V2EX = way to explore
V2EX 是一个关于分享和探索的地方
现在注册
已注册用户请  登录
推荐学习书目
Learn Python the Hard Way
Python Sites
PyPI - Python Package Index
http://diveintopython.org/toc/index.html
Pocoo
值得关注的项目
PyPy
Celery
Jinja2
Read the Docs
gevent
pyenv
virtualenv
Stackless Python
Beautiful Soup
结巴中文分词
Green Unicorn
Sentry
Shovel
Pyflakes
pytest
Python 编程
pep8 Checker
Styles
PEP 8
Google Python Style Guide
Code Style from The Hitchhiker's Guide
HFX3389
V2EX  ›  Python

Python3 如何正确处理超时并重试?

  •  
  •   HFX3389 · 2017-02-24 02:07:16 +08:00 · 11674 次点击
    这是一个创建于 2811 天前的主题,其中的信息可能已经有所发展或是发生改变。

    一段为 POST 获取数据的代码,

    #! /usr/local/python3
    import urllib
    import http
    import time
    import socket
    
    timeout = 10
    socket.setdefaulttimeout(timeout)
    
    
    def up_post(username, password, page):
        data = {'username': username, 'password' : password}
        url = r"http://192.168.2.8/?page" + page
        postdata = bytes(urllib.parse.urlencode(data), encoding='gbk')
        response = urllib.request.urlopen(url, data=postdata)
        text = response.read().decode('gbk')
        print(text)
        if text.find("成功") != -1:
            return page + ":" + text
        else:
            return page + ":error"
    
    username = "aaa"
    password = "bbb"
    page = 1
    while page <= 99:
        try:
            static = up_post(username, password, page)
        except ( http.client.IncompleteRead, urllib.error.URLError, socket.timeout, ConnectionResetError) as e:
            time.sleep(5)
            static = up_post(username, password, page)
            while True:
                if static == page + ":" + static or static == page + ":error" or static == "":
                    break
    

    运行一段时间过后,会连续提示两个 timed out :

    Traceback (most recent call last):
      File "mvc.py", line 103, in <module>
        static = up_post(username, password, page)
      File "mvc.py", line 27, in lan_up_post
        response = urllib.request.urlopen(url, data=postdata)
      File "/usr/local/lib/python3.6/urllib/request.py", line 223, in urlopen
        return opener.open(url, data, timeout)
      File "/usr/local/lib/python3.6/urllib/request.py", line 526, in open
        response = self._open(req, data)
      File "/usr/local/lib/python3.6/urllib/request.py", line 544, in _open
        '_open', req)
      File "/usr/local/lib/python3.6/urllib/request.py", line 504, in _call_chain
        result = func(*args)
      File "/usr/local/lib/python3.6/urllib/request.py", line 1346, in http_open
        return self.do_open( http.client.HTTPConnection, req)
      File "/usr/local/lib/python3.6/urllib/request.py", line 1321, in do_open
        r = h.getresponse()
      File "/usr/local/lib/python3.6/http/client.py", line 1331, in getresponse
        response.begin()
      File "/usr/local/lib/python3.6/http/client.py", line 297, in begin
        version, status, reason = self._read_status()
      File "/usr/local/lib/python3.6/http/client.py", line 258, in _read_status
        line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
      File "/usr/local/lib/python3.6/socket.py", line 586, in readinto
        return self._sock.recv_into(b)
    socket.timeout: timed out
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "mvc.py", line 106, in <module>
        static = up_post(username, password, page)
      File "mvc.py", line 27, in lan_up_post
        response = urllib.request.urlopen(url, data=postdata)
      File "/usr/local/lib/python3.6/urllib/request.py", line 223, in urlopen
        return opener.open(url, data, timeout)
      File "/usr/local/lib/python3.6/urllib/request.py", line 526, in open
        response = self._open(req, data)
      File "/usr/local/lib/python3.6/urllib/request.py", line 544, in _open
        '_open', req)
      File "/usr/local/lib/python3.6/urllib/request.py", line 504, in _call_chain
        result = func(*args)
      File "/usr/local/lib/python3.6/urllib/request.py", line 1346, in http_open
        return self.do_open( http.client.HTTPConnection, req)
      File "/usr/local/lib/python3.6/urllib/request.py", line 1321, in do_open
        r = h.getresponse()
      File "/usr/local/lib/python3.6/http/client.py", line 1331, in getresponse
        response.begin()
      File "/usr/local/lib/python3.6/http/client.py", line 297, in begin
        version, status, reason = self._read_status()
      File "/usr/local/lib/python3.6/http/client.py", line 258, in _read_status
        line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
      File "/usr/local/lib/python3.6/socket.py", line 586, in readinto
        return self._sock.recv_into(b)
    socket.timeout: timed out
    

    超时处理我只会 sleep 以后再重新运行一次获取代码,但是还是会出现两个 timed out 然后强行终止程序。

    它提示 timed out 可以,但是我不想让它强行终止程序,我想让它不断重试!

    咋搞啊?

    第 1 条附言  ·  2017-02-25 22:56:35 +08:00
    已经用了 #3 推荐的 retrying ,不错挺好用的!
    13 条回复    2017-02-24 18:31:37 +08:00
    zeroten
        1
    zeroten  
       2017-02-24 02:11:20 +08:00 via iPhone
    把外部调用的函数加个装饰器,捕获异常并重试
    HFX3389
        2
    HFX3389  
    OP
       2017-02-24 02:36:40 +08:00
    这节点没有 APPEND 嘛?
    我突然发现,似乎
    if static == page + ":" + static or static == page + ":error" or static == "":
    break
    并没有用啊。
    加不加都会 timed out
    argsno
        3
    argsno  
       2017-02-24 06:20:55 +08:00 via Android   ❤️ 1
    kindjeff
        4
    kindjeff  
       2017-02-24 07:08:13 +08:00 via iPhone
    循环 try catch 呗
    wannianma
        5
    wannianma  
       2017-02-24 07:57:24 +08:00 via iPhone
    最好设置个最高重试次数
    cszeus
        6
    cszeus  
       2017-02-24 08:04:59 +08:00
    用 2 层循环?
    while page <= 99:
    gotData = False
    reTry = 1
    while reTry < 5 and not gotData:
    try:
    reTry += 1
    static = up_post()
    gotData = True
    except:
    pass
    rogerchen
        7
    rogerchen  
       2017-02-24 08:10:43 +08:00 via Android
    while try catch timeout break
    herozhang
        8
    herozhang  
       2017-02-24 08:11:56 +08:00 via iPhone
    有个包叫做 retry ,可以试一下
    zh5e
        9
    zh5e  
       2017-02-24 09:05:03 +08:00
    @argsno 这个方法好
    lerry
        10
    lerry  
       2017-02-24 09:29:30 +08:00
    web 请求函数单独摘出来,手动捕获超时,超时就递归请求,正常就返回,还可以传一个最大尝试次数的参数
    awanabe
        11
    awanabe  
       2017-02-24 10:32:17 +08:00
    requests 库有 timeout 选项的
    Ge4Los
        12
    Ge4Los  
       2017-02-24 10:49:16 +08:00
    requests 有 timeout 和 retry 的配置。可以分别设置连接超时和读取超时。
    phithon
        13
    phithon  
       2017-02-24 18:31:37 +08:00
    @argsno 对,我一直用这个库
    关于   ·   帮助文档   ·   博客   ·   API   ·   FAQ   ·   实用小工具   ·   1145 人在线   最高记录 6679   ·     Select Language
    创意工作者们的社区
    World is powered by solitude
    VERSION: 3.9.8.5 · 26ms · UTC 18:33 · PVG 02:33 · LAX 10:33 · JFK 13:33
    Developed with CodeLauncher
    ♥ Do have faith in what you're doing.