首页 注册 登录
V2EX = way to explore
V2EX 是一个关于分享和探索的地方
现在注册
已注册用户请  登录
推荐学习书目
› Learn Python the Hard Way
Python Sites
› PyPI - Python Package Index
› http://diveintopython.org/toc/index.html
› Pocoo
值得关注的项目
› PyPy
› Celery
› Jinja2
› Read the Docs
› gevent
› pyenv
› virtualenv
› Stackless Python
› Beautiful Soup
› 结巴中文分词
› Green Unicorn
› Sentry
› Shovel
› Pyflakes
› pytest
Python 编程
› pep8 Checker
Styles
› PEP 8
› Google Python Style Guide
› Code Style from The Hitchhiker's Guide
hanssx
V2EX  ›  Python

使用 pycharm 对 Python 项目调试遇到非常诡异的现象

  •   
  •   hanssx · 2019-03-13 21:47:12 +08:00 · 5274 次点击
    这是一个创建于 2111 天前的主题，其中的信息可能已经有所发展或是发生改变。

    调试项目： https://github.com/WhaleShark-Team/cobra

    脚本命令：python cobra.py -t /root/php/dvwa/vulnerabilities/sqli_blind/ -d

    我在公司开发机上面 pipenv 搭建使用的是 py3.7，一开始我是用的 pycharm 远程调试，Run 没有任何问题，但是 Debug 会在进程池代码 multiprocessing.Pool()附近报错 KeyboardInterrupt 吗，实际上我什么也没有做，Run 没事儿，Debug 点一下就会自动中断。

    ssh://[email protected]:22/root/.local/share/virtualenvs/cobra-CV_8c13e/bin/python -u /root/.pycharm_helpers/pydev/pydevd.py --multiproc --qt-support --client '0.0.0.0' --port 40819 --file /root/python/cobra/cobra.py -t /root/php/dvwa/vulnerabilities/sqli_blind/ -d
warning: Debugger speedups using cython not found. Run '"/root/.local/share/virtualenvs/cobra-CV_8c13e/bin/python" "/root/.pycharm_helpers/pydev/setup_cython.py" build_ext --inplace' to build.
pydev debugger: process 26949 is connecting

Connected to pydev debugger (build 171.4694.94)
[21:37:52] [DEBUG] [INIT] set logging level: debug
[21:37:52] [DEBUG] [INIT] start scanning...
[21:37:52] [INFO] [REPORT] Report URL: ?sid=a7df1eil4boi
[21:37:52] [DEBUG] [PARSE-ARGS] Target Mode: folder
[21:37:52] [DEBUG] [PARSE-ARGS] Output Mode: stream
[21:37:52] [DEBUG] [PARSE-ARGS] target directory: /root/php/dvwa/vulnerabilities/sqli_blind/
[21:37:52] [INFO] [CLI] Target directory: /root/php/dvwa/vulnerabilities/sqli_blind
[21:37:52] [DEBUG] [PICKUP] /root/php/dvwa/vulnerabilities/sqli_blind
[21:37:52] [DEBUG] [PICKUP] [FILES] |--index.php
[21:37:52] [DEBUG] [PICKUP] [FILES] |--source
[21:37:52] [DEBUG] [PICKUP] [FILES] |  |--medium.php
[21:37:52] [DEBUG] [PICKUP] [FILES] |  |--impossible.php
[21:37:52] [DEBUG] [PICKUP] [FILES] |  |--low.php
[21:37:52] [DEBUG] [PICKUP] [FILES] |  |--high.php
[21:37:52] [DEBUG] [PICKUP] [FILES] |--help
[21:37:52] [DEBUG] [PICKUP] [FILES] |  |--help.php
[21:37:52] [DEBUG] [PICKUP] [FILES] |--cookie-input.php
[21:37:52] [DEBUG] [PICKUP] [EXTENSION-COUNT] .php : 7
[21:37:52] [DEBUG] [DETECTION] [LANGUAGE] .php 7
[21:37:52] [DEBUG] [DETECTION] [LANGUAGE] found the chiefly language(php), maybe have largest, continue...
[21:37:52] [DEBUG] [DETECTION] [LANGUAGE] main language(php), tmp language(None)
[21:37:52] [DEBUG] Dependency analysis cannot be done without finding dependency files
[21:37:52] [INFO] [DETECTION] [FRAMEWORK] Unknown Framework
[21:37:52] [INFO] [CLI] [STATISTIC] Language: php Framework: Unknown Framework
[21:37:52] [INFO] [CLI] [STATISTIC] Files: 7, Extensions:1, Consume: 0.0
[21:37:52] [DEBUG] filter dependency rules
[21:37:52] [DEBUG] filter dependency rules
[21:37:53] [DEBUG] filter dependency rules
[21:37:53] [DEBUG] filter dependency rules
[21:37:53] [DEBUG] filter dependency rules
[21:37:53] [DEBUG] filter dependency rules
[21:37:53] [DEBUG] filter dependency rules
[21:37:53] [DEBUG] filter dependency rules
[21:37:53] [DEBUG] filter dependency rules
[21:37:53] [DEBUG] filter dependency rules
[21:37:53] [DEBUG] filter dependency rules
[21:37:53] [DEBUG] filter dependency rules
[21:37:54] [DEBUG] filter dependency rules
[21:37:54] [DEBUG] filter dependency rules
[21:37:54] [DEBUG] filter dependency rules
[21:37:54] [DEBUG] filter dependency rules
[21:37:54] [DEBUG] filter dependency rules
[21:37:54] [DEBUG] filter dependency rules
[21:37:54] [INFO] [PUSH] 18 CVE Rules
[21:37:55] [DEBUG] Dependency analysis cannot be done without finding dependency files
[21:37:55] [DEBUG] [SCAN] [STORE] Not found vulnerabilities on this rule!
^CTraceback (most recent call last):
  File "/root/.pycharm_helpers/pydev/pydevd.py", line 1591, in <module>
    globals = debugger.run(setup['file'], None, None, is_module)
  File "/root/.pycharm_helpers/pydev/pydevd.py", line 1018, in run
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/root/.pycharm_helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/root/python/cobra/cobra.py", line 22, in <module>
    sys.exit(main())
  File "/root/python/cobra/cobra/__init__.py", line 110, in main
    cli.start(args.target, args.format, args.output, args.special_rules, a_sid, args.dels)
  File "/root/python/cobra/cobra/cli.py", line 94, in start
    language=main_language, framework=main_framework, file_count=file_count, extension_count=len(files))
  File "/root/python/cobra/cobra/engine.py", line 160, in scan
    cve_vuls = scan_cve(target_directory)
  File "/root/python/cobra/cobra/cve.py", line 406, in scan_cve
    pool.join()
  File "/usr/local/python3/lib/python3.7/multiprocessing/pool.py", line 560, in join
    p.join()
  File "/usr/local/python3/lib/python3.7/multiprocessing/process.py", line 140, in join
    res = self._popen.wait(timeout)
  File "/usr/local/python3/lib/python3.7/multiprocessing/popen_fork.py", line 48, in wait
    return self.poll(os.WNOHANG if timeout == 0.0 else 0)
  File "/usr/local/python3/lib/python3.7/multiprocessing/popen_fork.py", line 28, in poll
    pid, sts = os.waitpid(self.pid, flag)
KeyboardInterrupt

Process finished with exit code 1

    因为看不懂报错代码，所以我就使用了我搭建在 vmware 中的虚拟机 centos7，然后同样使用 pipenv 安装了 py3.7 的虚拟环境，但是诡异得是 Run 都会产生问题了，而且报错也和之前远程调试 Debug 不一样，

    /root/.local/share/virtualenvs/cobra-CV_8c13e/bin/python /root/python/cobra/cobra.py -t /root/python/cobra/tests/vulnerabilities/
--- Logging error ---
Traceback (most recent call last):
  File "/root/python/cobra/cobra/log.py", line 123, in emit
    stream.write(message)
TypeError: write() argument must be str, not bytes
Call stack:
  File "/root/python/cobra/cobra.py", line 22, in <module>
    sys.exit(main())
  File "/root/python/cobra/cobra/__init__.py", line 110, in main
    cli.start(args.target, args.format, args.output, args.special_rules, a_sid, args.dels)
  File "/root/python/cobra/cobra/cli.py", line 64, in start
    logger.info('[REPORT] Report URL: {u}'.format(u=report))
  File "/usr/local/python3/lib/python3.7/logging/__init__.py", line 1332, in info
    self._log(INFO, msg, args, **kwargs)
  File "/usr/local/python3/lib/python3.7/logging/__init__.py", line 1468, in _log
    self.handle(record)
  File "/usr/local/python3/lib/python3.7/logging/__init__.py", line 1478, in handle
    self.callHandlers(record)
  File "/usr/local/python3/lib/python3.7/logging/__init__.py", line 1540, in callHandlers
    hdlr.handle(record)
  File "/usr/local/python3/lib/python3.7/logging/__init__.py", line 854, in handle
    self.emit(record)
  File "/root/python/cobra/cobra/log.py", line 134, in emit
    self.handleError(record)
Message: '[REPORT] Report URL: ?sid=a9d766qddopy'
Arguments: ()
--- Logging error ---
Traceback (most recent call last):
  File "/root/python/cobra/cobra/log.py", line 123, in emit
    stream.write(message)
TypeError: write() argument must be str, not bytes
Call stack:
  File "/root/python/cobra/cobra.py", line 22, in <module>
    sys.exit(main())
  File "/root/python/cobra/cobra/__init__.py", line 110, in main
    cli.start(args.target, args.format, args.output, args.special_rules, a_sid, args.dels)
  File "/root/python/cobra/cobra/cli.py", line 75, in start
    logger.info('[CLI] Target directory: {d}'.format(d=target_directory))
  File "/usr/local/python3/lib/python3.7/logging/__init__.py", line 1332, in info
    self._log(INFO, msg, args, **kwargs)
  File "/usr/local/python3/lib/python3.7/logging/__init__.py", line 1468, in _log
    self.handle(record)
  File "/usr/local/python3/lib/python3.7/logging/__init__.py", line 1478, in handle
    self.callHandlers(record)
  File "/usr/local/python3/lib/python3.7/logging/__init__.py", line 1540, in callHandlers
    hdlr.handle(record)
  File "/usr/local/python3/lib/python3.7/logging/__init__.py", line 854, in handle
    self.emit(record)
  File "/root/python/cobra/cobra/log.py", line 134, in emit
    self.handleError(record)
Message: '[CLI] Target directory: /root/python/cobra/tests/vulnerabilities'
Arguments: ()
--- Logging error ---

    但是我直接在 centos 命令行下运行 pycharm 运行的命令是没有问题的，我是真不知道咋办了，只能打扰求助各位前辈，愿意花 20 块钱求教到底为何，聊表心意，困扰了一晚上。

    [root@xiaopo-centos cobra]# /root/.local/share/virtualenvs/cobra-CV_8c13e/bin/python /root/python/cobra/cobra.py -t /root/python/cobra/tests/vulnerabilities/
[21:05:09] [INFO] [REPORT] Report URL: ?sid=a9d7661ujlcb
[21:05:09] [INFO] [CLI] Target directory: /root/python/cobra/tests/vulnerabilities
[21:05:09] [INFO] [CLI] [STATISTIC] Language: php Framework: Spring
[21:05:09] [INFO] [CLI] [STATISTIC] Files: 15, Extensions:14, Consume: 0.0
[21:05:13] [INFO] [PUSH] 18 CVE Rules
[21:05:15] [INFO] [PUSH] 95 Rules
[21:05:18] [INFO] [SCAN] Trigger Rules/Not Trigger Rules/Off Rules: 65/2/29 Vulnerabilities (69)
  • Python
  • debug
  • Cobra
  • file
    8 条回复  •  2019-03-15 15:29:06 +08:00
    hanssx
        1
    hanssx  
    OP
       2019-03-13 22:28:03 +08:00
    pycon 群里刘师傅说了一个问题，“你用的是虚拟环境下的 python，为啥报错目录是 /usr/local/python3 呢”
    我也感觉好奇怪，按道理来说，报错应该是在虚拟环境下的吧？！
    chenqh
        2
    chenqh  
       2019-03-13 22:31:23 +08:00   ❤️ 2
    感觉是 log.py 文件的问题
    ```
    def emit(self, record):
    try:
    message = stdout_encode(self.format(record))
    stream = self.stream

    if not self.is_tty:
    if message and message[0] == "\r":
    message = message[1:]
    stream.write(message)
    else:
    self.output_colorized(message)
    stream.write(getattr(self, 'terminator', '\n'))

    self.flush()
    except (KeyboardInterrupt, SystemExit):
    raise
    except IOError:
    pass
    except Exception as e:
    self.handleError(record)
    ```
    这个改成
    ```
    def emit(self, record):
    try:
    message = stdout_encode(self.format(record))
    stream = self.stream

    if not self.is_tty:
    if message and message[0] == "\r":
    message = message[1:]
    stream.write(message.decode(sys.stdout.encoding))
    else:
    self.output_colorized(message)
    stream.write(getattr(self, 'terminator', '\n'))

    self.flush()
    except (KeyboardInterrupt, SystemExit):
    raise
    except IOError:
    pass
    except Exception as e:
    self.handleError(record)
    ```
    感觉是 pycharm log 走的
    ```
    if not self.is_tty:
    if message and message[0] == "\r":
    message = message[1:]
    stream.write(message.decode(sys.stdout.encoding))
    ```
    而 terminator 走的是
    ```
    else:
    self.output_colorized(message)
    ```
    lniwn
        3
    lniwn  
       2019-03-13 22:36:15 +08:00 via iPhone
    以前遇到过 pycharm 调试 asyncio，调试时下断点，pycharm 偶发退出，但是提前下好断点，再启动调试就 OK。
    hanssx
        4
    hanssx  
    OP
       2019-03-13 23:03:17 +08:00
    @chenqh 多谢师傅回复，没想到确实是这样，stream.write(message)改成 stream.write(message.decode('utf-8'))就可以了。
    hanssx
        5
    hanssx  
    OP
       2019-03-13 23:09:55 +08:00
    @chenqh 但是我发现在虚拟机中使用 pycharm 调试，在使用了进程池的附近也会报一样的 KeyboardInterrupt 错误，看来不是远程调试的问题，不过师傅已经帮我解决了一个问题，师傅请问微信或者 qq 私聊或者发我吧，我给师傅转钱，我的 qq 是 9614 六 2392
    @lniwn 提前下好，下到出错的后面了，也不行呢，感谢师傅。
    hanssx
        6
    hanssx  
    OP
       2019-03-14 11:45:51 +08:00
    最后发现好像是 py3.7 的问题，换成 py2.7 可以了。
    参考： https://forums.fast.ai/t/pycharm-ide-debugger-exception/28909/4
    hanssx
        7
    hanssx  
    OP
       2019-03-14 13:29:17 +08:00
    经测试，py3.6.8 亦可，大概率真是 py3.7 的问题。。
    EdwardChu
        8
    EdwardChu  
       2019-03-15 15:29:06 +08:00
    py3.7 好像真的有些问题，我 pyltp 都装不了，只能把 python 版本改成 3.6 的才装上去了。
    关于   ·   帮助文档   ·   博客   ·   API   ·   FAQ   ·   实用小工具   ·   3437 人在线   最高记录 6679   ·     Select Language
    创意工作者们的社区
    World is powered by solitude
    VERSION: 3.9.8.5 · 29ms · UTC 11:15 · PVG 19:15 · LAX 03:15 · JFK 06:15
    Developed with CodeLauncher
    ♥ Do have faith in what you're doing.