python - pyspider調試的時候沒有任何問題,點run就報編碼問題

問題描述

調試的時候沒有任何問題,點run就報編碼問題。同樣2個采集就這一個老報錯,另外一個完全沒問題

taskid

d7221a2be620c4ef60e874a1d93e79d1

lastcrawltime

1499144004.75187 (20 minutes ago)

updatetime

1499144004.7518892 (20 minutes ago)

exetime

1499144014.7518687 (20 minutes ago)

track.fetch 1.32ms

{ 'content': '', 'encoding': null, 'error': '’ascii’ codec can’t encode character ’uff09’ in position 94: ordinal not in range(128)', 'headers': {}, 'ok': false, 'redirect_url': null, 'status_code': 599, 'time': 0.0013222694396972656}

track.process 0.83ms

’ascii’ codec can’t encode character ’uff09’ in position 94: ordinal not in range(128) = self.gen.throw(*exc_info) File '/root/workspaces/pyspider3/lib/python3.5/site-packages/pyspider/fetcher/tornado_fetcher.py', line 378, in http_fetchresponse = yield gen.maybe_future(self.http_client.fetch(request)) File '/root/workspaces/pyspider3/lib/python3.5/site-packages/tornado/gen.py', line 1055, in runvalue = future.result() File '/root/workspaces/pyspider3/lib/python3.5/site-packages/tornado/concurrent.py', line 238, in resultraise_exc_info(self._exc_info) File '<string>', line 4, in raise_exc_info File '/root/workspaces/pyspider3/lib/python3.5/site-packages/tornado/curl_httpclient.py', line 214, in _process_queuecurl.info['headers']) File '/root/workspaces/pyspider3/lib/python3.5/site-packages/tornado/curl_httpclient.py', line 306, in _curl_setup_requestfor k, v in request.headers.get_all()]) Exception: ’ascii’ codec can’t encode character ’uff09’ in position 94: ordinal not in range(128){ 'exception': '’ascii’ codec can’t encode character ’uff09’ in position 94: ordinal not in range(128)', 'follows': 0, 'logs': ' = self.gen.throw(*exc_info)n File '/root/workspaces/pyspider3/lib/python3.5/site-packages/pyspider/fetcher/tornado_fetcher.py', line 378, in http_fetchnresponse = yield gen.maybe_future(self.http_client.fetch(request))n File '/root/workspaces/pyspider3/lib/python3.5/site-packages/tornado/gen.py', line 1055, in runnvalue = future.result()n File '/root/workspaces/pyspider3/lib/python3.5/site-packages/tornado/concurrent.py', line 238, in resultnraise_exc_info(self._exc_info)n File '<string>', line 4, in raise_exc_infon File '/root/workspaces/pyspider3/lib/python3.5/site-packages/tornado/curl_httpclient.py', line 214, in _process_queuencurl.info['headers'])n File '/root/workspaces/pyspider3/lib/python3.5/site-packages/tornado/curl_httpclient.py', line 306, in _curl_setup_requestnfor k, v in request.headers.get_all()])n Exception: ’ascii’ codec can’t encode character ’uff09’ in position 94: ordinal not in range(128)n', 'ok': false, 'result': null, 'time': 0.0008292198181152344}

schedule

{ 'age': 10, 'exetime': 1499144014.7518687, 'retried': 3}

process

{ 'callback': 'index_page'}

fetch

{}

問題解答

headers設置有問題,我刪掉直接秒好.

#!/usr/bin/env python# -*- encoding: utf-8 -*-

看看你的代碼前兩行是這個吧