问使用urllib2请求与build_opener发布JSON之间的意外行为(Python2.7)
EN

Stack Overflow用户

提问于 2017-02-24 11:40:19

回答 1查看 166关注 0票数 0

编辑：

找到了解决办法。因为RBC端点是https，所以处理程序类也需要有一个https_request函数，而不仅仅是一个http_request。

Solution for http endpoints

class ChangeTypeProcessor(urllib2.BaseHandler):
    def http_request(self, req):
        req.unredirected_hdrs["Content-type"] = "application/json"
        req.unredirected_hdrs["User-Agent"] = "Python-urllib/2.7"
        return req

    def https_request(self, req):
        req.unredirected_hdrs["Content-type"] = "application/json"
        req.unredirected_hdrs["User-Agent"] = "Python-urllib/2.7"
        return req

在使用python中使用urllib2库发布JSON数据的两种不同方法时，我遇到了奇怪的行为差异。尽管我知道请求是存在的，但我正在使用遗留软件，这使得切换变得更加困难。遗留软件还使用了发布数据的build_opener方法。

在尝试将JSON发布到RBC的求职时，request对象能够成功地获得JSON数据/响应。然而，当将JSON发布到RBC的作业库时，OpenerDirector并不能成功地获得响应。

以下是我试图从RBC获取工作数据的测试器：

import json
import urllib2
import urlparse

class ChangeTypeProcessor(urllib2.BaseHandler):
    def http_request(self, req):
        req.unredirected_hdrs["Content-type"] = "application/json"
        req.unredirected_hdrs["User-Agent"] = "Python-urllib/2.7"
        return req

data = '{"lang":"en_ca","deviceType":"desktop","country":"ca","ddoKey":"refineSearch","sortBy":"","subsearch":"","from":100,"all_fields":[],"pageName":"search-results","counts":true,"jobs":true,"keywords":"","global":true,"size":50,"sele
cted_fields":null,"sort":null}'
url = "https://jobs.rbc.com/widgets"
#url = "http://httpbin.org/post"
request = urllib2.Request(url)
request.add_header('Content-Type', 'application/json')

response = urllib2.urlopen(request, data)

urlopener = urllib2.build_opener()
urlopener.add_handler(ChangeTypeProcessor())
print(response.read())

connection = urlopener.open(url, data)
print(connection.read())

以下是OpenDirector在发布到站点时产生的错误：

Traceback (most recent call last):
  File "jsonPost.py", line 23, in <module>
    connection = urlopener.open(url, data)
  File "/usr/local/lib/python2.7/urllib2.py", line 437, in open
    response = meth(req, response)
  File "/usr/local/lib/python2.7/urllib2.py", line 550, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/local/lib/python2.7/urllib2.py", line 475, in error
    return self._call_chain(*args)
  File "/usr/local/lib/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/usr/local/lib/python2.7/urllib2.py", line 558, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 500: Internal Server Error

以下是用http://httpbin.org/post发布的数据：

请求对象的响应：

{
  "args": {},
  "data": "{\"lang\":\"en_ca\",\"deviceType\":\"desktop\",\"country\":\"ca\",\"ddoKey\":\"refineSearch\",\"sortBy\":\"\",\"subsearch\":\"\",\"from\":100,\"all_fields\":[],\"pageName\":\"search-results\",\"counts\":true,\"jobs\":true,\"keywords\":\"\",\"global\":true,\"size\":50,\"selected_fields\":null,\"sort\":null}",
  "files": {},
  "form": {},
  "headers": {
    "Accept-Encoding": "identity",
    "Content-Length": "259",
    "Content-Type": "application/json",
    "Host": "httpbin.org",
    "User-Agent": "Python-urllib/2.7"
  },
  "json": {
    "all_fields": [],
    "country": "ca",
    "counts": true,
    "ddoKey": "refineSearch",
    "deviceType": "desktop",
    "from": 100,
    "global": true,
    "jobs": true,
    "keywords": "",
    "lang": "en_ca",
    "pageName": "search-results",
    "selected_fields": null,
    "size": 50,
    "sort": null,
    "sortBy": "",
    "subsearch": ""
  },
  "origin": "<hidden>",
  "url": "http://httpbin.org/post"
}

OpenDirector的响应：

{
  "args": {},
  "data": "{\"lang\":\"en_ca\",\"deviceType\":\"desktop\",\"country\":\"ca\",\"ddoKey\":\"refineSearch\",\"sortBy\":\"\",\"subsearch\":\"\",\"from\":100,\"all_fields\":[],\"pageName\":\"search-results\",\"counts\":true,\"jobs\":true,\"keywords\":\"\",\"global\":true,\"size\":50,\"selected_fields\":null,\"sort\":null}",
  "files": {},
  "form": {},
  "headers": {
    "Accept-Encoding": "identity",
    "Content-Length": "259",
    "Content-Type": "application/json",
    "Host": "httpbin.org",
    "User-Agent": "Python-urllib/2.7"
  },
  "json": {
    "all_fields": [],
    "country": "ca",
    "counts": true,
    "ddoKey": "refineSearch",
    "deviceType": "desktop",
    "from": 100,
    "global": true,
    "jobs": true,
    "keywords": "",
    "lang": "en_ca",
    "pageName": "search-results",
    "selected_fields": null,
    "size": 50,
    "sort": null,
    "sortBy": "",
    "subsearch": ""
  },
  "origin": "<hidden>",
  "url": "http://httpbin.org/post"
}

值得注意的是，httpbin提供的信息表明所发布的数据完全相同，但由于某种原因，OpenDirector无法获得适当的响应。

我想要的解决方案是使OpenDirector成功地获得适当的响应，而不需要重新实现使用请求或请求对象的代码库。

有没有人知道是什么原因造成了这种差异？

python

json

python-2.7

回答 1

Stack Overflow用户

回答已采纳

发布于 2017-02-24 13:16:50

找到了解决办法。因为RBC端点是https，所以处理程序类也需要有一个https_request函数，而不仅仅是一个http_request。

Solution for http endpoints

class ChangeTypeProcessor(urllib2.BaseHandler):
    def http_request(self, req):
        req.unredirected_hdrs["Content-type"] = "application/json"
        req.unredirected_hdrs["User-Agent"] = "Python-urllib/2.7"
        return req

    def https_request(self, req):
        req.unredirected_hdrs["Content-type"] = "application/json"
        req.unredirected_hdrs["User-Agent"] = "Python-urllib/2.7"
        return req