python Tripartite Library requests - Quick Start

Based on 2.21.0

Send request

Send GET requests:

r = requests.get('https://api.github.com/events')

Send POST request:

r = requests.post('https://httpbin.org/post', data={'key':'value'})

Other request interfaces are identical to HTTP request types, such as PUT, DELETE, HEAD, OPTIONS, etc.

Using parameters in URL query strings

Pass a dictionary object to the params parameter:

>>> payload = {'key1': 'value1', 'key2': 'value2'}
>>> r = requests.get('https://httpbin.org/get', params=payload)
>>> print(r.url)
https://httpbin.org/get?key2=value2&key1=value1

Dictionary values can also be a list:

>>> payload = {'key1': 'value1', 'key2': ['value2', 'value3']}
>>> r = requests.get('https://httpbin.org/get', params=payload)
>>> print(r.url)
https://httpbin.org/get?key1=value1&key2=value2&key2=value3

The key pair whose parameter median is None will not be added to the query string

Text response content

The text attribute of the Response object retrieves the text form of the server response content, and Requests decodes automatically:

>>> r = requests.get('https://api.github.com/events')
>>> r.text
'[{"id":"9167113775","type":"PushEvent","actor"...

When accessing Response.text, Requests will guess the response content encoding based on the HTTP header. Using the Response.encoding attribute, you can view or change the encoding used by Requests:

>>> r.encoding
'utf-8'
>>> r.encoding = 'ISO-8859-1'

Binary response content

The content attribute of the Response object can obtain the binary form of the server response content:

>>> r.content
b'[{"id":"9167113775","type":"PushEvent","actor"...

JSON response content

The json() method of the Response object can obtain the JSON form of the server response content:

>>> r = requests.get('https://api.github.com/events')
>>> r.json()
[{'repo': {'url': 'https://api.github.com/...

If JSON decoding fails, an exception is thrown.

Original response content

In rare cases, access to the server's original socket response may be required. By setting the stream=True parameter in the request and accessing the raw attribute of the Response object:

>>> r = requests.get('https://api.github.com/events', stream=True)
>>> r.raw
<urllib3.response.HTTPResponse object at 0x101194810>
>>> r.raw.read(10)
'\x1f\x8b\x08\x00\x00\x00\x00\x00\x00\x03'

The usual usage is to save the original response content to a file in the following way. The Response.iter_content method automatically decodes gzip and deflate transport codes:

with open(filename, 'wb') as fd:
    for chunk in r.iter_content(chunk_size=128):
        fd.write(chunk)

Customized request header

Passing a dict object to the headers parameter adds an HTTP request header:

>>> url = 'https://api.github.com/some/endpoint'
>>> headers = {'user-agent': 'my-app/0.0.1'}

>>> r = requests.get(url, headers=headers)

Customized header s have low priority and may be overwritten in some scenarios or conditions.

All header s must be of string, bytestring or Unicode type. However, it is recommended to avoid passing unicode-type values as much as possible.

POST A Multipart-Encoded File

Upload files:

>>> url = 'https://httpbin.org/post'
>>> files = {'file': open('report.xls', 'rb')}
>>> r = requests.post(url, files=files)

Explicitly set the file name, content-type, and request header:

>>> url = 'https://httpbin.org/post'
>>> files = {'file': ('report.xls', open('report.xls', 'rb'), 'application/vnd.ms-excel', {'Expires': '0'})}
>>> r = requests.post(url, files=files)

You can even send strings received as files:

>>> url = 'http://httpbin.org/post'
>>> files = {'file': ('report.csv', 'some,data,to,send\nanother,row,to,send\n')}
>>> r = requests.post(url, files=files)

If the file sent is too large, it is recommended to use the third-party package requests-toolbelt as a data stream.

It is strongly recommended that files be opened in binary mode, because Requests may set Content-Length with byte length in the file

Response State Code

The status_code attribute of the Response object can obtain the response state:

>>> r = requests.get('https://httpbin.org/get')
>>> r.status_code
200

The requests library also has built-in status codes for reference:

>>> r.status_code == requests.codes.ok
True

If an exception is requested (a client error with status code 420 or a server error with 5XX), the raise_for_status() method can be called to throw the exception:

>>> bad_r = requests.get('https://httpbin.org/status/404')
>>> bad_r.status_code
404
>>> bad_r.raise_for_status()
Traceback (most recent call last):
  File "requests/models.py", line 832, in raise_for_status
    raise http_error
requests.exceptions.HTTPError: 404 Client Error

Response head

The headers attribute of the Response object can obtain the response header, which is a dictionary object with keys case-insensitive:

>>> r.headers
{
    'content-encoding': 'gzip',
    'transfer-encoding': 'chunked',
    'connection': 'close',
    'server': 'nginx/1.0.4',
    'x-runtime': '148ms',
    'etag': '"e1ca502697e5c9317743dc078f67693f"',
    'content-type': 'application/json'
}
>>> r.headers['Content-Type']
'application/json'
>>> r.headers.get('content-type')
'application/json'

Cookies

The cookies attribute of the Response object can obtain cookie information in the response:

>>> url = 'http://example.com/some/cookie/setting/url'
>>> r = requests.get(url)
>>> r.cookies['example_cookie_name']
'example_cookie_value'

Using cookies parameters, cookie information can be sent:

>>> url = 'https://httpbin.org/cookies'
>>> cookies = dict(cookies_are='working')
>>> r = requests.get(url, cookies=cookies)

Response.cookies returns a Requests CookieJar object, similar to a dictionary, but provides an additional interface for multiple domain names or multiple paths, or can be passed in a request:

>>> jar = requests.cookies.RequestsCookieJar()
>>> jar.set('tasty_cookie', 'yum', domain='httpbin.org', path='/cookies')
>>> jar.set('gross_cookie', 'blech', domain='httpbin.org', path='/elsewhere')
>>> url = 'https://httpbin.org/cookies'
>>> r = requests.get(url, cookies=jar)
>>> r.text
'{"cookies": {"tasty_cookie": "yum"}}'

Redirection and request history

Requests by default perform address redirection for all requests except HEAD. The Response.history attribute traces the redirection history and returns a list containing all Response objects created to complete the request and sorted from old to new.

The following is a use case for HTTP redirection to HTTPS:

>>> r = requests.get('http://github.com/')
>>> r.url
'https://github.com/'
>>> r.status_code
200
>>> r.history
[<Response [301]>]

Using the allow_redirects parameter, redirection can be disabled:

>>> r = requests.get('http://github.com/', allow_redirects=False)
>>> r.status_code
301
>>> r.history
[]

If you are using a HEAD request, you can also use the allow_redirects parameter to allow redirection:

>>> r = requests.head('http://github.com/', allow_redirects=True)
>>> r.url
'https://github.com/'
>>> r.history
[<Response [301]>]

request timeout

Use the timeout parameter to set the maximum waiting time for the server to return the response:

>>> requests.get('https://github.com/', timeout=0.001)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
requests.exceptions.Timeout: HTTPConnectionPool(host='github.com', port=80): Request timed out. (timeout=0.001)

Errors and exceptions

ConnectionError: Network anomalies, such as DNS errors, connection rejection, etc.
HTTPError: Calling Response.raise_for_status() throws this exception if the request returns a 4XX or 5XX status code.
Timeout: Connection timeout.
TooManyRedirects: Requests exceed the maximum number of redirections configured.
RequestException: Exception base class.

Keywords: Python github JSON Attribute encoding

Added by Herk on Thu, 16 May 2019 18:33:00 +0300

Programming VIP