下面是 tornado.httpclient.AsyncHTTPClient
类的 fetch()
方法的源代码。我没有在里面找到任何"fetch"的动作,它是怎么实现 “ Executes a request, asynchronously returning an HTTPResponse
”的?
完整代码在: https://github.com/tornadoweb/tornado/blob/master/tornado/httpclient.py
def fetch(self, request, callback=None, raise_error=True, **kwargs):
"""Executes a request, asynchronously returning an `HTTPResponse`.
The request may be either a string URL or an `HTTPRequest` object.
If it is a string, we construct an `HTTPRequest` using any additional
kwargs: ``HTTPRequest(request, **kwargs)``
This method returns a `.Future` whose result is an
`HTTPResponse`. By default, the ``Future`` will raise an
`HTTPError` if the request returned a non-200 response code
(other errors may also be raised if the server could not be
contacted). Instead, if ``raise_error`` is set to False, the
response will always be returned regardless of the response
code.
If a ``callback`` is given, it will be invoked with the `HTTPResponse`.
In the callback interface, `HTTPError` is not automatically raised.
Instead, you must check the response's ``error`` attribute or
call its `~HTTPResponse.rethrow` method.
"""
if self._closed:
raise RuntimeError("fetch() called on closed AsyncHTTPClient")
if not isinstance(request, HTTPRequest):
request = HTTPRequest(url=request, **kwargs)
else:
if kwargs:
raise ValueError("kwargs can't be used if request is an HTTPRequest object")
# We may modify this (to add Host, Accept-Encoding, etc),
# so make sure we don't modify the caller's object. This is also
# where normal dicts get converted to HTTPHeaders objects.
request.headers = httputil.HTTPHeaders(request.headers)
request = _RequestProxy(request, self.defaults)
future = TracebackFuture()
if callback is not None:
callback = stack_context.wrap(callback)
def handle_future(future):
exc = future.exception()
if isinstance(exc, HTTPError) and exc.response is not None:
response = exc.response
elif exc is not None:
response = HTTPResponse(
request, 599, error=exc,
request_time=time.time() - request.start_time)
else:
response = future.result()
self.io_loop.add_callback(callback, response)
future.add_done_callback(handle_future)
def handle_response(response):
if raise_error and response.error:
future.set_exception(response.error)
else:
future.set_result(response)
self.fetch_impl(request, handle_response)
return future
def fetch_impl(self, request, callback):
raise NotImplementedError()
1
phithon 2016-01-05 20:26:06 +08:00 1
![]( )
impl 看名字应该是个接口,等着子类去实现的。 你可以看 AsyncHTTPClient 的 `__new__` 方法,实际上执行 `new AsyncHTTPClient()` ,其返回的对象是一个 SimpleAsyncHTTPClient 实例, SimpleAsyncHTTPClient 是对 AsyncHTTPClient 的实现。 所以,最后执行的 fetch_impl 方法,实际上在这里: https://github.com/tornadoweb/tornado/blob/6b312f7a73dc78de4b1be2a71a5bbfa51acc725d/tornado/simple_httpclient.py#L122 |
3
sujin190 2016-01-05 20:45:30 +08:00
|
4
sujin190 2016-01-05 20:47:41 +08:00
不过比较坑的是 SimpleAsyncHTTPClient 本身不支持连接池,每次都是打开新连接,除此之外还有一个实现是 CurlAsyncHTTPClient ,支持连接池,不过在 pypy 下似乎会出现栈溢出错误
|
6
mqingyn616 2016-01-06 23:50:35 +08:00
@sujin190 这个问题有一个比较靠谱的解决方案,可以看 github 上的这个 pr , https://github.com/tornadoweb/tornado/pull/1622
给 SimpleAsyncHTTPClient 增加连接池,亲测可用。 |