What’s new in Tornado 3.0

Mar 29, 2013

Highlights

  • The callback argument to many asynchronous methods is now optional, and these methods return a Future. The tornado.gen module now understands Futures, and these methods can be used directly without a gen.Task wrapper.
  • New function IOLoop.current returns the IOLoop that is running on the current thread (as opposed to IOLoop.instance, which returns a specific thread’s (usually the main thread’s) IOLoop.
  • New class tornado.netutil.Resolver provides an asynchronous interface to DNS resolution. The default implementation is still blocking, but non-blocking implementations are available using one of three optional dependencies: ThreadedResolver using the concurrent.futures thread pool, tornado.platform.caresresolver.CaresResolver using the pycares library, or tornado.platform.twisted.TwistedResolver using twisted
  • Tornado’s logging is now less noisy, and it no longer goes directly to the root logger, allowing for finer-grained configuration.
  • New class tornado.process.Subprocess wraps subprocess.Popen with PipeIOStream access to the child’s file descriptors.
  • IOLoop now has a static configure method like the one on AsyncHTTPClient, which can be used to select an IOLoop implementation other than the default.
  • IOLoop can now optionally use a monotonic clock if available (see below for more details).

Backwards-incompatible changes

  • Python 2.5 is no longer supported. Python 3 is now supported in a single codebase instead of using 2to3
  • The tornado.database module has been removed. It is now available as a separate package, torndb
  • Functions that take an io_loop parameter now default to IOLoop.current() instead of IOLoop.instance().
  • Empty HTTP request arguments are no longer ignored. This applies to HTTPRequest.arguments and RequestHandler.get_argument[s] in WSGI and non-WSGI modes.
  • On Python 3, tornado.escape.json_encode no longer accepts byte strings.
  • On Python 3, the get_authenticated_user methods in tornado.auth now return character strings instead of byte strings.
  • tornado.netutil.TCPServer has moved to its own module, tornado.tcpserver.
  • The Tornado test suite now requires unittest2 when run on Python 2.6.
  • tornado.options.options is no longer a subclass of dict; attribute-style access is now required.

Detailed changes by module

Multiple modules

  • Tornado no longer logs to the root logger. Details on the new logging scheme can be found under the tornado.log module. Note that in some cases this will require that you add an explicit logging configuration in order to see any output (perhaps just calling logging.basicConfig()), although both IOLoop.start() and tornado.options.parse_command_line will do this for you.
  • On python 3.2+, methods that take an ssl_options argument (on SSLIOStream, TCPServer, and HTTPServer) now accept either a dictionary of options or an ssl.SSLContext object.
  • New optional dependency on concurrent.futures to provide better support for working with threads. concurrent.futures is in the standard library for Python 3.2+, and can be installed on older versions with pip install futures.

tornado.autoreload

  • tornado.autoreload is now more reliable when there are errors at import time.
  • Calling tornado.autoreload.start (or creating an Application with debug=True) twice on the same IOLoop now does nothing (instead of creating multiple periodic callbacks). Starting autoreload on more than one IOLoop in the same process now logs a warning.
  • Scripts run by autoreload no longer inherit __future__ imports used by Tornado.

tornado.auth

  • On Python 3, the get_authenticated_user method family now returns character strings instead of byte strings.
  • Asynchronous methods defined in tornado.auth now return a Future, and their callback argument is optional. The Future interface is preferred as it offers better error handling (the previous interface just logged a warning and returned None).
  • The tornado.auth mixin classes now define a method get_auth_http_client, which can be overridden to use a non-default AsyncHTTPClient instance (e.g. to use a different IOLoop)
  • Subclasses of OAuthMixin are encouraged to override OAuthMixin._oauth_get_user_future instead of _oauth_get_user, although both methods are still supported.

tornado.concurrent

tornado.curl_httpclient

  • Preliminary support for tornado.curl_httpclient on Python 3. The latest official release of pycurl only supports Python 2, but Ubuntu has a port available in 12.10 (apt-get install python3-pycurl). This port currently has bugs that prevent it from handling arbitrary binary data but it should work for textual (utf8) resources.
  • Fix a crash with libcurl 7.29.0 if a curl object is created and closed without being used.

tornado.escape

  • On Python 3, json_encode no longer accepts byte strings. This mirrors the behavior of the underlying json module. Python 2 behavior is unchanged but should be faster.

tornado.gen

  • New decorator @gen.coroutine is available as an alternative to @gen.engine. It automatically returns a Future, and within the function instead of calling a callback you return a value with raise gen.Return(value) (or simply return value in Python 3.3).
  • Generators may now yield Future objects.
  • Callbacks produced by gen.Callback and gen.Task are now automatically stack-context-wrapped, to minimize the risk of context leaks when used with asynchronous functions that don’t do their own wrapping.
  • Fixed a memory leak involving generators, RequestHandler.flush, and clients closing connections while output is being written.
  • Yielding a large list no longer has quadratic performance.

tornado.httpclient

  • AsyncHTTPClient.fetch now returns a Future and its callback argument is optional. When the future interface is used, any error will be raised automatically, as if HTTPResponse.rethrow was called.
  • AsyncHTTPClient.configure and all AsyncHTTPClient constructors now take a defaults keyword argument. This argument should be a dictionary, and its values will be used in place of corresponding attributes of HTTPRequest that are not set.
  • All unset attributes of tornado.httpclient.HTTPRequest are now None. The default values of some attributes (connect_timeout, request_timeout, follow_redirects, max_redirects, use_gzip, proxy_password, allow_nonstandard_methods, and validate_cert have been moved from HTTPRequest to the client implementations.
  • The max_clients argument to AsyncHTTPClient is now a keyword-only argument.
  • Keyword arguments to AsyncHTTPClient.configure are no longer used when instantiating an implementation subclass directly.
  • Secondary AsyncHTTPClient callbacks (streaming_callback, header_callback, and prepare_curl_callback) now respect StackContext.

tornado.httpserver

  • HTTPServer no longer logs an error when it is unable to read a second request from an HTTP 1.1 keep-alive connection.
  • HTTPServer now takes a protocol keyword argument which can be set to https if the server is behind an SSL-decoding proxy that does not set any supported X-headers.
  • tornado.httpserver.HTTPConnection now has a set_close_callback method that should be used instead of reaching into its stream attribute.
  • Empty HTTP request arguments are no longer ignored. This applies to HTTPRequest.arguments and RequestHandler.get_argument[s] in WSGI and non-WSGI modes.

tornado.ioloop

  • New function IOLoop.current returns the IOLoop that is running on the current thread (as opposed to IOLoop.instance, which returns a specific thread’s (usually the main thread’s) IOLoop).
  • New method IOLoop.add_future to run a callback on the IOLoop when an asynchronous Future finishes.
  • IOLoop now has a static configure method like the one on AsyncHTTPClient, which can be used to select an IOLoop implementation other than the default.
  • The IOLoop poller implementations (select, epoll, kqueue) are now available as distinct subclasses of IOLoop. Instantiating IOLoop will continue to automatically choose the best available implementation.
  • The IOLoop constructor has a new keyword argument time_func, which can be used to set the time function used when scheduling callbacks. This is most useful with the time.monotonic function, introduced in Python 3.3 and backported to older versions via the monotime module. Using a monotonic clock here avoids problems when the system clock is changed.
  • New function IOLoop.time returns the current time according to the IOLoop. To use the new monotonic clock functionality, all calls to IOLoop.add_timeout must be either pass a datetime.timedelta or a time relative to IOLoop.time, not time.time. (time.time will continue to work only as long as the IOLoop’s time_func argument is not used).
  • New convenience method IOLoop.run_sync can be used to start an IOLoop just long enough to run a single coroutine.
  • New method IOLoop.add_callback_from_signal is safe to use in a signal handler (the regular add_callback method may deadlock).
  • IOLoop now uses signal.set_wakeup_fd where available (Python 2.6+ on Unix) to avoid a race condition that could result in Python signal handlers being delayed.
  • Method IOLoop.running() has been removed.
  • IOLoop has been refactored to better support subclassing.
  • IOLoop.add_callback and add_callback_from_signal now take *args, **kwargs to pass along to the callback.

tornado.iostream

  • IOStream.connect now has an optional server_hostname argument which will be used for SSL certificate validation when applicable. Additionally, when supported (on Python 3.2+), this hostname will be sent via SNI (and this is supported by tornado.simple_httpclient)
  • Much of IOStream has been refactored into a separate class BaseIOStream.
  • New class tornado.iostream.PipeIOStream provides the IOStream interface on pipe file descriptors.
  • IOStream now raises a new exception tornado.iostream.StreamClosedError when you attempt to read or write after the stream has been closed (by either side).
  • IOStream now simply closes the connection when it gets an ECONNRESET error, rather than logging it as an error.
  • IOStream.error no longer picks up unrelated exceptions.
  • BaseIOStream.close now has an exc_info argument (similar to the one used in the logging module) that can be used to set the stream’s error attribute when closing it.
  • BaseIOStream.read_until_close now works correctly when it is called while there is buffered data.
  • Fixed a major performance regression when run on PyPy (introduced in Tornado 2.3).

tornado.log

tornado.netutil

tornado.options

tornado.platform.caresresolver

  • New module containing an asynchronous implementation of the Resolver interface, using the pycares library.

tornado.platform.twisted

tornado.process

tornado.simple_httpclient

  • SimpleAsyncHTTPClient now takes a resolver keyword argument (which may be passed to either the constructor or configure), to allow it to use the new non-blocking tornado.netutil.Resolver.
  • When following redirects, SimpleAsyncHTTPClient now treats a 302 response code the same as a 303. This is contrary to the HTTP spec but consistent with all browsers and other major HTTP clients (including CurlAsyncHTTPClient).
  • The behavior of header_callback with SimpleAsyncHTTPClient has changed and is now the same as that of CurlAsyncHTTPClient. The header callback now receives the first line of the response (e.g. HTTP/1.0 200 OK) and the final empty line.
  • tornado.simple_httpclient now accepts responses with a 304 status code that include a Content-Length header.
  • Fixed a bug in which SimpleAsyncHTTPClient callbacks were being run in the client’s stack_context.

tornado.stack_context

  • stack_context.wrap now runs the wrapped callback in a more consistent environment by recreating contexts even if they already exist on the stack.
  • Fixed a bug in which stack contexts could leak from one callback chain to another.
  • Yield statements inside a with statement can cause stack contexts to become inconsistent; an exception will now be raised when this case is detected.

tornado.template

  • Errors while rendering templates no longer log the generated code, since the enhanced stack traces (from version 2.1) should make this unnecessary.
  • The {% apply %} directive now works properly with functions that return both unicode strings and byte strings (previously only byte strings were supported).
  • Code in templates is no longer affected by Tornado’s __future__ imports (which previously included absolute_import and division).

tornado.testing

tornado.util

  • tornado.util.b (which was only intended for internal use) is gone.

tornado.web

  • RequestHandler.set_header now overwrites previous header values case-insensitively.
  • tornado.web.RequestHandler has new attributes path_args and path_kwargs, which contain the positional and keyword arguments that are passed to the get/post/etc method. These attributes are set before those methods are called, so they are available during prepare()
  • tornado.web.ErrorHandler no longer requires XSRF tokens on POST requests, so posts to an unknown url will always return 404 instead of complaining about XSRF tokens.
  • Several methods related to HTTP status codes now take a reason keyword argument to specify an alternate “reason” string (i.e. the “Not Found” in “HTTP/1.1 404 Not Found”). It is now possible to set status codes other than those defined in the spec, as long as a reason string is given.
  • The Date HTTP header is now set by default on all responses.
  • Etag/If-None-Match requests now work with StaticFileHandler.
  • StaticFileHandler no longer sets Cache-Control: public unnecessarily.
  • When gzip is enabled in a tornado.web.Application, appropriate Vary: Accept-Encoding headers are now sent.
  • It is no longer necessary to pass all handlers for a host in a single Application.add_handlers call. Now the request will be matched against the handlers for any host_pattern that includes the request’s Host header.

tornado.websocket