<< Tech Portfolio /

Real Time Web Service with Tornado

I built this for a startup in San Francisco.

Client : stealth

Features

The problem is to build a web service platform that can serve a request in sub 10 milli-second (ms) time frame. And serve millions of requests per day. The requests needed to be filled dynamically, they do access the database.

Lot of existing web frameworks didn't fit the bill, as they are oriented towards serving web pages (1-5 seconds response time). Plus they have lots of functionality to help render web pages (html generators, templates ..etc). We don't need any of that for a straight forward web service. So I ruled them out. Even Apache web server is dropped from the architecture.

I evaluated writing a simple 'socket server' in Java or Python. Then I discovered Tornado Web Server. Tornado was developed at Friend Feed to serve up their web pages. When the company was acquired by Faceobook, Tornado was open-sourced.

From Tornado website:

The framework is distinct from most mainstream web server frameworks (and certainly most Python frameworks) because it is non-blocking and reasonably fast. Because it is non-blocking and uses epoll, it can handle thousands of simultaneous standing connections, which means it is ideal for real-time web services. We built the web server specifically to handle FriendFeed's real-time features — every active user of FriendFeed maintains an open connection to the FriendFeed servers

I choose Tornado because

  • very light weight compared to other web frameworks
  • in Python
  • essentially a 'socket listener' that delegates handling different URLs to 'python handlers'
  • handlers are very simple to write
  • non-blocking and can serve a lot of connections at the same time

I developed a webservice platform based on Tornado server. The system has served us very well....

  • Tornado framework is very light weight and easy to learn. So I could focus on writing custom code (handlers) rather than learning in-and-outs of the framework
  • soo fast, when we measured the performance using Apache Bench (ab) the client couldn't max it out!
  • one tornado server per CPU core. So I can run multiple tornado instances on a multi-core machine
  • with a load balancer and multiple machines, we can easily scale to handle the work load
  • can easily add alternate 'url handlers' to serve different types of web service requests
  • being in Python means great runtime performance. Plus we have access to all open-source python libraries (JSON, mysql ..etc)
  • all infrastructure is built on Amazon EC2 environment