1 of 19

Async networking in python

 

With

gevent,concurrence,twisted,eventlet(and node.js)

Presentation by Uriel Katz

2 of 19

Lecture Topics

  • Asynchronous networking in a nutshell
  • Quick overview of python async networking frameworks 
  • Node.js as example for frameworks in other languages
  • Comet chat example - client side
  • Comet chat backend example in each framework
  • My work on gevnet

3 of 19

Async networking in a nutshell

  • another way to solve the problem of handling many conncurrent users.
  • the basic idea is to get a notification when a socket is ready for some IO operation and continue with other stuff until IO is available.
  • using one thread so blocking calls block all requests!
  • only good for IO-heavy servers,computation heavy code must run in other thread.
  • the basic way to achive this is using the select syscall,select accept three sets of file descriptors
  • select(read_fds,write_fds,error_fds) blocks until a event occurs

4 of 19

Overview of python frameworks

  • Gevent - libevent based with greenlets (started as a fork of eventlet).
  • Twisted - reactor based with callbacks
  • Concurrence - libevent based with greenlet/stackless support,from Hyves(the Dutch facebook).
  • Eventlet - libevent/epoll based with greenlets from Linden Lab(Second Life creators).

5 of 19

Twisted

  • probably the first answer to the question: "how do I make a high performance server in python?"
  • based on a reactor pattern and deferred(aka Future).
  • very mature and used by many projects.
  • has thread support and support many network protocols. 
  • integrates with other event loops.
  • has a steep learning curve.
  • uses callbacks with deferreds which lead to quite a messy code.

6 of 19

Gevent

  • started as a a fork of Eventlet,now a library by itself.
  • provide a API consistent with the standard library.
  • built around libevent and greenlet.
  • has a wsgi server based on libevent-http - very fast!
  • very good documentation and community
  • monkey patching that works!
  • gives you a synchronous API on top of asynchronous networks.
  • used by Omegle.com in production,replaced twisted.
  • very young and cutting edge.
  • my favourite of all python frameworks.

7 of 19

Concurrence

  • from Hyves(the dutch facebook)
  • libevent based with greenlets(also work with stackless).
  • uses channels as primitives to communicate between tasklets(their name for greenlets).
  • has a DBAPI 2.0 compatible MySQL driver!
  • doesn`t seem active.
  • quite slow.
  • doesn`t have a  standard socket implementation and no monkey patching :( 

8 of 19

Eventlet

  • from Linden Labs,the guys who made Second Life.
  • similar to gevent but:
  • has monkey patching on steroids!
  • thread-safe,you can have multiple threads running their own loop.
  • thread pool builtin.
  • DB API pool using threads - access to all DBAPI drivers
  • Websockets support.
  • Has a pluggable event loop.
  • Slower than gevent due to pure python event loops.
  • Big and messy compared to gevent.

9 of 19

Something extra - Node.js

  • Evented IO on top of Google V8 JavaScript Engine.
  • uses libev for the event loop,libeio for file IO.
  • Lets you write high performance servers in JavaScript!
  • Uses callbacks as a notifications,similar to how event callbacks work in the browser.
  • has async file IO(this thing is very hard!)
  • has a rock solid HTTP implementation.
  • has a strong community and is the current buzz.
  • very fast,almost as fast as nginx in micro benchmarks.
  • by using callbacks it suffer from the same problems as twisted,for complex code you will get a lot of nested callbacks.

10 of 19

Comet chat examples

  • a simple chat room using long polling(aka COMET).
  • the client side is the same for all frameworks
  • the following urls are implemented in each framework:
    • / - just serve the index.html
    • /wait - the comet long polling,this page doesn`t return until a new message is recived
    • /new_message - used to post a new message to the room
  •  uses jQuery and jQuery UI.

11 of 19

Comet chat - client side code

<html>�  <head>�    <script src="http://ajax.googleapis.com/ajax/libs/jquery/1.4.2/jquery.min.js"></script>�    <script src="http://ajax.googleapis.com/ajax/libs/jqueryui/1.8.2/jquery-ui.min.js"></script>�    <link href="http://ajax.googleapis.com/ajax/libs/jqueryui/1.7.0/themes/base/jquery-ui.css" rel="stylesheet"/>�    <script>�      $(function()�      {�    $("#dialog-form").dialog({�        autoOpen: false,�        height: 300,�        width: 350,�        modal: true,�        buttons: {�          'OK': function() {�              $(this).dialog('close');�              waitForMessage();�          },�        }�    });��    function newMessage()�    {�        $.get('/new_message?msg=' + $("#txtMsg").val() + '&name=' + $("#dialog-form input").val(),function(){});�        $("#txtMsg").val("");�    }�

12 of 19

Comet chat - client side code(cont.)

�    function waitForMessage() {�        $.get('/wait',function(msg){�        var div = $("<div>" + msg + "</div>");�        $("#messages").append(div);�        div.effect("highlight", {}, 1000);�        waitForMessage();�        });�    }��    $("input[type=button]").click(newMessage);�    $("#dialog-form").dialog('open');�      });�    </script>�  </head>�  <body>�      <div id="messages" style="border:1px solid #C0C0C0;width:600px;height:300px;overflow-y: scroll;">�    �      </div>�      <input type="text" id="txtMsg"/>�      <input type="button" value="post"/>�      <div id="dialog-form">�    Enter your name:<input type="text"/>�      </div>�  </body>�</html>

13 of 19

Twisted example

from twisted.internet import reactor,defer�from twisted.web.resource import Resource�from twisted.web.static import File�from twisted.web.server import NOT_DONE_YET,Site��clients = []��class WaitForMessages(Resource):�    def render_GET(self, request):�    clients.append(request)�        return NOT_DONE_YET�        �class NewMessage(Resource):�    def render_GET(self,request):�    global clients�    for client in clients:�      try:�        client.write(request.args["name"][0] + ":" + request.args["msg"][0])�        client.finish()�      except:#disconnected�        pass�    clients = []�    request.write("1")�    request.finish()��resource = Resource()�resource.putChild("",File("index.html"))�resource.putChild("wait",WaitForMessages())�resource.putChild("new_message",NewMessage())��factory = Site(resource)�reactor.listenTCP(8080, factory)�reactor.run()��

14 of 19

Concurrence example

import urlparse�from concurrence import dispatch, Tasklet, Message�from concurrence.http import WSGIServer��class MSG(Message): pass�connected_clients = set()��def chat(env, start_response):�    start_response('200 OK', [('Content-Type', 'text/html')])�    if env['PATH_INFO'] == '/':�    return [open("index.html","rb").read()]�    elif env['PATH_INFO'] == '/wait':�      connected_clients.add(Tasklet.current())�      try:�    for msg,args,kargs in Tasklet.receive():�      return [args[0]]�      except Exception,e:�    return [str(e)]�    elif env['PATH_INFO'].startswith('/new_message'):�      qs = urlparse.parse_qs(env['QUERY_STRING'])�      for client in connected_clients:�    try:�      MSG.send(client)(qs["name"][0] + ":" + qs["msg"][0])�    except:#disconnected�      pass�      return ["1"]�    else:�      return ["Not Found"]�def main():�    server = WSGIServer(chat)�    server.serve(('localhost', 8084))�if __name__ == '__main__':�    dispatch(main)�

15 of 19

Gevent example

import urlparse�from gevent import event,pywsgi��new_message = event.AsyncResult()��def chat(env, start_response):�    global new_message�    start_response('200 OK', [('Content-Type', 'text/html')])�    if env['PATH_INFO'] == '/':�    return [open("index.html","rb").read()]�    elif env['PATH_INFO'] == '/wait':�      result = new_message.wait()�      return [result]�    elif env['PATH_INFO'].startswith('/new_message'):�      qs = urlparse.parse_qs(env['QUERY_STRING'])�      new_message.set(qs["name"][0] + ":" + qs["msg"][0])�      new_message = event.AsyncResult()�      return ["1"]�    else:�      return ["Not Found"]�        �server = pywsgi.WSGIServer(('', 8083), chat)�server.serve_forever()

16 of 19

Eventlet example

import eventlet,urlparse�from eventlet import wsgi,event��new_message = event.Event()��def chat(env, start_response):�    start_response('200 OK', [('Content-Type', 'text/html')])�    if env['PATH_INFO'] == '/':�    return [open("index.html","rb").read()]�    elif env['PATH_INFO'] == '/wait':�      result = new_message.wait()�      return [result]�    elif env['PATH_INFO'].startswith('/new_message'):�      global new_message�      qs = urlparse.parse_qs(env['QUERY_STRING'])�      new_message.send(qs["name"][0] + ":" + qs["msg"][0])�      new_message = event.Event()�      return ["1"]�    else:�      return ["Not Found"]�        �wsgi.server(eventlet.listen(('', 8081)), chat)�

17 of 19

Node.js example

var http = require('http'),fs=require('fs');�var clients = [];�http.createServer(function (request, response) {�  var parts = require('url').parse(request.url,true);�  response.writeHead(200, {'Content-Type': 'text/html'});�  if(parts.pathname == "/")�  {�    var data = fs.readFileSync("index.html");�    response.end(data);�  }�  else if(parts.pathname == "/wait")�  {�    clients.push(response);�  }�  else if(parts.pathname == "/new_message")�  {�    for(var i=0;i<clients.length;i++)�      clients[i].end(parts.query.name + ":" + parts.query.msg);�    clients = [];�    response.end("1");�  }�  else�  {�      response.end('Not Found');�  }�}).listen(8082);

18 of 19

My work on and with gevent

  • convincing that pywsgi is needed,it was deprecated in 0.12,now it is part of 0.13
  • patches to pywsgi (applied in 0.13)
  • waitany function(issue 23)
  • a streaming proxy to Rackspace cloud files,handles 500 concurrent connections with 45MB of memory!
  • comet message bus.
  • real time drawing like google drawing.

Stuff done at work:

19 of 19

Thanks!

Slides and code available at:�http://www.urielkatz.com