Running services in Python as threads: Tornado server and RabbitMQ client

multithreading python rabbitmq tornado

Using the threading module we can easily start multiple services in our application. In this simple example we are going to run a Tornado web server and a RabbitMQ consumer queue using Pika.

import threading
import pika
import tornado.ioloop
import tornado.web

Define a Tornado application with an endpoint to publish a message into the "test" queue:

class MessageHandler(tornado.web.RequestHandler):

    def post(self):
        print 'POST /'
        channel.basic_publish(exchange='',
                              routing_key='test',
                              body=self.get_argument('message'))
        self.set_status(202)


def run_tornado():
    print 'running tornado...'
    app = tornado.web.Application([(r'/', MessageHandler)])
    app.listen(8888)
    ioloop.start()

Create a RabbitMQ consumer and print the messages received:

def consumer_callback(channel, method, properties, body):
    print 'message received: "%s"' % body


def run_pika():
    print 'running pika...'
    channel.basic_consume(consumer_callback, queue='test')
    channel.start_consuming()

In the __main__ define the Tornado event loop and the RabbitMQ channel, run the two services as daemons and allow the user to exit:

if __name__ == '__main__':
    ioloop = tornado.ioloop.IOLoop.instance()
    rabbit = pika.BlockingConnection()
    channel = rabbit.channel()
    channel.queue_declare(queue='test')

    for target in (run_pika, run_tornado):
        thread = threading.Thread(target=target)
        thread.daemon = True
        thread.start()

    print 'services running, press ctrl+c to stop'
    try:
        while True:
            raw_input('')
    except KeyboardInterrupt:
        print 'stopping services...'
        ioloop.stop()
        print 'tornado server stopped'
        rabbit.close()
        print 'rabbitmq connection closed'
        exit(0)

To test it, run the file:

$ python app.py
running pika...
running tornado...
services running, press ctrl+c to stop

Send a message to the queue via the web server:

$ curl -X POST --data "message=hello world" localhost:8888

The message will be printed in the terminal window where you run the file as:

[...]
message received: "hello world"