Blog Archives

Tornado – Escape – Json

Tornado web server exports methods for escape/unescape html, xml, Json among others. This blog discusses about encoding and decoding JSON format.

Tornado has the following methods:

  • tornado.escape.json_encode(value) – JSON’ify the Python object passed to it as argument
  • tornado.escape.json_decode(value) – Converts the JSON string into Python Object

Here’s a usage example:

In this example,

1. When user browses to http://localhost:8888/blog, jsonform.html is rendered that asks for ‘Title’ and ‘Author’

2. On filling this form, a POST request is sent to /blog URL, where the posted arguments are encoded to JSON string with tornado.escape.json_encode() and rendered on the user browser

3. class Language is request handler that caters to http://localhost:8888/lang. In this class, a Python dictionary object is converted to JSON string with tornado.escape.json_encode() and responds with this JSON string to any client request.

4. When tornadojsonclient.py makes a GET request to /lang URL, JSON string is sent as a response. The client decodes this JSON string to a Python dictionary object with tornado.escape.json_decode() method

Code for tornadojsonclient.py below

Tornado – Database MySQL Client Wrapper

Tornado provides a simple MySQL wrapper for performing database operations. Class tornado.databse.Connection acts as a wrapper over MySQLdb DB-API connection.

Consider you have a MySQL installed on your system and you create a DB ‘mydb’ with table ‘post’ and records as below:

mysql> use mydb
Database changed

mysql> create table post (Id int, Title char(50), Author char(50));
Query OK, 0 rows affected (0.03 sec)

mysql> insert into post values (1, 'Tornado Database', 'TechnoBeans');
Query OK, 1 row affected (0.01 sec)

mysql> insert into post values (2, 'Tornado Authentication', 'TechnoBeans');
Query OK, 1 row affected (0.00 sec)

mysql> insert into post values (3, 'Tornado Locale', 'TechnoBeans');
Query OK, 1 row affected (0.00 sec)

mysql> select * from post;
+------+------------------------+-------------+
| Id   | Title                  | Author      |
+------+------------------------+-------------+
|    1 | Tornado Database       | TechnoBeans |
|    2 | Tornado Authentication | TechnoBeans |
|    3 | Tornado Locale         | TechnoBeans |
+------+------------------------+-------------+
3 rows in set (0.02 sec)

Tornado helps you to connect, retrieve and display records from ‘mydb’ MySQL DB as shown in the example below

Output

 

In this example:

1. tornado.database.Connection connects to MySQL DB instance with appropriate server, database name, username and passwd

2. Object of class tornado.database.Connection ‘db’ is then used to query the records of table ‘post’

3. DB connection is then closed with db.close()

4. Records are then rendered by the webserver on accessing http://localhost/posts URL

Note:

While trying out this example, you may bump into this error

ubuntu@ubuntu:~/tornado-2.2$ python tornadodatabase.py 
Traceback (most recent call last):
  File "tornadodatabase.py", line 4, in 
    import tornado.database
  File "/home/ubuntu/tornado-2.2/tornado/database.py", line 20, in 
    import MySQLdb.constants
ImportError: No module named MySQLdb.constants

This is because python-mysqldb is not installed on your system. On ubuntu systems, you can get it with

sudo apt-get install python-mysqldb

Tornado – Asynchronous Requests

Tornado is a non-blocking I/O web server. In Tornado, when the request handler serves the request made by the client, the request is closed by itself. Python decorator @tornado.web.asynchronous can be used to override the default behavior (of automatically finishing the request) and keep the request open. So, its the duty of the server developer to finish the request.

Example

In the above example, When user browses to http://127.0.0.1/, AsyncHandler handles it and sends a GET request to http://google.co.in and receives the response. Processing of this response is done by _async_callback() method. Thus in this example, when get() returns, the request has not been finished. Once the response is processed and self.finish() is called, only then request is completed.

If the developer fails to finish the request with self.finish(), the browser hangs as in the picture below.

Tornado – Error Handling

Web development often calls for gracefully handling of non-existent pages or broken links that users request for. A simple way to achieve error handling in Tornado is with:

  • tornado.web.HTTPError - Exception handler that can used to generate error code of specified type. 
    It is often used to raise an exception as we would see in the example below
  • send_error - Generates an error page of the status code provided along with error description if debug=True

In the example below, when user generates a GET request on http://127.0.0.1:8888/articles/, articles along with author names are shown up.

But when GET request is made on http://127.0.0.1:8888/articles/1, since there is no Id implementation for articles page, error is handled with self.send_error(500) or raise tornado.web.HTTPError(404, ‘Page Not Found Error!!’)

Case1: raise tornado.web.HTTPError(404, ‘Page Not Found Error!!’) is enabled

  • Traceback (most recent call last):                                          —– Traceback is noticed on web page
    File “/home/ubuntu/tornado-2.2/tornado/web.py”, line 988, in _execute
    getattr(self, self.request.method.lower())(*args, **kwargs)
    File “errorhandling.py”, line 12, in get
    raise tornado.web.HTTPError(400, ‘Page Not Found Error!!’)
    HTTPError: HTTP 400: Bad Request (Page Not Found Error!!)
  • WARNING:root:400 GET /articles/1 (127.0.0.1): Page Not Found Error!!     —– Error description where web server is running
    WARNING:root:400 GET /articles/1 (127.0.0.1) 10.68ms

Case2: self.send_error(500) is enabled

  • ’500: Internal Server Error’ web page shows up
  • ERROR:root:500 GET /articles/1 (127.0.0.1) 0.41ms error line is generated where web server is running

Example:

Tornado – Templates – Run time and Cached

Website development often calls for reuse of pages. For instance, when you open your Citibank account after login the welcome page you see is same what other bank customers see but its customized with your name and settings. Do you think Citi creates so many web pages for all its customers? Well, they use, what’s called as template. Template is a layout or a skeleton that separates content from presentation and helps in mass production of web pages..

Control Statements in Templates

Tornado supports templates with expressions and control statements; straight from Tornadoweb, “Control statements are surronded by {% and %}, e.g., {% if len(items) > 2 %}. Expressions are surrounded by {{ and }}, e.g., {{ items[0] }}.”

In the above example, we have created a template (template.html) that requires a scalar value for title and a dictionary for listing dictionary items in the form of a table (List data types also work well in Tornado templates). The python code provide these values in the code line

self.render(‘template.html’, title=’Articles’, dict=articles)

Output

tornado templates example

Run-time Template Generation and Compiling Templates

Let’s now understand two important methods in Tornado templates, how do they work and how are they implemented:

  1. Template()
  2. Loader()

Template() – helps in generating a template at run time. For instance, there is a web page with lesser content and that need not be rendered quite often, we can utilize this Template method.

Loader() – helps in compiling and caching templates from a predefined location of your machine. Compiling and caching improves performance of web server while rendering web pages.

Example 2

In example 2,

t = Template(“<html>{{ name }}</html>”) — generates template at run time

self.write(loader.load(“template.html”).generate(name=”John”))  — template is compiled, cached and loaded

Python Easter Eggs

This is something I discovered incidentally… On the Python interpreter try out these commands

C:\Python27\pylibrary\PyLibrary>python
ActivePython 2.7.2.5 (ActiveState Software Inc.) based on
Python 2.7.2 (default, Jun 24 2011, 12:21:10) [MSC v.1500 32 bit (Intel)] on win32
Type “help”, “copyright”, “credits” or “license” for more information.

>>> import this  –> Prints out Zen of Python
The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren’t special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one– and preferably only one –obvious way to do it.
Although that way may not be obvious at first unless you’re Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it’s a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea — let’s do more of those!

>>> from __future__ import braces  –> Suggests there are no braces in Python (not a chance :) )
File “<stdin>”, line 1
SyntaxError: not a chance

>>> import antigravity  –> opens up xkcd on Python

>>> import __hello__
Hello world…

>>> import __hello__  –> doesnt print anything, it needs to be reloaded like below

>>> reload(__hello__)  –> reloading __hello__
Hello world…
<module ‘__hello__’ from ‘<frozen>’>

 

5 Ways of Fibonacci in Python

After learning so much about development in Python, I thought this article would be interesting for readers and to myself…

This is about 5 different ways of calculating Fibonacci numbers in Python

## Example 1: Using looping technique
def fib(n):
 a,b = 1,1
 for i in range(n-1):
  a,b = b,a+b
 return a
print fib(5)

## Example 2: Using recursion
def fibR(n):
 if n==1 or n==2:
  return 1
 return fib(n-1)+fib(n-2)
print fibR(5)

## Example 3: Using generators
a,b = 0,1
def fibI():
 global a,b
 while True:
  a,b = b, a+b
  yield a
f=fibI()
f.next()
f.next()
f.next()
f.next()
print f.next()

## Example 4: Using memoization
def memoize(fn, arg):
 memo = {}
 if arg not in memo:
  memo[arg] = fn(arg)
  return memo[arg]

## fib() as written in example 1.
fibm = memoize(fib,5)
print fibm

## Example 5: Using memoization as decorator
class Memoize:
 def __init__(self, fn):
  self.fn = fn
  self.memo = {}
 def __call__(self, arg):
  if arg not in self.memo:
   self.memo[arg] = self.fn(arg)
   return self.memo[arg]

@Memoize
def fib(n):
 a,b = 1,1
 for i in range(n-1):
  a,b = b,a+b
 return a
print fib(5)

You may ask, all this is okay, but what’s the best way? Wait for another post on performance of these… Its on the way!

Overloading in Python

Some say Python doesn’t allow overloading of methods or constructors. Well, they are right in some way!  A coding example below,

def add(a,b):
    return a+b

def add(a,b,c):
    return a+b+c

print add(4,5)

If you try to run the above piece of code, you get an error stating, “TypeError: add() takes exactly 3 arguments (2 given)”. This is because, Python understands the latest definition of method add() which takes only two arguments. Even though a method add() that takes care of three arguments exists, it didn’t get called. Hence you would be safe to say, overloading methods in Python is not supported.

But, then there are folks who are more than willing to say, ‘Oh! Python supports all!’ Yes, Python supports overloading but in a Pythonic way. Here’s an example,

def add(instanceOf, *args):
    if instanceOf == 'int':
        result = 0
    if instanceOf == 'str':
        result = ''
    for i in args:
        result = result + i
    return result

print add('int', 3,4,5)
print add('str', 'I',' am',' in', ' Python')

Output:
12
I am in Python

In the above code snippet, two things are achieved:

-          Irrespective of the different number of arguments, method add() works well

-          Also, based on the data type of input, data  type of output is changed

So, overloading IS there in Python!! :)

Performance for testing memberships: list vs tuples vs sets

Sets in Python are often used for two purposes:

1. Removing the duplicate entries in a collection

2. For membership testing.

By membership, here we mean to find existence of element in a collection

The focus of this post is to evaluate performance of list, tuple and set data structures with respect to each other on the basis of membership testing.

How do I do it? Well, some code, some data collection and some analysis

from datetime import datetime
listA = range(10000000)
setA = set(listA)
tupA = tuple(listA)

def calc(data, type):
    start = datetime.now()
    if data in type:
        print ""
    end = datetime.now()
    print end-start

calc(9999, listA)
calc(9999, tupA)
calc(9999, setA)

Below is the average of 10 iterations for the time taken by lists/tuples and sets for testing membership of 9999, 99999, 999999 in 10000000

Search 9999 in  10000000
list tuple set
0.8 0.8 1.9
Search 99999 in  10000000
list tuple set
2.6 2.8 1.7
Search 999999 in  10000000
list tuple set
21.4 21.6 0.8

Conclusions

1. for testing membership of elements at the beginning of the collection, lists or tuples perform more than 100% better than sets

2. As soon as you start testing membership of elements that are in the middle or end of the collection, sets perform 40% – 1800% better than lists or tuples

You now have a fair idea why you should think of using sets for large collections…

Note

Hardware I used: windows 7, x64, 4GB RAM

Python version: 2.7.2

Python Decorators

Let’s start with something simple..What is a decorator?

According to python.org, “A decorator is the name used for a software design pattern. Decorators dynamically alter the functionality of a function, method, or class without having to directly use subclasses or change the source code of the function being decorated”

A classic example tat I can think of right now is related to performance. What if we have requirement to calculate the total time taken by a method to perform its operations. Will you always end up writing code for getting the time difference for every method call you make? Well, yes if I had to do it for 4-5 methods, but what if I have to do for 50s or 60s of methods across my Python code? Then? Python decorators come to the rescue.

How? You could write a simple Python decorator method that:

1. Gets the start time

2. Calls the method for which the time taken has to be calculated

3. Gets the time when the method has completed its computation

4. Prints the time difference between end time (3) and start time (1)

Simple! Let’s see some code

from datetime import datetime
def decorate(f):
    start = datetime.now()
    f(3000)
    end = datetime.now()
    print end-start

@decorate
def fact(n):
    fact = 1
    for i in range(1,n+1):
        fact *= i

## Decorator on argument function

from datetime import datetime

def timer(fn):
 first = datetime.now()
 fn(3000)
 sec = datetime.now()
 print sec-first

@timer
def fn(*args):
 for i in range(args[0]):
  pass

Output of this code snippet is 0:00:00.006000 (time taken to calculate factorial of 3000 on my system)

Follow

Get every new post delivered to your Inbox.