Author Archives:
Express for Node.js
Express is a simple web application framework for building hybrid web applications. You can render web pages, generate customized responses and create user-friendly APIs quickly with ease. It just fits the trade with Node.js.
This post covers the installation and hello world example:
Installation:
buntu@ubuntu:~$ sudo npm install express
express@3.1.0 node_modules/express
├── methods@0.0.1
├── fresh@0.1.0
├── cookie-signature@0.0.1
├── range-parser@0.0.4
├── buffer-crc32@0.1.1
├── cookie@0.0.5
├── debug@0.7.2
├── commander@0.6.1
├── mkdirp@0.3.3
├── send@0.1.0 (mime@1.2.6)
└── connect@2.7.2 (pause@0.0.1, bytes@0.1.0, formidable@1.0.11, qs@0.5.1)
Hello World Example:
In the above example:
- 1. express module is imported and application object is created
- 2. Object’s get() method is called and callback is set to respond with ‘hello world’
- 3. When user sends a GET request at port 3000, an event is generated that is handled by callback function(request, response)
Installing Node.js on Ubuntu
I would continue explaining more about Node.js, but for all those folks who do things and understand them, here’s the way to install Node.js on Ubuntu machines..
You need to first install necessary packages like git-core and libssl-dev
sudo apt-get update sudo apt-get install g++ curl libssl-dev apache2-utils sudo apt-get install git-core
Download the latest source and build it
wget http://nodejs.org/dist/v0.8.2/node-v0.8.2.tar.gz gunzip node-v0.8.2.tar.gz tar xvf node-v0.8.2.tar.gz cd node-v0.8.2 ./configure sudo make install
Check the version after install
node -v
Node.js Async
In my previous post I discussed how asynchronous (with callbacks) nature of Node.js is useful to develop non-blocking server side implementations. Let’s now see how async functions are developed with an example.
Async functions are typically developed with a single callback with result arguments and an optional error.
Here’s an example code:
Now if you run the above code as: node async.js the output is:
step1 Out of async step2 step3 value is:6.082818640342675e+62
Let’s explain what we did here:
We first define a function async_factorial with a callback. Inside async_factorial we do something interesting; we use process.nextTick. We know node runs an event loop and handle i/o operations asynchronously with the help of callback invocations. With process.nextTick, you register the supplied function or piece of code to be executed during the next iteration of event loop. Typically a queue of functions is maintained. So whenever the event loop is free from I/O events, it starts with the next tick or next loop, it then first checks for the queue and executes the functions in the queue and then checks for I/O events and continues the loop. Thus if you need to perform cpu intensive operation and at the same time don’t want to block any I/O event, you can use this trick to perform cpu operation in async way with callbacks using nextTick. This ensures that event loop is free and handle I/O as well. In this case, we process factorial(50) in async way with nextTick.
Execution flow:
1. On calling async_factorial the flow control reaches step1.
2. With process.nextTick we expect, calculation of factorial(50) happens in the next iteration event loop and only would the callback(fact) gets called.
3. Since async_factorial is waiting on callback for result, the next statement ‘Out of asyc’ gets executed in this iteration of event loop.
4. When the next event loop begins, the flow goes to step2, calculates factorial and returns the factorial value through callback
5. Since the result is now available the code inside async_factorial gets executed and we see step3 and value is:6.082818640342675e+62 being printed out.
Note: You could also use setTimeout instead of nextTick that provides you the option of setting the time to wait before performing operation. But as the documentation says, nextTick is more efficient.
Understanding node.js
Essentially Node has an approach of making I/O operations non-blocking. All the I/O operations run independently and generate an event when they are done. Node follows an execution model of having an event loop (a concept existing in JavaScript) that handles events that are generated by I/O operations ( like database, HTTP requests, file I/O s). Whenever an I/O operation occurs, the event loop handles the event and processes it by converting it into callback invocation. With this mechanism, the I/O operation is handled in the background and the callback gets executed when the required data is available. This is also called as asynchronous way of handling events.
Node expects that there is no fixed time within which the I/O is performed. So instead of waiting for the result, Node generates an event (which are in-turn handled by callbacks) and hence the tasks get performed in “parallel” (nothing to do with multi-core and true parallelism; in fact Node can’t take advantage of multi-core as its a single threaded event loop).
In the hello world example, we have a HTTP server listening on port 8888. When the server receives an I/O activity (http GET request) an event (request) is fired which is tied to callback (anonymous function(request,response) ) which in turn prints ‘Hello World’. Node uses Linux libraries like epoll, select (as does Python Tornado) to get notified that a user has made a connection with the server.
Let’s understand the concept with an example of database call being made in the web server.
var result = mysql.execute(query);
console.log("I am done");
The above code is synchronous in nature. If a database call takes 15 secs to run, HTTP server is blocked. One db request like above can block the web server and as a result any other following requests can’t be served until the first request is complete.
Solution? Let’s try asynchronous then:
mysql.execute(query, function(result) {
console.log("callback");
});
console.log("I am done");
In the above code, database request is made async with the help of callback. So whenever the database query results are available, an event is generated that is handled by the callback & “callback” string is printed out, but before that the event loop moves ahead and prints “I am done”. It’s crucial to note that mysql.execute function itself should be written in an async manner for this to be successful.
Almost all the node api’s are async in nature and can be taken advantage of. But how do I develop my async function? Hmm.. you must read my next post 
Node.js
This is an introduction post to node.js..
Node.js is an event driven, non blocking (async) I/O style software that is used to develop server side implementations. (If you’re a Python fan, it’s like programming in Twisted!). It’s also built on Google’s V8 JavaScript engine. Like other event driven servers, Node.js, runs an event loop and handles the events asynchronously with callback invocations.
A typical ‘ hello world ‘ web server implementation can be found below. Run as: node helloworld.js
In this example,
1. we import module http and create a http server that listens on port 8888.
2. When the user makes a Get request on http://localhost:8888, the web server renders Hello World on the web browser. You can play around with request and response variables as used in the server code.
3. It’s interesting to note that if reponse.end() statement is commented out, the request doesn’t complete and the server hangs. If you press Ctrl+C, only then the request completes and ‘Hello World’ is rendered on the browser. So server developers, beware!
You may ask, what’s this function(request, response) and why it’s anonymous? Well, that’s a motivation for you to read my next post!
20K + hits on Technobeans…
Just another milestone that proves Technobeans is meeting its objective of making community aware of technologies and helping them solve their problem.. Thanks
Tornado Internals
Well, what a piece of technology and every time you read more about it the better you know about it and can better appreciate it. Yes, I’m talking about the very own Tornado Web Server. My ‘attempt’ here is to tell you about the workflow of Tornado internals…
Tornado is a non blocking web server as we all know and understand. But how does it get this non blocking thing going its way?
Let’s first understand that Tornado is event driven web server. But what does event driven programming mean? Well it means your application thrives an event loop (single threaded) that keeps polling for events, identifying them as wanted and then handling them. Tornado also works on similar principles.
Tornado web server runs an ioloop, a single threaded main event loop and is at the crust of async nature of Tornado. ioloop.add_handler(fd, handlers, events) maintains a list of file descriptors, events to be watched and the corresponding handlers for each of these fd.
ioloop is a user space construct but who listens to events on the fd’s. That should be a kernel library and Tornado uses epoll, kqueue(BSD) – libraries that provide event notifications in a non-blocking way. epoll has three main functions:
- epoll_create – creates an epoll object
- epoll_ctl – controls the file descriptors and events to be watched
- epoll_watch – waits until registered event occurs or wait till timeout
epoll thus watches file descriptors (sockets) and returns needed (READ, WRITE & ERROR) events.
As described above, Tornado’s ioloop consumes these events (for the file descriptors) and run associated handlers for these events.
tornado.IOStream works as an abstraction layer on top of sockets. It provides three methods:
- read_until() – reads the socket until it finds empty line delimiter that suggests completion of HTTP headers
- read_bytes() – reads N number of bytes from socket
- write() – write a buffer to socket
All of these methods can call a callback when their job is done.
tornado.httpserver is a non blocking http server that accepts connections from clients on a defined port by adding the sockets to the ioloop.
- http_server = httpserver.HTTPServer(handle_request)
- http_server.listen(8888)
- ioloop.IOLoop.instance().start()
handler argument as mentioned in ioloop is a callback accepts the new connection, creates a IOStream, and creates a HTTPConnection object of httpserver class that is now responsible handling all client requests.
Selenium with Python bindings
After a lot of posts on Tornado web server and understanding BDD, lets get to testing our website. What better than to you selenium. Lets go through the setup and create our first test..
Prerequisites
1. Python bindings for Selenium – Go to, selenium site and download the package
Install as:
- tar xvf selenium-2.25.0.tar.gz
- cd selenium-2.25.0
- sudo python setup.py install
2. Java Server – Download the server from here
Run as:
- java -jar selenium-server-standalone-2.25.0.jar
Here we discuss the usage of Selenium 2.0 Web Driver, with/without selenium server. Below are the examples of each of these:
Just a bit of history first… Web Driver aims to improve Selenium 1.0 Remote Control. The distinguishing factors being:
- Object Oriented APIs
- More features
- Web Driver uses the APIs exported by the browser for automated testing while Selenium Remote Control injects Javascript to run the test
Web Driver without selenium server
from selenium import webdriver
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.common.keys import Keys
import time
browser = webdriver.Firefox() # Get local session of firefox
browser.get("http://www.yahoo.com") # Load page
assert "Yahoo!" in browser.title
Web Driver with selenium server – WebDriver Remote
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
driver = webdriver.Remote(
command_executor='http://127.0.0.1:4444/wd/hub',
desired_capabilities=DesiredCapabilities.FIREFOX)
driver.get("http://www.python.org")
driver.close()
BDD in Python with lettuce
Behavior Driven Development, also known as BDD, is a concept developed by Dan North and is based on a popular and well adopted TDD. As in Dan’s words -
‘BDD is a second-generation, outside–in, pull-based, multiple-stakeholder, multiple-scale, high-automation, agile methodology. It describes a cycle of interactions with well-defined outputs, resulting in the delivery of working, tested software that matters.’
BDD provides a framework where QA, Business Analysts and other stake-holders communicate and collaborate on sotware development. While TDD emphasized on developing tests for unit piece of code. BDD insists on developing tests for business scenarios or use cases or behavioral specification of software being developed. According to Dan, BDD tests should be written as user stories ‘As a [role] I want [feature] so that [benefit]‘ and Acceptance criteria should be defined as ‘Given [initial context], when [event occurs], then [ensure some outcomes].‘
lettuce is typically used in Python to implement BDD. This blog covers the installation of lettuce on Ubuntu and its application with an example of fibonacci function
Installation
buntu@ubuntu:~$ sudo pip install lettuce
[sudo] password for buntu:
Downloading/unpacking lettuce
Downloading lettuce-0.2.9.tar.gz (40Kb): 40Kb downloaded
Running setup.py egg_info for package lettuce
Downloading/unpacking sure (from lettuce)
Downloading sure-1.0.6.tar.gz
Running setup.py egg_info for package sure
Downloading/unpacking fuzzywuzzy (from lettuce)
Downloading fuzzywuzzy-0.1.tar.gz
Running setup.py egg_info for package fuzzywuzzy
Installing collected packages: fuzzywuzzy, lettuce, sure
Running setup.py install for lettuce
Installing lettuce script to /usr/local/bin
Running setup.py install for sure
Running setup.py install for fuzzywuzzy
Successfully installed lettuce
Setup
Let’s first create a directory structure that looks like this
buntu@ubuntu:~$ tree lettucetests/
lettucetests/
|– features
| |– fib.feature
| |– test.py
`– test.feature
1 directory, 3 files
Define Features
Write Tests
