Fork me on GitHub

Monday, January 18, 2010

Are we stuck in a Bratty Programming Model?


This is more of an essay or more of my thoughts than a blog post. So brace yourself to read some lengthy writing :).

Warning - Thoughts are evolving and so may not be complete as you think. If you can help it to move in the right direction, I welcome you.

Ever had your day spoiled by a kid who demands what he needs and doesn't budge until you give it to him (cries, pleads, rolls on the floor and on extreme cases punches and kicks :). The problem with him is he won't let you do anything else till he gets what he wants. But how does it relate to programming? I think most of our programs behave in the same way as the bratty kid.

Let's take a problem - Say we are working on some ticket reservation app. The need is - Given a ticket id, get the reservation info, do some processing on that and then go on to do some other important stuff.

Expressing the above spec in ruby:


reservation = remote_service.get_reservation_info(ticket_id)
process_ticket(reservation)
#Program wishes to continue to do some other important stuff.


The above code is simple and we have all written this code at some point in time. But there seems to be a problem with the piece of code. It demands for some information from a remote service and doesn't move from there until it gets what it demands. Also it doesn't let rest of the code to execute as well until its demands are met. It is not just for service calls.. Think of using the same mode for network calls, database access, file system operations, long running computations. All of them can become bratty and stop your program from moving forward.

Are programs supposed to behave like this? Is this a better method of writing code? For simple code of not much serious use, the above model is acceptable. But when we start moving to systems which need to grow in scale this programming model is not sustainable.

Programs are supposed to be magnanimous in nature. A line of code should behave in a way that if it is not able to get something for moving forward, should move out of the line to let others execute and wait for its input. How would that actually work out in code?

Now lets jump into code which can give a hint of how to solve this. Below is the same ruby code written in a different way.


remote_service.get_reservation_info(ticket_id).addCallback {|reservation| process_ticket(reservation)} # I will get the input when it is available and process it
# other lines of code executing when above code waits for reservation info...


So the above lines of code does the same as above semantically. But the difference between the two is in this piece of code the retrieving of reservation information and processing it doesn't seem to block the program from moving forward. I know just adding a callback doesn't solve the problem. There must be an underlying infrastructure to handle this. And this is where event loops come in. But I think you get the idea of what I am trying to say.

Event loops are a bigger subject of discussion. But we have been using them most of the times. Take GUI programming as example.

The I/O or more importantly programing models we have been using till now have predominantly been focusing being imperative (Do this step, get the result, then do that and continue...) but I think it's time we understand that this model can't help us long. If we need our programs to scale or even be reasonable we need to move from bratty to a magnanimous programming model. Blocking other's from moving because we still need something is not a very scalable model both in real life and programming :).

This is my rant of current programming models. There may be better ways to do things and if you know any please let me know. A lot of this essay is inspired by node.js work done by Ryan Dhal and EventMachine which made non bratty programming so much easier for me to handle.

P.S. Using event loops has a different set of constraints like new way of thinking (inverted control), different way of exception handling as well using different frameworks which we may not be familiar with.

Saturday, January 16, 2010

DNS lookup Experiment in Ruby - Blocking and NonBlocking Modes

Normal DNS lookup in Ruby (Blocking Mode)

require "resolv"

hosts = %w[www.yahoo.com www.google.com twitter.com github.com]

start = Time.now

hosts.each do |host|
p "Getting address for #{host}"
p Resolv.getaddress(host)
end

finish = Time.now

p "Time take to finish #{finish - start} seconds"


Execution Result -

"Getting address for www.yahoo.com"
"69.147.76.15"
"Getting address for www.google.com"
"209.85.231.104"
"Getting address for twitter.com"
"168.143.171.84"
"Getting address for github.com"
"207.97.227.239"
"Time take to finish 40.313232 seconds"

Non Blocking DNS Lookup in Ruby (Using EventMachine Deferrable)

require "rubygems"
require "resolv"
require "eventmachine"

class Dns
include EM::Deferrable
def resolve_hostname(hostname)
begin
ip = Resolv.getaddress(hostname)
set_deferred_status :succeeded, ip
rescue Exception => ex
set_deferred_status :failed, ex.to_s
end
end
end

start = Time.now
EventMachine.run {
p "Requesting DNS info for yahoo"
dns0 = Dns.new
dns0.callback {|response| p "For yahoo #{response}"}
dns0.errback {|response| p "For yahoo #{response}"}
Thread.new { dns0.resolve_hostname "www.yahoo.com"}

p "Requesting DNS info for google"
dns1 = Dns.new
dns1.callback {|response| p "For google #{response}"}
dns1.errback {|response| p "For google #{response}"}
Thread.new { dns1.resolve_hostname "www.google.com"}

p "Requesting DNS info for twitter"
dns2 = Dns.new
dns2.callback {|response| p "For twitter #{response}"}
dns2.errback {|response| p "For twitter #{response}"}
Thread.new { dns2.resolve_hostname "twitter.com"}

p "Requesting DNS info for github"
dns3 = Dns.new
dns3.callback {|response| p "For github #{response}"}
dns3.errback {|response| p "For github #{response}"}
Thread.new {dns3.resolve_hostname "github.com"; EM.stop }
}
finish = Time.now

p "Time take for querying #{finish - start} seconds"



Execution Result -

"Requesting DNS info for yahoo"
"Requesting DNS info for google"
"Requesting DNS info for twitter"
"Requesting DNS info for github"
"For google 209.85.231.99"
"For yahoo 69.147.76.15"
"For github 207.97.227.239"
"For twitter 128.121.146.100"
"Time take for querying 20.311271 seconds"

The results are obvious :). You can see the difference between the blocking and non blocking modes and the effect they may have on your systems. I am also planning to write a DNS library based on NeverBlock which uses Fibers as a way to do non blocking concurrency.

EventMachine is an excellent piece of software which gives a lot of functionality to do things asynchronously. Especially the Deferrable pattern is an awesome way to do long running jobs or calculations and still not block the whole world from running. I have been exploring EventMachine for sometime since I started writing em-couchdb.

The code is written in a simplified way and may not be of direct use to production but hope it reflects the principles of value of doing things in non blocking way.

Wednesday, January 13, 2010

ChromeWatir + Watir-WebDriver Update [Important]

Interesting development in Watir world but not a completely unanticipated one. I started playing around with using WebDriver as platform for Watir in the form of FireDriver and ChromeWatir.

Yesterday Jarib has released a Watir abstraction on top of WebDriver. I am planning to move the effort of working on ChromeDriver to Watir-WebDriver which has ChromeDriver in it.

Installation of Watir-WebDriver

> sudo gem install selenium-webdriver
> git clone git://github.com/saivenkat/watir-webdriver.git

Example code in watir-webdriver (ChromeDriver)

require "watir-webdriver"
browser = Watir::Browser.new(:chrome)
browser.goto "http://www.google.com"
browser.text_field(:name, "q").set "Watir"
browser.button(:name, "btnG").click

The code is same as Watir except the driver changes internally.
The repo is in http://github.com/saivenkat/watir-webdriver. Feel free to fork and enjoy...

Tuesday, January 12, 2010

What is your favorite code metric?

You don't need to run any build or external tool to get this. It is always there when you code and changes actively as your code evolves. What would it be?

My most favorite is obviously "Line Numbers".

Line numbers are available in all IDEs but mostly not actively used as a code metric during coding. Switch them on and notice them actively when you code, they tend to have an interesting effect. Probably the classes and functions will be shorter, less bloated and concise. Based on personal experience :)

What is your simple metric to improve code?