Fork me on GitHub

Saturday, August 30, 2008

Announcing the release of rrd-rb

I am happy and thrilled to announce the release of my second open source project rrd-rb. rrd-rb is a simple round robin database binding written in ruby. I normally use it to visualize the response times of the websites I monitor using schnell.

Round Robin database is an open source industry standard for graphing solutions and can be used to store large amounts of time series of data in a minimal space. A good tutorial about the Round Robin databases is available in http://oss.oetiker.ch/rrdtool/tut/rrdtutorial.en.html.

Currently as the code is still under improvement I have not made any release. The source is available in SVN and can be downloaded. As well wiki has the installation instructions. The code comes with an example usage of the API. More examples are on the way.

Sample graph generated from random numbers -



Please let me know your view as well as comments. If you wish to contribute, please let me know. Till then happy coding.

Thank you Corey Goldberg for creating the rrdpy (Python binding) which gave me an idea to do this ruby binding.

Wednesday, August 27, 2008

Building a Google Reader API in Ruby


I was getting a little bored with the usual work and was trying to pep up my work by doing some hacking. I always wanted a API for Google Reader so that I can integrate with something or may be build a console app to read the posts :). So I started my journey into the mysterious land of APIs and service interfaces.

I found that there was no API published by Google for reader. So I thought may be I can put a prototype to read my subscriptions from the Google reader and print them. I used Mechanize for this and following is the code I wrote for this.

require "rubygems"
require "mechanize"



base_url = "http://www.google.com"
login_url ="https://www.google.com/accounts/Login?continue=http://www.google.com/&hl=en"
login_done_url = "https://www.google.com/accounts/CheckCookie?continue=http%3A%2F%2Fwww.google.com%2F&hl=en&chtml=LoginDoneHtml"
reader_subscriptions_url = "http://www.google.com/reader/settings?display=edit-subscriptions"



agent = WWW::Mechanize.new
base_page = agent.get base_url
login_page = agent.click(base_page.links.text(/Reader/))

login_form = login_page.forms.first
login_form.Email = "your google id"
login_form.Passwd = "your google password"

begin
agent.submit(login_form)
rescue WWW::Mechanize::ResponseCodeError
end

agent.get login_done_url

subscriptions_page = agent.get reader_subscriptions_url

subscription_names = []
feed_urls = []
names = subscriptions_page.search("//div[@class='subscription-title']")

names.each {|name| subscription_names << name.innerText}

urls = subscriptions_page.search("//input[@class='chkbox']")
urls.each{|feed| feed_urls << feed['value'].sub!('feed/','') unless feed['value'].grep(/feed/).empty?}

subscription_names.each_with_index{|feed_name,index| puts "#{feed_name} --- #{feed_urls[index]}"}


This code will show subscription names and the url of each subscription. I was planning to add the labels into it but it needs more work. As well there is some problem in the login, it logs in but it fails with a HTTP 501 exception and so the code is surrounded with exception handling (Need to figure that out). May be in future when I find time I will try to write this as a complete Google Reader API.

DISCLAIMER - THIS IS NOT OFFICIALLY ASSOCIATED WITH GOOGLE. THE CODE MAY NOT WORK IF THE GOOGLE READER INTERFACE CHANGES (which I seriously hope will not).

Tuesday, August 26, 2008

Do we need to test hidden fields?

I got into this moral dilemma when I was writing the schnell-driver’s hidden field (input type="hidden") tests. When I was writing the schnell with htmlunit I used something close to Watir API and so I wrote the hidden field tests especially had the hidden set values as a part of the test suite. But when writing the webdriver porting I suddenly got into the question of the need to test the hidden field set value tests.

In normal programming terms when writing unit tests we will not test the private members of a class because we will be exercising them from the public interface available. So there is no need to test the private members and if we feel we need to do then it means that we have some hidden complexity in the private method’s code which needs to be refactored. In the same lines the hidden field can be considered as private member or more appropriately private variable of the form. The value which goes into a hidden field may be directly filled in to create the query string or they may be manipulated by JavaScript before being used in to form post. If we need to manipulate the hidden fields we need to know all these implementation details and ask our testing code to do them, which is not a good idea.

This leads me to a question. Do we need to test or more preferably set values in hidden fields when we test an application? In my opinion we should not as hidden fields are private members of html language. It is better to their working through the form post or the JavaScript than us to deal with them directly and test them individually.

Sunday, August 10, 2008

schnell moved to jruby 1.1.3 and htmlunit 2.2

Update - Available on 13/8/2008 as I have some issue to fix :)

I have been busy for the past two weeks on a few things. So no major work was done. But I found that htmlunit 2.2 and jruby 1.1.3 were released. So I moved schnell to the new versions and ran the unit test... Everything ran fine (Tests are the source of my happy life :)). As well I have added the support for hidden elements. The new version of schnell 0.2.2 is available in the Google Code project.

As well I moved my laptop OS from Mandriva 2007 to OpenSUSE 11.0. Will write about it seperately... I may stick to it as I have everything I need except a few quirky issues (Seperate post 8-)).

Download the latest version of schnell and enjoy hacking...

Thursday, August 7, 2008

Ruby: A Simple Web Site Monitoring Using Schnell

Update - A small correction in the notify method. Please refer the code in the blog post for the change.

I have been working on schnell for quiet sometime and am moving towards another release. But the beginning of schnell is not as an automation testing tool. I started writing schnell as a wrapper to WWW::Mechanize to help my team to do the automated monitoring of websites. We had the requirement of monitoring a set of production websites to see their status every few minutes. But the condition was not to just see if the websites are accessible but to run a quick smoke test on them and see if they work.

A typical scenario would be to check if the website is available, log into it and see if it works. Here is the sample code of the site monitor we used to have. This has taken a mature version now which I am planning to release as open source.

You need to have mechanize, rubyful_soup, rufus_scheduler, log4r for this code to run. You can also do this with the latest jruby version of schell with a few changes in the notify method (Exceptions are handled differently in jruby version). The advantage of mechanize version is it can be run on native ruby. If you are running this code on latest version use jruby instead of ruby.

THIS IS ONLY A SAMPLE CODE.

require "../hui"
require "log4r"
require "rufus/scheduler"
include Log4r

class TestSite1
URL = "http://localhost:9999"
def self.monitor
browser = Browser.new
browser.goto URL
browser.link(:text, "buttons1.html").click
browser.button(:value,"Click Me").click
raise "Not able to get PASS" unless browser.text.include?("PASS")
end
end

class TestSite2
URL = "http://localhost:8888"
def self.monitor
browser = Browser.new
browser.goto URL
browser.link(:text, "links1.html").click
browser.link(:text,"test1").click
raise "Not able to get Links2-Pass" unless browser.text.include?("Links2-Pass")
end
end

$sitelog = Logger.new 'sitelog'
$sitelog.outputters = Outputter.stdout

scheduler = Rufus::Scheduler.new

trap("INT") do
$sitelog.info "stopping SiteMon..."
scheduler.stop
end

scheduler.start
monitered_websites = [TestSite1,TestSite2]

def notify(website,message)
if message.include? "Unable to navigate"
$sitelog.error "#{website.const_get('URL')} #{message}"
else
$sitelog.warn "#{website.const_get('URL')} #{message}"
end
end

$sitelog.info "starting SiteMon..."

scheduler.schedule_every "60s", :first_in => "5s" do
monitered_websites.each do |site|
begin
site.monitor
rescue => ex
notify site, ex.message
end
end
end

scheduler.join


There are a few things I am exploring now. Parallelism in monitoring, notification through Microsoft Office Communicator (We use mails as of now.) and a better modular structure.

I have published this code as well as the mechanize version of schnell (Used to call it hui) in GPL license as a download here. It is a very basic version of schnell and not all conditions and exceptions are covered. If you want to have better functionality, you can use the latest version with schnell in jruby. Please let me know if you have any suggestions and queries.

Download