Fork me on GitHub

Monday, April 21, 2008

Meta programming and documentation

I have been working on ruby for sometime and one of the things which attracted me to ruby is its meta programming capability. I work in DSLs a lot in Ruby and am currently working on a fluent interface for WATIR. Naturally meta programming comes into the picture in a lot of places in this implementation.

There are two problems I faced during the usage of meta programming. One is it hinders the way of using auto document generation like rdoc, the other is without proper documentation it is difficult to understand what you have written after sometime.

The first problem is more with the end user documentation. When I code an API I normally have comments in the methods and classes as here documents and at a later stage use rdoc to generate the documents. But when I use meta programming the code I am writing becomes concise but looses its meaning.

For example in one of the projects I wrote a geocoding API wrapper for yahoo geocoding service. I used a class called location for which I can set street, city, zip code or address as its attributes. Initially I wrote it as individual accessors but I found it to be too verbose and repetitive so I used method_missing to accomplish this.

class Location
def method_missing(method_name, args)
attr_name = method_name.to_s.chomp("=").to_sym
if %w[street city state zip address].include? attr_name.to_s then
@messages << attr_name.to_s
eval %(
class << self
attr_accessor :#{attr_name}
send(method_name, *args) if respond_to?(method_name)
raise "No method of name #{attr_name} error"

The job is accomplished easily but its meaning is a bit lost. I can still give the comment there and the rdoc will pick it and will use it. But still it will not be as effective as giving the normal documentation. The only solution which is ineffective that I can think of is to use dummy methods which mock the actual methods with comments during the document generation. I am not able to think of a better way.

The second problem is more on the developer's perspective. When using meta programming like this most of us feel happy that the work is done with lesser number of lines. But unfortunately the code's expressiveness is lost. So when you read the code after sometime it is difficult to understand why you used it actually. I normally follow TDD and even with the tests expressing my intent I find it difficult to understand the meta programmed code normally after a few days of coding it.

One of the important things when using meta programming is to write the code comment conveying your intent along with the code you are writing. This will go a long way in helping you at a later stage when you want to change something.

If you have anything related to this or want to discuss more in this regard please drop me a mail or add it to the comments.


Dave said...

Probably not the answer you're looking for, but maybe missing_method isn't the best solution? You're not trying to handle a missing method, you're providing several methods with the same basic body.

In Python I would use a utility function that would create the method body (with documentation attached) and then call that within the class definition. You could also use a class factory or a meta-class.

Your class will have the correct methods when run, so live introspection (help or dir in Python) will find the methods. Source scanners won't work, though. (I've never used rdoc, so I can't say how that would work.)

Mikoangelo said...

May I ask why you're doing it in such a convoluted way? As an example, I can perhaps understand it, but this is code you have actually used since you state it as a solution to a problem.

Why aren't you generating the accessor methods until they're used the first time? Since you have hard-coded the attributes you're making faux-accessors (in lack of a better term) for, wouldn't it have been easier to just make the accessors in the first place?

class Location
attr_accessor :street, :city, :state, :zip, :address

Or even if you have an affinity for %w[]:

class Location
attr_accessor *%w[street city state zip address].map { |a| a.to_sym} # .map(&:to_sym) with Symbol#to_proc

Unless, of course, the otherwise unmentioned `@messages << attr_name.to_s` serves a necessary purpose, in which case I'd opt for either `%w[...].each do |a| define_method(...) end` or a home-rolled messaging_attr_accessor.

On a related note: Why are you breaking NoMethodError raising by raising a String instance on an invalid method name instead? I'd choose to just write `super` instead on line 12.

Sai said...

Dave and Miko,
The example may not be entirely good but the point was in Meta Programming the code is semantically correct but syntactically different from the normal code. I wanted to write the post about its effects on documentation. There may be better ways to write this code :).

Dave said...


I guess I agree with the premise that it's different, but not that it's bad. Well, as long as the language facilities are adequate and your tools work well with it.

So I guess if this case was refactored (which was no insult to the original code, btw) it would be different, but not incomprehensible. Similarly OO and functional are different, but once you learn the idioms they're both readable.

I've been bitten in the past by writing meta-programming code that didn't play right with language tools. Basically I wrote your example in Python and then the built-in introspection (dir, help) didn't work, which was annoying.

Language and tool support for meta programming also helps, which is why no one touts C macros as being wonderful meta-programming. I don't know lisp, but I believe it has macro expanders so you can debug your macros. Python's decorators and meta-classes really help with meta-programming.

So I guess I'll extend your argument to say "if languages want to be friendly to meta programming they should provide tools to make it as semantically similar as possible :)