A few days ago, my shared hosting provider ( Site5 – I do recommend them ) moved my account to a new server. Understandably, this new server had new versions of Ruby and Ruby on Rails. Sadly, my site uses Ozimodo, a (ancient) tumblelog application built for Rails 1.2.3, and I had been unknowingly depending on the global version of Rails. Naturally, my site went kerplumpf once it was moved to the new server.

After fighting with it for some time, I foolishly thought that I was able to just switch directly to Rails 2.3.8. This actually seemed to work for a moment (the main page displayed!) but everything else was still broken.

I attempted to vendor Rails (that is, have a copy of the right version of Rails inside my rails app in vendor/rails) but I kept getting errors like this:

$ rake rails:freeze:gems
(in /home/fair/pkgs/ozimodo)
rake aborted!
undefined method `manage_gems' for Gem:Module

After fussing about for a long while, trying to use different versions of RubyGems and all sorts of silly ideas, I went ahead and found the rails:freeze:gems task (it’s in framework.rake) and manually completed what it is supposed to do automatically:

1. Install Rails 1.2.3 if you have not already
2. Create vendor/rails
3. gem unpack actionmailer (1.3.3), actionpack (1.13.3), actionwebservice (1.2.3), activerecord (1.15.3), activesupport (1.4.2), and rails (1.2.3) into `vendor/rails`
4. Rename each directory without the version name (e.g., “actionmailer-1.3.3” -> “actionmailer”) except rails.
5. Rename rails-1.2.3 to railties

You may also need to set GEM_PATH to /home/yourname/ruby/gems, I can’t remember if that was necessary or not.

This was the process, as well as I can remember, which got this site back up and running again.

I have been trying to do a bit more advertising of Brakeman lately. I have two reasons for doing so: firstly, I think it’s a really useful tool and will make the world a safer place. Secondly, I keep hoping someone else will find bugs in it for me.

One of my design decisions for Brakeman is that it should never crash and should always output some kind of report. I don’t always meet this goal, so I am very interested in any cases where a Rails app will actually make Brakeman fall on its face.

Lately, I have (finally) been putting together a test suite. I’m already finding issues with features I thought were working. Hopefully it will help me avoid embarrassing regressions in the future, too.

If you are in the Los Angeles area, I will be presenting on using Brakeman with Jenkins at the LA OWASP on Wednesday the 25th. Then, next month, I have a similar but unconfirmed talk at LA Ruby. The difference between the two is that I will talk more about Ruby and Rails at the OWASP meeting. At least, I think I will. This blog post is part of my avoiding actually making the slides.

Oh, what is Brakeman? It is a static analysis tool for finding security vulnerabilities in Ruby on Rails applications. It does a pretty good job, most of the time.

Brakeman is a static analysis tool for Ruby on Rails that looks for security vulnerabilities. The cool thing about Brakeman is that pretty much all you need is your code: there is no need to set up a database or a web server or anything of that sort. The code doesn’t even need to be working completely.

Brakeman has been working pretty well for Rails 2.x code, but no one really cares about that any more. Rails 3.x is the new hotness. So I have been working on getting Brakeman to work with Rails 3. The latest gems will attempt to detect Rails 3 automatically (otherwise, there is the `-3` options). There are some known issues to be fixed, but I think it’s past the “catastrophic crash and burn” stage. I would encourage people to try it out.

Also, I have released a Jenkins/Hudson plugin for Brakeman With a minimal amount of setup, you can have Jenkins run Brakeman automatically. It uses another nice plugin to make pretty graphs and tracks the vulnerabilities found so you can see right away when things go wrong.

There are a few examples around the ‘net of chat servers in Ruby, which seem to me to be unnecessarily complex or else too simplistic.

This is a simple version, too, as it is intended to meet the requirements over at RosettaCode. However, I think I’ve covered most of the common failure points.

Caution

If connecting with a client like PuTTY, use “raw” mode to avoid telnet negotiation.

I spent this summer doing an internship at ATTi, during which I developed a static analysis tool called brakeman for finding security vulnerabilities in Ruby on Rails applications.

What it is

Brakeman uses Ryan Davis’ Ruby Parser to parse the code of your RoR application, mangles it a bit, extracts some information, and then runs various checks on the result. It then uses Ruport to generate a report.

The HTML reports look like this.

Because brakeman analyzes the source code, there is no need to wait until the application is deployed to start testing it. Brakeman can be run at any point in the development process.

What it can do

Right now, brakeman can find these kinds of problems:

• Cross site scripting vulnerabilities
• SQL injection
• Command injection
• Unsafe redirects
• Unrestricted mass assignments
• Insufficient validation regexes
• Default routes
• Dynamic render paths

It can also check configuration settings, such as cross site request forgery protection and session secret length.

Unfortunately, it is not (yet?) compatible with Rails 3.0. Hopefully it will still be of use to a lot of people, though.

Installation

Brakeman can be installed as a gem (and, in fact, that is how I would recommend doing it):

gem install brakeman

(may require sudo).

Documentation

brakeman -h provides information on the options available. I’ve also been working on fleshing out the wiki with more detailed info.

Problems?

I really want this to be a useful tool, so if it does not work for you or there are any problems, please file an issue or even just leave a comment on this post. I’ll do my best to get everything fixed up.

As a result of a Ruby bounty, I wrote a patch for RubyGems which provides suggestions when you try to install a gem and you get the name a little wrong. You can see all the details here.

Now that RubyGems 1.3.7 is out, I’m going to go ahead and provide a patched version for those who frequently forget, mistype, or misspell gem names (this includes myself).

Download .tar.gz

Download .zip

After decompressing, I recommend installing it with

ruby setup.rb --no-rdoc --no-ri

(and probably sudo).

Here are some examples of what this adds to your RubyGems experience:

$ gem install Blah
ERROR:  Could not find a valid gem 'Blah' (>= 0) in any repository
        Possible alternatives: blahblahblah

$ gem install Nkojiri
ERROR:  Could not find a valid gem 'Nkojiri' (>= 0) in any repository
        Possible alternatives: nokogiri

$ gem install blue
ERROR:  Could not find a valid gem 'blue' (>= 0) in any repository
        Possible alternatives: bluecloth, BlueCloth, bluefeather,
         blue_light_special, blue_light_special_heroku_fork, glue, bluepill,
         blueprintr, blueprints, bluepay, ...

$ gem install sinatar-capcha
ERROR:  Could not find a valid gem 'sinatar-capcha' (>= 0) in any repository
        Possible alternatives: sinatra-captcha, sinatra-cache, sinatra-cas, sinatra-auth,
         sinatra, sinatra-any, sinatra_app_gen, sinatra-flash, sinatra-dm, sinatra-dm-auth,
         sinatra-doc, sinatra-erb, sinatra-compass, ...

Recently, the main Ruby Gems hosting has moved from RubyForge to Gemcutter. This is pretty cool, and I was excited to publish my first gem there (the JRuby gdbm library I’ve mentioned before).

I get excited imagining that people actual use my stuff, so I’ve been tracking the downloads reported by Gemcutter. I seem to recall there being a problem a little while ago with numbers being off, but I thought that had been solved. In any case, I was surprised by the number of downloads (up to 137 now). It’s not very high for most things, I guess, but for a library for a specific dbm for a specific Ruby implementation, it seems suspicious.

My current theory is that people are installing it on accident, perhaps thinking that they need it to use gdbm in the main Ruby implementation (or perhaps others?). Fortunately, I’ve checked it out, and it appears that even if you install it this way, it will not interfere with the gdbm library in Ruby’s standard library. This is likely because the gem paths are later in the search path. Of course, if Ruby’s standard gdbm is not working for you, then my gem probably won’t either, unless you are using it with JRuby, as intended.

In retrospect, perhaps I should have named the gem a little bit better, to avoid confusion…

Here’s some links to interesting Ruby information. I intend to update it periodically.

Ruby Doc – Best site for looking up the core Ruby API and the standard libraries.

AllGems – Documentation for most Ruby gems.

Ruby Forum – Mirrors of the main Ruby mailing lists. Some are read-only.

Gemcutter – The new gem hosting site.

RubyForge – The old Ruby project hosting site. It is slowly being phased out.

Try Ruby – The new Try Ruby page, where you can go through an interactive tutorial directly in your browser.

Ruby Inside – Popular Ruby blog, with frequent articles.

Ruby Flow – Community-submitted Ruby links.

Ruby Trends – View and vote for trends in Ruby software, books, and more.

Ruby Pulse – Screencasts of Ruby software and libraries.

Ruby Subreddit – Reddit site for Ruby. Not incredibly active.

Ruby Stack Overflow – Ruby questions on Stack Overlow.

Confreaks – Not strictly Ruby, but videos from many Ruby events are available for streaming.

I’ve been working on my little language a bit more lately, instead of doing work I really should be doing.

Now that Brat is working pretty well, I thought I would make a release. It’s basically just a snapshot of the current Subversion repository, but I know people would rather download a tar file than use SVN to check something out. It is only for Linux at the moment, but maybe I will get it working on Windows sometime. It should work with Ruby 1.8.6, 1.8.7, and 1.9.

Alright, on to language stuff. I recently changed the scoping rules to be more what a person might expect: any scope can access its outer scopes (rather than before, which used the Neko default of copying the outer scope). This makes the toplevel variables something like globals, except not quite.

a = { x }
x = 1
p a   #Error - x was not defined when a was
b = { x }
p b   #Prints '1'
c = { x = x + 1 }
c
p b   #Prints '2'

And recursive functions no longer need to be attached to an object:

rec = { x | false? x < 1, { p x; rec(x - 1) } }
rec 10

Bad news is that my current approach does not do a good job of releasing memory. But that is a problem for another day.

Of course, many bugs large and small have been fixed along the way. I hope to at least get this language to the point where I use it regularly. I think it is an interesting and probably odd mix of object-oriented and functional programming.

Also, I set up a discussion group in case Brat generates any attention.

I’ve released a new version of kams (0.2.0).

Today is the first day of the new quarter, but I don’t have class and I’m sick. So I’m staying at home. Yay.

Okay, in my last post I was talking about some changes I was making to kams to make it work with Ruby 1.9.1.

Today I happened to glance at the memory usage. This is just after starting up the server and I have about ~830 game objects in the world (I expect that to be a small number, given that I only have 152 rooms, 325 exits, and the rest are random things like clothing). Under Ruby 1.8.7, this takes ~36MB of memory. Under 1.9.1RC1, it’s only 16MB. Less than half.

Nice. Now to look for areas where memory may be leaking…

Here are a few things I have found so far while making kams compatible with Ruby 1.9.1:

• Small eventmachine change needed as shown here.
• require 'md5' changed to require 'digest/md5' and MD5 changed to Digest::MD5 (this works with 1.8.7, too.)
• Any hashes which were defined not using => (i.e., {'a', '1', 'b', '2'} ) have to now use =>
• One issue required some version checking: the Observable module in 1.8.7 uses an array, in 1.9.1 it uses a hash. This doesn’t matter much to me, except when Observable objects are marshalled and this goes along with them. What I have done for now is set it to an array or hash as appropriate when loading the objects (there is no content for it, in either case).
• Array#to_s has changed. In 1.8.7, [].to_s => ””. In 1.9.1, [].to_s => “[]”. Somewhat annoying, but easy to handle.
• Have to deal with Object#instance_variables and Object#instance_methods returning arrays of symbols instead of strings.
• Ran into an issue with Mutex#try_lock. Asking about a solution now…

Update – No one answered my question about Mutex#try_lock, but someone fixed it anyway :)

Rublar is a very neat little website for testing out Ruby regular expressions.

I typically use Liferea for keeping track of RSS feeds (works great at school so I don’t waste as much time browsing websites) but, for whatever reason it isn’t working right for me on my laptop. “So,” I thought, “why not create my own little dealie using some Ruby and some pairs of Shoes? I mean, I’m sure it will be straightforward, right?”

Ha. Ha.

First of all, I went looking for some Ruby RSS libraries. There are a few out there. I tried simple-rss, because it seemed nice and…simple. Installed just fine, ran it using this site’s feed. It crashed. Didn’t work at all. Lame. Looked a few others, but they seemed to have a few too many dependencies or were out of date.

But, wait! Ruby comes with it’s very own RSS parser (and generator) in the standard library! Great! Too bad the documentation is lacking and confusing at best. I managed to find a nice little article over at rubybook.ca (haven’t really investigated the site much) which has some details about parsing RSS and Atom with the standard library. Awesome.

Then I got sidetracked by something else. I wanted to be hip and not disturb the webservers too much, so naturally I wanted to use conditional GETs so that if a feed hasn’t been updated I don’t download the whole thing again. Seems like a great idea. Then I got seriously confused. Why? Because this ‘Last-Modified’ header is not something that is sent back all the time. I wasted a bunch of time trying to figure out if there is a way to request the information, but it doesn’t sound like it. If it’s there, great, if not, either make up your own timestamps or forget about it.

I managed to get back on track and even wrote up a nice little class for handling subscriptions when I found out the really bad part: My own website’s feed didn’t provide the data Ruby’s RSS parser expected. Great. I looked at a few others. It seems like RSS feeds are generally okay, but Atom feeds are all over the place. After fiddling around for a while, I made my code a little flexible and moved on.

I wrote a hideous shoes application.

Then I gave up, because I was tired of dealing with it. Maybe I’ll come back to it later.

However, I did change this site’s feed over to an RSS feed instead of Atom. If you were actually using the Atom feed, you can still do so, but the auto-discovery and link at the bottom are set to RSS.

Yeah, pretty much as the title says. I only think to do this when I am trying to think of anything possible I can do to avoid doing work.

So, in lieu of anything actually insightful to share, have a look at this, a little ways down. Note how the new 1.9.1 version of Ruby is blowing everything else away. Pretty awesome. I’m excited for the real release and for certain things (cougheventmachine cough) to have official support for it. Things are looking up in Rubyland.