A few days ago, my shared hosting provider ( Site5 – I do recommend them ) moved my account to a new server. Understandably, this new server had new versions of Ruby and Ruby on Rails. Sadly, my site uses Ozimodo, a (ancient) tumblelog application built for Rails 1.2.3, and I had been unknowingly depending on the global version of Rails. Naturally, my site went kerplumpf once it was moved to the new server.

After fighting with it for some time, I foolishly thought that I was able to just switch directly to Rails 2.3.8. This actually seemed to work for a moment (the main page displayed!) but everything else was still broken.

I attempted to vendor Rails (that is, have a copy of the right version of Rails inside my rails app in vendor/rails) but I kept getting errors like this:

$ rake rails:freeze:gems
(in /home/fair/pkgs/ozimodo)
rake aborted!
undefined method `manage_gems' for Gem:Module

After fussing about for a long while, trying to use different versions of RubyGems and all sorts of silly ideas, I went ahead and found the rails:freeze:gems task (it’s in framework.rake) and manually completed what it is supposed to do automatically:

1. Install Rails 1.2.3 if you have not already
2. Create vendor/rails
3. gem unpack actionmailer (1.3.3), actionpack (1.13.3), actionwebservice (1.2.3), activerecord (1.15.3), activesupport (1.4.2), and rails (1.2.3) into `vendor/rails`
4. Rename each directory without the version name (e.g., “actionmailer-1.3.3” -> “actionmailer”) except rails.
5. Rename rails-1.2.3 to railties

You may also need to set GEM_PATH to /home/yourname/ruby/gems, I can’t remember if that was necessary or not.

This was the process, as well as I can remember, which got this site back up and running again.

I have been trying to do a bit more advertising of Brakeman lately. I have two reasons for doing so: firstly, I think it’s a really useful tool and will make the world a safer place. Secondly, I keep hoping someone else will find bugs in it for me.

One of my design decisions for Brakeman is that it should never crash and should always output some kind of report. I don’t always meet this goal, so I am very interested in any cases where a Rails app will actually make Brakeman fall on its face.

Lately, I have (finally) been putting together a test suite. I’m already finding issues with features I thought were working. Hopefully it will help me avoid embarrassing regressions in the future, too.

If you are in the Los Angeles area, I will be presenting on using Brakeman with Jenkins at the LA OWASP on Wednesday the 25th. Then, next month, I have a similar but unconfirmed talk at LA Ruby. The difference between the two is that I will talk more about Ruby and Rails at the OWASP meeting. At least, I think I will. This blog post is part of my avoiding actually making the slides.

Oh, what is Brakeman? It is a static analysis tool for finding security vulnerabilities in Ruby on Rails applications. It does a pretty good job, most of the time.

Brakeman is a static analysis tool for Ruby on Rails that looks for security vulnerabilities. The cool thing about Brakeman is that pretty much all you need is your code: there is no need to set up a database or a web server or anything of that sort. The code doesn’t even need to be working completely.

Brakeman has been working pretty well for Rails 2.x code, but no one really cares about that any more. Rails 3.x is the new hotness. So I have been working on getting Brakeman to work with Rails 3. The latest gems will attempt to detect Rails 3 automatically (otherwise, there is the `-3` options). There are some known issues to be fixed, but I think it’s past the “catastrophic crash and burn” stage. I would encourage people to try it out.

Also, I have released a Jenkins/Hudson plugin for Brakeman With a minimal amount of setup, you can have Jenkins run Brakeman automatically. It uses another nice plugin to make pretty graphs and tracks the vulnerabilities found so you can see right away when things go wrong.

I have started to feel guilty about neglecting this site, so here is a recap of my life right now:

School drags on, but if all goes well I will at least advance to candidacy at the end of this week. I’m pretty nervous about it, though.

Work is weird because I work at AT&T Interactive but I work for AT&T, yet I do work for AT&T Interactive. Even more confusingly, the sign outside the building still says YellowPages.com.

My hobbies now include archery, which is thoroughly addictive, so you will find me on the range most Saturdays and Sundays and even on some weekday evenings.

In my more personal life, I will be getting married in approximately five months. I’m pretty happy about that.

And that’s a pretty good summary of where I am right now. A good portion of my activities can been seen on GitHub if you like more frequent updates.

From Tom Sawyer:

...Work consists of whatever a body is obliged to do, and that Play consists of whatever a body is not obliged to do.

It is common in the open source world to hear that someone wrote some software to “scratch their own itch.” The implication is usually that they wrote the code to meet some need (or annoyance?) they had, and it may, by mere coincidence, be useful to others as well. Sometimes this phrase also used to excuse the software or lack of support for it. It may also be used to provide guidance for those wanting to write software: do something useful for yourself.

I think there is a considerable amount of wisdom in this approach. Nothing motivates a person like a genuine interest in a project. The best learning (in my non-scientific opinion) occurs when a person is driven by their own interest. Outside pressures can only do so much.

I often see Project Euler or some kind of “programming koans” offered up as suggestions to improve one’s programming skills. I find these to be unhelpful for me, as I cannot make myself be interested in them. In fact, I should not need to force myself to be interested at all: that pushes the activity into the “work” category and the exercise has already failed.

For programmers, we should be writing code which we want to write, solving problems we want to solve, and producing something we want to make.

Along the way, we may accidentally learn all kinds of new things, but only because those things are on the path of building our project. It is not about making something useful, or good, or desirable. It is about the joy of creation.

That does not mean it is all bunnies and sunshine, though. I have hit my head against the wall more times over software I was writing for fun than any homework problem. Why? Because they were problems I intensely desired to overcome – for myself.

The point, in case I have not beaten it to death already, is that you should write code for you. It doesn’t matter if it isn’t perfect or won’t compile on anyone else’s machine. It doesn’t matter if twenty other programs exist that do the same thing. All that matters is that you write it. Learning is merely a side effect.

There are a few examples around the ‘net of chat servers in Ruby, which seem to me to be unnecessarily complex or else too simplistic.

This is a simple version, too, as it is intended to meet the requirements over at RosettaCode. However, I think I’ve covered most of the common failure points.

Caution

If connecting with a client like PuTTY, use “raw” mode to avoid telnet negotiation.

I spent this summer doing an internship at ATTi, during which I developed a static analysis tool called brakeman for finding security vulnerabilities in Ruby on Rails applications.

What it is

Brakeman uses Ryan Davis’ Ruby Parser to parse the code of your RoR application, mangles it a bit, extracts some information, and then runs various checks on the result. It then uses Ruport to generate a report.

The HTML reports look like this.

Because brakeman analyzes the source code, there is no need to wait until the application is deployed to start testing it. Brakeman can be run at any point in the development process.

What it can do

Right now, brakeman can find these kinds of problems:

• Cross site scripting vulnerabilities
• SQL injection
• Command injection
• Unsafe redirects
• Unrestricted mass assignments
• Insufficient validation regexes
• Default routes
• Dynamic render paths

It can also check configuration settings, such as cross site request forgery protection and session secret length.

Unfortunately, it is not (yet?) compatible with Rails 3.0. Hopefully it will still be of use to a lot of people, though.

Installation

Brakeman can be installed as a gem (and, in fact, that is how I would recommend doing it):

gem install brakeman

(may require sudo).

Documentation

brakeman -h provides information on the options available. I’ve also been working on fleshing out the wiki with more detailed info.

Problems?

I really want this to be a useful tool, so if it does not work for you or there are any problems, please file an issue or even just leave a comment on this post. I’ll do my best to get everything fixed up.

As a result of a Ruby bounty, I wrote a patch for RubyGems which provides suggestions when you try to install a gem and you get the name a little wrong. You can see all the details here.

Now that RubyGems 1.3.7 is out, I’m going to go ahead and provide a patched version for those who frequently forget, mistype, or misspell gem names (this includes myself).

Download .tar.gz

Download .zip

After decompressing, I recommend installing it with

ruby setup.rb --no-rdoc --no-ri

(and probably sudo).

Here are some examples of what this adds to your RubyGems experience:

$ gem install Blah
ERROR:  Could not find a valid gem 'Blah' (>= 0) in any repository
        Possible alternatives: blahblahblah

$ gem install Nkojiri
ERROR:  Could not find a valid gem 'Nkojiri' (>= 0) in any repository
        Possible alternatives: nokogiri

$ gem install blue
ERROR:  Could not find a valid gem 'blue' (>= 0) in any repository
        Possible alternatives: bluecloth, BlueCloth, bluefeather,
         blue_light_special, blue_light_special_heroku_fork, glue, bluepill,
         blueprintr, blueprints, bluepay, ...

$ gem install sinatar-capcha
ERROR:  Could not find a valid gem 'sinatar-capcha' (>= 0) in any repository
        Possible alternatives: sinatra-captcha, sinatra-cache, sinatra-cas, sinatra-auth,
         sinatra, sinatra-any, sinatra_app_gen, sinatra-flash, sinatra-dm, sinatra-dm-auth,
         sinatra-doc, sinatra-erb, sinatra-compass, ...

I’m putting that first for people who don’t need a story.

To be honest, I cannot believe it took me so long to find this out. It seems like such a simple thing. You have a git repo. You’ve made changes to a few different files, when you realize you just want to start over on one of them. You could do git reset --hard, but then you lose all of your changes. You could commit all the other changed files, then do a git reset, but what if you aren’t done with editing the other files?

In Subversion you can svn revert a single file. Since that is what I fairly frequently wish to do, I searched high and low for a way to do this seemingly simple operation.

Then, one day glorious day, I found it. And now so have you.

Yum.

Since it is almost the last day of the year, I suppose it is time to take a look back and see what has happened.

My research has progressed, but not very quickly. I missed several conference deadlines, and I’m still trying to get a paper done. However, I’m quite close!

My advisor retired in February. Not much changed until recently, when he announced he was not sure about funding those of us left in the lab next year. This spurred some panic, but we have time to figure it out. I’m thinking of taking an adjunct teaching position somewhere.

Inspired by _why’s potion language, I started working on my own little programming language called Brat. It’s been my best project, keeping me busy pretty much all year. I’m always running into or thinking of new things to do with it. Writing your own language forces you to learn a lot of things. Not just about parsers and compilers, but nearly everything, since any libraries will have to be written (or at least wrapped up) by yourself.

My parents came and visited around my birthday. That was pretty fun, although I now have a better list of things to do if they come visit again.

I was considerably absent last school year. Accordingly, this fall I made a resolution to be more involved, especially with the CSGSC. So far, I’ve been successful with this.

I also had an interesting programming experience wrapping up gdbm using FFI, making it usable from JRuby and other Ruby VMs. It was my first time using FFI, writing a library for JRuby, and packaging up something to be used as a gem. Quite a learning experience, but one I really enjoyed. Hopefully, it will eventually get included in JRuby itself.

I started a couple little websites: the Fledgling Languages List and a Neko Tutorial.

I picked up a T-Mobile myTouch, thinking I would get into Android programming. Hasn’t happened yet, but I’m still hoping to get going on it. I understand it’s pretty straightforward once you get the dev environment all set up, I just haven’t gotten past that part yet.

I lived through Michael Jackson’s death and Los Angeles’ frenzy over it. He was taken to the UCLA hospital, so I got to see some of it first hand.

The summer roadtrip I had hoped to take didn’t work out, although my girlfriend and I did get to go to Ohio for a little family reunion. We just flew instead of driving.

I was able to see Zao play a concert in a tiny little place in the middle of nowhere.

Despite having completed the WQE two years ago, I just now filled out the paperwork to get my Masters’s. Now it only takes 3-4 months to get my diploma.

Now I just have to get in gear for the new year!

Recently, the main Ruby Gems hosting has moved from RubyForge to Gemcutter. This is pretty cool, and I was excited to publish my first gem there (the JRuby gdbm library I’ve mentioned before).

I get excited imagining that people actual use my stuff, so I’ve been tracking the downloads reported by Gemcutter. I seem to recall there being a problem a little while ago with numbers being off, but I thought that had been solved. In any case, I was surprised by the number of downloads (up to 137 now). It’s not very high for most things, I guess, but for a library for a specific dbm for a specific Ruby implementation, it seems suspicious.

My current theory is that people are installing it on accident, perhaps thinking that they need it to use gdbm in the main Ruby implementation (or perhaps others?). Fortunately, I’ve checked it out, and it appears that even if you install it this way, it will not interfere with the gdbm library in Ruby’s standard library. This is likely because the gem paths are later in the search path. Of course, if Ruby’s standard gdbm is not working for you, then my gem probably won’t either, unless you are using it with JRuby, as intended.

In retrospect, perhaps I should have named the gem a little bit better, to avoid confusion…

Here’s some links to interesting Ruby information. I intend to update it periodically.

Ruby Doc – Best site for looking up the core Ruby API and the standard libraries.

AllGems – Documentation for most Ruby gems.

Ruby Forum – Mirrors of the main Ruby mailing lists. Some are read-only.

Gemcutter – The new gem hosting site.

RubyForge – The old Ruby project hosting site. It is slowly being phased out.

Try Ruby – The new Try Ruby page, where you can go through an interactive tutorial directly in your browser.

Ruby Inside – Popular Ruby blog, with frequent articles.

Ruby Flow – Community-submitted Ruby links.

Ruby Trends – View and vote for trends in Ruby software, books, and more.

Ruby Pulse – Screencasts of Ruby software and libraries.

Ruby Subreddit – Reddit site for Ruby. Not incredibly active.

Ruby Stack Overflow – Ruby questions on Stack Overlow.

Confreaks – Not strictly Ruby, but videos from many Ruby events are available for streaming.

I am just so excited right now. I was watching Dave Thomas’ (not the Wendy’s guy) keynote talk from RubyConf 2008, in which he proposes several “forks” of Ruby. At about 40 minutes in, he discusses having a fork of Ruby that has real closures instead of blocks. I thought to myself, “Self, that sounds a lot like Brat.” Five seconds later, he mentioned the problem with passing in multiple closure literals to a function: the comma is ugly.

some_method { do_stuff }, { 1 + 2 }

I agree! I think this is an ugly issue in Brat right now.

But then he proposed an awesome solution: if two closures are next to each other in the argument list, you don’t need a comma! Now you can do this:

some_method { do_stuff } { 1 + 2 }

This took approximately 2 minutes to implement for Brat.

Now instead of

while { x < 1 },
    { 
        p x
        x = x + 1 
    }

You can do

while { x < 1 } { 
        p x
        x = x + 1 
    }

I think this is awesome and makes Brat a lot more attractive.

In fact, I went totally crazy so now you can do

x = 101
true? x > 100
      { p "> 100!" }
      { p "<= 100!" }

Just be careful when using bare variables:

true? a { p "truth!" } { p "lies!" }

This is parsed into:

true?(a({ p "truth!" }, { p "lies!" }))

This is so cool…now to update all of Brat’s docs with this syntax. :)

After nearly three years of using the T-Mobile Dash, I was finally seduced into getting a new phone. Let me say upfront that I am no Microsoft/Windows fan, but once I got a good ROM with the latest Windows Mobile, the phone was really good. There was really nothing wrong with my Dash, the only real issue was me getting a little bit tired of EDGE speeds. But Android sounded pretty appealing, as did the potential of writing my own apps (and possibly making a little bit of money from them (also, this ).

Packaging

I have no idea why, but they shipped the myTouch in a really nice hard case (never to be used again) with everything packed neatly in foam. Pretty fancy. It came with a two-piece charger (wall plugin + USB cable), earphones, earphone adapter (more on that later), a nice little bag for the phone, a screen protector, and a 4GB microSD card (installed).

Unfortunately, it was quite light on instructions, and I had to go online to figure out how to open the battery cover. (I wasn’t going to start randomly pushing and pulling things on a brand-new phone.) The battery was charged, as usual.

Updating Services

For some reason, I was expecting the T-Mobile website to detect my new phone. I thought it did that before. With it still thinking I had the Dash, the site (wisely?) did not offer the myTouch data plans. Once I manually set my phone, it came up and I signed up for the $25/month plan.

Using the Phone

The onscreen keyboard takes some getting used to. However, after several hours of messing with the phone, I realized it expects you to use your thumbs, not fingertips. I would rather use my finger, personally, but it responds much better to “fatter” touches. Scrolling also takes some practice. For one thing, I’m used to down being down and up being up, although I understand the abstraction of “throwing” the screen. It’s also a little tricky to scroll instead of clicking on things. I assume in time I’ll get the hang of it, though. Oh, and sweaty fingers don’t work at all.

One other thing about the fingers: I was holding the phone landscape-style between the forefinger and thumb on my right hand, and touching the screen with my right pointer finger (probably behavior from playing the DS.) The problem with that is the volume key then directly underneath my thumb. Switching to thumbs-mode (like playing a NES, I guess) “fixes” this.

Another thing I kept confusing was the “disconnect” (red phone) key locking the phone. I am used to that being the “close application” or “cancel” button. But I got over that pretty quickly.

Besides basic dialing/browsing, the initial setup on the phone makes it kind of difficult to find anything. Settings are buried quite deep and I’m not sure they are organized as best as they could be.

Strangely, I often feel lost when using applications on the phone. Most apps don’t have “Okay” or any kind of confirmation buttons, you just use the ”<-” key. It’s hard to tell where you are or what you are expected to do.

Features

I started missing some things immediately. First, being able to quickly call a contact. On the Dash, I had it down to two keypresses. Second, I’m used to being able to see unread email counts right upfront, even when the phone is locked.

For the first issue, I found that “AnyCut” (free from the Market) will allow you to put a single-press shortcut on the home screen to call a contact. Problem solved.

For the email issue, I did not really find a solution. If you are an expert at writing email clients, here’s your opportunity. Search for “android email client” and you will see huge numbers of complaints. I’m using “K-9 Mail” now, which is a fork of the default client, but it’s still not as convenient or snappy as the Windows Mobile client (which was nothing special). K-9 still uses the “notifications” mechanism to tell you about unread emails, which is workable, but I would really prefer some kind of widget deal.

To combat the buried settings, I installed an app called “Toggle Settings” and dropped it on the home screen. Works for me. That’s how I found out the GPS was turned off.

Applications

Anyhow, now for the stuff that worked. The “voice search” is pretty cool and works well, although I don’t see myself using it. Google Maps is as good as you would expect, and that thing they showed in the G1 commercials with the street view actually works, too. Google SkyView is awesome. Of course, I had to get it during one of the rare cloudy days here, so I couldn’t try it outside.

The Android Market is decent. I hear it’s better in 1.6, but my phone hasn’t updated yet. Going through popular apps is alright, and I like the way you can immediately see ratings and comments from people. If you are just browsing, you can sort by category and then popularity or date. Unfortunately, though, if you just do a search you cannot sort the results at all (unless I missed something).

“Locale” is an application I’ve wanted forever. You can make all kinds of phone settings change according to your calendar, dates/times, location, or who is calling. You can set up all kinds of different things, and it’s really useful. No more embarrassing phone sounds in the middle of silent rooms!

The YouTube app works. Since I mostly use YouTube to lookup/listen to specific songs, I can see this being useful. I assume the video quality is a function of both the device and the uploaded video, so I am reserving my opinion of that until I fiddle with it more.

I also installed “Power Widget,” which simply displays the percentage of your battery left. The battery icon at the top of the screen is definitely misleading (right now shows full for 86%).

The calendar it came with is decent. My only complaint is that, again, the Dash would show upcoming appointments on the home screen, which was pretty nice. Perhaps there is an app somewhere for that, I haven’t looked yet.

I tried using one of the “home replacement” apps, but I could tell it slowed everything down, so I got rid of it pretty quickly. Like one blog I read, I think using the native one is going to provide the best performance/power usage for now.

Leftovers

I promised to get back to the earphone adapter, and then I forgot. Basically, it’s really short (like 3 inches), putting the microphone close to the phone and far from your mouth. I tried out the earphone/microphone deal from the Dash and it worked just fine. On the other hand, the adapter lets you use whatever headphones you want.

Anyhow, that’s most of what I’ve explored so far on the phone. I’ve come to the conclusion that I would probably not pay $199 for the phone, because somehow that sounds like a lot more than $149. But at $149, it’s pretty slick, and I imagine it will continue to improve as the Market expands and Android becomes better.