We won the Rally on Rails!
Aug
28

Last Thursday it was announced that we were the winners of the Rally on Rails ’11! We won with TimeHub, an utility that lets you generate invoices from your GitHub repositories.

You can check the scoreboard out at http://ranking.timehub.net (this is a little Sinatra app we coded after the competition, which is open-source, by the way).

I want to thank the organizers, sponsors and judges for this awesome contest! There were a few problems, sure, but overall it was an awesome contest. There were way more good things than bad things.

Predictions

These are the predictions about the winners I had made last week:

1. TimeHub.net by Funky Town (Definitely the most useful app in the competition. Of course, I’m biased.)
2. SecretLiker.com by Yotoco. (Very fun to use, specially when you sign in with Pemberthy’s account and start saying your male friends are sexy without Pemberthy actually knowing.)
3. GemTalk by 25cm+ (Very pretty design.)

I was only right in the most important one: us! SecretLiker.com ended up being 6th. I think they just needed to spam for more votes, because their idea and execution was way better than that of the apps that ended 2nd and 3rd. GemTalk ended up being 17th. It looks like they didn’t even tell their parents to vote for them.

I had also predicted Dumuzzi would be last. I nearly nailed this one; they ended second-to-last.

Things that need to be improved for next year

Although the contest ran relatively smoothly, there are a few suggestions I have for next year.

Decide a ranking scheme before the contest starts, and stick with it

The way teams were ranked was decided the same day the voting phase ended. Furthermore, it was changed several times in the lapse of just a few hours.

This shouldn’t have been so. Next year, decide how you’ll rank teams before the contest starts and don’t modify that during the contest.

There was a lot of discussion about whether teams should be ranked by their average score or not. Some people said this was unfair, supposedly because you can vote 1 star on the teams you don’t like and make their average go down. In my opinion, this is completely fair and acceptable. What’s the difference between voting 5 stars for a team you like to make its average go up and voting 1 star for a team you don’t like to make its average go down? There’s absolutely no difference; both approaches are equally biased.

To avoid this kind of pointless discussions, I suggest to ditch the average thing altogether. Make it so that only judges can vote with a number of stars and make the general public vote either 1 point or no points at all. Like presidential elections.

Also, I like they way you gave more weight to judges’ votes when calculating the final score. This is definitely important to avoid lame apps being on top just because they have an army of spam bots asking for votes.

No cheating

Gotta admit it, this episode was embarrassing. Make sure the app is cheat-proof. Ask for help if you need it, I’m sure that a lot of people (including myself) would have volunteered to help find this kind of vulnerabilities.

Add your own scoreboard

We had to scrape the Rally on Rails site because it didn’t have a scoreboard. Not nice! Implement an official scoreboard. An scoreboard is indispensable; it’s very fun to follow how everyone’s going.

Implement a public API

A public API would make a lot of tasks easier. For example:

  • Create custom scoreboard without having to scrape the website
  • Create graphs of votes and analyze them so we can answer questions like “at what time of the day are our spamming sessions on Twitter really working?”
  • Detect cheating easier.

Fix the user experience when voting

Voting for a team was absurdly difficult. Really difficult! Here’s what you had to do:

1. Go to the team page.
2. Click one of the authentication buttons on the top right corner.
3. After authorizing the app, you came back to the home page (this was later fixed and you ended back at the team page).
4. Scroll down until you see the stars.
5. Click the stars.
6. Wonder if your vote was counted or not.

Here are some of the questions I had to answer when I was asking my friends to vote for me:

“I already clicked on the stars, but nothing happened. Was my vote counted?”
“Why can’t I click on the first 4 rows of stars?”
“I clicked the Facebook button and it looks like I’m the home page. What’s your team’s name again?”
“After I clicked the stars I didn’t get a message or anything. Did I do something wrong?”

Here’s how the user experience should have been:

1. Your friend gives you a link to vote for his app. You click it.
2. You see a huge button that says “Vote for this team”. It’s obvious what you have to do.
3. You click the button. A little dialog asking you to identify yourself neatly slides down. Once again, it’s obvious what needs to be done next.

4. You click one of the authentication buttons that just appeared. You are redirected to the third-party, where you authorize to continue:

5. You come back to the app, and you see a green message saying that your vote has been counted. That’s it!

What do you think about that?

Thanks again

Thanks again for running this awesome contest! I know at the end it was harder than what it initially appeared to be. I’m looking forward to participate next year. I’ll be glad to help with anything you need about organizing it. Just ask!