Two weeks ago, Co.Labs editor Chris Dannen approached me with a problem: We were about to receive hundreds of entries for our Co.Labs/Target Retail Accelerator and we had no judging mechanism ready to go. Because of the volume of submissions and unique free-form entry format on GitHub, judging the contest using a spreadsheet or some other traditional method was going to be a time-consuming pain for judges and the people who would need to compile the entries–namely us. But we only had a short time to design, implement, and test a solution, and it would only make sense if the judging software took significantly less time to build than it saved.
We’ve written before about the pitfalls of looking for a technological solution to every problem, especially when you’re in lean startup mode. It occurred to me that many of the decisions I had to make applied equally to all kinds of businesses who need to decide whether to deploy humans or machines for small projects. Here’s what I learned.
Building a better solution would only make sense if the hours I spent building the project didn’t exceed the time the judges would save using my solution plus the time it would have taken to collect all the entries manually. In order to figure out how much time I had to build the app, I did a quick back-of-the-envelope calculation:
First, I looked at how much time it would take to compile a manual solution. Downloading the contest directory from GitHub is easy enough, but we would still need to manually compile a spreadsheet of each entry for the judges to keep track of their scores and progress. It seemed likely that we were going to get at least 100 entries, and I figured that each entry would take about 2 minutes to compile. This meant that we would spend at least three hours compiling entries.
Next, I thought about how I could save time for the judges. With a manual solution, the biggest waste of time for them would be moving between each entry’s folder, and the judging form would also take time. If you’re like me, you often get lost switching between apps. Call it three minutes per entry. If I could eliminate just this one piece of complexity, I could get rid of at least 4.5 hours of extra work per judge, or 18 hours in total.
Altogether, I technically had 21 hours to work with. But there was something else to consider, too: my time. If it takes me 15 hours to save 21 hours over the alternative, is it really worth those six total hours of savings? It would make more sense from an opportunity cost perspective if I could spend around the same time building the app that it would take to solve the problem manually. That way, all those savings would come without opportunity cost.
When Chris first approached me with the project, my first thought was to use the GitHub API to pull entry files into a web app and display them nicely alongside a judging form that would submit to Wufoo or Google Forms. This would be an ideal solution, because every action would happen on one screen and we could easily keep track of each judge’s progress in an internal database.
It took about two minutes to realize that building an app with that much complexity was going to take way more time than I had. My next thought was to instead piggyback on the GitHub web interface by embedding our repo an iFrame on a page that also included a judging form. This idea eliminated the need to use the GitHub API, but it still required building a backend to keep track of judge progress, which continued to feel like too heavy of a solution for such a simple task.
Deciding not to use a database to keep track of votes introduced another common problem for anyone who’s ever used Microsoft Excel: Storing variables as strings. To avoid judges accidentally editing entry names and breaking our ability to count votes, I used regex to isolate the top-level directory name of each entry and automatically fill out the name portion of the judging form.
This extension will definitely operate on the “security through obscurity” principle, because we’ll be distributing it directly to judges, not uploading it to Chrome’s Play store. But in case someone who wants to influence voting does get their hands on the extension, I built in a tiny bit of security to make sure we only count votes from official judges.
The first time you install the extension, it generates a random ID number and saves it to Chrome’s local storage. When judges submit votes, the extension automatically adds this unique identifier to their submission. This way, if someone tries to spoof one of our judges, we’ll know and be able to discard their votes with a simple filter.
If I hadn’t been so focused on saving time, I might have gone for one of my earlier, more robust but time-consuming app designs. There are numerous obvious flaws with my extension, and either of my earlier designs probably would have produced a more secure and robust app. But this app didn’t need to be robust or secure. It just needed to be a more efficient use of my time than manually compiling entries.
In the end, it took me about 4 hours to design, build, test and refine the extension. We received about 75 entries to the contest, so my efforts will probably save around 17 hours of other people’s time. From my perspective, this was a big win. I had a lot more fun designing and building the extension than I would have manually collating entries, and I think our judges will be a lot happier with this voting method.
Want to keep up with our accelerator? Take a look at all the entries on GitHub and let us know which ones are your favorites by tweeting at us @fastcolabs. Judging begins now, and finalists will be announced on April 30. For more information, visit the contest page.
[Photo by Flickr user Axion23]