On the face of it SnappyCam, a $1 iOS app, is like many similar “continuous shooting” apps that try to emulate the ultra-rapid frame capture sequence that happens when you hold down the shutter on a good DSLR. But then you discover its performance, and your mind should boggle: While Apple’s own camera system can manage a full-frame 8-megapixel photo capture rate of around four or five frames a second, SnappyCam can pull 20 full-resolution camera shots off the sensor and into the iPhone’s memory in the same interval.
How did developer John Papandriopoulos manage this?
It’s actually very simple. Instead of using built-in code inside iOS and his app, he took the image-capturing part of SnappyCam right back to first principles. When you snap a photo using your smartphone, the sensor captures light and turns it into a numerical array that matches the intensity of light hitting each colored pixel. The phone takes this array, comprising millions of numbers, and compresses them using a standard algorithm–typically JPEG–so that they take up far less room in the limited space in the device’s menu. The photo hardware components in this process are typically quite speedy nowadays, but the image processing pieces don’t necessarily match this speed, partly because of the limited processor power of a phone, and the fact the hardware is having to do a lot of different processes on a relatively simple slice of silicon.
Speaking to TechCrunch, Papandriopoulos explained that he recently realized that he could achieve huge efficiencies in this process if he applied some thinking to the JPEG compression system itself. Specifically he coded a unique discrete cosine transform algorithm. The DCT is one of the tricks that JPEG compression uses to make images smaller than a raw bitmap. The JPEG format uses a normalized two-dimensional DCT based on academic work published in 1974, and like any Fourier-like compression it turns a complex signal (the raw image) into a simpler string of numbers that describe the transform of the signal (the smaller encoded image file). The coded algorithm to create JPEGs is, of course, well known and reused endlessly across all sorts of different devices running on different operating systems inside different hardware. That’s why it’s a standard.
Instead Papandriopoulos and team hand-coded a compression algorithm that used more efficient code methods to calculate DCTs, ones that were optimized for the Arm hardware on the iPhone. In fact the code for the entire app is optimized precisely for the power of the dual core processors in the iPhone 5.
Effectively SnappyCam uses a unique JPEG compression engine that’s tuned for the vehicle it’s running in–its code grabs an image from the iPhone’s sensor, compresses it as fast as possible, shifts it into memory and then returns to grab more sensor data as quickly as it can. Much quicker than Apple’s own code does.
And this last point is interesting. Apple’s code is written by Apple’s own expert coders, who are familiar with every detail of the hardware and software of their own device. But by starting at first principles of what his app is all about, Papandrioupoulos reminds every developer that there’s plenty of room for innovation even in a densely populated field, if you turn your usual expectations on their head and think about the fine details, then get seriously nerdy with your code for a moment.
[Image: Flickr user Don McCullough]