If you’ve ever watched A&E’s Hoarders, or more recently, SyFy’s Collection Intervention with horror and disbelief, you might want to take a look at your PC or Mac before you pass judgment.
I have personally gone through the hoarding of 5 ¼" floppy disks, 3 ½" floppies, CDs, Zip disks, flash drives, memory cards, and hard drives. What are we hoarding now? Online storage. Have you collected Microsoft SkyDrive, Google Drive, Box, and Amazon cloud storage like flash drives to some virtual key ring? Do you have images scattered across Apple’s Photostream, Google + and Picassa, Flickr, and Facebook?
If your answer is yes, then you have Distributed Data Disorder (DDD). This is no imaginary aliment. I spent nearly five hours over the weekend rationalizing the gigabytes of data that I had stored in SkyDrive and Dropbox. Duplicates, in some cases—folders with the same intent but with different names, each containing bits of my writing, my business, my personal life.
Distributed Data Disorder goes far beyond file management. It also includes contacts, and to some degree, calendars, music, and other file types. Think about your contacts. LinkedIn, Constant Contact, Twitter, and Facebook—oh my! Google+, Exchange with Outlook, Hotmail, and Yahoo. Like your files, your relationships are also scattered across multiple services, all free, all accepting of whatever data you choose to upload, and with little or no recognition that the other systems exist except for initial searching for others that may have also chosen to fragment their relationships, in order to draw your fragments to theirs.
Cases of DDD can be diagnosed, but the cures are harder for now. But like all diseases, hope is on the way.
Distributed Data Disorder symptoms include hoarding online storage, not remembering where your files are, fractured mobile indexes, and performance hits on your primary system. Let’s take a look at each symptom.
Hoarding Online Storage We’ve already touched on the first symptom: online storage hoarding. Because people have always needed more storage, any free storage is accepted. Think about it. Have you ever turned down a free USB drive from anybody? Look in your desk drawer and face the truth as all the market, shiny, glowy, key-chainy baubles rattle against one another. Are you actively inviting people to Dropbox to increase your storage quota? Did you sign up for iTunes match?
Loss of File Memory Let’s be frank. You just don’t know where it is. The family portrait from two years ago. Your girlfriend's resume. The presentation you gave a couple of months ago. You just don’t have enough addressable memory space in your brain to keep track. Sure, on your desktop computer you have search that helps bring things back, but if you haven’t installed the client then the files are in the cloud, not on the desktop, and therefore not indexed. I don’t know about you, but I have so much data that I can’t remember precisely what I called something, which means even search can’t locate it. Unlike Google on the web, local search on the PC or Mac doesn’t guess at what you want. You either know, or it remains lost.
Fractured Mobile Indexes For those cloud storage services that replicate to the desktop, Microsoft or Apple indexes that replicated content, but mobile platforms don’t have that advantage. That means if you want to find something in your cloud stores when on a mobile device, you have to first install the client and then remember where you put what you are looking for.
Performance Hits on Your Primary System Synchronization is a minor background process, but if you have been hoarding cloud storage, all those little applications that synchronize data with the cloud take just a little off the top of your CPU. Not, perhaps, enough to notice in isolation, but the random pause here, the delayed reaction there, might be exacerbated by your cloud storage addiction.
APItis If you have a mobile device, your cloud storage options are limited to the partnerships that have been struck between app vendors and cloud storage providers. Most cloud storage services offer APIs for integration, but most apps currently align with just one or two, especially if they are a cloud storage provider themselves. You can’t save a Google word processing document to Dropbox, or open an iOS Microsoft Onenote document from Box. Many apps connect to Dropbox, some to Box, some to Google Drive, but most aren’t storage agnostic. That all translates into APItis, or the failure of your favorite app to connect to your preferred cloud storage repository when you want it to.
The cure for DDD is federation. I don’t mean "The Federation" that Captains Kirk and Picard work for. I mean a bringing together, a consolidation. In the technology world, federation is common in relational databases, which also suffer from DDD, but because most of them run a standard called SQL, they can be more easily integrated (yes, I know, DBAs, not all that easy when talking hierarchical, relational or object databases, but I digress).
Federation in this case means metadata about files, including full indexing of content, to create the illusion of integration. The files and contacts remain strewn across the data centers of the world, but when you are working, they appear to you with a modicum of cohesion.
I have recently explored two products that start to bring order to those suffering with DDD.
The first product, Skoot from Topia Technologies, offers data federation. They give you 2GB of free space to store your most secure personal data, and then they federate two of the more popular cloud storage services, Dropbox and Google Drive. Topia offers a minimalist client that exposes the various cloud stores for viewing, as well as searching. Skoot's sweet spot is helping you find cloud content regardless of where it was stored, and to deliver it to you securely. Topia has spent years creating distributed data solutions for military applications. Skoot rips your files apart, encrypts the pieces and only assembles them when at the authenticated client that requested them. Currently Skoot only works with Dropbox and Google Docs, but more cloud services are on the way.
The second product to address the symptoms of DDD comes from Xobni and its Smartr Contacts mobile app. On the PC using Outlook, or in Gmail and Yahoo mail, Xobni integrates mail from all services. Search for contacts by first name, last name, or company name, and even if they aren’t in your local contact list, Xobni will find them. Same is true for BlackBerry, Android, and iOS with Smartr Contacts. If you want advanced services, like full integration of Gmail, Android, and iPhone contacts from Outlook, or their Ranked AutoSuggest for email recipients feature, you will need to pay $7.99 a month, or $47.95 a year, for those features.
More products in both camps are likely on the way. In a Google briefing with their Apps team they hinted at the number of startups they were watching in the cloud federation space. Competition will likely drive both innovation and choice, and like all cures, perhaps side effects we’ll have to track.
Xobni’s basic services are free, as are the Smartr Contacts mobile apps. No brainer. Download and integrate. Skoot is new and just rolling out, and they have user interface and integration work to do, but it’s a start. You can experiment with Skoot for free, but eventually Topia will introduce a fee-for-service model.
New forms of DDD are on the horizon. Apple’s iTunes Match and Amazon’s Cloud Player offer options for fragmenting your music collection. They both offer the illusion of a solution, but actually increase the severity of DDD if you choose to use both. Those $5 digital download deals from Amazon now likely exist in both the Amazon Cloud and on iTunes, and perhaps matched in the iTunes cloud. In cases like this you have to own the integration, because nobody else does at the moment. Before you scatter and duplicate your music assets, pick a spot and stick with it.
As we go forward, the applications of federation are much more interesting than the curating. Apple, Microsoft, and the Linux community have all failed to reinvent personal content storage management metaphors and interfaces at the core of their systems. Sure they introduced indexing, search, smart folders, and libraries, but these were pretty basic enhancements to metaphors that haven’t changed for decades.
I’m looking forward to the cloud federation suppliers to not only cure my DDD, but to also improve my quality of life. Imagine services that help you rationalize your content, telling you where you have duplicate files and offering ways to eliminate those, freeing precious cloud space for other content—analyzing content so the big files that take up all the space can be easily identified. Imagine rich metadata that lets you visualize your content in new ways, based on projects, or topic, or people who received it, or might want to receive it. I hope these new services aren’t just utilitarian answers to Distributed Data Disorder, but new, and innovative ways, for people to interact with their content.
[Image: Flickr user Worapol Sittiphaet]