As the world becomes more digital, every text, email, Web page browsed, location app updated, Facebook Like, Google search, and financial transaction creates data. Collectively, the tools used to mine enormous amounts of data for nuggets of information are called Big Data.
But the "big" part of the name doesn't tell the entire story. Data scientists (a real job description, and now in great demand) point out that Big Data includes unstructured data as well. Texts, emails, user comments on websites, uploaded photos, and Twitter feeds follow no pattern.
Big Data requires massively parallel server computing clusters with terabytes upon terabytes of fast-access storage. Companies like IBM, HP, SAP, Oracle, and SAS help enterprises with Big Data projects using budgets far beyond the reach of SMBs. However, the open source movement and academia have created most of the basic tools used in Big Data, including Hadoop, NoSQL, and MongoDB.
Senior Analyst Jim Bagley of Storage Strategies NOW reports a surprising number of small businesses need Big Data-level solutions. "Radiologists have to keep massive MRI scans of patients forever, and this is a growing area of interest. Media, oil and gas consultants, and research organizations have giant Big Data needs." His advice for resellers? "A savvy reseller who understands client metadata needs can come up with some pretty slick cloud applications quickly, and build from there."
Luckily, the costs of handling Big Data have dropped dramatically, more tools than ever are available, and third parties will collect your data into their cloud computing system and format it for you, sometimes for just a few dollars per month.
What follows is a sampling of solutions for companies of all sizes and pocketbooks, plus a case study on SNAP Interactive's experience with a Big Data provider.