In a recent episode of Robin Williams’s sitcom “The Crazy Ones,” a client of his advertising firm forces him to bring on a “quant,” or quantitative analyst. Williams’s character has no idea what a quant is, but his daughter explains that they are an essential part of business nowadays.
To me, this is just one more sign that big data has jumped the shark (another TV-based expression) and is no longer in its hype phase. Instead, big data is a normal part of the business lexicon, and organizations dealing with analytics projects assume they will involve big data.
Another signal that big data is everyday, not headline-making news anymore, is that we can all finally agree on a definition. For years, I’d have to ask sources I’ve interviewed what they meant by big data, because there hadn’t been general consensus. Some referred to big data as the need for larger storage systems, because they felt they had to keep every bit of information about their business and customers. Trust me, the storage vendors weren’t complaining.
Some felt big data was more about the gathering of data via new front-end systems that would soak up as many details about their clients, product manufacturing, and other essential aspects of business as possible. Again, no software vendors were shooting this idea down.
Still others focused on the churn of data to pop out analytics reports. What companies did with that insight was neither here nor there; they just were cranking out reports based on the big data. Their goal: to satisfy the CEO’s demand for a big data project, which he or she issued after reading about it in a back-of-the-airplane-seat magazine.
All of the above
What I’ve learned over the years is that big data, if done correctly, creates a fine-tuned cycle out of all these elements. Yes, businesses do have to glean information from customers, business partners, and others in their sphere of influence. However, big data can help them pinpoint the information that’s most relevant and useful.
Yes, big data requires the storage of terabytes and petabytes of information. But it can also help pare down what needs to be stored by identifying the essential and unnecessary.
And yes, big data is about leveraging analytics, but not to generate superfluous reports. Instead, it’s about action and competitive advantage. At its best, big data is not about reaction; its strength is in prediction and strategy.
The maturation of the market has meant a coming together on what big data actually means, and that has pushed it over the hype hump.
Laura Madsen, leader of the healthcare practice at BI consulting firm Lancet Data Sciences, noted in a Data-Informed.com column last year that big data may have actually hit its tipping point in early 2010.
According to Google Trends, we are at the high-point of searches associated with the term ‘big data’ and we have been for a few months. It’s not clear how long this will last. And although theories differ on this, I believe that February, 2010 was the tipping point. The Economist published a feature called ‘The Data Deluge,’ and according to Google Trends, just nine short months later the increase in searches for big data began its meteoric rise.
Madsen summed it up best when she said, “I have a confession to make. Anytime I hear the term ‘big data,’ I simply zone out and hear the sound of Charlie Brown’s teacher (Wah wah wah wah…).”
Maybe instead of dreaming of being a famous World War I fighter pilot, Snoopy one day will lay on the doghouse thinking about being a quant.