Posts tagged big data

It’s that time of the year when technology pundits, mavens and analysts gaze into their crystal balls and make their predictions and prognostications for the next 12 months.

One of the hottest technology sectors over the past year and one that most industry observers agree will continue to grow in importance and reach in 2013 is Big Data. Big Data is more than just a buzzword. It’s real, significant and is driving disruptive changes in major markets like healthcare, finance, retail, media and advertising. Gartner forecasts that in 2013, Big Data will drive $34 billion of enterprise spending. 

While the overall market for Big Data will grow by over 20% this year, many industry experts anticipate significant changes in the type of Big Data solutions that will gain market share in 2013. For example, Forrester Research predicts that “real-time architectures will swing to prominence in 2013.” Specifically, Forrester states that:

Firms will seek out streaming, event processing, and in-memory data technologies to provide real-time analytics and run predictive models…In 2013, enterprise architects will step out of their ivory towers to once again focus on technology — real-time technology that is highly available, scalable, and performant.

Forrester also predicts that in 2013 the demand for actionable insights and advanced analytics will require Big Data solutions to extend beyond simple rules-based models and incorporate sophisticated statistical algorithms. 

Advanced analytics uses advanced statistical, data mining, and machine learning algorithms to dig deeper to find patterns that you can’t see using traditional BI tools, simple queries, or rules. Big data is the fuel. Algorithms are the engine to explore virgin data, seek out new meaning and models, and deliver more personalized, contextualized customer experiences.

Investors are also predicting that 2013 will witness major changes in the way Big Data solutions are delivered and architected. For example, Hollie Moore Haynes, managing director with tech-focused private equity firm Silver Lake, predicts that 2013 will usher in Big Data’s second act. 

The second act of Big Data is about action. This time intelligence will go beyond reporting. It will live at the heart of the organization, not at the edges. It will involve moving from historical insight to predictive information, enabling managers to make smarter decisions.

So there you have it. Big Data in 2013 will continue to be a big deal. More importantly, the solutions that will likely win the day (or the year, as it were) will look a lot like TrendSpottr: real-time, predictive and algorithmic. 

Here’s to the pundits and to a fabulously successful 2013!  

Earlier this week, Fred Gallagher of Actian Corporation, wrote a guest post on GigaOm titled, “Big Data: The quick and the dead”. In the post, Fred challenges the conventional thinking about Big Data; specifically, “the implied premise that bigger is better and that adding more data will naturally produce insights.” Fred goes on to note that about 95 percent of data in a data warehouse is dormant and that adding terabytes or petabytes of unstructured data to this data warehouse is likely to make dormant data, dead data. What businesses need and what ultimately drives business value, Fred correctly notes is “live “quick” data not dormant or dead data.”

At TrendSpottr, we believe that one of the most disruptive Big Data opportunities is the real-time analysis of “live” data elements. More specifically, our focus is on identifying trends and predictive insights from real-time, incremental streams of Big Data; i.e, a predictive early warning system based on real-time pattern recognition and analysis of the velocity and acceleration of living data. The reality, as Fred notes, is that most of the data that is being analyzed today by Big Data implementations is dormant or dead. By focusing on only those data elements that provide the potential for insight and action, we can leverage the potential of Big Data by making sure it’s alive! 

Finally, a quick plug for TrendSpottr cofounder and CTO, Alain Chesnais. In addition to being the technical architect of the TrendSpottr service and the source of one of our favourite recent quotes, “What we’re doing is actually playing the role of an automated data scientist for you. Now you can consume big data without needing a Ph.D in statistics”, Alain also serves as President of the Association for Computing Machinery (ACM), the world’s largest scientific and computing organization.

This week, the ACM is in Warsaw, Poland hosting the prestigious and highly competitive International Collegiate Programming Contest. The results are now in and the winner of this year’s ICPC is St. Petersburg State University. The University of Warsaw came in second place and our very own University of Waterloo came in ninth place. Congratulations to all the teams and to the ACM and Alain for hosting this amazing event.