Big Data and the Cloud facilitates innovation in data management

From innovative software applications and social media insights to Internet search results and email, the rate of big data creation is growing exponentially with no signs of abating. This has resulted in some companies having to play catchup. Those that are clever enough to lead in front of the pack, ensure their survival or disrupt the marketplace. This week we had the unveiling of microsoft’s xbox one, and saw how tightly integrated it is to our living rooms and entertainment experience.

Google is also coveting the TV space and is slowly rolling out a trojan horse with the aim To Disrupt TV  and it’s all based on really Big data. You can read about that here.

Companies have for years, seen information as the holy grail of competitive differentiation – if only we had more customer data, if only we knew more about market patterns. Despite all the focus, the actual meaning of big data and its application aren’t always clearly articulated, but the concept is actually  fairly simple. A simplified definition of big data could be- huge volumes of information that are extremely large and growing. That can mean data already being generated within an organization by systems or machines, like manufacturing output or customer buying patterns; or increasingly, data that’s available for companies to buy, provided from sources like social media vehicles or Internet search providers.

Enter big data solutions designed to turn this information tsunami into an information gold mine.  However a recent study found that an accelerated fragmentation of data standards plus more information sources than ever before are clashing with the growing need for decisions makers to gain instant insight.

“While the data management leaders have greatly increased functionality, specifically around the cloud, we are seeing tremendous innovation from the facilitator quadrant. Among them, Kalido, Pervasive and Attunity focus on a specific set of data management capabilities and have emerged as three innovators to watch”. “Big data and the need to capture information from disparate resources, both internally and externally, has shifted IT environments from relatively structured data to the Wild West where anything goes. The personalization of IT, where end users are taking increasing control of data environments is only compounding the problem,” said Ian Campbell, CEO of Nucleus Research.

To handle increasingly unstructured, multi-source data environments, leading vendors are leveraging cloud technology as a means to move data across organizations and make information readily available as needed. The clear leader in the current matrix, Informatica provides a single, unified metadata platform that streamlines the sharing of data across a complex, unstructured environment. IBM also scored high on the matrix with the ability to support a high degree of customization and scalability.

“While the data management leaders have greatly increased functionality, specifically around the cloud, we are seeing tremendous innovation from the facilitator quadrant. Among them, Kalido, Pervasive and Attunity focus on a specific set of data management capabilities and have emerged as three innovators to watch,” said Hyoun Park, principal analyst, Nucleus Research.

As we approach the midway point of the year, I thought it would be good to revisit Sap’s 2013 IT Market Trends And Predictions On Big Data. With the issues that decision makers face and the fact that the cloud continues to emerge as a value differentiator for leading vendors. I think they got it spot on. What do you think?

Some of the key trends included:

  • M&A: Analysts predict that Big Data technology and services will reach $10 billion in 2013, primarily driven by M&A activities in predictive analytics, rich media analytics, and data discovery.
  •  In-memory/real-time enterprise: With declining hardware costs and ever increasing demand for faster performance and real time computing, in-memory computing will gain popularity.
  • Industry-specific offerings: With the evolution of different use cases around leveraging Big Data,” there is a renewed focus on industry-specific offerings.  Retailers will try to maximize the value delivered to consumers, while manufacturing and financial services are trying to improve their operational productivity.
  • Volume: As the data avalanche continues, the “digital universe” is expected to explode.
  • Hadoop: Increased data-mining demands on large sets of data coupled with need for cost effective solutions have driven demand for the Hadoop/ MapReduce software ecosystem.
  • Governance: Big Data is becoming more of a mix of structured and unstructured data.

[xyz-ihs snippet=”it”]