How a Hybrid Solution Addresses the Crisis of Data Speed

Subscribers to our Statistica monthly newsletter have already been made aware of the latest EMA/9sight survey results that found data-driven businesses are becoming more interested in speed than volume. This shift in focus will change market dynamics, and that’s what the survey’s executive summary is all about.

When you read the summary report, you will learn some interesting things:

  • Speed Is Driving Competition – Speed of processing response was the most frequently indicated use case by respondents, at nearly 20%.
  • Time to Value with Applications – Over 20% of respondents implemented big data projects using customizable applications from external providers.
  • Low Latency is High Profile – Big data projects are overwhelmingly near real-time, with over 32% described as real-time or near real-time processing of data.
  • Two-time Use Case Champion – For the second year in a row, the top use case for big data initiatives is speed of processing response, at nearly 20% of mentions.

Clearly, a growing portion of respondents are feeling the need for speed.

Of course, speed is intrinsically related to volume and data structure. It is because of the growing volume and variety of data—especially with the onset of the Internet of Things—that data collection and preparation now require extra attention so that time-to-value (i.e., speed) can be maintained or improved. It is also true that not every business is ready to roll with 100% all-new infrastructure (hardware + software + sensors + workflows) to handle all this change from Day One, which means that most—if not all—businesses are likely implementing their data-driven strategies in piecemeal fashion, with a mix of old and new technologies plus a wish list for more.

This is a good place to mention the “Hybrid Data Ecosystem” as a valid means of addressing the speed issue. EMA originally defined the big data Hybrid Data Ecosystem (HDE) several years ago through end-user surveys and interviews with technology thought leaders, implementation specialists, and software vendor experts. Each platform within a HDE supports a particular combination of business requirements along with operational or analytical processing challenges. Rather than advocating a single data store that supports all business and technical requirements at the center of its architecture, the HDE seeks to determine the best platforms for supporting particular sets of requirements and then links those platforms together. In this sense, HDE makes the most of the messiness of reality and the overlap of various technologies that exist side-by-side in many businesses today.

Let’s face it: conversions, migrations, and upgrades don’t happen overnight and usually involve transition periods that may last into perpetuity. Accordingly, the Hybrid Data Ecosystem is constantly refined. This year, for instance, EMA expanded the HDE scope to include the influence and impact of the cloud on big data environments and data consumers.