Streaming Analytics is impacting a wide variety of today’s hottest technology domains. As such, I and my fellow technologists at Striim have put together our Top 20 Predictions for 2016 across many of the areas that touch Streaming Analytics. Enjoy!
Big Data/Data Lakes
1. Disillusionment with Data Lakes will intensify through 2016. Enterprises will take a much needed step back and focus on use-case-based loading of Lakes, rather than arbitrary data dumping.
2. Data Lakes will be primarily used for historical analysis and archiving purposes with Enterprises moving to streaming analytics for real time insights and operational analytics.
3. More Data Lakes will turn into Data Swamps due to an absence of an analytics strategy for pre-processing.
Transactional Systems
4. Enterprises building Data Lakes will realize that their 360-degree view is incomplete without the Transactional Data originating in Enterprise Databases, and endeavor to include this. These Enterprises may stumble using traditional ETL or querying techniques, but will move steadily towards standardizing on Change Data Capture for this purpose.
5. Enterprises will continue the trend of moving Databases to the cloud. The initial focus will be on non-mission critical data stores. However, as more of the enterprise is hosted using hybrid cloud models, more mission critical databases will also be migrated.
Cloud
6. Cloud expenditure will increase through 2016 as benefits of scaling, redundancy, and geographic distribution bear fruit. Hybrid cloud models using public cloud infrastructure privately through VPNs will enable elastic scaling without many of the perceived security issues of public cloud.
7. IoT will drive cloud adoption as more data processing is performed in a scalable fashion. Cloud infrastructures will also enable edge processing and analysis of data as much of the IoT data originates in a globally distributed fashion.
Open Source Technologies
8. A backlash against open source will occur in 2016 as Enterprises discover the true costs of maintaining teams of engineers to develop and maintain complex infrastructure to support data processing efforts.
9. The Open Source market place will continue to grow and diverge in almost all data processing and analytics related areas. This will add to confusion and frustration amongst Enterprise customers who will be spending more and more time evaluating and attempting to integrate different solutions.
Data Integration
10. Data Integration requirements will rapidly expand in 2016 as more sources of data continually come online. Data Integration platforms will not only need to address these new sources, but also scale for exponentially higher volumes, and manage diverse processing requirements on-premise, in-cloud and in hybrid cloud models.
11. As enterprises increasingly adopt IoT, Big Data Analytics, and Cloud Computing in their mission critical operations, there will be a requirement for true real-time data ingestion and integration that can handle the scale and speed of next-generation platforms.
12. Next-generation data integration players will emerge to support real-time, operational data that provides analytics while the data is in-motion.
Complex Event Processing
13. Complex Event Processing will cease to be a standalone technology, and will instead be an integrated requirement of all streaming analytics platforms.
Log Monitoring
14. As streaming integration platforms expand over the enterprise, log data will switch from being used for historical analysis and forensics purposes to being an essential aspect of Enterprise operational intelligence.
Fraud
15. In 2016 Fraud Prevention will be a key driver of competitive advantage. Current, after-the-fact, fraud detection techniques will not be tolerated by customers who expect organizations to have real-time insight into what is happening. Customer loyalty will depend on Enterprises spotting fraud as it is occurring and taking active steps to prevent fraudulent events.
Edge Processing
16. As ever-larger volumes of IoT data come online, it will be essential for more processing to be moved closer to the devices in order to mitigate network overload and growing storage costs. This processing must happen in real-time as soon as the data is produced in order to remove duplicate or useless data. The edge will start out in the cloud, but will move into vehicles, plants, factories and stores throughout 2016.
Customer Experience Management
17. Customer loyalty will be an ever growing concern through 2016. Enterprises will need to have up-to-the-second customer information in order to remain competitive. Customer experience with real-time apps, and current information in the social media realms will drive expectations for Enterprise to understand what the customer knows and wants, as soon as, or before, they know it.
18. As smart devices infiltrate all aspects of daily life, real-time analytics will be critical to Customer Experience management systems. Enterprises who are not able to understand their customers needs in real time will see an unprecedented increase in customer churn.
IoT
19. IoT will be the #1 driver of data growth in 2016. Enterprises dealing with IoT data that do not move to edge processing will be threatened with network and storage overload.
20. The variety of devices producing real-time data will continue to grow in 2016, with cities, cars, factories, farms, utilities, and shipping being the major drivers. The rate at which data can be consumed to provide meaningful insights will lag behind the rate of data increase until Enterprise embrace edge processing and real-time analytics.
