Leading Logistics Technology Provider Leverages Striim Cloud to Build Rapid and Intelligent Data Pipelines

The Challenge

The main challenge for this business was to maintain real-time synchronization with the ever-changing status of trucks on the road. Any delays in data updates could cause discrepancies between real-world scenarios and the dispatch planning system. Furthermore, they needed a solution to put a manageable technological demand on their small team.

The company’s business model relies on aggregating their customer’s data from various sources including legacy systems such as Transportation Management Systems (TMS) and Electronic Logging Devices (ELD). They used a system called “completeness, accuracy, and timeliness” for data delivery but found that their previous vendor limited the speed at which the data flowed into their platform. Determined to disrupt the status quo, the team decided to remain at the forefront of modernizing the trucking industry and pursue their search for a real-time ETL solution. 

Their customers’ needs primarily drove the decision to explore new providers for faster and more accurate dispatch solutions. They required an ETL system to process and deliver large volumes of data (10,000 – 100,000 records) in under a minute.

The Solution: Striim Cloud – Fully managed Change Data Capture and Streaming SQL

After a comprehensive evaluation process against competitors like AWS and Talend, this logistic technology provider chose Striim. The deciding factors were Striim’s superior speed, performance, user-friendly interface, and cost-effectiveness. In an industry where trucks are always moving in real time on the road providing a constant stream of data to the operators, speed is critical. 

Striim’s ability to reduce data ingestion and transformation time from five minutes to less than one minute were a critical factor for this logistics tech provider. Streaming SQL and in-stream transformations were the key components leveraged to optimize the performance of their pipelines. Moreover, Striim’s Out-Of-The-Box (OOTB) components and drag-and-drop interface enabled the team to build data applications without heavy coding. 

Finally, Striim Cloud’s usage based pricing and the flexibility to strategically use credits for cost planning further cemented its appeal.

Implementation

The company’s transition to Striim was smooth and streamlined, facilitated by Striim’s customer support and solutions team. After defining the scope with Striim’s team, this logistics tech provider began with the platform before shifting to the cloud.

The proof-of-concept exercises and the initial setup were completed within a matter of weeks, during which Striim’s comprehensive documentation and templates helped their team build applications faster. Within two weeks, the company was up and running with its first customer in a sandbox for testing. After that, they gradually moved other pipelines over to Striim, gaining confidence in the robustness and reliability of the Striim platform.

The team at this logistics technology provider leveraged Striim’s Streaming SQL capabilities to perform aggregates with in-memory sliding windows that apply logic and case statements on real-time streaming data. They also built their own UDFs (User Defined Functions) in Java to apply flexible procedural logic on data streams in the cases where SQL was not flexible enough.

The team also successfully conducted a proof-of-concept (PoC) with Striim, demonstrating their capability to read CSV files from an AWS S3 bucket and efficiently write the data to a MySQL database in real time. The PoC primarily focused on processing and transforming customer, trucking, and fleet information using extensive SQL transformations, aggregations, and joins.

By leveraging Striim’s powerful data integration and streaming analytics platform, this logistics technology provider seamlessly ingested the CSV files from the AWS S3 bucket, ensuring a continuous flow of data from the source. Striim’s intuitive interface and robust connectors facilitated the integration process, enabling the extraction of valuable insights from the customer, trucking, and fleet data. 

*To enable real-time analytics, the company implemented advanced SQL transformations, aggregations, and joins using Striim’s SQL-based streaming analytics capabilities. These operations allowed them to perform complex data manipulations and derive meaningful patterns and correlations from the raw data streams.

Throughout the PoC, the team demonstrated their expertise in leveraging Striim’s platform to handle large volumes of data and process it in near real-time. By effectively utilizing Striim’s capabilities, the company efficiently executed the necessary SQL operations to enrich, cleanse, and standardize the customer, trucking, and fleet information.

The successful completion of this PoC highlights the logistics provider’s proficiency in leveraging Striim’s streaming data integration and analytics capabilities to empower organizations with real-time insights. With their demonstrated ability to process data from AWS S3, perform SQL transformations, aggregations, and joins, this company paved the way for organizations to unlock the full potential of their customer, trucking, and fleet information for enhanced decision-making and operational efficiency.

The Impact

Since implementing Striim, the team at this logistics provider has seen transformative changes in its business operations. The significantly reduced data ingestion and transformation time, which resulted in quicker dispatch planning. This enabled their customers to see results up to 80% faster than their previous SLA.