Real-Time Stream Processing

Scalable and robust engine for processing and acting on streaming data

Data-driven applications need to process data as it arrives, acting on data in real-time to be able to deliver information to applications and users as quickly as possible. However, existing technologies, born in a batch processing world, were not designed for this.

Streamlio address this need with proven, best-of-breed technology for real-time and streaming data processing. Building on the the streaming processing capabilities in Apache Pulsar and the Streamlio team’s experience as co-creators of the Apache Heron* real-time engine at Twitter, Streamlio enables enterprises to deliver data-driven applications that can meet demanding SLAs at any scale.

Streamlio co-founder Sanjeev Kulkarni explains how Pulsar provides a serverless approach to stream processing

Highlights

Real-time stream processing

Streamlio’s event-based design makes it possible to process data immediately, as it arrives. Tune latency and throughput to meet demanding SLAs with end-to-end latencies in tens of milliseconds

Proven solution

Building on Apache Pulsar, the streaming solution proven in production at Yahoo, and experiences building and operating Apache Heron, the streaming data processing engine in production at Twitter, the Streamlio solution can handle 100s of billions of messages per day.

Scalable performance

With a multi-tenant architecture that ensures workload isolation, the ability to tune throughput and latency for individual workloads, and stream-native processing jobs, the Streamlio platform can meet demanding SLAs even as data and workloads scale

Self-healing

Automated monitoring and healing features regulate data flow and processing to automatically address bottlenecks and ensure resiliency even as workload demands fluctuate and failures occur

Development flexibility

Support for multiple processing semantics–at least once, at most once, effectively once–together with support for a serverless-inspired approach to stream-native processing makes it easy to meet diverse development needs

Microservices design

Modular, extensible architecture designed to take advantage of containers and frameworks such as Docker, Kubernetes and Mesos to optimize resource management and provide deployment flexibility

Diagram illustrating how Pulsar Functions process streaming data

Blog: Introducing Pulsar Functions

Read about how Pulsar Functions provides an easy way to create and deploy stream processing functions in Apache Pulsar.

Image illustrating streaming lights at an intersection

Webcast: Evaluating Streaming Data Solutions

Streamlio's Sanjeev Kulkarni looks at the requirements for modern streaming solutions and what Apache Pulsar* provides to meet those requirements