Stream Storage

Scalable technology for event and message storage

Streaming data messaging and processing require a robust storage solution to ensure scalability, performance, and data durability. Storage that is just patched into messaging or processing technology forces compromises that impact performance, durability, and scalability.

Streamlio uses Apache BookKeeper as its foundational storage system for streaming data. Apache BookKeeper is a scalable, fault-tolerant, and low-latency storage service optimized for streaming data. Originally developed at Yahoo!, BookKeeper has been widely adopted by enterprises including Twitter, Yahoo! and Salesforce to store and serve mission-critical data in a variety of use cases.

Streamlio's Ivan Kelly explains how Streamlio uses Apache BookKeeper

Highlights

Low latency

Performance isolation of reads and writes ensures fast response times even under load, without sacrificing consistency and durability.

Built-in replication

Data is replicated and durably stored on multiple machines and can easily be replicated across datacenters in different regions or geographies

Data durability

Data is guaranteed to be written to persistent storage and replicated before write operations complete to avoid data loss scenarios

Data consistency

Simple, repeatable read consistency model guarantees that all readers can see a consistent view of data

Availability

Ensemble changes and speculative reads ensure write and read availability without compromising consistency and durability

Segment offload in Apache Pulsar tiered storage

Blog: Tiered Storage in Apache Pulsar

Streamlio's Ivan Kelly explains how Pulsar can automatically and transparently offload data to external storage.