Resources and tutorials

FEATURED

Streamlio Solution Brief

Read about the Streamlio intelligent platform for fast data.

Read More

Resource Library

solution brief

Streamlio Real-Time Solution

Streamlio Real-Time Solution

Overview of Streamlio's solution for connecting, processing, and moving streaming data

blog

Open-Sourcing Pulsar, Pub-Sub Messaging at Scale

Open-Sourcing Pulsar, Pub-Sub Messaging at Scale

Read about the origins and usage of Pulsar at Yahoo in this blog

presentation

Pulsar, a highly scalable, low latency pub-sub messaging system

Pulsar, a highly scalable, low latency pub-sub messaging system

Presentation by Yahoo! providing a brief technical overview of Apache Pulsar

presentation

Pulsar -- Distributed Pub/Sub Platform

Pulsar -- Distributed Pub/Sub Platform

A technical dive into the Apache Pulsar architecture, features, and use cases

whitepaper

Twitter Heron: Towards Extensible Streaming Engines

Twitter Heron: Towards Extensible Streaming Engines

Paper from IEEE on the Apache Heron goals and architecture

whitepaper

Dhalion: Self Regulating Stream Processing in Heron

Dhalion: Self Regulating Stream Processing in Heron

VLDB paper from Microsoft Research covering the self-regulating features of Dhalion and its implementation for Apache Heron

video

Twitter Heron on Apache Aurora

Twitter Heron on Apache Aurora

Read why Twitter replaced Apache Storm with Apache Heron for real-time data processing

video

Flying Faster with Heron

Flying Faster with Heron

Karthik Ramasamy presents an overview of the design and implementation of Apache Heron

video

Heron: Real-Time Stream Data Processing at Twitter

Heron: Real-Time Stream Data Processing at Twitter

See Karthik Ramasamy explain why Twitter created Heron to replace Apache Storm for real-time processing

presentation

Real-Time Analytics: Algorithms and Systems

Real-Time Analytics: Algorithms and Systems

An in-depth overview of streaming applications, algorithms, and platforms

presentation

Stream Processing and Anomaly Detection

Stream Processing and Anomaly Detection

Presentation on the use of streaming and real-time processing for anomaly detection at Twitter

whitepaper

Low Latency Stream Processing: Apache Heron with Infiniband & Intel Omni-Path

Low Latency Stream Processing: Apache Heron with Infiniband & Intel Omni-Path

Read how to deploy Heron with Infiniband and IntelOmniPath for extreme performance

whitepaper

Streaming@Twitter

Streaming@Twitter

IEEE paper covering the design goals and architecture of Apache Heron

presentation

Apache BookKeeper: A High Performance and Low Latency Storage Service

Apache BookKeeper: A High Performance and Low Latency Storage Service

Presentation about BookKeeper's origins, design and use cases

whitepaper

DistributedLog: A High Performance Replicated Log Service

DistributedLog: A High Performance Replicated Log Service

IEEE paper on DistributedLog, the open source log solution built on Apache BookKeeper

whitepaper

A Technical Review of Kafka and DistributedLog

A Technical Review of Kafka and DistributedLog

Technical comparison of how Apache Kafka and Apache BookKeeper store data

whitepaper

Durability with BookKeeper

Durability with BookKeeper

An ACM paper providing an overview of replication and striping for performance and availability in BookKeeper

presentation

Apache BookKeeper as a Distributed Store

Apache BookKeeper as a Distributed Store

Presentation on how Salesforce uses Apache BookKeeper

presentation

Cloud Messaging Service

Cloud Messaging Service

Presentation explaining the motivations and design decisions made in created Apache Pulsar

presentation

Building reliable systems with Apache BookKeeper

Building reliable systems with Apache BookKeeper

Presentation on key considerations in the design of resilient systems such as Apache BookKeeper

Video

Microservices Part 3: Security Considerations

Microservices Part 3: Security Considerations

Part 3 in our tutorial series on microservices, covering key security needs and implications.

Video

Microservices Part 2: How to break up a monolithic application: guidelines and best practices

Microservices Part 2: How to break up a monolithic application: guidelines and best practices

Part 2 of our tutorial series on microservices, covering how to plan and get started.

Video

Microservices Part 1: What are they, what aren't they? An interview with an expert

Microservices Part 1: What are they, what aren't they? An interview with an expert

Tutorial series on microservices, providing an overview of key concepts.

Video

Unified queuing and streaming: Part 2

Unified queuing and streaming: Part 2

A dive into a use case requiring queuing and streaming.

Video

Unified queuing and streaming: Part 1

Unified queuing and streaming: Part 1

Explains the differences between message queuing and streaming.

Video

Migrating an Apache Kafka application to Apache Pulsar with no code changes

Migrating an Apache Kafka application to Apache Pulsar with no code changes

Walks through how to migrate Kafka applications to Pulsar.

Video

Install Apache Pulsar and get up and running in 4 minutes

Install Apache Pulsar and get up and running in 4 minutes

A step-by-step demonstration of setting up Apache Pulsar on your laptop.

Video

Introduction to microservices and real-time data processing

Introduction to microservices and real-time data processing

Presentation on the relationship between microservices and real-time processing.

Video

Running Heron on Google Kubernetes Engine

Running Heron on Google Kubernetes Engine

Demo on how to set up Apache Heron on GKE.

Video

How Apache Pulsar uses Apache BookKeeper to store topics

How Apache Pulsar uses Apache BookKeeper to store topics

How Apache Pulsar uses Apache BookKeeper to store and replicate topics.

Video

Message guarantees in Apache Pulsar with Apache BookKeeper

Message guarantees in Apache Pulsar with Apache BookKeeper

How Apache Pulsar processes messages in order to ensure that guarantees are met.

Video

Experiences in production with Heron

Experiences in production with Heron

Lessons learned from running Apache Heron in production at Twitter.

Video

How Apache Pulsar stores cursors using Apache BookKeeper

How Apache Pulsar stores cursors using Apache BookKeeper

How Apache Pulsar uses Apache BookKeeper to store and manage cursors.

Video

Introduction to the Heron Stream Processing Engine's Architecture

Introduction to the Heron Stream Processing Engine's Architecture

Video

Microsoft: Self-regulating stream processing in Heron, Part 4

Microsoft: Self-regulating stream processing in Heron, Part 4

Video

Microsoft: Self-regulating stream processing in Heron, Part 3

Microsoft: Self-regulating stream processing in Heron, Part 3

Video

Microsoft: Self-regulating stream processing in Heron, Part 2

Microsoft: Self-regulating stream processing in Heron, Part 2

Video

Microsoft: Self-regulating stream processing in Heron, Part 1

Microsoft: Self-regulating stream processing in Heron, Part 1

Video

Multi-tenant Messaging with Apache Pulsar (and Monopoly!)

Multi-tenant Messaging with Apache Pulsar (and Monopoly!)

Video

Introduction to the Apache Pulsar Messaging System

Introduction to the Apache Pulsar Messaging System

Video

Heron Delivery Semantics, Part 2

Heron Delivery Semantics, Part 2

Video

Heron Delivery Semantics, Part 1

Heron Delivery Semantics, Part 1

Video

The Streamlio sandbox walkthrough

The Streamlio sandbox walkthrough

Video

Apache Pulsar (incubating) Concepts and Terminology

Apache Pulsar (incubating) Concepts and Terminology

Video

Heron Stream Processing Architecture and Terminology

Heron Stream Processing Architecture and Terminology