In this post and accompanying video we introduce Pulsar Functions, a new stream-native processing capability for Apache Pulsar. Pulsar Functions allows developers to create fast, flexible data processing tasks that operate on data in motion as it passes through Pulsar, without requiring external systems or add-ons.
We explain the impetus for building this functionality into Pulsar, which comes down to the fact that for a developer looking to solve a fairly simple business problem it often involves multiple separate systems including a message queue, stream processing engine, and data serving system. For example, let’s consider the business case of an image uploading capability, where a stream of images comes into the application, and they need to be resized and delivered back to the application.
Apache Pulsar makes this type of use case very easy to code, by eliminating the complexity of having multiple systems and needing to learn different APIs. In the following video we will walk you through the architecture of Apache Pulsar and the new Pulsar functions, and then show you how to code the example outlined above: