1 of 35

Let’s Spring Forth and Stream

with Apache Pulsar

1

@KoTurk77

May 2023

2 of 35

https://bit.ly/41J3Xw1

This slide deck can be accessed here:

3 of 35

Speakers

Christophe Bornet

SWE on Pulsar team @ DataStax
@cbornet_
https://github.com/cbornet
Open Source Committer

Mary Grygleski

Streaming Developer Advocate @ DataStax
@mgrygles
https://github.com/mgrygles
Java Champion
President of the Chicago Java Users Group

4 of 35

Agenda

Introduction to Spring / Pulsar
High-level overview
Live demo
Feature roadmap
Q+A

Cover w/ Image

5 of 35

Introduction

01

6 of 35

What is Apache Pulsar?

Open-source

Distributed messaging + streaming

Cloud-native + multi-tenant + high-performance solution

Best of both worlds (messaging + streaming)

Popular - Became top-level Apache project in 2018 has 12.4K Github stars and 601 total contributors and 69 active contributors

7 of 35

What is Spring for Apache Pulsar?

Brings Apache Pulsar into Spring ecosystem

Adds Apache Pulsar to the list of supported messaging systems

A library that makes it easy to create stand-alone, production-grade Spring based applications using Apache Pulsar that you can “just run”

8 of 35

Backstory

Soby Chacko and Chris Bono kicked off the project

“Where’s my Reactive?”

Community (Paired w/ DataStax on the Reactive side)

9 of 35

High-Level Overview of Apache Pulsar

02

10 of 35

Built on Pub-Sub Pattern

Message: the data that moves through the service.
Topic: a named entity that represents a feed of messages.
Subscription: a named entity that represents an interest in receiving messages on a particular topic.
Publisher (ie. producer): creates messages and sends them to the messaging service on a specified topic.
Subscriber (ie. consumer): receives messages on a specified subscription.

11 of 35

Flexible Subscriptions

12 of 35

Architecture

Pulsar Client APIs

Producers

Broker

Consumers

Topics

13 of 35

Architecture

Producers

Broker

Consumers

Topics

Reactive

client

Imperative

client

14 of 35

Architecture

Producers

Broker

Consumers

Topics

Reactive

client

Imperative

client

Spring Pulsar

Spring Framework

Auto-configured clients

Spring Boot

SPRING PULSAR

15 of 35

Architecture

Producers

Broker

Consumers

Topics

Reactive

client

Imperative

client

Spring Pulsar

Spring Framework

Auto-configured clients

Spring Boot

PulsarTemplate

@PulsarListener

SPRING PULSAR

16 of 35

Architecture

Producers

Broker

Consumers

Topics

Reactive

client

Imperative

client

Spring Pulsar

Spring Framework

Auto-configured clients

Spring Boot

PulsarTemplate

@PulsarListener

SPRING PULSAR

ReactivePulsarTemplate

@ReactivePulsarListener

17 of 35

Produce / consume (imperative)

@SpringBootApplication�class SpringPulsarHelloWorld { public static void main(String[] args) {� SpringApplication.run(SpringPulsarHelloWorld.class, args);� }�� @Bean� ApplicationRunner runner(PulsarTemplate<String> template) {� return (args) -> template.send("hello-topic", "Hello World!");� }�� @PulsarListener(topics = "hello-topic")� void listen(String msg) {� System.out.println("Message Received: " + msg);� }�} // spring.pulsar.consumer.subscription-initial-position=earliest

18 of 35

Produce / consume (reactive)

@SpringBootApplication�class SpringPulsarReactiveHelloWorld { public static void main(String[] args) {� SpringApplication.run(SpringPulsarReactiveHelloWorld.class, args);� }�� @Bean� ApplicationRunner runner(ReactivePulsarTemplate<String> template) {� return (args) -> template.send("hello-topic", "Hello World!").subscribe();� }�� @ReactivePulsarListener(topics = "hello-topic")� Mono<Void> listen(String msg) {� System.out.println("Message Received: " + msg); return Mono.empty();� }�} // spring.pulsar.reactiveconsumer.subscription-initial-position=earliest

19 of 35

Additional Noteworthy Features

Pulsar IO / Pulsar Functions support

Error handling + Dead-Letter-Topic + retries (imperative)

Pulsar Reader API support

Automatic schema & topic detection/resolution

20 of 35

21 of 35

Additional Noteworthy Features

Spring Cloud Stream Pulsar binder

GraalVM native image support

Built-in Observability (imperative side)

22 of 35

Benefits

Increase DV (Developer Velocity)

Increase DX (Developer Experience)

23 of 35

The live demo…

03

24 of 35

Source code

https://github.com/cbornet/intro-spring-pulsar-demo

25 of 35

What’s next?

Transactions
Automatic failover support

26 of 35

Core

Auto-configuration

Pulsar Binder

Spring Pulsar v0.2.0

Project evolution

Spring

Pulsar

27 of 35

Project evolution

Spring

Boot

Spring �Cloud Stream

Spring Boot v3.2

Spring Cloud Stream v4.1

Core

Auto-configuration

Pulsar Binder

Spring Pulsar v1.0

Spring

Pulsar

28 of 35

Project evolution

Spring

Boot

Spring �Cloud Stream

Spring Boot v3.2

Spring Cloud Stream v4.1

Core

Auto-configuration

Pulsar Binder

Spring Pulsar v1.0

Spring

Pulsar

29 of 35

Project evolution

Spring

Boot

Spring �Cloud Stream

Spring Boot v3.2

Spring Cloud Stream v4.1

Core

Auto-configuration

Pulsar Binder

Spring Pulsar v1.0

Spring

Pulsar

30 of 35

What about GraalVM?…

04

31 of 35

The Q + A …

05

32 of 35

Appendix / Resources

33 of 35

Spring / Pulsar Resources

34 of 35

Pulsar / DataStax Resources

https://astra.datastax.com

https://www.datastax.com/products/astra-streaming

https://www.datastax.com/products/luna-streaming

CDC for Astra: https://docs.datastax.com/en/astra/docs/astream-cdc.html

34

https://pulsar.apache.org/

https://bookkeeper.apache.org/

https://zookeeper.apache.org

35 of 35

Thank You

Mary Grygleski

https://www.linkedin.com/in/mary-grygleski/

@mgrygles

https://discord.gg/RMU4Juw

Christophe Bornet

https://www.linkedin.com/in/christophe-bornet-bab1193/

@cbornet_

https://github.com/cbornet