Everything you need to know about Kafka Streams

An easy-to-follow Kafka Streams workshopP

Antón Rodríguez Yuste - @antonmry

Data Engineer @ inditex.com

Agenda

Some key points

1. Introduction to Kafka - 15 min
2. Exercise 0: KStreams - 15 min
3. Exercise 1: topology - 15 min
4. Exercise 2: branch - 15 min (1 hour)
5. Exercise 3: state store - 15 min
6. Exercise 4: joins - 15 min
7. Exercise 5: KTables - 15 min

​

Let’s start

Download the code and check it

​

https://github.com/antonmry/kafka-streams-workshop

​

Linux / Mac:

./gradlew test --tests KafkaStreamsIntegrationTest

Windows:

gradlew.bat test --tests KafkaStreamsIntegrationTest

​

​

Goals

​

• Understand what you can do with Kafka Streams

​

• Have a minimal practical base

https://github.com/antonmry/kafka-streams-workshop

What’s Kafka?

​

• Kafka is like REST but asynchronous?

​

• Kafka is like a service bus?
• Kafka is like a database?

Designing Event-Driven Systems

https://www.confluent.io/designing-event-driven-systems

What’s Kafka

Streams?

https://kafka.apache.org/documentation/streams/

Let’s code!

KSQL

SQL with streams!

https://www.confluent.io/blog/ksql-streaming-sql-for-apache-kafka/

CREATE STREAM vip_users AS

SELECT userid, page, action

FROM clickstream c

LEFT JOIN users u ON c.userid = u.user_id

WHERE u.level = 'Platinum';

Monitoring

and debugging

​

• Interceptors

​

• Metrics via JMX
• Topology.describe()

​

Takeaways

Thanks!

@antonmry

anton@galiglobal.com

Feel free to reach out!

Networking

🍻

galiglobal.com

Source code: https://github.com/antonmry/kafka-streams-workshop/