Entry Slide With Personal Detail

StyleLounge

Me, Alexander Thiel

  • Price comparison for fashion
  • 2.5 years old
  • 3M visits in January
  • 6 countries (ch,de,fr,nl,se,us)
  • 30+ people
  • 10 devs

For team, photos and stack go to jobs.stylelounge.de

@alexanderthiel
me@alexthiel.de
github.com/reasn
linkedin.com/in/reasn
xing.com/profile/Alex_Thiel

  • Since I was a kid I love how code lets you make anything imaginable.
  • I manage @StyleLounge's great engineers.
  • We build reactive microservices at scale.

Migrating to
Reactive Microservices

Werde nicht darüber reden

Think different

Requirements I

Scalability

Maintainability

ACID
Eventual Consistency

Requirements II

  • Continuous integration
  • Automated testing
  • Infrastructure as code
  • Collective code ownership

Death to the Monolith

Starting Point

Core DB

Importer

Cockpit

Webapp

Scoring

Updater

BI

Webapp

Core DB

Importer

Cockpit

Updater

Webapp

Webapp

Scoring

BI

Webapp

Core DB

Importer

Cockpit

Updater

Webapp

Webapp

Scoring

BI

Microservices to the Rescue

Cockpit

Webapp

Scoring

Products

Tracking

Users

Data Responsibility Problems

Cockpit

Webapp

Scoring

Products

Tracking

Users

Scalability Problems

Cockpit

Webapp

Scoring

Products

Tracking

Cockpit

Cockpit

Webapp

Webapp

Users

Users

Users

Products

Products

Resilience Problems

Cockpit

Webapp

Scoring

Products

Tracking

Cockpit

Cockpit

Webapp

Webapp

Users

Users

Users

Products

Products

Problems with Classic Microservices

  • Difficult data responsibilities
  • Load cascades into the system
  • Errors cascade to the user

These problems are amplified with additional service edges

Message-Driven to the Rescue

Webapp

Scoring

Products

Tracking

Cockpit

Webapp

Webapp

Users

Event Sourcing

Events ...

  • are the single source of truth
  • are stored forever
  • can be easily replayed

Benefits of Event Sourcing

  • Nothing overwritten is ever lost
  • Easily restore all microservices
  • Polyglot persistence is easy

Reactivity

The Reactive Manifesto

Reactive Software is

  • Responsive
  • Resilient
  • Elastic
  • Message Driven

Service Oriented
Architectures

Microservices

Reactive Micro
Services

Focus on Data Flow

B

A

C

D

A queries DB

DB

B polls A

B push-notifies C

D subscribes to C

Focus on Data Flow

B

A

C

D

A queries DB

DB

B polls A

B push-notifies C

D subscribes to C

Service Interactions

Webapp

Scoring

Products

Tracking

Users

Importer

Erklären, was Services tun

Als letztes: Webapp -> Users|Tracking

Service Interactions

Publish/Subscribe

Push Notifications

Webapp

Scoring

Products

Tracking

Users

Importer

Resilience

Webapp

Scoring

Products

Tracking

Users

Importer

Limitations

Making microservices reactive is harder if …

  • they have a less directed data flow
  • you require complex transactions
  • you need ACID

How to Start

Possible Starting Points

Start with jobs for …

… your first reactive microservice

  • Sending Emails
  • Scoring products
  • Writing tracking events to the database
  • Importing from / observing external systems
  • Mailing
  • Scoring
  • Tracking
  • Feed Importer

Interfacing With the Legacy System

Legacy Macroservice

Database

Interfacing With the Legacy System

New Process: Emitter

  • Triggered e.g. by cron
  • Scans through tables
  • Compares with cache
    (e.g. redis)
  • Publishes lifecycle events:
    New, Updated, Deleted

Legacy Macroservice

Emitter

Database

Interfacing With the Legacy System

Legacy Macroservice

Reactive Microservice

Emitter

Database

Interfacing With the Legacy System

Persister subscribes to microservice’s events and persists changes to database

Legacy Macroservice

Reactive Microservice

Emitter

Persister

Database

Microwave

Microwave at StyleLounge

  • Scores are calculated reactive

Scoring

Legacy Macroservice

Product Emitter

Score Persister

Database

Microwave at StyleLounge

Legacy Macroservice

Product Emitter

Database

Scoring

Webapp

Microwave at StyleLounge

Scoring

Webapp

Tracking

Product + X

Macroservice

Microwave at StyleLounge

Scoring

Webapp

Tracking

Product + X

Macroservice

Microwave at StyleLounge

  • Monolith gone
  • New User Service

Scoring

Webapp

Tracking

Bidding

Products

Users

Naming

Surprise, Surprise.

These are not individual processes.

Scoring

Webapp

Tracking

Bidding

Products

Users

Die Innere Ordnung von Parteien muss demokratischen Grundsätzen entsprechen.

Artikel 21 Grundgesetz

A Closer Look

Scoring

Webapp

Tracking

Bidding

Products

Users

  • boxes = subservices
  • most exist for each of our 6 countries

tracking
persister

calculator

product
persister

Redis

Hosted Google BigQuery

Elasticsearch Cluster

Scoring

Webapp

Tracking

Cache

We coined the term subservice. Happy to hear that something better exists

Darauf eingehen, was passiert, wenn diverse Services ausfallen

  • boxes = subservices
  • most exist for each of our 6 countries
  • some are clustered ( )

search-cache
persister

search-cache

endpoint

frontend

MailChimp

tracking
persister

calculator

product
persister

Redis

Hosted Google BigQuery

Elasticsearch Cluster

Scoring

Webapp

Tracking

Cache

We coined the term subservice. Happy to hear that something better exists

Darauf eingehen, was passiert, wenn diverse Services ausfallen

Scalability

  • Stateless services scale linear

tracking
persister

calculator

product
persister

calculator

calculator

Cache
Cluster

Scalability

  • Stateless services scale linear

Stateful services

  • persistence layer is the bottleneck
  • polyglot persistence makes this quite easy

product
persister

tracking
persister

Cache

tracking
persister

calculator

product
persister

calculator

calculator

Cache
Cluster

product
persister

tracking
persister

Scalability

  • Operations can be computational costly
    when parallelized with queues
  • Scaling purpose-optimized persistence
    in a small microservice is easy

Data Warehouse

How to Make BI Microservices

The data-warehouse microservice …

  • subscribes to all other microservices
  • persists entities from events into database
  • provides a combined view to BI

Create a new reactive microservice.

Wrap-Up

Wrap-Up

  • Message-driven architectures ...
    • simplify resilience
    • simplify scaling
    • are persistence-agnostic
  • Scaling persistence is simplified by
    • polyglot persistence
    • small services
  • Scaling computation is simplified by asynchronicity and queues
  • Feasibility is domain-dependent

Thank You.

Don’t Be Afraid.

  • We have 9 microservices (~ +1/month)
  • Each microservice consists of 2 to 7 subservices
  • We have about 170 services over our 6 countries
    (including DBs, caches, brokers)
  • Some subservices consist of only 3 classes
  • ~4 devs built these within 10 months
  • We spend little time on maintenance and bug chasing
  • We had no idea when we started
  • Invest in research, reading, understanding and discussions

Nichts sagen! Stehen lassen, während nach Fragen gefragt werden ;)

🤖 Just in case you actually read all the speaker notes: Why not head over to jobs.stylelounge.de and join the team? 👾

References & Further Reading

2017-01-31 Alexander Thiel - Migrating to Reactive Microservices - Google Slides