Reactive Microservices

​

Matt Clement - @mallize

​

​

​

LifeWay IT

tech.lifeway.com @lifewaytech

​

Reactive Microservices

​

​

Matt Clement - @mallize

LifeWay IT - tech.lifeway.com - @lifewaytech

LifeWay

• Much more than a bookstore
• 150 employees in IT alone
• Everything from back-office applications, big data, consumer applications, mobile applications, etc.
• A lot of legacy systems… a lot

​

​

LifeWay

​

​

​

Evolutionary Architecture

• Modularity and Loose Coupling
• Separate along well defined boundaries
• Not a ‘big ball of mud’
• Organized around Business Capabilities
• Domain Driven Design
• Supports Experimentation
• Canary releases
• A/B testing
• Bring the Pain Forward
• Continuous Delivery (automate everything)
• Last Responsible Moment
• Allow the architecture to evolve

https://www.thoughtworks.com/insights/blog/microservices-evolutionary-architecture

​

Evolutionary Architecture

Microservices

&

Reactive Programming

​

Microservices

Microservices vs Monolithic

• Component types
• Client-side user interface
• Database (often relational)
• Server-side business logic
• Pros
• Simple Deployment
• Run entire stack on a dev machine
• Cons
• Modules are deployed together (tight coupling)
• Leaky abstractions
• Vertical scaling only
• Monoliths of monoliths

​

​

Microservices vs Service Oriented (SOA)

• Component Types
• Providers
• Consumers
• Transport/Translation (Enterprise Service Bus, ESB)
• Pros
• ‘Kind of’ Loosely Coupled
• Reusability
• Independent Scalability
• Cons
• Shared Data Model
• Often used shared physical DB
• Often deployed as distributed monoliths
• Easy to have cascading failures

​

​

Microservices

​

Breaking down an application into a suite of small, well-defined services that are built around business capabilities, independently deployable and do not share their data model.

​

Built around business capabilities

​

• Domain Driven Design is an architectural strategy focusing on the core domain of system or subsystem.
• A domain is considered a bounded context. The internal model and behavior are not exposed.
• Context maps and anti-corruption layers help keep the domains pure while allowing collaboration.
• Ubiquitous Language - The shared language for business and code within a bounded context

Domain Driven Design

Independently Deployable

• Each microservice should be able to deploy to production independent of other services
• Continuous Integration, Continuous Delivery and Continuous Deployment
• Backwards compatibility is a must
• CDD - Consumer Driven Contracts help identify breaking changes before deploying.

Share Nothing

SOA is built on the concept of a share-as-much- as-possible architecture style, whereas microservices architecture is built on the concept of a share-as-little-as-possible architecture style.

~Mark Richards

• Reuse through creating enterprise capabilities
• Microservices have their own database(s)
• Document store for views
• Graph for searching

​

Microservices Suggestions

API Gateways for client applications

​

​

Microservices Suggestions

Create facades in front of capabilities.

Allows for evolutionary architecture.

​

​

Microservices Suggestions

Create 12 factor apps (https://12factor.net/)

​

​

​

Microservices Prerequisites

• DevOps mentality
• Designing microservices goes beyond code. You have to consider the environment that you are deploying to. How do you leverage the capabilities of your cloud provider?
• Service Discovery, HA, Security, etc
• Continuous Delivery pipeline
• Monitoring and Log aggregation

​

Microservices Prerequisites - Build Pipeline

Each microservice should have it's own build pipeline in GoCD. An example build pipeline is shown in the diagram below. Variations of this pipeline will exist for different services but the required pipeline steps are:

• Unit Tests
• Build Artifact
• Automated deploy to an dev/ci environment
• Functional tests (full regression can happen in a separate build step if needed)
• Automated Stage deploy
• Load tests
• Consumer-Driven Contract tests (1-N)

​

​

Microservices

​

Breaking down an application into a suite of small, well-defined services that are built around business capabilities, independently deployable and do not share their data model.

​

Reactive Programming

http://www.reactivemanifesto.org

Responsive

• The system responds in a timely manner if at all possible.
• Problems may be detected quickly and dealt with effectively.
• Rapid and consistent response times, establishing reliable upper bounds so they deliver a consistent quality of service.
• This consistent behaviour in turn simplifies error handling, builds end user confidence, and encourages further interaction.

Resilient

• Stays responsive in the face of failure.
• Resilience is achieved by replication, containment, isolation and delegation.
• Failures are contained within each component, isolating components from each other and thereby ensuring that parts of the system can fail and recover without compromising the system as a whole.
• Recovery of each component is delegated to another (external) component and high-availability is ensured by replication where necessary.
• The client of a component is not burdened with handling its failures.

Elastic

• Stays responsive under varying workload.
• Can react to changes in the input rate by increasing or decreasing the resources.
• No contention points or central bottlenecks, resulting in the ability to shard or replicate components.
• Support predictive, as well as Reactive, scaling algorithms by providing relevant live performance measures.
• Achieve elasticity in a cost-effective way on commodity hardware and software platforms.

Message Driven

• Rely on asynchronous message-passing
• Enables load management, elasticity, and flow control by shaping and monitoring the message queues in the system and applying back-pressure when necessary.
• Location transparency
• Non-blocking communication allows recipients to only consume resources while active, leading to less system overhead.

Reactive Microservices

Jonas Bonér lists the following as characteristics of Reactive Microservices

• Isolation
• Act Autonomously
• Do one thing and do it well
• Own your state
• Embrace Asynchronous Message-Passing
• Stay Mobile, but Addressable

Isolation

Isolation has a huge impact on design and should be considered at the start of designing a system.

Conway’s Law

"Any organization that designs a system (defined more broadly here than just information systems) will inevitably produce a design whose structure is a copy of the organization's communication structure." ~ Melvin Conway

Good or bad is a matter of perspective. Create teams around suites of services or capabilities.

​

Isolation

Isolation allows us to reduce the blast radius of errors. We don’t want cascading failures to take down our system.

​

​

​

Isolation

If service E goes down...

​

​

​

Isolation

We don’t want all other services to be affected

​

​

​

Isolation

Similar to ship bulkheading

​

​

​

Isolation

Similar to ship bulkheading

​

​

​

Isolation - How?

Circuit Breaker Pattern

​

​

​

Isolation - Circuit Breaker Pattern

A degraded experience is better than no experience.

​

​

​

Isolation

In a highly distributed system, synchronous API calls are dangerous.

​

​

​

​

​

Isolation

Temporally decouple components through message driven communication.

Message Driven at LifeWay

• Enterprise Events - Streaming
• Used to communicate between bounded contexts
• Built on Kafka
• Domains Subscribe to Events
• Temporally decouple Microservices / Bounded Contexts
• CQRS
• Command Query Responsibility Segregation
• Different Read model from Write model
• We use event sourcing for our writes
• Several read models possible
• Denormalized views
• Search indexes
• Commands mutate state and do not return data
• Queries return data but do not mutate state

Message Driven - CQRS

​

“CQRS is simply the creation of two objects where there was previously only one. The separation occurs based upon whether the methods are a command or a query (the same definition that is used by Meyer in Command and Query Separation: a command is any method that mutates state and a query is any method that returns a value). —Greg Young

Message Driven - CQRS

Reason for segregation

• Large imbalance of reads and writes
• Command side involves complex business logic
• Read side should be optimized for reads
• Write side should be optimized for writes
• One model does neither very well

Message Driven - CQRS

• Traditional read/write domain models have an impedance mismatch between the persistence and actual model.
• ORM (object relational mapping) tools try to ease this pain, but can become very complex.
• A SQL database in 5th normal form doesn’t map to a flat domain model.

Message Driven - CQRS - Event Sourcing

​

Event Sourcing - Storing changes as a set of event logs rather than just the current state. The current state is derived from replaying the events.

​

Message Driven - CQRS - Event Sourcing

Benefits of using event sourcing for the write side

• You can reconstruct the state of your domain at any point in history
• Events are immutable ‘facts’. You can not change them. You can only compensate for errors (much like real life communication).
• Free audit log
• Events can be forwarded so that 1-N event handlers can take action on the events
• The most common is a view handler which creates a denormalized view of the ‘current state’.
• You can create new views that business didn’t know they wanted.

​

CQRS

CQRS

Event Log Current State

CQRS end to end demo

https://www.smallgroup.com

Procedural REST Calls

Async Message Passing (Events)

What about transactions?

Saga - A finite state machine that’s state transitions are based on the events pertaining to a certain business process. Errors captured include invalid state transitions, timeouts and errors in messaging when orchestrating API calls.

Compensating events can be used to undo portions of a saga (similar to rolling back).

Message Driven

• Embrace Eventual Consistency
• Matter of ms normally
• Create SLAs
• Not always the best fit; stock trader apps
• ATM Machines
• Airlines
• Resilient publishers
• Resilient consumers
• Pull model
• Back pressure

Physical Architecture

• Each service is deployed on separate isolated instances
• Each service auto scales based on its load across multiple AZ’s.
• Each service has in internal ELB for managing that service’s traffic

Scaling under 3x avg load

Latency under 3x load

Summary

Microservices provide the means for creating autonomous, single responsibility services that can provide quick turn around for a company's feature sets. Because of their limited scope of responsibility they are easy to implement, change and maintain.

Complexity is contextual. The complexity in microservices moves out of the services themselves and into their deployment strategy, communication patterns and HA strategy.

Reactive Programming provides guidelines for creating systems that are designed to handle failure, load and asynchronous communication.

​

​

Questions?

Twitter:

@mallize

@lifewaytech

​

GitHub: github.com/LifeWay

​

More Info: http://tech.lifeway.com/more-info/