1 of 36

Istio to the rescue

Techorama 2021�Ricardo Castro

https://mccricardo.com

@mccricardo

2 of 36

Agenda

Microservices
Service Mesh
Istio

Traffic Management
Security
Observability

https://mccricardo.com

@mccricardo

3 of 36

The great transition

Big ball of things

Certainly, some storage

Small thingy

Flashy storage

The monolith

The microservices

https://mccricardo.com

@mccricardo

Describe monolith and breaking into microservices

Microservices because we’re in the era of digital transformation and in the pursuit to better serve our customers. Means faster development cycles as well as more and faster deploys.

---

In this era of digital transformation, we have embarked on a relentless pursuit of better serving customers and users.

Developers have not only evolved into faster development cycles based on Agile principles, but require vastly faster deployment times.

No longer the deployment (even if only by clicking a button or even automated) of a monolithic application once a month or even a week is acceptable.

Breaking up the application into smaller units with smaller team sizes, each with independent workflows, governance model and deployment pipeline is nowadays a standard practice.

The industry defined this approach as microservices architecture.

4 of 36

Why do we do it?

Maintainability and scalability
Loose coupling
Deployability independence
Organization around business capabilities
Small team ownership

https://mccricardo.com

@mccricardo

5 of 36

Is it all sunshine and rainbows?

Faster delivery
Isolation
Scaling
Culture
Flexibility

Service discovery
Load balancing
Fault tolerance
Distributed tracing
Metrics
Security

https://mccricardo.com

@mccricardo

6 of 36

Mesh what?

A mesh network is a local network topology in which the infrastructure nodes (i.e. bridges, switches and other infrastructure devices) connect directly, dynamically and non-hierarchically to as many other nodes as possible and cooperate with one another to efficiently route data from/to clients.

https://mccricardo.com

@mccricardo

7 of 36

Service Mesh

A service mesh is the connective tissue between our services that adds additional capabilities like traffic control, service discovery, load balancing, resilience, observability, security, and so on.

A service mesh allows applications to offload these capabilities from application-level libraries and allow developers to focus on differentiating business logic.

https://mccricardo.com

@mccricardo

8 of 36

Service Mesh

Service proxy

A service proxy is a proxy on which an application service relies for additional capabilities. The service calls through the service proxy any time it needs to communicate with the outside world (i.e., over the network). The proxy acts as an intermediary or interceptor that can add capabilities like automatic retries, timeouts, circuit breaker, service discovery, security, and more.

Sidecar

A service that is attached to a parent application and provides supporting features for the application. The sidecar also shares the same lifecycle as the parent application, being created and retired alongside the parent. The sidecar pattern is sometimes referred to as the sidekick pattern and is a decomposition pattern.

https://mccricardo.com

@mccricardo

9 of 36

Service Mesh

Image source: https://bit.ly/2HczDpq

https://mccricardo.com

@mccricardo

10 of 36

Istio

Istio provides behavioral insights and operational control over the service mesh as a whole, offering a complete solution to satisfy the diverse requirements of microservice applications.

Istio makes it easy to create a network of deployed services with load balancing, service-to-service authentication, monitoring, and more, with few or no code changes in service code. We add Istio support to services by deploying a special sidecar proxy throughout our environment that intercepts all network communication between microservices, then configure and manage Istio using its control plane functionality.

Istio is designed for extensibility and meets diverse deployment needs.

https://mccricardo.com

@mccricardo

11 of 36

Istio Architecture

Image source: https://bit.ly/3efLlhI

https://mccricardo.com

@mccricardo

12 of 36

Envoy

High-performance proxy developed in C++ to mediate all inbound and outbound traffic for all services in the service mesh.

It’s deployed as a sidecar to the relevant service in the same Kubernetes pod. This deployment allows Istio to extract a wealth of signals about traffic behavior as attributes. Istio can, in turn, use these attributes to enforce policy decisions, and send them to monitoring systems to provide information about the behavior of the entire mesh.

The sidecar proxy model also allows us to add Istio capabilities to an existing deployment with no need to rearchitect or rewrite code.

https://mccricardo.com

@mccricardo

13 of 36

Istiod

Istio control plane.

Istiod provides service discovery, configuration and certificate management.

Istiod converts high level routing rules that control traffic behavior into Envoy-specific configurations, and propagates them to the sidecars at runtime.

https://mccricardo.com

@mccricardo

It’s the control plan. Provides discovery, configuration and certificate management.

Converts high rules into something Envoy understands.

Interact with with it via Management API.

Possesses security features for service-to service and end-user authentication and authorization.

Can act as a CA and generate certificates for mTLS.

---

Istiod is Istio’s control plane.

Istiod provides service discovery, configuration and certificate management. It converts high level routing rules that control traffic behavior into Envoy-specific configurations, and propagates them to the sidecars at runtime.

Istio's Management API can be used to instruct Istiod to refine Envoy configuration to use more granular control over our service mesh.

Istiod possesses security features that enable strong service-to-service and end-user authentication along with built-on identity and credential management. We can leverage Istio to upgrade traffic in the service mesh.

By making use of Istio, operators can enforce policies based on service identity rather than on relatively unstable layer 3 or layer 4 network identifiers as well as using Istio's authorization feature to control who can access your services.

Istiod can act as a Certificate Authority (CA) and generate certificates to allow secure mTLS communication in the data plane.

14 of 36

Traffic Management

Istio has support for traffic rules that allow us to easily control the flow of traffic and API requests between services.

With these type of features there are some traffic-control patterns that can be taken advantage like, canary deployments, dark launches or A/B tests.

Big internet companies like Netflix, Amazon, or Facebook use these patterns frequently.

Concepts: virtual services, destination rules, gateways, service Entries, and network resilience and fault injection.

https://mccricardo.com

@mccricardo

Traffic rules to control traffic between services. Makes our lives easier (e.g. timeouts, retries, circuit breaking, A/B tests, canary rollouts, etc).

Big internet companies like Netflix, Amazon, or Facebook, just to mention a few examples, use these patterns frequently.

Out-of-box failure recovery features will make our services more robust against failures of dependent services or the network.

---

Istio has support for traffic rules that allow us to easily control the flow of traffic and API requests between services. This will make our lives easier when we need to configure service-level properties like

timeouts, retries and circuit breakers and also allows us to perform things like A/B tests, canary rollouts, or staged rollouts with percentage-based traffic splits.

Big internet companies like Netflix, Amazon, or Facebook, just to mention a few examples, use these patterns frequently.

Also, it’s out-of-box failure recovery features will make our services more robust against failures of dependent services or the network.

15 of 36

Traffic Management: Virtual Service

https://mccricardo.com

@mccricardo

How requests are routed to a service within the service mesh.

Set of rules evaluated in order allowing istio to match each given request to it’s real destination

(Going through the code)

Hosts - user-addressable destinations. the addresses the client uses when sending requests to the service

HTTP - virtual service’s routing rules, describing match conditions and actions for routing. A routing rule consists of the destination where we want the traffic to go and zero or more match conditions, depending on use case. (IN the example) the first routing rule has a condition and so it begins with the match field. In this case we want this routing to apply to all requests from the user “mccricardo”, so we use the header, end-user, and exact fields to select the appropriate requests.

Route - field specifies the actual destination for traffic that matches this condition and it has to be real destination that exists in Istio’s service registry or Envoy won’t know where to send traffic. Can have rule at the end.

---

In Istio, a virtual service lets us configure how requests are routed to a service within the service mesh, building on the basic connectivity and discovery provided by Istio and our platform.

Each virtual service consists of a set of routing rules that are evaluated in order, letting Istio match each given request to the virtual service to a specific real destination within the mesh.

(Going through the code)

The hosts field lists the virtual service’s hosts - meaning, the user-addressable destinations that these routing rules apply to. These are the addresses the client uses when sending requests to the service.

The http section contains the virtual service’s routing rules, describing match conditions and actions for routing HTTP/1.1, HTTP2, and gRPC traffic sent to the destination(s) specified in the hosts field. A routing rule consists of the destination where we want the traffic to go and zero or more match conditions, depending on use case

In this example, the first routing rule has a condition and so it begins with the match field. In this case we want this routing to apply to all requests from the user “mccricardo”, so we use the header, end-user, and exact fields to select the appropriate requests.

The route section’s destination field specifies the actual destination for traffic that matches this condition and it has to be real destination that exists in Istio’s service registry or Envoy won’t know where to send traffic. This can be a mesh service with proxies or a non-mesh service added using a service entry. In this case we’re running on Kubernetes and the host name is a Kubernetes service name.

Routing rules are evaluated in sequential order from top to bottom. In the example we want anything that doesn’t match the first routing rule to go to a default destination, specified in the another rule.

16 of 36

Traffic Management: Virtual Service

https://mccricardo.com

@mccricardo

Can have multiple match conditions or multiple match blocks making it as complex as we want.

We can distribute traffic by percentage “weight. (show example)

We can also use routing rules to perform some actions on the traffic, for example, append or remove headers or rewrite the URL.

---

As we’ve seen before, we can add multiple match conditions to the same match block or add multiple match blocks to the same rule. This lets us make your routing conditions as complex or simple as we like within a single virtual service.

In addition to using match conditions, we can distribute traffic by percentage “weight”. This is useful for A/B testing and canary rollouts, for example.

We can also use routing rules to perform some actions on the traffic, for example, append or remove headers or rewrite the URL.

17 of 36

Traffic Management: Destination Rule

https://mccricardo.com

@mccricardo

Virtual services -how we route your traffic to a given destination, destination rules - what happens to traffic for that destination. Destination rules are applied after virtual service routing rules are evaluated, so they apply to the traffic’s “real” destination.

Use destination rules to specify named service subsets, for example as grouping all a given service’s instances by version that can be used in the routing rules of virtual services.

Load balancing - default round-robin. Can also be: random, weighted or least requests.

Each subset is defined based on one or more labels, which in Kubernetes are key/value pairs that are attached to objects such as Pods.

(Looking at the example)

simple random load balancer for the v1 and v3 subsets, v3 also being the default

v2 policy, a round-robin load balancer

---

We can think of virtual services as how we route your traffic to a given destination, and then we use destination rules to configure what happens to traffic for that destination. Destination rules are applied after virtual service routing rules are evaluated, so they apply to the traffic’s “real” destination.

In particular, we use destination rules to specify named service subsets, such as grouping all a given service’s instances by version that can be used in the routing rules of virtual services to control the traffic to different instances of your services.

By default, Istio uses a round-robin load balancing policy. Istio also supports the following load balancing models:

- Random: requests are forwarded at random to instances in the pool.

- Weighted: requests are forwarded to instances in the pool according to a specific percentage.

- Least requests: requests are forwarded to instances with the least number of requests.

Each subset is defined based on one or more labels, which in Kubernetes are key/value pairs that are attached to objects such as Pods.

(Looking at the example)

As well as defining subsets, this destination rule has both a default traffic policy for all subsets in this destination and a subset-specific policy that overrides it for just that subset.

In the example, the default policy subsets field sets a simple random load balancer for the v1 and v3 subsets while the in the v2 policy, a round-robin load balancer is specified in the corresponding subset’s field.

18 of 36

Traffic Management: Gateway

https://mccricardo.com

@mccricardo

Used to manage inbound and outbound traffic for our mesh. Applied to standalone Envoy proxies that are running at the edge of the mesh.

Primarily used to manage ingress traffic, can be used for egress traffic. Lets us configure a dedicated exit node for the traffic leaving the mesh,limiting which services can or should access external networks, or to enable secure control of egress traffic to add security to our mesh.

(Using the example)

Lets HTTPS traffic from mccricardo.com into the mesh on port 443, but doesn’t specify any routing for the traffic.

To specify routing and for the gateway to work as intended, we must also bind the gateway to a virtual service. We do this using the virtual service’s gateways field.

We can then configure the virtual service with routing rules for the external traffic.

---

Gateways are used to manage inbound and outbound traffic for our mesh, letting us specify which traffic we want to enter or leave the mesh. Gateway configurations are applied to standalone Envoy proxies that are running at the edge of the mesh, rather than sidecar Envoy proxies running alongside your service workloads.

Gateways are primarily used to manage ingress traffic, but we can also configure egress gateways. An egress gateway lets us configure a dedicated exit node for the traffic leaving the mesh, letting us limit which services can or should access external networks, or to enable secure control of egress traffic to add security to our mesh, for example.

(Using the example)

This gateway configuration lets HTTPS traffic from mccricardo.com into the mesh on port 443, but doesn’t specify any routing for the traffic.

To specify routing and for the gateway to work as intended, we must also bind the gateway to a virtual service. We do this using the virtual service’s gateways field.

We can then configure the virtual service with routing rules for the external traffic.

19 of 36

Traffic Management: Service Entry

https://mccricardo.com

@mccricardo

Used o add an entry to the service registry that Istio maintains internally. That means that after we add the service entry, the Envoy proxies can send traffic to the service as if it was a service in your mesh.

Configuring service entries allows us to manage traffic for services running outside of the mesh, and treat them as they were part of the mesh (e.g timeout, retires, fault injection, etc).

We can also add services from a different cluster to the mesh to configure a multicluster Istio mesh on Kubernetes.

We don’t need to add a service entry for every external service that you want your mesh services to use because, by default, Istio configures the Envoy proxies to passthrough requests to unknown services. However, we can’t use Istio features to control the traffic to destinations that aren’t registered in the mesh.

For example, this destination rule config is enforcing the use of mTLS to a service defined as a ServiceEntry that is external do the mesh.

20 of 36

Traffic Management: Network Resilience

https://mccricardo.com

@mccricardo

21 of 36

Traffic Management: Fault Injection

https://mccricardo.com

@mccricardo

22 of 36

Security

Image source: https://bit.ly/3nPo0GO

https://mccricardo.com

@mccricardo

Breaking down a monolithic application into atomic services offers various benefits, including better agility, better scalability and better ability to reuse services. However, microservices also have particular security needs:

To defend against man-in-the-middle attacks - traffic encryption.
To provide flexible service access control - mutual TLS and fine-grained access policies.
To determine who did what at what time - auditing tools.

Istio Security provides a comprehensive security solution to solve these issues. In particular, Istio security mitigates both insider and external threats against your data, endpoints, communication, and platform.

The Istio security features provide strong identity, powerful policy, transparent TLS encryption, and authentication, authorization and audit (AAA) tools to protect your services and data. �

(In necessary)�The goals of Istio security are:

Security by default: no changes needed to application code and infrastructure
Defense in depth: integrate with existing security systems to provide multiple layers of defense
Zero-trust network: build security solutions on distrusted networks

23 of 36

Security: Architecture

Image source: https://bit.ly/3nPo0GO

https://mccricardo.com

@mccricardo

24 of 36

Security: Authentication

Istio provides two types of authentication:

Peer authentication
Request authentication

https://mccricardo.com

@mccricardo

Istio provides two types of authentication:

Peer authentication: used for service-to-service authentication to verify the client making the connection. Istio offers mutual TLS as a full stack solution for transport authentication, which can be enabled without requiring service code changes. ��(If necessary: This solution:

Provides each service with a strong identity representing its role to enable interoperability across clusters and clouds.
Secures service-to-service communication.
Provides a key management system to automate key and certificate generation, distribution, and rotation.)�

Request authentication: Used for end-user authentication to verify the credential attached to the request. Istio enables request-level authentication with JSON Web Token (JWT) validation and a streamlined developer experience using a custom authentication provider or any OpenID Connect providers, for example:

In all cases, Istio stores the authentication policies in the Istio config store via a custom Kubernetes API. Istiod keeps them up-to-date for each proxy, along with the keys where appropriate. Additionally, Istio supports authentication in permissive mode to help you understand how a policy change can affect your security posture before it is enforced.

25 of 36

Security: Peer Authentication

https://mccricardo.com

@mccricardo

26 of 36

Security: Request Authentication

https://mccricardo.com

@mccricardo

27 of 36

Security: Authorization

Image source: https://bit.ly/3xKhd5I

https://mccricardo.com

@mccricardo

Istio’s authorization features provide mesh-, namespace-, and workload-wide access control for your workloads in the mesh.

(If necessary: This level of control provides the following benefits:

Workload-to-workload and end-user-to-workload authorization.
A Simple API: it includes a single AuthorizationPolicy CRD, which is easy to use and maintain.
Flexible semantics: operators can define custom conditions on Istio attributes, and use DENY and ALLOW actions.
High performance: Istio authorization is enforced natively on Envoy.
High compatibility: supports gRPC, HTTP, HTTPS and HTTP2 natively, as well as any plain TCP protocols.)

The authorization policy enforces access control to the inbound traffic in the server side Envoy proxy. Each Envoy proxy runs an authorization engine that authorizes requests at runtime. When a request comes to the proxy, the authorization engine evaluates the request context against the current authorization policies, and returns the authorization result, either ALLOW or DENY.

We don’t need to explicitly enable Istio’s authorization features. Just apply an authorization policy to the workloads to enforce access control. For workloads without authorization policies applied, Istio doesn’t enforce access control allowing all requests.

28 of 36

Security: Authorization Policies

https://mccricardo.com

@mccricardo

29 of 36

Security: Authorization Policies

https://mccricardo.com

@mccricardo

30 of 36

Security: Authorization Policies

https://mccricardo.com

@mccricardo

31 of 36

Observability

Istio generates detailed telemetry for all service communications within a mesh.

It provides observability of service behavior, empowering operators to troubleshoot, maintain, and optimize their applications – without imposing any additional burdens on service developers.

Operators gain a deeper understanding of how services are interacting, both with other services and with the Istio components themselves.

https://mccricardo.com

@mccricardo

Istio generates the following types of telemetry in order to provide overall service mesh observability:

Metrics. Istio generates a set of service metrics based on the four “golden signals” of monitoring (latency, traffic, errors, and saturation). Istio also provides detailed metrics for the mesh control plane. A default set of mesh monitoring dashboards built on top of these metrics is also provided.
Distributed Traces. Istio generates distributed trace spans for each service, providing operators with a detailed understanding of call flows and service dependencies within a mesh.
Access Logs. As traffic flows into a service within a mesh, Istio can generate a full record of each request, including source and destination metadata. This information enables operators to audit service behavior down to the individual workload instance level.

32 of 36

Observability: Metrics

Types of metrics:

Proxy-level
Service-level
Control plane

https://mccricardo.com

@mccricardo

Metrics provide a way of monitoring and understanding behavior in aggregate.

To monitor service behavior, Istio generates metrics for all service traffic in, out, and within an Istio service mesh. These metrics provide information on behaviors such as the overall volume of traffic, the error rates within the traffic, and the response times for requests.

In addition to monitoring the behavior of services within a mesh, it is also important to monitor the behavior of the mesh itself. Istio components export metrics on their own internal behaviors to provide insight on the health and function of the mesh control plane.

Istio metrics collection begins with the sidecar proxies (Envoy), the proxy-level metrics. Each proxy generates a rich set of metrics about all traffic passing through the proxy (both inbound and outbound). The proxies also provide detailed statistics about the administrative functions of the proxy itself, including configuration and health information.

Istio provides a set of service-oriented metrics for monitoring service communications. These metrics cover the four basic service monitoring needs: latency, traffic, errors, and saturation.

The Istio control plane also provides a collection of self-monitoring metrics. These metrics allow monitoring of the behavior of Istio itself (as distinct from that of the services within the mesh).

33 of 36

Observability: Distributed traces

Image source: https://bit.ly/3efAd4c

https://mccricardo.com

@mccricardo

Distributed tracing provides a way to monitor and understand behavior by monitoring individual requests as they flow through a mesh. This feature empowers operators to understand service dependencies and the sources of latency within their service mesh.

Istio supports distributed tracing through the Envoy proxies. The proxies automatically generate trace spans on behalf of the applications they proxy, requiring only that the applications forward the appropriate request context.

Istio supports a number of tracing backends, including Zipkin, Jaeger, Lightstep, and Datadog. Operators control the sampling rate for trace generation (that is, the rate at which tracing data is generated per request). This allows operators to control the amount and rate of tracing data being produced for their mesh.

Example of Istio with Zipkin.

34 of 36

Observability: Logs

https://mccricardo.com

@mccricardo

35 of 36

And much more...

Multicluster mesh
Istio CNI plugin
VM: service credential distribution
WebAssembly extension
Request classification
Custom Envoy filters
DNS proxying
...

https://mccricardo.com

@mccricardo

36 of 36

Thank you!

Senior Site Reliability Engineer at Farfetch (We’re hiring)
DevOps Porto meetup co-organizer
DevOpsDays Portugal conference co-organizer
CKAD and CKA by the CNCF
Strong believer in culture and teamwork
Open source passionate (former Mozilla contributor)
Taekwondo amateur
Metal lover
Where to find me:

https://mccricardo.com
@mccricardo
Linkedin: Ricardo Castro

https://mccricardo.com

@mccricardo