1 of 28

Terraform providers for Cloud Foundry

Guillaume Berche & Mevan Samaratunga

@gberche

@mevansam

1

Latest slides: https://goo.gl/5GRV1t

2 of 28

Team introduction

Guillaume Berche �@gberche, Orange�product management

Arthur Halet �Orange, �development

Mevan Samaratunga �@mevansam, Pivotal development

Xavier Marcelet�Orange, �development

Samed Guner @samedguener, SAP �development

Janos Binder �SAP,�development

Jim Carrothers�SAP, �development

Xilin Li�Sap, �development

2

3 of 28

Agenda

Why terraform for cloudfoundry ? (Use cases)

For admin/operators
For app developers

How is terraform helping ?

Reminder about Terraform model/syntax and their benefits
Getting started demo
Sample TF configs addressing use-cases

What is terraform for cloudfoundry (Implementation) ?

History of the providers and community convergence
Details of each contributed provider and their status
Backlog, challenges & future work

3

4 of 28

Why terraform for CF? CF admins use cases

4

#! /bin/bash

$ cf create-

5 of 28

Why terraform for CF? Developers use cases

Provision CF resources not covered by CF app manifest

Space
Space user role
Network policies
Service instances / UPS
Space-scoped service brokers

Deploy a µs-based application from CI/CD on multiple CF deployments

Download app binaries/sources
Handle potential dependencies among apps
Lookup domains on each CF deployment

Cross reference resources from credhub, CFAR, CFCR

5

6 of 28

Agenda

Why terraform for cloudfoundry ? (Use cases)

For admin/operators
For app developers

How is terraform helping ?

Reminder about Terraform model/syntax and their benefits
Getting started demo
Sample TF configs addressing use-cases

What is terraform for cloudfoundry (Implementation) ?

History of the providers and community convergence
Details of each contributed provider and their status
Backlog, challenges & future work

6

7 of 28

How is terraform helping ?

Terraform (https://www.terraform.io /) “enables you to safely and predictably create, change, and improve your production platforms and apps. It is an open source tool that codifies APIs into declarative configuration files that can be shared amongst team members, treated as code, edited, reviewed, and versioned.”

7

8 of 28

8

9 of 28

9

UAA

10 of 28

10

11 of 28

Getting started demo

Set Up

Download terraform
Download cloudfoundry provider
Initialize cloudfoundry provider (provider cloudfoundry)

Write

Configure IDE for HCL support and cloudfoundry provider grammar
Look up an existing org (datasource cloudfoundry_org)
Create a space (resource cloudfoundry_space)
Create a service instance (resource cloudfoundry_service_instance)
Push an app from a github release (resource cloudfoundry_app)

Plan

Review changes

Create

11

https://github.com/gberche-orange/terraform-cloudfoundry-demo

12 of 28

Demo

Short demo (9 mins)

Long demo (28 mins)

12

13 of 28

Admin case study: ASG, isolation segment

13

space:

space1

organization:

public

isolation-segment:

public

asg:

dns

asg:

orange-dns

default-running-asgs

default-staging-asgs

IS Entitled to org

Space

assigned

to IS

Includes

cloudfoundry

uaa

credhub

Provider color code:

Includes

14 of 28

14

15 of 28

Admin case study: credhub, uaa, cf ar

15

app: cf-wall

Mapped to route �(as a route service)

Bound to app

References data from

user-provided- service:�cf-wall-smtp

user-provided- service:�cf-wall-config

uaa-client:�cf-wall

credhub-user:�cf-wall

route: cf-wall

service-instance:�Gobis �(OAuth gateway)

cloudfoundry

uaa

credhub

Provider color code:

16 of 28

16

17 of 28

Admin case study: Cloudflare & CFAR domains

17

cloudfoundry

cloudflare

Provider color code:

app: �myapp

route:

myapp.domain.com �(host=””)

domain:�myapp.domain.com

cloudflare_record:�myapp.domain.com

References data from

18 of 28

Admin case study: Cloudflare & CFAR domains

18

TF on-demand invocation through a service broker: orange-cloudfoundry/cf-ops-automation-broker

19 of 28

Developer case study 1: CFAR, CFCR

19

Collection of Spring based Microservices to track and manage vehicle fleets at a telco.

Originally built and run on Docker.

Migrated Spring Microservices to CFCR

Migrated CQRS event back-bone to CFCR

Terraform used to wire the architecture together and deploy continuously in an idempotent manner

I will grab the baton from here to briefly showcase a few application use cases.

The first use case describes a solution which we piloted for a telco in the Middle East.

You can see that in this solution we had a microservices landscape that worked well in CFCR.

These services depended on a set of backing services that had an architecture that was more suited for Kubernetes.

For this pilot we deployed CFCR - Kubo at the time - alongside their production CFAR environment.

We used the Kubernetes Terraform provider to declare resources in CFCR.

Then we used the services and application resources of the CF Terraform provider to wire them as user-provided services to services deployed to CFCR.

The goal of this pilot was to demonstrate how they could consistently deliver this system to their customers and eliminate the manual toil that was evident in the original docker based solution.

20 of 28

Architecture overview of SAP Leonardo Machine Learning Foundation

20

Developer case study 2: CFAR, CFCR

Here you can see the high-level architecture of the SAP Leonardo Machine Learning Foundation.

On the top part you can see a few CF-based microservices.

On the compute tier, there are some Kubernetes-based services (mostly services, which needs GPU support).

Last on the persistence tier, there are some services, which are running natively on the cloud, e.g. AWS S3 storage.

As you can see SAP needed to deploy microservices over 3 different landscapes, which was only possible, with terraform and its plugins.

To do this they needed to use the CF provider’s application and service resources along with resources from the Kubernetes and AWS providers.

They also had to extend the application and service resources to support certain features which were missing in the previous version of the CF provider.

21 of 28

Integrating TF in CI/CD 1/2

ljfranklin/terraform-resource/

21

orange-cloudfoundry/cf-ops-automation

orange-cloudfoundry/terraform-secure-backend

22 of 28

Integrating TF in CI/CD 2/2: SAP experience

22

Get credentials & tf state file from Vault

Run aliveness & integration tests

Apply changes using tf

Store changed tf state in Vault

This Jenkins pipeline shows how the SAP Machine Learning Foundation uses Terraform to deploy new releases of services across Cloud Foundry and Kubernetes.

(If someone asks, Kubernetes is needed for long running jobs and for GPU acceleration).

There are four steps in this pipeline.

In the first step the credential a retrieved from Hashicorp vault (this includes the terraform state file).

In the terraform apply step, terraform discovers, which applications have been updated and deploys them, and it also configures any required dependencies.

Then they run aliveness and integration tests against the functional services to see whether the integrity of the landscape has been kept.

At the last step the changed terraform state file is persisted to Vault using the Terraform Vault backend, as it contains sensitive credentials

I will now hand it back to Guillaume to talk about the past and current state of the development of the CF provider.

23 of 28

Agenda

Why terraform for cloudfoundry ? (Use cases)

For admin/operators
For app developers

How is terraform helping ?

Reminder about Terraform model/syntax and their benefits
Getting started demo
Sample TF configs addressing use-cases

What is terraform for cloudfoundry (Implementation) ?

History of the providers and community convergence
Details of each contributed provider and their status
Backlog, challenges & future work

23

24 of 28

History of the CFAR providers and community convergence

24

orange-cloudfoundry/

terraform-provider-cloudfoundry

mevansam /

terraform-provider-cf

2019

2018

2017

Converged efforts

terraform-providers/

terraform-provider-cloudfoundry

Official release

25 of 28

Details of each contributed CFAR providers and their status

25

orange-cloudfoundry/

terraform-provider-credhub

mevansam/

terraform-provider-uaa

2019

2018

2017

orange-cloudfoundry/

terraform-secure-backend

26 of 28

Community CFCR (K8S) providers

26

terraform-providers/�terraform-provider-kubernetes

2018

2017

mcuadros/terraform-provider-helm

2019

27 of 28

CFAR provider backlog, challenges & future work

Backlog:

zero downtime deployment / blue/green support #25
network policy support #33
V3 CC API not yet covered
Pending support for TF 0.12 #126

Challenges:

Hard to reuse CF CLI code base while CC API workflows are complex. #120

On going CF library extraction promising CF-CLI ##3960386

Acceptance test environment maintenance not yet leveraging CFF RelInt’s latest

Other Future work

Extract cloudfoundry_app artefacts fetching support (url, git, github release) into their own datasource
K8S helm provider improvements
Bosh director config support

Suggestions, comments, and contributions are welcome!

27

28 of 28

Thanks! Questions ?

Slides: https://goo.gl/5GRV1t

Reach us on cloudfoundry slack #terraform

https://cloudfoundry.slack.com/messages/C7JRBR8CV/

28