1 of 62

“Look ma, no hands”

Jenkins Configuration-as-Code

1

2 of 62

Who are we?

Name: Ewelina Wilkosz

Work: IT Consultant @ Praqma

Previous experience: Software Developer

@ Ericsson (6 years) in Krakow

Tools I work with: Jenkins (as Code), Git, Docker

@ewelinawilkosz

ewe@praqma.com

Jenkins Configuration-as-Code

3 of 62

Who are we?

Name: Nicolas De Loof

Work: Hacker @ CloudBees

Jenkins contributor & Docker Captain

Conference organizer and Video maker

@ndeloof

ndeloof@cloudbees.com

Jenkins Configuration-as-Code

4 of 62

2018 is “ * as code”

Jenkins Configuration-as-Code

5 of 62

Infrastructure as Code

Environment as Code

Architecture as Code

CI/CD as Code

5

6 of 62

Manage Jenkins as Code

Jenkins infrastructure

Jenkins job configuration

Jenkins system configuration

6

7 of 62

Jenkins infrastructure

7

8 of 62

Jenkins infrastructure

Using external tools

Jenkins CLI
REST API
Python-jenkins
Jenkins-client (Java, golang)
...

8

9 of 62

Jenkins infrastructure

Ansible, Chef, Puppet

Docker

9

10 of 62

jobs configuration

10

11 of 62

Jenkins job configuration

JobDSL plugin (groovy)
Job builder plugin (yaml)
...
Jenkins Pipeline

Multibranch
Organizations folders

11

12 of 62

JobDSL

job('gr8 example') {� scm {� github 'sheehan/job-dsl-gradle-example'� }� triggers {� scm 'H/5 * * * *'� }� steps {� gradle 'clean test'� }� publishers {� archiveJunit 'build/test-results/**/*.xml'� extendedEmail 'mr.sheehan@gmail.com'� }�}

12

13 of 62

Jenkins �master configuration

13

14 of 62

“Jenkins can be installed through native system packages, Docker, or run standalone by any machine with a Java Runtime Environment (JRE) installed…”

--- an enthusiast Jenkins user

Jenkins Configuration-as-Code

15 of 62

“… but it has to be configured manually”�--- a not so enthusiast Jenkins user

Jenkins Configuration-as-Code

16 of 62

looong scroll down

Jenkins Configuration-as-Code

17 of 62

And we don’t (always) like that

Jenkins Configuration-as-Code

18 of 62

Jenkins Configuration-as-Code

19 of 62

So how do we solve it?

Jenkins Configuration-as-Code

20 of 62

Jenkins system configuration

init.groovy
scriptler
scm-sync-configuration 😱

20

21 of 62

21

22 of 62

We’re not alone

22

23 of 62

JENKINS-31094 (system-config-dsl)
XML templating (seen at JenkinsWorld 2017)
Various Groovy bindings
Praqma’s “JenkinsAsCodeReference”
CloudBees CTO Office’s prototype

23

24 of 62

& & to join forces

Both had working prototypes last summer
Praqma focusing on:

real world usage by customers

CloudBees focusing on:

community adoption
out-of-the box support for our products

⇒ https://github.com/jenkinsci/configuration-as-code-plugin

24

25 of 62

Let’s make it as easy as possible

Jenkins Configuration-as-Code

26 of 62

jenkins:� systemMessage: "JCasC Demo"

numExecutors: 1

scmCheckoutRetryCount: 4

mode: NORMAL

securityRealm:

local:

allowsSignup: false

users:

- id: demoAdmin

password: ${adminpw}

jenkins.yaml

Jenkins Configuration-as-Code

27 of 62

Main benefits

Safety
Traceability
Speed
Easy to use
Easy to reuse

Jenkins Configuration-as-Code

28 of 62

There are challenges

Manage configuration as human-readable config file(s)�
Self-describing model to reflect Web UI�
Configure all jenkins initial setup (including plugins)�
Support most (*) plugins without extra development effort�
Generate documentation and validation tools (schema)�

28

29 of 62

human-readable config file(s)

Structured content
Nothing language centric

No groovy / ruby / xx

Readable and commentable

29

30 of 62

YAML ...

Indentation matters

photo credit :�Justin Palmer @Caged

30

31 of 62

Web UI as implicit documentation

Config element in web UI

==

Config element in YAML

“ No need to be a Jenkins expert to do it right” � -- Obi Wan Kenobi

31

32 of 62

Configure Jenkins in yaml

Obvious, isn’t it ?

32

jenkins:

securityRealm:

ldap:

configurations:

- server: ldap.acme.com

rootDN: dc=acme,dc=fr

managerPasswordSecret: ${LDAP_PASSWORD}

cache:

size: 100

ttl: 10

userIdStrategy: CaseSensitive

groupIdStrategy: CaseSensitive

tool:

git:

installations:

- name: git

- path: /bin/git

32

33 of 62

Configure ALL jenkins initial setup

No hand on keyboard

No click on web UI

to deploy

a fully working Jenkins master

33

34 of 62

Support ALL plugins

No need to write glue code for every supported plugin�
Most(*) plugins supported out of the box�
Others can bundle adapter code

(*) could require some minor changes

34

!! we require configuration-as-code-support plugin to be installed, for now !!

34

35 of 62

Generate documentation and validation tools

Can validate without running a test master
IDE support

35

36 of 62

Here comes JCasC

Jenkins Configuration-as-Code

37 of 62

Where to find it?!

https://github.com/jenkinsci/configuration-as-code-plugin

Implementation details and guide for plugin developers available in plugin’s github repository

Jenkins Configuration-as-Code

38 of 62

DEMO

38

39 of 62

How it works

39

40 of 62

Live Jenkins instance

Core + plugins

Yaml parser
Doc generator
Schema validator

Data �model

40

41 of 62

Introspection

Jenkins-core 2.xx + plugins [ git:3.7.0, ...]

Jenkins root instance
Descriptors (global configuration)
+ Special component with CasC support

=> hierarchical data model, trying to mimic Jenkins UI

41

42 of 62

Requirements

Target components need to follow some basic design rules

We rely on UI data binding mechanism (@DataBound)

Component to directly parse StaplerRequest / JsonObject can’t be introspected

Recommendations to plugin developers�github.com/jenkinsci/configuration-as-code-plugin/blob/master/PLUGINS.md
Pull requests on major plugins we want to support�github.com/jenkinsci/mailer-plugin/pull/39

42

43 of 62

Doc/Schema Generation

JENKISN/plugin/configuration-as-code/

JENKINS/plugin/configuration-as-code/schema

43

44 of 62

Corner cases

Some components hardly fit this model

For those we can develop dedicated Configurator adapter classes.

44

45 of 62

Under the hood

45

46 of 62

Root Elements → RootElementConfigurator

46

jenkins:

securityRealm:

ldap:

configurations:

- server: ldap.acme.com

rootDN: dc=acme,dc=fr

managerPasswordSecret: ${LDAP_PASSWORD}

cache:

size: 100

ttl: 10

userIdStrategy: CaseSensitive

groupIdStrategy: CaseSensitive

tool:

git:

installations:

- name: git

- path: /bin/git

46

47 of 62

Root Element

JenkinsConfigurator�“jenkins” → Jenkins.instance root object�
GlobalConfigurationCategoryConfigurator�“tools”, “security”, … → Descriptors grouped by categories�
DescriptorRootElementConfigurator�Uncategorized Descriptors with a global configuration page�“mailer”, ...�
CredentialsRootConfigurator�“credentials” → Glue code for credentials plugin (more on this later)�

47

48 of 62

Child element → Attribute

48

jenkins:

securityRealm:

ldap:

configurations:

- server: ldap.acme.com

rootDN: dc=acme,dc=fr

managerPasswordSecret: ${LDAP_PASSWORD}

cache:

size: 100

ttl: 10

userIdStrategy: CaseSensitive

groupIdStrategy: CaseSensitive

48

49 of 62

Attribute

Configurator do describe a target component as a set of Attributes

Attribute handle :

Name
Type (inferred by reflection on generics)
Multiplicity (Collection<x>)
Setting value

49

50 of 62

Generic Attribute

writable JavaBean property | DataBound constructor parameter

public void setSecurityRealm(SecurityRealm securityRealm) {

SecurityRealm is an ExtensionPoint (abstract)

Configuration-as-Code need to identify implementation

50

jenkins:

securityRealm:

ldap:

50

51 of 62

Extension point implementation

SecurityRealm is an ExtensionPoints => candidates implementations:

�LegacySecurityRealm → @Symbol(“legacy”) → legacy�HudsonPrivateSecurityRealm → @Symbol(“local”) → local�ActiveDirectorySecurityRealm → ActiveDirectory → activedirectory�LDAPSecurityRealm → LDAP → ldap

51

jenkins:

securityRealm:

ldap:

51

52 of 62

Build target Component

@DataBoundConstructor public LDAPSecurityRealm(� List<LDAPConfiguration> configurations, � boolean disableMailAddressResolver,� CacheConfiguration cache, � IdStrategy userIdStrategy, � IdStrategy groupIdStrategy)

+ DataBoundSetters

52

jenkins:

securityRealm:

ldap:

configurations:

...

cache:

size: 100

ttl: 10

userIdStrategy: CaseSensitive

groupIdStrategy: CaseSensitive

52

53 of 62

Corner cases

Setter method defined for internal needs / backward compatibility�We exclude @Deprecated and @Restricted �
[WiP] Technical facing Property name : “labelString”�We support @Symbol on setters�
Not a Describable / Internal model is … weird for end-user�Custom Configurator | Attribute implementation�

53

54 of 62

Custom Configurator, a.k.a “Glue Code”

Sample : expose a user-friendly credentials model

54

credentials:

system:

domainCredentials:

# global credentials

- credentials:

- certificate:

scope: SYSTEM

id: ssh_private_key

password: ${SSH_KEY_PASSWORD}

keyStoreSource:

fileOnMaster:

keyStoreFile: /docker/secret/id_rsa

CredentialsRootConfigurator�custom code

A fake Attribute "system"�to expose DomainCredentials (List)

with custom setter implementation:�

target.setDomainCredentialsMap(

DomainCredentials.asMap(value)

)

54

55 of 62

Status

55

56 of 62

1.0 is there !

… even 1.3 (released last week)

We welcome Feedback !

56

57 of 62

Features

Read configuration from local drive or url, REST API or CLI
Reload configuration (Manage Jenkins → Configuration as Code → Reload)
Export existing jenkins instance configuration into yaml (here be dragons)
Compatibility dashboard : https://issues.jenkins-ci.org/secure/Dashboard.jspa?selectPageId=17346�Please report issues with “jcasc-compatibility” label

Additionally docker demo setup (which can be easily adapted for different than demo purpose): https://github.com/Praqma/praqma-jenkins-casc

57

58 of 62

JEP-201

Make this THE configuration component for Jenkins community

https://github.com/jenkinsci/jep/blob/master/jep/201/README.adoc

58

59 of 62

Give it a try

Report missing plugin support / broken features

Contribute test cases (easy) or fixes (not so easy :P)

Jenkins Configuration-as-Code

60 of 62

How to talk to us?

github issues working well for reporting problems
we’re monitoring Jenkins Users, Jenkins Developers mailing lists

but...

gitter channel is a place to go to: https://gitter.im/jenkinsci/configuration-as-code-plugin

60

Jenkins Configuration-as-Code

61 of 62

Questions?

Jenkins Configuration-as-Code

62 of 62

Thank you!

62