1 of 60

Configuration-as-Code

1

Jenkins Configuration-as-Code

​

2 of 60

Who are we?

Name: Ewelina Wilkosz

Work: IT Consultant @ Praqma

Previous experience: Software Developer

@ Ericsson (6 years) in Krakow

Tools I work with: Jenkins (as Code), Git, Docker

​

​

@ewelinawilkosz

@ewelinawilkosz

ewe@praqma.com

Jenkins Configuration-as-Code

​

3 of 60

Who are we?

Name: Nicolas De Loof

Work: Hacker @ CloudBees

Jenkins contributor & Docker Captain

Conference organizer and Video maker

​

​

​

​

@ndeloof

@ndeloof

ndeloof@cloudbees.com

Jenkins Configuration-as-Code

​

4 of 60

2018 is “ * as code”

​

Jenkins Configuration-as-Code

​

​

​

Jenkins Configuration-as-Code

​

5 of 60

Infrastructure as Code

Environment as Code

Architecture as Code

CI/CD as Code

​

5

Jenkins Configuration-as-Code

​

6 of 60

Manage Jenkins as Code

  • Jenkins infrastructure

​

  • Jenkins job configuration

​

  • Jenkins system configuration

6

Jenkins Configuration-as-Code

​

7 of 60

Jenkins infrastructure

7

Jenkins Configuration-as-Code

​

8 of 60

Jenkins infrastructure

​

Using external tools

  • Jenkins CLI
  • REST API
  • Python-jenkins
  • Jenkins-client (Java, golang)
  • ...

8

Jenkins Configuration-as-Code

​

9 of 60

Jenkins infrastructure

Ansible, Chef, Puppet

​

Docker

9

Jenkins Configuration-as-Code

​

10 of 60

jobs configuration

10

Jenkins Configuration-as-Code

​

11 of 60

Jenkins job configuration

  • JobDSL plugin (groovy)
  • Job builder plugin (yaml)
  • ...
  • Jenkins Pipeline
    • Multibranch
    • Organizations folders

11

Jenkins Configuration-as-Code

​

12 of 60

JobDSL

job('gr8 example') {� scm {� github 'sheehan/job-dsl-gradle-example'� }� triggers {� scm 'H/5 * * * *'� }� steps {� gradle 'clean test'� }� publishers {� archiveJunit 'build/test-results/**/*.xml'� extendedEmail 'mr.sheehan@gmail.com'� }�}

12

Jenkins Configuration-as-Code

​

13 of 60

Jenkins master configuration

13

Jenkins Configuration-as-Code

​

14 of 60

Jenkins can be installed through native system packages, Docker, or run standalone by any machine with a Java Runtime Environment (JRE) installed...

Jenkins Configuration-as-Code

​

​

​

Jenkins Configuration-as-Code

​

15 of 60

… and it is manually configurable

Jenkins Configuration-as-Code

​

​

​

Jenkins Configuration-as-Code

​

16 of 60

Jenkins Configuration-as-Code

​

17 of 60

And we don’t (always) like that

Jenkins Configuration-as-Code

​

18 of 60

Jenkins Configuration-as-Code

​

19 of 60

So how do we solve it?

Jenkins Configuration-as-Code

​

​

Jenkins Configuration-as-Code

​

20 of 60

Jenkins system configuration

  • init.groovy
  • scriptler
  • scm-sync-configuration 😱

20

Jenkins Configuration-as-Code

​

21 of 60

21

Jenkins Configuration-as-Code

​

22 of 60

We’re not alone

22

Jenkins Configuration-as-Code

​

23 of 60

  • JENKINS-31094 (system-config-dsl)
  • XML templating (seen at JenkinsWorld 2017)
  • Various Groovy bindings
  • Praqma’s “JenkinsAsCodeReference”
  • CloudBees CTO Office’s prototype

23

Jenkins Configuration-as-Code

​

24 of 60

& & to join forces

​

  • Both had working prototypes last summer
  • Praqma focusing on:
    • real world usage by customers
  • CloudBees focusing on:
    • community adoption
    • out-of-the box support for our products

⇒ https://github.com/jenkinsci/configuration-as-code-plugin

24

Jenkins Configuration-as-Code

​

25 of 60

Let’s make it as easy as possible

Jenkins Configuration-as-Code

​

​

​

Jenkins Configuration-as-Code

​

26 of 60

jenkins:� systemMessage: "JCasC Demo"

numExecutors: 1

scmCheckoutRetryCount: 4

mode: NORMAL

securityRealm:

local:

allowsSignup: false

users:

- id: demoAdmin

password: ${adminpw}

​

​

​

jenkins.yaml

Jenkins Configuration-as-Code

​

27 of 60

Main benefits

  • Safety/Security
  • Traceability
  • Speed
  • Easy to use
  • Easy to reuse

Jenkins Configuration-as-Code

​

28 of 60

There are challenges

  • Manage configuration as human-readable config file(s)�
  • Self-describing model to reflect Web UI�
  • Configure all jenkins initial setup (including plugins)�
  • Support most (*) plugins without extra development effort�
  • Generate documentation and validation tools (schema)�

​

28

Jenkins Configuration-as-Code

​

29 of 60

human-readable config file(s)

  • Structured content
  • Nothing language centric
    • No groovy / ruby / xx
  • Readable and commentable

​

​

29

Jenkins Configuration-as-Code

​

30 of 60

Web UI as implicit documentation

Config element in web UI

==

Config element in YAML

​

“ No need to be a Jenkins expert to do it right” � -- Obi Wan Kenobi

30

Jenkins Configuration-as-Code

​

31 of 60

Configure Jenkins in yaml

Obvious, isn’t it ?

31

jenkins:

securityRealm:

ldap:

configurations:

- server: ldap.acme.com

rootDN: dc=acme,dc=fr

managerPasswordSecret: ${LDAP_PASSWORD}

cache:

size: 100

ttl: 10

userIdStrategy: CaseSensitive

groupIdStrategy: CaseSensitive

​

tool:

git:

installations:

- name: git

- path: /bin/git

​

Jenkins Configuration-as-Code

​

32 of 60

Configure ALL jenkins initial setup

​

No hand on keyboard

No click on web UI

to deploy

​

a fully working Jenkins master

32

Jenkins Configuration-as-Code

​

33 of 60

Support ALL plugins

  • No need to write glue code for every supported plugin�
  • Most(*) plugins supported out of the box�
  • Others can bundle adapter code

​

​

(*) could require some minor changes

​

33

Jenkins Configuration-as-Code

​

34 of 60

Generate documentation and validation tools

​

  • Can validate without running a test master
  • IDE support

​

​

34

Jenkins Configuration-as-Code

​

35 of 60

Here comes JCasC

Jenkins Configuration-as-Code

​

​

​

Jenkins Configuration-as-Code

​

36 of 60

Where to find more information

Implementation details and guide for plugin developers available in plugin’s github repository:

https://github.com/jenkinsci/configuration-as-code-plugin

​

Jenkins Configuration-as-Code

​

37 of 60

DEMO

37

Jenkins Configuration-as-Code

​

38 of 60

How it works

38

Jenkins Configuration-as-Code

​

39 of 60

Live Jenkins instance

Core + plugins

  • Yaml parser
  • Doc generator
  • Schema validator

Data �model

39

Jenkins Configuration-as-Code

​

40 of 60

Introspection

Jenkins-core 2.xx + plugins [ git:3.7.0, ...]

  • Jenkins root instance
  • Descriptors (global configuration)
  • + Special component with CasC support

=> hierarchical data model, trying to mimic Jenkins UI

40

Jenkins Configuration-as-Code

​

41 of 60

Requirements

Target components need to follow some basic design rules

We rely on UI data binding mechanism (@DataBound)

Component to directly parse StaplerRequest / JsonObject can’t be introspected

​

​

41

Jenkins Configuration-as-Code

​

42 of 60

Doc/Schema Generation

JENKISN/plugin/configuration-as-code/

JENKINS/plugin/configuration-as-code/schema

42

Jenkins Configuration-as-Code

​

43 of 60

Corner cases

Some components hardly fit this model

For those we can develop dedicated Configurator adapter classes.

​

43

Jenkins Configuration-as-Code

​

44 of 60

Under the hood

44

Jenkins Configuration-as-Code

​

45 of 60

Root Elements → RootElementConfigurator

45

jenkins:

securityRealm:

ldap:

configurations:

- server: ldap.acme.com

rootDN: dc=acme,dc=fr

managerPasswordSecret: ${LDAP_PASSWORD}

cache:

size: 100

ttl: 10

userIdStrategy: CaseSensitive

groupIdStrategy: CaseSensitive

​

tool:

git:

installations:

- name: git

- path: /bin/git

​

Jenkins Configuration-as-Code

​

46 of 60

Root Element

  • JenkinsConfigurator�“jenkins” → Jenkins.instance root object�
  • GlobalConfigurationCategoryConfigurator�“tools”, “security”, … → Descriptors grouped by categories�
  • DescriptorRootElementConfigurator�Uncategorized Descriptors with a global configuration page�“mailer”, ...�
  • CredentialsRootConfigurator�“credentials” → Glue code for credentials plugin (more on this later)�

46

Jenkins Configuration-as-Code

​

47 of 60

Child element → Attribute

47

jenkins:

securityRealm:

ldap:

configurations:

- server: ldap.acme.com

rootDN: dc=acme,dc=fr

managerPasswordSecret: ${LDAP_PASSWORD}

cache:

size: 100

ttl: 10

userIdStrategy: CaseSensitive

groupIdStrategy: CaseSensitive

​

​

Jenkins Configuration-as-Code

​

48 of 60

Attribute

Configurator do describe a target component as a set of Attributes

Attribute handle :

  • Name
  • Type (inferred by reflection on generics)
  • Multiplicity (Collection<x>)
  • Setting value

48

Jenkins Configuration-as-Code

​

49 of 60

Generic Attribute

writable JavaBean property | DataBound constructor parameter

public void setSecurityRealm(SecurityRealm securityRealm) {

​

​

SecurityRealm is an ExtensionPoint (abstract)

Configuration-as-Code need to identify implementation

​

​

​

49

jenkins:

securityRealm:

ldap:

​

​

​

Jenkins Configuration-as-Code

​

50 of 60

Extension point implementation

SecurityRealm is an ExtensionPoints => candidates implementations:

�LegacySecurityRealm → @Symbol(“legacy”) → legacy�HudsonPrivateSecurityRealm → @Symbol(“local”) → local�ActiveDirectorySecurityRealm → ActiveDirectory → activedirectory�LDAPSecurityRealm → LDAP → ldap

​

​

​

50

jenkins:

securityRealm:

ldap:

​

​

​

Jenkins Configuration-as-Code

​

51 of 60

Build target Component

@DataBoundConstructor public LDAPSecurityRealm(� List<LDAPConfiguration> configurations, � boolean disableMailAddressResolver,� CacheConfiguration cache, � IdStrategy userIdStrategy, � IdStrategy groupIdStrategy)

​

+ DataBoundSetters

​

51

jenkins:

securityRealm:

ldap:

configurations:

...

cache:

size: 100

ttl: 10

userIdStrategy: CaseSensitive

groupIdStrategy: CaseSensitive

​

​

​

Jenkins Configuration-as-Code

​

52 of 60

Corner cases

  • Setter method defined for internal needs / backward compatibility�We exclude @Deprecated and @Restricted �
  • [WiP] Technical facing Property name : “labelString”�We support @Symbol on setters�
  • Not a Describable / Internal model is … weird for end-user�Custom Configurator | Attribute implementation�

​

​

​

52

Jenkins Configuration-as-Code

​

53 of 60

Custom Configurator, a.k.a “Glue Code”

Sample : expose a user-friendly credentials model

​

​

​

​

​

​

​

53

credentials:

system:

domainCredentials:

# global credentials

- credentials:

- certificate:

scope: SYSTEM

id: ssh_private_key

password: ${SSH_KEY_PASSWORD}

keyStoreSource:

fileOnMaster:

keyStoreFile: /docker/secret/id_rsa

​

​

​

​

CredentialsRootConfigurator�custom code

​

A fake Attribute "system"�to expose DomainCredentials (List)

with custom setter implementation:�

​

​

target.setDomainCredentialsMap(

DomainCredentials.asMap(value)

)

​

Jenkins Configuration-as-Code

​

54 of 60

Status

54

Jenkins Configuration-as-Code

​

55 of 60

Features

  • Read configuration from local drive or url
  • Reload configuration (Manage Jenkins → Configuration as Code → Reload)
  • Export existing jenkins instance configuration into yaml
  • Supported plugins: https://github.com/jenkinsci/configuration-as-code-plugin/tree/master/demos and more

​

  • Additionally docker demo setup (which can be easily adapted for different than demo purpose):

​

Jenkins Configuration-as-Code

​

56 of 60

JEP-201

Make this THE configuration component for Jenkins community

https://github.com/jenkinsci/jep/blob/master/jep/201/README.adoc

56

Jenkins Configuration-as-Code

​

57 of 60

alpha

0.9-alpha released

Use Experimental Plugins Update Center to install, details here

​

Feedback requested (jenkins-users mailing list)

Main Target audience : early adopters & plugin maintainers

57

Jenkins Configuration-as-Code

​

58 of 60

Give it a try

Report missing plugin support / broken features

Contribute test cases (easy) or fixes (not so easy :P)

Jenkins Configuration-as-Code

​

​

​

Jenkins Configuration-as-Code

​

59 of 60

Questions?

Jenkins Configuration-as-Code

​

​

Jenkins Configuration-as-Code

​

60 of 60

Thank you!

Jenkins Configuration-as-Code

​