1 of 32

How I learned to stop worrying about updates and love GitOps for Windows

Presented by: Sage Belrose, ICPSR

2 of 32

About Us

“Always remember that you are absolutely unique. Just like everyone else.” - Margaret Mead

​

3 of 32

Sage Belrose (She/They)

  • Background: K-12 IT and Sociology Degree
  • Job: DevOps Engineer at ICPSR
  • Automation Junkie
  • Trans
  • Neurodiverse
  • Opinionated
  • Eccentric
  • Pretty Smart
  • Engineer, not Project Manager

4 of 32

Jared Cohn (He/Him)

  • Background: MSP
  • Job: DevOps Engineer at ICPSR
  • Networks/Firewalls
  • On-Premise Infrastructure
    • VMWare
    • Backups
  • Other Duties as Assigned
  • Rockstar
  • Cool Dude
  • Can’t be here today

5 of 32

Problem

“Our OS and applications are all updated manually. Here is the checklist.”

6 of 32

Environment

  • VDI
  • Windows 10
  • Restricted Data Enclave
  • FISMA Moderate
  • VMWare Horizon + vSphere

7 of 32

The Spreadsheet

  • Every square is a task
  • Red is good!
  • Green/Black is action needed.
  • Auditing? Nah.

8 of 32

Deployment via GUI

9 of 32

Deployment via GUI

10 of 32

Deployment via GUI

11 of 32

Humans are Bad

  • Error prone
  • Do not generate log files

​

  • Do not have version control
  • Slower than code

​

12 of 32

Why Not SCCM?

  • Lack of Windows Admins
  • Bad history in ICPSR
  • “Just make it like Linux!”
    • Packer
    • Terraform
    • Ansible
    • yum/apt

13 of 32

Action

“Your first task, should you choose to accept it, is to automate the VDE Updates.”

14 of 32

Packer? Nah

  • Collaboration is hard
  • We don’t have root access

15 of 32

Chocolatey

  • Wait, there’s no yum or apt?
  • Trustworthy?

16 of 32

New Problems (Chocolatey)

  • No software pre-packaged
    • Build it!
  • Need a repository storage solution
    • SMB Share works
  • Monthly updates didn’t go away…
    • MSU support!
  • Version control and Auditing?
    • Gitlab!
  • Deployment?
    • IDEAL: Gitlab Pipeline -> Ansible -> Chocolatey Packages -> VM

17 of 32

ReplaceVersion.ps1

  • Just download the binary
  • Rename it
  • Run the script!

18 of 32

ReplaceVersion.ps1

19 of 32

ReplaceVersion.ps1

20 of 32

ReplaceVersion.ps1

  • Output →
  • Commit Hash
  • Timestamp
  • Automated
  • ReplaceAllVersions.ps1

21 of 32

Terraform? Nah, PowerCLI!

22 of 32

Snapshot Automation!

  • Snapshot VMs.ps1

23 of 32

Deploy it!

24 of 32

Result

“Weeklong update cycles reduced to hours.”

25 of 32

Efficiency Improvements

  • Original task →

​

​

​

  • Scope Creep →

26 of 32

Audit/Security Improvements

  • Sha256 hashes available and enforced
  • Packages are datestamped
  • Package source is commit-hashed
  • Download once, install ad infinitum
  • Remove internet access need on base images

27 of 32

What Next?

“All we should ever have to do is click approve on a Merge Request.”

28 of 32

TODO: Evolve from DevOps to GitOps

  • Ansibilization
  • Gitlab Pipeline
  • Configuration Drift Monitoring
  • Automated downloads/merge requests
  • Automated changelogs
  • Automated End-User Communication

29 of 32

TODO: Evolve from DevOps to GitOps

30 of 32

TODO: Evolve from DevOps to GitOps

31 of 32

TODO: Evolve from DevOps to GitOps

Orange

Is

TODO

Green

Is

Done

32 of 32

Questions?

Breakout Time!