Deploying DC/OS On Prem

(With a little help from your friends)

Who’s the guy at the front?

Tom Barber

• NASA JPL Devops and Data Developer
• Spicule & Meteorite BI
• Data guy and open source hacker… erm developer.

A bit of history…...

Caveat

We want to run our services on hardware supplied by

​

​

​

We also want to retain control of the underlying hardware to control costs, networking and scalability.

Our Goal

We also want to be able to run our services in different data centres for resilience and data locality purposes.

So how can we achieve our goals?

Container Management Systems

• Docker Swarm
• Kubernetes
• Apache Mesos
• CoreOS
• DC/OS
• Deployed to fixed hosts

I’m a sucker for a nice looking dashboard

The next question we asked was:

“We have a bunch of Bigstep bare metal servers, what's the most flexible way to manage them?”

Introducing Metal As A Service!

• Deploy workloads onto bare metal servers
• Integration with Juju to manage the services
• DNS & IP management
• Internal fabric networking
• Operating system agnostic

​

We already manage a lot of our services using Juju… so could we make DC/OS work with Juju?

Application modeling with Juju

Juju Key Concepts

• Bootstrap
• Deploy
• Relate
• Scale
• Repeat!

Bootstrap

• Pick a cloud
• AWS
• Azure
• CloudSigma
• Google
• Joyent
• Rackspace
• LXD Local
• Manual
• MAAS!

Deploy

juju deploy dcos-master

juju deploy dcos-agent

​

Relate

juju add-relation dcos-master dcos-agent

​

Scale

juju add-unit -n 10 dcos-agent

​

Repeat! (Anywhere)

Export my model to a YAML file

Import into a new cloud or MAAS service

Deploys in the same manner

Crowdsourcing Ops

• Majority of users get Charms from the Juju Charm Store. Similar to PuppetForge et al
• Charms written by a user or company and then added to and extended by others
• Charms that have been validated and have a test suite are often promulgated to the top level recommended charms to try and ensure users have a good experience

Development flow to enhance charms

Monitoring DC/OS

juju deploy nagios

juju deploy nrpe

juju add-relation nagios:monitors nrpe:monitors

juju add-relation dcos-master:local-monitors nrpe:local-monitors

Logging DC/OS

juju add-relation logstash dcos-master

(Finally) Deploying Containers to DC/OS

juju deploy dcos-nginx

juju add-relation dcos-nginx dcos-master

Enriching your deployment with Metadata

You can make use of non dockerized or Mesos-ized services from other charms

Passing around metadata allows you to configure applications dynamically even within and across DC/OS core systems

Current State of Play

DCOS Master Nodes (Scalable)

DCOS Client Nodes (Scalable)

Monitoring connectivity

Ability to deploy docker containers

Actions to do various admin tasks

Roadmap

Implement disk mounts for NFS and Loopback devices.

Add more actions.

Implement Monitoring.

Improve Security.

Expose running app ports automatically.

Simplify the upgrade process.

ELK to Beats?

Custom Cluster Name

Private Docker Reg

Marathon Load Balancer

​

Support and Testing

info@spicule.co.uk <- me

juju@lists.ubuntu.com <- the community

tom@analytical-labs.com <- also me

https://jujucharms.com/u/spicule/basic-dcos-bundle <- a bundle to get you going

https://github.com/buggtb/dcos-master-charm <- the master

https://github.com/buggtb/dcos-agent-charm <- the agents

​

But it doesn’t end there…..

DC/OS is hard to configure once installed

You can’t easily change from public to private units

You can’t enable or disable OAuth logins

You can’t enable or disable telemetry

Layered Mesos

Mesos master and slaves

Marathon support for Docker deployment

Utilise the Zookeeper charm

SSL Communication

HA support

SDN support