1 of 27

CNCF TOC F2F Meeting

August 3rd, 2016

2 of 27

LF Antitrust Policy Notice

CNCF meetings involve participation by industry competitors, and it is the intention of the Linux Foundation to conduct all of its activities in accordance with applicable antitrust and competition laws. It is therefore extremely important that attendees adhere to meeting agendas, and be aware of, and not participate in, any activities that are prohibited under applicable US state, federal or foreign antitrust and competition laws.

​

Examples of types of actions that are prohibited at CNCF meetings and in connection with Linux Foundation activities are described in the Linux Foundation Antitrust Policy available at http://www.linuxfoundation.org/antitrust-policy. If you have questions about these matters, please contact your company counsel, or if you are a member of the Linux Foundation, feel free to contact Andrew Updegrove of the firm of Gesmer Updegrove LLP, which provides legal counsel to the Linux Foundation.

​

2

3 of 27

Meeting Logistics

Time: August 3rd 8AM (Pacific)

http://uberconference.com/cloudnative

Conference Number: +1-415-579-0198 No Pin Needed

3

4 of 27

​

TOC - Members Present Today

​

4

Company

Apache Software Foundation

Cisco

CoreOS

Docker

GoDaddy

Google

Joyent

Mesosphere

Weaveworks

​

Note: TOC meetings shall require a quorum of two-thirds of the TOC total members to take a vote or make any decision. If a TOC meeting fails to meet the quorum requirement, discussions may proceed, however there shall be no voting or decisions.

Name

Camille Fournier

Ken Owens (said will join late)

Jonathan Boulle

Solomon Hykes

Elissa Murphy

Brian Grant

Bryan Cantrill (vacation)

Benjamin Hindman (travelling)

Alexis Richardson

​

​

Email

skamille@gmail.com

kenowens@cisco.com

jonathan.boulle@coreos.com

solomon.hykes@docker.com

elissam@godaddy.com

briangrant@google.com

bryan@joyent.com

benh@mesosphere.io

alexis@weave.works

​

​

​

5 of 27

Agenda

  • Projects: Pipeline, Incubation
    • CNCF Proposals Process
    • Presentation: Fluentd
    • Presentation: Heron
    • Presentation: Minio
  • Helping Projects: Marketing, Docs, Tests, Other
  • CNCF Reference Architecture + Landscape
  • AOB

​

​

​

​

5

6 of 27

Proposals Process

Proposal: ratify & publish a “proposals process”.

This will require a vote.

​

This week: For discussion, please review email

Also - quick checkpoint on Incubation.

6

7 of 27

“Below the waterline” - DAN & CHRIS

Want to provide regular updates on work actions agreed in prior TOC calls, eg. website.

  • Slack / SlackArchive
  • PonyMail (https://ponymail.cncf.io/)
    • better mailing list archives
    • piloting Discourse as an alternative (http://discuss.cncf.io/)
  • CLABot update

7

8 of 27

Projects: Pipeline

  • NATS
    • Working on project proposal and legal issues
  • Heron (streams)
  • Fluentd (logging)
    • Testing the waters (Treasure Data joined CNCF)
  • Minio (storage)
    • Met Alexis & Chris - project is promising & needs a sponsor
  • OpenTracing & OpenZipkin
    • Further noodling - will update on TOC call

​

​

​

​

​

​

8

9 of 27

Fluentd: Overview and History

  • https://github.com/fluent/fluentd/
  • http://www.fluentd.org/
  • Started at Treasure Data in 2011
    • Sadayuki “Sada” Furuhashi: MessagePack and Embulk creator
  • Solve MxN problem in log collection reliably
    • M(any) inputs & N(umerous) outputs make log delivery hard

​

​

9

10 of 27

Fluentd: Milestones

  • 118 releases & 2,362 commits, 704 PRs (17 open)
    • 90 contributors
    • 50% of top 10 contributors commercially sponsored
  • 4,368 stars, 518 forks
  • 625 plugins
  • In production@Atlassian, Nintendo, LINE, Microsoft OMS, etc.

10

11 of 27

Fluentd: Key Features

  • Structural logging + Tag-based routing
    • parse data as JSON (msgpack internally)
    • Tags for declarative event routing
  • Built-in buffering
    • No external queue required for retries
  • Plugin system
    • Input/Parser/Filter/Buffer/Output/Formatter all pluggable
  • High performance
    • Mostly in Ruby, performance critical part in C

11

12 of 27

Fluentd: Architecture

12

TCP/UDP

Syslog

HTTP(S)

File

​

HDFS

Elasticsearch

S3

GCS

Azure Blob

Treasure Data

SaaS API

etc.

13 of 27

Fluentd: Docker Setup

  • Supported by Docker Log Driver API
    • Stream container logs directly to Fluentd (running on host or container)
  • Kubernetes Logging Pod

13

14 of 27

Heron

  • Started in 2013 at Twitter
    • Sanjeev, Maosong and Karthik
  • Motivated by several issues with Apache Storm
    • Inability to debug/tune/resource reservation
    • Several GC incidents
  • Open sourced May 25, 2016 (see blog post)
  • Apache 2.0 License

14

15 of 27

Heron: Community

  • 1260 commits
  • 40 contributors
  • 2210 github stars
  • 795 pull requests - 28 open
  • 141 open issues
  • Production at Twitter > 2 years
  • Going into production at WeChat and Microsoft

15

16 of 27

Heron: Key Features

  • Fully API compatible with Apache Storm
  • Task isolation
  • Developer productivity
  • Ease of manageability
  • Use of mainstream languages C++/Java/Python
  • Kubernetes support planned (see github)

​

​

16

17 of 27

Heron: Architecture

17

18 of 27

Minio Overview

Minio is an object storage server released under Apache License v2.0.

​

Github: https://github.com/minio�Home: www.minio.io

18

19 of 27

Minio Key Features

  • Minimal Design
  • Amazon S3 API Compatible
  • Erasure Code & Bitrot Protection
  • Event Notification & Lambda Computing

19

20 of 27

Minio Community

Minio Server, Minio Client, Minio SDK, MinL λ

  • 86 Contributors
  • 341 Gitter members
  • 2602 Github stars
  • 19K+ Downloads by unique IP’s
  • 20K+ Docker pulls (official and community together)
  • Adopters include:
    • Deis, Workday, Appknox, Codecasts, TransferXL, Watchup, SemanticMD

20

21 of 27

Scale using Microstorage Architecture

21

Stateless

Stateless

MINIO

Data Volume

Data Volume

Data Volume Containers / VM

​

Provision object storage like microservices

22 of 27

Distributed Minio (Erasure Code)

22

Minio 1

JBOD

Erasure Code (4x4, 6x6, 8x8) - Maximum 16 Disks / Setup

Minio 2

JBOD

Minio 16

JBOD

Minio 3

JBOD

23 of 27

CNCF Reference Architecture - KEN

  • Landscape introduced in email here
  • Any comments? Vote?
  • See slides

​

​

​

​

​

23

Application Definition / Development

Cloud Native Services

Cloud Native Runtime

Cloud Native OS / Provisioning

Infrastructure (Bare Metal/Cloud)

24 of 27

Cadence

  • Last meeting slides on 7/20

​

  • Meeting Cadence Reminder
    • First and third Wednesday of every month
    • Calls without TOC quorum will still be used to review issues

​

  • Next Meeting: 8/17/2016

​

​

24

25 of 27

Thank You

25

26 of 27

Other topics to discuss in the future

  1. Example Apps & Patterns
  2. Website “how we help” etc.
  3. “Below the waterline”: Define Incubation, Development Model & other community management issues; Advice; Advocacy; ..

​

​

​

26

27 of 27

Marketing - Request

This is going well for Prometheus eg. with 1.0

Please contribute if you can

Some asks:

  1. We need speakers for events - email pr@cncf.io
  2. We need TOC members to volunteer for triaging CFP responses for CloudNativeCon (email events@cncf.io)

​

27