1 of 42

CNCF TOC Meeting

July 3, 2018

© 2018 Cloud Native Computing Foundation

1

2 of 42

LF Antitrust Policy Notice

CNCF meetings involve participation by industry competitors, and it is the intention of the Linux Foundation to conduct all of its activities in accordance with applicable antitrust and competition laws. It is therefore extremely important that attendees adhere to meeting agendas, and be aware of, and not participate in, any activities that are prohibited under applicable US state, federal or foreign antitrust and competition laws.

​

Examples of types of actions that are prohibited at CNCF meetings and in connection with Linux Foundation activities are described in the Linux Foundation Antitrust Policy available at http://www.linuxfoundation.org/antitrust-policy. If you have questions about these matters, please contact your company counsel, or if you are a member of the Linux Foundation, feel free to contact Andrew Updegrove of the firm of Gesmer Updegrove LLP, which provides legal counsel to the Linux Foundation.

​

© 2018 Cloud Native Computing Foundation

2

3 of 42

Meeting Logistics

  • Time: July 3 2018 8AM (PT)
  • https://zoom.us/j/263858603
  • Or Telephone:
    • Dial:
      • +1 646 558 8656 (US Toll) or +1 408 638 0968 (US Toll)
      • +1 855 880 1246 (US Toll Free) or +1 877 369 0926 (US Toll Free)
    • Meeting ID: 263 858 603
    • International numbers: https://zoom.us/zoomconference

© 2018 Cloud Native Computing Foundation

3

4 of 42

TOC - Members Present Today

​

Note: TOC meetings require a quorum of two-thirds of the TOC total members to take a vote or make any decision. If a TOC meeting fails to meet the quorum requirement, discussions may proceed, however there shall be no voting or decisions.

© 2018 Cloud Native Computing Foundation

4

5 of 42

Agenda

  • Standard Agenda
    • Community Presentation: TiKV
    • Community Backlog
    • Open Q&A

​

​

© 2018 Cloud Native Computing Foundation

5

6 of 42

CNCF Project Proposals: RFC DD

  • Call to action for TOC Contributors to help review proposals or upcoming project presentations:

​

Upcoming Project Proposals:

Backlog:

© 2018 Cloud Native Computing Foundation

6

7 of 42

TiKV

© 2018 Cloud Native Computing Foundation

7

8 of 42

​

An Open-Source Distributed Transactional Key-Value Store

Make Data Creative

​

Ed Huang & Kevin Xu

https://github.com/pingcap/tikv

​

9 of 42

Agenda

  • History, Community, Status
  • Technical Walkthrough
  • Use Case with Ele.me
  • Demo
  • Q&A

2018 PingCAP

10 of 42

History

  • Founded in April 2015 by 3 infrastructure engineers
  • TiDB platform: (Ti = Titanium)
    • TiDB (stateless SQL layer; MySQL compatible)
    • *TiKV* (distributed transactional key-value store)
    • TiSpark (Apache Spark plug-in for TiKV)
    • Placement Driver (metadata store, scheduling, timestamp allocator)
  • TiKV open-sourced on April 1, 2016
    • Current version: 2.0
    • License: Apache 2.0
    • Link: https://github.com/pingcap/tikv

​

2018 PingCAP

11 of 42

Community

Stars

  • TiDB: 13,500+
  • *TiKV*: 3300+

Contributors

  • TiDB: 180+
  • *TiKV*: 70+
    • ~3000 Commits

2018 PingCAP

12 of 42

Outside Institutional Contributors

  • Samsung
  • Mobike
  • Toutiao.com
  • Ele.me
  • Tencent Cloud
  • UCloud

2018 PingCAP

13 of 42

Pain Point

  • World needs an open-source, unifying distributed storage layer that supports:
    • Strong consistency
    • Distributed transaction (ACID compliant)
    • Horizontal scalability
    • Cloud-native architecture
  • Building block to simplify building other systems
    • So far: TiDB (MySQL), TiSpark (SparkSQL), Toutiao.com (metadata service for their own S3), Ele.me (Redis Protocol Layer)
    • Sky is the limit!

2018 PingCAP

14 of 42

TiKV Architecture

15 of 42

Overview

TiDB

TiDB

Worker

Spark Driver

TiKV Cluster (Storage)

Metadata

TiKV

TiKV

TiKV

Data location

Job

TiSpark

DistSQL API

TiKV

TiDB

TSO/Data location

Worker

Worker

Spark Cluster

TiDB Cluster

TiDB

DistSQL API

PD

PD Cluster

TiKV

TiKV

TiDB

KV API

MySQL

MySQL

Spark SQL

PD

PD

Spark SQL

2018 PingCAP

16 of 42

TiKV Architecture

RocksDB

Raft

Transaction

Txn KV API

Coprocessor API

RocksDB

Raft

Transaction

Txn KV API

Coprocessor API

RocksDB

Raft

Transaction

Txn KV API

Coprocessor API

Raft Group

Client

gRPC

TiKV Instance

TiKV Instance

TiKV Instance

gRPC

gRPC

PD Cluster

2018 PingCAP

17 of 42

Technical highlights

  • Scheduling and Auto-Balancing
  • Multi-Raft
  • Dynamic Range-based Partition
    • Split / Merge / Leader transfer (how we resolve hot spots)
  • ACID Transaction
    • 2-Phase Commit with Optimistic Lock
  • Written in Rust (No GC stoptime / No runtime cost)

2018 PingCAP

18 of 42

Example 1: SQL above TiKV

Node1

Node2

Node3

Node4

MySQL Network Protocol

SQL Parser

Cost-based Optimizer

Distributed Executor (Coprocessor)

ODBC/JDBC

MySQL Client

Any ORM which supports MySQL

TiDB

TiKV

2018 PingCAP

19 of 42

Relational -> KV

ID

Name

Email

1

Edward

h@pingcap.com

2

Tom

tom@pingcap.com

...

​

user/1

Edward,h@pingcap.com

user/2

Tom,tom@pingcap.com

...

​

In TiKV

-∞

+∞

(-∞, +∞)

Sorted map

“User” Table

Some region...

2018 PingCAP

20 of 42

SQL -> Parser -> Optimizer -> Coprocessor

​

2018 PingCAP

21 of 42

Example 2: Distributed Object Storage

TiKV

2018 PingCAP

22 of 42

YCSB Benchmark (June 2018)

Environment:

  • Raw KV API
  • 3 TiKV nodes (thus 3 replicas)
  • Intel(R) Xeon(R) Gold 6142 CPU @ 2.60GHz / 192GB RAM / 700 NVMe SSD
  • 10m record, 10 field count, 100 field length, uniform request distribution, 3000 threads

​

100% Insert TPS 82500 ops/s

Insert Latency Avg(us): 63824, Min(us): 577, Max(us): 1895394, 95th(us): 118000, 99th(us): 169000

​

100% Read QPS 157400 ops/s

Read Latency Avg(us): 36308, Min(us): 280, Max(us): 12360007, 95th(us): 94000, 99th(us): 164000

​

2018 PingCAP

23 of 42

Comparison

​

Cassandra

MongoDB

TiKV

Auto-scaling

Y

Optional

Y

ACID transaction

N

Maybe?

Y

Strong consistency replication

Optional

N

Y

Geo-based replication

N

N

Y

Self-healing

N

N

Y

SQL Compatibility

Partial (CQL)

N

Complete (with TiDB)

2018 PingCAP

24 of 42

Dynamic Split and Merge

Region A

Region A

Region B

Region A

Region A

Region B

Split

Region A

Region A

Region B

Merge

2018 PingCAP

25 of 42

Hotspot Removal

*Region A*

*Region B*

Region A

Region B

Workload

*Region A*

Region B

Region A

*Region B*

Workload

Workload

Hotspot Schedule

(Raft leader transfer)

2018 PingCAP

26 of 42

Cloud-Native Progress

...

...

2018 PingCAP

27 of 42

Cloud-Native Synergy

  • Deployment
    • Docker-compose deployment (for test/dev on local machine)
    • Kubernetes with TiDB-Operator tool (soon to be open source)
    • One-click Helm Chart deployment of monitoring and logging
  • Prometheus* (maintain Rust implementation)
  • gRPC* (maintain Rust implementation)
  • etcd* (active contributor, e.g. “Learner” feature, bug fixes)
  • Grafana

2018 PingCAP

28 of 42

TiKV In Production

29 of 42

Who’s Using TiKV?

​

200+ Companies

2018 PingCAP

30 of 42

Ele.me ( ) Before TiKV

  • Food delivery platform with 260 million users (Grubhub + Doordash + Deliveroo + Postmates, etc.)
  • Recently acquired by Alibaba for 9.5B USD
  • Bulk of its data is in key-value format
    • Hodgepodge of: MySQL, Mongo, Cassandra, Redis…
  • Need a single storage layer

​

2018 PingCAP

31 of 42

Ele.me ( ) After TiKV

  • Deploys TiKV as a unifying KV storage layer
  • TiKV affects 80% of platform’s traffic
  • 25+ TBs, 10+ clusters, 4 IDCs, 100+ nodes
  • Built own Redis proxy on top of TiKV
  • Detailed use case story written by Ele.me engineers: https://www.pingcap.com/blog/use-case-tidb-in-eleme/

​

2018 PingCAP

32 of 42

Demo

2018 PingCAP

33 of 42

Ask CNCF to accept TiKV as Incubation or Sandbox-level project.

​

​

2018 PingCAP

34 of 42

More Contribution, Bigger Community

  • Bigger community, vendor neutrality, help with project governance
  • More contribution to build components like:
    • Support more languages (right now: Go client for TiDB; Java client for TiSpark)
    • Redis Proxy (like Tidis: https://github.com/yongman/tidis)
    • Column family (already started: https://github.com/pingcap/tikv/pull/2852)
    • Useful features:
      • Compare and Swap
      • Hybrid storage engine
    • Multiple programming paradigms
      • KV / Document / Table …

​

2018 PingCAP

35 of 42

Would you be our TOC sponsor? (Bryan Cantrill volunteers)

Thank You!

Ed Huang (huang@pingcap.com; @c4pt0r)

Kevin Xu (kevin@pingcap.com; @kevinsxu)

​

2018 PingCAP

36 of 42

WG Updates

© 2018 Cloud Native Computing Foundation

36

37 of 42

CNCF WG Updates

https://github.com/cncf/toc#working-groups

© 2018 Cloud Native Computing Foundation

37

38 of 42

Project Review/Backlog

  • See spreadsheets!
    • Project priorities
    • Review backlog
    • → send suggestions to TOC list

​

​

​

​

© 2018 Cloud Native Computing Foundation

38

39 of 42

Upcoming Events

  • Main CNCF Events:
    • Nov 14-15, 2018: KubeCon + CloudNativeCon Shanghai
    • Dec 11-13, 2018: KubeCon + CloudNativeCon Seattle
    • May 21-23, 2019: KubeCon + CloudNativeCon Barcelona

​

    • Sponsorship info available!

© 2018 Cloud Native Computing Foundation

39

40 of 42

Upcoming Meetings

  • Next Meeting is July 17th
    • Community Presentation: Falco
    • Working Group readouts: TBD
  • Meetings are the first and third Tuesdays of the month

​

© 2018 Cloud Native Computing Foundation

40

41 of 42

Open Q&A

© 2018 Cloud Native Computing Foundation

41

42 of 42

Thank You

© 2018 Cloud Native Computing Foundation

42