1 of 17

Ceilometer

The OpenStack metering project

26 July 2012 @ XLCloud

Julien Danjou

​

julien@danjou.info

http://julien.danjou.info

Twitter: @juldanjou

2 of 17

Problems to solve

  • Collecting per user/tenant usage data
    • For every OpenStack component
    • In a single place

​

  • Retrieving usage data
    • From a single place

​

  • Doing this with an open source project
    • Everyone did this in its corner so far :-(

​

​

​

3 of 17

Ultimate goal?

or whatever else you want, like capacity planning or pretty usage statistics…

Billing customers!

4 of 17

Ceilometer begins

  • Starts in May 2012
    • eNovance
    • Canonical
  • Developed in StackForge
    • Same process as OpenStack
  • Minimal set of meters defined
  • Targetting OpenStack core

5 of 17

Design: base

Notification

bus

Nova

Compute

Cinder

Glance

Swift

6 of 17

Design: agent

Agent

Notification

bus

Nova

Compute

Cinder

Glance

Swift

Direct polling

7 of 17

Ceilometer Agent

  • One agent per OpenStack component
  • Polling
    • Time based
      • Swift storage usage
      • Nova instance CPU usage
    • Time granularity configurable
  • Send results to Ceilometer Collector
    • Using openstack.common.rpc
  • Extensible via plugins

8 of 17

Design: collector

Collector

Notification

bus

Nova

Compute

Cinder

DB

Glance

Swift

Agent

Direct polling

9 of 17

Ceilometer Collector

  • Listen to events from Ceilometer Agent
    • Verify message signature
  • Listen to notifications
    • From Nova
    • From Nova-volume / Cinder
    • From Glance
  • Store to database
    • Engine based, pick the one you want or implement it!
      • MongoDB (started)
      • SQLAlchemy (planned)
    • No consolidation done, raw events

10 of 17

Design: bus

Agent

Collector

Ceilometer bus

Notification

bus

Nova

Compute

Cinder

DB

Glance

Swift

Direct polling

11 of 17

Design: API

Agent

Collector

Ceilometer bus

Notification

bus

Nova

Compute

Cinder

DB

API

Glance

Swift

Direct polling

12 of 17

Ceilometer API

  • Access the database via engine
  • Export data via REST API
    • GET /v1/[SOURCES/<SOURCE>/]USERS/<USER_ID>/<METER>/VOLUME
    • GET /v1/[SOURCES/<SOURCE>/]USERS/<USER_ID>/RESOURCES/<RESOURCE_ID>/<METER>/DURATION
    • ...

​

13 of 17

Design keys

  • Scalable
    • …if your database is too
  • Message signature
    • Non-repudiation is planned
  • Only one entry point to get data
  • Extensible, add your own:
    • Agent
    • Agent plugin
    • Storage engine
    • Meters
  • Use openstack.common components

​

14 of 17

Roadmap

  • Version 1 (delivered with Folsom)
    • Collect base metering info
    • Provides basic API access
  • Version 2 (delivered with Grizzly)
    • API extension
    • Integrate with Horizon
    • New agent for other OpenStack components
      • Heat
      • Quantum
  • Version 3 (delivered with H)
    • Core project

15 of 17

Project technical status

  • Agent
    • 80 % communication with the collector
    • 60 % of Nova-compute
    • 0 % of Nova-network
    • 0 % of Swift
  • Collector
    • 80 % communication with the agent
    • 30 % database storage
    • 0 % of Nova-volume / Cinder
    • 0 % of Glance
  • API
    • 0 %

​

16 of 17

Project status

  • Now leaded by:
    • Julien Danjou
    • Dreamhost (Doug Hellmann)
    • Canonical (Nick Barcet & more to come)
  • Proposed for incubation
  • Project Technical Leader election
    • Starting today!

17 of 17

Questions?

Julien Danjou

​

julien@danjou.info

http://julien.danjou.info

Twitter: @juldanjou