1 of 18

Production Grade Big Data

Cory Johns, Kevin Monroe

​

2 of 18

Warm the cache!

  • SSH us! (it’ll fail, and that’s OK)
    • We’ll use these hits in our live demo
    • Be professional!

​

​

​

​

​

​

​

​

  • ssh <FUNNY>@juju.does-it.net

3 of 18

Juju & Big Data

  • Modelling language for service oriented environments

​

    • Deploy, relate, scale
    • Reliable
    • Repeatable
    • Observable

4 of 18

Plugging in to Hadoop

juju quickstart apache-core-batch-processing

5 of 18

Add Apache Flume

juju quickstart u/bigdata-dev/apache-ingestion-flume

6 of 18

Add Apache Kafka

juju quickstart u/bigdata-dev/apache-flume-ingestion-kafka

7 of 18

Add Apache Pig

juju quickstart apache-analytics-pig

8 of 18

Add Apache Hive

juju quickstart apache-analytics-sql

9 of 18

Add Apache Spark

juju quickstart apache-hadoop-spark

10 of 18

Add iPy Notebook

juju quickstart apache-hadoop-spark-notebook

11 of 18

Add Apache Zeppelin

juju quickstart apache-hadoop-spark-zeppelin

12 of 18

Focus on the Science

13 of 18

Syslog Analytics

juju quickstart realtime-syslog-analytics

​

14 of 18

Production Grade

15 of 18

Production Grade

  • Big Data focus areas for us:

​

    • High Availability
    • Security
    • Scalability (> 1000 nodes)

16 of 18

The Struggle is Real

  • Tell us about your workflow
  • Big Data Friction
  • Missing Big Data applications

​

​

​

​

​

​

​

​

​

  • Improving charm offerings helps the entire Big Data community!

17 of 18

Resources

18 of 18

Thanks!