EdgeNet: A Lightweight, Scalable Edge Cloud

Rick McGeer

May 30, 2019

Obligatory Ad…

Read all about it!

The Cloud as it is

EdgeNet: the Cloud as it will be

PlanetLab: The First Edge Cloud

• Peak: 1353 nodes at 717 sites worldwide
• Invented the concepts of a “slice” and “sliver”
• Over 150 publications at top-5 venues in 5 years!

• Container-based infrastructure
• Sliver: standard container at site
• Slice: network of containers

What Killed It?

• Hardware cost: $800K/year just to maintain PL hardware
• Lack of central support
• Maintaining distributed hardware infrastructures is expensive
• $5K-$15K/site/year
• Lack of onsite support
• Research boxes aren’t a local admin’s day job
• Hardware replacement/installation are slow, networks reassigned...
• Custom software/control stack
• Maintenance, documentation, training burden without value-add

Bottom Line: $10,000 site/year (PlanetLab); $50,000 site/year (GENI)

Edge Infrastructure Must Be

• Scalable: Able to grow without action by central administration
• Sustainable:
• Central Marginal costs of additional nodes → $0
• Per-site node costs must be small
• Software mostly widely-available, externally maintained, automatically improved
• Usable: Copious tools and documentation available

Four Big Takeaways

• Long-term maintenance hard to fund
• Historic record of distributed infrastructures is that funding ceases
• PlanetLab (four years), SAVI (four years), GENI (8 years, now maintenance only), G-Lab (4 years), V-Node (4 years)...
• Have to run long-term on minimal funding
• Hardware is heinous
• Breaks and needs fixing
• Bulk of the cost of both onsite and centralized support
• PlanetLab reached its maintenance limit
• Don’t write software unless you’re actually adding value
• If industry-standard stuff works, use it
• Free maintenance
• Free tutorials/answers/educational material
• Be as cheap and easy as possible for local sites

Successful Infrastructures Live Off The Land

• Use local resources
• Spread by local action: software download and configuration
• Ex: Internet used existing computers, phone lines
• Ex: Web used existing Internet and services
• FTP
• Nine-line shell script was first web server!

=

+

“Hello, World” around the world in 5 minutes

VMs Only

Sustainable ✔

Automated Install

Scalable ✔

Doumentation, Maintenance

Usable ✔

Hello, World, Around The World!

Add A Node in 1 Minute!

EdgeNet Today

What’s It Good For?

• Distributed computing across the wide area
• Low-latency services distributed worldwide
• Network experimentation and mapping
• Robust, large-scale distributed systems
• Internet observation and measurement
• ….

What’s Left to Build?

• Everything!
• 150+ papers in top-five conferences in first five years of PlanetLab
• Almost all about building services and utilities on/for PlanetLab
• We need the same thing!
• Tons of things to do…

Clouds, NSF Cloud, EdgeNet

Tomorrow’s Exercises

• Get to know EdgeNet
• “Hello, World” around the world!
• Run a real experiment
• Deploy Paris Traceroute
• Measure multilevel traceroutes from any vantage point

Preparation

• Get an EdgeNet account at http://www.edge-net.org
• Agree to the AUP
• I’ll be watching to approve
• Download your config file
• Store it in ~/.kube/config
• Play with Docker and Kubernetes tonight
• Download and install kubectl

Getting An Account

Click Here

Agree here

Click Here

Then you see this

Wait until you have a namespace allocated

Thanks to Collaborators

Eric Allman, Hadi Bannazadeh, Ilya Baldin, Andy Bavier, Andi Bergen, Mark Berman, Ketan Bhardwaj, Sushil Bhojwani, Marshall Brinn, Justin Cappos, Yvonne Coady, Chip Elliott, Timur Friedman, Sudhakar Ganthi, Ada Gavrilovska, Jim Griffoen, Matt Hemmings, John Kubiatowicz, Al Leon-Garcia, Thomas Lin, Ken Lutz, Sean McGeer, Nitesh Mor, Hausi Muller, Aki Nakao, Pat O’Connell, Riz Panjwani, Tom Mitchell, Larry Peterson, Albert Rafetseder, Glenn Ricart, Rob Ricci, Ciro Scognagmiglio, Robert Taylor, Ulrike Stege, Vic Thomas

https://www.edge-net.org