Distributed Edge Compute PoC

The Opportunity

We understand that with increasing IoT deployments, we will see an increase in local compute power. However, IoT applications are not designed to run 100% of the time, which results in the waste of computing resources.

We believe that we can create a distributed edge compute solution to efficiently use compute resources that are readily available to us.

As a result of using our solution, our clients will realize a benefit of cost reduction.

This goal of this POC is to create a distributed edge computing solution that will::

• Lower the cost of running applications by using devices’ processing power when they are idle or when they have additional capacities.
• Improve application performance and ensure data locality by processing closer to the users.

System Overview

The overall system comprises of three big architecture components. The orange layer Is the core that enables this distributed compute platform on the edge.

​

Hardware:

• user has the ability to scale compute resources by adding more nodes onto the cluster

​

• user can define limits on resources to be utilized by the cluster*

​

Core:

• Microk8s is a lightweight distribution of kubernetes from canonical

​

• three key services were enabled as part of the core set up:
• metalb: load balancing of incoming request to the cluster
• Prometheum & grafana are use for monitoring and dashboarding

​

Deployments:

• Deployments of application are done through containers

​

Workload Workflow example

1

Master Server: used to intake data and distribute data to each of the node for processing

2

Nodes: These are IoT devices, with idle compute resources, used to process the data. Resource allocations would be control to ensure primary workloads are not disrupted

Consolidation Server: processed data would be consolidated and send to the requested client

3

Demo

We containerized an object detection model and deploy it onto the platform. The detection model is running on all three nodes.

​

Project Outcomes

Outcomes from deploying detection model onto the platform:

• Data is processed and kept local
• Improvement on performance would be highly dependent on the application. In theory, by running application locally we should see an increase in performance; but we did not test a cloud-based scenario in this prototype.
• The theoretical cost should be lower when running application locally vs on the cloud; however, same setup should be replicate onto the cloud along with cost analysis to validate this hypothesis.

​

​

Other Learnings

Few observations worth noting as we worked through this prototype...

• The performance difference between ARM & x86 is significant in the case of tensorflow serving model.
• The user can limit resource on the each deployment, but ideally, the platform would allow user to limit resource constraint on a node level.
• It is not recommended to have a hybrid cluster, like the one we have in this prototype, because there will be performance differences between different hardware architecture

​

Next Steps

• Deploy and run same tests on the cloud to validate cost-saving and performance improvement.
• Experiment with integration of this edge platform with cloud platforms (the idea is so data can be process locally on this edge platform and would only send relevant information to the cloud)