Samo-prilagodljive �pametne storitve��Self-Adaptive �Smart Services
1
Samo-prilagodljive pametne storitve imajo vgrajene mehanizme, ki jim omogočajo, da spreminjajo režim svojega delovanja z namenom doseganja različnih ciljev.
Self-adaptive smart services have built-in mechanisms that allow them to change the regime of their operation in order to achieve different goals.
2
Definicija
Definition
3
ref.: D. Weyns, Software Engineering of Self-Adaptive Systems: An Organised Tour and Future Challenges, Chapter for Handbook of Software Engineering, Editors: Richard Taylor, Kyo Chul Kang and Sungdeok Cha, Springer 2017
Šest valov samo-prilagodljivih sistemov
Six waves of self-adaptive systems
4
ref.: D. Weyns, Software Engineering of Self-Adaptive Systems: An Organised Tour and Future Challenges, Chapter for Handbook of Software Engineering, Editors: Richard Taylor, Kyo Chul Kang and Sungdeok Cha, Springer 2017
Načela samo-prilagodljivih sistemov
Principles of self-adaptive systems
A system that can independently manage changes and uncertainties in its environment, the system itself and its goals (i.e. without or with minimal intervention).
The system consists of two different parts: the first part interacts with the environment and addresses the requirements or realizes the goals for which it was built; the second part interacts with the first, monitors its environment and is responsible for its adaptation.
5
ref.: D. Weyns, Software Engineering of Self-Adaptive Systems: An Organised Tour and Future Challenges, Chapter for Handbook of Software Engineering, Editors: Richard Taylor, Kyo Chul Kang and Sungdeok Cha, Springer 2017
Konceptualni model samo-prilagodljivega sistema
Conceptual model of a self-adaptive system
6
MAPE-K
link: J. Kephart and D. Chess. The Vision of Autonomic Computing. Computer, 36(1):41–50, 2003.
Structure of an autonomous manager
Struktura avtonomnega upravitelja
7
Pristopa k reševanju problemov adaptacije
Approaches to solving adaptation problems
A proactive approach
focuses on fixing problems before they occur.
A reactive approach
is based on responding to events as they happened.
8
Pametne storitve in njihove zahteve
Smart services and their requirements
Več primerov samo-prilagodljivih aplikacij: https://www.hpi.uni-potsdam.de/giese/public/selfadapt/exemplars/model-problem-znn-com/
9
{
"bounding-box": "350,287,50,213",
"classification": "person",
"helmet": "yes"
}
AI models
People detection
Podatki, metapodatki, modeli UI, metode in digitalni dvojčki
Data, metadata, AI models, methods and digital twins
10
11
Računalništvo v megli
Fog Computing
12
DataMiningGrid (2004-2007)
Arhitektura
Testno okolje
FHG
UL
UU
~80 računalnikov
~40 računalnikov
~5 računalnikov
Delovanje
Razvojno okolje DataMiningGrid
17
mOSAIC (2010-2013)
Primera uporabe
Bent
Helix
Aplikacija
REQ.
HTML
20
ENTICE (2015-2018)
ENTICE VM Image Portal
21
LEFT IMAGE: Environmental statistics and storage statistics.
RIGHT IMAGE: Form for launching Virtual Images covering all industrial partners use cases specific attributes.
Več kriterijska optimizacija
22
23
SWITCH (2015-2018)
Architecture & Workflow of Decision Making process
Software Workbench for Interactive, Time Critical and Highly self-adaptive Cloud applications
24
Software Workbench for Interactive, Time Critical and Highly self-adaptive Cloud applications
25
Project meeting Porto, 1-3 December 2015
Early Warning System
26
Štefanič, P., Kimovski, D., Suciu, G., Stankovski, V. Non-functional requirements optimisation for multi-tier cloud applications: An early warning system case study, https://ieeexplore.ieee.org/document/8397637
27
DECENTER (2018-2021)
Research and Innovation Aspects
28
Define resource models and associated SLAs for in-border and cross-border services
Develop a robust multi-tier fog platform to manage cloud-to-edge resources
Define and implement resource orchestration strategies to satisfy app requirements
Implement blockchain-based smart contracts and probing tools to validate their fulfillment
Privacy-preserving mechanism to represent users’ context
Facilitate interoperable IoT device management in the infrastructure
Develop hierarchical methods to map AI algorithms from the cloud to the edge
Develop application data management over the infrastructure for cross-border AI
Validate DECENTER innovations, and demonstrate them in real world pilots
Promote the standardisation and industrial application of DECENTER results
DECENTER Architecture
29
30
DECENTER Fog and Brokerage Platforms
Brokerage Platform
Fog Computing Platform
Includes all the virtualized environment to recognize, abstract, expose, share and orchestrate resources from cloud to edge in support to AI applications
Expected Benefits
31
Issues
Computing latency
Bandwidth limitations
Resource availability �and ownership
Data privacy
Decentralised fog/edge computing offers timely computation for AI applications
Distributing AI in the fog reduces need of bandwidth between cloud and edge
Smart contracts enable short-lived and trusted resource sharing between providers
Data fusion, filtering, and digital reality provided at the edge allow to preserve confidential data
…
DECENTER Fog and Brokerage Platforms
32
Fog Computing
Brokerage
QoS & SLA & monitoring
Security & Robustness
Architecture
Orchestration
IoT fabric
Data brokerage @ edge
Resource Federation
Design
Implementation: Brokerage Platform�Initial tech scenario demonstration�
DECENTER services and tools
33
AI preparation
AI delivery
Model preparation
Model repository
AI into microservices
DECENTER base container (deliver AI Service)
Digital twin
AI App
Output
Distributed AI App
Platform
Implementation of Containerized AI
34
Base Container Implementation
Implementation of VGG16 Microservice
Implementation of Yolov3 Microservice
AI functionalities exposed with RESTful API
DECENTER Fog and Brokerage Platforms
35
Resource owner / �Infrastructure provider B
Interface 1
Application Composer GUI
Orchestrator A
INFR. A (Cloud+Fog)
Resource Selector
Ethereum Blockchain
IaaS Manager
Reosurce Manager
Orchestrator B
INFR. B (Cloud+Fog)
IaaS Manager
Resource Manager
Resource Seller
Resource Seller
REB_A
REB_B
Interface 1
Interface 1
Interface 3
Resource owner / �Infrastructure provider A
Resource Exchange Broker
Interface 2
Brokerage Platform
Fog Computing Platform(s)
/
Monitoring System
SLA Manager
Monitoring System
Expected benefits��- Saving Time�- Saving Money�- Energy Efficiency and Sustainability�- Quality Working Conditions�- Safer Working Conditions�- Improved Planning�- Early Disaster Warning�- … �
Štefanič, M., and Stankovski, V. A review of technologies and applications for smart construction. Proceedings of the Institution of Civil Engineers – Civil Engineering, https://doi.org/10.1680/jcien.17.00050
A Review of Technologies and Applications for Smart Construction
Distributed AI Applications for Construction and Civil Engineering in General
37
38
Supporting Smart Construction with Dependable Edge Computing Infrastructures and Applications
39
Kochovski, P., Stankovski, V. Supporting smart construction with dependable edge computing infrastructures and applications, Automation in Construction, DOI: 10.1016/j.autcon.2017.10.008
Documenting
Collaborative engineering
Setting Up a Smart & Safe Construction Site
40
Construction Site Smart Management
Planning
Precision
Monitoring
Tracking
Progress
Process
Budget
Collaboration
Type of material
Quantity
Notification
Notification
Smart Management of Resources and Assets
Construction worker
Supervisor
Construction site manager
How many workers are currently at the site?
Was the construction site manager present at 12:00 CET today and where?
Design study: 2D floor plan. Placing objects and persons on a 2D floor plan or 3D model.
Person without a helmet in safety zone
Splitting the site map into zones of interest, e.g. danger zones.
Observing the Site
The worker observed in the video:
44
PRIVACY!
SECURITY
SAFETY
Safety at Work: Helmet Detection
45
25
Example: TensorFlow
Helmet
detection
Car plate
number
recognition
Material
type recognition
……
Deep Learning in the Fog
Using AI (Deep Learning)
47
{
"bounding-box": "350,287,50,213",
"classification": "person",
"helmet": "yes"
}
AI models
People detection
Use Case Examples – Easy or Difficult?
48
The system shall be able to identify if the workers are wearing a helmet and a safety vest.
The system shall be able to identify the quantity/assets of materials. The system shall identify peoples (workers/visitors).
EASY
DIFFICULT
Use Case Examples – Easy or Difficult?
49
The system shall be able to identify the number of workers and the color of their helmets.
The system shall be able to identify people‘s position in a restricted area.
EASY
DIFFICULT
Use Case Examples – Easy or Difficult?
50
The system shall be able to identify the type of a vehicle and the number on its licence plate.
The system shall be able to identify the type of the vehicle and direction of its movement.
EASY
DIFFICULT
Video Stream AI Processing Pipeline
51
Video stream ingestion
Knowledge Base
Stream preprocessing
Features processing
Notifications
Connection
Connection through Pub/Sub, MQ, buffer
AI models
Preprocessors
Augmented Virtuality: From Reality to Augmented BIM
52
Construction site
Building Information Model
Phase I
Phase II
Phase V
Construction progress tracking
27
Is everyone wearing a helmet?
If not, ring the alarm bell.
Is everyone wearing an uniform?
If not, ring the alarm bell.
What type of building materials are currently on site?
Bricks should be stored.
If found, notify the site engineer.
Are there any unregistered cars on the site?
If yes, notify the security.
Is anyone unhappy?
Who is the unknown visitor?
Are there any bricks around?
Integrating the information
Digital Twin
Air temperature is exceeding working limits, notify the site engineer.
CO2 Emission levels are higher than it should be, notify the site engineer.
Possible colision between crane and electric wires detected, notify the site engineer.
Earthquake warning recieved from earthquake observatory, ring the emergency alarm bell.
Smart and Safe Construction App
54
Single AI App Designed to Run in Two Tiers
55
TIER 2
TIER 1
Data Pre-processor
Data Post-processor
AI Method (AI Model Rear Part)
Backend
50 Mbps
50 Mbps
10 FPS
(0.1 s/frame)
10 FPS
(0.1 s/frame)
5.3 Mbps
10 FPS
(0.1 s/frame)
IoT Frontend
4k@60 FPS
H.265 (HEVC)
MP4 or MKV
10-20 Mbps
180 Mbps
10 FPS
(0.1 s/frame)
10 FPS
(0.1 s/frame)
50 Mbps
10 FPS
(0.1 s/frame)
Data Pre-processor
AI Method (AI Model Front Part)
Data Post-processor
The Internet
Pod
Pod
Box = container
Pod = deployment unit
boxes = AI solution service
boxes = Platform services
boxes = AI solution service, deployed separately
YOLOv3 study AI model
Deployment Assumption:
two-component & two-tier application
50 Mbps
64 kbps
The person is wearing a helmet.
Direction of moving veichle identified.
The person is wearing a uniform.
License plate
İdentified.
Type of material identified from texture.
0.00 s
0.15 s
0.12 s
0.10 s
0.16 s
0.17 s
0.24 s
0.21 s
0.20 s
0.27 s
0.30 s
0.32 s
57
Functional (FR)
System (SR)
Non-Functional (NFR)
SR
23,3%
NFR
30,1%
FR
46,6%
Summary of Requirements
58
Zaključki
59
Cross-border AI management
Application scenario
60
STOP C1
START C2
STOP C2
START C4+C5
Orchestrator
alarm
Backend
AI MODEL REPOSITORY
SERVICE
REPOSITORY
BLOCKCHAIN
(SC execution)
61
Where to find us?