1 of 51

1

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

2 of 51

Welcome to the lecture �Applied Machine & Deep Learning (190.015)

Telefon: +43 3842 402 - 1901 �Email: teaching@ai-lab.science

Univ.-Prof. Dr. Elmar Rueckert�

​

​

WO AUS FORSCHUNG ZUKUNFT WIRD

Chair of Cyber-Physical-Systems

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

3 of 51

What do we expect from you?

3

  • Get excited by the world of Machine & Deep Learning.
  • Visit our lectures in the first week or �watch the lecture recordings put on Moodle.
  • Form a team and clone our git repository template.
  • Work on your project, submit your code and documentation.
  • Evaluate this course and discuss with us the results.

Details about the organization, the grading, the links to our services, etc. will follow after the lunch break today.

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

4 of 51

Let’s start with an introduction to machine learning.

4

Quiz on ML, please visit: https://tweedback.de/e44m

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

5 of 51

Definition of Machine Learning

5

Machine learning is a branch of artificial intelligence (AI) focusing on the design and development of algorithms that allow computers to learn from data.

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

6 of 51

Definition of Deep Learning

6

Deep learning is a subfield of machine learning that involves the use of artificial neural networks, specifically deep neural networks, to model and solve complex problems.

Yadav, Harsh; Xue, Honghu; Rudall, Yan; Bakr, Mohamed; Hein, Benedikt; Rueckert, Elmar; Nguyen, Ngoc Thinh. Deep Reinforcement Learning for Mapless Navigation of Autonomous Mobile Robot

International Conference on System Theory, Control and Computing (ICSTCC), 2023.

​

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

7 of 51

Definition of Artificial Intelligence

7

AI are algorithms for problem solving, inference and learning with the goal of mimicking human intelligence and behavior.

From Annamacharya Institute of Technology & Sciences Autonomous / Rajampet, India.

​

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

8 of 51

The Big Picture

8

Deep Learning

Machine Learning

Artificial Intelligence

Mimicking the human intelligence or behavior, or of any other living entity.

Techniques that allow machines to learn from data.

ML based on neural networks inspired by our brain’s network of neurons.

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

9 of 51

The Machine Learning Process

9

Figure by Google Cloud Tech, https://youtu.be/HcqpanDadyQ

  • Using data
    • audio, video, text docs, streams, gene dbs…
    • noise, uncertainty, errors
  • For answering questions
    • to understand the data
    • to make predictions
    • compute control signals

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

10 of 51

Types of Machine Learning

3. Behavioral or �Reinforcement Learning

2. Descriptive or Unsupervised Learning

1. Predictive or �Supervised Learning

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

11 of 51

Types of Machine Learning

3. Behavioral or �Reinforcement Learning

2. Descriptive or Unsupervised Learning

1. Predictive or �Supervised Learning

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

12 of 51

Types of Machine Learning

3. Behavioral or �Reinforcement Learning

2. Descriptive or Unsupervised Learning

1. Predictive or �Supervised Learning

  • Classification
  • Linear Regression / Function Approximation, i.e.,
  • Clustering, e.g., k-means
  • Feature learning, e.g., autoencoder
  • Dimensionality reduction, e.g., PCA
  • Decision Making
  • Policy Search
  • Deep Q-Learning
  • Bandits

​

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

13 of 51

Types of Machine Learning

13

Supervised Learning

Unsupervised Learning

Reinforcement Learning

  • Predictive Learning.�
  • Requires target values or labels for every input.�
  • Key idea of the learning algorithms is to minimize an objective.�
  • Examples: Regression: Time Series Prediction, Robot Motor Skill Learning, Clustering: Facial Detection
  • Descriptive Learning.�
  • Only unlabelled data samples are given.�
  • Key idea of the learning algorithms is to extract representative features, compress the data or identify similarities.�
  • Examples: Autoencoder, Clustering.
  • Behavioral Learning.�
  • Occasionally rewards are given for an unknown number of past actions.�
  • Key idea of the learning algorithms is to maximize a cumulative reward.�
  • Examples: Autonomous Decision Making.

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

14 of 51

Is it sufficient to just apply some tools?

14

+

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

15 of 51

Motivation for Studying Machine Learning Concepts

15

  • Domain expertise is needed to understand the data.
    • Is the data collection representative for the problem?
    • Is the data balanced?
    • Corrupted by outlier?
    • Inhomogeneous noise?

From ThyssenKrupp, Oct. 2023.

Prediction results of steelmaking defects from the Master thesis of Melanie Neubauer, M.Sc. from 2023.

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

16 of 51

Motivation for Studying Machine Learning Concepts

16

  • ML fundamentals are needed to
    • Understand, visualize and interpret the data.
    • Preprocess the data (filtering, normalization).
    • Create expressive training and test data sets.
    • Analyze the complexity of the problem.
    • Select the right model and an adequate model complexity (e.g., number of layers & neurons in neural networks).

Have a look at our book.

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

17 of 51

Motivation for Studying Machine Learning Concepts

17

  • Statistics fundamentals are needed to
    • Generate statistically significant results.
    • Validate the robustness of the model (will it work on new data?).

Typical evaluation of multiple learning methods on multiple ‘runs’, from Sheller et al. 2020.

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

18 of 51

ML Examples

18

Figure from https://www.learncomputerscienceonline.com/what-is-machine-learning/

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

19 of 51

Some Projects

19

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

20 of 51

20

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

21 of 51

21

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

22 of 51

Research Projects I

22

  • NNATT FFG 01/2024-12/2026 �2 Ph.Ds. 295k€�Smart recycling of excavated materials.�
  • K1-MET FFG 07/2023-06/206

1 Stud. Ass. 80k€

Hybrid modelling of steel casting processes.

​

  • KIRAMET FFG 07/2023-06/2026�1 Ph.D. 304k€�AI based recycling of metal composite waste.�
  • AI-Robot-Lab Infrastructure Fund 03/2022�5 Robots 120k€�Research lab - Industrie 5.0.�
  • TRAIN DFG 07/2020-01/2025�1 Ph.D. 322k€ Active transfer learning in Human-Robot Co-Worker tasks.

​

https://superior-ind.com

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

23 of 51

Research Projects

23

  • NNATT FFG 01/2024-12/2026 �2 Ph.Ds. 295k€�Smart recycling of excavated materials.�
  • K1-MET FFG 07/2023-06/206

1 Stud. Ass. 80k€

Hybrid modelling of steel casting processes.

​

  • KIRAMET FFG 07/2023-06/2026�1 Ph.D. 304k€�AI based recycling of metal composite waste.�
  • AI-Robot-Lab Infrastructure Fund 03/2022�5 Robots 120k€�Research lab - Industrie 5.0.�
  • TRAIN DFG 07/2020-01/2025�1 Ph.D. 322k€ Active transfer learning in Human-Robot Co-Worker tasks.

​

https://superior-ind.com

voestalpine Stahl GmbH

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

24 of 51

Research Projects

24

  • NNATT FFG 01/2024-12/2026 �2 Ph.Ds. 295k€�Smart recycling of excavated materials.�
  • K1-MET FFG 07/2023-06/206

1 Stud. Ass. 80k€

Hybrid modelling of steel casting processes.

​

  • KIRAMET FFG 07/2023-06/2026�1 Ph.D. 304k€�AI based recycling of metal composite waste.�
  • AI-Robot-Lab Infrastructure Fund 03/2022�5 Robots 120k€�Research lab - Industrie 5.0.�
  • TRAIN DFG 07/2020-01/2025�1 Ph.D. 322k€ Active transfer learning in Human-Robot Co-Worker tasks.

​

https://superior-ind.com

voestalpine Stahl GmbH

Particle Tracking, CPS 2023

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

25 of 51

Research Projects

25

  • NNATT FFG 01/2024-12/2026 �2 Ph.Ds. 295k€�Smart recycling of excavated materials.�
  • K1-MET FFG 07/2023-06/206

1 Stud. Ass. 80k€

Hybrid modelling of steel casting processes.

​

  • KIRAMET FFG 07/2023-06/2026�1 Ph.D. 304k€�AI based recycling of metal composite waste.�
  • AI-Robot-Lab Infrastructure Fund 03/2022�5 Robots 120k€�Research lab - Industrie 5.0.�
  • TRAIN DFG 07/2020-01/2025�1 Ph.D. 322k€ Active transfer learning in Human-Robot Co-Worker tasks.

​

https://superior-ind.com

voestalpine Stahl GmbH

Particle Tracking, CPS 2023

Robot LAB CPS

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

26 of 51

Research Projects

26

  • NNATT FFG 01/2024-12/2026 �2 Ph.Ds. 295k€�Smart recycling of excavated materials.�
  • K1-MET FFG 07/2023-06/206

1 Stud. Ass. 80k€

Hybrid modelling of steel casting processes.

​

  • KIRAMET FFG 07/2023-06/2026�1 Ph.D. 304k€�AI based recycling of metal composite waste.�
  • AI-Robot-Lab Infrastructure Fund 03/2022�5 Robots 120k€�Research lab - Industrie 5.0.�
  • TRAIN DFG 07/2020-01/2025�1 Ph.D. 322k€ Active transfer learning in Human-Robot Co-Worker tasks.

​

https://superior-ind.com

voestalpine Stahl GmbH

Particle Tracking, CPS 2023

Robot LAB CPS

CPS TRAIN

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

27 of 51

Industrial Projects

27

  • Ottronic GmbH �Masterthesis�Process modeling of thermoset resins.�
  • voestalpine Böhler GmbH

Bachelorthesis

Deep learning for manufacturing time predictions from 3D CAD files.

​

  • qoncept dx GmbH�Masterthesis, FFG project (under review)�Optimization of metralurgical processes.�
  • voestalpine Stahl GmbH�Masterthesis, K1-MET project�Computer vision for metralurgical processes.

​

  • Stahl- und Walzwerk Marienhütte GmbH�Contract research project�Deep learning for predicting yield strengths.�
  • Firma Geislinger GmbH�Contract research project�Deep Learning for commissioning.

​

  • LUPA Electroinics GmbH �Bachelorthesis�Autonomous driving, sensor fusion.�
  • Theresianische Militärakademie �260k€ 2-year Project�Multi-modal, AI-based gesture recognition �system.

�

​

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

28 of 51

Probabilistic Robot Learning

  • The goal: Combine tactile and proprioceptive sensory data for learning motor skills.�
  • The model: A powerful Gaussian mixture model for multi-modal data.�
  • The results: Complex grasping skills can be inferred solely by conditioning on desired future tactile sensations.

Dave, Vedant; Rueckert, Elmar. Predicting full-arm grasping motions from anticipated tactile responses. International Conference on Humanoid Robots (Humanoids 2022), 2022.

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

29 of 51

Autonomous Navigation and Mapping

29

  • The hospital contacted us and wanted to buy our system.�
  • The goal: Develop an mobile robot to guide hospital patients to their room. �
  • Subgoals are to carry bags, files, communicate, use the elevators, etc.

Nwankwo, Linus; Fritze, Clemens; Bartsch, Konrad; Rueckert, Elmar. O2S: Open-source open shuttle. Journal Article under review, 2022.

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

30 of 51

Startup on Autonomous Hospital Guides

30

Proof of Concept (done)

Technology demonstrator (Aug/2023)

Research

Technology

Commercial Partner

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

31 of 51

Startup on Autonomous Hospital Guides

  • Interesting Research Problem: All state-of-the-art algorithms fail to create accurate maps.
  • Our approach is to use floor maps as prior.

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

32 of 51

Outdoor Navigation, Mapping & LLMs

32

Dynamic obst.

Elevated terrain

Start pose

  • Terrain profile not correctly estimated
  • Pose drift due to dynamic obstacles
  • Light penetration & reflectivity
  • Increased uncertainty in the motion estimation.

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

33 of 51

33

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

34 of 51

Industrial Process Modelling

34

  • The goal: Predict a product quality factor (a scalar) from sensory data of the production line.
  • Our Results: Deep Learning approach predicts the metric with an accuracy of < 5% for 95% of the data.

Industrial Project with the Stahl- und Walzwerk Graz Marienhütte

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

35 of 51

AI-based Recycling of Steel Scrap (KIRAMET)

35

  • The steel production industry is face hard challenges in the energy costs, available resources and the climate targets.�
  • The goal of this project is to enhance the re-use of secondary steel scrap for the production of high quality steel products. �
  • Our contribution is the development of cloud databases, computer vision algorithms and smart annotation tools to detect, classify and track particles.

​

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

36 of 51

36

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

37 of 51

37

Rueckert, Elmar; Kappel, David; Tanneberg, Daniel; Pecevski, Dejan; Peters, Jan

Recurrent Spiking Networks Solve Planning Tasks Journal Article

In: Nature Publishing Group: Scientific Reports, vol. 6, no. 21142, 2016.

​

​

Tanneberg, Daniel; Paraschos, Alexandros; Peters, Jan; Rueckert, Elmar

Deep Spiking Networks for Model-based Planning in Humanoids Proceedings Article

In: Proceedings of the International Conference on Humanoid Robots (HUMANOIDS), 2016.

​

​

Tanneberg, Daniel; Peters, Jan; Rueckert, Elmar

Intrinsic Motivation and Mental Replay enable Efficient Online Adaptation in Stochastic Recurrent Networks Journal Article

In: Neural Networks – Elsevier, vol. 109, pp. 67-80, 2019, ISBN: 0893-6080, (Impact Factor of 7.197 (2017)).

​

​

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

38 of 51

38

A Neuroinspired Plannig Approach

Pfeiffer, B. & Foster, D. Hippocampal place-cell sequences depict future paths to remembered goals. Nature 497, 74–79 (2013).

​

​

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

39 of 51

A Neuroinspired Planning Approach

39

Planning as Inference

Planning in Spiking Neural Networks

xt

xT

Learning

Inference

Adaptive/Dynamic Constraints

xt+1

Proof for optimal planning as inference in recurrent neural networks!

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

40 of 51

A Neuroinspired Planning Approach

40

Rueckert, Elmar; Kappel, David; Tanneberg, Daniel; Pecevski, Dejan; Peters, Jan. Recurrent Spiking Networks Solve Planning Tasks. Nature Publishing Group: Scientific Reports, 6 (21142), 2016.

The Model:

  • Supervised learning to learn the dynamics model (the effect of the control commands on the state)!�
  • Reinforcement learning to learn optimal strategies (motion plans through local Hebbian learning)!�
  • Competition through local inhibition among neurons (the best neuron wins)!�

Sensor/Motor Inputs

System Dynamics

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

41 of 51

A simple Motion Planning Example

41

Rueckert, Elmar; Kappel, David; Tanneberg, Daniel; Pecevski, Dejan; Peters, Jan. Recurrent Spiking Networks Solve Planning Tasks. Nature Publishing Group: Scientific Reports, 6 (21142), 2016.

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

42 of 51

Tanneberg, Daniel; Peters, Jan; Rueckert, Elmar. Intrinsic Motivation and Mental Replay enable Efficient Online Adaptation in Stochastic Recurrent Networks. Neural Networks - Elsevier, 109 , pp. 67-80, 2019, ISBN: 0893-6080, (Impact Factor of 7.197 (2017)).

​

Real-Time Planning & Control

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

43 of 51

Reinforcement learning of motion plans

Supervised state transition model learning

Real-Time Planning & Control

Factorized Population Codes

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

44 of 51

44

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

45 of 51

45

Xue, Honghu; Hein, Benedikt; Bakr, Mohamed; Schildbach, Georg; Abel, Bengt; Rueckert, Elmar

Using Deep Reinforcement Learning with Automatic Curriculum Learning for Mapless Navigation in Intralogistics Journal Article

In: Applied Sciences (MDPI), Special Issue on Intelligent Robotics, 2022.

​

Yadav, Harsh; Xue, Honghu; Rudall, Yan; Bakr, Mohamed; Hein, Benedikt; Rueckert, Elmar; Nguyen, Ngoc Thinh

Deep Reinforcement Learning for Mapless Navigation of Autonomous Mobile Robot Proceedings Article

In: International Conference on System Theory, Control and Computing (ICSTCC), 2023, (October 11-13, 2023, Timisoara, Romania.).

​

Yadav, Harsh; Xue, Honghu; Rudall, Yan; Bakr, Mohamed; Hein, Benedikt; Rueckert, Elmar; Nguyen, Thinh

Deep Reinforcement Learning for Autonomous Navigation in Intralogistics Workshop

2023, (Extended Abstract.).

​

Xue, Honghu; Song, Rui; Petzold, Julian; Hein, Benedikt; Hamann, Heiko; Rueckert, Elmar

End-To-End Deep Reinforcement Learning for First-Person Pedestrian Visual Navigation in Urban Environments Proceedings Article

In: International Conference on Humanoid Robots (Humanoids 2022), 2022.

​

​

​

​

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

46 of 51

46

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

47 of 51

47

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

48 of 51

Reinforcement Learning

Movement Primitives

Humanoid Robotics

Spiking Neural Networks

Intrinsic Motivations

Human-Machine Int.

Robotics Health Care

Postural Control

Muscle Synergies

Tactile Learning

Model Learning

Probabilistic Inference

Probabilistic Computational Neuroscience

​

Medical Robotics and Human Motor Control

​

Probabilistic Robot Learning

​

Stochastic Machine and Deep Learning

​

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

49 of 51

49

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

50 of 51

Course Materials, Links & Literature

50

Other ML books

  • Kevin P. Murphy 2012. Machine Learning: A Probabilistic Perspective, MIT Press.
  • Bishop 2006. Pattern Recognition and Machine Learning, Springer.
  • Barber 2007. Bayesian Reasoning and Machine Learning, Cambridge University Press.
  • Kristian Kersting, Christoph Lampert, Constantin Rothkopf. 2019. Wie Maschinen lernen, Springer.
  • James-A. 2020. Goulet Probabilistic Machine Learning for Civil Engineers, MIT Press.

​

Video Lectures:

  • videolectures.net on Machine Learning
  • coursea.org on AI

​

free online version

At our library

free online version

https://bibliothek.unileoben.ac.at/en/

At our library

At our library

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS

51 of 51

Thank you for your attention!

Visit our Youtube Channel: https://youtube.com/@CPSAustria

​

​

Phone: +43 3842 402 – 1901 (Sekretariat CPS)

Email: cps@unileoben.ac.at

Web: https://cps.unileoben.ac.at

​

Disclaimer: The lecture notes posted on this website are for personal use only. The material is intended for educational purposes only. Reproduction of the material for any purposes other than what is intended is prohibited. The content is to be used for educational and non-commercial purposes only and is not to be changed, altered, or used for any commercial endeavor without the express written permission of Professor Rueckert.

MONTANUNIVERSITÄT LEOBEN

CYBER-PHYSICAL-SYSTEMS