1 of 21

2 of 21

WP2 - AI

  • Collect training data in each school, in mobilities, in trips etc (20+ students involved)
    • Garbage
    • Non-garbage
  • AI model trained on our common collection
  • Collection to upload for free use
    • Result: >1000 training images collected by students in all schools

3 of 21

4 of 21

1629 images gathered

> 1000

5 of 21

Split them into training & test sets 80%-20%

6 of 21

Split them into training & test sets 80%-20%

Training set

Test set

7 of 21

Split them into training & test sets �80%-20%

8 of 21

Demo

teachablemachine.withgoogle.com/

9 of 21

AES - Silves

  • Overall Accuracy: 79.76%

​

    • Garbage Accuracy: 92.73% (51/55)

​

    • Non-garbage Accuracy: 55.17% (16/29)

10 of 21

AES - Silves

  • Mistaken cases

​

  • Garbage : 51/55

​

  • Non-garbage 16/29

11 of 21

EJH - Alexandroupolis

  • Overall Accuracy: 80.72%

​

    • Garbage Accuracy: 62.07% (18/29)

​

    • Non-garbage Accuracy: 90.74% (49/54)

12 of 21

EJH �Alexandroupolis

  • Mistaken cases

​

  • Garbage: 18/29
  • Non-garbage: 49/54

​

13 of 21

EBCC - Madeira

  • Overall Accuracy: 80.2%

​

    • Garbage Accuracy: 76.19% 32/42

​

    • Non-garbage Accuracy: 83.05% 49/59

14 of 21

EBCC �Madeira

  • Mistaken cases
  • Garbage: 32/42
  • Non-garbage: 49/59

​

15 of 21

SGME - Craiova

  • Overall Accuracy: 90.16%

​

    • Garbage Accuracy: 90.32% (28/31)

​

    • Non-garbage Accuracy: 90.0% (27/30)

16 of 21

SGME - Craiova

  • Mistaken cases

​

  • Garbage: 28/31

​

  • Non-garbage: 27/30

​

​

17 of 21

Overall

  • Gathered train images from all schools

​

  • Gathered test images from all schools

​

  • We trained a new model

18 of 21

Overall

  • Overall Accuracy: 79.33%

​

    • Garbage Accuracy: 74.52% (117/157)

​

    • Non-garbage Accuracy: 83.72% (144/172)

19 of 21

Trained on data from one school, applied to the entire test set

Training from

Garbage

Non-garbage

Overall

Accuracy

Accuracy

Correct/All

Accuracy

Correct/All

AES

87.90%

138/157

54.07%

93/172

70.21%

EJH

62.42%

98/157

87.79%

151/172

75.68%

EBCC

75.16%

118/157

77.33%

133/172

76.29%

SGME

89.81%

141/157

56.4%

97/172

72.34%

All schools

74.52%

117/157

83.72%

144/172

79.33%

Together we get the best overall accuracy result!

20 of 21

Why the wrong cases? The model’s fault?

(Ambiguous) samples from the training set...

Non-Garbage

Garbage

21 of 21

Conclusions – lessons learned

  • Good balance between classes is important.
    • More images in one class makes the model advantage it.
  • The AI needs accurate images for each class.
    • The model gets confused if we label similar images as either garbage or

non-garbage

  • The model that has more samples to learn from is more robust.
  • You were able to train your first AI model!
    • From gathering data, uploading them based on classes, training & testing an AI model!

​