1 of 20

Changing paradigms in AI prototyping

From Minimum Viable Product to a First Draft approach

Carlos Toxtli

@ctoxtli

2 of 20

3 of 20

Minimum “Viable” Product

4 of 20

Expectations in AI

Reality

@ctoxtli

5 of 20

Why?

  • Conventional solutions are deterministic

1 + 1 = 2 (100% sure)

  • AI solutions are stochastic (randomness)

There is a cat in the image (86% sure) ???

@ctoxtli

6 of 20

So how can we effectively prototype AI?

Lets learn from academia

@ctoxtli

7 of 20

8 of 20

9 of 20

10 of 20

In fact, Data scientists currently code in notebooks

11 of 20

Hypothesis and baseline definition

You want to prove that your proposed algorithm is better than existing ones.

Try with the simplest algorithms first.

@ctoxtli

12 of 20

13 of 20

Common AI pipeline in Industry

  • Collect data (tons of data)
  • Pre process data (refine the data to its most optimal representation)
  • Find best approach to implement (usually the state-of-the-art)
  • Implement the algorithm (usually using a published code)

@ctoxtli

14 of 20

First Draft AI pipeline

  • Collect data (small data but well stratified and the most discriminative)
  • Pre process data (simplest data cleaning)
  • Find the simplest approach (use vanilla versions of algorithms or one-shot learning)
  • Implement the algorithm (use a widely and easy to use framework i.e. sklearn, keras)

15 of 20

16 of 20

Quick iterations

  • Fine tune it (hyper parameters)
  • Try several already implemented ML algorithms and ensembled models.
  • Evaluate and create a benchmark

@ctoxtli

17 of 20

Define your error mitigation strategy

  • Step 1: Try a mechanism to give a solution when accuracy is low (i.e. human-in-the-loop, heuristics)
  • Step 2: Try both (AI model and mitigation strategy) triggered by a certain accuracy threshold.
  • Step 3: Try the UX when the system is only driven by AI.
  • Evaluate the UX of AI vs Mitig vs AI+Mitig

@ctoxtli

18 of 20

Hey! now you have a working solution

  • This version 0 would be super inefficient but the model and the mitigation process are implemented.
  • Implementing the state-of-the-art algorithm will be a simple module replacement task.

@ctoxtli

19 of 20

Conclusions

  • An AI MVP may never see the light if we are looking for a "Viable" first version.
  • Having an implemented first draft of an AI model and a mitigation strategy can boost the development of robust AI solutions.

20 of 20

Thanks

​

http://www.carlostoxtli.com

​

@ctoxtli