1 of 28

What’s behind EmberObserver’s Scores?

March 20, 2019 EmberConf, Portland, OR

@katiegengler

2 of 28

About me

codeallday.com

About me

@katiegengler

3 of 28

EmberObserver

EmberObserver.com

About me

@katiegengler

4 of 28

EmberObserver

EmberObserver Score

1 - Sustainability

1 - Popularity

1 - Interest

2 - Maintained

2 - Being open source

1 - Tests

1 - Documentation

1 - Running those tests

About me

@katiegengler

This is a slide from a previous talk I gave. The score was a fixed set of checks adding up to 10!

This is why it should be meaningful to you. It’s NOT just me assigning a number.

Here is the point breakdown, or at least what the metrics are meant to represent.

1 - Sustainability, which comes from looking at the # of contributors,

1 - Popularity - whether it was in the top 10% of downloads,

1 - Interest - whether it was in the top 10% of github stars,

2 - Maintained - having been released in the last 3 months and having more than 1 commit in the last 3 months),

2 - Open Source - for being open source and not an empty addon, these will likely be replaced by other metrics

1 - for having meaningful tests, a manual check

1 - for having CI build to run those tests

1 - for having meaningful documentation.

The ones in bold are the only ones that are manually done.

5 of 28

EmberObserver

Can you score on accessibility?

About me

@katiegengler

6 of 28

EmberObserver

No :-/

About me

@katiegengler

7 of 28

EmberObserver

Can you encourage CHANGELOG updates through the score?

About me

@katiegengler

8 of 28

EmberObserver

No :-/

About me

@katiegengler

9 of 28

EmberObserver

Can you score addons

on whether they are `Octane-ready`?

About me

@katiegengler

10 of 28

EmberObserver

No :-/

About me

@katiegengler

11 of 28

EmberObserver

MATH

About me

@katiegengler

12 of 28

EmberObserver

Weighted Average

About me

@katiegengler

13 of 28

EmberObserver

In programmer speak

For each score component:

normalizedValue = (checkValue/maxCheckValue)

weightedValue = normalizedValue * weight

About me

@katiegengler

14 of 28

EmberObserver

In programmer speak

weights = checks.map((c) => c.weight));

weightedValues = checks.map((c) => c.weightedValue)

weightedValues.sum()

weights.sum()

x

10

About me

@katiegengler

15 of 28

EmberObserver

Example

(1*1) + (1*1) + (1*1)

(1 + 1 + 1)

x

10

About me

@katiegengler

16 of 28

EmberObserver

Before

About me

@katiegengler

17 of 28

EmberObserver

EmberObserver Score

1 - Sustainability

1 - Popularity

1 - Interest

2 - Maintained

2 - Being open source

1 - Tests

1 - Documentation

1 - Running those tests

About me

@katiegengler

18 of 28

EmberObserver

After rebalance

About me

@katiegengler

19 of 28

EmberObserver

After rebalance

About me

@katiegengler

20 of 28

EmberObserver

Reweighting

About me

@katiegengler

21 of 28

EmberObserver

Reweighted

About me

@katiegengler

22 of 28

EmberObserver

Smarter checks

and partial credit

About me

@katiegengler

23 of 28

EmberObserver

New Checks

About me

@katiegengler

24 of 28

EmberObserver

New Checks

Non-default description

About me

@katiegengler

25 of 28

EmberObserver

New Checks

Maintainer Count

About me

@katiegengler

26 of 28

EmberObserver

New Checks

> 1.0

About me

@katiegengler

27 of 28

EmberObserver

About me

@katiegengler

28 of 28

The Future

March 20, 2019 EmberConf, Portland, OR

@katiegengler