Experiment Quality Scorecard

JavaScript isn't enabled in your browser, so this file can't be opened. Enable and reload.

The scoring guidance is provided on the premise that our experiment design template has been filled i.e. basic experiment setting are in place (hypothesis, power, min detectable effect, etc).

Experiment id (Please use exact value in "Test name" as shared in AB Test Run Log)

Ideation & Planning

Problem statement (Source of the experiment idea) *

A) Random idea or from a top-down mandate with unclear user value: SCORE -10

B) Why we want to run this specific experiment needs further clarity: SCORE -5

C) The situations described in options (A) and (B) don’t apply : SCORE 0

D) Option (C) holds true, and the problem statement builds off prior data analysis or UX research insights: SCORE 10

Remarks (if any)

Experiment hypothesis (Quality of the experiment idea) *

A) Not defined or non-falsifiable: SCORE -10

B) Describes the product change to be evaluated and expected impact: SCORE 0

C) Option (B) holds true, and the hypothesis addresses a proven user pain/need: SCORE 10

Remarks (if any)

Rigour of Decision matrix *

A) There are no success metrics OR a decision plan is missing for an outcome: SCORE -10

B) Situation described in option (A) don’t apply, but there are more than two success metrics OR missing guardrail metrics: SCORE -5

C) Decision plan is to ship even if success metric doesn't improve or has stat. insignificant change: SCORE 0

D) Decision plan is to ship only if success metric improves with stat. significant change: SCORE 10

Remarks (if any)

Decision

Post-experiment outcome *

A) No clear decision is taken: SCORE -10

B) Decision taken, but not in line with pre-aligned decision matrix: SCORE -5

C) No decision is taken due to an issue with the experiment: SCORE 0

D) Decision taken in line with pre-aligned decision matrix: SCORE 10

Remarks (if any)

Submit

Clear form

Never submit passwords through Google Forms.

This form was created inside of Delivery Hero SE.

Does this form look suspicious? Report

Forms