Test Pilot - All Test Suites
 Share
The version of the browser you are using is no longer supported. Please upgrade to a supported browser.Dismiss

 
View only
 
 
ABCDEFGHIJKLMNOPQRSTUVWXYZ
1
NumberTitlePreconditionsStepsExpected Results
2
1Test Pilot - Localized testing prerequisites (mandatory if testing localized version of website)In order to test Test Pilot on one of the available Localization, you'll need to either download and install / extract one of the localized builds from "http://ftp.mozilla.org/pub/firefox/releases/". Or to perform the next steps in order to see only the page content in the desired localization.

Note: The Test Pilot toolbar button and elements are not localized if the used build for testing is not localized. The button is part of the browser UI and not from the page content, thats why the language selection is not affecting it.
1. Open Firefox browser and click on the "Menu" button.
2. Click on "Options" button (cogwheel).
3. Click on "Content" section from left sidebar.
4. From "Languages" section click on "Choose..." button.
5. Click on "Select a language to add.." dropdown and write the name of the language you want to add (Ger for German [de] language).
6. Press the "Enter" key on the keyboard and then click on "Add" button from right side of the dropdown.
7. Make sure that desired language is on top of the list by using the "Move Up" button, for the content localization to work.
8. Click the "OK" button and return / navigate to the desired Test Pilot server web page to test.
1. Firefox menu doorhanger opens.
2. You are redirected to Firefox "General" options page.
3. Firefox Content options are displayed.
4. Languages selection pop up is displayed.
5. The desired language is highlighted in list if available.
6. The highlighted language is selected and added to the browser Languages list.
7. The desired language is on top of the list.
8. Test Pilot page is correctly localized in the selected language if available.
3
2Test Pilot (Not installed) - Browser version unsupported - Homepage UI- A profile with "Never check for updates (not recommended: security risk)" option selected was created.
- Firefox v48.0.2 or lower installed / extracted (http://ftp.mozilla.org/pub/firefox/releases/48.0.2/).
1. Start Firefox v48.0.2 (or under) with the profile from prerequisites.
2. Navigate to one of the servers depending on where you need to test:
- https://testpilot.firefox.com (public server)
- https://testpilot.stage.mozaws.net (staging server)
- https://testpilot.dev.mozaws.net (development server)
1. Firefox is correctly launched and homepage is displayed.
2. The Test Pilot page is rendered as expected and contains the next elements:
Top Part
- Blue background with stars and constellations
- Firefox icon and "Firefox Test Pilot" text in top left side of the page
- Test Pilot logo image on the left side
- "Go beyond... " title text on right side
- "Test new features. Give your feedback. Help build Firefox." text sentences one under the other
- "Test Pilot requires Firefox 49 or higher." message displayed under
- "Upgrade Firefox Free Download" green button with white text and Fox icon.

Middle part
- Semitransparent white overlay over the blue background
- "Try out the latest experimental features" text centrally positioned
- Experiments tiles with icon on top and under, title + description + nr of participants + "Get Started" blue button with white text, all over a white rectangle
- "View Past Experiments" button

Bottom part
- "Get started in 3,2,1" text in white
- 3 section area with separators containing "Puzzle" icon and "Get the Test Pilot add-on" on left, "UFO" icon with "Enable experimental features" in the middle and "Chat bubbles" icon with "Tell us what you think" on the right
- "Test Pilot requires Firefox 49 or higher." message displayed under
- "Upgrade Firefox Free Download" green button with white text and Fox icon.
- Test Pilot Newsletter sign up section.
- Footer links for "moz://a" homepage, Legal, About Test Pilot, Privacy, Terms, Cookies on the left and Github and Twitter social media buttons on the right.
4
3Test Pilot (Not installed) - Browser version unsupported - Experiment page UI- A profile with "Never check for updates (not recommended: security risk)" option selected was created.
- Firefox v48.0.2 or lower installed / extracted (http://ftp.mozilla.org/pub/firefox/releases/48.0.2/).

1. Click on one of the available experiments from middle section and observe the page UI.
2. Return to Homepage and click the "Get Started" blue button from experiment card.
1. You are redirected to experiment details page for the users that don't have Test Pilot installed and which contains the next elements:
Top Part
- Blue background with stars and constellations
- Firefox icon and "Firefox Test Pilot" text in top left side of the page
- "Ready for Takeoff?" title text
- "We're building next-generation features for Firefox. Install Test Pilot to try them!" sentence under title
- "Test Pilot requires Firefox 49 or higher." message displayed under
- "Upgrade Firefox Free Download" green button with white text and Fox icon.
- Test Pilot logo image on the right side

Middle left
- White background
- Experiment title in large black chars
- Additional notes under title
- Experiment logo
- "There are x people trying <experiment_name> right now!" text in black over a gray rectangular background tied to the experiment logo
- Experiment description in black regular characters
- "Brought to you by" text in black chars followed under it by avatars and names of the peoples that participated in the development of the experiment
- "Your privacy" section with general paragraph over white background and the next custom paragraphs over a gray rectangular background
- "You can learn more about the data collection for <experiment_name> here." text link

Middle right
- Images from the experiment and descriptions under them

Bottom part
- Blue constellation background
- "Try out these experiments as well" message in large white chars centrally positioned
- All other available experiments tiles
- Test Pilot Newsletter sign up section
- Footer links (mozilla, Legal, About Test Pilot, Privacy, Terms, Cookies, GitHub and Twitter)
2. You are again redirected to experiments details page.
5
4Test Pilot (Not installed) - Browser version unsupported - Upgrade Firefox- A profile with "Never check for updates (not recommended: security risk)" option selected was created.
- Firefox v48.0.2 or lower installed / extracted (http://ftp.mozilla.org/pub/firefox/releases/48.0.2/).

1. Click on the "Upgrade Firefox" button from Experiments details page.
2. Return to the Test Pilot Homepage and click on the "Upgrade Firefox" button from top part of the page.
3. Return to the Test Pilot Homepage and click on the "Upgrade Firefox" button from bottom part of the page.
4. Update Firefox browser to latest version and navigate to the Test Pilot homepage.
1. You are redirected to Firefox download page.
2. You are redirected to Firefox download page.
3. You are redirected to Firefox download page.
4. The "Upgrade Firefox" button is no longer displayed. Install the Test Pilot add-on" button is displayed instead.
6
5Test Pilot (Not installed) - Homepage - UI- Navigate to about:addons page and make sure that you don't have any existing Test Pilot add-on or experiment add-ons installed.1. Open Firefox and navigate to one of the servers depending on where you need to test:
- https://testpilot.firefox.com (public server)
- https://testpilot.stage.mozaws.net (staging server)
- https://testpilot.dev.mozaws.net (development server)
2. Resize the browser window and observe the page UI
3. Maximize the browser window and press "F11" key.
1. The Test Pilot page is rendered as expected and contains the next elements:
Top Part
- Blue background with stars and constellations
- Firefox icon and "Firefox Test Pilot" text in top left side of the page
- Test Pilot logo image on the left side
- "Go beyound... " title text on right side
- "Test new features. Give your feedback. Help build Firefox." text sentences one under the other
- "Install the Test Pilot Add-on" green button with white text.
- "By proceeding, you agree to the Terms of Use and Privacy Notice of Test Pilot." agreement message under button

Middle part
- Semitransparent white overlay over the blue background
- "Try out the latest experimental features" text centrally positioned
- Experiments tiles with icon on top and under, title + description + nr of participants + "Get Started" blue button, all over a white rectangle
- "View Past Experiments" blue button (same as background) with white margin

Bottom part
- "Get started in 3,2,1" text in white
- 3 section area with separators containing "Puzzle" icon and "Get the Test Pilot add-on" on left, "UFO" icon with "Enable experimental features" in the middle and "Chat bubbles" icon with "Tell us what you think" on the right
- "Install the Test Pilot Add-on" green button with white text.
- "By proceeding, you agree to the Terms of Use and Privacy Notice of Test Pilot." agreement message under button
- Newsletter sign up section
- Footer links for "mozilla" homepage, Legal, Privacy, Terms, Cookies on the left and Github and Twitter social media buttons on the right.
2. Page elements are responsive and auto rearrange to accommodate the browser window size.
3. - The browser enters in Full Screen mode.
- Page elements are correctly rendered also in full screen mode.
7
6Test Pilot (Not installed) - Homepage - UI - Top part1. Click on "Firefox Test Pilot" icon or text in top left corner of the page.
2. Navigate to any of the website pages and click on the logo or text from top left corner.
3. Click on the "Terms of Use" text link under the green button.
4. Go back to the previous page and click on the "Privacy Notice" text link under the green button.
1. You redirected to Test Pilot Homepage.
2. You redirected to Test Pilot Homepage.
3. You are redirected to "Test Pilot Terms of Use" page (http://testpilot.dev.mozaws.net/terms).
4. You are redirected to "Test Pilot Privacy Notice" page (http://testpilot.dev.mozaws.net/privacy).
8
7Test Pilot (Not installed) - Homepage - UI - Middle part1. Return to Test Pilot Homepage
2. Observe the available experiments area.
3. Observe the experiments cards.
4. Hover over each experiment card.
5. Hover over "Get Started" button.
6. Click on one of the experiment cards.
7. Return to Test Pilot Homepage and click the "Get Started" button.
8. Return to Test Pilot Homepage and click the "View Past Experiments" button.
9. Click the "Hide Past Experiments" button.
10. Click again the "View Past Experiments" button and observe the past experiments cards.
11. Hover over each past experiments cards.
12. Hover over "Learn More" button.
13. Click on one of the past experiments cards.
14. Return to Test Pilot Homepage, reopen the "Past Experiments" section and click the "Learn More" button.
15. Return to Test Pilot Homepage and scroll to the bottom of the page.
1. Test Pilot Homepage is correctly loaded.
2. Experiments are centrally positioned on the page depending on the number of available experiments.
3. The experiments cards contain the next elements:
- "Just launched" blue label with white text in top left corner of the card (if experiment was just launched)
- Experiment white logo over experiment color, centered on top side of the card (aprox. 35% of the card space)
- White background covering the rest of the card under logo
- Title of the feature (large black chars)
- Additional note for the experiment (optional, eg:"Powered by the Wayback Machine")
- Experiment short description (medium black chars)
- "Get Started" blue button with white characters
- No. of participants in experiment (small gray chars)
4. - A selection border appears around each card.
- Experiment title and additional note (if available) are highlighted changing their color from black to blue.
5. The button gains focus effect and the blue color darkens a bit.
6. You are redirected to experiment details page for users that don't have Test Pilot installed.
7. You are redirected to experiment details page for users that don't have Test Pilot installed.
8. - The "View Past Experiments" button changes into a "Hide Past Experiments" one
- The middle section of the page expands displaying the past Test Pilot experiments that graduated.

9. - The middle section of the page is reduced, hiding the past experiments section.
10. The experiments cards contain the next elements:
- Experiment white logo over experiment color, centered on top side of the card
- White background covering the rest of the card under logo
- Title of the feature (large black chars)
- Additional note for the experiment (optional, eg:"Powered by the Wayback Machine")
- Experiment short description (medium black chars)
- "Learn More" gray button with black text
11. - A selection border appears around each card.
- Experiment title and additional note (if available) are highlighted changing their color from black to blue.
12. The button gains focus effect and the gray color darkens a bit.
13. You are redirected to experiment details page where a report with experiment results are displayed.
14. You are redirected to experiment details page where a report with experiment results are displayed.
15. Homepage is correctly loaded.
9
8Test Pilot (Not installed) - Homepage - UI - Bottom part1. Click on the "Terms of Use" text link under the green button.
2. Go back to the previous page and click on the "Privacy Notice" text link under the green button.
3. Return to Test Pilot Homepage and observe the Newsletter section.
4. Scroll to the bottom of the page and click on "moz://a" logo.
5. Return and click on "Legal" text link.
6. Return and click on "About Test Pilot" text link.
7. Click on "Mozilla" text link from the page
8. Return to "About Test Pilot" page and click on "...core beliefs in user privacy and choice" text link.
9. Return to "About Test Pilot" page, scroll to the bottom of it and click on "...you can find us on our forum" text link.
10. Return to Test Pilot Homepage and click the "Privacy" link in the footer.
11. Click on the "Mozilla Privacy Policy" link from first phrase.
12. Return and click both of the "Learn More" text buttons from the page.
13. Click on "...information" text link from the "Telemetry and Browser Data" sub section.
14. Return and collapse both the "Learn More" sections by clicking the "Show Less" text links.
15. Return to Test Pilot Homepage and click the "Terms" button in the footer.
16. Scroll through the "big wall of text" and click all the observed links.
17. Return to test Pilot Homepage and click the "Cookies" text button in the footer.
18. Return to test Pilot Homepage and click the "GitHub" logo button from bottom right side of the page.
19. Close the repo tab and click on "Twitter" logo button.
20. Close the Twitter tab in order to return to the Test Pilot Homepage.
1. You are redirected to "Test Pilot Terms of Use" page (http://testpilot.dev.mozaws.net/terms).
2. You are redirected to "Test Pilot Privacy Notice" page (http://testpilot.dev.mozaws.net/privacy).
3. The "Newsletter" section is correctly displayed and contains the next elements:
- White semitransparent background
- Envelope logo on the left side
- "Stay Informed" title in white large chars, next to envelope logo
- "Find out about new experiments and see test results for experiments you've tried." text in white normal chars under title
- White actionable text box with "YOUR EMAIL HERE" hint text in gray letters, on the right side
- "Sign UP Now" button with the same size as the email text box and with white slim border around it and white text.
4. You are redirected to the mozilla homepage (https://www.mozilla.org/en-US/).
5. You are redirected to mozilla general "Legal" page (https://www.mozilla.org/en-US/about/legal/).
6. You are redirected to "About Test Pilot" page where all the info is correctly rendered.
7. You are redirected to Mozilla homepage.
8. You are redirected to Mozilla Manifesto page.
9. You are redirected to Test Pilot Discourse forum page.
10. You are redirected to "Test Pilot Privacy" page (https://testpilot.firefox.com/privacy).
11. You are redirected to Mozilla "Privacy Policy" general page (https://www.mozilla.org/en-US/privacy/).
12. Both the sections are correctly expanded and the info inside is correctly rendered.
13. You are redirected to "Test Pilot Telemetry metrics page.
14. Both the sections are correctly collapsed and page looks again as when you first navigated to it.
15. You are redirected to the "Test Pilot Terms of Use" page (https://testpilot.firefox.com/terms).
16. Everything renders as expected and the correct pages are opened for each link.
17. You are redirected to Mozilla Cookies page (https://www.mozilla.org/en-US/privacy/websites/#cookies).
18. Test Pilot GitHub repository page is opened in a new tab.
19. Test Pilot oficial Twitter page is opened in a new tab.
20. The Homepage tab is refocused.
10
9Test Pilot (Not installed) - Homepage - UI - Bottom part - Newsletter sign up1. Scroll down to the bottom of the page until the "Newsletter" section is reached.
2. Click inside the email text box.
3. Enter at least one character in the text box and observe the page behavior.
4. Click on "... this privacy notice" words under the text box.
5. Return (back button) to the Homepage, at the Newsletter section.
6. Delete the entered character.
7. Click the "Sign Up Now" button with empty text box.
8. Enter a single character or invalid email address and click on "Sign Up Now" button.
9. Check the checkbox under and try to submit with the same string as above.
10. Remove the checkbox under the text box.
11. Enter a valid (preferably existing) email address and click on "Sign Up Now" button.
12. Check the checbox and click again on the "Sign Up Now" button.
13. In a new tab, open the email client and check for the sent email.
14. Return to the Test Pilot Home Page and refresh the page.
1. The "Newsletter" section is correctly displayed and contains the next elements:
- White semitransparent background
- Envelope logo on the left side
- "Stay Informed" title in white large chars, next to envelope logo
- "Find out about new experiments and see test results for experiments you've tried." text in white normal chars under title
- White actionable text box with "YOUR EMAIL HERE" hint text in gray letters, on the right side
- "Sign UP Now" button with the same size as the email text box and with white slim border around it and white text.
2. Writing pointer appears but hint text is still kept.
3. - The entered character is displayed and hint text is removed from the text box
- Checkbox + "I'm okay with Mozilla handling by info as explained in this privacy notice." text. Where "...this privacy notice" words have a link under them.
- "We will only send you Test Pilot-related information." text in white chars appears under "Sign Up Now" button.
4. You are redirected to the "Test Pilot Privacy Notice" page.
5. Newsletter section is correctly displayed.
6. - The Newsletter section retracts, removing the added checkbox and sentences.
- The hint text reappears in text box.
7. A hint tooltip appears pointing with an arrow to the beginning of the box and saying "Please fill out this field." text.
8. "Please enter an email address" text is displayed with the same type of tooltip and subscription confirmation is not displayed.
9. "Please enter an email address" text is displayed with the same type of tooltip and subscription confirmation is not displayed.
10. Check mark is removed.
11. "Please check this box if you want to proceed." text is displayed with the same type of tooltip and subscription confirmation is not displayed.
12. - The button changes into "Submitting..."
- Text box and button are dismissed after email is sent
- "Thanks!" text replaces the previous title
- "If you haven't previously confirmed a subscription to a Mozilla-related newsletter you may have to do so. Please check your inbox or your spam filter for an email from us." replaces the previous text under title.
13. The email is correctly received and contains the next message:
-
14. The Newsletter form resets and you can subscribe again.
11
10Test Pilot (Not installed) - Experiment details page1. Click on one of the experiments cards.
2. Scroll back to the top and click on "Terms of Use" and "Privacy Notice" buttons under installation button.
3. Return to the experiment page and click on the "Changelog" link.
4. Return to the experiment page and click on the "Contribute" link.
5. Return to the experiment page and click on the "Bug Reports" link.
6. Return to the experiment page and click on the "Discuss <experiment_name>" link.
7. Return to the experiment page and click on the "data" text link from "Your Privacy" section.
8. Return to the experiment page and click on any other links displayed on "Your Privacy" section.
9. Return to the experiment page and click on "You can learn more about the data collection for <experiment_name> here" text link.
10. Return to the experiment page and click on one of the other experiments cards.
11. Scroll to the bottom of the page and click on all the footer links.
12. Resize the browser window and observe the page UI
13. Maximize the browser window and press "F11" key.
14. Press "F11" key again to return to normal view.
1. You are redirected to experiment details page for the users that don't have Test Pilot installed and which contains the next elements:
Top Part
- Blue background with stars and constellations
- Firefox icon and "Firefox Test Pilot" text in top left side of the page
- "Ready for Takeoff?" title text
- "We're building next-generation features for Firefox. Install Test Pilot to try them!" sentence under title
- Green button with "Install Test Pilot &" text in smaller white chars + "Enable <experiment_name>" text under it, in larger and bold white chars
- "By proceeding, you agree to the Terms of Use and Privacy Notice of Test Pilot." message in white chars
- Test Pilot logo image on the right side

Middle left
- White background
- Experiment title in large black chars
- Additional notes under title
- Experiment logo
- "There are x people trying <experiment_name> right now!" text in black over a gray rectangular background tied to the experiment logo
- Experiment description in black regular characters
- Links for Changelog, Contribute, Bug Reports and Discuss page for current experiment
- [Depending on experiment] Experiment notification message in black over yellow rectangle (eg: No More 404s requires Firefox 48 or later.Make sure you're up to date.)
- "Brought to you by" text in black chars followed under it by avatars and names of the peoples that participated in the development of the experiment
- "Your privacy" section with general paragraph over white background, and the next experiment oriented paragraphs over a gray rectangular background
- "You can learn more about the data collection for <experiment_name> here." text link

Middle right
- Images from the experiment and descriptions under them

Bottom part
- Blue constellation background
- "Try out these experiments as well" message in large white chars centrally positioned
- All other available experiments tiles
- Footer links (mozilla, Legal, Privacy, Terms, Cookies, GitHub and Twitter)
2. You are redirected to the expected pages.
3. If you are logged with a valid GitHub account, you are redirected to experiment changelog page on GitHub. Else you are redirected to GitHub login page.
4. If you are logged with a valid GitHub account, you are redirected to experiment "Code" page on GitHub. Else you are redirected to GitHub login page.
5. If you are logged with a valid GitHub account, you are redirected to experiment "Issues" page on GitHub. Else you are redirected to GitHub login page.
6. You are redirected to experiment Discourse page.
7. You are redirected to "Test Pilot Privacy Notice" page (http://testpilot.firefox.com/privacy).
8. You are correctly redirected to the corresponding pages.
9. You are redirected to a specific page where the experiment details about data collection are detailed.
10. - You are redirected to the selected experiment details page.
- The same page format is displayed for each experiment.
11. You are redirected to the expected pages.
12. Page elements are responsive and auto rearrange to accommodate the browser window size.
13. - The browser enters in Full Screen mode.
- Page elements are correctly rendered also in full screen mode.
14. Browser returns to normal maximized view.
12
11Test Pilot - Homepage Install - Top of the page1. Click on the Firefox logo at the top part of the page and after click on "Install the Test Pilot Add-on" button.1. - The green button changes to "Installing..." and displays a barberpole animation under the text.
- After the installation is performed the page UI changes in order to display the functionality for users with installed add-on.
- "Spacecraft" Test Pilot button is added in the browser toolbar
- You are redirected to Test Pilot Experiments page
- Page is covered with a black semitransparent overlay
- A pop up is displayed in the center of the page, welcoming the user and provides the option to enroll for the newsletter
13
12Test Pilot - Install - Welcome + Newsletter pop up1. Observe the Welcome pop up displayed after install.
2. Click outside the pop up or press the "Esc" key.
3. Click the email text box.
4. Enter at least 1 character in text box.
5. Leave the text box empty and click on "Sign me up" button.
6. Enter an invalid email address and click on "Sign me up" button
eg: namedomain.com, name@domaincom, name@domain. , name.domain.com, @domain.com, etc
7. Enter a valid email address and click on "Sign me up" button.
8. Observe the confirmation pop up.
9. Open the email client on a new tab and check for the Newsletter subscription email.
10. Choose to unsubscribe from the bottom of the received email.
11. Return to the Test Pilot tab and click on the "On to the experiments..." button.
1. The welcome pop up contains the next elements:
- Blue header with white text (~15% of the pop up) + "Welcome to Test Pilot!" title + close "[x]" white button on top right corner
- White background covering the rest of the pop up
- "Find out about new experiments and see test results for experiments you've tried." message in black chars
- Email text box + "email goes here :)" hint text
- "Sign me up" blue button with white text
2. The pop up and overlay cannot be dismissed by clicking the page overlay or with "Esc" key.

3. Writing pointer appears but hint is not dismissed.
4. Hint message is dismissed.
5. "Please use a valid email address!" error message in red is displayed above the box.
6. "Please use a valid email address!" error message in red is displayed above the box.
7. The pop up confirms that the email is sent.
8. The confirmation pop up contains the next elements:
- Blue header with white text (15% of the pop up) + "Email Sent" title + "[x]" close button
- White background covering the rest of the pop up
- Envelope logo
- "When you get a chance, head over to your inbox to verify your email address. Thank you!" message in black chars
- "On to the experiments..." blue button with white chars.
9. Email is correctly received.
10. Unsubscribe action is confirmed.
11. Pop up and page overlay is dismissed and experiments page is accessible.
14
13Test Pilot - Install - Restart after install1. Close the Firefox browser and restart it with the same profile.
2. Navigate again to "testpilot.firefox.com" page and observe the displayed page.
3. Restart the browser again to see if any Test Pilot pages are opened.
1. Firefox browser opens with the Firefox Homepage displayed.
2. Test Pilot page for users with installed add-on is loaded and "Spaceship" button displayed in Fx toolbar.
3. No other pages related to Test Pilot are opened.
15
14Test Pilot - Experiments page1. Observe the Experiments page UI.
2. Resize the browser window and observe the page UI
3. Maximize the browser window and press "F11" key.
4. Press F11 again to return to normal view.
1. The page contains the following elements:
- Blue background with stars and constellations

Top Part
- Firefox icon and "Firefox Test Pilot" text in top left side of the page
- "Settings" button on the top right side
- Semitransparent banner + "Pick your experiments!" white text in large bold characters on left side + Test Pilot spaceship logo on the right side.

Middle
- Available experiments cards
- "View Past Experiments" blue button with white characters and white borders.

Bottom part
- Newsletter sign up section
- Footer links (moz://a, Legal, About Test Pilot, Privacy, Terms, Cookies, GitHub and Twitter)
2. Page elements are responsive and auto rearrange to accommodate the browser window size.
3. - The browser enters in Full Screen mode.
- Page elements are correctly rendered also in full screen mode.
4. Page elements are responsive and auto rearrange to accommodate the browser window size.
16
15Test Pilot - Experiments page - Functionality1. Click on the Firefox logo or "Firefox Test Pilot" text on top left side of the page.
2. Navigate to other Test Pilot page and click the Firefox logo or "Firefox Test Pilot" text on top left side of the page.
3. Hover over the available experiments cards.
4. Click on any of the available experiments cards.
5. Return to the "Experiments" page and click the "View Past Experiments" button.
6. Click the "Hide Past Experiments" button.
7. Click again the "View Past Experiments" button and observe the past experiments cards.
8. Hover over each past experiments cards.
9. Click on one of the past experiments cards.
10. Return to "Experiments" page, reopen the "Past Experiments" section and click the "Learn More" button.
11. Return to the Experiments page and click all the footer links (moz://a, Legal, Privacy, Terms, Cookies, GitHub and Twitter)
12. Return or close the pages opened in new tabs.
1. You are redirected to the same Experiments page.
2. You are redirected to Experiments page.
3. - A selection border appears around each card.
- Experiment title and additional note (if available) are highlighted changing their color from black to blue.
4. You are redirected to the experiments detailed page for users that have Test Pilot installed.
5. - The "View Past Experiments" button changes into a "Hide Past Experiments" one
- The middle section of the page expands displaying the past Test Pilot experiments that graduated.


6. The middle section of the page is reduced, hiding the past experiments section.
7. The experiments cards contain the next elements:
- Experiment white logo over experiment color, centered on top side of the card
- White background covering the rest of the card under logo
- Title of the feature (large black chars)
- Additional note for the experiment (optional, eg:"Powered by the Wayback Machine")
- Experiment short description (medium black chars)
- "Learn More" gray button with black text
8. - A selection border appears around each card.
- Experiment title and additional note (if available) are highlighted changing their color from black to blue.


9. You are redirected to experiment details page where a report with experiment results are displayed.
10. You are redirected to experiment details page where a report with experiment results are displayed.
11. You are redirected to the expected pages.
12. Experiments page is redisplayed.
17
16Test Pilot - Experiments page - Settings1. Scroll up to the top of the page and click on the "Settings" button from top right side of the page.
2. Click outside the doorhanger or on any page element.
3. Reopen the "Settings" doorhanger and click on "Test Pilot Wiki" text button.
4. Close the opened new tab and open again the "Settings" doorhanger.
5. Click the "Discuss Test Pilot" button from doorhanger.
6. Close the opened new tab and open again the "Settings" doorhanger.
7. Click on "File and Issue" text button.
8. Close the opened new tab for GitHub page.
1. A doorhanger menu opens displaying the next elements:
- White background
- "Test Pilot Wiki" text button
- "Discuss Test Pilot" text button
- "File an Issue" text button
- Gray separating line
- "Uninstall Test Pilot" text button
2. The doorhanger is automatically closed.
3. The Test Pilot mozilla wiki page is opened in a new tab.
4. Doorhanger is displayed.
5. The Mozilla Test Pilot discourse forum page is opened in a new tab.
6. Doorhanger is displayed.
7. The Test Pilot GitHub page is opened in a new tab where you are prompted to log in if not already logged, and where you can file an issue observed.
8. The "Experiments" page comes back in focus.
18
17Test Pilot - Experiments page - Settings - Uninstall1. Open the "Settings" button doorhanger and click on the "Uninstall Test Pilot" text button.
2. Click the "[x]l" button.
3. Click again on "Uninstall Test Pilot" from doorhanger and choose to "Proceed" on the displayed pop up.
4. Click on "Take a quick survey" button.
5. Complete and submit the survey or close the tab to return to Test Pilot uninstall page.
6. Click the "Home" text button on the pop up.
1. A confirmation dialog appears containing the next elements:
- A white pop up with a red header containing "Uninstall Test Pilot?" text in it
- Close "[x]" white button on right side of the red header
- "As you wish. This will disable any active tests, uninstall the add-on and remove your account info from our servers." text in medium black chars
- "To opt out of email updates, simply click the unsubscribe link on any Test Pilot email." small text in black chars
- "Proceed" red button with white text
2. The dialog is dismissed without removing the Test Pilot add-on.
3. A "Shutting down" barberpole loading animation is displayed and after a "Thanks for flying!" pop up is displayed containing the next elements:
- White background with blue header
- "Thanks for flying!" white text over blue header
- "Hope you had fun experimenting with us. Come back any time." black text over white background
- "Take a quick survey" blue button with white text
- "Home" blue text button.
4. Survey page is opened on a new tab.
5. Survey sent / Retire page is redisplayed.
6. You are redirected to Test Pilot Homepage.
19
18Test Pilot - Homepage install - Bottom button1. Scroll down to the bottom of the Homepage and click on the "Install the Test Pilot Add-on" button.
2. Dismiss the newsletter pop up by clicking on "[x]" button on top right side of the pop up.
1. - The green button changes to "Installing..." and displays a barberpole animation under the text.
- Page UI changes to display the functionality for the user with Test Pilot add-on installed.
- Welcome + Newsletter pop up is re-displayed
2. Pop up is dismissed and Experiments page content is displayed.
20
19Test Pilot - Experiment details1. Click on one of the available experiments cards and observe the displayed page.
2. Resize the browser widnow and observe the page UI
3. Maximize the browser window and press "F11" key.
4. Press "F11" key again to return in normal mode.
1. The experiment page for users with Test Pilot installed is loaded and contains the next elements:
Top Part
- Header with blue background with stars and constelations
- Firefox icon and "Firefox Test Pilot" text in top left side of the page
- "Settings" button on top right side of the page

Middle left
- White background
- Experiment title in large black chars
- Additional notes under title (if available)
- Experiment logo
- "There are x people trying <experiment_name> right now!" text in black over a gray rectangular background tied to the experiment logo
- Experiment details (Tour link, Changelog link, Contribute link, Bug Reports link and Discourse link)
- "Brought to you by" text in black chars followed under it by avatars and names of the peoples that participated in the development of the experiment
- "Your privacy" title in black for the section where privacy details are mentioned for each feature
- "You can learn more about the data collection for <experiment_name> here." text link

Middle right
- White background
- "Your privacy" blue text button + "Enable <experiment_name>" blue button with white text
- [Depending on experiment] Experiment notification message in black over red rectangle (eg: No More 404s requires Firefox 48 or later.Make sure you're up to date.)
- Experiment description in black regular characters
- Images from the experiment and descriptions under them

Bottom part
- Blue constellation background
- Test Pilot "Newsletter" sign up section.
- Footer links (moz://a, Legal, About test Pilot, Privacy, Terms, Cookies, GitHub and Twitter)
2. Page elements are responsive and auto rearrange to accomodate the browser window size.
3. - The browser enters in Full Screen mode.
- Page elements are correctly rendered also in full screen mode.
4. Broeser returns in normal mode and page elements are correctly rendered to acommodate the window size.
21
20Test Pilot - Experiment details - Page functionality1. While observing the top part of the page, scroll down.
2. Scroll back to the top of the page.
3. Click on "Tour" text button at experiments details section on left side.
4. Dismiss the tour and click on "Changelog" link at experiments details section on left side.
5. Return to experiment details page and click on the "Contribute" link from the same section.
6. Return to experiment details page and click on "Bug Reports" link from the same section.
7. Return to experiment details page and click on "Discuss" link from the same section.
8. Click the "Your privacy" text button from top right side of the page.
9. [Optional] If links are available in "Your privacy" section, check each of them.
10. [If available] Return to the experiment details page and click on version compatibility link from the displayed notification.
11. Prerequisite: Tab Groups add-on was installed from AMO.

Navigate to the "Experiments" page and click on "Tab Center" experiment.
12. Click on "disabling these add-ons" text link from the notification.
13. Return to experiment details page and observe the images and descriptions related to experiment.
14. Scroll to the bottom of the page and check each button / link from the footer.
1. When the page content under the Firefox logo reaches the browser frame, it transforms in a pinned floating bar containing:
Left side
- Experiment name
- Additional notes about it (if available)

Right side
- "Your privacy" blue text button
- "Enable <experiment_name>" blue button with white text
2. The pinned floating bar is hidden and its elements regain their position on the page content
3. A pop up with the experiment tour is displayed.
4. You are redirected in the same tab to GitHub repository changelog page for the selected experiment.
5. You are redirected in the same tab to GitHub repository code section where you can contribute to the experiment development.
6. You are redirected in the same tab to the GitHub "New issue" page (if logged in) where you can file an issue for the chosen experiment.
7. You are redirected in the same tab to experiment mozilla Discourse page.
8. Page is scrolled down to "Your privacy" section and the whole section is highlighted with a temporary blue border.
9. Each link redirects to the expected page.
10. You are redirected in a new tab to the mozilla support page where steps for how to find out what version you have installed and how to upgrade to latest one available are explained.
11. A notification appears on the top part of the page on yellow background informing the user about the next thing:
"This experiment may not be compatible with add-ons you have installed.
We recommend disabling these add-ons before activating this experiment:
- Tab Groups"
12. You are redirected to a Mozilla tutorial page for add-ons disable.
13. All the images and descriptions displayed are correctly rendered.
14. Each link redirects to the expected page.
22
21Test Pilot - Experiment details - Page functionality - Enable experiment1. Prerequisites: Disable the internet connection.

Return to the experiment details page and click on "Enable <experiment_name>" button without an internet connection
2. Scroll down on the page and observe the red error bar behavior.
3. Re-enable the internet connection and click again the "Enable <experiment_name> button.
1. - "Enable <experiment_name>" button changes into "Enabling..." and displays a barberpole animation under the text.
- After attempting the installation, a red bar is displayed between page header and experiment description area containing "Uh oh. â￾¨Universal Searchâ￾© could not be enabled. Try again later." error message.
2. The error red bar remains displayed on top of the pinned floating bar.
3. - "Enable <experiment_name>" button changes into "Enabling..." and displays a barberpole animation under the text.
- Experiment is enabled after a few seconds and changes in UI can also be observed for the ones that affect the normal functionality
- A black semitransparent overlay covers the entire page
- A pop up is displayed in the center of the page informing the user that the experiment was enabled.
23
22Test Pilot - Experiment details - Page functionality - Enable experiment pop up1. Observe the displayed pop up.
2. Hover the "[x]" close button.
3. Hover the navigation arrows.
4. Click on the "Right arrow" button.
5. After viewing the first slide, click on the "Left arrow" button.
6. Click the "Right arrow" button until reaching the last slide.
7. Click the "Left arrow" button until reaching the first slide.
8. Click on the navigation bubbles on bottom of the tour.
9. Click on the last bubble and after hover the "Done" button.
10. Click on "Done" button.
11. Relaunch the "Tour" from the button with the same name displayed in version section on the left side of the page.
12. Click the "[x]" closing button.
1. The experiment enable pop contains the next elements:
- Blue header with "<experiment_name> enabled!" text +"[x]" button to dismiss / skip the tour
- White background under the header on the remaining pop up surface that is populated with experiment presentation slides
- Navigation arrows (first only to right).
2. Button gains a square focused state.
3. The buttons gain a focus state where button darkens a bit.
4. Next slide is displayed for the enabled experiment.
5. The previous tour slide is displayed.
6. The right navigation arrow is replaced by "Done" white button with green text.
7. The first slide is correctly displayed.
8. The user can choose whatever slide desires.
9. The colors invert to green button with white text.
10. The pop up is dismissed and view returns to experiment details page.
11. Experiment tour is re-displayed.
12. The tour is again dismissed.
24
23Test Pilot - Experiment details - Enabled experiment - UI changes1. Observe the changes that occur on experiment details page after enabling the experiment.
2. Scroll down on the page and observe the confirmation green bar behavior.
3. Return to Test Pilot "Experiments" page or click the Firefox logo on the top of the page and observe the enabled experiment card.
4. Click on the "Manage" button for the enabled experiment.
1. - Green bar displayed between page header and experiment description area displaying "<experiment_name> is enabled." confirmation message.
- "Your privacy" blue text button was replaced by "Give Feedback" blue button with white text
- "Enable <experiment_name>" button changed into "Disable <experiment_name>".
2. The confirmation bar remains displayed on top of the pinned floating bar.
3. - "Enabled" green tag with white text displayed on top left corner of the enabled experiment card
- Green border surrounds the enabled experiment card.
- "Get Started" blue button was replaced with "Manage" gray button with black text.
4. You are redirected to the experiment details page.
25
24Test Pilot - Experiment details - Enabled experiment - New functionality1. While observing the top part of the page, scroll down.
2. Click on "Give Feedback" button from top right side of the page.
3. Complete and submit the suvrey or close the tab to return to chosen experiment page.
4. Check if the enabled experiment features are working
5. Chose to "Disable <experiment_name>.
6. Observe the displayed pop up.
7. Click the "Take a quick survey" button.
8. Complete and submit the suvrey or close the tab to return to chosen experiment page.
9. Click on the Firefox logo to return to "Experiments" page and observe the disabled experiment card.
1. The floating bar that appears, now also has the green bar with experiment enabled confirmation sticked on top of it while page is scrolled.
2. A new tab is opened and the experiment survey page is correctly loaded.
3. The experiment page is redisplayed.
4. The next changes occur for each of the experiments:
- Activity Stream >>> A new button for Activity Stream timeline is added in browser toolbar, New Tab page content is enhanced by new Activity Stream functionality and homepage is replaced.

- Tab Center >>> All the opened tabs are moved from horizontal position from top of the browser to a vertical position on left side of the browser window.

- Tracking Protection (Blok) >>> A shield icon is displayed at the end of the Awesomebar for the pages that track user actions (test page - http://goo.gl/d6Cn4E)

- Page Shot >>> Scissors button is added in Firefox Toolbar from which the user can start the screenshot functionality.

- Min Vid >>> Overlay icon displayed over videos (eg: YouTube, Vimeo, etc) and option to send to mini player available on context menu for video titles / links.

- NoMore404s >>> Notification bar displayed on top of the page to inform the user that the searched page that 404s, is saved and can be provided trough wayback machine internet archive. (eg: http://www.pfaw.org/attacks.htm)

German language set (de):

- Cliqz >>> The browser Awesomebar is unified with the search bar providing both the functionalities in only one universal bar. Enhanced content is displayed for the search suggestions and also the "New Tab" page is replaced with Cliqz content.
5. - Green bar at the top is removed.
- The "Disable <experiment_name> button changes into "Disabling..."
- The active users number is reduced with 1
- A "Thank You!" pop up is displayed.
6. The pop up contains the next elements:
- White background with blue header
- "Thank You" white text over the blue header + "[x]" close button
- Test Pilot logo
- "Your participation in Firefox Test Pilot means a lot! Please check out our other experiments, and stay tuned for more to come!" text message in black characters
- "Take a quick survey" blue button with white text
7. A new tab is opened and you are redirected to the disabled experiment suvrvey page.
8. - The experiment page is redisplayed.
- "Disabling..." button changed into "Enable <experiment_name>" button.
9. The "Enabled" tag and the green border are no longer displayed.
26
25Test Pilot - Toolbar button1. Click on the "Test Pilot" spaceship icon from the main toolbar.1. You are redirected to the Test Pilot "Experiments" page.
27
26Test Pilot - All experiments enabled1. Enable all the available experiments either from "Experiments" page.
2. Click the "Test Pilot" spaceship icon from the main toolbar and observe the state of all experiments on "Experiments" page.
3. Click on one of the experiments cards.
1. All the available experiments were enabled and correctly work.
The next changes are observed for each of them:
- Activity Stream >>> A new button for Activity Stream timeline is added in browser toolbar, New Tab page content is enhanced by new Activity Stream functionality and homepage is replaced.

- Tab Center >>> All the opened tabs are moved from horizontal position from top of the browser to a vertical position on left side of the browser window.

- Tracking Protection (Blok) >>> A shield icon is displayed at the end of the Awesomebar for the pages that track user actions (test page - http://goo.gl/d6Cn4E)

- Page Shot >>> Scissors button is added in Firefox Toolbar from which the user can start the screenshot functionality.

- Min Vid >>> Overlay icon displayed over videos (eg: YouTube, Vimeo, etc) and option to send to mini player available on context menu for video titles / links.

- NoMore404s >>> Notification bar displayed on top of the page to inform the user that the searched page that 404s, is saved and can be provided trough wayback machine internet archive. (eg: http://www.pfaw.org/attacks.htm)

German language set (de):

- Cliqz >>> The browser Awesomebar is unified with the search bar providing both the functionalities in only one universal bar. Enhanced content is displayed for the search suggestions and also the "New Tab" page is replaced with Cliqz content.
2. All experiments are displayed as enabled with no rendering issue on the "Experiments" page.
"Enabled" green tag with white text displayed on top left corner of the enabled experiments cards
- Green border surrounds each of the enabled experiments cards.
3. Experiment details page is loaded.
28
27Test Pilot - Newsletter Subscribe- Test Pilot is already installed.1. Navigate to "Experiments" page and subscribe to the Newsletter.
2. Open email client and choose to "Unsubscribe" from Test Pilot Newsletter email (bottom links).
3. Click on the "Unsubscribe" radio button for Test Pilot newsletter option and click the "Save Preferences" button.
4. Navigate to experiment details page and subscribe again to the Newsletter.
5. Open email client, choose again to "Unsubscribe" from Test Pilot Newsletter and perform the previous steps to correctly unsubscribe.

Note: this step is not mandatory but should be performed in order to be able to use the same email address in the future for the above tests.
1. The request is correctly performed and email is sent and received.
2. You are redirected to a page where you can manage all mozila email preferences.
3. You are unsubscribed from the Test Pilot Newsletter and "Thanks for updating your email preferences." message is displayed.
4. The request is correctly performed and email is sent and received.
5. You are unsubscribed from the Test Pilot Newsletter and "Thanks for updating your email preferences." message is displayed.
29
28Test Pilot - Experiment details page - Settings - Uninstall1. Click the "Settings" button from top right side of the page.
2. Click on the "Uninstall Test Pilot" text button.
3. Click on "Proceed" button from the displaye pop up.
4. Click on "Home" button.
1. A doorhanger menu opens displaying the next elements:
- White background
- "Test Pilot Wiki" text button
- "Discuss Test Pilot" text button
- "File an Issue" text button
- Gray separating line
- "Uninstall Test Pilot" text button
2. A confirmation dialog appears containing the next elements:
- A white pop up with a red header containing "Uninstall Test Pilot ?" text in it
- "As you wish. This will disable any active tests, uninstall the add-on and remove your account info from our servers." text displayed on white part
- "Proceed" red button with white text
- "Cancel" blue text button."
3. A "Shutting down" loading spinner is displayed and after a "Thanks for flying!" pop up is displayed containing the next elements:
- White background with gray header
- "Thanks for flying!" black text over grey header
- "Hope you had fun experimenting with us.Come back any time." text over white background
- "Take a quick survey" blue button with white text
- "Home" blue text button.
4. You are redirected to "Test Pilot" homepage.
30
29Test Pilot - Install from Experiment page1. Click on one of the experiments cards.
2. Click the "Install te Test Pilot Add-on & Enable <experiment_name>" button from the top left part of the page.
3. Dismiss the tour pop up.
4. Observe the page changes occurred.
5. Verify if experiment functionality is present.
1. Experiment details page is loaded.
2. - The green button changes to "Installing..." and displays a barberpole animation under the text.
- After the installation is performed you are redirected to the experiment details page for the users with "Test Pillot" add-on installed.
- "Enable <experiment_name> button automatically changes into "Enabling..." and chosen experiment is also enabled.
- Experiment tour pop up is displayed.
3. Pop up is closed and experiment details page is displayed.
4. - Green bar appears at the top to confirm that the experiment is enabled
- "Your Privacy" button changes into "Give Feedback"
- "Enabling..." button changes into "Disable <experiment_name>".
5. The enabled experiment functionality is present.
31
30Test Pilot - about:addons - Experiments details- Enable multiple experiments.1. Open a "New Tab" page and navigate to "about:addons" page.
2. Click on "Extensions" tab button from left sidebar of the page.
3. In a new tab, click on "Test Pilot" spaceship toolbar button and compare if all the enabled experiments from "Experiments" page are present in "Extensions" page.
4. Verify each experiment logo and description.
5. Click on "More" button.
6. Verify that the experiment version matches the one from http://testpilot.dev.mozaws.net/__version__ page (depending on server to test).
7. Verify that the experiment "Homepage" provided link is functional and redirects to the expected page.
1. Firefox add-ons page is displayed.
2. A page with all installed extensions is displayed containing the next elements:
- Each experimment displayed individually in a list
- Each experiment has its own logo displayed on the left side of the page
- Each experiment title displayed in bold characters next to the logo
- Each experiment short description displayed in normal chars under the title (truncated if too long).
- Each experiment has a "More" blue text button displayed after its description
- "Options" button for the experiments that have additional options than normal ones
- "Disable" + "Remove" buttons displayed on the left side of the page for each experiment.
3. All the enabled experiments are correctly displayed in "Extensions" page.
4. All the experiments have the right logo and description displayed with no typos in it.
5. The "about:addons" experiment details page is displayed providing the next options and information:
- Large experiment logo on top left side
- Experiment title and version next to the logo on the right
- Experiment short description under title
- "Automatic Updates" option with radio buttons for "Default / On / Off" states
- "Last Updated" section where the latest date of installation / update is displayed
- Experiment "Homepage" section where the development team displays a homepage for the experiment
- Additional experiment options if available with short description and checkboxes
- "Disable" + "Remove" buttons
6. The version matches the one on the server page.
7. Homepage link is functional and correctly displays a page related to the experiment.
32
31Test Pilot - about:addons - Experiments - Manual disable1. Perform a manual disable for an experiment by clicking on the "Disabble" button.
2. Click on the "Test Pilot" spaceship icon from the main toolbar and observe the manually disabled experiment state on "Experiments" page.
3. Click on the disabled experiment card.
4. Scroll down on the page while observing the red error bar behavior.
5. Verifiy that the disabled experiment functionality is not available in the browser.
1. The experiment is successfuly disabled.
2. - The experiment card looses the "Enabled" tag and surrounding green border when experiment is manually disabled.
3. - The experiment details page is displayed with a yellow bar at the top
- "<experiment_name> was disabled in your "Add-ons Manager." message displayed over the yellow bar
where the experiment is displayed as manually disabled.
- The button for "Disable" experiment is grayed out. / The "Disable <experiment_name> button changes into "Enable <experiment_name>".

(TBD)
4. The yellow error bar remains displayed on top of the pinned floating bar.
5. The manually disabled experiment functionality is no longer available in the browser.
33
32Test Pilot - about:addons - Experiments - Manual re-enable1. Return to "about:addons" extensions tab and choose to "Enable" a previously disabled experiment.
2. Click on the "Test Pilot" spaceship button from main toolbar and observe the experiment card state on "Experiments" page.
3. Click on re-enabled experiment card and observe the "Experiment details" page.
4. Verifiy that the re-enabled experiment functionality is available again in the browser.
1. The experiment is successfuly re-enabled.
2. Experiment is displayed as enabled on "Experiments" page.
3. The experiment appears to be enabled and the green bar at top confirms this.
4. All the experiment functionality is again present in the browser.
34
33Test Pilot - about:addons - Experiments - Manual remove1. Return to "about:addons" extensions tab and choose to manually "Remove" one of the experiments.
2. Click on the "Test Pilot" spaceship icon from the main toolbar and observe the manually removed experiment on "Experiments" page.
3. Click on the disabled experiment card.
4. Verify if disabled experiment functionality is present.
1. Experiment is removed from the browser.
2. The experiment is no longer displayed as enabled on the "Experiments" page.
3. The experiment details page is displayed and the experiment is no longer displayed as enabled.
4. Disabled experiment functionality is no longer available.
35
34Test Pilot - about:addons - Enable manualy removed experiment1. Re-enable the manually removed add-on from Test Pilot experiment page
2. Navigate to "about:addons" page on "Extensions" tab and look for the re-enabled experiment.
3. Verify experiment functionality.
1. The experiment appears to be enabled.
2. The experiment that was manually removed is now displayed again on the page.
3. The experiment functionality is available and correctly working.
36
35Test Pilot - about:addons - Disable Test Pilot1. Having multiple experiments installed, choose to "Disable" Test Pilot.1. - Test Pilot is displayed as disabled.
- "Test Pilot" spaceship button is removed from Firefox toolbar
- Experiments that were installed trough Test Pilot remain unnaffected (enabled and functionality available).
37
36Test Pilot - about:addons - Re-enable Test Pilot1. Re-enable Test Pilot by clicking the "Enable" button.
2. Click the "test Pilot" spaceship icon from the main toolbar and observe the state of all experiments on "Experiments" page.
1. - Test Pilot is displayed as enabled
- "Test Pilot" spaceship button is re-desiplayed in Firefox Toolbar
2. The enabled experiments are displayed as "Enabled". Test Pilot manual disable and enable actions didn't affected other experiments installed trough it.
38
37Test Pilot - about:addons - Remove Test Pilot1. Return to "about:addons" extensions page and click the "Remove" button for Test Pilot.
2. Navigate to Test Pilot website and observe the page state.
3. Verify that none of the removed experiments functionality is available.
1. Test Pilot and all the experiments installed trough it are removed from the browser.
2. - Test Pilot spaceship icon is no longer present on the browser main toolbar
- The Test Pilot Homepage is correctly displayed
- Test Pilot install button is displayed
- None of the available experiments is displayed as enabled.
3. Browser returned to default functionality, no specific functionality related to the removed experiments is available.
39
38Test Pilot - Share page - UI1. Prerequisite: Test Pilot is not installed on the used profile.
Start Firefox and navigate to "testpilot.firefox.com/share" page (or corresponding server page).
2. Click the Firefox logo on top.
3. Install Test Pilot.
4. Navigate to "testpilot.firefox.com/share" page (or corresponding server page).
1. The Test Pilot share page is loaded and contains the next elements:
- Blue constellations background
- Firefox icon + "Firefox Test Pilot" title in white characters (Fx is in bold) on left side
- White pop up with down pointing arrow
- "Love Test Pilot? Help us find some new recruits." title in black on top part of the pop up
- "Facebook", "Twitter" and "envelope" buttons under the tile.
- "or just copy and paste this link..." text in black
- a box with black outline containing "https://testpilot.firefox.com/?utm_source=copy&utm_medium=social&utm_campaign=share-page" URL and a "copy" text button.
- Test Pilot logo under the pop up
- Footer links (mozilla, Legal, About Test Pilot, Privacy, Terms, Cookies, GitHub and Twitter).
2. You are redirected to Test Pilot Homepage.
3. Test Pilot add-on is correctly installed and you are redirected to "Experiments" page.
4. The Test Pilot share page is loaded and contains the next elements:
- Blue constellations background
- Firefox icon + "Firefox Test Pilot" title in white characters (Fx is in bold) on left side
- "Settings" Test Pilot button on top right side
- White pop up with down pointing arrow
- "Love Test Pilot? Help us find some new recruits." title in black on top part of the pop up
- "Facebook", "Twitter" and "envelope" buttons under the tile.
- "or just copy and paste this link..." text in black
- a box with black outline containing "https://testpilot.firefox.com/?utm_source=copy&utm_medium=social&utm_campaign=share-page" URL and a "copy" text button.
- Test Pilot logo under the pop up
- Footer links (mozilla, Legal, About Test Pilot, Privacy, Terms, Cookies, GitHub and Twitter).
40
39Test Pilot - Share page - Facebook share1. From the Test Pilot "Share" page, click on "F" button for Facebook share.
2. Prerequisite: Log in on a Facebook account.
Enter a description or leave it empty and click on "Post to Facebook.
3. In a new tab, navigate to the Facebook profile page and observe the shared page.
4. Click on the shared post and observe the URL of the opened page.
5. Return to the Test Pilot Share page.
1. You are redirected to Facebook login page / share Facebook page.
2. Facebook share page is dismissed and you are returned to Test Pilot Share page.
3. Facebook share post message contains:
- Test Pilot logo
- "Firefox Test Pilot" title in black chars
- "Test new Features. Give us feedback. Help build Firefox." text in black chars under title
- "TESTPILOT.FIREFOX.COM" page url in grey chars.
4. The opened page URL contains also "utm" parameters displayed.
5. Share page is correctly displayed.
41
40Test Pilot - Share page - Twitter share1. From the Test Pilot "Share" page, click on "bird" button for Twitter share.

2. Prerequisite: Log in on a Twitter account.
Observe the link displayed and click on "Tweet" button.
3. Open logged Twitter profile page and observe the tweeted page.
4. Click on the tweeted post and observe the URL of the opened page.
5. Return to the Test Pilot Share page.
1. You are redirected to Twitter login page / Twitter tweet page.

2. The link displayed contains "utm" parameters and after you choose to "Tweet", you are redirected to Twitter wall page.
3. Twitter tweet post message contains:
- Test Pilot logo on left side
- "Firefox Test Pilot" title in black chars
- "Test new Features. Give us feedback. Help build Firefox." text in black chars under title
- "testpilot.firefox.com" page url in grey chars.
4. The opened page URL contains also "utm" parameters displayed.
5. Share page is correctly displayed.
42
41Test Pilot - Share page - Email share1. From the Test Pilot "Share" page, click on "Envelope" button for email share.

2. Observe the link added to the email and after entering a valid email address, click on "Send" button.
3. Navigate to the email client page and look for the sent email.
4. Click on the sent link and observe the opened page URL.
5. Return to the Test Pilot Share page.
1. The set email client starts a new email window (if set).
2. Email is sent to the desired address.
3. The email is correctly displayed in Inbox.
4. The opened page URL contains also "utm" parameters displayed.

5. Share page is correctly displayed.
43
42Test Pilot - Share page - Link share1. From the Test Pilot "Share" page, click double click to select the URL in the box.
2. Verify if the link can be copied and pasted.
3. Prerequisite: Chose to copy a random string to dismiss the Test Pilot one.

Return to Share page and click the "copy" text button.
4. Choose to paste it on an area that accepts pasting content.
1. The URL is correctly selected.
2. The selected link can be copied and pasted.
3. The text in the box is highlighted and scrolled to end.
4. The URL is correctly pasted.
44
43Test Pilot - Restart browser- Test Pilot is already installed.1. Restart the browser with Test Pilot installed and a few or all experiments enabled and observe if user is spammed in some way.1. User should not be spammed with pages related to Test Pilot after browser restart.
45
44Chrome - UI rendering- Open your Developer Tools Console (for each browser), in order verify that no unexpected messages (console.log(), CSP errors, SRI errors, etc) are logged to the console.
- Go to https://testpilot.firefox.com/__version__ (or desired environment) and verify the deployed Circle-CI build, commit SHA and version.
1. Open Google Chrome and navigate to one of the servers depending on where you need to test:
- https://testpilot.firefox.com (public server)
- https://testpilot.stage.mozaws.net (staging server)
- https://testpilot.dev.mozaws.net (development server)
2. Hover the "Download" button.
3. Click the top "Download" button.
4. Return and click on one of the experiments cards.
5. Hover the "Download" button.
6. Click on the green "Download" button.
7. Return on the experiment details page and click on of the experiments cards from bottom of the page.
8. Click on the logo from the top of the page and hover the bottom "Download" button.
9. Click the bottom "Download" button.
1. The Test Pilot page is rendered as on Firefox. The only differences are the next ones:
- The agreement message from under the install button is replaced by "(Test Pilot is available for Firefox on Windows, OS X and Linux)" message above the green button.
- The "Install the Test Pilot Add-on" button is replaced with a green button with Firefox download logo + "Firefox" text in bold white + "Free Download" text in regular white.

Same thing in the bottom of the page.
2. - Button green background darkens.
- The download arrow displayed over Firefox icon animates and moves down a few pixels.
3. You are taken to the Firefox download page.
(https://www.mozilla.org/en-US/firefox/new/)
4. You are redirected to experiment details page.
5. - Button green background darkens.
- The download arrow displayed over Firefox icon animates and moves down a few pixels.
6. You are taken to the Firefox download page.
(https://www.mozilla.org/en-US/firefox/new/)
7. You are correctly redirected to the selected experiment details page.
8. - Button green background darkens.
- The download arrow displayed over Firefox icon animates and moves down a few pixels.
9. You are taken to the Firefox download page.
(https://www.mozilla.org/en-US/firefox/new/)
46
45Chrome - Footer links1. Open Google Chrome and navigate to one of the servers depending on where you need to test:
- https://testpilot.firefox.com (public server)
- https://testpilot.stage.mozaws.net (staging server)
- https://testpilot.dev.mozaws.net (development server)
2. Scroll to the bottom of the page and check each button / link from the footer.
1. The Test Pilot page is rendered as expected.
2. Each link redirects to the expected page.
47
46Chrome - Error 4041. Navigate to one of the next pages depending on server in test:
- http://testpilot.dev.mozaws.net/fourohfourohno
- http://testpilot.stage.mozaws.net/fourohfourohno
- http://testpilot.firefox.com/fourohfourohno
2. Click the "Home" button.
1. - In the center of the page a "Four Oh Four!" message is displayed.
- Under the message a "Home" button is displayed
- Test Pilot logo displayed under.
2. You are redirected to Test Pilot Homepage.
48
47Chrome - Experiment details page1. Click on one of the experiments cards.
2. Click on the "Changelog" link on the left side.
3. Return to the experiment page and click on the "Contribute" link.
4. Return to the experiment page and click on the "Bug Reports" link.
5. Return to the experiment page and click on the "Discuss <experiment_name>" link.
6. Return to the experiment page and click on the "data" text link from "Your Privacy" section.
7. Return to the experiment page and click on any other links displayed on "Your Privacy" section.
8. Return to the experiment page and click on "You can learn more about the data collection for <experiment_name> here" text link.
9. Return to the experiment page and click on one of the other experiments cards.
10. Scroll to the Newsletter section, enter a valid email, check the checkbox and sign up for it.
11. (Optional) Open the email client and follow the process to unsubscribe from the newsletter.
12. Return to the experiment details page and click on all the footer links.
13. Resize the browser window and observe the page UI
14. Maximize the browser window and press "F11" key.
15. Press "F11" key again to return to normal view.
1. You are redirected to experiment details page for the users that don't have Test Pilot installed and which contains the next elements:
Top Part
- Blue background with stars and constellations
- Firefox icon and "Firefox Test Pilot" text in top left side of the page
- "Ready for Takeoff?" title text
- "We're building next-generation features for Firefox. Install Test Pilot to try them!" sentence under title
- Green button with "Install Test Pilot &" text in smaller white chars + "Enable <experiment_name>" text under it, in larger and bold white chars
- "By proceeding, you agree to the Terms of Use and Privacy Notice of Test Pilot." message in white chars
- Test Pilot logo image on the right side

Middle left
- White background
- Experiment title in large black chars
- Additional notes under title
- Experiment logo
- "There are x people trying <experiment_name> right now!" text in black over a gray rectangular background tied to the experiment logo
- Experiment description in black regular characters
- Links for Changelog, Contribute, Bug Reports and Discuss page for current experiment
- [Depending on experiment] Experiment notification message in black over yellow rectangle (eg: No More 404s requires Firefox 48 or later.Make sure you're up to date.)
- "Brought to you by" text in black chars followed under it by avatars and names of the peoples that participated in the development of the experiment
- "Your privacy" section with general paragraph over white background, and the next experiment oriented paragraphs over a gray rectangular background
- "You can learn more about the data collection for <experiment_name> here." text link

Middle right
- Images from the experiment and descriptions under them

Bottom part
- Blue constellation background
- "Try out these experiments as well" message in large white chars centrally positioned
- All other available experiments tiles
- Footer links (mozilla, Legal, Privacy, Terms, Cookies, GitHub and Twitter)
2. If you are logged with a valid GitHub account, you are redirected to experiment changelog page on GitHub. Else you are redirected to GitHub login page.
3. If you are logged with a valid GitHub account, you are redirected to experiment "Code" page on GitHub. Else you are redirected to GitHub login page.
4. If you are logged with a valid GitHub account, you are redirected to experiment "Issues" page on GitHub. Else you are redirected to GitHub login page.
5. You are redirected to experiment Discourse page.
6. You are redirected to "Test Pilot Privacy Notice" page (http://testpilot.firefox.com/privacy).
7. You are correctly redirected to the corresponding pages.
8. You are redirected to a specific page where the experiment details about data collection are detailed.
9. - You are redirected to the selected experiment details page.
- The same page format is displayed for each experiment.
10. You are correctly subscribed to the newsletter.
11. You are correctly unsubscribed from the newsletter.
12. You are redirected to the expected pages.
13. Page elements are responsive and auto rearrange to accommodate the browser window size.
14. - The browser enters in Full Screen mode.
- Page elements are correctly rendered also in full screen mode.
15. Browser returns to normal maximized view.
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
Loading...
 
 
 
Test Pilot
Activity Stream
Min Vid
Page Shot
Snooze Tabs
Tab Center