| A | B | C | D | E | F | G | H | J | K | L | M | N | O | P | Q | R | S | T | U | V | W | X | Y | Z | AA | AB | AC | AD | AE | AF | AG | AH | AI | AJ | AK | AL | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
4 | VIOLET MEANS 'MUST BE DONE' DURING REGRESSION | ||||||||||||||||||||||||||||||||||||
5 | LIGHT GREY MEANS DEPRECATED OR NOT IMPLEMENTED YET | ||||||||||||||||||||||||||||||||||||
6 | https://caniuse.com | ||||||||||||||||||||||||||||||||||||
7 | browser usage statistics: https://gs.statcounter.com/ | ||||||||||||||||||||||||||||||||||||
8 | Download Sample files here: https://filesamples.com/ | SUPPORTED ENVIRONMENTS | |||||||||||||||||||||||||||||||||||
9 | Chrome 87 | Firefox 84 | Safari | Edge | |||||||||||||||||||||||||||||||||
10 | Win10 (version 2009) | ||||||||||||||||||||||||||||||||||||
11 | MacOS Catalina | ||||||||||||||||||||||||||||||||||||
12 | IPad | ||||||||||||||||||||||||||||||||||||
13 | Test your webcam: https://webcamtests.com/ | Android phone | |||||||||||||||||||||||||||||||||||
14 | Android tablet | ||||||||||||||||||||||||||||||||||||
15 | Check your screen resolution: https://whatismyresolution.com/ | ||||||||||||||||||||||||||||||||||||
16 | |||||||||||||||||||||||||||||||||||||
17 | Check what video/audio settings are supported by your browser: https://yari-demos.prod.mdn.mozit.cloud/en-US/docs/Web/API/Media_Streams_API/Constraints/_sample_.Example_Constraint_exerciser.html | ||||||||||||||||||||||||||||||||||||
18 | QA estimate | S | test | exp res | test res | comments | test res | comments | test res | comments | test res | comments | test res | comments | test res | comments | test res | comments | test res | comments | test res | comments | |||||||||||||||
19 | 16.04.2021 Smoke after 1.2 PROD | 15.04.2021 Regression before 1.2 release STG | 06.04.2021 Initial API testing | 05.04.2021 Initial API testing | 01.04.2021 Regression before 1.1 STG | 25.03.2021 GUI testing STG | 18.03.2021 Regression before 1.0 STG | 12.03.2021 Bug fixes for 'Wowza streaming' STG | 10.03.2021 Initial test for 'Wowza streaming' STG | ||||||||||||||||||||||||||||
20 | FT | fails in total | fails in total | fails in total | fails in total | fails in total | fails in total | fails in total | fails in total | fails in total | |||||||||||||||||||||||||||
21 | 0 | 0 | 0 | 6 | 1 | 5 | 3 | 1 | 6 | ||||||||||||||||||||||||||||
22 | TS | WOWZA CLOUD STREAMING | https://jira.trembit.com/browse/OMEGA-8 | 0 | 0 | 0 | 0 | - | 0 | - | 0 | 0 | 1 | - | 0 | 1 | 2 | 1 | 1 | 6 | 6 | ||||||||||||||||
23 | Check your camera resolution: https://webrtchacks.github.io/WebRTC-Camera-Resolution/ | General appearance | Home page contains: - Camera preview - RTMP input - KEY input - START STREAMING button When the stream is in progress, one button is available in the bottom of the screen: - STOP STREAMING | passed | passed | not tested | not tested | passed | not tested | passed | passed | passed | |||||||||||||||||||||||||
24 | Success basic flow: disabled auth on wowza | phone has internet connection on, you allowed the application to use the camera and mic => Stream is started without issue. | passed | passed | not tested | not tested | passed | not tested | passed | passed | passed | ||||||||||||||||||||||||||
25 | WebRTC diagnostic: https://networktest.twilio.com/ | Success basic flow: enabled auth on wowza | URL should be formed as: rtmp://user:pass@wowza.host/app => Strem is started without issues. | not tested | not tested | not tested | not tested | passed | not tested | passed | passed | failed | OMEGA-45 | ||||||||||||||||||||||||
26 | SRS: https://docs.google.com/document/d/16T7Y | Cam/Mic access | App is not launched until you give access to both cam and mic | not tested | not tested | not tested | not tested | passed | not tested | passed | passed | passed | |||||||||||||||||||||||||
27 | Figma stage 1: https://www.figma.com/file/istqjTd7 | Device orientation | Landscape is the only supported orientation. Home page and streaming page should look accurate in horizontal orientation The stream on wowza should look accurate for horizontal device orientations | passed | passed | not tested | not tested | passed | not tested | passed | passed | failed | OMEGA-46 | ||||||||||||||||||||||||
28 | Stream sound | Sound is streamed | passed | passed | not tested | not tested | passed | not tested | failed | OMEGA-59 | failed | OMEGA-59 | failed | OMEGA-47 | |||||||||||||||||||||||
29 | Stop streaming | When 'STOP STREAMING' is clicked - the stream is stopeed - you are redirected to home page | passed | passed | not tested | not tested | passed | not tested | passed | passed | passed | ||||||||||||||||||||||||||
30 | STAGE: https://stg.omega.video/ user with PRO subscription: frodo.qa+pro@trembit.com user with FREE subscription: frodo.qa+free@trembit.com | Long stream (more then 10 min) | Phone screen is not locked, stream goes smooth | not tested | not tested | not tested | not tested | not tested | not tested | passed | passed | failed | OMEGA-50 | ||||||||||||||||||||||||
31 | Internet connection issues: - simple 'off' case - 'blinking' connection case | App tryis to reconnect. If failed then 'failed to connect' error should be shown. | not tested | passed | not tested | not tested | not tested | not tested | passed | passed | failed | OMEGA-52 | |||||||||||||||||||||||||
32 | Error messages | Error messages are clear. You can't close the error pop up clicking outside of it. Error messages are inconsistent for different devices which is expected for now. | not tested | not tested | not tested | not tested | passed | not tested | passed | passed | passed | ||||||||||||||||||||||||||
33 | DEV: https://dev.omega.video/ user with PRO subscription: sam.qa+pro@trembit.com user with FREE subscription: sam.qa+free@trembit.com | Start stream when there is no internet connection | 'Failed to connect' error | not tested | not tested | not tested | not tested | passed | not tested | passed | passed | passed | |||||||||||||||||||||||||
34 | URL/Key/Credentials validation | - 'Failed to connect' error when URL is incorrect - 'Key' is not validated - Some error when username/password is incorrect for stream with auth. | not tested | not tested | not tested | not tested | not tested | not tested | passed | passed | failed | OMEGA-38 | |||||||||||||||||||||||||
35 | Try to start streaming while wowza stream is not started | Failed to connect' error | not tested | not tested | not tested | not tested | passed | not tested | passed | passed | passed | ||||||||||||||||||||||||||
36 | PROD: https://omega.video/ user with PRO subscription: bilbo.qa+pro@trembit.com user with FREE subscription: bilbo.qa+free@trembit.com | Try to stop wowza stream while your stream is in progress | Streamer should be available about stream issues on Wowza | not tested | passed | not tested | not tested | passed | OMEGA-64 | not tested | passed | OMEGA-64 | passed | passed | |||||||||||||||||||||||
37 | TS | RTMP: HOST FLOW | 0 | 0 | 0 | 0 | - | 0 | - | 0 | 1 | 1 | - | 0 | 1 | 1 | - | 0 | - | 0 | |||||||||||||||||
38 | FareBase stage (ask Sten for permissions): https://console.firebase.google.com/project/omega-staging-79856/database/data | RTMP studio available functionality | - View RTMP credentials - Copy RTMP credentials - Preview broadcast - Start/stop broadcast https://docs.google.com/document/doc | passed | passed | not tested | not tested | passed | not tested | passed | not tested | not tested | |||||||||||||||||||||||||
39 | Activity - Room relations | - Activities fall into the same room on Wowza until their time slots don't overlap - As soon as time slot for new activity overlapts with any of existing activities in the room, new room is created for new activity. | not tested | not tested | not tested | not tested | not tested | not tested | passed | not tested | not tested | ||||||||||||||||||||||||||
40 | Preview broadcast features | - ~20sec delay comparing to the source - Number of viewers when go live. Host is taken into account - Controls: - pause - volume - 'live' indicator - 'picture-in-picture' - full screen | not tested | passed | not tested | not tested | passed | not tested | passed | not tested | not tested | ||||||||||||||||||||||||||
41 | Number of viewers when go live (for host and for viewers) | - Host is taken into account - Only unique users are counted. Hence, you need to use different browsers to test this. (Incognito mode doesn't work for this case) | not tested | not tested | not tested | not tested | failed | OMEGA-68 | not tested | failed | OMEGA-68 | not tested | not tested | ||||||||||||||||||||||||
42 | TS | API | https://jira.trembit.com/browse/OMEGA-37 | - | 0 | 0 | 0 | 0 | 1 | 6 | 1 | - | 0 | - | 0 | - | 0 | - | 0 | - | 0 | ||||||||||||||||
43 | Create Activity | - activityID is unique - startTime is in ms, epoch format - duration is in min | not tested | passed | passed | failed | Configuration issues. Testing is blocked. DevOps is fixing the issue in scope of his other ticket. | not tested | not tested | not tested | not tested | not tested | |||||||||||||||||||||||||
44 | Update Activity | - works against both currently active activities and ended ones | not tested | not tested | passed | failed | not tested | not tested | not tested | not tested | not tested | ||||||||||||||||||||||||||
45 | if 'currently active' activity is updated - no new activities should be created - the KEY and URL should not be changed - HLS link should not be changed | not tested | not tested | passed | failed | not tested | not tested | not tested | not tested | not tested | |||||||||||||||||||||||||||
46 | If not active activity is updated - it is updated in FB - KEY and URL are generated from scratch - HLS is generated from scratch | not tested | not tested | passed | OMEGA-75 | failed | not tested | not tested | not tested | not tested | not tested | ||||||||||||||||||||||||||
47 | Delete Activity | - works for both currently active and ended activiries - ended activities are removed from DB - active activities are stopped, removed from wowza and from DB | not tested | passed | passed | failed | not tested | not tested | not tested | not tested | not tested | ||||||||||||||||||||||||||
48 | isStreamActive | - works only against currently active activities - 412 should be returned against an activity which is not active now - {activityId} is required param - returns true/false | not tested | not tested | passed | failed | not tested | not tested | not tested | not tested | not tested | ||||||||||||||||||||||||||
49 | TS | SECURITY | - | 0 | - | 0 | - | 0 | - | 0 | - | 0 | - | 0 | - | 0 | - | 0 | - | 0 | |||||||||||||||||
50 | DB backup | DB backup is saved once a week. | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | ||||||||||||||||||||||||||
51 | No personal data in console | There is no any personal data in console: use name, email, login etc | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | ||||||||||||||||||||||||||
52 | No debug info in console | There should be no debug info in console on any environment available for people from out of the team. There must not be any debug info in console for prod. | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | ||||||||||||||||||||||||||
53 | HTML source sanity | Page source doesn't content any unexpected into, such as - default test data, credentials - unexpected development process info | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | ||||||||||||||||||||||||||
54 | BA: session cookies have HttpOnly and Secure attributes | DevTools -> Storage -> Cookies | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | ||||||||||||||||||||||||||
55 | BA: no user enumeration vulnerability | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | |||||||||||||||||||||||||||
56 | BA: capcha/login blocking or other defend agains brute force | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | |||||||||||||||||||||||||||
57 | BA: session cookies are unique and elaborate enough | Should be just discussed if developers paid enough attention to cookie generation approach. | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | ||||||||||||||||||||||||||
58 | BA: session is stored on server side | no user info in tools->application, only cookies should be there | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | ||||||||||||||||||||||||||
59 | BA: session cookies are regenerated when preveleges are increased | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | |||||||||||||||||||||||||||
60 | BA: session is killed when log out | use back button or navigate to any internal page after log out check logaut for all availble 'log out' links if there are more the one. | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | ||||||||||||||||||||||||||
61 | BA: session cookies have relevant expiration date | the exp res should be discussed with BA and team | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | ||||||||||||||||||||||||||
62 | BA: no username/password/cookies in the URL | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | |||||||||||||||||||||||||||
63 | BA: passwords are hashed with BCrypt (PKDB2) OR Argon2 algorithms | developers should be just asked about this. If compromized hash algorithm is used, the point should be brought to BA. | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | ||||||||||||||||||||||||||
64 | BA: check if no sensitive data is cached | tools->storage->cache | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | ||||||||||||||||||||||||||
65 | BA: user is notified that someone is changing his password | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | |||||||||||||||||||||||||||
66 | BA: all BA checks should be done on all supported environments | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | |||||||||||||||||||||||||||
67 | TS | GUI DESIGN | - | 0 | 0 | 0 | - | 0 | - | 0 | 0 | 0 | 5 | 5 | 1 | 1 | - | 0 | - | 0 | |||||||||||||||||
68 | Fonts family and waight meets the requirements Use some tools, in instance What Font Chrome extension | not tested | not tested | not tested | not tested | passed | passed | not tested | not tested | not tested | |||||||||||||||||||||||||||
69 | All the colours on the page are as designed. Use some tools, in instance ColorZilla Chrome extension | not tested | not tested | not tested | not tested | passed | passed | not tested | not tested | not tested | |||||||||||||||||||||||||||
70 | Coursor behaviour. If it appears when it is expected? Does it looks as expected? | not tested | passed | not tested | not tested | passed | passed | not tested | not tested | not tested | |||||||||||||||||||||||||||
71 | Favicon is as designed | not tested | not tested | not tested | not tested | passed | failed | OMEGA-69 | failed | OMEGA-69 | not tested | not tested | |||||||||||||||||||||||||
72 | HTML/CSS standards are met. Use some checkers, in instance this one: https://validator.w3.org/ | not tested | not tested | not tested | not tested | passed | failed | OMEGA-75 | not tested | not tested | not tested | ||||||||||||||||||||||||||
73 | Page title is as designed. | not tested | passed | not tested | not tested | passed | failed | OMEGA-76 | not tested | not tested | not tested | ||||||||||||||||||||||||||
74 | Back button works as expected | not tested | passed | not tested | not tested | passed | passed | passed | not tested | not tested | |||||||||||||||||||||||||||
75 | Page looks fine when browser tab is resized or when it is opened on different devices. Use some tools, in instance Window Resizer Chrome extension | not tested | passed | not tested | not tested | passed | failed | OMEGA-79 | not tested | not tested | not tested | ||||||||||||||||||||||||||
76 | Horizontal and vertical scrolls look and work as expected | not tested | passed | not tested | not tested | passed | failed | OMEGA-79 | not tested | not tested | not tested | ||||||||||||||||||||||||||
77 | TS | WEBRTC CHECKLIST | use chrome://webrtc-internals | - | 0 | - | 0 | - | 0 | - | 0 | - | 0 | - | 0 | - | 0 | - | 0 | - | 0 | ||||||||||||||||
78 | Generic video features | Check the audio/video settings are applied correctly | Change video/audio quality in the application 'Settings' and verify if they are actually get changed. Use chrome://webrtc-internals -> 'GetUserMedia Requests'. | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | |||||||||||||||||||||||||
79 | Check the cases with muted mic and cam. | When camera is muted, there should be no video stream transmitted. Use chrome://webrtc-internals -> 'Stats graphs for RTCOutboundRTPVideoStream' to check. | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | ||||||||||||||||||||||||||
80 | When microphone is muted, there should be no audio stream transmitted. Use chrome://webrtc-internals -> 'Stats graphs for RTCOutboundRTPAudioStream' to check. | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | |||||||||||||||||||||||||||
81 | Check if actual video/audio quality corelate with the 'settings' paramethers | Compare the audio/video quality values you have set in the application 'Settings' with what you have in chrome://webrtc-internals. Use chrome://webrtc-internals -> 'Stats graphs for RTCOutboundRTPVideoStream' and chrome://webrtc-internals -> 'Stats graphs for RTCOutboundRTPAudioStream' to check. | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | ||||||||||||||||||||||||||
82 | Video layouts are correct under different window size | Check the video layout on all supported devices Resize the desktop browser window during the video call and while you are in the waiting room | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | ||||||||||||||||||||||||||
83 | Waiting room allows to set the settings and preview video stream | Waiting room looks and provided options accordingly to design and SRS. However, basically it should allow select the camera, microphone and provide the video stream preview. | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | ||||||||||||||||||||||||||
84 | Browser access to Camera/microphone | In general it should be possible to join the session without cam/mic or with blocked access to the cam/mic. This behaviour can be different if it is described in the SRS. | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | ||||||||||||||||||||||||||
85 | Behaviour in the browsers which don't support WebRTC | There should be some clear and user friendly behaviour for the browsers which don't support WebRTC. | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | ||||||||||||||||||||||||||
86 | Non fuctional features | Check if the session goes without issues for TURN network configurations. | To check this, let one participant connect from the office and another one - from out of the office. (We have TURN configured in the office) | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | |||||||||||||||||||||||||
87 | All server connections should be made via 443 port, includiing the TURN server case | Use devTools -> Network to check that the request is transmitted over https protocol | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | ||||||||||||||||||||||||||
88 | Check the CPU usage during the session | CPU usage should be moderate. CPU should not 'take off'. If CPU gets noisy the test can be considered to be failed. | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | not tested | ||||||||||||||||||||||||||
89 | theEnd | ||||||||||||||||||||||||||||||||||||
90 | |||||||||||||||||||||||||||||||||||||
91 | |||||||||||||||||||||||||||||||||||||
92 | |||||||||||||||||||||||||||||||||||||
93 | |||||||||||||||||||||||||||||||||||||
94 | |||||||||||||||||||||||||||||||||||||
95 | |||||||||||||||||||||||||||||||||||||
96 | |||||||||||||||||||||||||||||||||||||
97 | |||||||||||||||||||||||||||||||||||||
98 | |||||||||||||||||||||||||||||||||||||
99 | |||||||||||||||||||||||||||||||||||||
100 | |||||||||||||||||||||||||||||||||||||
101 | |||||||||||||||||||||||||||||||||||||
102 | |||||||||||||||||||||||||||||||||||||
103 | |||||||||||||||||||||||||||||||||||||