VR Vision Internal Standards

Development QA & UAT Standards

Standards and procedures for client User Acceptance Testing and internal development QA across VR Vision projects.

Part 1 — Client UAT Standards Process

Preliminary Setup

1. Users have watched the How-To Video for Quest 2 Headset
2. Users have watched the Tutorial Playthrough Video
3. Users have watched the Main Menu and Tool Table videos
4. Headsets are confirmed connected to Wi-Fi and to Vision Portal
5. Confirmation and retrieval of content to test

Performing UAT on Headset & First-Time User

1. Confirm headset is turned on
2. Confirm boundary is set
3. Confirm headset is connected to a stable Wi-Fi network
4. Confirm headset battery is above 80%
5. Confirm both controllers' batteries are above 50%
6. Confirm apps are updated to latest version:
   1. Version code will be verified by VR Vision before the revision period
   2. Launch the Main Menu application
   3. Confirm the Version Code in the VR Vision Main Menu UI
   4. If incorrect version, consult VR Vision or use ManageXR to confirm the device is up to date
7. Enter username or log in using SSO
8. If first-time user, load into the VR Vision Tutorial and complete it
9. Once finished, play with the Tool Table in the Main Menu
10. Once the user is feeling 70%+ confident, launch into a course. The user needs to demonstrate they can:
    1. Teleport
    2. Interact with UI
    3. Pick up and hold objects
    4. Do inspections
    5. Use the Watch UI
    6. Use objects on the Tool Table

Performing UAT Testing on Course Content

1. The course to launch into needs to be decided before UAT
2. Confirm all 10 steps above have been completed and the user is confident in their VR skills
3. To launch into the chosen course:
   1. Select the course in the main menu
   2. Select from the following modes: Single Player, Trainer Led, Trainee Led
   3. If the user has chosen trainer or trainee led (multiplayer), they can:
      1. Join an existing room listed on the right-hand side (displays the room creator's name)
      2. Create a new room so other users can join (for more advanced users)
   4. Trainer rooms are created by a trainer who has previously gone through the module in VR. This mode has no narration, subtitles only. Results screen shows for everyone including trainees. For inspections, trainees only need to grab their PPE at the start — they won't be part of the visual inspections.
   5. Trainee rooms can be created by any user. Same steps as above with no difference; anyone can join as an observer, grabbing PPE only. Everyone has narration and guidance.
4. Once a course is created, users will be in the lobby waiting room. When ready, they click and hold the start button to begin the module.
5. Once the course has begun, the UAT team should compare the steps to the signed-off storyboard to confirm all steps are correct and the flow is smooth.
6. If any issues arise, they should be entered on the Bugs & Revisions spreadsheet by the revision period. Pair bugs and revisions with a video or photo where possible.
7. Once a mode is completed, each remaining mode should be tested until all confirmed content has been covered.

Bugs & Revisions Spreadsheet

Filling out the Bugs & Revisions spreadsheet should be done during the UAT process.

What should be on the spreadsheet

1. Bugs with the course such as:
   1. Game-breaking issues (unable to continue application)
   2. Completely missed features
   3. Slightly breaking issues (able to continue) — e.g., tool disappears and comes back
   4. Minor bugs — incorrect spellings, difficult-to-reach inspections
   5. UI issues
   6. Visual issues
   7. Sound issues
   8. Any steps missing or order wrong in storyboard
2. Revisions are categorized as:
   1. Minor revisions: Model & texture changes, UI & sound changes, special effects, adding 2D elements (icons, etc.), changes to workflow order, changes to data range/output
   2. Major revisions: Any major development work not included in the ADD/SOW. Typically refers to creating new models & textures, adding new workflows or steps, adding new integration points for analytics, or any changes to the Main Menu or Tutorial.
3. The sheet should remain open during testing and be filled out in real time
4. Description should be a short one-sentence description of the issue
5. Steps to reproduce should expand on:
   1. What course, mode, and step
   2. Description of what happened and what led to the issue
   3. Can it be repeated
   4. Does it break or stop the user from continuing
   5. Attach photo and video if possible
6. Continue testing — do not stop and restart if the issue does not break the course. If unable to continue, use the Skip feature on Vision Portal.
7. Warn others of bugs and revisions before having a new user run the course to avoid repeated reports.
8. If there is a known reproducible bug that breaks the game and can be avoided using Vision Portal's Skip Step, use Skip Step.
9. Rebuilding the application to fix a course-breaking bug is not always possible on demand and must go through VR Vision's emergency QA policy — though this is extremely rare.

Single Sign-On (SSO)

1. User selects SSO Sign-In from the main menu
2. User is presented with the Microsoft sign-in screen
3. User inputs corporate email address
4. User uses MFA device to input the code shown on the VR screen
5. Verify username appears in the main menu

Vision Portal

A video will be provided for trainers using the portal to watch beforehand.

1. Confirm laptop is connected to a stable Wi-Fi network
2. Open Chrome and navigate to the client's Vision Portal instance
3. Log in with the credentials provided by VR Vision
4. Open the Remote Assistance > Trainer Yard tab
5. Confirm a VR user is inside the Main Menu, lobby, or training content
6. Observe the table — confirm the user is indicated as online and the Unit No matches the headset
7. Click Assist Panel and confirm you can see the VR user's screen share
8. Message the user and confirm they can receive messages (visible from screen share)
9. Ensure the user is inside the pre-decided course and use the Skip Step button
10. Scroll to the Course Launching area and follow the sequence to launch the user back to the Main Menu
11. Click the Analytics tab and observe the dashboard for usage and metrics

---

Part 2 — Development QA

Pre-Testing: Preliminary Setup

1. Headset is connected to Wi-Fi
2. Headset has battery above 80%
3. Controllers have battery above 50%
4. Install the proper APK to perform testing (Main Menu and course applications)
   1. Launch the Main Menu application
   2. Verify the Version Code in the VR Vision Main Menu UI with the developer
   3. If incorrect version, consult the developer

Notes for Developers

1. For first-time testers, developers should be observing via streaming Meta Quest Developer Hub
2. For subsequent testing, the developer can be inside the module or not involved entirely
3. User experience issues — order of operations:
   1. Sometimes it's better to say nothing and let testers problem-solve by observing
   2. Have testers think out loud
   3. Assist only as a worst-case scenario if they cannot continue
   4. Take notes and add as a task on Notion (not a bug)

Testing

• Record in headset. Instructions are available here.
• If Log Writer is not enabled, use Meta Quest Developer Hub to track logs
• Create a test thread so as not to spam the channel
• Document findings however is most efficient — just write things down
• Post these in the relevant Slack channel (for developer and tester)

Submitting Bugs

Post encountered bugs in the test thread.

1. Bugs with the course such as:
   1. Game-breaking issues (unable to continue application)
   2. Completely missed features
   3. Slightly breaking issues (able to continue application) — e.g., tool disappears and comes back
   4. Minor bugs such as incorrect spellings or difficult-to-reach inspections
   5. UI issues
   6. Visual issues
   7. Sound issues
   8. Any steps missing or order wrong in storyboard
2. Steps to reproduce should include:
   1. Who found it
   2. What course, mode, and step
   3. Description of what happened — one short sentence
   4. What led to the issue
   5. What you were doing in the last few seconds
   6. Is it repeatable (should be yes if it was the first encounter)
   7. Does it break or stop the user from continuing
   8. Attach video/photo
3. Continue testing — do not stop and restart if it does not break the course. If unable to continue, use the Skip feature.
4. Warn others of bugs before having a new user run the course, otherwise there will be repeated information and repeated issues.
5. If there is a known reproducible bug that breaks the game and can be avoided using the Vision Portal Skip Step, make sure to skip the step.

Bug Resolution

• Whitebox & blackbox testing. Test a bug in project context and in isolation, then compare results.
• Ask around. Someone may have already encountered the issue recently or in a previous project and knows the solution.
• Assume democracy 🐛