ABCDEFGHIJKLMNOPQRSTUVWXYZAAABACADAEAFAGAHAIAJAKALAMANAOAPAQARASATAUAVAWAXAYAZBABBBC
1
REF NUMBERTitleAuthorsYearExpert SpaceTrainee SpaceNr. CollaboratorsTimeApplication DomainSetup SymmetryInteraction Medium - Expert (to Trainee)Interaction Medium - Trainee (to expert)Output - ExpertOutput - TraineePerspective SharingUser Studies?Training / AssistanceNotesCountry where research was conducted
2
RealARVRRealARVRExpertsTraineesSynchAsynchHealthSports Education Industry and Maintenance Navigation Other?SymmAsymmControllersMidAir Gestures/Bare HandsTouch Screen
Keyboard&Mice
Head GazeEye GazeNo interactionControllersMidAir Gestures/Bare HandsTouch Screen
Keyboard&Mice
Head GazeEye GazeNo interactionVisualAudioHapticsSmellVisualAudioHapticsSmell1st Person PoV3rd Person PoVBothYes (Nr. People)No
3
1Distant Assist Cursor (DAC): Designing an Augmented Reality System to Facilitate Remote Collaboration for Novice UsersKervegant C,Castet J,Vauchez J,Bailly C2021x - desktopx - Projection based AR11xAll purpose approach/devicex - (desktop Real - Projection AR)expert uses a traditional computer and share the corresponding mouse cursor to guide the local userx - can only speakx - sees a portion of the trainees workspace that is being captured by a camera in 1st person view.x - phonex - sees a projected cue from the expert (white luminous dot) x - phonex - expert sees 1p pov of the trainee. 20 participants as expert + 1 local user (actor) AssistanceAR remote collaboration system for sharing a virtual cursor. DAC relies on a camera and a projector sharing the same optical center. France
4
2Mixed Reality Light Fields for Interactive Remote AssistanceMohr P,Mori S,Langlotz T,Thomas BH,Schmalstieg D,Kalkofen D2020x - mobile - Samsung Galaxy S9 smartphonex - mobile - Samsung Galaxy S9 smartphone111General Approach - use case scenarios: the calibration of an oscilloscope and the maintenance of a computer x (mobile AR - mobile AR)x - identification of objects (using a circular outline around the object
), the communication of object
movements (using a arrow), placement of objects (drawing the
outline of the object at the target place), handwriting.
xx - sees a portion of the local trainee's workspace: Local user captures an overview of the environment by acquiring a spaced collection of images by pointing the tracked mobile camera at objects identified as potentially interesting.x - sees real local environment + annotations from the expert in his environmentxexperiment 1 - 20, experiment 2 - 16, experiment 3 - 10assistanceThey introduce an interaction method for quickly annotating light fields in 3D space without requiring surface geometry to anchor annotations.AustriaNew ZealandAustralia
5
3A User Study on Mixed Reality Remote Collaboration with Eye Gaze and Hand Gesture SharingBai H,Sasikumar P,Yang J,Billinghurst M2020VR headset (HTC Vive
Pro Eye) with a built-in eye tracker + Leap Motion.
AR display (Magic Leap One)11xGeneral Approach x (AR-VR)x - Tracked with Leap Motionxx -immersed in 3D Live Panorama/reconstruction of trainees environment. Also sees own gaze ray.xx - sees real local environment + hands and gaze raycast from the expert in his environmentxxx - 24 participants in pairsassistanceSystem that enables a local worker to share a live 3D panorama of his/her surroundings with a remote expert. The remote expert can also share task instructions back to the local worker using visual cues in addition to verbal communication. User study shows sharing augmented gaze and gesture cues from the remote expert to the local worker provide a significantly stronger sense of co-presenceNew ZealandSwitzerland
6
4On the Shoulder of the Giant: A Multi-Scale Mixed Reality Collaboration with 360 Video Sharing and Tangible InteractionPiumsomboon T,Lee GA,Irlitti A,Ens B,Thomas BH,Billinghurst M2019x (HTC Vive)x (HTC Vive)11xGeneral Approachx(AR-VR)x "wave" - summoning and denying the expertx - immersed in 360 live video shared by the traineexx - sees real local environments + avatar representation of expert attached to the camera he controlsxx - local trainee can put expert on shoulder (1st px pov) or put in on a surface in a tripod (3rd p pov) x - 24 trainees / 1 "actor" expertassistanceExplored four 360-camera placement positions on the Giant. The results show users prefer a shoulder mounted camera view, while a view frustum with a complimentary avatar is a good visualization for the Miniature virtual representation.AustraliaNew Zealand
7
5Loki: Facilitating Remote Instruction of Physical Tasks Using Bi-Directional Mixed-Reality TelepresenceThoravi Kumaravel B,Anderson F,Fitzmaurice G,Hartmann B,Grossman T2019x - The user wears a mixed-reality display (HTC Vive + Zed Mini) allowing them to transition between AR and VRx - The user wears a mixed-reality display (HTC Vive + Zed Mini) allowing them to transition between AR and VRx - The user wears a mixed-reality display (HTC Vive + Zed Mini) allowing them to transition between AR and VRx - The user wears a mixed-reality display (HTC Vive + Zed Mini) allowing them to transition between AR and VR11xxGeneral Approachx - same hardware for both users but allows: VR -VR, AR-AR and AR-VR x - 6DoF Vive Controllersx - 6DoF VIVE controllersx - sees Avatar of trainee (AR) + Hologplyph (VR) + video of trainee (VR) and annotations (AR and VR)xx - sees Avatar of expert (AR) + Hologplyph (VR) + video of expert (VR) and annotations (AR and VR)xxtrainingapproach for bidirectional mixed-reality telepresence systems for teaching physical tasks, where they leverage video, audio, and spatial capture with both AR and VR to allow users to explore and annotate the local and remote environments.CanadaUSA
8
6Mini-Me: An Adaptive Avatar for Mixed Reality Remote CollaborationPiumsomboon T,Lee GA,Hart JD,Ens B,Lindeman RW,Thomas BH,Billinghurst M2018x (HTC Vive)x (Hololense HMD)11xGeneral Approach - use case scenarios: assisting a local worker and collaboration in urban planningx (AR-VR)Pointing (by holding down a trigger button on the controller) + Raycast"Pinch/air tap" - enabling/disabling the expertx - sees all recontructed environment of the local trainee in VR xx - sees real local environment + avatar representation of the expert in his environmentxxx - 16 trainees / 1 "actor" expertassistanceMini-Me avatar represents the VR user’s gaze direction and body gestures while it transforms in size and orientation to stay within the AR user’s field of view.AustraliaNew Zealand
9
7Multi-View Gesture Annotations in Image-Based 3D Reconstructed ScenesNuernberger B,Lien KC,Grinta L,Sweeney C,Turk M,Höllerer T2016x - desktopMobile AR11xxGeneral approachx (desktop - Mobile AR)xx - Google Project Tango TabletCan follow the trainee's device pose or virtually navigate independently of the local user’s view.xx - sees local environment + annotations of the expertxx - expert chooses12 participantsassistanceMulti-view annotation method. Interactive disambiguation of the 2D gestures via a second annotation drawn from another viewpoint, triangulating two drawings to achieve a 3D result.USA
10
8OptoBridge: Assisting Skill Acquisition in the Remote Experimental CollaborationSun H,Zhang Z,Liu Y,Duh HB2016x - desktop + LeapMotionOptical See Through HMD11xlearning of skills to set up an interferometer, calibrate the light path, and observe the
phenomena of equal inclination interference
x (desktop - AR)x - though leap motionxSees the trainee's environment streamed to his computer in a first person perspective (recorded by the trainee's HMD)x - sees local environment + AR tags placed by the expert + expert's handsx - trainee captures his perspective of the workspace with integrated camera on the HMD and streams to expert xTrainingEnables the remote teacher to monitor the experimental scenario as well as the detailed optical phenomena through the transmitted video captured on the local side. The local learner is equipped with HMD and can be indicated by virtual hands and augmented annotations controlled by the teacher’s gestures and follow the guidance to get their skills practiced.ChinaAustralia
11
9Virtual Replicas for Remote Assistance in Virtual and Augmented RealityOda O,Elvezio C,Sukan M,Feiner S,Tversky B2015x - can switch between AR and VR (Sony HMZ-T3W opaque stereo HWD)x - can switch between AR and VR (Sony HMZ-T3W opaque stereo HWD)x - Canon HM A1 stereo video-see-through HWD11xx - "maintenance and repair"x - (VR-VR)x - (AR-VR)Anotates and manipulates virtual replica through tracked mouse, lazy susan turntable, and Leonar3Do bird controller.xx - sees video stream of portion of the local trainee workspacex- sees anotated and manipulated virtual replicas created by the expert in his envoronmentExpert can switch seamlessly between a mirrored VR representation of the trainee's environment, and shared live or frozen AR views from the perspective of the local user.x - 22 participants in pairstask assistanceexpert can create and manipulate virtual replicas of physical objects in his virtaul environment to refer to parts of the trainees physical objects and to indicate actions on them. The expert points in 3D to portions of virtual replicas to annotate them. OR. The expert demonstrates actions in 3D by manipulating virtual replicas, supported by constraints and annotations. Assumes that manipulable physical objects have been modeled and are tracked in 6dof.USA
12
10In Touch with the Remote World: Remote Collaboration with Augmented Reality Drawings and Virtual NavigationGauglitz S,Nuernberger B,Turk M,Höllerer T2014x - desktop - touchscreen interfacex - mobile AR (tablet)11xGeneral approach - use case in industry/maintenancex - (desktop- mobile AR)x - Using a single finger, the remote user can draw annotations into the scene.x x - expert can freeze the viewpoint , save the current viewpoint, and go to the live view or any previously saved view. Note: only parts of the scene that have previously
been observed by the local user’s camera are known and can be
rendered.
x - sees annotations from the expert in his local environment through mobile ARExpert can see live (first person view) or navigate to parts that were previously captured by the trainee's camerax - 11 participantsassistanceTouchscreen interface for creating freehand drawings as world stabilized
annotations and for virtually navigating a scene reconstructed
live in 3D
USA
13
11Designing a Remote Video Collaboration System for Industrial SettingsDomova V,Vartiainen E,Englund M2014x - desktopx - mobile AR (tablet)11xxx - (desktop- mobile AR)x - can annotate (drawing ) and point (pointer based)x - can annotate (drawing ) and point (pointer based)x - sees live stream from local workspace and can take a snapshot "feezing" the view" + sees annotations and pointers fro the local traineex - sees annotations and pointer from the expert in his local environment through mobile AR + can see the "frozen" viewpoint from the expertxx - 4 trainees and 2 expertsassistanceSmartphone/tablet application used by the field worker to capture and stream video to a desktop application of the remote expert. Note: Participants expressely mentioned they missed a voice/chat environment that for them was vital in such settings.Sweden
14
12Object-Based Touch Manipulation for Remote Guidance of Physical TasksAdcock M,Ranatunga D,Smith R,Thomas BH2014x - desktopx - projection based AR (spacial augmented reality SAR)11xx - General Approachx- (desktop - SAR)x - can sketch on top of video stream or "move" boxed representations of the physical objects (tracked with a kinect)xx - sees live video of the trainee's workspace with the projected cues + a virtual representation of physical objects (bounded boxes)x - no audio channel but participants could talk during the sessions as they were co-locatedx - sees local environment with projected cues from the expertx - no audio channel but participants could talk during the sessions as they were co-locatedStudied a top and side camera - found top is better for object-based interactions while for scketching its same samex - 12 pairs (24)assistanceEnables a remote expert to observe a live stream of local worker’s environment and directly specify object movements via a touch display. Local trainee sees cues through SAR. They evaluate two methods of remote worker interaction, object-based and sketch-based, and also investigate the impact of two camera positions, top and side. Found for object-based interactions the top view was better, and no diference in sketching.Australia
15
133D Helping Hands: A Gesture Based MR System for Remote CollaborationTecchia F,Alem L,Huang W2012Stereoscopic HMDx - desktop11xGeneral approach - use case Lego assemblyx - (VR - Desktop)x - tracked with kinectxx - is immersed in a 3D rendering of the trainee's space (captured with a kinect) and seees own hands in that environmentx - sees a "replica" of own environment on the screen, with AR expert's hands superposed x 1 actor trainee + 4 expert participantsassistance3D meshes captured from the worker space and the helper space are co-located and fused together so that four hands are available in the virtual scene: 2 from the worker and 2 from the helper.Italy
16
14T.A.C: Augmented Reality System for Collaborative Tele-Assistance in the Field of Maintenance through InternetBottecchia S,Cieutat JM,Jessel JP2010x - desktopx - MOVST AR goggles11xxx - (Desktop - AR)x - able to select parts of the image to outline and place pre-defined alements such as arrows to direct local traineexx - can see a stream of the local trainee's environment on a screenXx -sees own environment with augmented cues from the expert (arrown, annotations, outlines)Xxyes but no info on how many participantsassistanceFrance
17
15ShiSha: Enabling Shared Perspective With Face-to-Face Collaboration Using Redirected Avatars in Virtual RealityHoppe AH,van de Camp F,Stiefelhagen R2021x - HTC Vive Prox - HTC Vive Pro1 +1 +xGeneral Approach x (VR HMD - VR HMD)xxx - sees virtual world + dyad of the other trainees/experts "artificalliy" positioned on his side/front, with ray cast to point to the workspacexx - sees virtual world + dyad of the other trainees/experts "artificalliy" positioned on his side/front, , with ray cast to point to the workspacexx - To provide all users with the same perspective, each authority-user stands at the ideal location, e.g. in front of an object of interest, in her/his authority world. To avoid overlapping with the avatars of connected-users, their avatars are shifted to different locations, e.g. to the left or right of the authority-user. This allows a shared perspective interaction with the advantages of a side-by side/face-to-face communication.32 in pairsboth ("support")They place all users at the same location to provide a shared perspective. To avoid overlapping body parts, the avatars of the other connected users are shifted to the side. A redirected body pose modification corrects the resulting inconsistencies.Germany
18
16I’m in Control! Transferring Object Ownership Between Remote Users with Haptic Props in Virtual RealityAuda J,Busse L,Pfeuffer K,Gruenefeld U,Rivu R,Alt F,Schneegass S2021x - VR HMD (HTC Vive Pro)x - Oculus Rift11xGeneral Approachx (VR HMD - VR HMD)x - hands tracked via tracking gloves: OptiTrack motion capture system (only wrist position - static fingers)x - hands tracked via tracking gloves: OptiTrack motion capture system (only wrist position - static fingers)x - sees virtual world + own hands + partner's hands.x - tecnique to enhance ownership of virtual objects with haptic props. SingleOwnership restricts virtual objects to local haptic props, while SharedOwnership allows collaborators to take over ownership of virtual objects using local haptic props. x - sees virtual world + own hands + partner's hands.x - tecnique to enhance ownership of virtual objects with haptic props. SingleOwnership restricts virtual objects to local haptic props, while SharedOwnership allows collaborators to take over ownership of virtual objects using local haptic props. x - face to face12 trainee and 2 actor expertstraining or assistanceFocus on enabling passive haptics for ownership of virtual objects. Germany
19
17How About the Mentor? Effective Workspace Visualization in AR TelementoringC. Lin; E. Rojas-Muñoz; M. E. Cabrera; N. Sanchez-Tamayo; D. Andersen; V. Popescu; J. A. Barragan Noguera; B. Zarzaur; P. Murphy; K. Anderson; T. Douglas; C. Griffis; J. Wachs2020x (LCD Monitor - desktop)x - Microsoft’s HoloLens11xSurgical telementoring, specifically for cricothyroidotomy training in austere settings.General Approachx (LCD tabletop - AR HMD)x - can anottate image with his fingers (free hand sketched incision lines, or dragged-and-dropped instrument icons.)xx - sees trainee's workspace
captured by a head mounted camera.
x - they can talk to each otherx - sees own local space + AR annotations from the expert rendered on top x - they can talk to each otherx - trainee wears an HMD with incorporated camera and streams image from 1st pov to the expertPilot with 30 people + study 20 trainees and 2 experts (actors) TrainingMethod for robust high-level stabilization of a mentee first-person
video to provide effective workspace visualization to a remote mentor.
Why? -> As the mentee moves their head, the mentor’s visualization of the workspace changes frequently, unexpectedly, and substantially and can cause disturbances.
USA
20
18An Augmented Reality Online Assistance Platform for Repair TasksSun L,Osman HA,Lang J2021x (mobile AR - Mi Note 2 smartphone)x (mobile AR - Honor 6x smartphone)11+xRepair Tasksx (mobile AR - mobile AR)x - to control interface BUT has a physical copy of the same componet as the traineexx - sees own physical replica of the object the trainee is trying to repair, and a virtual replica of the components the trainee is moving at that moment.x - they can talk to each otherx - sees own workspace with the expert's operations on his own replica displayed and overlaid as 3D virtual objects on itx - they can talk to each otherx30 "trainees" and 1 "actor" expertAssistanceMobile AR platform that assists a remote expert in guiding users through repair tasks online. Remote expert can compose complex instructions and present them to users using 3D virtual objects. Different from other remote assistance methods, the proposed method assumes the existence of two copies of the same product, one on the expert side and the other on the new user’s side.Canada
21
19A Technique for Mixed Reality Remote Collaboration Using 360 Panoramas in 3D Reconstructed ScenesTeo T,F. Hayati A,A. Lee G,Billinghurst M,Adcock M2019x - VR HMD (HTC Vive)x - Microsoft’s HoloLens + RICOH Theta V mounted on it11xGeneral approachx - (AR HMD - VR HMD)x - with raycasts to pointxx - is immersed in a static reconstruction of the trainee's space. Can move freely. Can opt to switch into a live 360 panorama video at any time by pressing a button on the VR controller. Also sees a representation of the trainee's head (ball)x - Voice Over IP (VoIP)x - sees own worspace augmeted with a representation of the expert (low polygon virtual head) plus a "hands" (cilindrical rep of controllers) with raycastsx - Voice Over IP (VoIP)x - Can either move freely and independently from trainee in a reconstructed environment, or opt to immerse into the same point of view as the local user and see live view of the local user’s environment (streamed through a camera). - expert loses the ability to move independently, but they can still look around independently.10 participants (in five pairs)AssistanceExplores a prototype system that combines 360 panoramas into a 3D reconstructed scene. Limitation: the prototype may not be applicable
for tasks that involves moving large objects that may change the room layout since there is no live 3D reconstruction of the environment
Australia
22
20360Anywhere: Mobile Ad-Hoc Collaboration in Any Environment Using 360 Video and Augmented RealitySpeicher M,Cao J,Yu A,Zhang H,Nebeling M2018x - Projection Based ARx - Projection Based AR1+1+x - ( extra feature: can rewind the live stream by 10 seconds, e.g., in case they missed something important)Provides functionality for saving all digital artifacts in a session at their exact positions, as well as a snapshot of the current 360-degree stream (including all annotations and projections), which can be loaded at a later point in time even if the system was restarted. General approachx (Projected AR - Projected AR)x - drawing, placing images, and writing text directly into the 360-degree stream.x - can see other collaborator's gaze zone and share their ownx - drawing, placing images, and writing text directly into the 360-degree stream.x - can see other collaborator's gaze zone and share their ownx - see augmeted annotations projected onto own environmnet provided by the trainees + trainees' environment through live 360 feed on screen + Gaze (“colored cones that indicate where each collaborator is looking in the 360-degree stream”)separate Skype-like channel for additional audio-visual communicationx - see augmeted annotations projected onto own environmnet provided by the experts + experts' environment through live 360 feed on screen + Gaze (“colored cones that indicate where each collaborator is looking in the 360-degree stream”)separate Skype-like channel for additional audio-visual communicationx - everyone can opt to “gain control of everyone’s 360 feed, so that the direction of viewing is synchronized across all collaborators” 10 studentsAssistanceThrough a projective AR set-up, remote collaborators are able to add digital annotations to a real-world environment
by drawing directly into a 360-degree video stream. Explore different features to tackle existing 360 stream approaches limitations such as lack of gaze awareness, out of synch views and missed gestures.
ChinaUSA
23
21Virtual Interactive Presence in Global Surgical Education: International Collaboration Through Augmented RealityDavis MC,Can DD,Pindrik J,Rocque BG,Johnston JM2016x - Mobile ARx - Big Screen11+ (co-located team of trainees)xsurgical assistance and trainingx - (Mobile AR - Desktop)x - to use gestures like pointing or show a tool x - draw on live streamxx - sees live video transmission of trainees from fixed perspective chosen by them.x - audio channelx - see a semitransparent overlay of remote image on their local
station image (as a single image)
xx - perspective is chosen by the trainee team27 ETV/CPCs were performed at
Children’s Hospital, all for aqueductal stenosis
bothAllows surgeons to provide real-time virtual assistance and training: provides a hybrid perspective of local and remote video feeds, allowing a remote surgeon to digitally" reach into the surgical field" to highlight anatomic structures and providing a visual demonstration of
complex operative techniques.
USA
24
22Onebody: Remote Posture Guidance System Using First Person View in Virtual EnvironmentHoang TN,Reinoso M,Vetere F,Tanin E2016x - Oculus Riftx - Oculus Rift11xx - posture training (martial arts but scalable)x (VR with kinects x2)xxx - sees movements of the trainee overlaid on his body x - sees movements of the instructor as if they were the instructor (skeletal tracking of an instructor and the student (kinect), rendered as overlaid avatars) - concurrent feedback-> skeleton has colour indications on whether the the trainee is doing poses correctly.x - allows student to step into the instructor’s bodyone instructor and
23 students.
Trainingremote posture guidance during sports or physical activity training. Skeletal tracking of the instructor and the students, rendered as virtual avatars. Using a virtual reality headset, the student can visualise the movement of the instructor’s avatar, rendered in place of their own body. Compared performance of Onebody in terms of posture matching accuracy and user’s preference, with pre-recorded video, video conferencing and third person view VR.Australia
25
23Comparing Visual Representations of Collaborative Map Interfaces for Immersive Virtual EnvironmentsA. Santos-Torres; T. Zarraonandia; P. Díaz; I. Aedo2022x - Oculus Questx - Oculus Quest11xSensemakingVRxxcondition 1 workspace + avatar
condition 2 workspace only
voice in one conditioncondition 1 workspace + avatar
condition 2 workspace only
voice in one conditionx24 participants (dyads)Bothstudy conducted colocated, but insights work too for remote. focus on whether or not to present users with an avatar. results indicate that adding an avatar is well perceived by users. scenario is emergency response. no distinction between expert and trainee, all equal collaborators.Spain
26
24Evaluating Remote Virtual Hands Models on Social Presence in Hand-based 3D Remote CollaborationB. Yoon; H. -i. Kim; S. Y. Oh; W. Woo2020x - HTC Vivex - Meta2 OST HMD111ASLx - (AR HMD - VR HMD)xxonly seeing hands only seeing handsx48 participantsbothexpliring the effects of different tyoes of hand rendering on collaboration. Korea
27
25Using a Head Pointer or Eye Gaze: The Effect of Gaze on Spatial AR Remote Collaboration for Physical TasksP. Wang; X. Bai; M. Billinghurst; S. Zhang; W. He; D. Han; Y. Wang; H. Min; W. Lan; S. Han2020x - HTC VIVE VR HMDx - projector-based AR111x (lego assembly)x - (VR HMD, SAR)x (annotations)xxExperts sees camera view of local worker.Sees annotations of remote expertexperts sees 3rd person view.
remote workers sees annotated real world
30 participantsbothexploring the effects of showing remote experts the head pointers and gaze pointer of the local worker.ChinaAustralia
28
26World-Stabilized Annotations and Virtual Scene Navigation for Remote CollaborationGauglitz S,Nuernberger B,Turk M,Höllerer T2014x - tabletx - smartphone111xx - smartphone AR / 2D desktop windowxx2d browser window with view of trainee. 3d reconstructionsees annotation of remote experts through smartphone in pass-through ARx60 participants (30 dyads)Assistancesmartphone-based system that allows trainees to see expert annotations in real-time as overlay. Presents expert with 1st person view of remote user and a 3D reconstruction so they can move the camera.USA
29
27Visual and haptic collaborative tele-presenceAnsar A,Rodrigues D,Desai JP,Daniilidis K,Kumar V,Campos MF2001x - HMDx10-1xxx - only expert has AR headsetxxexpert sees remote siterobotic armxexpert sees perspective of camerasxtrainingRemote system for surgical training and other tasks that require robotic manipulation. USABrazil
30
28Belonging There: VROOM-Ing into the Uncanny Valley of XR TelepresenceJones B,Zhang Y,Wong PN,Rintel S2021xx11xx - online meetings but the technology is not restrictedx - Trainee (local user) weas AR headset to see expert's avator overlay. Expert wears VR headset that livestreams trainee's 360 view and controls a remote robot placed in trainee's local space.x - to control remote robotxexpert sees 360 view of the traineeavatar overlay in AR - overlaid onto the physical robot in trainee's space, which is controlled by the expert x20 participants (10 dyads)Assistance (? more for meetings..)UK
31
29Do You Really Need to Know Where “That” Is? Enhancing Support for Referencing in Collaborative Mixed Reality EnvironmentsJohnson JG,Gasques D,Sharkey T,Schmitz E,Weibel N2021x - desktop + tabletx - HoloLens11xx - collaborativexxxmap and list for visual guidancesees visual guidance like arrows generated by the expertx40 participants (20 dyads)AssistanceStudy on the effect of visual referencing guidance USA
32
30ARTEMIS: A Collaborative Mixed-Reality System for Immersive Surgical TelementoringGasques D,Johnson JG,Sharkey T,Feng Y,Wang R,Xu ZR,Zavala E,Zhang Y,Xie W,Zhang X,Davis K,Yip M,Weibel N2021x - HTC Vive Pro, IMU-equipped gloves and a wireless mouse-pen used for annotationsx - HoloLens v111xx - medicalx - novice surgeon (mentee) sees AR with camera-equipped room for room reconstruction, expert surgeon (mentor) sees VR with hand-tracking devicesx - IMU-equipped gloves and wireless mouse-pen for annotationsx - hands need to be performing the surgeryexpert sees mentee's first person point of view and room surroudingsexpert's avatar, 3D annotations, and procedure clipsx - 6 different live video streams (2 on rolling wheels that can be moved by novice; 2 at the corners of the operating room; 1 for novice's first person point of view; 1 depth camera from the surgical lamp that the novice is holding6 novices and 5 expertsTrainingSurgical mentoring between novice surgeons and expert surgeons - Novice Surgeon in Augmented Reality receives help from a
remote expert on site. The Remote Expert Surgeon in VR interactes with a 3D point-cloud of the patient, and engages with the
novice on a surgical procedure.
USA
33
31User Behaviour Analysis of Mixed Reality Remote Collaboration with a Hybrid View InterfaceGao L,Bai H,Billinghurst M,Lindeman RW2020x - HTC Vivex - Magic Leap AR Display 111xx - guidance in remote collaborationxxxexpert can choose between the local worker's first-person view and 360 God-like viewvirtual remote guidance cues overlaid, remote expert's head as a red avatar with a semi-transparent view frustum to show the expert's location and view orientationx - first-person view on HMD or 36028 participants (14 dyads)AssistanceVisual guidance in remote collaboration. "Experts chose to use the 360 live view with independent view control rather than the 2D FPV with high-resolution"New Zealand
34
32Mixed Reality Remote Collaboration Combining 360 Video and 3D ReconstructionTeo T,Lawrence L,Lee GA,Billinghurst M,Adcock M2019x - Samsung Odysseyx - HoloLens11xx - collaborationxxxexpert can choose between first-person view from 360 camera or 3D reconstructionnatural hand gestures and visual cues (viewing direction, animated 3D hand model for gestures, and a ray pointer for pointing); arrow to indicate the view frame when out of user's FOV, halo to indicate virtual hands out of FOVx - first-person view (from 360) and 3D scene seeing each other from a second person view 20 participantsAssistancecomparing 360 vs. 3D reconstruction. "participants performed significantly better on collaborative search tasks in 360 and felt higher social presence, yet 3D also showed potential to complement"Australia
35
33Evaluating the Combination of Visual Communication Cues for HMD-Based Mixed Reality Remote CollaborationKim S,Lee G,Huang W,Kim H,Woo W,Billinghurst M2019x - FOVE VR HMD + Leap Motion hand trackerx - Meta2 AR HMD11xx - collaborationxxxlive video of AR scenevisual cues (virtual hand, sketch, pointer)x16 participants (8 pairs)AssistanceComparing hand only vs. hand + sketch vs. hand + pointer vs. hand + pointer + sketchAustralia Korea
36
34Hand Gestures and Visual Annotation in Live 360 Panorama-Based Mixed Reality Remote CollaborationTeo T,Lee GA,Billinghurst M,Adcock M2018x - HTC Vive + Leap Motionx - HoloLens + Ricoh Theta V11xx - collaborationxxxAR view seen by the traineeVisual annotations from the expert through virtual ray pointingx8 participantsAssistanceAustralia
37
35JackIn Space: Designing a Seamless Transition between First and Third Person View for Effective Telepresence CollaborationsKomiyama R,Miyaki T,Rekimoto J2017x - CAVE like projection screen environmentx - external cameras1Nxx - collaborationxxxx - first person view and third person viewsees his worspace only, nothing from the expertx8 participantsAssistanceI'm unsure if this paper needs to be included or not. There isn't any interaction between the two remote collaborators. It's not specifically training/assistance setting.Japan
38
36JackIn Head: Immersive Visual Telepresence System with Omnidirectional Wearable CameraS. Kasahara; S. Nagai; J. Rekimoto2017xx11xx - collaborationxxxx - first person view of trainee from head-worn 360 camerax6 participantsAssistanceJapan
39
38Workspace Awareness in Collaborative AR Using HMDs: A User Study Comparing Audio and Visual NotificationsCidota M,Lukosch S,Datcu D,Lukosch H2016x - laptopx - META SpaceGlassesxx - spatial problem solvingxxxx - video from local user's headsetx - Text messages and virtual objects from the remote user; visual notificationsx - audio notifications when the expert takes actions, which are auto-generated by online text-to-speech synthesizerx12 participantsAssistanceNetherlands
40
39RemoteFusion: Real Time Depth Camera Fusion for Remote Collaboration on Physical TasksAdcock M,Anderson S,Thomas B2013x - PCx - depth cameras and projectors11xx - annotationsxxxx - view of physical task scene constructed through volumetric fusion of depth camerasx - annotationsxxAssistanceAustralia
41
40ObserVAR: Visualization System for Observing Virtual Reality Users using Augmented RealityS. Thanyadit; P. Punpongsanon; T. Pong2019x - Hololensx - VR HMD (Cardboard)11+xx - Classroom (colocated students in vr)xxxAvatar virtual representation of trainees, and where they are looking at / visual cues Follow Visual cuesx - (trainee)9 participants (experts). The participants performed the task for each visualization setting in groups of 20, 30 and 40 students (pre-recorded student data)AssistanceHong Kong
42
41Your Place and Mine: Designing a Shared VR Experience for Remotely Located UsersSra M,Mottelson A,Maes P2018x - hmd vivex - hmd vive22xx - Social gathering, Dancing taskxxxfull body avatarFollow dancing movesx28 participants, groups of 2. (dance) Instructors were not participantsTrainningUSA
43
42Collaborative Haptic Exploration of Dynamic Remote EnvironmentsS. Kim; J. Park2018AR - Desktopx11xx - guidance, no domain task xx - omega.7 haptic devicexhaptic guidance input visual cues for guidancehaptic cues for guidancex17AssistanceKorea
44
43Towards Seamless Interaction between Physical and Virtual Locations for Asymmetric CollaborationClergeaud D,Roo JS,Hachet M,Guitton P2017xxx1+1xxEngineeringxCustom physical artifactsxexperts discuss the possibilities in a meeting rooma technician immersed in VR tests the selected alternativesxFeedback from industry partnersboth France
45
45Applications of Networked Virtual Reality for Tele-Operation and Tele-Assistance Systems in the Mining IndustryBednarz T,James C,Caris C,Haustein K,Adcock M,Gunn C2011x - Dome display or LCD monitor displayx - projector/camera pair 111mining tele-operationxpenxfeed with stickynote sketchesaumented feed with instructionsxxAssistanceAustralia
46
46Symmetric Model of Remote Collaborative MR Using Tangible ReplicasS. Yamamoto; H. Tamaki; Y. Okajima; K. Okada; Y. Bannai2008xxx111no domainxNot a controller, but a physical objectNot a controller, but a physical objectinteracting with a physical object object augmented with remote collaborato's gestures interacting with a physical onjectxFive pairs of subjectsbothJapan
47
47Face-to-Face Tabletop Remote Collaboration in Mixed RealityS. Minatani; I. Kitahara; Y. Kameda; Y. Ohta2007x - VSTx - VST111no domain xxxfull body avatar interacting in the same physical workspacefull body avatar interacting in the same physical workspacexxbothUSA
48
48Does Size Matter? Exploring How Standard and Large-Scale Displays Affect Off-Site Experts during AR-Remote CollaborationMarques B,Silva S,Dias P,Sousa-Santos B2022x- large scale display or standard displayx-pad ar111xxxxscreen shots from the pad that trainee is holding, which is an AR figure about the machine.an AR figure about the target machine captured by the pad backside cameraxx - 16 participants as the remote user, 1 researcher as the local userAssistancecomparing large screen/standard screen on expert's side for observing the ar scene captured by the trainee.Portugal
49
49Avatars for Teleconsultation: Effects of Avatar Embodiment Techniques on User Perception in 3D Asymmetric TelepresenceK. Yu; G. Gorbachev; U. Eck; F. Pankratz; N. Navab; D. Roth2021x-Vivex-Microsoft Hololens111no domain xx-vive tracker on arm/legsx3d point cloud / 3d virtual avatar of the trainee3d virtual avatar of the expertxx - 24 participants into 12 pairsbothcomparing 3d point cloud/avatar as the presentation of the local user in the view of the remote userGermany
50
50A User Study on View-sharing Techniques for One-to-Many Mixed Reality CollaborationsG. Lee; H. Kang; J. Lee; J. Han2020x-Vivex-Microsoft Hololens1nxsearch/assemblyxxx2d video/360 video/3d model(3rd perspective)x-instructionsxx - 2 local users, 8 remote expertsbothcomparing three ways of sharing the view of multiple local users to the remote expert, who gives instructionsKorea
51
51Augmented Virtual Teleportation for High-Fidelity TelecollaborationT. Rhee; S. Thompson; D. Medeiros; R. dos Anjos; A. Chalmers2020HTC Vive + Zed Minix-oculus rift11xdesign/training/assistancexxx3d avatar360 degrees videoxx - 40 in total, 20 local, 20 remotebothcontributes a system (setup) for High-Fidelity TelecollaborationNew Zealand
52
52Immersive Telepresence and Remote Collaboration using Mobile and Wearable DevicesJ. Young; T. Langlotz; M. Cook; S. Mills; H. Regenbrecht2019x-phone/headset based ar, shared environmentx-phone-based ar with the hand gesture of the remote user11xsharing the whole environment with remote userxx-when remote user wears ar headsetxxsegmented hand gesturesegmented hand gesturexx - 19 as the remote usersbothan interface to share the current environment with a remote userNew Zealand
53
53A User Study on MR Remote Collaboration Using Live 360 VideoG. A. Lee; T. Teo; S. Kim; M. Billinghurst2018x-HMD with leap motionx-see through AR HMD11xsearch taskxxx360 panorama video shared by host, with visual cues of hands/regionsview awareness cues(regions)xx -12 participants, 6 pairsbothinvestigating the effect of indepedent view of remote user in 360 panorama video shared by the local userAustralia
54
54Evaluation of Hand Gesture Annotation in Remote Collaboration Using Augmented RealityS. Yamada; N. P. Chandrasiri2018x-desktop screenx-Microsoft Hololens11xpuzzle placementxxxthe first person view shared by the worker (local user)virtual hand/sketches as the instructionsxx - 10 as workers, 1 as helperassistancecompare virtual hand vs sketches as the instructions from the helper to the workerJapan
55
55Collaborative Production Line Planning with Augmented FabricationD. Aschenbrenner; M. Li; R. Dukalski; J. Verlinden; S. Lukosch2018x-Microsoft Hololensx-Microsoft Hololens12xx-installation of a new production linex - remote expert with desktop as control interfacexxthree modes were discussed, 1) first person view of the local user, 2) 3rd person view of the scene, 3) "god mode" with miniature visualization of the sceneAR content of the components that need to be added to the product linexxassistancetwo local users have different roles, customer and consultant, the remote user is expert for providing advices, AR is used for different purposes, for visualization purpose for the local users, and for sense making for the remote userNetherlands
56
56Geometrically Consistent Projection-Based Tabletop Sharing for Remote CollaborationD. Iwai; R. Matsukage; S. Aoyama; T. Kikukawa; K. Sato2018x-projectorx-projector11xx-document sharing and discussionxx-touch/pointing gestures on the table captured by the camerax-touch/pointing gestures on the table captured by the cameraa visualization of the other user's upper limb, e.g., an arm extended from the screen to the table pointing at a sentence on the document on the tablea visualization of the other user's upper limb, e.g., an arm extended from the screen to the table pointing at a sentence on the document on the tablexx -10 participants, 5 pairsbothenabling correct visualization of the remote user's upper limb posture, which was incorrect due to different setup on the two sides.Japan
57
57With a Little Help from a Holographic Friend: The OpenIMPRESS Mixed Reality Telepresence Toolkit for Remote Collaboration SystemsKolkmeier J,Harmsen E,Giesselink S,Reidsma D,Theune M,Heylen D2018x-headsetx-Microsoft Hololens11xxx-block selection, block alignmentxxxspatial mesh of the environment, point clouds of the local user.visualization of the head and hands of the remote users captured by the headset and controllers.xx - 60 participants, 3 tasks * 10 pairsboththree key features: 1 free viewpoint of the remote user, 2. gesture tracking of the remote user, 3. remote user is visualized with their head and hand positions.Netherlands
58
58Remote Collaboration With Mixed Reality Displays: How Shared Virtual Landmarks Facilitate Spatial ReferencingMüller J,Rädle R,Reiterer H2017x-tabletx-tablet11xspatial referencing, building consistent spatial understanding between two collaboratorsxxxvirtual objects added to the physical environment and marks of where the other collaborator is atvirtual objects added to the physical environment and marks of where the other collaborator is atxx- 32 participants, 16 pairsassistancetwo collaborators are both remote in this project, no physical environment is shared, only the virtual space is shared to two physical environmentsGermanyDenmark
59
59TeleAdvisor: a versatile augmented reality tool for remote assistancePavel Gurevich, Joel Lanir, Benjamin Cohen, Ran Stone2012x - desktopx - projector11xxxxxCan control which part of the workspace he wants to see (camera view controled by positioning a robotic arm), can also zoom in and outxSees own workspace with annotations projected on top of it (pointer, freehand sketches, text and
images)
xExpert controles camera view, can have 1st person of third person viewx - 3 experts and 10 traineesassistancevideo camera and a small projector mounted at the end of a tele-operated robotic arm: enables a remote expert to view and interact with the workers’ workspace, while controlling the point of view.Israel
60
60Design and Implementation of TeleAdvisor: a Projection-Based Augmented Reality System for Remote CollaborationPavel Gurevich, Joel Lanir, Benjamin Cohen2015x - desktopx - projector11xxxxxCan control which part of the workspace he wants to see (camera view controled by positioning a robotic arm), can also zoom in and outxSees own workspace with annotations projected on top of it (pointer, freehand sketches, text and
images)
xExpert controles camera view, can have 1st person of third person viewx - 4 experts and 20 traineesassistancevideo camera and a small projector mounted at the end of a tele-operated robotic arm: enables a remote expert to view and interact with the workers’ workspace, while controlling the point of view. (Extension of the above with a robotic arm with more DoF)Israel
61
61The Effect of View Independence in a Collaborative AR SystemMatthew Tait, Mark Billinghurst 2015x - desktopx-headset Vuzix11xobject placementxxxthe remote user sees the 3d model of the scene from an independent camera they can controlxinstructions from the remote userindependent view of the scene, with an annotation of the local user's focus areax - 18 pairs of remote and local usersassistancecontributes an independent view of the remote user as the most significant contributionNew Zealand
62
62Do You See What I See? The Effect of Gaze Tracking on Task Space Remote CollaborationKunal Gupta, Gun A. Lee, Mark Billinghurst2016x-desktopx-headset AR11xbuilid LEGO structurexxxthe view of the local worker with their gaze focus area highlighted in red colorxscreen view of the remote helper overlayed to the local space via AR headsetxxx-13pairs of remote and local usersassistanceinvestigated whether the pointer position of the remote helper and/or the gaze focus of the local worker is helpful in co-presense if shared.New Zealand
63
63Can Eye Help You?: Effects of Visualizing Eye Fixations on Remote Collaboration Scenarios for Physical TasksKeita Higuch, Ryo Yonetani,
Yoichi Sato

2016x - tracked by POV camera and eye trackerx - HMD (Brother AirScouter WD200S) or Projector11xgeneral (eval with blocks)xxxspeechxExpert sees the first-person POV camera worn on the worker's headGaze and hand gesture of the expert either projected onto the space or seen as optical see-through ARxxx - 4 pairsassistancecollaboration in which one side gives instructions to the worker on the other sideJapan
64
64The Effect of Collaboration Styles and View Independence on Video-Mediated Remote CollaborationSeungwon Kim, Mark Billinghurst, Gun Lee2018x - desktopx - HMD with camera11xobject manipulation (user study using Tangram puzzle)xxxExpert sees the first-person POV camera attached to HMDAnnotations from the expertxx - 12 pairstraining (instructions and collaborations)two types of collaborations: (1) one expert gives instructions to the other trainee, (2) two collaborators work together on the puzzleAustralia
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100