Observation-by-proxy; placing safety critical healthcare environments within reach for human-technology interaction design

Gaining access to hard-to-reach environments, like hospitals, for research purposes is often restricted. In this study, we install a single 360° camera to emulate the viewpoint of an in-situ observer, achieving a less invasive method to evaluate interaction design of a safety-critical environment.

Like Comment
Read the paper

It is well published on the impact of the COVID-19 pandemic period to academic travel and a need to rethink many practices that we previously took for granted. This includes a constellation of optimism about inclusion, extended reach across scientific communities and reduced carbon footprint presented by conferences held on virtual platforms. I believe the need for creative thinking of this kind extends to scientists carrying out observations of hard-to-reach environments, like safety critical environments in hospitals, and it is our response to this challenge, that I, with my fellow collaborators embark on in this paper Making the invisible visible: New perspectives on the intersection of human–environment interactions of clinical teams in intensive care

Infection prevention is paramount in hospital spaces, and team-working complicated. In this paper, we report on a new practice of effectively observing these types of interactions, by installing a small (5cm) single 360° camera, in a position that emulates that of an observer (perhaps a scientist, researcher, designer or engineer) as if they were in the room but without needing to be there. We report on touch patterns, behaviours and interactions of a clinical team to target opportunities for touchless technologies – like speech and hand gesture recognition technologies that are intended to reduce avoidable touch.

The complex needs of the Neonatal Intensive Care Environment (NICU) require simultaneous use of multiple tools and technologies. This need has driven development of miniaturised, wearable and intelligent medical devices. NICU is complicated by frequent flows of decisions and therefore to be useful in NICU, new technologies not only need to match the reliability of traditional methods, they  also should be able to add valuable information to the clinical team as part of a clinical workflow, forming an integrated solution. 

Clinical simulation, described as “a technique to replace or amplify real experiences with guided experiences that evoke or replicate substantial aspects of the real world in a fully interactive manner” offers an ideal approach for agile interaction design studies. 

  • The focus of this study was to observe without intervention, the interactions of a clinical team in a safety-critical environment, to develop a coding system for human-environment interactions and thereafter inform touchless technology appropriation and adoption. 
  • To collect data, a 360◦ camera was installed to observe a simulation of a team that typically work together training to treat a deteriorating infant. A high-fidelity clinical environment was created using a preterm neonatal manikin, simulated monitoring and real medical equipment and devices. 
  • The scenario was conducted in real time, and was filmed with a small 360◦ camera mounted on the wall, at a height of 2 metres, fronting on to the cot space. This positioning was selected to give maximum view of the room. It provided a vantage point to view a wider aspect and angle of the room, view simultaneous tasks in different areas that would otherwise be missed and to do so without interruption to the task at hand. 
  • It is important to raise that even outside of pandemic restrictions, there are few qualitative studies of design considerations in NICU. This is because these studies typically require extensive on-site observations and interviews. By contrast, this study is interested in less invasive and remote methods of evaluating interaction design with computer interfaces.

Aerial positioning of the 360 camera to view a wide aspect of a clinical space

Aerial positioning of the 360 camera to view a wide aspect of a clinical space on a 360 media player and VR headset. There are several 360◦ cameras available on the consumer market, predominantly developed for extreme sports. The technical specifications of the selected 360◦ camera included: a 360◦ horizontal plane, 240◦ vertical plane, 4K, 16-megapixel camera and dual microphones. This was selected this due to its 8-element fisheye lens, super wide-angle field of view, lightweight and wipeable lens.

By taking a behavioural approach to analysis, a modified version of Interaction Analysis of the 360◦ video was completed on a 360◦ media player, and by viewing on a VR head-mounted device.

  • The analysis comprises touch frequency by single touch, sequential touch incidents and patterns of touch across individual team members by manual review. An episode is defined as a single touch and “a sequence” was identified as a string of two consecutive episodes. A surprisingly large number of sequences of touch were identified (n = 68) which make that on average 3-4 a minute. 
  • Together, these findings are used to develop a classifier of tasks and to infer the optimal environmental conditions for recognition-based touchless technologies in NICU, from observed characteristics of the physical environment and sensory modalities of light and sound.
  • This is collated as a 12- point Space-Lighting-Obstruction-Ambient Noise (SLOAN) framework, intended as a quick reference tool to drive cooperative design thinking with hospitals purchasing new technologies and the developers of these new technologies in mind. This is encouraged by recommendations from West et al (2020), who reported in May 2020 on effective interventions to reduce the spread of COVID-19, including touch by applying behavioural science methods and models.

SLOAN framework

This preliminary study observes a high-fidelity simulation of the heterogenous, real time environment of NICU conducted during the COVID-19 pandemic. The COVID-19 pandemic periodically halted several research initiatives in healthcare, including my own PhD research path, which motivated this new way to conduct observations-by-proxy. Whilst we report on one discrete example, this has been achieved with success during a pandemic period, and underscores important barriers to collaborative design practices in safety critical environments.

I believe this method offers a chance to encourage interdisciplinary collaboration remotely, in this case by bridging a gap between multidisciplinary healthcare professionals and technology design teams. In particular, the use of the 360◦ camera is intended to encourage an enriched, immersive and empathetic researcher experience when reviewing a predetermined timeline. There are several inaccessible or yet to exist places that 360◦ films of live spaces or design concepts could be used to evaluate. Therefore, the broader intention is to spur deliberations on placing hard-to-reach environments for scientific research within reach. 


West, R., Michie, S., Rubin, G.J. et al. Applying principles of behaviour change to reduce SARS-CoV-2 transmission. Nat Hum Behav 4, 451–459 (2020). https://doi.org/10.1038/s41562-020-0887-9


This study is with thanks to the partnership between Great Ormond Street Hospital (GOSH) for Children's DRIVE innovation centre, UCL Interaction Centre (UCLIC) and GOSH Clinical Simulation centre and my respective interdisciplinary co-authors: Dr. Laura Potts, Prof. Neil Sebire, Prof. Yvonne Rogers, Emma Broughton, Dr. Linda Chigaru & Dr. Pratheeban Nambyiah

Touch sequnce pairs

Touch sequence pairs. This analysis offers new insights into the workings of a clinical team by using human-centred design strategies to highlight potential targets for new touchless technologies. One such strategy is to identify common exchanges of touch, for instance between the incubator and infusion pump. I present findings as a graphical visualisation in a sankey chart, to engage on this topic in new and different ways.

Sheena Visram

Researching Emerging Technologies as part of doctorate in Computer Science | Human Computer Interaction, Great Ormond Street Hospital for Children DRIVE centre | UCL Interaction Centre

Currently researching the adoption of emerging technologies at the internationally renowned UCL Interaction Centre (UCLIC) and Great Ormond Street Hospital for Children’s DRIVE innovation lab, as part of a doctorate in Computer Science. This includes design thinking for new human-technology interactions for Touchless Computing.