How HoloLens and Dynamics 365 Remote Assist helps the NHS provide patient care

An NHS doctor at St. Mary’s hospital performs a ward round on a COVID-19 ward whilst wearing a Microsoft HoloLens 2

March 2020. As the COVID-19 reached the UK, we had a theory that Dynamics 365 Remote Assist could work beyond industrial scenarios. It could transform healthcare.

Introducing Microsoft HoloLens 2 for remote assistance

I recently spoke with James Kinross, Senior lecturer in colorectal surgery and consultant surgeon at Imperial College Healthcare Trust NHS. “If you wind back to early March, which in COVID-19 times seems like a lifetime ago, we had a genuine amount of fear and anxiety about what was going to happen,” he said. “As it is to this day, the major issues we have is around staff protection and delivery of patient care.”

Colin Brown is CCIO at University Hospitals of Morecambe Bay NHS Foundation Trust. His Trust has a small federated hospital with limited staffing in a semi-rural area. By the end of March, it was still uncertain as to whether the volume of patients with COVID-19 would overwhelm the NHS, hospital services, and their staffing limits.

NHS hospitals across the country, serving both adult and children in acute and convalescent care have never been tested against such pressure. Rafael Guerrero, from Alder Hey Children’s Hospital, was worried that there would be delays in children’s heart surgery. He was worried his team would run out of capacity to monitor and deliver care to the whole region of North West of England, Cumbria, and North Wales.

Over the last three months, we worked with a consortium of NHS Trusts, Medical iSight, and Insight to use HoloLens 2 and Remote Assist to begin solving these problems. We recently published our initial use case on how it helps protect doctors while providing patient care.

How HoloLens 2 with Dynamics 365 Remote Assist works

We recently launched HoloLens 2 – a fully untethered, wearable holographic computer. It allows users to place 3D digital models in the room alongside them and interact with them using gestures, gaze, and voice. Mixed reality is at the forefront of transformative technologies and the impact and results are substantial.

Using Dynamics 365 Remote Assist, doctors wearing the HoloLens 2 on COVID-19 wards can be heads-up and hands-free. They can hold video calls with colleagues and experts from anywhere in the world. They can receive advice, interact with the caller and the patient at the same time. Medical notes and x-rays can also be placed alongside the call in the wearer’s field of view.

In this series of blogs, I wanted to take the key learning points and share them across the healthcare community. I hope it continues to foster more dialogue on how we can use the innovations within HoloLens 2 and mixed reality to support and transform outcomes.

An NHS doctor at St. Mary’s Hospital speaks with a patient on a COVID-19 ward during the pandemic whilst wearing a Microsoft HoloLens 2

Defining and implementing HoloLens and Dynamics 365 Remote Assist

The principle priority across the NHS Trusts has been to use the technology to improve staff safety. Advanced technology, especially remote evaluations of patient and communication between specialist care teams have made this possible, whilst maintaining social and physical barriers. Also, the consequential outcome was to limit the demand for PPE usage, enabling supply to be adequately distributed between hospitals. Individual case examples arose from local needs by specific team structures. In this blog, we will go into more details on considerations the teams took.

NHS doctors remotely discuss a patient during a Microsoft HoloLens2 virtual ward round during the COVID-19 pandemicMeasuring patient access and clinical effectiveness

Imperial NHS Trust has taken the lead on collecting qualitative data. They have collected information relating to their key outcomes of patients safety and PPE use. They have been collecting feedback from staff relating to how easy the technology is to use. They’ve also looked at the perceived impact on care quality, communication, and teamwork. In this blog, we will share more specifics along with patient feedback.

Two surgeons who have just completed a case during the COVID-19 pandemic. Surgeon on left is wearing full PPE, surgeon on right is wearing customised PPE and a Microsoft HoloLens 2What’s next: How can other NHS Trusts get involved?

Now that we have established the use case for Dynamics 365 Remote Assist, our NHS partners are looking to embed the benefits of the HoloLens 2 as an everyday tool. To do this within healthcare, the approach they are taking is twofold: First, they are looking to create a structured approach for a wider trial to effectively demonstrate impact in a transparent and robust way. Second, they are exploring new use cases with our mixed reality Partners, who have existing applications for medical training, surgical planning, interoperative surgical capabilities, patient education, and more.

In this blog, we will share some of the examples of what our NHS Trusts are looking to do next, and also share ways for other NHS Trusts to get involved

NHS doctors at St. Mary’s hospital from different specialities discuss a patient on a COVID-19 wardWhat technology was needed to get started

HoloLens 2 is now widely available to businesses through authorised resellers such as Insight. Dynamics 365 Remote Assist is an application that can be securely managed within an enterprise environment. It is also further optimised with the Microsoft 365 and NHS recent announcements. With Medical iSight’s extensive experience with Imperial College NHS through their previous work in developing surgical guidance apps for plastic and reconstructive surgery, they were able to provide invaluable support to effective rapid deployment. They have worked with Insight across this consortium to package the best practices and created a free Remote Assist Fast Start for NHS organisations who purchase HoloLens 2 through Insight. In this blog, we’ll share more details on how to get started.

Thank you to the NHS and our partners

I’d like to thank the following for their contributions and penmanship. Colin Brown, CCIO at University Hospitals of Morecombe Bay; Guy Martin, Consulting Surgeon at Imperial College Healthcare NHS Trust; James Kinross, Senior Lecturer and Consulting Surgeon, Imperial College Healthcare NHS Trusts; Rafael Guerrero, Consultant Congenital Cardiac Surgeon, Clinical Director of Cardiac Services & Director of Innovation, Alder Hey Children’s NHS Foundation Trust; Phuoc Duong, Paediatric Cardiology Consultant, Alder Hey NHS Foundation Trust; Andrew Lewington, Associate Medical Director of Education, Leeds Teaching Hospital Trust;  Philip Pratt, Chief Scientific Officer, Medical iSight; Wes Platel, UK Digital Innovation Lead, Insight.

It’s been the greatest privilege to work alongside this collective of amazing people within the NHS, Medical iSight, Insight, and our Microsoft teams who have truly worked above and beyond to help protect NHS staff while delivering patient outcomes.

Find out more

Defining and implementing Remote Assist use cases

Measuring patient and clinical effectiveness

What’s next: How can other NHS Trusts get involved?

What technology was needed to get started

About the author

Headshot of Leila MartineLeila has been working on the commercial mixed reality business for the last five years and is based in the UK. She’s been working with some of the most advanced customers and partners in the world, helping them identify, pilot and deploy technologies that have a profound ability to transform their business.

Prior to joining the mixed reality team, she held several senior positions at Microsoft including leading consumer and commercial business units such as Server & Tools and Windows in the UK and Central & Eastern Europe