Skip to content
Skip to main content
Microsoft Industry Blogs - United Kingdom

Kelly Limonte-Microsoft Blogger Series-children in long term hospital care

Today’s story is a great example of how AI and robotics can be used to improve patient experience in a healthcare setting.

Microsoft has been working in partnership with Great Ormond Street Hospital (GOSH), University College London, and NTT DATA on an Industry Exchange Network programme (IXN). The programme gives computer science students the chance to work with academic and clinical supervisors on real-life AI projects as part of their degree.

One particular brief provided by a GOSH clinician brought home the stark reality of the impact long-term care can have on children. “When parents have a child in long-term care in hospital, the hard reality is that life must go on. Siblings at home need caring for too, and time is split between hospital and home. Children at hospital can miss out on valuable social interactions, especially at bedtime when they miss out on special moments, like a bedtime story,” said Dr Shankar Sridharan, Consultant Fetal and Paediatric Cardiologist & Chief Clinical Information Officer.

Using AI to provide social interaction

The StoryWall was a concept taken from an award-winning project from the IXN in 2018 with NTT DATA. The concept was further developed by Daiana Bassi as part of her MSc project with UCL. Daiana is now the Technical Lead in DRIVE, GOSH’s Digital Research, Informatics and Virtual Environments unit dedicated to the research and evaluation of new data driven technology.     SOTA robots

Daiana explains:

“StoryWall uses Microsoft’s Computer Vision API to identify a toy, or a medical object a child is holding when interacting with SOTA, which then delivers age-appropriate content – a comforting story, or reasons for a procedure – in the form of interactive storytelling.”

– Daiana Bassi, DRIVE Tech Lead

When the child holds a toy or item in front of the SOTA robot, it will recognise the object and read a related bedtime story to the child. For example, if child holds up an apple, the robot would read Snow White and the Seven Dwarfs.

SOTA robot interacting with a young boy

Let me explain the tech a bit more…

Azure Cognitive Services infuses SOTA the robot with intelligent algorithms which allow it to hear, see, speak, understand, and interpret the user’s needs through natural communication methods. SOTA extracts rich information from images to categorise and process visual data, allowing it to make the connection to a relevant story it’s been taught. It then ‘reads’ this story. Screenshot showing Computer Vision in action, translating an image of a man standing on a train platform and labelling the content with relevant tags, such as

“NTT DATA is delighted to have provided mentoring and the SOTA robot for this initiative. We’re investing in social robotics and cognitive solutions globally and are looking forward to progressing the work with SOTA and Microsoft in the coming months,” said Tom Winstanley, Vice president, Digital & Innovation

This is a really lovely example showing how Microsoft Azure Cognitive services can be paired with a partner’s device, such as SOTA, to bring the power of AI to life.

Learn more

Get hands on with Cognitive Services 

Maximising the AI opportunity 

Kelly Limonte headshotAbout the author

Kelly is the Healthcare Industry Manager at Microsoft UK, working with transformational digital partners and NHS customers to pilot solutions for collaborative working and empowering everyone to do more. She has 15 years’ experience working alongside the NHS, and is passionate about the power technology has to create positive change in healthcare.