Augmented Reality and Surgery: helping surgeons achieve ever-greater accuracy

Questions of health, well-being and personal independence are central preoccupations of all modern societies. Age-related illnesses, and our ageing populations, pose a number of social and economic challenges. Over the last two decades, information sciences and technologies have helped us rise to these challenges. Concepts such as distance health care and online treatment are now accepted as a part of everyday life. Among these new technologies, successive innovations in the field of medical imaging have been of particular significance. The Institut Mines-Télécom is a leading centre of research in this sector. Based at Télécom Bretagne, where she works with a multi-disciplinary team bringing together clinical scientists and researchers specialising in information processing, Chafiaâ Hamitouche has had a hand in numerous discoveries which have helped pave the way for the use of augmented reality technology in surgery.

Chafiaâ Hamitouche, a researcher at Télécom Bretagne, has long been interested in the potential of modelling technologies, starting from morphology modelling. “Form and function are interdependent. It’s thanks to the movement of the foetus inside the mother’s womb that joints are formed,” Dr. Hamitouche explains. Her work seeks to find  a new representation space forboth the morphology (form) and movement (function) of joints.

This research allows us to achieve a clearer understanding of certain joint and bone disorders, reflected in symptoms such as sprains and arthritis; treatment generally requires physiotherapy or surgery.

The rise of  Navigated Surgery

In 2001, Chafiaâ Hamitouche first began to take a particular interest in Computer Assisted  Orthopaedic Surgery. As she points out, “this is a sector where stakes are high: every year over 150,000 prostheses are implanted in France alone.” But decisions regarding the implant operation are still generally made on the basis of 2D X-rays, with no quantifiable control method. There is therefore always the risk that errors might arise, making the operation a failure.

The aim of  assisted surgery is to optimise the surgeon’s gesture, ensuring  its accuracy. This requires a change in the layout of operating room – introducing a 3D localizer and specially-adapted surgical instruments – and the use of guiding tools, some of which were developed at LaTIM[1]. These techniques allow surgeons to make their interventions less invasive, and sometimes shorter. They can also help avoid or limit X-ray exposure for patients and medical  staff.

And, as Dr. Hamitouche explains, “with 3D modelling and advanced imaging technology the surgeon can get a clear look at the bone structure of the patient before the operation, select the surgical instruments and plan the surgery accordingly.” During the operation, real-time fusion of multimodal (data from various sensors) information  is perfomed. This fusion of pre-operative and intra-operative data  allows the surgeon to see things which would otherwise be invisible or inaccessible due to the limited size of the incision or the presence of other tissues. Data  fusion can also help predict the consequences of a gesture (intervention) before it is made.

Mathematical modelling of biological forms, based on the morphogenesis process

Personalised treatment and surgery

Dr. Hamitouche continues to explore and expand the possibilities of guided surgery, and since 2010 has been working on radiofrequency ablation for bone tumours. “Augmented reality technology is the future of surgery,” she declares. Dr. Hamitouche and her team are currently working on an ambitious EU-backed project (ITEA2 programme), developing the interventional environment of next generation. The project is led by one of the world’s leading medical imaging specialists, Philips Healthcare, bringing together 27 partners from 4 different countries, including 6 French SMEs. Dr. Hamitouche is the coordinator of the French arm consortium, focusing on boosting the productivity and efficiency of surgical treatments and reducing patient risk and discomfort by helping doctors make the transition from invasive surgery towards minimally-invasive, image-guided techniques.

The operating room of tomorrow will be interactive and highly flexible, allowing professionals to tailor their procedures to the specific requirements of the patient. The techniques pioneered by Dr. Hamitouche and her team allow surgeons to plan their operations in great detail before they begin, and also to be more accurate and efficient in their use of radiation therapy to treat tumours.

[box type=”shadow” align=”” class=”” width=””]

OSTESYS and IMASCAP, two companies born in our laboratories

High tibial osteotomy is a surgical technique for treating arthritis in the knees. It is a cheap, non-invasive and relatively quick surgical procedure performed on around 7,000 patients every year. But to achieve satisfactory long term results, the operation needs to be extremely accurate. OSTESYS was founded in March 2009, a spin-off from a project developed by Chafiaâ Hamitouche’s research team and focusing on osteotomy of the knee. The company produces intelligent medical equipment integrating augmented reality technology, with innovative surgical instruments and implants which allow surgeons to achieve maximum accuracy using minimally-invasive techniques.

IMASCAP is an innovative start-up launched in 2009 as part of an ANR project conducted in partnership with LaTIM and Nice University Hospital, specialising in Image Guided Shoulder Surgery. Total Shoulder Arthroplasty pose a number of problems: the soft tissue environment where the muscles are of utmost importance, and it can be difficult to visualise the structure of the shoulder. The new technology developed, provides a 3D reconstruction showing the different structures in the shoulder, which can be reliably connected to key points identified by palpation of the target area. This process makes the operation possible even in highly complex cases. This procedure is performed on around 25,000 patients every year in France alone. Find out more [/box]

From fundamental research to clinical application

Augmented reality technologies of this nature are now widely used in clinical contexts. These new solutions have been developed with substantial input from surgeons, helping to make their interventions more precise, cut costs and above all to protect patients’ well-being. This is certainly true of the work undertaken by the team since 2006 on the development of a new generation of predictive autonomous implants  . These prostheses incorporate electronic components and adjustable mechanical elements allowing for post-operative functional modifications.

The team’s researchers are currently working on four key technological challenges: achieving autonomous power by generating energy within the prosthesis, analysing ligament imbalance, installing telemetry system and  actuation. Further opportunities to launch new start-ups and create new jobs? Watch this space…

[author title=”” image=” https://imtech.imt.fr/wp-content/uploads/2017/10/Portrait_Chafiaa_réduit.jpg”]Chafiaâ Hamitouche would have been able to study medicine, but ultimately decided that her real passion was mathematics. She obtained her engineering degree in Algeria in 1987, winning an international doctoral scholarship in the process. More interested in signal processing than electronics, Dr. Hamitouche embarked upon a research career and was soon drawn to the medical sector. In 1988 she developed an interest in three-dimensional medical imaging, and in 1991 submitted her doctoral thesis, prepared at the Image and Signal Processing Laboratory of the University of Rennes 1 and focusing on a very promising subject: the analysis of three-dimensional medical images, applied to the extraction of anatomical structures.

Having always lived close to the sea, she chooses to continue her work in Brest, attracted by both the quality of research and the quality of life in Brittany. In 2005 she was officially accredited as a Research Director in the field of geometric and kinematics modelling in 3D medical imaging. She is now Professor of Image and Signal Processing at Télécom Bretagne, and deputy Director of LaTIM.

Dr. Hamitouche is involved with a number of projects in the field of augmented reality and its surgical applications, and has coordinated various national and European projects as well as supervising more than 12 doctoral theses. She is listed as co-inventor of 5 national and international patents and software innovations.

Nonetheless, Dr. Hamitouche is always keen to underline the importance of teamwork. When she first arrived at Télécom Bretagne, Christian Roux, now the school’s Scientific Director, was already working on bone and joint structures. Éric Stindel, an orthopaedic surgeon and the current Director of LaTIM, played a crucial role in the development of numerous innovations in orthopaedic surgery, innovations to which Dr. Hamitouche contributed. “I’ve also been lucky enough to work with some great doctoral researchers,” she hastens to add, describing their passion for Computer Assisted surgery which, in some cases, has resulted in business start-ups.[/author]

[1] LaTIM, is the Laboratoire de Traitement de l’Information Médicale (Laboratory of Medical Information Processing),  a combined research unit affiliated to INSERM (UMR1101) and including researchers from Brest University Hospital, the Université de Bretagne Occidentale and Télécom Bretagne.