Mobilise-D Workshop at the International Conference on Ambulatory Monitoring of Physical Activity and Movement (ICAMPAM)

Workshop – Validating digital mobility tools: the Mobilise-D experience

Real-world monitoring of mobility and function (e.g. gait) is enabled by wearable devices including inertial measurement units (IMUs) that allow to quantify digital mobility outcomes (DMOs). While these devices and the associated DMOs are adopted more and more frequently, there is still limited awareness of how complex it is to ensure their validity and what could hinder comparability of data obtained during such assessments. In this workshop we will aim at raising this awareness by sharing the experience we gained as part of Mobilise-D, a project funded by the European Union (EU) as part of the Innovative Medicine Initiative, aiming at fostering regulatory approval and clinical adoption of DMOs.

To reach our aim we will share the complementary and multi-disciplinary experiences from a representative group of researchers involved in the project to discuss the various challenges that they encountered in association to the following activities:

Experimental protocols for the validation of the DMOs: the Mobilise-D Technical Validation Study. When thoroughly validating a system for the estimation of real-world DMOs, the optimal trade-off between clinical and technical requirements is necessary. In Mobilise-D, balancing inclusion of multiple pathological cohorts, refence systems and centres significantly increased the complexity of the protocol. This talk will present the protocol of the Technical Validation Study, it will describe instrumentation and type of assessments used, including how acceptability and participants’ opinions regarding the use of technology have been captured. It will also present solutions and challenges faced by the researchers in developing and running the study protocol.

Identification and characterisation of gold standards for real-world applications. This talk will outline the methods developed to characterize the gold standard solution and single sensor system used in the technical validation study. In particular, this talk will describe the reference system adopted as gold standard solutions to validate DMOs estimated from a single wearable device in real-world conditions (a wearable multi-sensing system including Inertial units, Distance sensors, and Pressure insoles: the INDIP system). The methodology and the adopted workflow to measure reference DMOs will be presented, highlighting strengths and limitations of the system. Challenges faced and solutions devised during the processing of the data collected will be presented.

A framework to compare and select top performing algorithms for quantifications of DMOs. Digital mobility outcomes (DMOs) can be obtained through algorithms processing a single sensor’s signals. This talk will present the pipeline (set of algorithms) that has been implemented for the calculation of real-world DMOs (e.g. cadence, step-length and walking speed estimation). But what do to when multiple algorithms are proposed for the evaluation of the same DMO? How can we compare those and select the “best” algorithm for that specific DMO? In this talk, a comprehensive methodology to compare and rank algorithms, depending on the DMO of interest, will be presented and techniques to select the top performers will be indicated.

The statistical analysis plan: how to validate DMOs? This talk will focus on the comprehensive statistical framework developed and implemented within Mobilise-D to evaluate DMOs criterion validity. Firstly, we will explain how the DMOs are obtained from wearable sensor assessments at lab and real-world contexts by running all available algorithms on an online platform. Considering the nature and level of aggregation of spatiotemporal DMOs and the characteristics of the reference systems, we will present the performance metrics of the analytical pipeline in multiple cohorts (e.g. healthy adults, Parkinson’s Disease, Multiple Sclerosis, Proximal Femoral Fracture (PFF) and Chronic Obstructive Pulmonary Disease (COPD)).

Interactive visualisation tools to enhance data exploration and interpretation. How to explore and make sense of large datasets of results obtained from statistical analyses? This talk will present the design of an automatic and interactive tool enabling the visual exploration and analysis of multivariate heterogeneous data. The talk will show the use of this interactive toolbox, focusing on selective DMOs, for data exploration, visualisation of different granularities/ aggregation levels of statistical analyses, and plot generation. It will be shown how this toolbox can facilitate access of data and results interpretation in large heterogeneous datasets.

The range of topics that will be covered is highly multi-disciplinary by definition. Each participant will be able to enhance or acquire new skills that would allow them to better navigate in the field of digital health. We will present some new data and results; we will also share our direct experience and tips for overcoming possible similar challenges in future studies. The techniques and analyses presented can be “translated” and applicable to other fields and topics (other than mobility), especially in circumstances where algorithm and DMOs validation is required. We will also share a number of papers and analytical tools that have already been published and shared with the goal of promoting standardisation and adoption.

Program ICAMPAM 2022