control systems theory and application -- neuroprosthetic systems -- real-time software design
I am a control scientist for application and theory with a focus on autonomous driving vehicles and biomedical engineering. I got my Ph.D. from Technische Universität Berlin, where I was working since 2010 in the Control Systems Group. For almost ten years, my focus was on the investigation of novel control schemes for neuroprosthetic devices to restore lost motor functions in case of stroke and Spinal Cord Injury (SCI).
After my Ph.D., I continued my work as a Postdoctoral Research Fellow in the Neuroingineering section at NearLab, Politecnico di Milano on the restoration of hand functions under paralysis. Today, I am working in the field of control for autonomous vehicles with TomTom Location Technology Germany GmbH.
Research on feedback controlled Neuro-Prosthetic Systems
Neuro-prosthetic systems using Functional Electrical Stimulation (FES) to restore or support motor functions in neurologically impaired patients are considered. FES stimulated muscles usually behave highly non-linear and time variant. Hence, an often encountered difficulty is the precise control of limb motion by adjusting the stimulation intensity.
The considered topics include:
Motion feedback control strategies applied to the upper limb extremities using Functional Electrical Stimulation (FES)
Mehods for improving feedback control of Neuro-prosthetic systems. An underlying feefback of the muscular recruitment (Lambda, λ-control) measured by the FES evoked EMG was introduced.
Methods for evaluating the volitional EMG
Investigation of co-contracted antagonistic muscle pairs with the aim to improve motor functions.
Variable sampling rate systems; The aim is the application to FES for reducing muscle fatigue.
A novel control method (Lambda, λ-control) was developed which linearises the static input non-linearity (recruitment curve) of FES-activated muscles by fast feedback of the detected muscle activity caused by FES. This activity is estimated by the FES evoked electromyogram (eEMG).
It could be shown that the developed approach robustly compensates the unwanted effects related to uncertainties and time-variances of the highly nonlinear muscular recruitment behavior. The Lambda-control method allows the precise adjustment of the muscular recruitment. Therefore, the feedback-controlled muscles are much easier to model and to control in Neuro-prosthetic systems.
Applications / Neuroprosthesis
FES Hand Neuroprosthesis
This is for people who have a paralysis of hand functions as a typical result of a stroke. Electrical pulses applied to the motor nerves induce artificial muscle contractions (functional electrical stimulation, FES). Three muscles are used to control arbitrary angles of wrist extension and to realize grasping. A feedback of the electrical muscle responses (electromyography, EMG) is used to ensure the desired reaction of the muscles to FES. Only one EMG-channel is required to observe all three muscles.
In an earlier stage (video B), motion control is open-loop but is superseded by automatic, feedback-control approaches in the final stage of development (video A). With active feedback control, the operator of the experiment manually issued movement commands (e.g. moving up, grasping).
The control system was tested in ten healthy subjects (not all shown in the video). They were asked to not volitionally participate in the hand movements. The placement of the stimulation electrodes required a trial-and-error procedure at the beginning of the trials, however, with an increasing number of tested participants, a better understanding of the placement, in general, could be obtained.
FES Arm Neuroprosthesis
This is a system intended for persons who have only weak residual activity in the shoulder deltoid muscle. By Functional Electrical Stimulation (FES) an artificial muscle contraction is added such that the full extent of arm movements is restored. Behind the scenes, a positive-feedback control system measures the elevation-angle to amplify volitional movements. Electromyography (EMG) is used to compensate for the effects of rapidly progressing muscle fatigue under FES.
In effect, the participant (stroke, paresis in shoulder deltoid muscle) shown in this video could elevate his arm much higher than without the arm neuroprosthesis. The entire movement is still under his control, as the system detects his intention of moving.
FES Arm Neuroprosthesis controlled by a Brain-computer Interface
A two-class brain-computer interface (Berlin-BCI) allows the user to select a target position. This position is then continuously realized by a feedback-controlled FES-system attached to the shoulder deltoid to produce horizontal arm movements.
FES Arm Neuroprosthesis and Exoskelleton
Within the European project MUNDUS, an assistive framework was developed for the support of arm and hand functions during daily life activities in severely impaired people. This contribution aims at designing a feedback control system for Neuro-Muscular Electrical Stimulation (NMES) to enable reaching functions in people with no residual voluntary control of the arm and shoulder due to high level spinal cord injury. NMES is applied to the deltoids and the biceps muscles and integrated with a three degrees of freedom (DoFs) passive exoskeleton, which partially compensates gravitational forces and allows to lock each DOF. The user is able to choose the target hand position and to trigger actions using an eyetracker system. The target position is selected by using the eyetracker and determined by a marker-based tracking system using Microsoft Kinect. A central controller, i.e., a finite state machine, issues a sequence of basic movement commands to the real-time arm controller. The NMES control algorithm sequentially controls each joint angle while locking the other DoFs. Daily activities, such as drinking, brushing hair, pushing an alarm button, etc., can be supported by the system. The robust and easily tunable control approach was evaluated with five healthy subjects during a drinking task. Subjects were asked to remain passive and to allow NMES to induce the movements. In all of them, the controller was able to perform the task, and a mean hand positioning error of less than five centimeters was achieved. The average total time duration for moving the hand from a rest position to a drinking cup, for moving the cup to the mouth and back, and for finally returning the arm to the rest position was 71 s.
Artificially induced hand functions (DAAD (German Academic Exchange Service), Politecnico di Milano 2018)
Support of the arm elevation in case of a paresis (only weak residual muscle activity in the shoulder deltoid is present) (BMBF (German Federal ministry of Education and Research), www.bemobil.net, TU-Berlin, 2014-2017)
Because of the pulse-based nature present in Functional Electrical Stimulation, the framework of discrete-time control perfectly suitable for control systems' design. Therefore, the following topics were addressed:
Discretisation and control of linear, time-invariant systmes under irregular output-sampling and actuation frequencies (variable sample rate systems).
Exact linearisation of nonlinear discrete-time systems.
Inertial Measurement Units (IMU)
Feedback control of limb movements requires a measurement of motion information e.g. inter-segment joint angles or velocities. Because of their small size, Inertial Motion Units are a perfect choice. Singal processing employing kinematic models allows obtaining the needed information in real-time.
Kalman Filter for orientation-estimation
Combination of multiple IMUs to determine inter-segment joint angles in the human body
Determination of joint-angle acceleration using kinematic models
Open-Source Software Projects
OpenRTdynamics.github.io (ORTD): A framework for the implementation of advanced real-time control systems aiming to be an open-source alternative to Simulink Coder. It provides features including state machines and, as a real-time capable interpreter is used, sub-simulations that can be exchanged during runtime. Besides, this framework properly handles multiple threads, their communication, allows to synchronize control systems to external events (e.g. even irregular timers or incoming network packages) and provides many other nice features. Because of a high-level schematic-description language -- based on Scilab commands provided by a toolbox -- lower implementation-effort results compared to C/C++.
This framework has grown since 2008 along with the needs for the implementation of the feedback-control schemes in the area of neuroprosthetics.