In 2017 Microsoft showed a prototype that would apply a surface vibration to a users wrist in order to “short-circuit” the feedback loop that was causing tremors. While no further research was released, various media stories have mentioned success of this approach with about half a dozen people. So here is my use case: I’ve basically been working on making an open source tremor dampening watch off and on for a couple of years with my students. I recently became aware of Edge Impulse and think it can help me with the last steps to get a functional prototype to start testing if this “surface haptic tremor watch” concept is worth prototyping into a more custom circuit. The process:
I will gather IMU data (ax, ay, az, gx, gy, gz) and an analog input that is the frequency of a motor that’s vibrating. That motor will be applying surface haptic feedback to an individual’s wrist that is experiencing some form of tremor. I will gather IMU tremor data as a function of motor vibration frequency. This part of the circuit is working and I can start getting data from volunteers. Each volunteer will have their own bespoke data solution as tremors are so variable and different for each person.
I will feed the IMU and frequency data into an Edge Impulse regression model to get a fit of vibration motor frequency to the IMU data. I kind of have this working.
I want a ML model that will then accept the IMU data in realtime, output a vibration frequency and gradually change that frequency to induce a change in tremor amplitude that will minimize its impact.
I’m curious if this sounds plausible with Edge Impulse and if anyone with more experience with the platform would have suggestions on which parts of the documentation I can look into towards making this a reality. Thanks in advance!
I assume you refer to the Emma Watch, Parkinson’s disease? Emma Watch
To answer you’re question. Yes, you can use Edge Impulse. I am working on a healthcare regression problem (https://mlate2dge.github.io) using Tensorflow/Keras with two essential MLOps tools: Edge Impulse and Weights and Biases (for experiment tracking). I mainly focus on the ML part.
You can use Edge Impulse Studio. As you mentioned, you need a split by person/patient ID, so you need to follow Metadata approach. However, I suggest using Bring your own model, also check Edge Impulse Python SDK , this approach is much easier.
You can try the 1DConv or feature engineering approach, for example, Spectral features. The DSP block code is also available on Github. In the project, I have used 1DConv. However, currently, I am doing feature engineering and trying to bring in uncertainty (aleatoric and epistemic) and model explainability (SHAP).
(By the way, this is a nice use case for reinforcement learning.)
Let me know if you have more questions (we can always connect).
Joeri
Indeed that is what i’m working on. I haven’t seen an open source implementation and it seems perfect for that use case. The project is being tracked here: Tremor-Duino | Hackaday.io
Its good to hear that i’m on the right path. I’ll definitely reach out as I expect i’ll hit multiple walls with this. At this point i’m just getting proof of concept with 1 or 2 people before tackling wider scale implementation.
In this case the idea is that motor vibration frequency can impact the magnitude of the tremor. For an individual person, the idea would be to gather data on how the IMU tremor changes while changing the the motor frequency. So I would set a motor frequency, gather a bunch of IMU data and then change the motor frequency and again gather a bunch of IMU data. So I’ll populate a wide space of motor frequency and IMU data to train on and develop a model that can extrapolate what frequency will minimize the tremor and deploy model to run in real-time. So your suggestion is I need to repeat that process with multiple people to get the quantities of data needed to get a good model trained?
So far, I understand the Emma Watch has some vibrating motion, a vibration at some frequency. It is unclear whether it is a one-tone or multitone. However, the pattern causes some ‘disturbance’ in the pathway between the brain and hand. The patient will ‘forget the tremor’; however, it is still there.
This will be the main challenge: finding that pattern. In my opinion, this is patient-dependent. Maybe there is some sweet spot, a pattern that works for most patients. But this will be part of the research. You need to start with a small pilot, collecting data from different patients (you need to pass a Medical Ethics Committee before you can start collecting data). How many patients? Difficult to tell. Consider also that age probably affects and the patient population (Parkinson’s, essential tremor patients, MS patients, nerve damage, …)
So, you need to start with collected IMU data without external vibration. Next, collecting data with different stimuli and investigating the effect on the tremor.
Indeed there are a lot of open questions about the Emma Watch. That is really the goal of the project to test the hypothesis that haptic vibrations and dampen the impact of tremors. I have medical advisors to work with and an IRB but want to get the workflow figured out before doing a research study. So at this point I just want to get the technology working. I think that means having a way to get the IMU data (done), vibrating a motor with different tones/patterns (done, though unclear what kind of pattern is needed), reading that data into edge (done), creating an appropriate Impulse (that’s what I’m working on now) and deploying back to the Arduino to control haptic as a function of IMU data in realtime (need to get a model deployed). I’ll be digging into the different references you suggested to get a model worked out that can do this simple task with some synthetic tremor data. Once I’m confident in the workflow, I can do as you say and design a good protocol for systematically testing vibration tones/patterns and their impact on a small cohort.