How to add Band pass filter and use external feature extraction

Help
#21

i have made a new version try with this one project ID:137064

#22

I did modify dsp-server.py to point to port 44461.

If you are getting Failed to Add Custom Block, then make sure

$ pip3 install -r requirements-blocks.txt
$ python3 dsp-server.py
$ ngrok http 44461

1 Like
#23

Ya at the beginning I got an error but when I changed the ngrok HTTP to 44461 it uploaded successfully
but when I generate the feature extraction it takes so long

#24

I was able to clone 137064. What is your configuration? I’ll test on my side for speed.

Scale axes                                 = ?
My Smooth It!!!                     = ?
Apply Low Pass Filter         = ?
Normalize Frequency               = ?
Order of the Filter               = ?
Critical Frequency                  = ?
Sample Rate [Hz]                     = ?
Apply Notch Filter                  = ?
Order of the Notch Filter  = ?
Lower Cutoff Frequency         = ?
Higher Cutoff Frequency      = ?

#25

The only import paramters i need to use is low pass filter 20Hz ,high pass filter 500HZ and notch filter at 50HZ
The order =4
The sampling rate is 4000HZ

#26

For me, it took about 50 minutes to extract Features. So at this point you’ll probably need to upgrade to an Enterprise Account that will speed up the process by:

  • allow running dsp.py on EI servers
  • gain access to a GPU which will probably speed things up by a factor of 10 to 100 times faster

For a slight increase in speed, take out the print statements in dsp.py.

#27

How can i accsess the Enterprise Account,
and when u did apply the feature extraction does it decremented the data nicely or there was an ability to differentiate

#28

Enterprise Account Info

My Training results were about 55% and Test results about 20%. Not so good.

Things to try:

  • Tweaking the filters is in order.
  • Analyzing the Feature Importance of the Spectral Power should help. See this video.
  • Add a Anomaly Detection Learning Block
#29

I can’t vesualize the features’ importance so I dont know what the features are included in the coustom block.
Is it also possible to implement other types of algorithm such as SVM,CNN , LDA ,etc… or is it only restricted with the ANN

#30

I am not sure about SVM,CNN , LDA ,etc. @Aurel can you answer this question about algorithms.

@Joker147 Today I created a new Impulse that has 2 each standard EI issued Spectral Analysis Processing Blocks (SAPB). I’ll make you a collaborator on this project so you can see the various settings. You will be able to see the Feature Importance for each SAPB. One thing I did was to re-distribute the data split between training and testing from the Dashboard page.

  • One SAPB acts as a Low Pass at 75Hz
  • One SAPB acts as a High Pass at 5Hz

The NN Classifier had a 90% accuracy.
The Model Testing had a 84% accuracy.

#31

That is impressive, but for the EMG signal usually the low pass filter is 20HZ and the high pass is 500HZ how it possible to use the low pass at 75HZ and the high pass at 5 HZ and you are able to obtain a high accuracy

#32

The feature importance is restricted to blocks with dimension reduction disabled. @Joker147, do you want to implement SVM or CNN within the DSP block? I’m not sure to understand the request.

On the low pass and high pass, if you set a low-pass to 20 Hz and high-pass to 500 Hz it will discard frequencies between 20 and 500 Hz which is probably not what you want.The project from @MMarcial looks quite good!

Aurelien

#33

@aurel This is the requirement of the signal analysis of this dataset so it is mentioned that the bandpass filter has been used to filter out the signal

image
image776×252 57.4 KB

#34

@Joker147 I arrived at my settings by interactively changing cutoff frequencies by evaluating the Spectral Power as Jan showed in his video mentioned above. I had no idea what the data meant at that time and was free to follow the data wherever it lead me.

I tried new settings: lowpass filter at 500 Hz and a highpass filter at 20 Hz and got even better results.

  • The NN Classifier shows 97% accuracy.
  • The Model Testing shows 95% accuracy.
  • The best EON Tuner shows 38% accuracy.

@aurel Why is EON Tuner so low?

  • Is this the Accuracy I would expect when running on the chosen Portenta MCU?
  • So in this case, I would not select any of the EON Tuner models, correct?
  • What Accuracy would I expect on the Portenta?

Project ID: 139295

@Joker147 I believe that the Model does not need a notch filter in this case if each and every sample have the 50 Hz line interference. The Model will just ignore this signal since it adds no new or different information. The beauty of AI, no?

#35

The thing you did is duplicated the spectral features and you used one for a low pass and the other one for a high pass. or there was an extra modification applied to the custom blocks?
Regarding the EON what is its purpose of it and what it leads to at the end
@aurel In that case I’m expecting to test the model in the real-time application and get the right classification?
and how to determine the amount of the sampling of the signal for the real-time prediction because im looking for a fast prediction to classify
How to prove there is no overfitting since we can’t visualize the ROC curve to see the behaviour of the training data?
@MMarcial


The signal supposed to be rectified to remove the negative values

clear

image
image717×322 30.2 KB

#36

@MMarcial, the current EON tuner version works with only 1 DSP block. But stay tuned there will be some improvements soon :slight_smile:

@Joker147, EON tuner is our AutoML feature that selects the best signal processing and learning blocks.
You can check the latency prediction for all blocks in your project. Just make sure to select the appropriate board from your project’s dashboard (project info section on the right side).

Aurelien

1 Like
#37
  • RE: Custom Blocks
    • I did not use an Custom Blocks in my latest Impulse. I was trying to see how good it could get with the standard Blocks.
  • RE: Sampling Frequency
    • The Sampling Frequency will depend on your MCU you choose to use in the final design.
  • RE: Fast Prediction
    • The Deployment page show Latency times of 2ms for an Aurduino Portenta H7.
  • RE: ROC
    • You can create the ROC table thru some manual work (I suppose the EI API could be used but I have not yet played with the API). On the Test Data page you can:
      1. Set the Confidence Threshold
      2. Click Classify All button
      3. Cut-n-Paste the table into a spreadsheet
      4. Repeat Step 1
#38

Regarding the rectification of the signal.Does it need to be rectified or i can just used it like that

#39

At this point it becomes all trial and error by adding or not adding features and comparing the outcome to all other runs. These guys used only the mean absolute value (MAV) (a form of rectification) and median frequency (MDF). They are trying to determine if less EMG parameters can still get accurate outcomes.

The Impulse I build uses different spectral power bands or power spectrum density, RMS, kurtosis and skew with good results.

#40

I have tried to use the live classification i got 95% anomly detection the thing the sensor im using has different specification than the one i use for the training so how can use it