Cropping feature set in NN Classifier

#1

The context of my problem:
I want to perform a NN Classification.
The input of my NN Classifier is the output of the Spectogram block.
For simplicity les us assume that this input is an array of 10 (time intervals) by 6 (frequency intervals).

What I want to achieve :
I want to do the NN Classification not on the complete input (= 10 by 6 array) but only on the input for the frequency intervals 3 - 5 (so excluding the lowest 2 frequency intervals and also the highest one) (= 10 by 3 array)

Is there a way to crop the input set to the frequency intervals 3-5 ?
And if so how ?

Rationale for this request:
I know that there is mainly noise in those frequency intervals I want to remove while the distinguishing features are visible in the frequency intervals that I want to keep.

My Neural Network architecture :

import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, InputLayer, Dropout, Conv1D, Conv2D, Flatten, Reshape, MaxPooling1D, MaxPooling2D, BatchNormalization
from tensorflow.keras.optimizers import Adam

# model architecture
model = Sequential()
model.add(Reshape((int(input_length / 6), 6), input_shape=(input_length, )))
model.add(Conv1D(8, kernel_size=3, activation='relu', padding='same'))
model.add(MaxPooling1D(pool_size=2, strides=2, padding='same'))
model.add(Dropout(0.25))
model.add(Conv1D(16, kernel_size=3, activation='relu', padding='same'))
model.add(MaxPooling1D(pool_size=2, strides=2, padding='same'))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(classes, activation='softmax', name='y_pred'))

# this controls the learning rate
opt = Adam(lr=0.005, beta_1=0.9, beta_2=0.999)
# this controls the batch size, or you can manipulate the tf.data.Dataset objects yourself
BATCH_SIZE = 32
train_dataset, validation_dataset = set_batch_size(BATCH_SIZE, train_dataset, validation_dataset)
callbacks.append(BatchLoggerCallback(BATCH_SIZE, train_sample_count))

# train the neural network
model.compile(loss='categorical_crossentropy', optimizer=opt, metrics=['accuracy'])
model.fit(train_dataset, epochs=100, validation_data=validation_dataset, verbose=2, callbacks=callbacks)
#2

@janvda I think you can use Cropping2D for that (but it requires a 2D convolutional neural network). E.g. this seems to work:

ROWS = 13
COLS = int(input_length / ROWS)

# model architecture
model = Sequential()
model.add(Reshape((COLS, ROWS, 1), input_shape=(input_length, )))
model.add(Cropping2D(cropping=((0, 1), (0, 1)), data_format=None))
model.add(Conv1D(8, kernel_size=3, activation='relu', padding='same'))
model.add(MaxPooling2D(pool_size=2, strides=2, padding='same'))
model.add(Dropout(0.25))
model.add(Conv1D(16, kernel_size=3, activation='relu', padding='same'))
model.add(MaxPooling2D(pool_size=2, strides=2, padding='same'))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(classes, activation='softmax', name='y_pred'))

(crops one row off on both sides).

We’re working on adding filters to the spectrogram / MFE / MFCC blocks in the meantime as well to solve the problem properly.

1 Like