XIAO ESP32S3 SENSE MIC Support? YES

#1

For users of the new XIAO ESP32S3 SENSE, looks like Edge Impulse Studio will soon support ESP32S3 MCU as target for projects…

While waiting, we can at least run EI projects on it by generating code (export Arduino Library .ZIP) for the already supported ESP-EYE, and making some little changes, so that it can run on the ESP32-S3 (like the XIAO ESP32S3 SENSE)…

Below I will describe how I was able to create a small “demo” like voice recognition project, export the code targeting the ESP-EYE, and make little changes and have the code running on the XIAO ESP32S3 SENSE…

Basically, 2 things are needed:

1. Change the I2S initialization code…
2. AI Functions on ESP32S3 are not yet supported, so I DISABLED IT on the code…

DETAILS

1. INITIALIZATION
Below are the full listing of the 2 functions that need changes:

static int i2s_init(uint32_t sampling_rate) {
  i2s_config_t i2s_config = {
    .mode                 = (i2s_mode_t)(I2S_MODE_MASTER | I2S_MODE_RX | I2S_MODE_PDM ),
    .sample_rate          = sampling_rate,
    .bits_per_sample      = I2S_BITS_PER_SAMPLE_16BIT,

    //.channel_format       = I2S_CHANNEL_FMT_ONLY_RIGHT,     // Also works
    .channel_format       = I2S_CHANNEL_FMT_ONLY_LEFT,
    //.communication_format = I2S_COMM_FORMAT_PCM,            // Also works
    .communication_format = I2S_COMM_FORMAT_I2S,
    
    .intr_alloc_flags     = ESP_INTR_FLAG_LEVEL1,
    .dma_buf_count        = 8,
    .dma_buf_len          = 512,
    .use_apll             = false,
    .tx_desc_auto_clear   = false,
    .fixed_mclk           = 0

  };

  i2s_pin_config_t pin_config = {
      .bck_io_num = -1,    // IIS_SCLK
      .ws_io_num = 42,     // IIS_LCLK
      .data_out_num = -1,  // IIS_DSIN
      .data_in_num = 41,   // IIS_DOUT
  
  };
  esp_err_t ret = 0;

  ret = i2s_driver_install((i2s_port_t)0, &i2s_config, 0, NULL);
  if (ret != ESP_OK) {
    ei_printf("Error in i2s_driver_install");
  }

  ret = i2s_set_pin((i2s_port_t)0, &pin_config);
  if (ret != ESP_OK) {
    ei_printf("Error in i2s_set_pin");
  }

  ret = i2s_zero_dma_buffer((i2s_port_t)0);
  if (ret != ESP_OK) {
    ei_printf("Error in initializing dma buffer with 0");
  }

  return int(ret);
}

static int i2s_deinit(void) {
    i2s_driver_uninstall((i2s_port_t)0); //stop & destroy i2s driver
    return 0;
}

Notice that, the port number changed to zero [(i2s_port_t)0]…

2. DISABLE AI NEURAL NET FUNCTIONS
After adding the exported Arduino Library .ZIP file to the ArduinoIDE, the files are expanded inside the Arduino “libraries” subdirectory…
This directory (folder) is locate inside the place where Arduino is installed…
Or, it is inside the “portable/sketchbook/libraries/” directory (folder) if you are running Arduino in “portable mode”.

Locate the file “ei_classifier_config.h” and, inside it, the following line:

#define EI_CLASSIFIER_TFLITE_ENABLE_ESP_NN 1

Change it to:

#define EI_CLASSIFIER_TFLITE_ENABLE_ESP_NN 0

You will find the file into something like the following:

libraries/YOURPROJECT_inferencing/src/edge-impulse-sdk/classifier/ei_classifier_config.h

After that, you can compile and upload the sketch to XIAO ESP32S3 SENSE board…

Ok, since it is not using the AI-NN functions, this is not a very good way…
My point here is that, it is possible to have the code running…

This is it,
regards all,
Valter

Arduino IDE 1.8.19
esp32 version 2.0.8
Machine: Debian Bullseye 64Bit GNU/Linux

[EI STUDIO, USING NEURAL NETWORK FUNCTIONS ON ESP32-S3?]
At the very begining, I was looking at the way to have the generated code running on the not-supported-yet ESP32-S3 with minimalist changes… that is the reason I did the test and now I am sharing here in the case it may also interest people… BUT… INSTEAD, if you want to use the ESP Neural Network Functions on the ESP32-S3, there is a kind of “preview” version that you can find on the following tutorial on Hackster IO, including instructions about how to use it (“activate it”)…

Go to hackster “dot” io, and search for the following project:
tinyml-made-easy-keyword-spotting-kws-5fa6e7

Considering what that tutorial states, the official support in EI Studio will be added soon…

3 Likes
#2

Hi.
Just in the case you want to check the data coming from the MIC to the MCU, I adapted the XIAO Wave Record Sample (from XIAO wiki) to use the same config listed above…
It record AUDIO and save WAVE file to the Micro SD Card…

/* 
 * DI2S Version ("driver/i2s.h"), 
 * WAV Recorder for Seeed XIAO ESP32S3 Sense
 * 
 * Original: https://wiki.seeedstudio.com/xiao_esp32s3_sense_mic/
 */

#include "driver/i2s.h"

#include "FS.h"
#include "SD.h"
#include "SPI.h"

// make changes as needed
#define RECORD_TIME   20  // seconds, The maximum value is 240
#define WAV_FILE_NAME "arduino_rec_DI2S"

// do not change for best
#define SAMPLE_RATE 16000U
#define SAMPLE_BITS 16
#define WAV_HEADER_SIZE 44
#define VOLUME_GAIN 2

void setup() {
  Serial.begin(115200);
  while (!Serial) ;

  i2s_init();
  
  if(!SD.begin(21)){
    Serial.println("Failed to mount SD Card!");
    while (1) ;
  }
  record_wav();

  i2s_deinit();
}

void loop() {
  delay(1000);
  Serial.printf(".");
}

void record_wav() {
  delay(2000); 
  
  uint32_t sample_size = 0;
  uint32_t record_size = (SAMPLE_RATE * SAMPLE_BITS / 8) * RECORD_TIME;
  uint8_t *rec_buffer = NULL;
  Serial.printf("Ready to start recording ...\n");

  File file = SD.open("/"WAV_FILE_NAME".wav", FILE_WRITE);
  // Write the header to the WAV file
  uint8_t wav_header[WAV_HEADER_SIZE];
  generate_wav_header(wav_header, record_size, SAMPLE_RATE);
  file.write(wav_header, WAV_HEADER_SIZE);

  // PSRAM malloc for recording
  rec_buffer = (uint8_t *)ps_malloc(record_size);
  if (rec_buffer == NULL) {
    Serial.printf("malloc failed!\n");
    while(1) ;
  }
  Serial.printf("Buffer: %d bytes\n", ESP.getPsramSize() - ESP.getFreePsram());

  // Start recording
  i2s_read((i2s_port_t)0, (void*)rec_buffer, record_size, &sample_size, portMAX_DELAY);
  
  if (sample_size == 0) {
    Serial.printf("Record Failed!\n");
  } else {
    Serial.printf("Record %d bytes\n", sample_size);
  }

  // Increase volume
  for (uint32_t i = 0; i < sample_size; i += SAMPLE_BITS/8) {
    (*(uint16_t *)(rec_buffer+i)) <<= VOLUME_GAIN;
  }

  // Write data to the WAV file
  Serial.printf("Writing to the file ...\n");
  if (file.write(rec_buffer, record_size) != record_size)
    Serial.printf("Write file Failed!\n");

  free(rec_buffer);
  file.close();
  Serial.printf("The recording is over.\n");
}

void generate_wav_header(uint8_t *wav_header, uint32_t wav_size, uint32_t sample_rate) {
  // See this for reference: http://soundfile.sapp.org/doc/WaveFormat/
  uint32_t file_size = wav_size + WAV_HEADER_SIZE - 8;
  uint32_t byte_rate = SAMPLE_RATE * SAMPLE_BITS / 8;
  const uint8_t set_wav_header[] = {
    'R', 'I', 'F', 'F', // ChunkID
    file_size, file_size >> 8, file_size >> 16, file_size >> 24, // ChunkSize
    'W', 'A', 'V', 'E', // Format
    'f', 'm', 't', ' ', // Subchunk1ID
    0x10, 0x00, 0x00, 0x00, // Subchunk1Size (16 for PCM)
    0x01, 0x00, // AudioFormat (1 for PCM)
    0x01, 0x00, // NumChannels (1 channel)
    sample_rate, sample_rate >> 8, sample_rate >> 16, sample_rate >> 24, // SampleRate
    byte_rate, byte_rate >> 8, byte_rate >> 16, byte_rate >> 24, // ByteRate
    0x02, 0x00, // BlockAlign
    0x10, 0x00, // BitsPerSample (16 bits)
    'd', 'a', 't', 'a', // Subchunk2ID
    wav_size, wav_size >> 8, wav_size >> 16, wav_size >> 24, // Subchunk2Size
  };
  memcpy(wav_header, set_wav_header, sizeof(set_wav_header));
}

static int i2s_init() {
  i2s_config_t i2s_config = {
    .mode                 = (i2s_mode_t)(I2S_MODE_MASTER | I2S_MODE_RX | I2S_MODE_PDM ),
    .sample_rate          = 16000U,
    .bits_per_sample      = I2S_BITS_PER_SAMPLE_16BIT,
    //.channel_format       = I2S_CHANNEL_FMT_ONLY_RIGHT,     // Also works
    .channel_format       = I2S_CHANNEL_FMT_ONLY_LEFT,
    //.communication_format = I2S_COMM_FORMAT_PCM,            // Also works
    .communication_format = I2S_COMM_FORMAT_I2S,   
    .intr_alloc_flags     = ESP_INTR_FLAG_LEVEL1,
    .dma_buf_count        = 8,
    .dma_buf_len          = 512,    
    .use_apll             = false,
    .tx_desc_auto_clear   = false,
    .fixed_mclk           = 0
  };
  
  i2s_pin_config_t pin_config = {
      .bck_io_num = -1,    // IIS_SCLK
      .ws_io_num = 42,     // IIS_LCLK
      .data_out_num = -1,  // IIS_DSIN
      .data_in_num = 41,   // IIS_DOUT
  
  };
  esp_err_t ret = 0;

  ret = i2s_driver_install((i2s_port_t)0, &i2s_config, 0, NULL);
  if (ret != ESP_OK) {
    Serial.printf("Error in i2s_driver_install");
  }

  ret = i2s_set_pin((i2s_port_t)0, &pin_config);
  if (ret != ESP_OK) {
    Serial.printf("Error in i2s_set_pin");
  }

  ret = i2s_zero_dma_buffer((i2s_port_t)0);
  if (ret != ESP_OK) {
    Serial.printf("Error in initializing dma buffer with 0");
  }

  return int(ret);
}

static int i2s_deinit(void) {
    i2s_driver_uninstall((i2s_port_t)0); //stop & destroy i2s driver
    return 0;
}

Regards,
Valter

3 Likes