Projet voice control

[Bob]

Bonjour,
J’ai reçu tout le matériel donc c’est parti mais voila…
J’ai suivi le branchement indiqué sur:

https://gist.github.com/EverythingSmartHome/055fbdde31a607ef9d695d5cac780e94

Même micro / ampli sur un ESP32WroomM32.

Mon yaml pour l’instant :

esphome:
  name: esp32-voice-control-1
  friendly_name: Esp32 Voice Control 1

esp32:
  board: esp32dev
  framework:
    type: arduino

# Enable logging
logger:

# Enable Home Assistant API
api:
  encryption:
    key: "q9W1KdaUqIRk4xxxxxxDhfIwh2r12VW7d7bSkke3f1U="

ota:
  password: "546340f29b91xxxxxxac0038e07534d9"

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password
  manual_ip:
    static_ip: 192.168.1.13
    gateway: 192.168.1.254
    subnet: 255.255.255.0

  # Enable fallback hotspot (captive portal) in case wifi connection fails
  ap:
    ssid: "Esp32-Voice-Control-1"
    password: "bDXXXxxx9KB6"

captive_portal:

switch:
  - platform: gpio
    name: "LedBleu2"
    pin: 2

# voice assistant
i2s_audio:
  - id: i2s_in
    i2s_lrclk_pin: GPIO26   #WS / LRC
    i2s_bclk_pin: GPIO25    #SCK /BCLK

microphone:
  - platform: i2s_audio
    adc_type: external
    pdm: false
    id: mic_i2s
    channel: left
    bits_per_sample: 32bit
    i2s_audio_id: i2s_in
    i2s_din_pin: GPIO33    #SD

speaker:
  - platform: i2s_audio
    id: my_speaker
    dac_type: external
    i2s_dout_pin: GPIO27   #DIN 
    mode: mono
    i2s_audio_id: i2s_in

# Example configuration entry
web_server:
  port: 80
    

Au flashage :

Uploading: [============================================================] 100% Done...

INFO Upload took 9.35 seconds, waiting for result...
INFO OTA successful
INFO Successfully uploaded program.
INFO Starting log output from 192.168.1.13 using esphome API
INFO Successfully connected to esp32-voice-control-1 @ 192.168.1.13 in 7.141s
INFO Successful handshake with esp32-voice-control-1 @ 192.168.1.13 in 0.119s
[17:37:09][I][app:102]: ESPHome version 2023.12.5 compiled on Dec 29 2023, 17:36:08
[17:37:09][C][wifi:573]: WiFi:
[17:37:09][C][wifi:405]:   Local MAC: B0:A7:32:22:77:A8
[17:37:09][C][wifi:410]:   SSID: [redacted]
[17:37:09][C][wifi:411]:   IP Address: 192.168.1.13
[17:37:09][C][wifi:413]:   BSSID: [redacted]
[17:37:09][C][wifi:414]:   Hostname: 'esp32-voice-control-1'
[17:37:09][C][wifi:416]:   Signal strength: -51 dB ▂▄▆█
[17:37:09][C][wifi:420]:   Channel: 11
[17:37:09][C][wifi:421]:   Subnet: 255.255.255.0
[17:37:09][C][wifi:422]:   Gateway: 192.168.1.254
[17:37:09][C][wifi:423]:   DNS1: 0.0.0.0
[17:37:09][C][wifi:424]:   DNS2: 0.0.0.0
[17:37:09][C][logger:443]: Logger:
[17:37:09][C][logger:444]:   Level: DEBUG
[17:37:09][C][logger:445]:   Log Baud Rate: 115200
[17:37:09][C][logger:447]:   Hardware UART: UART0
[17:37:09][C][switch.gpio:068]: GPIO Switch 'LedBleu2'
[17:37:09][C][switch.gpio:091]:   Restore Mode: always OFF
[17:37:09][C][switch.gpio:031]:   Pin: GPIO2
[17:37:09][C][captive_portal:088]: Captive Portal:
[17:37:09][C][web_server:168]: Web Server:
[17:37:09][C][web_server:169]:   Address: 192.168.1.13:80
[17:37:09][C][mdns:115]: mDNS:
[17:37:09][C][mdns:116]:   Hostname: esp32-voice-control-1
[17:37:09][C][ota:097]: Over-The-Air Updates:
[17:37:09][C][ota:098]:   Address: 192.168.1.13:3232
[17:37:09][C][ota:101]:   Using Password.
[17:37:09][C][api:139]: API Server:
[17:37:09][C][api:140]:   Address: 192.168.1.13:6053
[17:37:09][C][api:142]:   Using noise encryption: YES
[17:37:41][D][api:102]: Accepted 192.168.1.32
[17:37:42][W][api.connection:092]: : Socket operation failed: HANDSHAKESTATE_READ_FAILED errno=11

Je vois bien l’ESP et je peux piloter la LED bleu.

Une idée ?
PS : 192.168.1.32 c’est mon PI.

Bob

[WarC0zes]

Salut,
il te manque c’est parti du code.

voice_assistant:
  microphone: mic
  use_wake_word: false
  noise_suppression_level: 2
  auto_gain: 31dBFS
  volume_multiplier: 2.0
  speaker: big_speaker
  id: assist

switch:
  - platform: template
    name: Use wake word
    id: use_wake_word
    optimistic: true
    restore_mode: RESTORE_DEFAULT_ON
    entity_category: config
    on_turn_on:
      - lambda: id(assist).set_use_wake_word(true);
      - if:
          condition:
            not:
              - voice_assistant.is_running
          then:
            - voice_assistant.start_continuous
    on_turn_off:
      - voice_assistant.stop
      - lambda: id(assist).set_use_wake_word(false);

[Bob]

Bonsoir @WarC0zes
Merci beaucoup, je pensais avancer pas à pas et déjà trouver micro et ampli
C’est OK merci, reste à paramétrer le « word detected » avec snowboy ou autre
[18:35:21][E][voice_assistant:656]: Error: no_wake_word - No wake word detected

ça avance

captive_portal:


# voice assistant
i2s_audio:
  - id: i2s_in
    i2s_lrclk_pin: GPIO26   #WS / LRC
    i2s_bclk_pin: GPIO25    #SCK /BCLK

microphone:
  - platform: i2s_audio
    adc_type: external
    pdm: false
    id: mic_i2s
    channel: left
    bits_per_sample: 32bit
    i2s_audio_id: i2s_in
    i2s_din_pin: GPIO33    #SD

speaker:
  - platform: i2s_audio
    id: my_speaker
    dac_type: external
    i2s_dout_pin: GPIO27   #DIN 
    mode: mono
    i2s_audio_id: i2s_in

voice_assistant:
  microphone: mic_i2s
  id: assist
  noise_suppression_level: 2
  auto_gain: 31dBFS
  volume_multiplier: 2.0
  use_wake_word: true
  speaker: my_speaker


binary_sensor:
  - platform: status
    name: API Connection
    id: api_connection
    filters:
      - delayed_on: 1s
    on_press:
      - if:
          condition:
            switch.is_on: use_wake_word
          then:
            - voice_assistant.start_continuous:
    on_release:
      - if:
          condition:
            switch.is_on: use_wake_word
          then:
            - voice_assistant.stop:
              
switch:
  - platform: template
    name: Use wake word
    id: use_wake_word
    optimistic: true
    restore_mode: RESTORE_DEFAULT_ON
    entity_category: config
    on_turn_on:
      - lambda: id(assist).set_use_wake_word(true);
      - if:
          condition:
            not:
              - voice_assistant.is_running
          then:
            - voice_assistant.start_continuous
    
    on_turn_off:
      - voice_assistant.stop
      - lambda: id(assist).set_use_wake_word(false);

  - platform: gpio
    name: "LedBleu2"
    id: led
    pin: 2

Bob

[18yt8zhg]

@WarC0zes

Snowboy

Hello, désolé pour la réponse tardive, je n’ai pu m’en occuper avant. En effet, c’est beaucoup plus rapide mais toujours le même soucis, il ne comprend rien, même quand je lui demande d’allumer le four…

[WarC0zes]

Salut,
je suis passer sur porcupine ( GitHub - chatziko/wyoming-porcupine ) au lieu de snowboy et j’utilise un wake word perso, « Terminator » en FR .

Utilise Vosk ( https://github.com/rhasspy/hassio-addons/tree/master/vosk ) a la place de faster-whisper, si c’est pas déja fait.

Utilise l’option alias qui ce paramètre dans chaque entité, pour utiliser un nom simple de l’appareil si c’est un peu trop compliquer a comprendre.

exemple: quel est la température du séjour ?

Pour savoir les commandes pris en charge regarde les sentences en FR

[18yt8zhg]

Merci pour ton retour. Je regarde les alias, et si jamais je reconfigure ca.

[Krull56]

Hello,

Comme convenu, petit retour flash sur le korvo V1.1, reçu cet après-midi.
Pour mon premier test j’ai utilisé le yaml dispo ici :
korvo-esphome/esp32korvo_esphome.yaml at dev · haade-administrator/korvo-esphome (github.com)

Il est à modifier en partie pour prendre en compte la restriction d’utilisation multiple des gpio apparue dans la version 2023.12 d’EspHome.

Partie à modifier :

i2s_audio:
  - id: codec
    i2s_lrclk_pin: GPIO22
    i2s_bclk_pin: GPIO25
    i2s_mclk_pin: 
      number: GPIO0
      allow_other_uses: true
  - id: mic_adc
    i2s_lrclk_pin: GPIO26
    i2s_bclk_pin: GPIO27
    i2s_mclk_pin: 
      number: GPIO0
      allow_other_uses: true

Très très bonne sensibilité et portée du mic-array, même dans un environnement bruyant (sans autre voix , mais avec musique OK).
La partie Speaker ne fonctionne pas pour l’instant, mais il y a du monde sur le coup.

@+

[Bob]

Bonsoir,
comme indiqué j’ai suivi le câblage de:

https://gist.github.com/EverythingSmartHome/055fbdde31a607ef9d695d5cac780e94

Mon assistant (favoris):

Je teste avec perroquet pour l’instant mais je retrouve bien dans la liste le mot créé sur porcupine et déposé dans /shared/porcupine

Wyoming Protocol:

Mon ESP32 (autorisé à passer des appels de service HA):

Assist en cours est toujours désactivé et je ne vois jamais dans les logs une ligne de ce type:
[23:02:18][D][voice_assistant:538]: Wake word detected

Mon YAML d’ESPHome :

esphome:
  name: esp32-voice-control-1
  friendly_name: Esp32 Voice Control 1
  on_boot:
    - priority: -100
      then:
        - wait_until: api.connected
        - delay: 5s
        - speaker.play:
            data: [0x00]
        #- delay: 1s
        #- if:
        #    condition:
        #      switch.is_on: use_wake_word
        #    then:
        #      - voice_assistant.start_continuous:

esp32:
  board: esp32dev
  framework:
    type: arduino

# Enable logging
logger:

# Enable Home Assistant API
api:
  encryption:
    key: "q9W1KdaUqIRkxxxxxxxxxfIwh2r12VW7d7bSkke3f1U="

ota:
  password: "54634xxxxxxxxx94b2ac0038e07534d9"

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password
  manual_ip:
    static_ip: 192.168.1.13
    gateway: 192.168.1.254
    subnet: 255.255.255.0

  # Enable fallback hotspot (captive portal) in case wifi connection fails
  ap:
    ssid: "Esp32-Voice-Control-1"
    password: "bxxxxxxxxKB6"

captive_portal:


# voice assistant
i2s_audio:
  - id: i2s_in
    i2s_lrclk_pin: GPIO26   #WS / LRC
    i2s_bclk_pin: GPIO25    #SCK /BCLK

microphone:
  - platform: i2s_audio
    adc_type: external
    pdm: false
    id: mic_i2s
    channel: left
    bits_per_sample: 32bit
    i2s_audio_id: i2s_in
    i2s_din_pin: GPIO33    #SD

speaker:
  - platform: i2s_audio
    id: my_speaker
    dac_type: external
    i2s_dout_pin: GPIO27   #DIN 
    mode: mono
    i2s_audio_id: i2s_in

voice_assistant:
  microphone: mic_i2s
  id: brunoassist
  noise_suppression_level: 2
  auto_gain: 31dBFS
  volume_multiplier: 2.0
  use_wake_word: true
  speaker: my_speaker

  #on_error: 
  # - if:
  #      condition:
  #        switch.is_on: use_wake_word
  #      then:
  #        - switch.turn_off: use_wake_word
  #        - switch.turn_on: use_wake_word      

  on_client_connected:
    - if:
        condition:
          switch.is_on: use_wake_word
        then:
          - voice_assistant.start_continuous:

  on_client_disconnected:
    - if:
        condition:
          switch.is_on: use_wake_word
        then:
          - voice_assistant.stop:

binary_sensor:
  - platform: status
    name: API Connection
    id: api_connection
    filters:
      - delayed_on: 1s
    on_press:
      - if:
          condition:
            switch.is_on: use_wake_word
          then:
            - voice_assistant.start_continuous:
    on_release:
      - if:
          condition:
            switch.is_on: use_wake_word
          then:
            - voice_assistant.stop:
              
switch:
  - platform: template
    name: Use wake word
    id: use_wake_word
    optimistic: true
    restore_mode: RESTORE_DEFAULT_ON
    entity_category: config
    on_turn_on:
      - lambda: id(brunoassist).set_use_wake_word(true);
      - if:
          condition:
            not:
              - voice_assistant.is_running
          then:
            - voice_assistant.start_continuous
    
    on_turn_off:
      - voice_assistant.stop
      - lambda: id(brunoassist).set_use_wake_word(false);

  - platform: gpio
    name: "LedBleu2"
    id: led
    pin: 2

# Example configuration entry
web_server:
  port: 80
    

Un extrait des logs de l’ESP32 en prononçant le mot de réveil :

[19:50:40][D][voice_assistant:529]: Event Type: 0
[19:50:40][D][voice_assistant:529]: Event Type: 2
[19:50:40][D][voice_assistant:619]: Assist Pipeline ended
[19:50:40][D][voice_assistant:422]: State changed from STREAMING_MICROPHONE to IDLE
[19:50:40][D][voice_assistant:428]: Desired state set to IDLE
[19:50:40][D][voice_assistant:422]: State changed from IDLE to START_PIPELINE
[19:50:40][D][voice_assistant:428]: Desired state set to START_MICROPHONE
[19:50:40][D][voice_assistant:206]: Requesting start...
[19:50:40][D][voice_assistant:422]: State changed from START_PIPELINE to STARTING_PIPELINE
[19:50:40][D][voice_assistant:443]: Client started, streaming microphone
[19:50:40][D][voice_assistant:422]: State changed from STARTING_PIPELINE to STREAMING_MICROPHONE
[19:50:40][D][voice_assistant:428]: Desired state set to STREAMING_MICROPHONE
[19:50:40][D][voice_assistant:529]: Event Type: 1
[19:50:40][D][voice_assistant:532]: Assist Pipeline running
[19:50:40][D][voice_assistant:529]: Event Type: 9
[19:50:45][D][voice_assistant:529]: Event Type: 0
[19:50:45][D][voice_assistant:529]: Event Type: 2
[19:50:45][D][voice_assistant:619]: Assist Pipeline ended
[19:50:45][D][voice_assistant:422]: State changed from STREAMING_MICROPHONE to IDLE
[19:50:45][D][voice_assistant:428]: Desired state set to IDLE
[19:50:45][D][voice_assistant:422]: State changed from IDLE to START_PIPELINE
[19:50:45][D][voice_assistant:428]: Desired state set to START_MICROPHONE
[19:50:45][D][voice_assistant:206]: Requesting start...
[19:50:45][D][voice_assistant:422]: State changed from START_PIPELINE to STARTING_PIPELINE
[19:50:45][D][voice_assistant:443]: Client started, streaming microphone
[19:50:45][D][voice_assistant:422]: State changed from STARTING_PIPELINE to STREAMING_MICROPHONE
[19:50:45][D][voice_assistant:428]: Desired state set to STREAMING_MICROPHONE
[19:50:45][D][voice_assistant:529]: Event Type: 1
[19:50:45][D][voice_assistant:532]: Assist Pipeline running
[19:50:45][D][voice_assistant:529]: Event Type: 9
[19:50:50][D][voice_assistant:529]: Event Type: 0
[19:50:50][D][voice_assistant:529]: Event Type: 2
[19:50:50][D][voice_assistant:619]: Assist Pipeline ended
[19:50:50][D][voice_assistant:422]: State changed from STREAMING_MICROPHONE to IDLE
[19:50:50][D][voice_assistant:428]: Desired state set to IDLE
[19:50:50][D][voice_assistant:422]: State changed from IDLE to START_PIPELINE
[19:50:50][D][voice_assistant:428]: Desired state set to START_MICROPHONE
[19:50:50][D][voice_assistant:206]: Requesting start...
[19:50:50][D][voice_assistant:422]: State changed from START_PIPELINE to STARTING_PIPELINE
[19:50:50][D][voice_assistant:443]: Client started, streaming microphone
[19:50:50][D][voice_assistant:422]: State changed from STARTING_PIPELINE to STREAMING_MICROPHONE
[19:50:50][D][voice_assistant:428]: Desired state set to STREAMING_MICROPHONE
[19:50:50][D][voice_assistant:529]: Event Type: 1
[19:50:50][D][voice_assistant:532]: Assist Pipeline running
[19:50:50][D][voice_assistant:529]: Event Type: 9
[19:50:51][D][api:102]: Accepted 192.168.1.32
[19:50:51][W][api.connection:092]: : Socket operation failed: HANDSHAKESTATE_READ_FAILED errno=11
[19:50:55][D][voice_assistant:529]: Event Type: 0
[19:50:55][D][voice_assistant:529]: Event Type: 2
[19:50:55][D][voice_assistant:619]: Assist Pipeline ended
[19:50:55][D][voice_assistant:422]: State changed from STREAMING_MICROPHONE to IDLE
[19:50:55][D][voice_assistant:428]: Desired state set to IDLE
[19:50:55][D][voice_assistant:422]: State changed from IDLE to START_PIPELINE
[19:50:55][D][voice_assistant:428]: Desired state set to START_MICROPHONE
[19:50:55][D][voice_assistant:206]: Requesting start...
[19:50:55][D][voice_assistant:422]: State changed from START_PIPELINE to STARTING_PIPELINE
[19:50:55][D][voice_assistant:443]: Client started, streaming microphone
[19:50:55][D][voice_assistant:422]: State changed from STARTING_PIPELINE to STREAMING_MICROPHONE
[19:50:55][D][voice_assistant:428]: Desired state set to STREAMING_MICROPHONE
[19:50:55][D][voice_assistant:529]: Event Type: 1
[19:50:55][D][voice_assistant:532]: Assist Pipeline running
[19:50:55][D][voice_assistant:529]: Event Type: 9
[19:51:00][D][voice_assistant:529]: Event Type: 0
[19:51:00][D][voice_assistant:529]: Event Type: 2
[19:51:00][D][voice_assistant:619]: Assist Pipeline ended
[19:51:00][D][voice_assistant:422]: State changed from STREAMING_MICROPHONE to IDLE
[19:51:00][D][voice_assistant:428]: Desired state set to IDLE
[19:51:00][D][voice_assistant:422]: State changed from IDLE to START_PIPELINE
[19:51:00][D][voice_assistant:428]: Desired state set to START_MICROPHONE
[19:51:00][D][voice_assistant:206]: Requesting start...
[19:51:00][D][voice_assistant:422]: State changed from START_PIPELINE to STARTING_PIPELINE
[19:51:00][D][voice_assistant:443]: Client started, streaming microphone
[19:51:00][D][voice_assistant:422]: State changed from STARTING_PIPELINE to STREAMING_MICROPHONE
[19:51:00][D][voice_assistant:428]: Desired state set to STREAMING_MICROPHONE
[19:51:00][D][voice_assistant:529]: Event Type: 1
[19:51:00][D][voice_assistant:532]: Assist Pipeline running
[19:51:00][D][voice_assistant:529]: Event Type: 9

Avec de temps en temps cette ligne :

[19:59:53][W][api.connection:092]: : Socket operation failed: HANDSHAKESTATE_READ_FAILED errno=11

Voilà pour l’instant, pas de reconnaissance du mot de réveil alors si vous avez une petite ou grande idée
Je précise que via assist depuis le micro du PC j’arrive à allumer des lampes.

Merci d’avance pour votre aide.
Bob

[Krull56]

Hello @Bob

Mes meilleurs voeux pour cette nouvelle année.

Peut-être as-tu un pb de micro sur ton montage.
En ajoutant ces lignes à ton yaml, tu peux forcer l’écoute sans wakeword, et tu vas voir le résultat dans le pipeline

  - platform: template
    id: listen
    name: Listen
    optimistic: true
    restore_mode: ALWAYS_OFF
    on_turn_on:
      - switch.turn_off: use_wake_word
      - delay: 0.5s
      - voice_assistant.start
    on_turn_off:
      - switch.turn_on: use_wake_word

[Bob]

Bonjour @Krull56,
Meilleur vœux à toi aussi
J’attendais avec impatience une (ta) réponse , nous sommes nombreux je crois sur le sujet.
Merci, je teste ce midi, j’ai deux micros et 3 amplis d’avance au cas où.
Dans le pipeline je verrai le texte énoncé ?

Merci
Bob

[Krull56]

Oui, dans la partie débogage du pipeline

[Bob]

@Krull56
Je le colle où le code
En remplacement d’un autre code ?

J’ai testé quelques possibilités mais ça ne passe pas au flashage

Merci
Bob

[Krull56]

Effectivement, je viens de tester une recompile et aussi une erreur.
C’est passé en changeant listen par listen_force

On dirait que le mot listen est maintenant réservé par Esphome

Ce code est à ajouter au niveau des switchs

[Bob]

OK, merci @Krull56, je ferai un test ce soir, j’ai testé un deuxième micro sans meilleurs résultat.
Bob

[Bob]

J’ai bien le Listen maintenant mais je ne vois pas de résultat dans le pipeline.

Heureusement que le micro ne m’entend pas

Bob

[Krull56]

Quand tu active listen, tu devrais avoir assist en cours qui passe en activé.
Je pense pas que VOSK accepte les jurons

[Bob]

En effet, mais pas de résultat dans le pipeline si je parle
Puis assist en cours repasse a désactivé.

Bob

[Krull56]

Quand tu es dans le débogage du pipeline, tu peux acceder à l’historique en haut.
Tu n’as rien en revenant en arrière ?

[Bob]

Mais la majorité du temps j’ai :

Ou

Bob

[Krull56]

T’es sur une breadboard ?
Dans la positive, supprime le MAX et modifie ton yaml en conséquence (éventuellement branche l’INMP sur d’autres GPIO)