M5Stickc Plus 2 voice assistant , soucis de GPIO pour le microphone

@bentou le lien que tu ma mis est pour le PLUS, pas le PLUS 2.
C’est ici pour le PLUS 2 M5Stack M5StickC PLUS2 | devices.esphome.io

ils ont pas tester pour l’audio, les pin sont inverser en plus.

# note: Audio hasn't been tested
i2s_audio:
  id: bus_i2s
  i2s_lrclk_pin: G26
  i2s_bclk_pin: G0

microphone:
  - platform: i2s_audio
    i2s_din_pin: GPIO34
    i2s_audio_id: bus_i2s
    adc_type: external
    pdm: true
    id: mic

J’ai réussi , a faire reconnaitre un truc, j’y suis pas loin du but.

[09:19:02][D][voice_assistant:439]: State changed from STOPPING_MICROPHONE to AWAITING_RESPONSE
[09:19:05][D][voice_assistant:563]: Event Type: 4
[09:19:05][D][voice_assistant:591]: Speech recognised as: "ah"
[09:19:05][D][voice_assistant:563]: Event Type: 5
[09:19:05][D][voice_assistant:596]: Intent started
[09:19:06][D][voice_assistant:563]: Event Type: 6
[09:19:06][D][voice_assistant:563]: Event Type: 7
[09:19:06][D][voice_assistant:619]: Response: "Désolé, je n'ai pas compris"
[09:19:06][D][light:036]: 'M5StickC PLUS2 Voice' Setting:
[09:19:06][D][light:051]:   Brightness: 60%
[09:19:06][D][light:085]:   Transition length: 1.0s
[09:19:06][D][voice_assistant:563]: Event Type: 8
[09:19:06][D][voice_assistant:639]: Response URL: "http://192.168.1.xxxx:8123/api/tts_proxy/393247aa3dd2cd24b4ee2f8550489431d86f0c02_fr-fr_a022601376_tts.piper.mp3"
[09:19:06][D][voice_assistant:439]: State changed from AWAITING_RESPONSE to IDLE
[09:19:06][D][voice_assistant:445]: Desired state set to IDLE
i2s_audio:
  - id: mic_adc
    i2s_lrclk_pin:
       number: GPIO0
       allow_other_uses: true
       ignore_strapping_warning: true
  - id: spk_adc
    i2s_lrclk_pin:
       number: GPIO0
       allow_other_uses: true
       ignore_strapping_warning: true
    i2s_bclk_pin: GPIO26


microphone:
  - platform: i2s_audio
    id: echo_microphone
    i2s_din_pin: GPIO34
    adc_type: external
    i2s_audio_id: mic_adc
    bits_per_sample: 16bit
    sample_rate: 44100
    pdm: true

Ah oui, je viens de voir le M5Stack M5StickC PLUS2 | devices.esphome.io

yes !!!

[10:09:47][D][voice_assistant:439]: State changed from STOPPING_MICROPHONE to AWAITING_RESPONSE
[10:09:48][D][voice_assistant:563]: Event Type: 4
[10:09:48][D][voice_assistant:591]: Speech recognised as: "quelle est la température du salon"
[10:09:48][D][voice_assistant:563]: Event Type: 5
[10:09:48][D][voice_assistant:596]: Intent started
[10:09:48][D][voice_assistant:563]: Event Type: 6
[10:09:48][D][voice_assistant:563]: Event Type: 7
[10:09:48][D][voice_assistant:619]: Response: "La température est de 17.4 °"
[10:09:48][D][light:036]: 'M5StickC PLUS2 Voice' Setting:
[10:09:48][D][light:051]:   Brightness: 60%
[10:09:48][D][light:085]:   Transition length: 1.0s
[10:09:48][D][voice_assistant:563]: Event Type: 8
[10:09:48][D][voice_assistant:639]: Response URL: "http://192.168.1.xxx:8123/api/tts_proxy/b973e9a209b4b43bb13890560f26b8364a4864e3_fr-fr_a022601376_tts.piper.mp3"
[10:09:48][D][voice_assistant:439]: State changed from AWAITING_RESPONSE to IDLE

Plus cas ajouter le speaker et voir le résultat final :smiley:

1 « J'aime »

Bravo !!! tu affiche les dialogues sur l’écran ?

Pas essayer encore, je cherche a faire fonctionner le micro et speaker spk2.

j’ai déja des idées en tête, pour afficher sur l’écran.

Voila !

[20:08:16][D][voice_assistant:439]: State changed from STOPPING_MICROPHONE to AWAITING_RESPONSE
[20:08:18][D][voice_assistant:563]: Event Type: 4
[20:08:18][D][voice_assistant:591]: Speech recognised as: "éteindre cuisine"
[20:08:18][D][voice_assistant:563]: Event Type: 5
[20:08:18][D][voice_assistant:596]: Intent started
[20:08:18][D][voice_assistant:563]: Event Type: 6
[20:08:18][D][voice_assistant:563]: Event Type: 7
[20:08:18][D][voice_assistant:619]: Response: "Éteint"
[20:08:18][D][text_sensor:064]: 'text-to-speech': Sending state 'Éteint'
[20:08:18][D][light:036]: 'M5StickC PLUS2 Voice' Setting:
[20:08:18][D][light:051]:   Brightness: 60%
[20:08:18][D][light:085]:   Transition length: 1.0s

Joli !!!

au lieu de mettre question et réponse, ce qui prend de la place, il faudrait mettre des couleurs …

Oui la réponse prends de la place et j’ai pas de retour a la ligne, ca sort de l’écran quand c’est trop long.

Des couleurs Ok, pour ok ou erreur. Mais quand tu attends une réponse de température, météo, obliger de mettre du texte.

ah c’est couillon l’absence de retour à la ligne…

Oui, faut le coder lol.

j’ai tester, la réponse sur un google nest, nikel. Ca peut être une bonne alternative, si j’arrive pas a utiliser le speaker sur l’esp.

trigger:
  - platform: state
    entity_id:
      - sensor.m5stickc_plus2_voice_text_to_speech
    to: null
condition: []
action:
  - service: tts.speak
    metadata: {}
    data:
      cache: true
      media_player_entity_id: media_player.nest_mini
      message: "{{ states(\"sensor.m5stickc_plus2_voice_text_to_speech\") }}"
    target:
      entity_id: tts.google_fr_fr

Salut @Bob ,
j’ai un message texte a afficher sur un écran, ou la longueur peu varier. Si le message est trop long il sort de l’écran, et je reçois des warnings dans les logs.

Connais tu une façon de découper le texte sur deux lignes, ou faire du scrolling sous esphome ?

text_sensor:
  - platform: template
    name: "text-to-speech"
    id: tts

display:
  - platform: ili9xxx
    model: st7789v
    cs_pin: GPIO5
    dc_pin: GPIO14
    reset_pin: GPIO12
    rotation: 90
    dimensions:
      height: 240
      width: 135
      offset_height: 40
      offset_width: 52
    # Required or the colors are all inverted, and Black screen is White
    invert_colors: true
    lambda: |-
      it.printf(5, 4, id(roboto), id(my_white), "Response:");
      it.printf(5, 30, id(roboto), id(my_green), id(tts).state.c_str());

Sinon, tu sais comment mettre un accent pour c’est la ligne:

it.printf(5, 4, id(roboto), id(my_white), "Response:");

remplacer "Response:" par "Réponse:"

Bonsoir @WarC0zes
Pour les accents et symbole j’avais mis un exemple ici:

Désolé je ne suis pas sur PC et copier coller de code délicat :smirk:
Pour le scroll ou longueur pas d’idée à l’instant.
Bob

ok, rien de spécial enfaite :slight_smile:

it.print(5, 160, id(my_font18), "Humidité :");

j’ai bien la police avec accents, mais je pense que c’est un autre soucis pour la compilation.

it.printf(5, 4, id(roboto), id(my_white), "Réponse:");

Je resterais.

Tu as bien les caractères, accentuées dans glyphs: ?
Bob

Oui, mais c’est un autre soucis enfaite. c’est moi qui pensais que ca compiler pas a cause d’un accent dans la ligne it.print, mais non .

font:
  - file: "gfonts://Roboto"
    id: roboto
    size: 18
    glyphs: "<>!'%()+,-_.:;*=°?#0123456789AÀBCDEÉÈÊFGHIJKLMNOPQRSTUVWXYZ aàbcdeéèêfghijklmnopqrstuvwxyzôöç"

je pense que c’est un soucis ou j’ai flasher un firmware avec 4MB de flash au lieu de 8MB.
J’ai voulu reflasher en OTA pour passer en 8MB . :sweat_smile:
J’ai effacer le flash et reflasher :wink:

1 « J'aime »

Salut @bentou,
tu compte te lancer quand pour le M5Stick Plus 2 ?
Un coup de main serait pas de refus, pour essayer de finaliser un firmware stable pour un voice assistant.

j’ai essayé, mais je n’arrive pas à aller bien loin…
j’ai le plus2 et le spk2, chargé un esphome, mais il n’écoute pas, l’effronté !!!

AH ! ta commencer a bosser deçu :+1:

Impeccable, on a le même matos :wink:

Allez, je te partage les recherches que j’ai faite.

substitutions:
  name: m5stickc-plus2-voice
  friendly_name: M5StickC PLUS2 Voice

esphome:
  name: ${name}
  name_add_mac_suffix: false
  friendly_name: ${friendly_name}
  project:
    name: m5stack.atom-echo-voice-assistant
    version: "1.0"
  min_version: 2023.11.1
  platformio_options:
    upload_speed: 115200  
  on_boot:
    - priority: 600
      then: 
        - output.turn_on: output_high
    - then:
      - light.turn_on:
          id: display_bl
          brightness: 75%
       
  on_shutdown:
    - priority: 600
      then:
        - output.turn_off: output_high

esp32:
  board: m5stick-c
  framework:
    type: esp-idf
  flash_size: 8MB

# Enable logging
logger:

# Enable Home Assistant API
api:
  encryption:
    key: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

ota:
  password: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"


wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password

improv_serial:

esp32_improv:
  authorizer: none

script:
  - id: reset_led
    then:
      - if:
          condition:
            switch.is_on: use_wake_word
          then:
            - light.turn_on:
                id: led
          else:
            - light.turn_off: led


output:
  - platform: ledc
    pin: GPIO19
    id: led_pwm
  # Buzzer output    
  - platform: ledc
    pin:
      number: GPIO2
    id: buzzer
    inverted: false
  # Backlight TFT LCD
  - platform: ledc
    pin: 27
    inverted: false
    id: backlight
  # Battery output
  - platform: gpio
    id: output_high
    pin:
      number: GPIO4
      mode:
        output: true

spi:
  clk_pin: GPIO13
  mosi_pin: GPIO15

#i2s_audio:
#  i2s_lrclk_pin: GPIO0

i2s_audio:
  - id: mic_adc
    i2s_lrclk_pin:
       number: GPIO0
       allow_other_uses: true
       ignore_strapping_warning: true
  - id: spk_adc
    i2s_lrclk_pin:
       number: GPIO0
       allow_other_uses: true
       ignore_strapping_warning: true
    i2s_bclk_pin: GPIO26


microphone:
  - platform: i2s_audio
    id: echo_microphone
    i2s_din_pin: GPIO34
    adc_type: external
    channel: left
    i2s_audio_id: mic_adc
#    bits_per_sample: 16bit
#    sample_rate: 44100
    pdm: true

#speaker:
#  - platform: i2s_audio
#    dac_type: internal
#    i2s_audio_id: spk_adc
#    mode: right

#speaker:
#  - platform: i2s_audio
#    id: echo_speaker
#    i2s_dout_pin: GPIO25
#    i2s_audio_id: spk_adc
#    dac_type: external
#    mode: mono


voice_assistant:
  id: va
  microphone: echo_microphone
#  speaker: echo_speaker
  noise_suppression_level: 2
  auto_gain: 31dBFS
  volume_multiplier: 2.0
  vad_threshold: 3
  on_listening:
    - light.turn_on:
        id: led
        effect: "Listening"
  on_stt_vad_end:
    - light.turn_on:
        id: led
        effect: "Listening"        
  on_tts_start:
    - text_sensor.template.publish:
        id: tts
        state: !lambda 'return x;'
    - light.turn_on:
        id: led
        brightness: 60%
    - delay: 100ms
    - light.turn_off: led
  on_end:
    - delay: 100ms
#    - wait_until:
#        not:
#          speaker.is_playing:
    - script.execute: reset_led
  on_error:
    - light.turn_on:
        id: led
        effect: "Error"
    - delay: 2s
    - script.execute: reset_led
    - script.wait: reset_led
  on_client_connected:
    - if:
        condition:
          switch.is_on: use_wake_word
        then:
          - voice_assistant.start_continuous:
          - script.execute: reset_led
  on_client_disconnected:
    - if:
        condition:
          switch.is_on: use_wake_word
        then:
          - voice_assistant.stop:
          - light.turn_off: led


binary_sensor:
  - platform: gpio
    pin:
      number: GPIO37
      inverted: true
    name: Button A
    disabled_by_default: true
    entity_category: diagnostic
    id: a_button
    on_multi_click:
      - timing:
          - ON for 40ms to 400ms
          - OFF for at least 330ms
        then:
          - if:
              condition:
                switch.is_off: use_wake_word
              then:
                - if:
                    condition: voice_assistant.is_running
                    then:
                      - voice_assistant.stop:
                      - script.execute: reset_led
                    else:
                      - output.turn_off: buzzer
                      - voice_assistant.start:
              else:
                - voice_assistant.stop
                - delay: 1s
                - script.execute: reset_led
                - script.wait: reset_led
                - voice_assistant.start_continuous:
      - timing:
          - ON for 40ms to 400ms
          - OFF for 40ms to 300ms
          - ON for 40ms to 400ms
          - OFF for at least 330ms
        then:
          - light.toggle: display_bl    


light:
  - platform: monochromatic
    id: led
    name: None
    disabled_by_default: true
    entity_category: config
    output: led_pwm
    effects:
      - pulse:
          name: "Listening"
          transition_length: 1s
          update_interval: 1s
          min_brightness: 0%
          max_brightness: 30%
      - pulse:
          name: "Error"
          transition_length: 0.25s
          update_interval: 0.25s
          min_brightness: 0%
          max_brightness: 100%
  # Backlight TFT LCD
  - platform: monochromatic
    output:  backlight
    name: Backlight
    id: display_bl
    restore_mode: ALWAYS_ON

switch:
  - platform: template
    name: Use wake word
    id: use_wake_word
    optimistic: true
    restore_mode: RESTORE_DEFAULT_OFF
    entity_category: config
    on_turn_on:
      - lambda: id(va).set_use_wake_word(true);
      - if:
          condition:
            not:
              - voice_assistant.is_running
          then:
            - voice_assistant.start_continuous
      - script.execute: reset_led
    on_turn_off:
      - voice_assistant.stop
      - lambda: id(va).set_use_wake_word(false);
      - script.execute: reset_led

text_sensor:
  - platform: template
    name: "text-to-speech"
    id: tts

font:
  - file: "gfonts://Roboto"
    id: roboto
    size: 18
    glyphs: "<>!'%()+,-_.:;*=°?#0123456789AÀBCDEÉÈÊFGHIJKLMNOPQRSTUVWXYZ aàbcdeéèêfghijklmnopqrstuvwxyzôöç"

color:
  - id: my_white
    red: 100%
    green: 100%
    blue: 100%
  - id: my_blue
    red: 0%
    green: 0%
    blue: 100%
  - id: my_red
    red: 100%
    green: 0%
    blue: 0%
  - id: my_green
    red: 0%
    green: 100%
    blue: 0%
  - id: my_yellow
    red: 100%
    green: 100%
    blue: 0%
  - id: my_orange
    red: 100%
    green: 50%
    blue: 0%

# 1.14 inch, 135*240 Colorful TFT LCD, ST7789v2
display:
  - platform: ili9xxx
    model: st7789v
    cs_pin: GPIO5
    dc_pin: GPIO14
    reset_pin: GPIO12
    rotation: 90
    dimensions:
      height: 240
      width: 135
      offset_height: 40
      offset_width: 52
    # Required or the colors are all inverted, and Black screen is White
    invert_colors: true
    lambda: |-
      it.printf(5, 4, id(roboto), id(my_white), "Réponse:");
      it.printf(5, 30, id(roboto), id(my_green), id(tts).state.c_str());

external_components:
  - source: github://pr#5230
    components:
      - esp_adf
    refresh: 0s

esp_adf:

Avec c’est config, ta le led rouge , l’écran, le micro, la battery ( activer , si tu débranche l’usb ).
Le bouton A ( logo M5 ) active le wakeword et tu dit ta commande.
La réponse est écrite sur l’écran, mais attention si le texte est trop long, l’écran peut bugger ( un point a améliorer ).

Maintenant pour activer le speaker, dans le M5Stickc Plus 2 ta un buzzer ( speaker ) sur le GPIO2 et si il est utiliser le microphone ne fonctionne pas. Il utilise le même GPIO0 pour le LRCLK.
Faut désactiver le buzzer, pour que le micro fonctionne.

Pour le speaker SPK2, il utilise le GPIO0 LRCLK, le GPIO26 BCLK et le GPIO25 DATA. Ca reviens au même que le buzzer GPIO2 il utilise le même GPIO0 pour le LRCLK.

Le but serais de pouvoir, activer ou désactiver le micro et speaker suivant le besoin dans voice_assistant. Mais je bloque sur la facon de mettre ca en place.

J’ai un code arduino, pour utiliser le micro et speaker ( buzzer ou SPK2 ).

Mais du mal a convertir ca pour esphome.

Y’a un truc qui doit manquer chez moi… ???

11:20:16	[D]	[binary_sensor:036]	
'Button A': Sending state ON
11:20:19	[D]	[binary_sensor:036]	
'Button A': Sending state OFF
11:20:30	[W]	[web_server_idf:070]	
Only application/x-www-form-urlencoded supported for POST request
11:20:30	[D]	[light:036]	
'M5StickC PLUS2 Voice' Setting:
11:20:30	[D]	[light:047]	
  State: ON
11:20:30	[D]	[light:109]	
  Effect: 'Listening'
11:20:36	[D]	[esp-idf:000]	
e[0;33mW (42825) httpd_uri: httpd_uri: Method '2' not allowed for URI '/'
11:20:36	[D]	[esp-idf:000]	
e[0;33mW (42829) httpd_txrx: httpd_resp_send_err: 405 Method Not Allowed - Request method for this URI is not handled by server
11:20:36	[E]	[spi-esp-idf:081]	
Transmit failed - err 101
11:21:06	[D]	[esp-idf:000]	
e[0;33mW (72827) httpd_uri: httpd_uri: Method '2' not allowed for URI '/'
11:21:06	[D]	[esp-idf:000]	
e[0;33mW (72831) httpd_txrx: httpd_resp_send_err: 405 Method Not Allowed - Request method for this URI is not handled by server
11:21:06	[E]	[spi-esp-idf:081]	
Transmit failed - err 101
11:21:36	[D]	[esp-idf:000]	
e[0;33mW (102842) httpd_uri: httpd_uri: Method '2' not allowed for URI '/'
11:21:36	[D]	[esp-idf:000]	
e[0;33mW (102846) httpd_txrx: httpd_resp_send_err: 405 Method Not Allowed - Request method for this URI is not handled by server
11:21:58	[D]	[api:102]	
Accepted 192.168.92.25
11:21:58	[W]	[component:237]	
Component api took a long time for an operation (62 ms).
11:21:58	[W]	[component:238]	
Components should block for at most 30 ms.
11:21:59	[D]	[api.connection:1210]	
Home Assistant 2024.4.4 (192.168.92.25): Connected successfully
11:22:01	[E]	[spi-esp-idf:081]	
Transmit failed - err 101
11:22:01	[W]	[component:237]	
Component display took a long time for an operation (682 ms).
11:22:01	[W]	[component:238]	
Components should block for at most 30 ms.
11:22:01	[E]	[spi-esp-idf:081]	
Transmit failed - err 101
11:22:01	[E]	[voice_assistant:392]	
Multiple API Clients attempting to connect to Voice Assistant
11:22:01	[E]	[voice_assistant:393]	
Current client: Home Assistant 2024.4.4 (192.168.92.25)
11:22:01	[E]	[voice_assistant:394]	
New client: Home Assistant 2024.4.4 (192.168.92.25)
11:22:01	[E]	[spi-esp-idf:081]	
Transmit failed - err 101
11:22:06	[D]	[esp-idf:000]	
e[0;33mW (132831) httpd_uri: httpd_uri: Method '2' not allowed for URI '/'
11:22:06	[D]	[esp-idf:000]	
e[0;33mW (132836) httpd_txrx: httpd_resp_send_err: 405 Method Not Allowed - Request method for this URI is not handled by server
11:22:07	[W]	[api.connection:102]	
Home Assistant 2024.4.4 (192.168.92.25): Connection reset
11:22:36	[D]	[esp-idf:000]	
e[0;33mW (162845) httpd_uri: httpd_uri: Method '2' not allowed for URI '/'
11:22:36	[D]	[esp-idf:000]	
e[0;33mW (162848) httpd_txrx: httpd_resp_send_err: 405 Method Not Allowed - Request method for this URI is not handled by server
11:22:36	[E]	[spi-esp-idf:081]	
Transmit failed - err 101
11:23:08	[D]	[esp-idf:000]	
e[0;33mW (192839) httpd_uri: httpd_uri: Method '2' not allowed for URI '/'
11:23:08	[D]	[esp-idf:000]	
e[0;33mW (192843) httpd_txrx: httpd_resp_send_err: 405 Method Not Allowed - Request method for this URI is not handled by server
11:23:08	[E]	[spi-esp-idf:081]	
Transmit failed - err 101
11:23:08	[E]	[spi-esp-idf:081]	
Transmit failed - err 101
11:23:22	[E]	[spi-esp-idf:081]	
Transmit failed - err 101

la seule différence c’est çà:

# Enable Home Assistant API
api:
  encryption:
    key: !secret api_key

ota:
  password: !secret ota_password

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password

captive_portal:
web_server:
  port: 80

Oui, j’ai eu c’est erreur. Chez pas comment il remédier encore ( peut être activer - output.turn_off: buzzer au boot ).
Je pense que c’est le GPIO2 du buzzer qui est activer, et fait conflit avec le micro.
Essaye d’appuer sur le button A ( logo m5 ), pour activer le micro.

j’ai ajouter - output.turn_off: buzzer

on_multi_click:
  - timing:
      - ON for 40ms to 400ms
      - OFF for at least 330ms
    then:
      - if:
          condition:
            switch.is_off: use_wake_word
          then:
            - if:
                condition: voice_assistant.is_running
                then:
                  - voice_assistant.stop:
                  - script.execute: reset_led
                else:
                  - output.turn_off: buzzer
                  - voice_assistant.start:

Tu peut essayer d’éteindre l’esp et le rallumer aussi ( bouton C ).