Accés données long terme pour export vers BD externe

Bonjour à tous et merci pour votre aide.

Mon problème

J’aimerai accéder à mes données long terme concernant ma consommation, ma production et ma réinjection dans le but d’exporter chaque jour les données de la veille vers un BD externe mariadb.

Ma configuration


System Information

version core-2023.8.4
installation_type Home Assistant OS
dev false
hassio true
docker true
user root
virtualenv false
python_version 3.11.4
os_name Linux
os_version 6.1.45
arch x86_64
timezone Europe/Paris
config_dir /config
Home Assistant Community Store
GitHub API ok
GitHub Content ok
GitHub Web ok
GitHub API Calls Remaining 5000
Installed Version 1.32.1
Stage running
Available Repositories 1289
Downloaded Repositories 42
Home Assistant Cloud
logged_in false
can_reach_cert_server ok
can_reach_cloud_auth ok
can_reach_cloud ok
Home Assistant Supervisor
host_os Home Assistant OS 10.5
update_channel stable
supervisor_version supervisor-2023.08.1
agent_version 1.5.1
docker_version 23.0.6
disk_total 30.8 GB
disk_used 6.8 GB
healthy true
supported true
board ova
supervisor_api ok
version_api ok
installed_addons Samba share (10.0.2), Terminal & SSH (9.7.1), RPC Shutdown (2.4), MariaDB (2.6.1), Samba Backup (5.2.0), Mosquitto broker (6.3.0), File editor (5.6.0), Zigbee2MQTT (1.33.0-1)
Dashboards
dashboards 3
resources 25
views 33
mode storage
Recorder
oldest_recorder_run 29 août 2023 à 15:45
current_recorder_run 3 septembre 2023 à 10:57
estimated_db_size 344.16 MiB
database_engine sqlite
database_version 3.41.2
___

Bonjour,

de mon coté, j’aila base de donnée HA pour la gestion au quotidien et qui n’a QUE 7 jours d’historique, de l’autre InfluDB avec une selection d’entité a garder sur du long terme, comme la conso par exemple.
peut etre que MariaDb fait la même chose ?

influxdb:
  host: a0d7b954-influxdb
  port: 8086
  database: homeassistantDB
  username: !secret influxdb_user
  password: !secret influxdb_password
  max_retries: 3
  default_measurement: state
  include:
    entities:
      xxx
      xxx
      xxx
      xxx

Bonjour,
Via un Google traduction, voici une explication que j’avais vu de pourquoi changer de base de données et comment faire…

Hello

Le truc c’est que le postulat de départ n’est pas tout à fait vrai …

Although the default SQLite is fine for the average user, and can be fine tuned to include or exclude entities for record keeping, we suggest switching to a better, more optimized database library, such as MariaDB . They are both Relational DBMS , and serve the same purpose, but MariaDB is superior in handling large datasets, easier extraction of data/integrations and is faster when displaying history in Home Assistant .

Avec un bon tuning de la base (include/exclude+stats) la base est petite (quelques 100aines de Mo et pas pour 30J comme dans ce tuto ! )… donc on a pas besoin de manipuler des grosses quantités de données (autant en écriture, en recherche qu’en consultation d’historique)… Quant à l’export des données (et on oublie l’import, c’est même pas envisageable) vue le modèle de données, je doute que ce soit à la portée de beaucoup de monde…
Autant partir sur une base influxdb pour ‹ sortir › les quelques données intéressantes, c’est bien plus rapide

Salut,

j’ai exactement ça, car je voulais quelque chose de plus manipulable que InfluxDB que je n’aimais pas trop après plusieurs mois d’utilisation.

J’ai un serveur MariaDB sur lequel j’ai ma base de HA, mais aussi ma base à long terme.
Un évènement de la DB qui toutes les nuits récupère de HA les infos qui m’intéressent pour les coller dans ma table conso_maison, qui me permet de garder tout ça.

Depuis que j’ai fait ça avec HA (3 ans), j’ai eu 3 fois des corrections à faire à cause de l’évolution du modèle de données de HA, le changement majeur c’était en début 2023.

Ma base contient ces mêmes infos depuis presque 10ans car j’y ai déjà mis les données que j’avais récupéré de Domoticz et d’une autre système.

Le SQL est un peu bourrin mais ça marche. Il y a en complément une première requête qui créé l’entrée de chaque jour. Et celle-ci récupère les données:

UPDATE conso 
SET
VOITURE = (SELECT max(CAST(state as decimal(10,3))) - min(CAST(state as decimal(10,3))) FROM homeassistant.states 
      WHERE metadata_id = "4185" AND state <> "unknown" AND state <> "unavailable" AND state <> '' AND DATE(FROM_UNIXTIME(last_updated_ts + 7200)) = DATE(NOW() - INTERVAL 1 DAY)),
CONSO_MAISON = (SELECT max(CAST(state as decimal(10,3))) - min(CAST(state as decimal(10,3))) FROM homeassistant.states 
      WHERE metadata_id = "65" AND state <> "unknown" AND state <> "unavailable" AND state <> '' AND DATE(FROM_UNIXTIME(last_updated_ts + 7200)) = DATE(NOW() - INTERVAL 1 DAY)),
CONSO_SOLAR = (SELECT max(CAST(state as decimal(10,3))) - min(CAST(state as decimal(10,3))) FROM homeassistant.states 
      WHERE metadata_id = "120" AND state <> "unknown" AND state <> "unavailable" AND state <> '' AND DATE(FROM_UNIXTIME(last_updated_ts + 7200)) = DATE(NOW() - INTERVAL 1 DAY)),
EXPORT = (SELECT max(CAST(state as decimal(10,3))) - min(CAST(state as decimal(10,3))) FROM homeassistant.states 
      WHERE metadata_id = "119" AND state <> "unknown" AND state <> "unavailable" AND state <> '' AND DATE(FROM_UNIXTIME(last_updated_ts + 7200)) = DATE(NOW() - INTERVAL 1 DAY)),
PROD_SOLAR = (SELECT (max(CAST(state as decimal(10,3))) - min(CAST(state as decimal(10,3)))) / 1000 FROM homeassistant.states 
      WHERE metadata_id = "200" AND state <> "unknown" AND state <> "unavailable" AND state <> '' AND DATE(FROM_UNIXTIME(last_updated_ts + 7200)) = DATE(NOW() - INTERVAL 1 DAY)),
HC = (SELECT max(CAST(state as decimal(10,3))) - min(CAST(state as decimal(10,3))) FROM homeassistant.states 
      WHERE metadata_id = "52" AND state <> "unknown" AND state <> "unavailable" AND state <> '' AND DATE(FROM_UNIXTIME(last_updated_ts + 7200)) = DATE(NOW() - INTERVAL 1 DAY)),
HC_INDEX = (SELECT max(CAST(state as decimal(10,3))) FROM homeassistant.states 
            WHERE metadata_id = "52" AND state <> "unknown" AND state <> "unavailable" AND state <> ''  AND DATE(FROM_UNIXTIME(last_updated_ts + 7200)) = DATE(NOW() - INTERVAL 1 DAY)),
HP = (SELECT max(CAST(state as decimal(10,3))) - min(CAST(state as decimal(10,3))) FROM homeassistant.states 
      WHERE metadata_id = "237"  AND state <> "unknown" AND state <> "unavailable" AND state <> '' AND DATE(FROM_UNIXTIME(last_updated_ts + 7200)) = DATE(NOW() - INTERVAL 1 DAY)),
HP_INDEX = (SELECT max(CAST(state as decimal(10,3))) FROM homeassistant.states 
            WHERE metadata_id = "237"  AND state <> "unknown" AND state <> "unavailable" AND state <> '' AND DATE(FROM_UNIXTIME(last_updated_ts + 7200)) = DATE(NOW() - INTERVAL 1 DAY)),
demarrages = (SELECT max(CAST(state as decimal(10,3))) - min(CAST(state as decimal(10,3))) FROM homeassistant.states 
      WHERE metadata_id = "103"  AND state <> "unknown" AND state <> "unavailable" AND state <> '' AND DATE(FROM_UNIXTIME(last_updated_ts + 7200)) = DATE(NOW() - INTERVAL 1 DAY)),
heures_appoint1 = (SELECT max(CAST(state as decimal(10,3))) - min(CAST(state as decimal(10,3))) FROM homeassistant.states 
      WHERE metadata_id = "3"  AND state <> "unknown" AND state <> "unavailable" AND state <> '' AND DATE(FROM_UNIXTIME(last_updated_ts + 7200)) = DATE(NOW() - INTERVAL 1 DAY)),
heures_appoint2 = (SELECT max(CAST(state as decimal(10,3))) - min(CAST(state as decimal(10,3))) FROM homeassistant.states 
      WHERE metadata_id = "102"  AND state <> "unknown" AND state <> "unavailable" AND state <> '' AND DATE(FROM_UNIXTIME(last_updated_ts + 7200)) = DATE(NOW() - INTERVAL 1 DAY)),
heures_comp = (SELECT max(CAST(state as decimal(10,3))) - min(CAST(state as decimal(10,3))) FROM homeassistant.states 
      WHERE metadata_id = "43"  AND state <> "unknown" AND state <> "unavailable" AND state <> '' AND DATE(FROM_UNIXTIME(last_updated_ts + 7200)) = DATE(NOW() - INTERVAL 1 DAY)),
puissance_conso = (SELECT max(CAST(state as decimal(10,3))) - min(CAST(state as decimal(10,3))) FROM homeassistant.states 
      WHERE metadata_id = "29"  AND state <> "unknown" AND state <> "unavailable" AND state <> '' AND DATE(FROM_UNIXTIME(last_updated_ts + 7200)) = DATE(NOW() - INTERVAL 1 DAY)),
puissance_chauffage = (SELECT max(CAST(state as decimal(10,3))) - min(CAST(state as decimal(10,3))) FROM homeassistant.states 
      WHERE metadata_id = "54"  AND state <> "unknown" AND state <> "unavailable" AND state <> '' AND DATE(FROM_UNIXTIME(last_updated_ts + 7200)) = DATE(NOW() - INTERVAL 1 DAY)),
temp_ext = (SELECT avg(CAST(state as decimal(10,3))) FROM homeassistant.states 
      WHERE metadata_id = "37"  AND state <> "unknown" AND state <> "unavailable" AND state <> '' AND DATE(FROM_UNIXTIME(last_updated_ts + 7200)) = DATE(NOW() - INTERVAL 1 DAY)),
temp_ext_min = (SELECT min(CAST(state as decimal(10,3))) FROM homeassistant.states 
      WHERE metadata_id = "37"  AND state <> "unknown" AND state <> "unavailable" AND state <> '' AND DATE(FROM_UNIXTIME(last_updated_ts + 7200)) = DATE(NOW() - INTERVAL 1 DAY)),
temp_ext_max = (SELECT max(CAST(state as decimal(10,3))) FROM homeassistant.states 
      WHERE metadata_id = "37"  AND state <> "unknown" AND state <> "unavailable" AND state <> '' AND DATE(FROM_UNIXTIME(last_updated_ts + 7200)) = DATE(NOW() - INTERVAL 1 DAY)),	  
temp_depart = (SELECT avg(CAST(state as decimal(10,3))) FROM homeassistant.states 
      WHERE metadata_id = "101"  AND state <> "unknown" AND state <> "unavailable" AND state <> '' AND DATE(FROM_UNIXTIME(last_updated_ts + 7200)) = DATE(NOW() - INTERVAL 1 DAY)),
temp_consigne = (SELECT avg(CAST(state as decimal(10,3))) FROM homeassistant.states 
      WHERE metadata_id = "9"  AND state <> "unknown" AND state <> "unavailable" AND state <> '' AND DATE(FROM_UNIXTIME(last_updated_ts + 7200)) = DATE(NOW() - INTERVAL 1 DAY)),
temp_jean = (SELECT avg(CAST(state as decimal(10,3))) FROM homeassistant.states 
      WHERE metadata_id = "243"  AND state <> "unknown" AND state <> "unavailable" AND state <> '' AND DATE(FROM_UNIXTIME(last_updated_ts + 7200)) = DATE(NOW() - INTERVAL 1 DAY)),
temp_parents = (SELECT avg(CAST(state as decimal(10,3))) FROM homeassistant.states 
      WHERE metadata_id = "90"  AND state <> "unknown" AND state <> "unavailable" AND state <> '' AND DATE(FROM_UNIXTIME(last_updated_ts + 7200)) = DATE(NOW() - INTERVAL 1 DAY)),
temp_couloir = (SELECT avg(CAST(state as decimal(10,3))) FROM homeassistant.states 
      WHERE metadata_id = "241" AND state <> "unknown" AND state <> "unavailable" AND state <> '' AND DATE(FROM_UNIXTIME(last_updated_ts + 7200)) = DATE(NOW() - INTERVAL 1 DAY)),
temp_garage = (SELECT avg(CAST(state as decimal(10,3))) FROM homeassistant.states 
      WHERE metadata_id = "40"  AND state <> "unknown" AND state <> "unavailable" AND state <> '' AND DATE(FROM_UNIXTIME(last_updated_ts + 7200)) = DATE(NOW() - INTERVAL 1 DAY))
WHERE date = date(NOW() - INTERVAL 1 DAY)

Merci pour vos premières réponses cependant je maîtrise la partie recorder et je n’ai pas de problème de taille de base de données. Je veux simplement accéder aux valeurs de certaines statistiques pour l’exporter ailleur.

La question est comment faire un « get prod yesterday »?