<!--
You are amazing! Thanks for contributing to our project!
Please, DO N…OT DELETE ANY TEXT from this template! (unless instructed).
-->
## Proposed change
<!--
Describe the big picture of your changes here to communicate to the
maintainers why we should accept this pull request. If it fixes a bug
or resolves a feature request, be sure to link to that issue in the
additional information section.
-->
Rework of https://github.com/home-assistant/core/pull/109030 which introduces a new device based MQTT discovery schema.
The original PR was reverted during the beta of HA Core 2024.6.0. With PR and branch we aim to test and optimize large MQTT setups that run via MQTT discovery.
The new device based discovery payload allows multiple `mqtt` components to set up with one configuration payload.
### Context
The current MQTT discovery model only allows to setup per component, if a device has multiple entities to published, the device context, availability and origin information needs to be duplicated.
This PR [reduces IO overhead in the paho client on processing discovery packages](https://github.com/home-assistant/core/pull/109030#issuecomment-2145649738): The baseline was 10000 packets, with device discovery it 6250 packet reads so a 37.5% reduction in I/O. Processing less I/O reduced the paho client run time by ~18%. The grouped discovery reduced the run time in the HA client by ~12%.
This PR adds a way to discover all components for a device with one discovery. The device, availability and origin mapping is only submitted once. Also the following options are allowed in the shared context:
- `state_topic`
- `command_topic`
- `encoding`
- `qos`
Except for `device` and `origin` options, supported shared options can be overridden in the component config.
Discovery updates and removal is fully supported.
The device based discovery coexists with the component based discovery, so no deprecation is planned for discovery with the single component schema to guarantee older devices keep working.
> Note that some code and schemas that are related to device, availability and origin validation was moved to a new module `schemas.py`.
### Example
The example below is of a device based auto discovery that supplies a `sensor`, a `binary_sensor` and a `tag`.
Discovery topic:
```text
homeassistant/device/test123/config
```
Payload:
```json
{
"dev": {
"identifiers": [
"mysensors123"
]
},
"avty": {
"topic": "avty-topic"
},
"cmps": {
"bins1": {
"p": "binary_sensor",
"unique_id": "bins1_key",
"name": "Beer",
"state_topic": "test-topic"
},
"sens1": {
"p": "sensor",
"unique_id": "sens1_key",
"name": "Milk",
"state_topic": "test-topic"
},
"tag1": {
"p": "tag",
"topic": "foobar/tags/tag1/see"
}
},
"o": {
"name": "My org",
"sw": "bla2mqtt"
},
"qos": 0
}
```
Note that a required `platform` option is added to identify the component platform. The keys under the `components` (abbreviated `cmp`), are treated as `object_id` and are used to create a unique device `discovery_id`. In this case the `discover_id's` are `test123 bins1` and `test123 sens1`.
For entity components defined in the device based configuration payload the `unique_id` is required.
```text
homeassistant/device/node123/test123/config
```
... this will become part of the `object_id`.
The `discovery_id` 's become `test123 node123 bins1` and `test123 node123 sens1`.
### Migration from single topic to device based discovery
To allow a smooth migration from single topic discovery to device based discovery, the single platform platform discovery configs must have a `unique_id` and `device` config. Note that migration is only supported if the entities in the device based payload have the same `unique_id` and `device` identifier.
To allow a take over via the device based discovery all included items must be made ready for migration first by publishing the `{"migrate_discovery": true }` payload to the existing platform based discovery topics. This will allow migration and clearing the migrated discovery topics without removing the registry settings with possible customization's. Migration is supported in both directions. When a rollback is intended `{"migrate_discovery": true }` is to be published to the device based discovery topic, and then included components can be migrated back to single component discovery topics.
#### Example (device automation and 2 sensors):
##### Discovery topic single: `homeassistant/device_automation/0AFFD2/bla1/config`
Discovery id: `0AFFD2 bla1`
Discovery payload single:
```json
{
"automation_type": "trigger",
"device": {
"identifiers": [
"0AFFD2"
]
},
"o": {
"name": "foobar"
},
"payload": "short_press",
"topic": "foobar/triggers/button1",
"type": "button_short_press",
"subtype": "button_1"
}
```
##### Discovery topic single: `homeassistant/sensor/0AFFD2/bla2/config`
Discovery id: `0AFFD2 bla2`
Discovery payload single:
```json
{
"automation_type": "trigger",
"device": {
"identifiers": [
"0AFFD2"
]
},
"o": {
"name": "foobar"
},
"state_topic": "foobar/bla2/state",
"unique_id": "bla2"
}
```
#### Migrate to a one device config discovery payload:
First enable discovery migration:
Publish:
```json
{"migrate_discovery": true }
```
To the topics:
- `homeassistant/device_automation/0AFFD2/bla1/config`
- `homeassistant/sensor/0AFFD2/bla2/config`
Now the new device based discovery payload must be send to migrate the discovery topic.
Discovery topic device: `homeassistant/device/0AFFD2/config`
Discovery id: `0AFFD2`
Discovery payload device:
```json
{
"device": {
"identifiers": [
"0AFFD2"
]
},
"o": {
"name": "foobar"
},
"cmps": {
"bla1": {
"automation_type": "trigger",
"payload": "short_press",
"topic": "foobar/triggers/button1",
"type": "button_short_press",
"subtype": "button_1",
"p": "device_automation"
},
"bla2": {
"state_topic": "foobar/bla2/state",
"p": "sensor",
"unique_id": "bla2"
}
},
}
```
Note that in this example the `node_id` becomes the `object_id` in the device based discovery topic, and the `object_id` from the single component is placed as a key under the `components` (`cmp`) key. The `device` part is now shared and all entities have a `unique_id`.
The source integration, responsible for the entities and the migration, is also responsible for cleaning up the old discovery topics after the migration has been completed. This the source integration should publish empty payloads to the old configuration topics, if these topic are retained. Note that retained configs must be cleared with a retained empty payload.
In the example a retained empty payload is to be published to the topics:
- `homeassistant/device_automation/0AFFD2/bla1/config`
- `homeassistant/sensor/0AFFD2/bla2/config`
> Each migration step is logged as an info message to the HA Core logs. This allow you to check that each step was executed correctly.
#### Removing the device or components
Updating works the same as with single component based discovery.
To remove all components at once a (retained) empty payload must be send to the discovery topic `homeassistant/device/0AFFD2/config`.
To remove a component from an existing example configuration, only the `platform` key must be specified.
To remove the `device_automation` component from the the example discover publish:
```json
{
"device": {
"identifiers": [
"0AFFD2"
]
},
"o": {
"name": "foobar"
},
"cmps": {
"bla1": {
"p": "device_automation"
},
"bla2": {
"state_topic": "foobar/bla2/state",
"p": "sensor",
"unique_id": "bla2"
}
},
}
```
## Type of change
<!--
What type of change does your PR introduce to Home Assistant?
NOTE: Please, check only 1! box!
If your PR requires multiple boxes to be checked, you'll most likely need to
split it into multiple PRs. This makes things easier and faster to code review.
-->
- [ ] Dependency upgrade
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New integration (thank you!)
- [x] New feature (which adds functionality to an existing integration)
- [ ] Deprecation (breaking change to happen in the future)
- [ ] Breaking change (fix/feature causing existing functionality to break)
- [ ] Code quality improvements to existing code or addition of tests
## Additional information
<!--
Details are important, and help maintainers processing your PR.
Please be sure to fill out additional details, if applicable.
-->
- This PR fixes or closes issue: fixes #
- This PR is related to issue:
- Link to documentation pull request: https://github.com/home-assistant/home-assistant.io/pull/33072
## Checklist
<!--
Put an `x` in the boxes that apply. You can also fill these out after
creating the PR. If you're unsure about any of them, don't hesitate to ask.
We're here to help! This is simply a reminder of what we are going to look
for before merging your code.
-->
- [x] The code change is tested and works locally.
- [x] Local tests pass. **Your PR cannot be merged unless tests pass**
- [x] There is no commented out code in this PR.
- [x] I have followed the [development checklist][dev-checklist]
- [x] I have followed the [perfect PR recommendations][perfect-pr]
- [x] The code has been formatted using Ruff (`ruff format homeassistant tests`)
- [x] Tests have been added to verify that the new code works.
If user exposed functionality or configuration variables are added/changed:
- [ ] Documentation added/updated for [www.home-assistant.io][docs-repository]
If the code communicates with devices, web services, or third-party tools:
- [ ] The [manifest file][manifest-docs] has all fields filled out correctly.
Updated and included derived files by running: `python3 -m script.hassfest`.
- [ ] New or updated dependencies have been added to `requirements_all.txt`.
Updated by running `python3 -m script.gen_requirements_all`.
- [ ] For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.
- [ ] Untested files have been added to `.coveragerc`.
<!--
This project is very active and we have a high turnover of pull requests.
Unfortunately, the number of incoming pull requests is higher than what our
reviewers can review and merge so there is a long backlog of pull requests
waiting for review. You can help here!
By reviewing another pull request, you will help raise the code quality of
that pull request and the final review will be faster. This way the general
pace of pull request reviews will go up and your wait time will go down.
When picking a pull request to review, try to choose one that hasn't yet
been reviewed.
Thanks for helping out!
-->
To help with the load of incoming pull requests:
- [ ] I have reviewed two other [open pull requests][prs] in this repository.
[prs]: https://github.com/home-assistant/core/pulls?q=is%3Aopen+is%3Apr+-author%3A%40me+-draft%3Atrue+-label%3Awaiting-for-upstream+sort%3Acreated-desc+review%3Anone+-status%3Afailure
<!--
Thank you for contributing <3
Below, some useful links you could explore:
-->
[dev-checklist]: https://developers.home-assistant.io/docs/development_checklist/
[manifest-docs]: https://developers.home-assistant.io/docs/creating_integration_manifest/
[quality-scale]: https://developers.home-assistant.io/docs/integration_quality_scale_index/
[docs-repository]: https://github.com/home-assistant/home-assistant.io
[perfect-pr]: https://developers.home-assistant.io/docs/review-process/#creating-the-perfect-pr
).
Por favor
parce celle existante est nulle !