Custom Tasmota MQTT discovery format to enable native HA integration

[emontnemery]

Have you looked for this feature in other issues and in the docs?
Yes

Is your feature request related to a problem? Please describe.
Home Assistant's MQTT discovery protocol includes a lot of redundant information which increases Tasmota's code size, and has some crucial limitations e.g. RGBxx lights flickering when turning on from HA.
Also, Home Assistant's MQTT discovery protocol is not easy to add new features or make breaking changes to.

Describe the solution you'd like
Acceptance of a custom MQTT discovery protocol for Tasmota. Work has started, but some feedback is needed.
An estimate is that 2-3kB code can be saved.

Short term, this will have to co-exist with Home Assistant's MQTT discovery format:
Maybe:

• When SetOption19 = 0, send Tasmota's custom MQTT discovery format
• When SetOption19 = 1, send Home Assistant's MQTT discovery format

Longer term, support for Home Assistant's MQTT discovery format can be removed from Tasmota or included as a non default compile-time option if needed by other SW, e.g. for OpenHab support.

Describe alternatives you've considered

1. Build a native HA integration but keep using HA's MQTT discovery protocol.
   Drawback: It will still not be possible to keep compliant to HA's MQTT discovery protocol and also add new features.
2. Don't use any discovery format at all, send MQTT messages to query the Tasmota device about its configuration.
   Drawback: Not all necessary information can (easily) be read by MQTT messages.

Additional context
Work on a Home Assistant native integration is started, with an MVP PR home-assistant/core#39624 which supports:

• Discovery, update and removal of Tasmota devices
• Discovery, update and removal of relays (switch entities in HA)

The planned next steps are:

• Discovery, update and removal of lights
• Discovery, update and removal of switches
• Discovery, update and removal of buttons
• Discovery, update and removal of sensors
• Add support to manage the Tasmota from HA
• Add Tasmota console in HA
• Ideas??

Preliminary discovery message format example
Topic: tasmota/discovery/49A3BC
Payload:

{
  "ip":"192.168.15.10",                             // IP address
  "dn":"Living Room",                               // Device name
  "fn":["Ceiling Lamp", "Floor Lamp"],              // List of friendly names
  "hn":"tasmota_49A3BC-0956",                       // Hostname
  "id":"49A3BC",                                    // ChipID
  "md":"Sonoff Dual",                               // Module
  "of":"Offline",                                   // D_OFFLINE
  "on":"Online",                                    // D_ONLINE
  "st":["OFF","ON","TOGGLE","HOLD"],                // StateText[0..3]
  "bd":"8.4.0.2",                                   // Tasmota SW build version
  "t":"tasmota_49A3BC",                             // Topic
  "ft":"%prefix%/%topic%/",                         // Fulltopic
  "tp":["cmnd","stat","tele"],                      // [SUB_PREFIX, PUB_PREFIX, PUB_PREFIX2]
  "li":[0,0,0,0,0,0,0,0],                           // Lights, 0: disabled, 1: Enabled
  "rl":[0,0,0,0,0,0,0,0],                           // Relays, 0: disabled, 1: relay, 2.. future extension (fan, shutter?)
  "sw":[0,0,0,0,0,0,0,0],                           // Switches, 0: disabled: 1: enabled
  "bt":[0,0,0,0],                                   // Buttons, 0: disabled: 1: enabled
  "so":{"13":0,"17":1,"30":0,"37":1,"68":0,"73":1}, // SetOption needed by HA to map Tasmota devices to HA entities and triggers
  "lt_st":0,                                        // Light subtype
  "se":[0],                                         // Sensors, 0: disabled, 0..xx index in kHAssJsonSensorTypes (??)
  "ver":1                                           // Discovery protocol version, must be 1
}

Screenshots
Add Tasmota integration to HA:

Discovery message received by HA:

(Please, remember to close the issue when the problem has been addressed)

[blakadder]

My first concert is that a user will not be able to manually add entities under a device by sending the autodiscovery payload

See for what I mean:

https://blakadder.com/pir-in-tasmota/

As clunky as it is the current MQTT discovery has a wide range of possibilities

[arendst]

If this is the way HASS discovery can be made less tasmota version dependant it's fine with me.

As suggested pls leave the option to fallback to the current discovery process. In time we can decide to remove support for it and keep the new process only.

[digiblur]

Very cool stuff! I'm all for removing some steps and confusion with users. Not to mention saving code space as well! Well done!

[sfromis]

I came to wonder about a couple of small details with the new discovery.

Assuming that the new tasmota/discovery/<mac>/sensors message published to MQTT has the purpose of allowing the subscribing backend system to setup for what sensors and values can be expected, this seems to imply that the "model" sensor JSON payload should contain all relevant keys, but this may not be the case, even during entirely normal operation. Specifically, the ENERGY object does not contain the "Period" key; consumption within TelePeriod. I suppose that this makes sense, as it does not have a "start of TelePeriod" value to compare with. However, this would mean that a backend system might not be prepared for getting this value in a following message. Example:

tasmota/discovery/CC50E3******/sensors {"sn":{"Time":"2020-10-17T18:56:16.440+02:00","ENERGY":{"TotalStartTime":"2020-01-17T19:18:31","Total":261.244,"Yesterday":1.737,"Today":1.163,"Power":76.200,"ApparentPower":131.265,"ReactivePower":106.900,"Factor":0.58,"Voltage":231.100,"Current":0.568}},"ver":1}

Also, I'm used to all published MQTT messages to be logged in the console with MQT: tags, but this does not happen for the two messages for new discovery. In the nitpicky department, I'm a bit surprised that the MAC is uppercased in the topic, usually it is with lowercase a-f.

[emontnemery]

That's an interesting observation about the energy sensor, and you're correct that HA will need all keys in the initial message. What does the period key do?

The MQTT messages are hidden on purpose to not spam the log.

I don't think HA minds if MAC address is with lower or upper case letters 😁

[sfromis]

The Period field only appears for messages triggered by TelePeriod, and is energy usage in Wh for that interval, with decimals set by WattRes. It does not appear for extra reporting triggered by PowerDelta or Status 10, which seems to be similar to this initial reporting.

Example:
********/tele/SENSOR = {"Time":"2020-10-17T20:06:19.473+02:00","ENERGY":{"TotalStartTime":"2020-01-17T19:18:31","Total":261.428,"Yesterday":1.737,"Today":1.347,"Period":7.020,"Power":84.500,"ApparentPower":135.313,"ReactivePower":105.700,"Factor":0.62,"Voltage":231.700,"Current":0.584}}
For the standard TelePeriod of 5 minutes, the Period value of 7.020 would (if stable over one hour) be 84.240 Wh, pretty close to the Power value of 84.500 W (which is just the momental value at time of reporting).

[effelle]

Discovery /sensors topic grab everything is present on MqttShowSensor(); at boot time after mqtt init.
If something is not there means Tasmota has not initialized a particolar sensor at the time. That's the case of Period, becasue it needs two values to compare and compute.
There isn't much to do in this case @emontnemery , the only thing we could do is forcing the sensor creation directly in Python if a power meter is detected.

[sfromis]

I've also seen at least one case of a sensor not responding initially, but coming online a bit later, meaning that config based on the /sensors topic would be incomplete. I remember a case of a DS18B20 being "late", which can be said to be a issue with the sensor, but his could lead to confused users when it was not discovered backend.

[effelle]

I'll see if I can add a mild delay just to be sure.