Part 1: Smart Home Matter and Thread Deep Dive

The modern Smart Home has a host of new standards to enable what is supposed to be a seamless experience across multiple ecosystems. These new standards include Matter and Thread, both of which the likes of Amazon, Google, Apple, Samsung, and Home Assistant support to various degrees.

There’s A LOT of confusion and mystery around these two new standards, and troubleshooting any issues can be hard. In this series of posts I’ll do a deep dive in parts of Matter, Thread, IPv6 addressing and more with the goal of being an educational resource on your smart home journey.

Part 1: Smart Home Matter and Thread Deep Dive

What is Matter?
What is Thread?
What can you deploy today?
The Thread Network State
Thread IPv6 Addressing
What is Multicast?
Discovering Your Matter Devices with mDNS
Decoding Home Assistant IPv6 Addresses

Part 2: Smart Home Matter and Thread Deep Dive

Exploring IPv6 Routing Details
Home Assistant Matter IPv6 Routing Failover

Part 3: Smart Home Matter and Thread Deep Dive

Home Assistant Deployment Model for Matter
Home Assistant Matter Controller Logs
Proxmox IPv6 Tips
mDNS Networking Tips
The Sad State of Switch/Router Firmware
The Sad State of the Linux Kernel IPv6 Routing

Yes that is a lot! But this will be a fun journey and you should learn a lot. Even if you aren’t using Home Assistant or Apple TV smart hubs, all of this knowledge will apply to Matter and Thread in general so keep reading.

November 28, 2023 Update: I modified a couple of sections in Part 1 to better clarify Thread border router options. I also clarified the state for Home Assistant users. I added a new section called “What can you deploy today?”.

November 13, 2023 Update: Updated the Thread border router section and recommendations. Google Nest and Eero routers now support Thread v1.3 with TREL, joining Apple. That makes all three border routers solid options.

October 29, 2023 Update: I added two sections to Part 3 covering the sad state of Switch firmware, and the sad state of Linux IPv6 routing. Most of the information came directly from Home Assistant developers for Matter and Thread. Lots of good information!

What is Matter?

Matter is an open-source, unified IP-based connectivity protocol built for smart home devices, backed by major companies like Apple, Google, Amazon, and the CSA (Connectivity Standards Alliance). Matter aims to increase compatibility among smart home devices and increase the security of these devices. Key features of Matter include:

Local Unified IP-based connectivity: Devices from different manufacturers can communicate directly with each other (without the cloud–all locally) using this standard IPv6-based protocol over common networking technologies like Ethernet, Wi-Fi and Thread. It does NOT support z-wave or Zigbee. Bluetooth is ONLY used for commissioning a device, not for ongoing communications.
Security: Matter focuses heavily on security, requiring device-to-device encrypted communications. This includes secure setup codes to prevent unauthorized access. Matter security is superior to z-wave security, as Matter requires a minimum (good) security baseline, device attestation, and always uses encrypted communications.
Ease of use: Matter devices use a simple setup process, enabling consumers to easily add and control devices within a smart home environment, across manufacturers. Apple donated much of the Homekit protocol, which was used as the basis for Matter commissioning.
Compatibility: Matter is designed to support a wide range of device types, from lighting and electrical products to smart home controls, access control, security, and more. The Matter v1.2 spec adds a number of new device types and is now public. You can read the Matter 1.2 press release.

Bottom Line: Matter is a SOFTWARE control plane protocol (NOT a networking protocol) for your smart home. It’s built on IPv6 and is independent of your underlying network protocols, like Thread or WiFi. Apple, Google, Amazon, Samsung, Home Assistant, etc. all have Matter controllers. Matter devices, unlike z-wave or Zigbee, must support multiple simultaneous controllers. So, for example, Home Assistant and Apple Home can both directly control a Matter device and the device’s state is reflected in real time to both controllers. This cross-vendor multi-controller support is new in the smart home space and very exciting.

What is Thread?

Thread is an IPv6-based, low-power, secure, and interoperable radio frequency mesh networking technology specifically designed for communicating with and among devices within a smart home environment. It’s designed to overcome some limitations of older technologies to provide a more reliable wireless communication protocol especially for home automation. Thread has been around for several years. Apple home hubs have used it for some HomeKit devices.

Key features of Thread include:

Reliable – Thread networks are self-healing. It supports mesh networking, which means each device can act as a node that can relay data for other devices. The more devices, the stronger the network.
Standards based – It’s IP-based (IPv6), facilitating direct addressability of devices on the network from the Internet. Thread runs on 802.15.4 radio hardware, the same protocol used by Zigbee. Some newer 802.15.4 radios can be flashed to support Thread, and some can even do Zigbee and Thread at the same time.
Secure – End-to-end secure and encrypted communications are a baseline requirement. This is different from Z-wave where security is optional and adds additional overhead on your Z-wave network.
Scalable – Thread networks are designed to scale to hundreds of devices.
Power Efficiency – Low-power thread device can be battery powered and can operate for years.

A Thread Border Router (TBR) is a device that provides connectivity from the Thread network, which is a low-power wireless device network, to other more traditional networks, typically IP-based networks like WiFi or Ethernet. A TBR has several important functions:

Gateway to the Internet: It connects Thread devices to your local LAN, and by extension the Internet. The TBR acts as the bridge between these two kinds of networks, allowing Thread devices to access Internet-based services.
Network Configuration and Management: A TBR controls the formation of the Thread network, determining which devices can join the network, enforcing security policies, and routing data between devices.
Mesh Network Formation: As part of the Thread network, a TBR can also help to form and extend the mesh network topology, relaying communications between Thread devices that can’t directly communicate due to distance or obstructions.

Some smart home hubs, such as Apple TVs on the latest tvOS release support TREL. TREL allows any Thread border router that has other IP-based links (e.g. WiFi or Ethernet) to incorporate it into the Thread mesh topology. This allows the Thread network to leverage the benefits of the other links (e.g. higher throughout, capacity, better coverage etc.) and keep it more connected. It also can help reduce traffic over the Thread radios by allowing border routers to talk to each other over their higher capacity WiFi/Ethernet links.

The latest version of Thread is v1.3 and it was released to the public on February 27, 2023. Smart home hub manufacturers such as Apple, Google, Amazon and Samsung often include Thread radios in their devices. You can read more about Thread here.

What can you deploy today?

Despite some of the messiness of Thread and Matter in their current state, today you can build a rock solid and very responsive smart home with Thread and Matter components. Do all device types come in Thread/Matter variants? Absolutely not. Thus, it’s unlikely you can exclusively use Thread/Matter devices. You can easily use other protocols in your smart home for other devices, such as Zigbee or even z-wave. A great feature of Home Assistant is that you can use multiple smart home protocols, pretty much seamlessly. If purchasing a new IoT device today, would first consider the thread/matter version if it exists. If so, go that route. If not, then pick a fallback protocol like Zigbee.

Follow this simple recipe for the best Thread and Matter experience:

Use a Thread border router from Apple or Google. $99 USD entry point.
Deploy Home Assistant OS (HAOS) – Not Docker, not supervised.
Ensure your network gear doesn’t have broken IPv6 multicast support.
Ensure all of your IoT devices and ALL smart hubs/controllers, like Home Assistant, are on the same VLAN. The VLAN can be isolated, with firewalls ports open into the IoT VLAN as needed.

Each of these points are covered in detail in this series. Matter enabled Thread devices should have a nearly instant response time in all of your Matter controllers, like Home Assistant. It should not be laggy or unreliable.

There are some pain points with Thread and Matter today, so it’s not all roses or perfection. Solutions are still maturing. But I have over 20 Thread/Matter devices on my network with Home Assistant and Apple Home. Response times are nearly instant and the network is highly reliable. But it took a bit of work to get there (more on that in Part 3). This series will help you avoid the pitfalls I ran into and deploy a highly reliable and responsive smart home from the get go.

The Thread Network State

The best advice is to pick a single vendor (e.g. Google, Apple, etc.) and always commission your new Thread devices to that Thread Border router. This ensures that all of your Thread devices are on the same Thread network. This is because Thread credential sharing, which is coming, isn’t yet ubiquitous so “islands” of Thread networks can exist today if you aren’t careful. You can commission Thread devices to different Thread border router vendors in your house and it will work, but that’s just messy and complicates matters for no good reason.

Think of Thread as a WiFi network and keep all devices on the same ‘SSID’. In your home you don’t mix and match WiFi APs from different vendors, as that creates a mess. You don’t get one WAP from Netgear, another from Eero, and another from Asus. If you want a mesh WiFi network you invest in ONE of the many WAP ecosystems, like Orbi or Eero. Same rule applies to Thread, for now (more on that below).

Your Matter smart home hub controllers (such as Home Assistant) can all talk to that one unified Thread network via any IPv6 Thread border router so everything can seamlessly communicate. So it’s mostly irrelevant to Home Assistant users that the Thread RF communications is handled by a separate device made by Google or Apple. Home Assistant doesn’t care that your WiFi is provided by Eero, for example, and that’s the way it should be.

The Thread group wants to eventually publish a standard for Thread credential sharing. Thread credential sharing will propel Thread past WiFi in terms of vendor interop, and allow Thread Border routers from Apple, Google, Amazon, HA/Skyconnect, etc. to all use the same Thread network. Parts of that reality are beginning to emerge today.

However, right now it’s up to each vendor to define their own APIs for Thread credential sharing. As of October 2023, not all vendors have published Thread credential APIs. Google and Apple have released Thread credential APIs, but to my knowledge nobody else has. But this really isn’t a problem, as home WiFi users are used to relying on a single WAP brand. It’s just the same rule (right now) for Thread.

As of November 2023, the best Thread border routers are from either Apple or Google. For Apple you can use certain Apple TV 4Ks, or the latest HomePod or HomePod Minis. Check out this link for more Apple information. For Google you can choose from Nest Hub (2nd Gen, $99 USD), Nest Hub Max, Nest WiFi Pro (WiFi 6E), or Nest Wifi. Check out this link for more Google information. The cheapest Google entry point for a Thead border router is $99, using the Nest Hub 2nd Gen. The Apple HomePod mini is also $99 USD.

The latest Eero WiFi router firmware now supports Thread v1.3 and TREL as well. This means Apple, Google and Eero all use the latest Thread version with TREL, which is great. As of November 2023 Amazon Echo and Smarthings are on an older Thead version without TREL support. I would actively avoid them as Thread Border routers, until they support Thread v1.3 with TREL.

Home Assistant Users: As mentioned above, Thread border routers handle the RF communications for Thread devices. This is just like a generic WiFi access point for your WiFi network. So don’t get hung up on using a third-party Thread Border router. Apple and Google have very strong, and affordable, solutions in this space. There’s almost no logical reason to avoid them.

In the long term (I’d hope by late 2024), Home Assistant can be a production ready Thread border router so a third party router would not be required. However, the current development state (November 2023) of the HA Thread border router is such that end users should NOT use it. HA developers are working hard on this feature, so no doubt it will materialize. Bottom line, don’t purchase a Skyconnect anytime soon and expect to use it with Thread as a border router.

The Home Assistant team is actively working on Thread credential sharing with commercial border router vendors, like Apple and Google. Once that is in place and the HA Thread border router code matures, then you will be able to add a Skyconnect to your existing Thread network (managed by Apple, Google, etc.) and seamlessly integrate.

In addition, now that Apple, Google and other vendors are offering Thread Border routers which support TREL, adding Skyconnect or Home Assistant Yellow negates any advantages of TREL. TREL support will come to Home Assistant, but that’s not currently supported. The Matter controller in Home Assistant works very well with the Thread border routers in other ecosystems without needing its own local Thread radio.

Thread IPv6 Addressing

Understanding Thread device IPv6 addressing is important. As we dive deeper into Thread this will become apparent. Thread device will have multiple IPv6 addresses assigned to them. One category of addresses are called Endpoint Identifiers (EIDs), which identify a unique Thread interface within a Thread network partition. EIDs are independent of Thread network topology. A few Thread unicast EIDs include:

Link-Local Address (LLA): An EID that identifies a Thread interface reachable by a single radio transmission. Always has a prefix of fe80::/16.
Mesh-Local EID (ML-EID): An EID that identifies a Thread interface, independent of network topology. Used to reach a Thread interface within the same Thread partition. Also called a Unique Local Address (ULA). Always has a prefix fd00::/8.
Global Unicast Address (GUA): An EID that identifies a Thread interface on a global scope, beyond a Thread network. These prefixes can vary depending on how your smart home hub wants to do addressing. This is the address which the rest of your home LAN sees for your thread devices.

Later in this post we will see how to discover those IPv6 addresses for each Matter enabled Thread device in Home Assistant. But just to give you a sneak peek, here’s an example for a Thread device:

What is Multicast?

Understanding multicast is key to understanding Matter. Matter relies on IPv6 mDNS (multicast DNS) to advertise services. If multicast is somehow broken on your network, you will likely have BIG random Matter headaches.

Multicast is a method used in the Internet Protocol (IP) to transmit data packets to multiple destinations simultaneously. It uses efficient network routing and minimizes bandwidth usage since a single packet can be received by multiple recipients, rather than sending individual packets to each recipient. Multicast uses UDP. UDP is connectionless and does not reorder packets or request retransmission of lost packets, thus making it suitable for multicast delivery where delivery speed is more important than reliability.

IGMP and MLD snooping are techniques that network devices can use to conserve bandwidth by reducing multicast traffic. Both IGMP and MLD snooping improve the performance of Ethernet networks by limiting the extent of the multicast domain and decreasing unnecessary traffic. The Internet Group Management Protocol (IGMP) is a communications protocol used by hosts and adjacent routers on an IPv4 network to establish multicast group memberships. It allows a computer to tell its local router that it wants to receive multicast traffic for a specific group.

Multicast Listener Discovery (MLD) is a component of the Internet Protocol version 6 (IPv6) suite. MLD is derived from IGMP; MLD version 1 (MLDv1) is equivalent to IGMPv2, and MLD version 2 (MLDv2) is equivalent to IGMPv3. MLD is a subprotocol of Internet Control Message Protocol version 6 (ICMPv6), and MLD messages are a subset of ICMPv6 messages.

Apple Home Hub Setup

On my network I have two Apple TV 4Ks that have both Thread and Ethernet. This means I have two Thread border routers on my network. Both run tvOS 17.1. Apple designates one home hub as “Connected” and all others as “Standby“. The “Connected” home hub is the one which advertises the Matter controller services, although both are active v1.3 Thread Border routers with TREL support. But let’s not get ahead of ourselves..we are only getting started.

Discovering Your Matter Devices with mDNS

If you have an iPhone or Mac you can discover your mDNS enabled devices, including Matter, by using the app Flame. Simply install Flame on your iPhone or Mac, launch it, and wait a few seconds for the list to populate. Depending on how many devices you have on your network, you may see a long list that includes printers, NAS, network switches, receivers, etc. There is another iPhone/Mac app called Discovery – DNS-SD Browser that presents the same mDNS data but in a different format. However, it has no data export feature. But its tree view can come in handy.

To find all Matter enabled devices in Flame you can simply type Matter in the search box. To easily view all of the data, you can use the share icon in the upper right corner and airdrop the JSON file to your Mac from your iPhone. Or if you are running it on your Mac, it can directly save a JSON file. I suggest opening the JSON file in a code editor, such as Microsoft Visual Code Studio.

In the Flame output you can see the IPv6 address of each device and the 16 character hostname. This 16 character hostname is the hex code version of the randomly generated extended MAC address of the thread radio after commissioning is complete.

In my case the global IPv6 address space that all of my Matter devices are using is prefixed with fd5d:443b:99ef. This is known as the off-mesh routable (OMR) prefix. This prefix is not universal, so your Thread network will likely have a different prefix. Thread devices are assigned two more IPv6 addresses: link-local and mesh-local. Flame will not show you the other two IPv6 address of a Matter device, but as mentioned before, Home Assistant can provide this information.

In the figure below if you look at the two Apple TVs listed on the left, you see one has 7 services and the other has 8. The one with 8 is the “Connected” Apple TV which is your active Matter controller. The other Apple TV doesn’t broadcast the “Matter” service as it’s in standby. However, both Apple TVs do broadcast as a thread border router with TREL support (_meshcorp._udp and _trel._udp, respectively). So for IPv6 packets going to your Thread devices, both Apple TVs can route packets in an active/active fashion. If the network connection to the Loft ATV gets interrupted, the living room ATV will take over as the Matter controller and advertise the _matter._tcp service.

If you take a look at the Flame JSON output for an Apple TV we can see some interesting fields. Super nerdy so won’t cover here, but take a gander.

Decoding Home Assistant IPv6 Addresses

As of HA 2023.10, finding the IPv6 addresses of your Matter devices in Home Assistant needs a bit of work. Unfortunately this is not yet surfaced in the Home Assistant UI, but it is coming pretty soon. But in the mean time, lets use the manual method. In the Matter diagnostics file for a device the IPv6 addresses are base-64 encoded and we can easily decode them. Let’s get the device’s JSON diagnostics file:

Settings -> Devices & services -> Matter -> Click on xx Devices -> Click on any device -> DOWNLOAD DIAGNOSTICS.

Open the JSON file and search for IPv6Add and you should see a block of base-64 encoded strings between quotes that start with a / and end with ==. In my case I see three blocks of text.

To decode those base-64 strings into a hex string formatted like an IPv6 address, you can use the following Python script (courtesy of ChatGPT). Just replace the base-64 strings on lines 4-6 with your strings. Run the script, and it should output a nicely formatted IPv6 address. If you don’t mind manually putting in colons, you can use this website to do the base64 to hex conversion: Base64Guru. The Python code can easily be run without any fuss with VS Code (free).

				
					import base64

base64_strings = [
"/oAAAAAAAAAUwgnlaZDaPg==",
"/RysaSl2AADWZBzKf93Rtg==",
"/V1EO5nvAAAEYQB6xk26Yg=="
]

for base64_str in base64_strings:
    # Decode the Base64 string to bytes
    bytes_arr = base64.b64decode(base64_str)

    # Convert the bytes to a Hex string
    hex_str = ''.join(f'{byte:02x}' for byte in bytes_arr)

    # arrange the Hex code into the IPv6 format
    ipv6_str = ':'.join(hex_str[i:i + 4] for i in range(0, len(hex_str), 4)) 

    print(ipv6_str)

The image below is the Python script output of the Matter device IPv6 addresses. You will recognize the fdf5:44eb:99ef: prefix is the one shown in Flame. So we know this is the global IPv6 address that we can use to ping the Thread device.

Take note that sometimes Apple assigns a new IPv6 address to Thread devices. So if you ping a device and it’s not responding, double check you have the right IPv6 address..it may have changed. While I was doing some Apple TV routing failover tvOS 17 decided to get a brand new Thread IPv6 GBA, so all devices had a brand new IPv6 prefix. As of HAOS 2023.10, the diagnostics file does NOT get regularly updated with the new IPv6 address. So until that is addressed (coming soon), use my next method below.

There is an alternate way to also find the IPv6 addresses of your Home Assistant Thread devices. The best UI for this is the Discovery DNS-SD browser I mentioned earlier. In the long list of Matter devices it will show you, there will be a block of addresses where the second half of the name is all zeros except for a couple of digits at the end (depending on the number of devices you have). Here, the hex encoded numbers correspond to the “node ID” (in decimal) that appears in the Home Assistant Matter logs and diagnostics file. Although hard to see, but you can expand the mDNS entry and see the faint IPv6 address.

For example, hex 33 (the last entry in the red boxed area), is 51 decimal. If I dump all the diagnostics files for all of my Matter devices, I can see node 51 is one of my window shades.

Summary

In this post we covered what is Matter, Thread Networks, Thread IPv6 addressing, multicast, discovering your mDNS services, and how to decode Home Assistant IPv6 addresses. In Part 2 I do a deep dive on Matter IPv6 border router failover with two Apple TVs and Home Assistant. Check it out for some heavy networking knowledge.