"Interface route" cannot be modified and is set to 0 metric

[ArcherSeven]

Bug Report

Description

Talos creates both a default route with metric 1024 and what I will naively call an "interface route" set to metric of 0 for each configured interface.

If multiple interfaces connect to the same subnet, it is possible to modify the default/gateway route, but not the interface route, and since the interface route is set to metric 0, it cannot be overwritten.

Example config (where an unconfigured interface is left to use DHCP)

    network:
        interfaces:
            - addresses:
                - 172.22.1.21/24
              deviceSelector:
                busPath: 0000:02:00.0
              routes:
                - gateway: 172.22.1.1
                  metric: 2048
                  network: 0.0.0.0/0
                - network: 172.22.1.0/24
                  metric: 2048

Resulting routes:

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         172.22.1.1      0.0.0.0         UG    1024   0        0 eno1
0.0.0.0         172.22.1.1      0.0.0.0         UG    2048   0        0 enp2s0
10.244.1.0      0.0.0.0         255.255.255.0   U     0      0        0 cni0
172.22.1.0      0.0.0.0         255.255.255.0   U     0      0        0 eno1
172.22.1.0      0.0.0.0         255.255.255.0   U     0      0        0 enp2s0
172.22.1.0      0.0.0.0         255.255.255.0   U     2048   0        0 enp2s0

This means that the actual order of preference for the routes is based on order of creation, so in the above situation, when the router reboots, the routes end up as:

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         172.22.1.1      0.0.0.0         UG    1024   0        0 eno1
0.0.0.0         172.22.1.1      0.0.0.0         UG    2048   0        0 enp2s0
172.22.1.0      0.0.0.0         255.255.255.0   U     0      0        0 enp2s0
172.22.1.0      0.0.0.0         255.255.255.0   U     0      0        0 eno1
172.22.1.0      0.0.0.0         255.255.255.0   U     2048   0        0 enp2s0

At which point no destination is pingable (since enp2s0 is configured, but the interface isn't actually plugged in)

We need to either be able to change these "interface" routes, or have them set to a higher metric so we can override them.

Environment

• Talos version: v1.9.2 (other versions 1.7+ also tested)
• Kubernetes version: v1.27.15
• Platform: Baremetal x86_64

[smira]

Talos doesn't create these "interface" routes, they are created by Linux automatically by Linux as the address is assigned, but we'll look into this.

[ArcherSeven]

As a point of comparison, Manjaro Linux with Network Manager, configures the same routes with a metric of 100, which allows it to be overwritten if needed.

❯ ip route
default via 192.168.12.1 dev eno2 proto dhcp src 192.168.12.10 metric 100 
192.168.12.0/23 dev eno2 proto kernel scope link src 192.168.12.10 metric 100 

[smira]

Linux supports this via IFA_RT_PRIORITY in the RTM_NEWADDR message, we need to add support for it to rtnetlink library first

[smira]

In Talos >= v1.10.0-beta.1 the DHCP with configured route metric will use that metric for prefix routes generated for addresses.

For static addresses, this support will be added in 1.11+ as we work on new network machine configuration.