Is it possible to set up Policy Based Routing with NetworkManager in RHEL?

Solution Verified - Updated

Environment

  • Red Hat Enterprise Linux 10
  • Red Hat Enterprise Linux 9
  • Red Hat Enterprise Linux 8
  • Red Hat Enterprise Linux 7
  • NetworkManager
  • Policy Based Routing

Issue

  • Is it possible to set up Policy Based Routing with NetworkManager in RHEL?

Resolution

For NetworkManager 1.18.0 or later

Set up the routes for the additional routing table using the connection's ipv4.routes property

  • Assign a unique routing table number (table ID) and add routes using this table number
  • This number should be in the range 1 to 252 inclusive
  • In the following examples, the connection named enp7s0 is set to use table number 10
  • Note the plus (+) used to append to the property in these examples
### Example of a route for the locally connected subnet
# nmcli con mod enp7s0 +ipv4.routes "192.168.0.0/24 table=10"

### Example of a route to send a remote subnet (10.0.0.0/24) via a specific gateway (192.168.0.254)
# nmcli con mod enp7s0 +ipv4.routes "10.0.0.0/24 192.168.0.254 table=10"

The default route syntax differs on RHEL7 and later, as the earlier NM does not support 0.0.0.0/0 so two /1 routes are used instead:

### RHEL 7 Example of a default route via a specific gateway (192.168.0.1)
# nmcli con mod enp7s0 +ipv4.routes "0.0.0.0/1 192.168.0.1 table=10, 128.0.0.0/1 192.168.0.1 table=10"

This limitation does not apply on RHEL8 and later, so 0.0.0.0/0 can be used directly:

### RHEL 8+ Example of a default route via a specific gateway (192.168.0.1)
# nmcli con mod enp7s0 +ipv4.routes "0.0.0.0/0 192.168.0.1 table=10"

Set up the policy rules using the connection's `ipv4.routing-rules' property.

  • In the following examples, the most common types of routes are defined via three separate commands for an interface named enp7s0 with the IP address 192.168.0.2/24
  • Each rule requires a unique priority in the range 1 to 32765 inclusive, lower priority takes precedence
  • Note the plus (+) used to append to the property in these examples
### Anything which comes in this interface, send to table 10
# nmcli con mod enp7s0 ipv4.routing-rules "priority 100 iif enp7s0 table 10"

### Anything from this system's local IP, send to table 10
# nmcli con mod enp7s0 +ipv4.routing-rules "priority 110 from 192.168.0.2 table 10"

### Anything to the local subnet, send to table 10
# nmcli con mod enp7s0 +ipv4.routing-rules "priority 120 to 192.168.0.0/24 table 10"

After modifying all of the connection properties, put the settings into place with the nmcli dev reapply command:

# nmcli dev reapply enp7s0
Connection successfully reapplied to device 'enp7s0'.

Note that the nmcli dev reapply command is for if any routes or rules are added, not removed. Any changes that may involve removing rules and/or routes should be applied with nmcli connection down and nmcli connection up commands, or (if it is not safe to bring a connection down but safe to remove routes and rules live) manually removing the entry in memory with the ip rule or ip route commands:

# nmcli connection down enps70
# nmcli connection up enps70

The syntax for IPv6 is described at:

For NetworkManager versions below 1.18.0

Ensure the NetworkManager-dispatcher-routing-rules package is installed.

  • In RHEL 8 this package is part of the standard repositories.
  • In RHEL 7 this package is in the Optional repository.
### For RHEL 8:
# yum install NetworkManager-dispatcher-routing-rules
    
### For RHEL 7:
# yum install --enablerepo=rhel-7-server-optional-rpms NetworkManager-dispatcher-routing-rules

Configure Policy Based Routing using the legacy network service scripts route & rule file format as described in:

Root Cause

NetworkManager 1.18.0 and later has native support for Policy Based Routing. This does not require the NetworkManager-dispatcher-routing-rules package.

NetworkManager 1.18.0 and later still can use the optional NetworkManager-dispatcher-routing-rules package if the old configuration files format is required.

Diagnostic Steps

Confirm configuration

Once the policy routes are set up and activated, confirm they are in place as expected. The following confirms the example configuration from the above Resolution section.

Confirm the routes and rules are in place:

# nmcli con show enp7s0
....
ipv4.routes:           { ip = 192.168.0.0/24 table=10 };
                       { ip = 10.0.0.0/24, nh = 192.168.0.254 table=10 };
                       { ip = 0.0.0.0/0, nh = 192.168.0.1 table=10 }

ipv4.routing-rules:    priority 100 from 0.0.0.0/0 iif enp7s0 table 10,
                       priority 110 from 192.168.0.2 table 10,
                       priority 120 to 192.168.0.0/24 table 10

Confirm the kernel routing table and IP rules are applied:

# ip route show table 10

default via 192.168.0.1 dev enp7s0 proto static metric 101 
10.0.0.0/24 via 192.168.0.254 dev enp7s0 proto static metric 101
192.168.0.0/24 dev enp7s0 proto kernel scope link src 192.168.0.2 metric 101 
    
# ip rule

100:    from all iif enp7s0 lookup 10 
110:    from 192.168.0.2 lookup 10 
120:    from all to 192.168.0.0/24 lookup 10

Note: ipv4.route-table property

NetworkManager 1.18.0 also has a new ipv4.route-table property controls which routing table all routes specified by the ipv4.routes and ipv4.gateway properties are installed into.

When an ipv4.route-table is set, NetworkManager will not install any route for the connection in the system's main routing table unless explicitly told to.

This is usually the opposite of what most customers want to achieve with policy routing, so this knowledgebase solution does not use the ipv4.route-table property.

To specify a route be installed in a table other than the ipv4.route-table table set an ipv4.routes entry's table property value to some other value. Note 254 is the table number of the main table.

So if using the ipv4.route-table property and a route in the main table is required:

# nmcli con mod enp7s0 +ipv4.routes "172.16.100.0/24 192.168.0.254 table=254"
SBR
Product(s)
Components
Category
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.