How to Configure Source-Based Routing on Linux

How to Configure Source-Based Routing on Linux

Source-based routing (also called policy routing) allows your Breeze instance to route packets based on their source address rather than just the destination. This is essential when your server has multiple network interfaces, multiple ISPs, or when you need different traffic paths for different services.

Why Source-Based Routing

By default, Linux routes all traffic through a single default gateway regardless of which interface received the packet. This causes problems in multi-homed setups where reply packets might exit through a different interface than the one that received the original request, resulting in dropped connections due to asymmetric routing.

Prerequisites

Ensure the iproute2 package is installed (it is on virtually all modern Linux distributions). You also need the ip rule and ip route commands, which support multiple routing tables.

Setting Up Routing Tables

First, define custom routing table names in /etc/iproute2/rt_tables:

echo "100 isp1" | sudo tee -a /etc/iproute2/rt_tables
echo "200 isp2" | sudo tee -a /etc/iproute2/rt_tables

Configuring Multi-ISP Source Routing

Assume your Breeze has two interfaces:

• eth0: 203.0.113.10/24 via gateway 203.0.113.1 (ISP 1)
• eth1: 198.51.100.10/24 via gateway 198.51.100.1 (ISP 2)

Configure the routing tables:

# ISP 1 routing table
sudo ip route add 203.0.113.0/24 dev eth0 src 203.0.113.10 table isp1
sudo ip route add default via 203.0.113.1 table isp1

# ISP 2 routing table
sudo ip route add 198.51.100.0/24 dev eth1 src 198.51.100.10 table isp2
sudo ip route add default via 198.51.100.1 table isp2

Adding Policy Rules

Create rules that direct traffic to the appropriate routing table based on source address:

# Traffic from ISP 1 IP uses ISP 1 table
sudo ip rule add from 203.0.113.10 table isp1 priority 100

# Traffic from ISP 2 IP uses ISP 2 table
sudo ip rule add from 198.51.100.10 table isp2 priority 200

Verify the rules:

ip rule show
ip route show table isp1
ip route show table isp2

Service-Based Routing with Marks

Route specific services through different ISPs using iptables marks and routing rules:

# Mark web server traffic to use ISP 2
sudo iptables -t mangle -A OUTPUT -p tcp --sport 443 -j MARK --set-mark 2
sudo ip rule add fwmark 2 table isp2 priority 150

Making Routes Persistent

On Ubuntu with Netplan, add routing policies to your configuration:

network:
  version: 2
  ethernets:
    eth0:
      addresses: [203.0.113.10/24]
      routes:
        - to: default
          via: 203.0.113.1
          table: 100
      routing-policy:
        - from: 203.0.113.10
          table: 100
          priority: 100
    eth1:
      addresses: [198.51.100.10/24]
      routes:
        - to: default
          via: 198.51.100.1
          table: 200
      routing-policy:
        - from: 198.51.100.10
          table: 200
          priority: 200

Troubleshooting

Use the ip route get command to verify which route a packet will take:

ip route get 8.8.8.8 from 203.0.113.10
ip route get 8.8.8.8 from 198.51.100.10

If replies are being dropped, check for reverse path filtering:

sudo sysctl -w net.ipv4.conf.all.rp_filter=2
sudo sysctl -w net.ipv4.conf.eth0.rp_filter=2
sudo sysctl -w net.ipv4.conf.eth1.rp_filter=2

Setting rp_filter to 2 enables loose mode, which is more permissive and necessary for asymmetric routing scenarios on your Breeze.