Suppose there are two NIC on one Ubuntu box:
nic0: eth0
ip=192.168.127.88
gateway=192.168.127.2
nic1: eth1
ip=172.24.220.72
gateway=172.24.220.1
And the original networking configuration is:
> netstat -arp
172.24.0.0 * 255.255.0.0 U 0 0 0 eth1
link-local * 255.255.0.0 U 0 0 0 eth0
192.168.0.0 * 255.255.0.0 U 0 0 0 eth0
default 192.168.127.2 0.0.0.0 UG 0 0 0 eth0
In the above network configuration, eth0 will be used as default gateway of server, all output traffic will go through eth0, except the target address is 172.24.0.0/16.
issue:
if one 172.24.0.0/16 client access service via eth0, the return IP trafic will go through eth1, instead of erh0, according to the route table.
Solution: Using policy route mechanism of linux:
Step 1 ****** Add extra route table for eth1 & eth0:
# echo “251 eth1” >>/etc/iproute2/rt_tables
# echo “250 eth1” >>/etc/iproute2/rt_tables
Step 2 ****** Setup eth1 route table:
# ip route add 172.24.220.0/24 dev eth1 src 172.24.220.72 table eth1
# ip route add default via 172.24.220.1 dev eth1 table eth1
# ip route add 192.168.127.0/24 dev eth0 src 192.168.127.88 table eth0
# ip route add default via 192.168.127.2 dev eth0 table eth0
Step 3 ****** Setup rule of eth0 & eth1 table:
# ip rule del prio 1000
# ip rule del prio 2000
# ip rule add from 172.24.220.72/16 table eth1 prio 1000
# ip rule add from 192.168.127.88/16 table eth0 prio 2000
Step 4 ****** Remove eth1 entry in the default table:
# sudo route del -net 172.24.0.0/16
Step 5 ****** Clean route table cache
# sudo ip -s -s route flush cache
Example:
# cat /etc/network/interfaces
auto lo
iface lo inet loopback
auto eth0
iface eth0 inet dhcp
auto eth1
iface eth1 inet dhcp
post-up /etc/init.d/seteth1mask.sh
#cat /etc/init.d/seteth1mask.sh
#! /bin/sh
/sbin/ifconfig eth1 netmask 255.255.0.0
/sbin/route add -host 172.24.61.252 gw 192.168.127.2 dev eth0
ip route add 172.24.220.0/24 dev eth1 src 172.24.220.72 table eth1
ip route add default via 172.24.220.1 dev eth1 table eth1
ip route add 192.168.127.0/24 dev eth0 src 192.168.127.88 table eth0
ip route add default via 192.168.127.2 dev eth0 table eth0
ip rule del prio 1000
ip rule del prio 2000
ip rule add from 172.24.220.72/16 table eth1 prio 1000
ip rule add from 192.168.127.88/16 table eth0 prio 2000
ip route flush cache
/sbin/route del -net 172.24.0.0/16
****** Note ******
Before network link switch on windows xp/samba client, need to clean the network cache on windows system by logout action