ARP is sourced from loopback address

Mailman List Mirror (Read Only)
1

Hey All,

Anycast related.

Is this normal behavior? Whats the workaround? Why havent I run into this before?

192.168.76.1 is a HSRP address on a ring of routers transiting a private non routed vlan to the service addresses hosted on systems that have independent management interfaces.

Best,

Joe

root@debian31:~# ifconfig lo:0
lo:0 Link encap:Local Loopback
           inet addr:209.54.140.64 Mask:255.255.255.255
           UP LOOPBACK RUNNING MTU:16436 Metric:1

root@debian31:~# ip rule list
0: from all lookup local
32764: from 209.54.140.0/24 lookup pbr1-exit
32765: from 216.222.144.16/28 lookup pbr1-exit
32766: from all lookup main
32767: from all lookup default
root@debian31:~# ip route list table pbr1-exit
default via 192.168.76.1 dev eth1
192.168.34.0/24 dev eth1 scope link src 192.168.76.16
192.168.76.0/24 dev eth1 scope link src 192.168.76.16
root@debian31:~# tcpdump -i eth1
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes

11:08:09.053943 ARP, Request who-has 192.168.76.1 tell 209.54.140.64, length 28
11:08:10.035126 IP noc08rt08.noc08.chl.net > 209.54.140.64: ICMP echo request, id 517, seq 0, length 80
11:08:10.051276 ARP, Request who-has 192.168.76.1 tell 209.54.140.64, length 28
11:08:11.052548 ARP, Request who-has 192.168.76.1 tell 209.54.140.64, length 28
11:08:12.035964 IP noc08rt08.noc08.chl.net > 209.54.140.64: ICMP echo request, id 517, seq 1, length 80
^C

root@debian31:~# ip neigh
fe80::230:71ff:fe3b:6808 dev eth0 lladdr 00:30:71:3b:68:08 router STALE
192.168.76.1 dev eth1 FAILED
192.168.34.254 dev eth0 lladdr 00:11:93:04:7a:1b DELAY
192.168.34.48 dev eth0 lladdr 00:0c:29:fd:64:8a STALE

root@debian31:~# uname -a
Linux debian31 3.2.0-1-686-pae #1 SMP Tue Jan 24 06:09:30 UTC 2012 i686 GNU/Linux

root@debian31:~# ping 192.168.76.1
PING 192.168.76.1 (192.168.76.1) 56(84) bytes of data.
64 bytes from 192.168.76.1: icmp_req=1 ttl=255 time=2.95 ms
^C
--- 192.168.76.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 2.952/2.952/2.952/0.000 ms
root@debian31:~# ip neigh
fe80::230:71ff:fe3b:6808 dev eth0 lladdr 00:30:71:3b:68:08 router STALE
192.168.76.1 dev eth1 lladdr 00:00:0c:9f:f0:01 REACHABLE
192.168.34.254 dev eth0 lladdr 00:11:93:04:7a:1b REACHABLE
192.168.34.48 dev eth0 lladdr 00:0c:29:fd:64:8a STALE
192.168.76.2 dev eth1 lladdr 00:b0:4a:9e:54:00 STALE
root@debian31:~# !tcp
tcpdump -i eth1
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes
11:12:22.476479 IP noc08rt08-l0.noc08.chl.net > 209.54.140.64: ICMP echo request, id 518, seq 0, length 80
11:12:22.476572 IP 209.54.140.64 > noc08rt08-l0.noc08.chl.net: ICMP echo reply, id 518, seq 0, length 80
11:12:22.479495 IP noc08rt08-l0.noc08.chl.net > 209.54.140.64: ICMP echo request, id 518, seq 1, length 80
11:12:22.479533 IP 209.54.140.64 > noc08rt08-l0.noc08.chl.net: ICMP echo reply, id 518, seq 1, length 80
11:12:22.484346 IP noc08rt08-l0.noc08.chl.net > 209.54.140.64: ICMP echo request, id 518, seq 2, length 80
11:12:22.484392 IP 209.54.140.64 > noc08rt08-l0.noc08.chl.net: ICMP echo reply, id 518, seq 2, length 80
11:12:22.487670 IP noc08rt08-l0.noc08.chl.net > 209.54.140.64: ICMP echo request, id 518, seq 3, length 80
11:12:22.487705 IP 209.54.140.64 > noc08rt08-l0.noc08.chl.net: ICMP echo reply, id 518, seq 3, length 80
11:12:22.490639 IP noc08rt08-l0.noc08.chl.net > 209.54.140.64: ICMP echo request, id 518, seq 4, length 80
11:12:22.490675 IP 209.54.140.64 > noc08rt08-l0.noc08.chl.net: ICMP echo reply, id 518, seq 4, length 80
^C

2

Even though TCP dump doesn't show it the ARP packets should have a
source mac address that is reachable on the link. I think the reply
is unicast to that mac address regardless of the IP in the request.
Otherwise the receiving station would have to do an arp request for
the source IP in the packet before it replied, in order to reply that
station would need to have the very mapping it just requested making
the whole thing useless. I've never seen arp sourced from a
non-local interface IP unless there was some sort of tunnel or
bridging configured, but then again I don't spend my days staring at
ARP packets so I could be missing something.

3

Hi Joe,

Linux frequently does Really Stupid Things with ARP. You can generally
force it to do the right thing with the arp_announce, arp_ignore and
arp_filter sysctl's as well as the arptables command.

If I understand your problem correctly, you have a virtual IP on a
loopback interface and when that virtual IP is pinged, the Linux box
uses it as the source address in the arp request instead of using the
correct source address for that interface. Because the source address
is not valid for that LAN, the router does not respond.

Workaround:

vi /etc/sysctl.conf:
net.ipv4.conf.all.arp_announce = 1
net.ipv4.conf.eth1.arp_announce = 1

sysctl -p

This forces the box to use eth1's IP address when making an ARP
request from eth1 instead of using the VIP in the source address of
the IP packet (the default behavior).

#arp_announce - INTEGER
# Define different restriction levels for announcing the local
# source IP address from IP packets in ARP requests sent on
# interface:
# 0 - (default) Use any local address, configured on any interface
# 1 - Try to avoid local addresses that are not in the target's
# subnet for this interface.
# 2 - Always use the best local address for this target.
# In this mode we ignore the source address in the IP packet
# and try to select local address that we prefer for talks with
# the target host.

Regards,
Bill Herrin

4

Thanks for the reply.

Yes, it does appear to have the correct mac.

root@debian31:~# tcpdump -e -n -i eth1
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes
12:54:17.882537 00:03:fd:03:38:08 > 00:0c:29:b8:2a:14, ethertype IPv4 (0x0800), length 114: 69.90.15.224 > 216.222.144.24: ICMP echo request, id 161, seq 4, length 80
12:54:18.084320 00:0c:29:b8:2a:14 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 192.168.76.1 tell 209.54.140.64, length 28
12:54:19.083580 00:0c:29:b8:2a:14 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 192.168.76.1 tell 209.54.140.64, length 28
12:54:19.838376 00:03:fd:03:38:08 > 00:0c:29:b8:2a:14, ethertype IPv4 (0x0800), length 407: 69.90.15.224.179 > 216.222.144.24.60714: Flags [P.], seq 4062306194:4062306547, ack 170308540, win 16365, length 353: BGP, length: 353
12:54:20.083649 00:0c:29:b8:2a:14 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 192.168.76.1 tell 209.54.140.64, length 28

^C

root@debian31:~# ifconfig eth1
eth1 Link encap:Ethernet HWaddr 00:0c:29:b8:2a:14
           inet addr:192.168.76.16 Bcast:192.168.76.255 Mask:255.255.255.0

Keegan Holley wrote:

5

Golden.

Thank you, William.

Joe

William Herrin wrote:

6

Hi Joe,

You're welcome. The flip side of Linux's arp funkiness is that you can
get it to do some nifty stuff. For example, a /32 ethernet looks more
or less like this:

ifconfig lo:1 198.51.100.1 netmask 255.255.255.255
ifconfig eth1 192.168.0.1 netmask 255.255.255.252
ip route add 198.51.100.44/32 dev eth1 src 198.51.100.1
arptables --out-interface eth1 -j mangle -s 192.168.0.1 --mangle-ip-s
198.51.100.1

The implicit proxy arp takes care of the rest with the machine hanging
off the interface thinking that it's part of a /24.

This sort of thing is how I'm using all 17 of the IP addresses in my
Cox /28. :slight_smile:

Regards,
Bill Herrin

7

We ran into a lot of quirkiness with Linux when we started rolling out
Linux-based CPE with XORP as a routing engine.

I've thrown some sane defaults you might want to consider into a text file at:

http://soucy.org/xorp/xorp-1.7-pre/TUNING

Specifically, you prob. want option 2 instead of 1 for arp_ignore,
otherwise you'll see funkiness with ARPs coming from the wrong IP in a
multi-interface configuration.

----8<----
ARP_IGNORE values:

0 - Reply for any local address.
1 - Reply only if the target IP is configured on the receiving interface.
2 - Like 1, but the source IP (sender's address) must belong to the
same subnet as the target IP.
3 - Reply only if the scope of the target IP is not the local host
(e.g., that address is not used to communicate with other hosts).
4-7 - Reserved.
8 - Do not reply.

8 - Unknown value; accept request.

----8<----

Hope this helps,

8

That's still a different part of the packet. Below is the source
address in the ethernet header used to deliver the arp request itself.
In side the ARP payload there is also a field for source and
destination mac. I couldn't get tcpdump to show it even with the -n
and -vvv switches. Wireshark will show it though. You may be able to
use -w and -s0 to save to a cap file and then look at arp in
wireshark. There still seem to be no responses. You can try the
tweaks suggested by others. I've sent traffic from a loopback before
and I've never seen this problem though.