Charter ARP Leak

Mailman List Mirror (Read Only)
1

Hello,

I recently swapped out a home router for a SRX at home. Any charter techs able to take a look at the following? It looks like I am seeing some arp broadcast leaks towards my home router.

Here is a small excerpt I am seeing.

06:04:04.760869 In 00:21:a0:fb:53:d9 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: arp who-has 97.85.59.219 tell 97.85.58.1
06:04:04.761950 In 00:21:a0:fb:53:d9 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: arp who-has 75.135.155.27 tell 75.135.152.1
06:04:04.765870 In 00:21:a0:fb:53:d9 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: arp who-has 96.36.45.180 tell 96.36.44.1
06:04:04.802309 In 00:21:a0:fb:53:d9 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: arp who-has 68.188.219.125 tell 68.188.218.1
06:04:04.847125 In 00:21:a0:fb:53:d9 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: arp who-has 71.89.171.238 tell 71.89.168.1
06:04:04.873828 In 00:21:a0:fb:53:d9 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: arp who-has 24.247.247.159 tell 24.247.247.1
06:04:04.879921 In 00:21:a0:fb:53:d9 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: arp who-has 71.89.171.68 tell 71.89.168.1
06:04:04.890323 In 00:21:a0:fb:53:d9 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: arp who-has 96.36.45.161 tell 96.36.44.1
06:04:04.896711 In 00:21:a0:fb:53:d9 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: arp who-has 66.227.246.238 tell 66.227.240.1
06:04:04.901874 In 00:21:a0:fb:53:d9 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: arp who-has 24.247.247.205 tell 24.247.247.1
06:04:04.938238 In 00:21:a0:fb:53:d9 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: arp who-has 66.227.241.137 tell 66.227.240.1
06:04:04.965508 In 00:21:a0:fb:53:d9 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: arp who-has 71.89.171.119 tell 71.89.168.1
06:04:04.973382 In 00:21:a0:fb:53:d9 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: arp who-has 66.227.247.55 tell 66.227.240.1

Stephen Carter | IT Systems Administrator | Gun Lake Tribal Gaming Commission
1123 129th Avenue, Wayland, MI 49348
Phone 269.792.1773
[cid:image001.png@01CF83DD.3875D090]

<br><hr><font face='Arial' color='Gray' size='1'>The information contained in this electronic transmission (email) is confidential information and may be subject to attorney/client privilege. It is intended only for the use of the individual or entity named above. ANY DISTRIBUTION OR COPYING OF THIS MESSAGE IS PROHIBITED, except by the intended recipient. Attempts to intercept this message are in violation of 18 U.S.C. 2511(1) of the Electronic Communications Privacy Act (ECPA), which subjects the interceptor to fines, imprisonment and/or civil damages.</font>

2

This is normal for a cable modem network. These are broadcast packets so
they get delivered to everybody on that node.

ARP uses layer-2 broadcast to ask for the owner of a given IP to respond
with its MAC so that subsequent communication with that IP can be addressed
directly.

[sent from mobile device]

3

The interesting thing is that they're all .1 addresses. It's almost as if
the one broadcast domain has at least 7 different address spaces on it.

I've long seen similar in Comcast country. My CPE router has an upstream
interface:

ge00 Link encap:Ethernet HWaddr 10:0D:7F:64:CA:0C
          inet addr:73.171.123.11 Bcast:73.171.123.255 Mask:255.255.254.0

but yet I see a continual background flux of 6-8 arp requests a second, mostly
from what appear to be routers for other subnets:

# cpdump -i ge00 -n arp -c 2000 | awk '{print $7}' | sort | uniq -c
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ge00, link-type EN10MB (Ethernet), capture size 65535 bytes
2000 packets captured
2012 packets received by filter
0 packets dropped by kernel
     38 100.93.216.1,
     16 184.121.18.1,
     18 184.126.32.1,
     36 24.127.42.1,
     34 24.127.50.1,
     20 24.131.5.1,
     18 50.134.17.1,
     17 50.134.55.1,
     37 50.134.64.1,
     91 50.218.88.1,
    142 50.220.88.1,
    298 71.197.0.1,
    183 71.62.120.1,
     81 71.63.61.1,
    167 73.171.122.1, (my putative upstream router)
      1 73.171.123.11, (my box timed out its arp entry for upstream)
    131 73.171.77.1,
    511 73.31.150.1,
    157 73.31.41.1,
      3 96.120.18.205,

I've annotated the 2 lines I *expected* to see...

The other odd part is that of 20 sources, only 7 appear to have PTR entries....

When I first noticed this and mentioned it to somebody, they responded
"Forget it, Jake. It's Chinatown".

4

Valdis, you are correct. What your seeing is caused by multiple IP blocks
being assigned to the same CMTS interface.

5

Am I incorrect, though, in believing that ARP packets should only be visible
within a broadcast domain, and that because of that, they should not be
being passed through a cablemodem attached to such a CMTS interface unless
they're within the IP network in which that interface lives (which is
probably not 0/0)?

This sounds like a firmware bug in either the CMTS or the cablemodem.

Cheers,
-- jra

6

>
> Valdis, you are correct. What your seeing is caused by multiple IP
> blocks being assigned to the same CMTS interface.

Am I incorrect, though, in believing that ARP packets should only be visible
within a broadcast domain,

broadcast domain != subnet

and that because of that, they should not be
being passed through a cablemodem attached to such a CMTS interface unless
they're within the IP network in which that interface lives (which is
probably not 0/0)?

This sounds like a firmware bug in either the CMTS or the cablemodem.

int ethernet 0/0
  ip address 10.0.0.1 255.255.0.0
  ip address 11.0.0.1 255.255.0.0 secondary
  ip address 12.0.0.1 255.255.0.0 secondary

The broadcast domain will have ARP broadcasts for all three subnets.

Doing it over a CMTS doesn't change that.

     -- Brett

7

From: "Brett Frankenberger" <rbf@rbfnet.com>

> >
> > Valdis, you are correct. What your seeing is caused by multiple IP
> > blocks being assigned to the same CMTS interface.
>
> Am I incorrect, though, in believing that ARP packets should only be
> visible
> within a broadcast domain,

broadcast domain != subnet

Yeah; I didn't use the right term. That's why my networks are small. :slight_smile:

> and that because of that, they should not be
> being passed through a cablemodem attached to such a CMTS interface
> unless
> they're within the IP network in which that interface lives (which
> is
> probably not 0/0)?
>
> This sounds like a firmware bug in either the CMTS or the
> cablemodem.

int ethernet 0/0
ip address 10.0.0.1 255.255.0.0
ip address 11.0.0.1 255.255.0.0 secondary
ip address 12.0.0.1 255.255.0.0 secondary

The broadcast domain will have ARP broadcasts for all three subnets.

Doing it over a CMTS doesn't change that.

Ok. But the interface to which the cablemodem is attached, in the general
single-DHCP-IP case, is a /24, is it not?

The example Valdis posted had 5 or 6 different /24s from all over the v4
address space; that seems exceptionally sloppy routing...

I have seen ARP-traffic-not-for-me come through a cablemodem in the past as
well, but it was *uniformly* for the /24 in which my modem's address lived
that day.

Cheers,
-- jra

8

I would be way more interested in seeing the NDP entries than the ARP
entries.

  - Jared

9

Ok. But the interface to which the cablemodem is attached, in the general
single-DHCP-IP case, is a /24, is it not?

I'm on TWC. The IP address I get from them is on a /20.

104.230.32.0/20 dev eth7 proto kernel scope link src 104.230.32.x

The example Valdis posted had 5 or 6 different /24s from all over the v4
address space; that seems exceptionally sloppy routing...

fw-1:/root # tcpdump -ni eth7 arp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth7, link-type EN10MB (Ethernet), capture size 65535 bytes
12:54:21.354278 ARP, Request who-has 173.89.105.161 tell 173.89.96.1, length 46
12:54:21.355881 ARP, Request who-has 104.230.27.232 tell 104.230.0.1, length 46

We all knows it's easier to add another secondary IP to the interface and add a new DHCP scope than to try to expand a subnet.

Not sure I understand what all the excitement is about?

10

Ok. But the interface to which the cablemodem is attached, in the general
single-DHCP-IP case, is a /24, is it not?

No, I've seen multiple IPv4 /21s assigned to a single customer interface on a CMTS. The newer CMTS are beastly large boxes.

The example Valdis posted had 5 or 6 different /24s from all over the v4
address space; that seems exceptionally sloppy routing...

It's just the nature of having multiple secondary IP addresses on the same RF interface facing the customers

I have seen ARP-traffic-not-for-me come through a cablemodem in the past as
well, but it was *uniformly* for the /24 in which my modem's address lived
that day.

Cable modems are typically bridges (at least the ones that Work Right, IMHO), so it makes sense that you'll see all layer 2 broadcasts. If you live in a small enough town, or have business class service on your modem, you may only see a smaller or single subnet. On the residential side in a larger town you'll see lots of layer 2 stuff.

--Chris

11

The CM is just a bridge for that traffic. It has a management IP assigned to it by the provider but that's a different network so to speak.

Phil

12

add an *adjacent* block, not one halfway across the address space, no?

Cheers,
-- jra

13

We'll I would for one be very interested if the 8 ARP packets a second count against the caps.

Given len of 46 or 60 is not much, but that's about a gig of traffic almost assuming 8 of those a second happen(and my cold medicine addled mind is working). I'm sure it's not just that when it comes to garbage received on your interface either.

14

Valdis, you are correct. What your seeing is caused by multiple IP
blocks being assigned to the same CMTS interface.

Am I incorrect, though, in believing that ARP packets should only be visible
within a broadcast domain,

broadcast domain != subnet

It surprises me that in this day and age, in a forum like this that has an active thread about kids being taught archaic concepts, we see language like "broadcast domain != subnet" and a perceived need to explain it.

[no longer germane material deleted to reduce excess baggage charges]

int ethernet 0/0
   ip address 10.0.0.1 255.255.0.0
   ip address 11.0.0.1 255.255.0.0 secondary
   ip address 12.0.0.1 255.255.0.0 secondary

The broadcast domain will have ARP broadcasts for all three subnets.

This are not "subnets"! They are IP addresses in three different IP networks.

Doing it over a CMTS doesn't change that.

Communication here perceived as hostile is apologized-for.

15

Well sure they are subnets :slight_smile: of 0.0.0.0/4

range: 0.0.0.0 > 15.255.255.255
range b10: 0 > 268435455
range b16: 0x0 > 0xfffffff
hosts: 268435456
prefixlen: 4
mask: 240.0.0.0

Doubt anyone should ever describe them as such unless they own all that space though. May God rest their soul if they do.

16

Depends on where and what counters they probe. I would assume they look at "unicast" fields, so it wouldn't counted. (of course, *I* would use netflow accounting, but I care about my data being accurate.)

--Ricky

17

I probably should know--what kind of packets get counted? All? TCP & UDP? ICMP? BDU's?

18

They generally use IPDR on the CMTS for accounting, and I don't believe it counts ARP.

Phil

19

One never knows how the address space is carved up. Changing what
were once deemed reasonable addressing ideas, ultimately becoming
grossly suboptimal, often loses out to competing interests.

A long time ago, I arrived at a network where there were two major
sites with many LANs at each site. Generally speaking each LAN was a
department, but a department spanned both sites. Each department/LAN
at a site started off with less than a /25 worth of nodes.

This was apparently all done at a time when RIPv1 was the norm and
multiple subnet sizes were not widely deployed if even available in the
gear deployed.

The arrangement I inherited was such that a department was assiged
a /24, with the lower half (a /25) network at one site, and the upper
half at the other. As long as the organization's assigned /16 always
used /25's per network and departments split between sites fit into
the /25 things might have been fine for awhile. By the time I arrived
the address space was impossibly fragmented with some router
interfaces having many secondaries as departments arose, grew, split,
ceased to exist and new sites came online. This had the now
predictable effect of turning a seemingly nice day one addressing plan
into a fragmented and secondary mess. That was over 15 years
ago and there are still remnants of the originally addressing plan in
place.

I wouldn't be too surprised or even too concerned about these sorts of
configurations that appear poorly designed in hindsight. They are
natural for most any complex system as it evolves. It is all part of
the fun.

John