for i in /proc/sys/net/ipv4/conf/*/arp_announce; do echo 2 > $i;done
+1 setting arp_announce in Linux is essential if being used as a router
with more than one subnet.
I would also recommend setting arp_ignore. For Linux-based routers, I've
found the following settings to be optimal:
echo 1 > /proc/sys/net/ipv4/conf/all/arp_announce
echo 2 > /proc/sys/net/ipv4/conf/all/arp_ignore
On a side note, this underscores what a lot of people on-list are saying:
If you don't understand the internals of a Linux system, for example,
"rolling your own" will bite you.
It's also pretty rare to find a network engineer who is also a Linux
system-level developer, so finding and maintaining that talent can often be
a challenge.
Many make a leap and go on to assert that because of this software-based
systems can never be viable, which I disagree with. After all, the latest
OS offerings from Cisco run a Linux kernel. Nearly all the Ciena DWDM and
ME gear I run is built on Linux. These companies aren't doing quite as
much with hardware acceleration as they would lead you to believe.
I think Intel DPDK will be a disruptive technology for networking.
At the end of the day, I'm pretty anxious to see the days of over-priced
routers driving up network service costs go away.
> (...)
> The users each have a unique VLAN (Q-in-Q). The question is, what do I
put
> on those VLANs, if I do not want to put a full IPv4 subnet on each?
>
> My own answer to that is to have the users share a larger subnet, for
> example I could have a full class C sized subnet shared between 253
> users/VLANs.
>
> To allow these users to communicate with each other, and so they can
> communicate with the default gateway IP, I will need proxy arp. And in a
> non-OpenFlow solution, also the associated security functions such as
> DHCP-snooping to prevent hijacking of IP addresses.
>
> Which devices can solve this task?
Hi Baldur,
Assuming you manage 1.1.1.0/24 and 2001:db8:0::/48 and
have a Linux box on both ends you can get rid of
IPv4 and v6 interco subnets and arp proxy the following way:
1/ on the gateway
ip addr add 1.1.1.0/32 dev lo
for all client VLAN "NN" on eth0 :
ip -6 addr add fe80::1/64 dev eth0.NN
ip -6 route add 2001:db8:0:NN00::/56 via fe80::1:NN dev eth0.NN
2/ on user CPE number "NN" CPE WAN interface being eth0 :
ip addr add 1.1.1.NN/32 dev eth0
ip route add 1.1.1.0/32 dev eth0
ip route add default via 1.1.1.0
ip -6 addr add fe80::1:NN/64 dev eth0
ip -6 route add default via fe80::1 dev eth0
# ip -6 addr add 2001:db8:0:NN00::1/56 dev eth0 # optional
Note: NN in hex for IPv6
The trick in IPv4 is that linux by default will answer to ARP requests
for "1.1.1.0" on all interfaces even if the adress is on the loopback.
And in IPv6 use static link local on both ends. You can replace
"1.1.1.0" by any IPv4, but since ".0" are rarely assigned to end users
it doesn't waste anything and keep traceroute with public IPv4.
The nice thing of this setup is that it "virtualizes" the routing from
the client point of view: you can split/balance your clients on multiple
physical gateways and not change a line to the client configuration
while it's being moved, you just have to configure your IGP between
gateways to properly distribute internal routes.
We (AS197422 / tetaneutral.net) use this for virtual machines too (with
"tapNN" interfaces from KVM instead of "eth0.NN"): it allows us to move
virtual machines around physical machines without user reconfiguration,
not waste any IPv4 and avoid all issues with shared L2 (rogue RA/ARP
spoofing/whatever) since there's no shared L2 anymore between user VM.
It also allows us to not pre split our IPv4 space in a fixed scheme,
we manage only /32 so no waste at all.
Of course you still have work to do on PPS tuning.
Sincerely,
Laurent GUERBY
AS197422 http://tetaneutral.net peering http://as197422.net
PS: minimum settings on a Linux router
echo 1 > /proc/sys/net/ipv4/ip_forward
for i in /proc/sys/net/ipv6/conf/*; do for j in autoconf accept_ra; do
echo 0 > $i/$j; done;done