Hi,
We receive a BGP feed from different providers on two
different routers. While one seems to be a reasonable amount
of feeds after reviewing the CIDR report, the other is anywhere
from 3K to 10K more routes.
Is there a utility that I can use that will pull the
routes off each router (Foundry preferred), and then compare
them as best it can to see why there is such a difference?
I can understand a handful of routes over what CIDR says,
but a minimum of 3K more?
Thanks, Tuc/TBOH
Is one of them as4323?
> Is there a utility that I can use that will pull the
> routes off each router (Foundry preferred), and then compare
> them as best it can to see why there is such a difference?
> I can understand a handful of routes over what CIDR says,
> but a minimum of 3K more?Is one of them as4323?
Actually, no. I wasn't wanting to "name names" to
protect the innocent... BUT....
ROUTER1:
Neighbor Address AS# State Time Rt:Accepted Filtered Sent ToSend
64.200.58.69 7911 ESTAB 4d21h57m 182287 0 4 0
ROUTER2:
Neighbor Address AS# State Time Rt:Accepted Filtered Sent ToSend
69.28.152.229 22822 ESTAB 18d16h51m 186379 0 4 0
Tuc/TBOH
I don't know anything about foundry, but if you can simply display
the routing table from a terminal, you can go the hacky unix cli
tool way. For example, use 'script' to log your terminal session
to a file, then presuming you can show the route table and each
route includes a 'via upstream-address-line' line for each route
(completely untested and I'm sure someone could come up with
something much simpler and better):
grep 'via upstream?' script > upstream?
perl -ne 'print "$1\n" if /(\d{1,3}(?:\.\d{1,3}){3}\/\d{1,3})/' upstream? |
sort > upstream?.sored
comm -23 upstream1.txt upstream2.txt
comm -13 upstream1.txt upstream2.txt
John
I have one, but it's cisco-specific:
http://www.bofh.it/~md/software/cisco-tools-0.2.tgz (the dumppeers script)
Then you can easily find the missing routes with commands like:
awk '{print $1}' < ../routes/1.2.3.4 | sort > ROUTER1
awk '{print $1}' < ../routes/1.2.3.5 | sort > ROUTER2
comm -23 ROUTER1 ROUTER2 > MISSING2
This is actually fairly common. There are a lot of folks out there who
announce more specifics to one network but not another, or who apply no
export or limited export community tags in various places. Also, every
network has a different filter policy of what they will and won't accept.
FWIW my "exported to bgp speaking customers" count at this moment is
182525. I wouldn't get concerned about it unless the network with more
prefixes is doing something absurdly stupid like sending you internal /30s
and such (which, well, a lot of people do :P). It could also be something
like peers agreeing to traffic engineer by sending each other more
specifics w/meds, though if they were smart they would be doing that with
no-export so as to not make your TE job more difficult. If you really want
to compare the differences, try something like:
telnet yourrouter | tee outputfile
term length 0
sh ip bgp nei x.x.x.x received-routes
quit
Followed by 30 secs with awk(1), cut(1), diff(1), etc. For floundry,
something dirt simple like "grep / | awk '{ print $2 }'" should do the
trick.
Were I faced with this reporting equirement on an on-going basis, I'd suggest establishing a read-only BGP peer with both devices and comparing directly. I've got a perl BGP peering daemon that feeds and maintains a mirror of the BGP routing table into SQL, applying updates and withdrawals as they come in. Setting up something similar, and adding some additional metrics to keep entries unique by peer source would facilitate your end goal with simple SQL grouping mechanics.
- billn
Much of what Bill described below is already present using Nick Feamster's bgptools release: http://nms.lcs.mit.edu/software/bgp/bgptools/
Start with zebra / quagga / etc., which do a great job of dumping tables and updates.
Then use bgptools to take the MRT-formatted dumps that Zebra spits out and turn them into text, etc. With the '-q' option, can insert the BGP updates or table snapshot directly into a SQL database.
then the libbgpdump.a library gives you lots of cool things on top of that. You'd have to do a little work to get the analysis tool you want, but it's pretty easy. Use the 'buildtree' starting program to build the prefix tree from each provider and then compare those two trees (see which prefixes are present/not present, see if any parts of the IP space are unreachable in in one and unreachable in the other, etc.)
It starts as Bill suggested - a read-only BGP peer from the devices, which takes about 3 seconds to set up.
-Dave
Hi,
Thanks for all the replies! I've consolidated them here hoping to save
some noise....
From: Bill Nash <billn@odyssey.billn.net>
Were I faced with this reporting equirement on an on-going basis, I'd
suggest establishing a read-only BGP peer with both devices and comparing
directly. I've got a perl BGP peering daemon that feeds and maintains a
mirror of the BGP routing table into SQL, applying updates and withdrawals
as they come in. Setting up something similar, and adding some additional
metrics to keep entries unique by peer source would facilitate your end
goal with simple SQL grouping mechanics.
This is an idea, thank you. I was hoping for something that would
be a bit more "smarter" than BGP . What I was looking for would be something
that could say :
Router A has route 216.231.96.0/24, 216.231.97.0/24, (etc) while
Router B has 216.231.96.0/19
Router B has the following /30's :
A.B.C.D, E.F.G.H, I.J.K.L
Router A has 216.231.96.0/24, 216.231.97.0/24, but Router B has
a route of 216.231.96.0/19 but none of the other /24's.
From: Richard A Steenbergen <ras@e-gerbil.net>
This is actually fairly common. There are a lot of folks out there who
announce more specifics to one network but not another, or who apply no
export or limited export community tags in various places. Also, every
network has a different filter policy of what they will and won't accept.
I understood that this happened, but didn't think it could account
for 3K to 10K routes. Guess it can. 
FWIW my "exported to bgp speaking customers" count at this moment is
182525.
Thats in line with the CIDR report, and I wouldn't mind.
I wouldn't get concerned about it unless the network with more
prefixes is doing something absurdly stupid like sending you internal /30s
and such (which, well, a lot of people do :P). It could also be something
like peers agreeing to traffic engineer by sending each other more
specifics w/meds, though if they were smart they would be doing that with
no-export so as to not make your TE job more difficult.
Thats what I'm hoping to find out.
If you really want
to compare the differences, try something like:telnet yourrouter | tee outputfile
term length 0
sh ip bgp nei x.x.x.x received-routes
quitFollowed by 30 secs with awk(1), cut(1), diff(1), etc. For floundry,
something dirt simple like "grep / | awk '{ print $2 }'" should do the
trick.
(See above what I was looking for the output, but again, something
to start with, thanks!)
From: md@Linux.IT (Marco d'Itri)
Is there a utility that I can use that will pull the
routes off each router (Foundry preferred), and then compare
them as best it can to see why there is such a difference?I have one, but it's cisco-specific:
http://www.bofh.it/~md/software/cisco-tools-0.2.tgz (the dumppeers script)
himinbjorg# fetch http://www.bofh.it/~md/software/cisco-tools-0.2.tgz
fetch: http://www.bofh.it/~md/software/cisco-tools-0.2.tgz: Not Found
Then you can easily find the missing routes with commands like:
awk '{print $1}' < ../routes/1.2.3.4 | sort > ROUTER1
awk '{print $1}' < ../routes/1.2.3.5 | sort > ROUTER2
comm -23 ROUTER1 ROUTER2 > MISSING2
No worries, I'll take a look at it and then see if I can
"Foundryize" it. Its not such a case of "missing" but maybe more
aggregated differently, etc. But again, all leads will be taken!
From: John Kristoff <jtk@ultradns.net>
Is there a utility that I can use that will pull the
routes off each router (Foundry preferred), and then compare
them as best it can to see why there is such a difference?I don't know anything about foundry, but if you can simply display
the routing table from a terminal, you can go the hacky unix cli
tool way. For example, use 'script' to log your terminal session
to a file, then presuming you can show the route table and each
route includes a 'via upstream-address-line' line for each route
(completely untested and I'm sure someone could come up with
something much simpler and better):grep 'via upstream?' script > upstream?
perl -ne 'print "$1\n" if /(\d{1,3}(?:\.\d{1,3}){3}\/\d{1,3})/' upstream? |
sort > upstream?.sored
comm -23 upstream1.txt upstream2.txt
comm -13 upstream1.txt upstream2.txt
Thanks!
From: Warren Kumari <warren@kumari.net>
Sounds to me like one of your providers is not feeding you the full
internet routing table. Have you checked with them to see if they are
providing you that?Sounds to me like a: you are only looking at best routes or b: one of
the providers is sending you more specific customer routes (that they
summarize before sending to non-customers).Personally I would just slurp one set of routes into an array in perl
and then delete them if they appear in the other set. Any left over
in either set are unique....
It wouldn't take aggregate differences into account.
From: "Majdi S. Abbas" <msa@latt.net>
We receive a BGP feed from different providers on two
different routers. While one seems to be a reasonable amount
of feeds after reviewing the CIDR report, the other is anywhere
from 3K to 10K more routes.Thanks, Tuc/TBOH
-snip-
I refer both of you to the following message that I posted a
few years ago, rather than restate it all:http://www.merit.edu/mail.archives/nanog/2001-02/msg00347.html
Hope this helps.
--msa
No, I agree, I don't think I'm MISSING, just want to know what
the differences are to see why there is such a disparity. Maybe I need
to get the provider to filter or change communities, etc.
Much of what Bill described below is already present using Nick Feamster's bgptools release: http://nms.lcs.mit.edu/software/bgp/bgptools/
Start with zebra / quagga / etc., which do a great job of dumping tables and updates.
Then use bgptools to take the MRT-formatted dumps that Zebra spits out and turn them into text, etc. With the '-q' option, can insert the BGP updates or table snapshot directly into a SQL database.
My peer actually comes from a Zebra box, so I'm not talking directly to any production devices, in the event that I want to bounce my db feed up and down (debugging, featuritis treatments, etc) Z/Q + bgptools is a great suggestion for doing complex reporting/comparison on the routing tables, though. I've got a need for a more real-time view, so my setup fits me a little better than your suggestion, but potato/potatoe. =)
then the libbgpdump.a library gives you lots of cool things on top of that. You'd have to do a little work to get the analysis tool you want, but it's pretty easy. Use the 'buildtree' starting program to build the prefix tree from each provider and then compare those two trees (see which prefixes are present/not present, see if any parts of the IP space are unreachable in in one and unreachable in the other, etc.)
This is pretty interesting, I'll have to tinker with it, especially since I know one of my providers doesn't give me a full routing table.
It starts as Bill suggested - a read-only BGP peer from the devices, which takes about 3 seconds to set up.
And for folks to whom this is new stuff: don't be an idiot, put Zebra/Quagga up as a peer/buffer for attaching analysis tools to your network. *Never* attach development grade tools to a production device, most especially when you're dealing with a routing table. Not that I've ever taken down a live router in this manner[1], I'm just saying.. 
- billn
[1] All smirking current/past coworkers are kindly invited to stfu. =)