Hello all!!! First post here
Some carrier, somewhere between us and the service provider is selectively dropping the IKE packets originating from our VPN gateway and destined for our Brazil gateway. Other traffic is able to pass, as are the IKE packets coming back from Brazil to us. This is effectively preventing us from establishing the IPSEC tunnel between our gateways.
Also something else is awry, for two given hosts on the same subnet (x.y.z.52 and x.y.z.53), they take two wildly divergent paths:
For dest x.y.x.52
16 ms 3 ms 3 ms xe-2-3-0.cr1.lga5.us.above.net [64.125.31.34]
2 ms 2 ms 2 ms xe-0-1-0.er1.lga5.us.above.net [64.125.27.61]
12 ms 3 ms 13 ms e2-4-10000m.ar9.nyc1.gblx.net [208.178.58.197]
117 ms 118 ms 118 ms te3-4-10g.asr1.gru1.gblx.net [67.16.142.238]
136 ms 137 ms 136 ms ctbc-multimidia-data-net-s-a.gigabitethernet1-2.ar5.gru1.gblx.net [207.138.94.102]
157 ms 136 ms 138 ms xe-3-2-0-0.core-b.ula001.ctbc.com.br [201.48.44.165]
132 ms 132 ms 141 ms ge-3-0-0-0.core-b.spo511.ctbc.com.br [201.48.44.14]
135 ms 133 ms 134 ms ae1-0.edge-a.spo511.ctbc.com.br [201.48.44.93]
For dest x.y.x.53
3 ms 2 ms 3 ms xe-2-3-0.cr1.lga5.us.above.net [64.125.31.34]
2 ms 2 ms 2 ms xe-1-1-0.er1.lga5.us.above.net [64.125.26.162]
19 ms 3 ms 12 ms e2-4-10000m.ar9.nyc1.gblx.net [208.178.58.197]
117 ms 117 ms 117 ms te3-4-10g.asr1.gru1.gblx.net [67.16.142.238]
117 ms 117 ms 118 ms 64.209.106.170
118 ms 118 ms 118 ms ae1-0.edge-a.spo511.ctbc.com.br [201.48.44.93]
Anyone have any insight on to what may be occurring?
OMG !!!...l know this sounds weird and you wouldn't believe me...Yes,
am really stuck out here in the UK and it's so devastating at the
moment i really need your help l wish l could cal but l don't have
access to phone at the moment , I really need your help to get myself
out of this mess. The hotel management has been kind to let have an
access to a library to get across to anybody to help me since my
luggage was also taken away.Hope to hear from you soon
Isabel - I am a network engineer working for a small financial firm outside Philadelphia.
Can you offer any insight?
I just need you to help me with some cash just till i get back home
I still don't know where you are and the simulation you are doing .....can you
be more specific ?
normally address to service providers that sit behind a block of addresses and
are able to provide visibility from their networks , looking glasses are as
usefull but if you need someone to be as cleaver as the ones on training at
Holloway just be more specific as there are plenty of options around the world
and also in the UK ......
Some carrier, somewhere between us and the service provider is selectively
dropping the IKE packets originating from our VPN gateway and destined for
our Brazil gateway. Other traffic is able to pass, as are the IKE packets coming
back from Brazil to us. This is effectively preventing us from establishing
the IPSEC tunnel between our gateways.
Has IKE been known to work to that location before? Or is this something new?
My first guess is some chucklehead banana-eater at the service provider either
fat-fingered the firewall config, or semi-intentionally blocked it because it
was "traffic on a protocol/port number they didn't understand so it must be
evil".
Also something else is awry, for two given hosts on the same subnet (x.y.z.52
and x.y.z.53), they take two wildly divergent paths:
Anyone have any insight on to what may be occurring?
The paths appear to diverge at 67.16.142.238. I wonder if that's gear trying
to do some load-balancing across 2 paths, and it's using the source IP as a
major part of the selector function ("route to round-robin interface source-IP
mod N" or similar?).
The other possibility is your two traceroutes happened to catch a routing flap in
progress (obviously not the case if the two routes are remaining stable).
Sorry I can't be more helpful than that...
Thanks for the response.
Ike had worked great up until Monday. The provider did a local test and our box saw the Ike packets so it appears to lie somewhere upstream. (GLBX may be a good guess)
Also - the paths are stable and we are sourcing from the same ip - very strange behaivor. Hope someone from GLBX or CTBC (or someone who had experienced an issue like this) can assist
Thanks to all for their feedback so far.
SD
We saw similar issues with IKE through Global Crossing (as odd as that sounds) out of the NYC market at the same time. We routed around them and problem solved. Still scratching our heads on that one... In my experiences, GLBX has numerous odd issues to the point where it's become a bad joke anytime something breaks with connectivity... we blame them. It's kind of not funny though because it's almost always true. Taking them out of the equation usually fixes the problem. One of our customers who is frequently affected by GBLX problems jumps to the (often correct) conclusion that they are causing problems. :-/
-Vinny
Very interesting. I have had similar issues with connectivity to my Brazil office, and oddly enough it involved GBLX and CTBC (now called Algar Telecom). I also vastly divergent paths to a couple hosts in the same subnet. I ended up communicating with GBLX via email (who were actually really great in corresponding with me)...the engineer pointed to some sort of link capacity issue...i'll dig up the thread...
That thread detail would be very interesting to me. Thanks for the heads up.
Thanks Vinny - how did you route around? There seems to be one path from the US to Brazil via GBLX and CTBC. Are you leveraging leased connectivity? Thanks for the info!!
SD
CTBC has capacity from GBLX, TIWS and SEABONE, although not all
prefixes are announced to all providers. TIWS usual path in the US is
thru Level 3, so steering the traffic to Level 3 might do the trick.
Rubens
Very simply. We chose to stop accepting prefixes from and announcing
prefixes to them. You could attempt this in more elaborate and less
forceful ways if you're in the right position, but we encounter issues
like this too much and it affects critical clients that cannot afford
any downtime, and we have plenty of other transit connections. We are in
a position where we have direct connectivity with them (which based on
our track record won't be much longer), so it might be much easier for
us. Otherwise you'd have to hope your upstream is the one connected to
them and has communities available to tinker with to withdraw your
tagged prefixes from being announced to them, and just change the local
preference or however you prefer to do it on the inbound routes from
your upstream, or better yet filter on as-path.
-Vinny