asymmetric routes/security concerns/Fortinet

Mailman List Mirror (Read Only)
1

Hello,

we have multiple internet connections of which one is a research network where many medical institutions and universities are also connected to threw out the country. This research network (ORION) also has internet access but is not meant to be used as a primary path to the internet by its customers. Connected to the ORION network are many sites we exchange email with daily who also have multiple internet connections. One of these sites is not reachable by us. After investigating, it was discovered this site is dropping our connections as the path back to use would use a different interface on the firewall ( a Fortinet device) than that which it arrived upon.

The admins at this university claim this is by design and for security reasons.. My response was the entire internet is asymmetrical and while this may of been a legitimate concern in the 90's, I don't think its a real concern anymore if things are set up correctly. They suggested we add static routes to our equipment to address this… This seems like a bad idea and I am not comfortable adjusting my routing table to address one site's issues on the internet due to their (not ours) routing/security policies.

am I correct here? any comments on this would be greatly appreciated as I'll be called into a meeting to discuss this further (they are digging in their heals in on this, and higher ups are getting involved now). I'd like to arm myself with a few perspectives.

thanks very much for your time again,

greg

2

The admins at this university claim this is by design and for security reasons.. My response was the entire internet is asymmetrical and while this may of been a legitimate concern in the 90's, I don't think its a real concern anymore if things are set up correctly. They suggested we add static routes to our equipment to address this… This seems like a bad idea and I am not comfortable adjusting my routing table to address one site's issues on the internet due to their (not ours) routing/security policies.

Working in a university environment like you, we do have connectivity to some of those high-speed R&E networks, and or routing policy generally prefers to use those paths if they are available, for reasons of performance (offloading traffic from more traditional transit paths) and cost/cost avoidance, as others have mentioned. Asymmetric routing is always a possibility between two multi-homed networks. I still occasionally have to wrestle with the notion that many people have that asymmetric routing is bad...

If the organization at the far end is doing stateful firewalling at the borders of their multi-homed network, then they are probably accustomed to things 'just breaking' more often then they're willing to admit :wink:

jms

3

On Fri, 7 Jan 2011 12:40:32 -0500

we have multiple internet connections of which one is a research
network where many medical institutions and universities are also
connected to threw out the country. This research network (ORION)
also has internet access but is not meant to be used as a primary
path to the internet by its customers. Connected to the ORION
network are many sites we exchange email with daily who also have
multiple internet connections. One of these sites is not reachable
by us. After investigating, it was discovered this site is
dropping our connections as the path back to use would use a
different interface on the firewall ( a Fortinet device) than that
which it arrived upon.

Correct me if I'm wrong, I'm not very familiar with ORION, but if it's
like some of the research networks in the U.S. have been built in the
past, ORION is dedicated high speed, low latency network that
interconnects research institutions together. The way these are often
used is that you localpref routes you learn from ORION participants so
that traffic between each of you goes over the research network. You'd
typically want this since the performance is good and there is plenty of
capacity available, but it is also paid for, probably through some
research grant, helping to reduce the use and expense of your commercial
transit.

You should be sending your traffic to them via ORION and they
likewise. However, if that path is down, then it would make sense for
it to go via another route. Hence, asymmetry may happen.

Are you not sending the traffic via ORION? If so, then I'd suggest you
both have something to fix. :slight_smile:

John

4

Thanks John for your input.

You are correct, ORION is a dedicated high speed research network.

Based on the fact that we access ORION via one of our ISPs (3rd party, we don't BGP/directly peer with ORION), I'm not sure if i can use this solution here. I could do that for the routes learned from that ISP, but we receive the entire internet routing table from them… I'd have to understand things more before I went down that road. perhaps I shouldn't be accepting the full table from them.

the localpref is something I'll look at, thanks for that. I'm not a BGP expert by any stretch, and our requirements here are "simple". we are not a transit. I've only attempted to make the config safe, not efficient.

i'd like to hear what you have to say about the original question, is there good reason in this day and age to drop traffic as described in the original post in your opinion?

-g

5

Based on the fact that we access ORION via one of our ISPs (3rd party, we

  don't BGP/directly peer with ORION), I'm not sure if i can use this solution
  here. I could do that for the routes learned from that ISP, but we receive
  the entire internet routing table from them? I'd have to understand things
  more before I went down that road. perhaps I shouldn't be accepting the
  full table from them.
  >
  >the localpref is something I'll look at, thanks for that. I'm not a BGP
  expert by any stretch, and our requirements here are "simple". we are not a
  transit. I've only attempted to make the config safe, not efficient.
  >
  > i'd like to hear what you have to say about the original question, is
  there good reason in this day and age to drop traffic as described in the
  original post in your opinion?

It sounds like the target site has a possible misconfiguration if this is a
long term issue. If they're using the open internet to get back to you and not
ORION (when your packets arrived from ORION-based connection), then something
is misconfigured or down. The problem is a conflict in the way BGP works and
how people assume it works :slight_smile: BGP is designed to get packets to where they
want to go, not drop them if they're going the wrong way.

The route back to you via ORION might not be available temporarily for a
legitimate reason (outtage, etc), or due to other unintentional side effects
of preference configurations. They are likely not seeing a route to you being
preferred via ORION. Try some traceroutes from your end and from their end and
compare. They're likely different paths. However, that shouldnt be suprising
- or a reason to filter traffic, really.

Symmetric routes across any chunk (big or small) of the internet are a
mythological thing of the past. Don't rely on that being the case at
any time.

As for your getting a full table from your upstream, you would likely
expect and want that. Mixed in would be ORION's prefixes, and if things
are working right, you are using an ORION path to get to your target. That's
up to the upstream's config preferences for packets destined to go through
ORION, so if you are the one using the open internet to get to the target
(check your traceroutes), then you need to talk to your upstream and get them
to fix their route preferences or whatever link or peering session is down.

As for dropping traffic, that's a strong fail-fast signal there. If you want
to ensure you are getting the intended bandwidth, say if you needed a 100mbps
path guaranteed that ORION can easily give you but the open internet/your
respective ISPs cant or wont (or you didnt pay for nor want to), then having
it fail immediately instead of using a slow backup path might be what you
want. There's a balance to be struck between failing fast thus generating
sufficient complaints that pressure to fix the best (and only) path that has
the required capacity is done ASAP, but then you are also left with no
alternate connectivity to the target in the meantime. Which scenario you
prefer is up to you and dependant on your organizations' needs.

An alternative is to generate sufficient alarms on the best connection's
failure, fail over to slower alternates, alert users to the degraded quality
(and modify their expectations and behaviour), and have your respective tech
teams respond promptly. This all requires awareness, training and a more
sophisticated failure-handling system. (How do you automatically alert all
users that the alternate degraded path is in use for eg? Email? Pager?) Having
alternate connectivity tends to dilute responce urgency from my experience. A
question of discipline/(dis)incentives. :slight_smile:

As for using an outtage as a security measure, yes you will reduce the
opportunities for the open internet to be a source of spoofed packets from
other 'semi-trusted' entities that you expect to only go across ORION for.

However, it sounds like you have no opportunity to determine such packets'
arrivals/departures, as it all goes through your upstream(s). The other end
however seems to have a firewall system that does determine this disparity of
paths. This is a minor security benefit, IMHO, which may be worth it to you,
depending on how important some connectivity is vs the increased risk of
spoofed packets from the general internet during the primary link via ORION's
downtime. And, it seems this is the other org's decision to make, not yours,
since you dont directly control your own ORION peering, and you are getting a
full routing table from your upstream.

If you wanted to control your ORION traffic, you would likely get a second BGP
feed and link from your upstream if you cant connect to ORION directly somehow.

Are you not on TORIX? If so you could connect to ORION directly as we at Heavy
Computing do.

/kc

6

You can allow asymmetric traffic on the Fortinet, but you lose some
functionality. Firewalls aren't routers and pretty much all of them
behave in the similar manner.

7

Thanks Ken,

Some good stuff there, thanks.

Since my original email, i think i've come up with a partial solution not requiring the far end's involvement. If not, at least it would get us into a better position to utilize the ORION network when possible. We peer over a L2 tunnel with a router down in the states threw one of our ISP's 10G links, I'm going to see if ORION will do the same with us. This would allow us to establish a BGP session directly with the ORION router, then I could use the localpref options, which may help.

this problem is intermitting, most of the time things are fine. doing the above isn't going to help if path/route conditions change, but at least we'll have done all we could within reason and have a proper config.

I didn't consider the reasons you mentioned related to 'fail fast', that does make a lot of sense. this is not the reason they claim this policy is in place, it is for security reasons.

we access ORION via GTAnet, they are within/part of/something to do with the UoT, and we are across the street.

take care,
greg

@Anthony Pardini <tony@pardini.org>

  Firewalls aren't routers and pretty much all of them
behave in the similar manner.

oh! thanks. 8)

8

Thanks Ken,

  >
  >Some good stuff there, thanks.
  >
  >Since my original email, i think i've come up with a partial solution not requiring the far end's involvement. If not, at least it would get us into a better position to utilize the ORION network when possible. We peer over a L2 tunnel with a router down in the states threw one of our ISP's 10G links, I'm going to see if ORION will do the same with us. This would allow us to establish a BGP session directly with the ORION router, then I could use the localpref options, which may help.
  >
  >this problem is intermitting, most of the time things are fine. doing the above isn't going to help if path/route conditions change, but at least we'll have done all we could within reason and have a proper config.
  >
  >I didn't consider the reasons you mentioned related to 'fail fast', that does make a lot of sense. this is not the reason they claim this policy is in place, it is for security reasons.
  >
  >we access ORION via GTAnet, they are within/part of/something to do with the UoT, and we are across the street.

Intermittent sounds exactly like what's happening - and alternate routes are
being chosen when the main link or peering session is down. And their firewall
isnt liking the alternate route and blocking packets. (oddly if their policy
is simple enough, you sending packets to them also across the open internet
so you both are using it to eachother, might make things work - with a further
reduction in (perceived?) security. :slight_smile:

Yes, peering directly with ORION would allow you some control of outgoing
packets if that's the issue - ie the issue is you sending via open internet.

But if the issue is you receiving via open internet (ie the far side is
sending via open internet because their ORION routes are not being preferred
to you due to outtage, etc), then you'll have to work with them on their side
of the issue. Localpref and other big hammer approaches to preferences are
effective but indelicate, and only work on outgoing traffic. Engineering
incoming traffic is a black art (there are several vendors that will sell you
gear and access to help you of course :slight_smile:

If you are going through an upstream that is also on ORION, their concerns for
routing to your target should be convergent with yours. Im suspecting that the
issue is at the far end with their firewalling policy and link/session, which
are harder for you to fix directly. You could also get a lan extension
between your site and theirs directly if you wanted to peer with that entity
directly, if signficant/frequent valuable traffic between you is sufficient, and
you do not want it to ever pass over the open internet.

good luck.

/kc

9

Randy your assumptions are correct, all outbounds get that slapped on them, automagically. good thing you have read the same magic book and can counter! 8)

I don't or ever did expect anything from you, not sure why you thought i might.

do you think I should quit this organization because we do this, and not consider the good work and intentions they have? (finding a cure for cancer) I don't think the legal's deparments intention was to force anything onto the world.

sounds like you have a lot of anger issues to work out. 8)

take care and have a great weekend,

greg

10

On Fri, 7 Jan 2011 13:56:00 -0500

the localpref is something I'll look at, thanks for that. I'm not
a BGP expert by any stretch, and our requirements here are
"simple". we are not a transit. I've only attempted to make the
config safe, not efficient.

I'm not quite sure I understand what the paths look like, but you could
also append your ASN once or twice for your routes on the less
preferred path to make the other institution use the more preferred one
as long as it is available.

i'd like to hear what you have to say about the original question,
is there good reason in this day and age to drop traffic as described
in the original post in your opinion?

Depends on who you ask. I think it clearly shows a mismatch in the
assumptions of security devices, engineers and the actual behavior of
some deployed networks.

Since you're both part of ORION, ideally packets would be following the
same path in both directions. I suggest you endeavor to make that the
common case.

However, to answer your question, dropping packets because the path is
asymmetrical would not be something I'd want my university network to
be doing. I'd love for them to tell me it's happening though.

John

11

you have sent a message to me which seems to contain a legal
warning on who can read it, or how it may be distributed, or
whether it may be archived, etc.

i do not accept such email. my mail user agent detected a legal
notice when i was opening your mail, and automatically deleted it.
so do not expect further response.

yes, i know your mail environment automatically added the legal
notice. well, my mail environment automatically detected it,
deleted it, and sent this message to you. so don't expect a lot
of sympathy.

and if you choose to work for some enterprise clueless enough to
think that they can force this silliness on the world, use gmail,
hotmail, ...

randy

12
13

Tarig Yassin Ahmed

You can allow asymmetric traffic on the Fortinet, but you lose some
functionality. Firewalls aren't routers and pretty much all of them
behave in the similar manner.

Hi

I think u can solve this issue only by adding router between the firewall and the Internet.

in multihoming metwork, Internet connections should be connect to routers then afterthat come the the firewall to avoid such problems.

Thanks