ISP inbound failover without BGP

This may sound like dumb question, but... I'm used to asking those.

Here's the scenario

Another ISP, say AT&T, is the primary ISP for a customer.

Customer has publicly accessible servers in their office, using the AT&T address space.

I am the customer's secondary ISP.

Now, if AT&T link fails, I can provide the customer outbound Internet access fairly easily. So they can surf and get to the Internet.

What about the publicly accessible servers that have AT&T addresses, though?

One thought I had was having them use Dynamic DNS service.

Are there any other solutions, short of using BGP multihoming and having them try to get their own ASN and IPv4 /24 block?

It looks like a few router manufacturers have devices that might work, but it looks like a short DNS TTL (or Dynamic DNS) needs to be set so when the primary ISP fails, the secondary ISP address is advertised.

This may sound like dumb question, but... I'm used to asking those.=0A=0AHe=
re's the scenario=0A=0AAnother ISP, say AT&T, is the primary ISP for a cust=
omer.=0A=0ACustomer has publicly accessible servers in their office, using =
the AT&T address space.=0A=0AI am the customer's secondary ISP.=0A=0ANow, i=
f AT&T link fails, I can provide the customer outbound Internet access fair=
ly easily.=A0 So they can surf and get to the Internet.=0A=0AWhat about the=
publicly accessible servers that have AT&T addresses, though?=0A=0AOne tho=
ught I had was having them use Dynamic DNS service.=A0 =0A=0AAre there any =
other solutions, short of using BGP multihoming and having them try to get =
their own ASN and IPv4 /24 block?=0A=0A=0AIt looks like a few router manufa=
cturers have devices that might work, but it looks like a short DNS TTL (or=
Dynamic DNS) needs to be set so when the primary ISP fails, the secondary =
ISP address is advertised.

The usual solution is to get the public servers stuck in a colo that's
multihomed.

Most of the other solutions tend to be a bit dodgy. If your gear is
sufficiently competent, you can hack up a solution with multiple
addresses for each of the servers (one on each ISP) and then use a
short TTL to fail over, but this has more of "fail" than "fail over"
about it, because there are a bunch of issues that typically result.

... JG

Depends on the application,

SIP, VPN, SMTP, etc just setup both IPs and let the end-user application figure it out (SIP-UA register to both IPs for example)

HTTP/HTTPS setup a proxy server in a colo that is multi-homed to frontend the requests. Then it can load balance traffic over both IPs.

DNS TTL ‘tricks’ are just that, they work ‘kinda’

Fatpipe? Crazy expensive IMHO but I hear they work ok.

-Matt

For what it sounds like the customer wants to do, this really is the right solution. Most everything else has some level of 'ugly hack' attached to it, that can cause reliability/reachability problems at inopportune times (as opposed to problems that happen at opportune times).

If the customer is just looking for failover capabilities, they can take default via BGP from both providers, prefer one over the other, and use some other bits to prefer one provider over for inbound connectivity. They would not need very beefy routers for this.

That will get you basic service redundancy. Add a second router, and they can have some protection in the event of a router failure.

It all really boils down to what the customer wants and is willing to pay for. If they have services that need to be reliably reachable, then using a time-tested design is a prudent decision.

jms

Depending on their business, using dynamic DNS providers could be a really bad idea. If they deal only with home users who won't even know, it'll probably work. If their customers are security-aware businesses, they probably block all sites hosted with dynamic DNS systems.

Ray

Depending on their business=2C using dynamic DNS providers could be a reall=
y bad idea. If they deal only with home users who won't even know=2C it'll =
probably work. If their customers are security-aware businesses=2C they pro=
bably block all sites hosted with dynamic DNS systems.

Where do dynamic DNS /providers/ enter the picture?

You update *your own* DNS records. Those could conceivably be hosted
with a dynamic DNS provider, but why not just use the API for whatever
system you currently use for DNS?

... JG

Is there some technical reason that BGP is not an option? You could allow them to announce their AT&T space via you as a secondary.

-Randy

That's a good point Ray - thank you.

Hi Eric,

I went through this a couple years ago with continuity of operations
planning. The bottom line is: with the notable exception of
low-activity electronic mail, switching the address record in the DNS
entry will generally not work as expected. For folks serious about
reliable access to their servers, BGP isn't just the best way, it's
the only way.

Reasons why dynamic DNS fails to perform as expected include:

* Web browser DNS pinning can result in a customer's web browser
holding the old IP address indefinitely.

* Host-level caching of looked up names which discards the TTL.
Remember: your desktop or laptop performs lookups against multiple
name services, e.g. DNS, /etc/hosts, lmhosts, NIS+. DNS TTL is no
longer in scope once the name to address map enters the generic host
lookup mechanism. Most OSes have a fixed timeout of one sort or
another, some old ones as long as 24 hours.

* Custom applications with either IP addresses hardcoded into the
configuration or with getaddrinfo() called only once and the resulting
IP address held for the lifetime of the application.

* Anti-spam systems block IP addresses when receiving large quantities
of email from formerly-quiescent IP addresses. This is a problem if
your mail server sends a lot of email and suddenly switches to a new
sending IP address.

Regards,
Bill Herrin

unless it is a /26, /25 or something shorter.

Even with a /24 things may get messy.

IPv4 is coming to an end, but it is not possible for the customer to get
their own space and use BGP?

Regards,
as

Honestly? Because the end-customers are not technically competent enough to run dual-homed BGP, and we don't want to be their managed service providers on the IT side. And announcing the AT&T space is fine until something goes wrong, and I have to troubleshoot the problem (Customer - "How come AT&T is down, and we're not getting inbound traffic to our servers?", and I discover L3 or CenturyLink isn't accepting my advertisement for some weird reason, but they won't fess up to it for a few frustrating hours)

If they're not technically competent enough to handle BGP, they won't be technically competent enough to deal with solutions that play the short DNS TTL game.

As someone else mentioned in this thread - would colocating the servers be a workable solution for them? Put the servers some place where the redundancy exists already.

jms

There are other elaborate solutions to accomplish this, however all of them would require a competent IT/Network Person to manage the network.

If we were the ISP, we would look at such a case an an opportunity, and become the managed service provider, for a fee (typically a premium), and provide the service.

As service providers, we all complain about the end-customer being a pain, but we often forget that it the the PITA end-customers that give us the ability to earn our daily bread!.... I think too many of us are overworked and providing highly under-paid services for peanuts, where we often overlook at opportunities to get premium value as a PITA, and not worth it...

:slight_smile:

Just my personal two cents,.....

Faisal Imtiaz
Snappy Internet & Telecom

With the risk of starting holy war on how BGP works on dialup and that
providers should permit such, the OP has not specified what transport
methods is in play with AT&T which could limit such an option.

~Seth

My vote goes to the traditional BGP multihomed solution. It's the right way to solve the problem and the easiest to manage.

If getting AT&T to do BGP and buying a BGP capable router (they don't even need full routes...so so anything that'll speak BGP, take a pair of default routes, and handle whatever their traffic level is will do) is too costly[1], another possible option I've not seen mentioned is VPN. They could put one machine/router somewhere with decent redundancy and setup a VPN gateway at their office that connects to the colo'd device.

You might even offer this as a service.

Spammers have been doing this for years. It makes moving their operations easier as their public facing servers get cancelled. All they do is move the VPN server(s) and their systems that do all the "work" remain online and hidden.

[1] If only I had a dollar for every time a client said redundancy was too expensive to have, but when their non-redundant stuff went offline, they claimed to be losing millions of $ per small unit of time.

* Eyeball ISPs' DNS resolvers might tamper with TTL values.

-- SEBASTIAN SPIES lnked.in/sspies vastly.de

I've been doing the suggestion below for many years using the IP addresses that Cogent gives us. All I needed to do is get LOA from them and submit it to my backup ISP. I've never had an issue with my Cogent IP's *not* being advertised by my other ISP and I really don't think there is very much management overhead for the customer once this is setup. I have an SNMP based alerting system (Cacti) set up so I can be alerted if too much traffic ever "shifts" to the backup link.

The client getting their own ASN is the better way to go but you should be able to do the above until that comes through.