VPN over slow Internet connections

Dear all,

Can anyone share any thoughts or experiences for VPN links running over slow Internet connections, typically 2kB/s - 3kB/s (think 33.6k modem)?

We are looking into utilising OpenVPN for out-of-office workers who would be running mobile broadband in rural areas. Typical data across the wire would be SQL queries for custom applications and not much else.

Some initial thoughts include...

   * How well would the connection handle certificate (>= 2048 bit key) based authentication?
   * Is UDP or TCP better considering the speed and possibility of packet loss (no figures to hand)?
   * Is VPN over this type of connection simply a bad idea?

Many thanks in advance.

Kind regards,
Ben Whorwood

If I had to guestimate, the performance would be horrible considering the VPN overhead in itself.

You can't choose UDP or TCP, that is all based on the applications being used within the tunnel.
So the apps will decide what protocols they will need to use, which will then be encapsulated by IPSEC.

It could work, but you may not be happy and it may not provide the desired performance that you need
to be productive....

Ben Whorwood (bw-ml) writes:

Some initial thoughts include...

  * How well would the connection handle certificate (>= 2048 bit
key) based authentication?
  * Is UDP or TCP better considering the speed and possibility of
packet loss (no figures to hand)?

  I'd go for a UDP tunnel, as you wouldn't have to renegotiate
  a TCP session for the tunnel *and* whatever connection you've
  got going through that.

  * Is VPN over this type of connection simply a bad idea?

  I don't think it's a particularly bad idea. But why don't you
  make you own tests using FreeBSD/dummynet, simulating 1-2%
  packet loss, limit bandwidth to 33 Kbit/s, and corresponding
  latency (say 100ms).

  I'd say your biggest concern won't be the VPN (you can make
  it completely stateless with static keys), but whatever protocol
  you've got running on top of that, and how it deals with the
  loss.
  
  Cheers,
  Phil

There's not that much overhead--your certs should be ok. TCP for SQL would just make sense. I personally wouldn't want to do what you are contemplating. Here's some stuff to think about:

1. your modems will not be able to do compression. You can't easily compress random data (e.g. encrypted).
2. you won't get 33.6 unless your phone lines are pristine. You better plan on 28.8--if you are lucky.
3. I would hone my SQL sharply so it produces the smallest most relevant data sets possible.

4. you might want to give them some kind of termnial/shell access for doing their SQL remotely, instead of from home. Telnet or SSH. If you used SSH you could obviate using a separate VPN, you could use -C for compression, and you could do your SQL on the server side (or the on-site side)--all in all a speedier alternative.

--Patrick Darden

   * How well would the connection handle certificate (>= 2048 bit key)
based authentication?

It will hiccup for a moment (maybe a quarter or half second) for the data. The
certificate exchange is the least of your problems.

   * Is VPN over this type of connection simply a bad idea?

Well, 33.6k is a Bad Idea right there. :slight_smile: But if you're stuck with that
for technical reasons, but need a VPN for security reasons, it won't
be all *that* much worse, unless you're doing a lot of SSH or similar
short-packet single-keystroke traffic, where the VPN overhead will
start being a bit painful. Shouldn't be too hard to model the traffic
involved to see if it's too painful - FreeBSD has dummynet IIRC.

We are looking into utilising OpenVPN for out-of-office workers who
would be running mobile broadband in rural areas. Typical data across
the wire would be SQL queries for custom applications and not much else.

I agree with Patrick, SSH would do nicely. You could even setup a
tunnel, and the sql queries could hit localhost:3306 (for mysql for
example) and connect, encrypted to a remote server. At 2-3KB/sec, I
wouldn't expect the ability to have too many people concurrently
connecting to the database.

Can anyone share any thoughts or experiences for VPN links running over slow
Internet connections, typically 2kB/s - 3kB/s (think 33.6k modem)?

We are looking into utilising OpenVPN for out-of-office workers who would be
running mobile broadband in rural areas. Typical data across the wire would
be SQL queries for custom applications and not much else.

Some initial thoughts include...

* How well would the connection handle certificate (>= 2048 bit key) based
authentication?

Fine. The certificate isn't sent very often and is only 256 bytes when
it is sent.

* Is UDP or TCP better considering the speed and possibility of packet loss
(no figures to hand)?

TCP is more likely to pass firewalls at the user's end, especially if
you put your VPN server on port 443. UDP will allow the user's various
sessions to recover from packet loss independently (i.e. faster). I
would pick UDP and provide an alternate TCP configuration for users
who experience trouble.

* Is VPN over this type of connection simply a bad idea?

No worse than using this slow a connection in the first place. VPN
overhead is 5% to 10% tops. I would use a split tunnel though; let
general internet destinations go directly through the Internet
connection rather than through the VPN.

Regards,
Bill Herrin

I vote for Patrick's idea of allowing the end user to remote into a machine where the SQL resides.

This would eliminate a lot of potential issues....wish I had thought of that first!!!

If you haven't deployed your VPN environment yet I would seriously consider using SSL VPN instead of IPSec as your tunneling protocol. SSL VPN gives you a lot more options than IPSec.

Gary

Hi Gary,

Ben was looking at OpenVPN, not IPSec.. He seems to have a field
application he wants to talk directly to his SQL server. An SSL VPN
that does such a good job protecting interior web services, sometimes
also called a Web VPN or a clientless VPN, may not be on the table.

Regards,
Bill Herrin

I third this idea. Using screen would be a good idea as well.

This reminds me a project I worked on last century were we had people direct dialing our facility over modems to use a custom DB front end presented using Citrix.

One of the big challenges was dropped calls. Persistence is your friend under these circumstances. At least the end users don't lose work.

Dear all,

Can anyone share any thoughts or experiences for VPN links running over slow Internet connections, typically 2kB/s - 3kB/s (think 33.6k modem)?

We are looking into utilising OpenVPN for out-of-office workers who would be running mobile broadband in rural areas. Typical data across the wire would be SQL queries for custom applications and not much else.

Some initial thoughts include...

* How well would the connection handle certificate (>= 2048 bit key) based authentication?

Should be fine. Might take 30 seconds to connect, but after connection it makes no difference

* Is UDP or TCP better considering the speed and possibility of packet loss (no figures to hand)?

Since you're running TCP applications (database connections), you definitely want UDP. TCP-in-UDP behaves correctly in the presence of packet loss, TCP-in-TCP behaves horribly (it causes exponential backoff on the outer VPN connection, which causes queueing of the inner packets when they should be dropped. I've seen 20-30 second latencies with TCP VPNs over slow/lossy links).

* Is VPN over this type of connection simply a bad idea?

It shouldn't be any worse than running directly over the connection. With a UDP VPN it does packet-by-packet encapsulation, so it only adds the fixed per-packet overhead.

Nothing like getting into the groove, then losing your connection, waiting for the modem to dial back up
and then try to figure out what you were just doing!!! Again, it goes back to what I mentioned, it "could" work
but how will that affect your overall productivity.

Is over the air 3G or 4G not available? I'm assuming that modem is being used because broadband is not in the area....

I would think so too. When I first moved to the States I lived in rural Oregon and used an (attempted) always on dialup for years. I could mostly get a steady 56K connection, except when there was a nest of mice in the local phone exchange/switch/box thing (whatever they're called) or a tree fell in the wrong place or it rained for weeks. Then it would be a tad slower (the fallen tree would only cause an electricity outage :wink: the phoneline often was still fine), but never go below 33.6K. Though the mice were annoying in causing a disconnect every half hour or so.

Anyways, at 56K speeds 3/4 of the time and I could stream low bandwidth real audio radio and do some browsing all at the same time whilst chatting. So going from that I would say that a 33.6K connection dedicated to just VPN connectivity would work fine for things such as sql queries.

Greetings,
Jeroen

Nothing like getting into the groove, then losing your connection, waiting for the modem to dial back up
and then try to figure out what you were just doing!!! Again, it goes back to what I mentioned, it "could" work
but how will that affect your overall productivity.

Is over the air 3G or 4G not available? I'm assuming that modem is being used because broadband is not in the area....

The OP said:

Can anyone share any thoughts or experiences for VPN links running over slow Internet connections, typically 2kB/s - 3kB/s (think 33.6k modem)?

We are looking into utilising OpenVPN for out-of-office workers who would be running mobile broadband in rural areas.

It looks like they are using the fastest mobile (cellular) access solutions available in these rural areas with throughput speeds that are *about* the same as dial-up modem speeds. Mobile broadband (sic) is tuned for faster downloads, upload speeds can be very very slow, especially if you are on the edge of their service range. (I used Verizon EVDO devices for several years, have extensive first-hand experience in how slowly they work in many rural areas.)

jc

Assuming the slow connection, I would guess there would be some packet loss to go along with it which could make not an ideal situation even worse.

We've all done things we're not proud of because of business need, from the outside looking in I would definitely hammer home very low expectations before rolling it out. If they think it's going to be great they've got another thing coming, if they expect it to be barely usable they just might be pleasantly surprised.

-wil

Dear all,

Can anyone share any thoughts or experiences for VPN links running over slow Internet connections, typically 2kB/s - 3kB/s (think 33.6k modem)?

We are looking into utilising OpenVPN for out-of-office workers who would be running mobile broadband in rural areas. Typical data across the wire would be SQL queries for custom applications and not much else.

Some initial thoughts include...

* How well would the connection handle certificate (>= 2048 bit key) based authentication?

You're doing this rarely; it shouldn't be a problem.

* Is UDP or TCP better considering the speed and possibility of packet loss (no figures to hand)?

For your application or for the VPN? For the VPN, I *strongly*
suggest you use UDP, or you're going to get dueling retransmissions
and spend a lot of time sending many copies of the same thing. Consider:
if a packet is dropped, either due to line noise or queuing delay for
the slow link, the sending TCP will resend. If you're using TCP for
OpenVPN, that session's TCP will resend. Of course, the TCP running
on top of it will resend as well, so you'll get two copies of the data
sent to the application's TCP, wasting precious bandwidth. If, on the
other hand, OpenVPN is running UDP, it won't resend; the application's
TCP will, so you'll only get one copy.

I should note: IPsec, being datagram-based, will also work well. PPTP,
which runs over TCP as far as I know, will suffer all of the ills I just
outlined.

I'm assuming that your application is using TCP. Unless the data
characteristics are such that you're able to fit every query and every
response into a single packet, you'll spend more effort (and probably
bandwidth) doing your own retransmissions, backoffs, segementation,
etc.

* Is VPN over this type of connection simply a bad idea?

If you do it correctly, a VPN is actually better: you can assign a
static internal IP address to each certificate. If the modem connection
drops, when you reconnect the applications will still have the same
IP address, so their connections won't be interrupted.

You do have to watch out for queue limits -- as Jim Gettys has reminded
us, many of today's queues are far too long, so we're not getting the
very beneficial effects of early drop and consequent TCP slow-down.
This will require tuning your end nodes OS and/or the router at your
head end. Use active queue management (e.g., RED), and consider a
priority queueing scheme. Watch out for other applications -- I've had
trouble with MacOS's Mail.App on slow links; it's gotten very confused
and more or less forgotten about my mail folders, with the consequent need
to rebuild them, reactivate my sorting rules, etc. (Note that this paragraph
applies whether or not you're using a VPN; it's the effect of a slow
connection, not the crypto.)

The real VPN question is what the overhead is. I've never calculated
it for OpenVPN; I did for IPsec some years ago (long enough ago that
IPsec was still using DES or 3DES because AES and its 16-byte blocksize
didn't exist); using average packet size distribution, it came to (as
I recall) about 12%. That's unlikely to make or break your system.
However, there's no substitute for real data -- what do your packets
look like? Fairly obviously, the shorter the packet, the higher the
overhead percentage. Someone suggested trying it using a FreeBSD
flakeway; that's a good idea.

In short -- if your VPN is set up properly, any ill effects are much
more likely to be from the link speed itself, not the VPN.

    --Steve Bellovin, https://www.cs.columbia.edu/~smb

Steven Bellovin (smb) writes:

I should note: IPsec, being datagram-based, will also work well. PPTP,
which runs over TCP as far as I know, will suffer all of the ills I just
outlined.

  PPTP uses 1723/tcp for control, but the tunneled traffic is GRE,
  so that would work fine as well.

If you do it correctly, a VPN is actually better: you can assign a
static internal IP address to each certificate. If the modem connection
drops, when you reconnect the applications will still have the same
IP address, so their connections won't be interrupted.

  Absolutely, that's the case with OpenVPN, if you assign static IPs to
  each profile. PPtP can do this as well, for instance using MPD.
  Very big advantage in fact.

Someone suggested trying it using a FreeBSD flakeway; that's a good idea.

  Using a dummynet box as a router (or bridge for that matter), you have
  the benefit that you can run tcpdump on the trafic, and record the packet
  sizes with and and without VPN, then derive the actual observed overhead.

  Cheers,
  Phil

Steven Bellovin (smb) writes:

I should note: IPsec, being datagram-based, will also work well. PPTP,
which runs over TCP as far as I know, will suffer all of the ills I just
outlined.

  PPTP uses 1723/tcp for control, but the tunneled traffic is GRE,
  so that would work fine as well.

Ah, thanks for the correction.

If you do it correctly, a VPN is actually better: you can assign a
static internal IP address to each certificate. If the modem connection
drops, when you reconnect the applications will still have the same
IP address, so their connections won't be interrupted.

  Absolutely, that's the case with OpenVPN, if you assign static IPs to
  each profile. PPtP can do this as well, for instance using MPD.
  Very big advantage in fact.

Yup, I've done this myself with OpenVPN.

    --Steve Bellovin, https://www.cs.columbia.edu/~smb

Is this actually how OpenVPN's TCP encapsulation works? I'd be curious to
know. It isn't how Cisco's TCP/10000 encapsulation works, at least not with
the IOS devices I have experience with.

Cisco's TCP/10000 looks like TCP to a firewall, but it really isn't. There
is no reliability -- no retransmits, etc. It's pretty close to UDP behavior
but with a TCP header, which was confusing to troubleshoot at first but
quickly made perfect sense to me for the reasons you state above.

-Terry