Would anyone mind sharing with me their first-hand experiences with
residential satellite internet?
Right now I am evaluating HughesNet Gen4 and ViaSat Exede and I'm thinking
specifically as a sysadmin who needs to use the uplink for work, not surf.
What are your experiences with the following applications?
-SSH, (specifically interactive CLI shell access)
-RDP
-SIP over SSL
-IPSec Tunneling (should be a non-starter due to latency)
-GRE Tunneling
Thank you,
-Nicholas
Hi Nicholas,
Two-way satellite systems based on SV's in geostationary orbit (like
the two you're considering) have high latency. 22,000 miles out,
another 22,000 miles back and do it again for the return packet.
You'll start around 500ms latency and go up from there. Any kind of
interactive session (like SSH and RDP) will be excruciating.
I'm not aware of any low earth orbit systems providing residential
Internet. Iridium tries to but the bandwidth is pathetic (2400bps or
so). LEO vehicles are only 500-1000 miles up. Latency is high compared
to wireline systems but within usable bounds for interactive sessions.
Regards,
Bill Herrin
SIP will suck. VPN will suck. RDP will suck.
Have you looked to see if you have any local wireless ISPs in your area?
Hit me up offlist if you want me to check for you.
-Mike
It is indeed. This is first-hand in the sense that I once worked for an earth station manufacturer and did a fair bit of work related to this environment, and second-hand in that my sister, for a while, used VSAT connectivity to her home.
The trick in the context is what's called a "performance-enhancing proxy", or PEP. What it does, in concept, is terminate a TCP connection at each earth station and use some form of private protocol over the bird. Cisco RBSCP (which maps TCP connections to SCTP sessions over the bird, http://www.cisco.com/c/en/us/td/docs/ios-xml/ios/interface/configuration/15-sy/ir-15-sy-book/ir-rt-bsd-sat.html) is an example of such a technology. The obvious benefit of a PEP is that it can convince a TCP sender to keep enough data in flight to make good use of the throughput rate of the satellite - you have a start-up issue with the first RTT, but after that it has essentially figured out what the effective window should be and makes that happen. The downside of a PEP is when the application is itself interactive (it's not about rate, it's about responsiveness clocked by end-to-end RTT) or the protocol in question isn't TCP (noting that TCP in IPsec ESP isn't TCP to the PEP - it's IPsec ESP, and can't be goosed along).
In my sister's case, her description of the service was somewhat colorful, and included the word "slow".
Interesting that you say that about sip. We had a client that would use it for sip on ships all the time. It wasn't the best but it worked. Ping times were between 500-700ms.
Regards,
Dovid
I never had good luck with VSAT and SIP. Maybe you had a better kit than I
did 
-Mike
Personally, 500-700ms of delay is well within distinguishable range and causes challenges in verbal communication. If the speakers are both expecting and accustomed to delay like that (e.g. sailors that are used to being hundreds/thousands of miles away from anywhere and any other comms solution sucks anyway), it could be workable.
For regular consumer/business voice applications, 100ms and lower is decent, but above that starts to get into various degrees of suckage.
Just my 2c.
I too have had customers in a previous life where the 500ms delay
really didn't cause any big issue.
Same with SSH and even heavier stuff like SMB. Sure, it was slower
than expected, but I could still saturate the pipe pretty good.
Thing is...the kind of setups where you're getting 500ms delay with
little jitter is stupidly expensive. Those are generally going to be
an SCPC (single carrier per channel) uplink with hopefully something
like IP over DVB providing a large pool of downlink bandwidth. Expect
to pay over 4k per Megahertz (roughly translated to 1 Mbps
unidirectional depending your link budgets) of bandwidth (sometimes
substantially more, depending on what bird and provider you're using).
O3B looks really interesting. I'm not aware of what they're current
state of deployment is, but they've got a MEO (I think) constellation
planned which will help a lot of with that latency. Viasat had
something that looked promising too.
I mean..if you're looking at doing sysadmin type stuff where you're
already going to be pulling out your hair at times, doing so over
hughesnet is going to suuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuck.
My experience with geostationary was that latencies were around 720 ms in practice. Telnet was painful and it turns out my brain didn’t like typing things while I wasn’t getting instant feedback, though I understand there’s software for that problem now.
Reliability was pretty good unless the satellite I was using happened to lose it’s control processors. I was using Panamsat Galaxy 4 when it failed. I don’t know how many angry phone calls we got about why we weren’t answering our pages about the entire network being down before we got into the office. In our case recovery involved getting access to another satellite and having people re-aim the dishes.
My only real recollection besides that was that the signal was bad enough/long enough to induce TCP Silly Window Syndrome; but I can’t imagine anyone’s running an OS that old anymore.
Yes, https://mosh.mit.edu/ is your friend if you want to do things interactively.
Still, satellite is painful, avoid if anything else decent is available.
Would anyone mind sharing with me their first-hand experiences with
residential satellite internet?Right now I am evaluating HughesNet Gen4 and ViaSat Exede and I'm thinking
specifically as a sysadmin who needs to use the uplink for work, not surf.My experience with geostationary was that latencies were around 720 ms in practice. Telnet was painful and it turns out my brain didn’t like typing things while I wasn’t getting instant feedback, though I understand there’s software for that problem now.
Mosh makes latencies like this a lot less painful. Still painful.
Interesting that you say that about sip. We had a client that would use it
for sip on ships all the time. It wasn't the best but it worked. Ping times
were between 500-700ms.
It really depends on your expectations - or more to the point, your end-users' expectations.
I've tested SIP in the lab up to 2000ms RTT. The protocols all hang together and keep working, but it's obviously very much in walkie-talkie mode, you can't hold a normal duplex conversation. 500ms there's more of the talking over each other / "sorry, you go" / "no, you go" dance, but it *is* workable. If your end-user is expecting land-line replacement though...
Regards,
Tim.
Working for an satellite ISP I can give you some technical background.
We're only target enterprise/government/military customers with more
specific use cases because offering satellite based Internet to
residential customers without making them angry while being profitable
is hard. So I've no experience with HughesNet Gen4 or ViaSat Exede as
products in particular but I know the underling platforms. In general
the systems are optimized for fast browsing and VoIP from the own
operator. The modems you'll use with the mentioned services will
include all kinds of acceleration features. General acceleration of
TCP sessions to work around TCP implementation issues in combination
the the high RTT (slow start, behavior during packet loss/high jitter,
window scaling, ...) are standard for these services. The residential
services usually use additional acceleration features like HTTP
prefetching/pushing. That's usually done using a transparent HTTP
proxy which sits at the teleport analyzing all HTTP
requests/responses, download images etc. already before they are
actually requested the the end users browser. They are then pushed to
the modem which will delivery them as soon as the end user requests
them. As a result the end user doesn't have to wait another RTT for
the images etc.. Similar sniffing/spoofing acceleration options are
available for other protocols. But with end to end encryption becoming
more common these days all these transparent higher level acceleration
features of the modems, etc. no longer work. Of course you still can
do the same but you have to move the acceleration to the client
device. That's not very common yet in the satellite industry.
Regarding phone conversations our experience is that the high RTT is
not that much of an problem in practice. People recognize the delay
and wait longer before starting to talk automatically.
But the experience might vary extremely depending on the operators
config and end devices. You need corresponding QoS settings to keep
latency/jitter low and stable. For residential services the return
channel will be most likely time division multiplexed. Once the
network is congested (=profitable for the operator) you'll see the
latency go beyond 1 second more or less often without proper QoS
settings. That of course will completely break your VoIP experience.
You should expect that the operator only has corresponding QoS
settings for their own VoIP service in place. Experience with third
party services might suck due to that. Another issue you might run
into are some VoIP phones. Some of them only support very small jitter
buffers (<10ms) which might cause problems.
IPsec tunneling, GRE tunneling etc. should work perfectly fine unless
it's intentionally filtered. But as soon as you do
tunneling/encryption you should expect that you byepass any
acceleration feature or high priority QoS rule.
Besides that both products you mentioned AFAIK are using Ka-Band spot
beam satellites. There's probably roughly one beam per US state.
Assuming 200 MHz per beam that translate to roughly to a maximum of
600-700 Mbit/s downstream capacity shared by all customers in that
beam (one state). Upstream is probably designed for half of that. This
grouping of customers also makes a simple experience comparison
difficult as your experience will heavily depend on the congestion
level in your beam. From other similar services we already know that
at the launch new customers are happy (always getting the maximum
speed) but as the network fills up they get angry due to bad
performance during peak times.
I really wouldn't recommend a sysadmin to use a geo stationary
satellite based connection for your daily work unless there's no other
way - simply due to the latency. You'll notice a significant
productivity impact.
Best Regards,
Frederik Kriewitz
Reading about SIP made it seem like latency alone is not an issue, aside
from delays which impact verbal communication as previously mentioned. What
is going to be much worse is jitter and packet loss. You can eventually get
used to a significant delay, but dropped calls and chopped sound renders
the service useless.
With modern software implementing a responsive jitter buffer, jitter
shouldn't be much of a problem. Practical effect would be a longer
delay as the receiver buffers enough packets to deal with the measured
variance in receipt times. Perhaps a few chops early in the
conversation as the software grows the buffer.
Not all SIP implementations are equal. Try yours in a high-jitter
environment and see what happens.
High packet loss is deadly. That'll depend on the satellite vendor's
network implementation, the weather, etc. But then high packet loss is
deadly to essentially all IP networking activity.
In situations where a high bit error rate (BER) is endemic, the
layer-2 vendor is expected to redress that with forward error
correction (FEC) and retransmission that trades jitter for loss. I
have no idea which satellite vendors are better or worse about this.
Regards,
Bill Herrin
Latency, latency, latency, RTTs, RTTs, RTTs.
Not an option for anything interactive, very poor for general user-type Internet access.
Thank you all for your responses.
This was exactly the kind of information and opinions I was hoping to find-
way better than reading tea leaves!
Just a minor nitpick - that's 22,300 miles above the equator at sea level.
You're probably closer to 22,500 miles away from the bird (as could your
uplink). That's just rough math adding the tangent of 1500 miles from the
equator in my head (plus the tangent of the curve distance from that base
line and angle of the bird ).
Typically further than that because you're not only not at the same
latitude as the bird, you're not at the same longitude either.
If you want to nitpick. 
-Bill
....
If you want to nitpick.
Well, if you are going to nitpick, the earth is modeled more
closely (but still not precisely) as an oblate spheroid than a
true sphere.
