10 Mbit/s problem in your network

Mailman List Mirror (Read Only)
1

Dear NANOG@,

In light of the recent discussion titled, "The 100 Gbit/s problem in
your network", I'd like to point out that smaller operators and
end-sites are currently very busy having and ignoring the 10 Mbit/s
problem in their networks.

Hotels in major metro areas, for example. Some have great
connectivity (e.g. through high-capacity microwave links), and always
have a latency of between 5ms and 15ms to the nearest internet
exchange, and YouTube and Netflix just work, always, and nearly
flawlessly, and in full HD.

Others think that load-balancing 150+ rooms with Fast Ethernet and
WiFi in every room, plus a couple of conference/meeting rooms (e.g.
potentially more than a single /24 worth of all sorts of devices) on a
couple of independent T1 and ADSL links is an acceptable practice.
Yes, a T1 and an ADSL, with some kind of Layer 3 / 4 balancing! This
is at a time when it would not be uncommon to travel with an Apple TV
or a Roku. And then not only even YouTube and cbs.com don't work, but
an average latency of above 500ms is not unusual in the evenings, and
ssh is practically unusable. (Or sometimes they do the balancing
wrong, and the ssh connections simply break every minute due to the
broken balancer.)

And this happens even with boutique hotels like the Joie de Vivre
brand in the Silicon Valley (Wild Palms on El Camino Real in Sunnyvale
has an absolutely horrible bandwidth even when it's half empty), or
with brand-new properties like Hyatt Place in the hometown of a rather
famous ILEC that has the whole town glassed up with fiber-optics (the
place is less than 2 years old, and Google Maps still shows it as
being constructed, yet independent T1 and ADSL links from two distinct
ILECs is the only connectivity they have!).

How should end-users deal with such broadband incompetence; why do
local carriers allow businesses to abuse their connections and their
own customers in such ways; why do the sub-contracted internet support
companies design and support such broken-by-design setups?

When you are staying at a 3* hotel, should you have no expectations
that you'll be getting at least a 3Mbps pipe and at least an under
100ms average latency, and won't be getting a balancer that would be
breaking up your ssh sessions?

Best regards,
Constantine.

2

"why do the sub-contracted internet support companies design and
support such broken-by-design setups?"

Because they don't know any better and lack the technical clue on how
to implement a network that can support a hotel-full (or half-full) of
people...

But i'm sure they all have their MCSEs and CCNAs so they are qualified :slight_smile:

-mike

3

Not really. Best way to improve this would probably be to get the hotel booking sites to include a separate rating for the internet connectivity.

Up until then, getting Internet connectivity into a hotel is either just cost (in case they offer it for free) or probably a badly performing profit center (because as soon as they try to charge their outrageous prices I imagine take up is abysmal).

If a good performing hotel actually got better rating out of having bad connectivity, and a badly performing hotel got worse rating at rating sites, then I'd imagine that more emphasis would be put on this.

*But* it also requires a standard test that people can run to understand if things are bad or good. For instance, my ISP guarantees to provide 50-100 megabit/s down and 7-10 up on my 100/10 home connection to a speed test site located on neutral ground here in Sweden.

So if the hotels could market themselves with some kind of lowest speed guarantee according to some standard, I believe things would improve. Especially if hotels.com (and others) had a special search item for this, where you could do a search and it would only show results for hotels that guaranteed a certain speed.

4

The problem here is that somehow someone at Hyatt decided that a
regular low-end asymmetrical ~10Mbps/~1Mbps fibre-optic connection
from SureWest could be shared (together with a lousy 1.5Mbps T1 from
T) between 151 rooms, when almost every single person staying in the
hotel has a connection at least twice as big back home, for their own
unshared use! Isn't the logical reasoning simply unbelievable?

I've tried calling their corporate office, but they, apparently, don't
have any kind of a corporate standard for internet connectivity,
saying that it's up to the individual hotels and the local conditions.

How anyone could math out that an average single-digit Mbps
asymmetrical connection can be shared with 151 rooms without any kind
of service degradation or outright periodic halts is rather beyond me.

Out of curiosity, I've tried going onto SureWestBusiness.com web-site
to see what kind of offers they provide for businesses, only to find
out that business FTTH connections max out at 10Mbps down and 1Mbps
up! Yeap, in a major metro area, that's definitely an ILEC for you!

Anyone from SureWest to comment how come residential fibre-optic
connections can have 50Mbps/50Mbps, but businesses that share their
connection with several hundred residents are limited to 10Mbps down
and 1Mbps up max? Why do you even need to have fibre-optics for that
kind of stone-age speeds? And I thought AT&T FTTU was slow!

C.

5

Others think that load-balancing 150+ rooms with Fast Ethernet and
WiFi in every room, plus a couple of conference/meeting rooms (e.g.
potentially more than a single /24 worth of all sorts of devices) on a
couple of independent T1 and ADSL links is an acceptable practice.
Yes, a T1 and an ADSL, with some kind of Layer 3 / 4 balancing! This

Not to be pedantic, but The Last Mile Cache will actually help you to
solve this problem, with a local cache server at the hotel.

The hotel's ISP must participate in TLMC before they, the hotel, can
have a cache server running.

6

Date: Sun, 10 Feb 2013 13:08:04 +0100
From: fredrik danerklint <fredan-nanog@fredan.se>
Subject: Re: 10 Mbit/s problem in your network

Not to be pedantic, but The Last Mile Cache will actually help you to
solve this problem, with a local cache server at the hotel.

The hotel's ISP must participate in TLMC before they, the hotel, can
have a cache server running.

And as a business traveller I want to have the ISP or Hotel cache (aka
be able to read and for others to be found!) my possibly very
sensitive corporate documents exactly _why_ ? The TLMC concept only
has possible applications in certain residential settings. And even
then it's very debatable as to how it could actually improve instead
of overcomplicate and deteriorate the entire service along the route.

Kind regards,
JP Velders

7

> Not to be pedantic, but The Last Mile Cache will actually help you to
> solve this problem, with a local cache server at the hotel.

And as a business traveller I want to have the ISP or Hotel cache (aka
be able to read and for others to be found!) my possibly very
sensitive corporate documents exactly _why_ ?

A VPN or SSH session (which is what most hotel guests traveling for
work will do) won't cache at all well, so this is a very bad idea. Might
improve some things, but not the really important ones.

8

Not to be pedantic, but The Last Mile Cache will actually help you to
solve this problem, with a local cache server at the hotel.

The hotel's ISP must participate in TLMC before they, the hotel, can
have a cache server running.

And as a business traveller I want to have the ISP or Hotel cache (aka
be able to read and for others to be found!) my possibly very
sensitive corporate documents exactly _why_ ?

Since when have you started to publish your sensitive corporate documents on public sites, cause that's what's needed for TLMC to
cache your documents in the first place.

Look,

If a CSP (Content Service Provider - where you host your documents)
does not want to have it's content cached, they don't need too. The
cache server(s) at the ISP:s around the world will then _not_ be able
to cache it.

The traffic will in this case, will be loaded directly from the CSP.

The TLMC concept only
has possible applications in certain residential settings.

No. It will help the ISP:s to distribute their loads in their network.

And even
then it's very debatable as to how it could actually improve instead
of overcomplicate and deteriorate the entire service along the route.

How about those who have limited bandwidth to the Internet? Like
ferries, trains, buses or satellite links...

9

Date: Sun, 10 Feb 2013 17:33:04 +0100
From: fredrik danerklint <fredan-nanog@fredan.se>
Subject: Re: 10 Mbit/s problem in your network

Since when have you started to publish your sensitive corporate
documents on public sites, cause that's what's needed for TLMC to
cache your documents in the first place.

You seem to be mistaken that any bandwidth issue will be remedied by
TLMC. A significant number (well over the 50% mark I'd wager) will not
be remedied. This thread was started over such a subject.

The Apple TV cited as an example was an example. Travellers, be they
corporate or leisure, have significant networking needs that the TLMC
cannot address. Just think of "The Cloud" (yes, I'll go and flog
myself for bringing it into a discussion on NANOG), where people are
storing their (semi-) private documents or files - in the end it's
similar to connecting back to the office to access the fileserver.

How about those who have limited bandwidth to the Internet? Like
ferries, trains, buses or satellite links...

And pray tell me, why should they all have TLMC's ? If the concepts
and technologies underlying "The Internet" were invented to have the
same ubiquitous speed for all, I think it would have a fairly
different design. Now if you're a content provider, then yes I can
imagine why you'd like everybody else to pay for better ways to
deliver your content without having to pay for it yourself.

The examples you cite are the prime examples where users either bring
their own entertainment, or it is already provided. On a long airplane
flight it is quite uncommon to not have some offering with movies or
audio, free or paid is outside scope since TLMC's won't be free
either. After all, when I sleep or travel on the road my bandwidth use
is vastly different from when at home, work or at a hotel.

Within this discussion we're talking about the actual last mile. A
proxy or cache won't be of any use if the users can't get to it with
sufficient bandwidth to make it work anyway.

Kind regards,
JP Velders

10

Dear NANOG@,

In light of the recent discussion titled, "The 100 Gbit/s problem in
your network", I'd like to point out that smaller operators and
end-sites are currently very busy having and ignoring the 10 Mbit/s
problem in their networks.

Hotels in major metro areas, for example. Some have great
connectivity (e.g. through high-capacity microwave links), and always
have a latency of between 5ms and 15ms to the nearest internet
exchange, and YouTube and Netflix just work, always, and nearly
flawlessly, and in full HD.

Others think that load-balancing 150+ rooms with Fast Ethernet and
WiFi in every room, plus a couple of conference/meeting rooms (e.g.
potentially more than a single /24 worth of all sorts of devices) on a
couple of independent T1 and ADSL links is an acceptable practice.
Yes, a T1 and an ADSL, with some kind of Layer 3 / 4 balancing! This
is at a time when it would not be uncommon to travel with an Apple TV
or a Roku. And then not only even YouTube and cbs.com don't work, but
an average latency of above 500ms is not unusual in the evenings, and
ssh is practically unusable. (Or sometimes they do the balancing
wrong, and the ssh connections simply break every minute due to the
broken balancer.)

And this happens even with boutique hotels like the Joie de Vivre
brand in the Silicon Valley (Wild Palms on El Camino Real in Sunnyvale
has an absolutely horrible bandwidth even when it's half empty), or
with brand-new properties like Hyatt Place in the hometown of a rather
famous ILEC that has the whole town glassed up with fiber-optics (the
place is less than 2 years old, and Google Maps still shows it as
being constructed, yet independent T1 and ADSL links from two distinct
ILECs is the only connectivity they have!).

Network is rather far outside the core competency of most hotel manangement corporations and REITS assuming they have any at all.

There's fairly abundant reasons reasons why they or their contractors might not be very good at it or be able to deliver a decent service at the pricepoint they have budgeted.

When you consider the alternative is bringing your own (in the form of HSDPA/LTE) and that might in many cases be an order of magnitude faster, it's hard to imagine how most of them would address that in a fashion that generates in cost recovery on the service or pricing power on rooms.

11

You seem to be mistaken that any bandwidth issue will be remedied by
TLMC. A significant number (well over the 50% mark I'd wager) will not
be remedied. This thread was started over such a subject.

And to save 1 - 5 Mbit/s of this bandwidth is wrong, how?

The Apple TV cited as an example was an example.

If the TV Show/films/movies/etc.. is static content, then we
should be able to cache it, at the hotel's cache server.

Travellers, be they
corporate or leisure, have significant networking needs that the TLMC
cannot address. Just think of "The Cloud" (yes, I'll go and flog
myself for bringing it into a discussion on NANOG), where people are
storing their (semi-) private documents or files - in the end it's
similar to connecting back to the office to access the fileserver.

(We have 1 - 5 Mbit/s of more bandwidth for these services).

What you are talking about here is dynamic content, which should not
be cached at all and everyone knows this.

How about those who have limited bandwidth to the Internet? Like
ferries, trains, buses or satellite links...

And pray tell me, why should they all have TLMC's ?

I'm not saying that they should have a cache server. I'm saying
that they could.

Now if you're a content provider, then yes I can
imagine why you'd like everybody else to pay for better ways to
deliver your content without having to pay for it yourself.

It does matter how you are going to try to solve this, it is always
the customer who is going to pay in the end.

Within this discussion we're talking about the actual last mile.

I call it "The Last Mile Cache", TLMC

A proxy or cache won't be of any use if the users can't get to it with
sufficient bandwidth to make it work anyway.

So, as long as a user does not have enough bandwidth, they should not
have a cache server on their side, correct?

12

Oh.

*Now* I understand the problem.

Do you really think that the content providers, and the delivery systems
they purposefully choose for that, actually make that possible, much less
practical?

Even in your country, much less the countries of, um, North America?

Cheers,
-- jra

13

Hello,

The Apple TV cited as an example was an example.

If the TV Show/films/movies/etc.. is static content, then we
should be able to cache it, at the hotel's cache server.

The question is "how much it helps". Everyone can easily find that caching Google logo is possible, also some pictures from big media companies webs. Also some program updates may help.

I'm not sure what will be cache hit ratio from YouTube (because of very log tail) or facebook pictures.

Number of hotel guests is really limited.

How about those who have limited bandwidth to the Internet? Like
ferries, trains, buses or satellite links...

And pray tell me, why should they all have TLMC's ?

I'm not saying that they should have a cache server. I'm saying
that they could.

The question is:
Is investment for buying TLMC and operation costs for TLMC profitable for the hotel?

Seems to me like question:
Is investment and operation costs for high bandwidth connection profitable for the hotel?

The discussion is really about the hotel business, the best way for the community is to provide a feedback for hotel managers what is expected (for free and for the money). And, eventually, provide a kind of metric. What is really annoying, is when you pay for broken connection.

         Regards
                 Michal

14

*Now* I understand the problem.

Do you really think that the content providers, and the delivery systems
they purposefully choose for that, actually make that possible, much less
practical?

(I'm not sure that I understand what you mean with that sentence).

If you mean that a CSP already has an agreement with a CDN, why
should they change it to something else since it works right now
for them?

If this is what you mean, yes the can add TLMC to their mix as well and continue with whatever they are using today for delivering their
contents.

Even in your country, much less the countries of, um, North America?

I think that has more to do with the CSP since they are actual needed
in the first place. After that it is the ISP, which in turns adds the
possibility for a end-user/customer/residence to set-up their own
cache server at home.

15

Dear NANOG@,

In light of the recent discussion titled, "The 100 Gbit/s problem in
your network", I'd like to point out that smaller operators and
end-sites are currently very busy having and ignoring the 10 Mbit/s
problem in their networks.

Hotels in major metro areas, for example. Some have great
connectivity (e.g. through high-capacity microwave links), and always
have a latency of between 5ms and 15ms to the nearest internet
exchange, and YouTube and Netflix just work, always, and nearly
flawlessly, and in full HD.

Others think that load-balancing 150+ rooms with Fast Ethernet and
WiFi in every room, plus a couple of conference/meeting rooms (e.g.
potentially more than a single /24 worth of all sorts of devices) on a
couple of independent T1 and ADSL links is an acceptable practice.
Yes, a T1 and an ADSL, with some kind of Layer 3 / 4 balancing! This
is at a time when it would not be uncommon to travel with an Apple TV
or a Roku. And then not only even YouTube and cbs.com don't work, but
an average latency of above 500ms is not unusual in the evenings, and
ssh is practically unusable. (Or sometimes they do the balancing
wrong, and the ssh connections simply break every minute due to the
broken balancer.)

And this happens even with boutique hotels like the Joie de Vivre
brand in the Silicon Valley (Wild Palms on El Camino Real in Sunnyvale
has an absolutely horrible bandwidth even when it's half empty), or
with brand-new properties like Hyatt Place in the hometown of a rather
famous ILEC that has the whole town glassed up with fiber-optics (the
place is less than 2 years old, and Google Maps still shows it as
being constructed, yet independent T1 and ADSL links from two distinct
ILECs is the only connectivity they have!).

Network is rather far outside the core competency of most hotel manangement
corporations and REITS assuming they have any at all.

There's fairly abundant reasons reasons why they or their contractors might
not be very good at it or be able to deliver a decent service at the
pricepoint they have budgeted.

When you consider the alternative is bringing your own (in the form of
HSDPA/LTE) and that might in many cases be an order of magnitude faster,
it's hard to imagine how most of them would address that in a fashion that
generates in cost recovery on the service or pricing power on rooms.

Well, let's do a thought experiment on cost comparison to put things
into perspective.

* How much do they pay for the actual pipe?
* How much do they pay for the outsourced maintenance and the
technical support contract? (Tech support is outsourced to New York.)
* How much do they pay to an average in-house employee?
* How much do they pay to receive and deliver the newspapers in the
mornings to at least half the rooms? (Potentially more than 2000
copies a month at just half the rooms.)

And:

* How much do they charge per night per room? Then times 151, then
times 31? Something along the lines of 200'000$/mo to 600'000$/mo?

Unless my guesstimates are wrong, even 100$/mo for the actual pipe is
completely and utterly nothing compared to all the other expenses (and
the revenue), and I think their 10Mbps down / 1Mbps up fibre-optic (or
ADSL?) connection from SureWest Business is even cheaper than that
(although their AT&T T1 is probably not, but then it's still rather
unclear why they even need to load-balance 151 rooms over a T1 in a
brand-new building in a major metro area in California in 2013
anyways).

Even at 500$/mo for the pipe it would still be SEVERAL TIMES cheaper
than delivering the daily newspaper to every guest every morning
alone.

Even with just a couple of speed-related tech-support calls per month,
it might still be cheaper to upgrade to a better pipe than to have the
guests needing to call the support and being inconvenienced with even
a very slight likelihood of not returning.

Besides, why would leisure and business travellers need a
complementary newspaper in the morning, but would be OK with 200ms
average latency and 300ms std-dev / jitter, and not being able to
watch YouTube, video conference or work remotely comfortably, is
something that I'm yet to comprehend.

Yet the local management of the hotel thinks that there's no problem.
"Internet works." "It's been fixed." "We've fixed your internet, sir
-- the system has been rebooted; please check again in a few minutes."
"Yes, sir, some customers occasionally do complain that the video
streaming doesn't work; but most people just check their email, and it
works." "Sir, I personally do have a 30Mbps connection at home, but
here at our hotel 10Mbps (+ 1.5Mbps) is shared between 151 suites;
what's your problem again, sir?"

I think it is honestly laughable that they must be spending about
3$/day (the price of coffee at some Starbucks locations) for their
actual internet pipe for the whole hotel. Also, do people really
think that a 10/1 connection is still a 10/1 "broadband" even when
shared with hundreds of folks?

Best regards,
Constantine.

16

When staying at Homestead a few years back, they would close my Internet
connection, because I was downloading movies via peer to peer. It took me
a while and escalating to a relatively competent network engineer to
figure it out: "Mate, I don't have any p2p software installed, may be my
computer is hacked, tell me what traffic you see that triggers your
system, so I can investigate". I came down that they did not like my Skype
trying to re-establish connections with contacts in Asia/Pacific (where I
lived then), instead of the USA.

I also organized conferences, and putting more than 20 people (with
various OS/hacked machines) on the same access point, is not standard
operations as in a company, you need some experience with that, something
that some ISPs (who were sponsoring the Internet) failed to understand.

17

Constantine A. Murenin wrote:

The problem here is that somehow someone at Hyatt decided that a
regular low-end asymmetrical ~10Mbps/~1Mbps fibre-optic connection
from SureWest could be shared (together with a lousy 1.5Mbps T1 from
T) between 151 rooms, when almost every single person staying in the
hotel has a connection at least twice as big back home, for their own
unshared use! Isn't the logical reasoning simply unbelievable?

I've tried calling their corporate office, but they, apparently, don't
have any kind of a corporate standard for internet connectivity,
saying that it's up to the individual hotels and the local conditions.

Yes, that is reasonable.

Just saying "Internet connectivity" is too broad for world wide
hotel operators. It's up to the local conditions, of course!!!

When I was at a resort in an isolated island in Pacific ocean
three yeas ago, only connectivity of the resort was through
satellite, shared by tens of rooms. There, of course, was no
local 2G/3G/4G services.

When I was at a hotel in Geneva about 10 years ago, the hotel
advertised to be Internet-capable, even though the hotel
only offered telephone connectivity to local and international
dial-up ISPs.

When I was at a resort in Africa more than 15 years ago,
there was no telephone connectivity, except for one by
private wireless relay maintained by the hotel for its
reservation and other its own business purposes.

Differentiating the "Internet connectivity" of hotels as:

  (No?) Internet connectivity (dial up)

  Internet connectivity (satellite)

  Internet connectivity (DSL)

  Internet connectivity (FTTH)

could be meaningful, for which NANOG could act for or against
it, but there can be no standard for "Internet connectivity"
defined world wide, unless you accept 110bps dial up good
enough.

            Masataka Ohta

PS

You can, of course, pay for private satellite connectivity
at certain bps available world wide.

18

If you don't like it, let them know, and stop providing them with your business. Money talks. They'll either decide
they need to invest in good Internet, or they'll decide that for their customer demographic it just isn't worth it.

I personally think you're being unreasonable on the bandwidth and latency expectations, Hotel Internet connections are
there as a convenience rather than some kind of business grade connection. If you are expecting a top quality
connection, expect to pay by the GB - so that greedy patrons watching Netflix HD pay for their bandwidth.

Broken SSH connections would annoy me though.

Graham.

19

  Internet connectivity (FTTH)

Fibre-To-The-Hotel, eh? :slight_smile: :slight_smile: :slight_smile: :slight_smile: :slight_smile:

-JimC

20

I personally think you're being unreasonable on the bandwidth and latency
expectations, Hotel Internet connections are
there as a convenience rather than some kind of business grade connection.

Hey, the name business grade connection is prejudiced, as if to imply,
that only businesses get it. I think the expectation from a
visitor, is only, that their internet experience will be comparable
to their home cable/dsl internet.

If it's not... that's fine, but they should provide disclosure of
that, whenever mentioning the feature, before a reservation could
be made.

Of course there can be no worldwide standard, but there should be a
standard, based on what is normal in the country.

If the advertising tells you, that the room has electric lights, air
conditioning, and cable tv; you don't want to see a room that just
has a 9 volt battery, a little LED lamp, as your light source -- a
portable battery powered fan.
And a single shared television in the lobby, plugged into a cable
provider that charges a per-minute fee to visitors wishing to see
anything other than channel 3.