Block all servers?

I think it's more complicated than "prevent residential users from
hosting servers".

You're right. As soon as we begin talking about
what all ISPs should do, we are out of the realm
of technical solutions and into the realm of
psychology and politics. After all, we first have
to convince all ISPs that something should be done
and we have to demonstrate that there is a way to
present the action to customers so that the customers
will accept it.

Customers generally don't like ISPs to tell them
"you can't do this" unless there is a very well
reasoned argument attached.

I suggest that people should start thinking about
ways to incorporate security services into their
broadband access products and allow customers the
choice of paying for the security services monthly
to the ISP or paying up front one time by buying
a broadband router.

NANOG could help by collecting together some of the
technical information about the various broadband
routers so that ISPs have an exhaustive and definitive
source to refer to.

--Michael Dillon

P.S. I have always used a router on my Internet connection
even when it was only a dialup connection. Back then it
was a FreeBSD box running TIS firewalls toolkit. Today it's
a Speedstream 510 DSL router.

I agree that Michael is "right on". The social, psychological and
financial issues are in many ways more tricky than the technical issus.
However, I think there are ways to help.

But first some history....

When I signed up for Cable broadband access several years ago, I was
told, "And of course you must not put a router on the network." The of
course was a surprise to me. That immediately meant (at least to me)
that I was going to be exposed to anything that came wandering past my
(dynamically assigned) IP address. Of course I put a router in place.
Was it something really good? No, it was what I could afford. A Linksys
broadband router. Was it misconfigured? Probably - I am after all an
applications guy not a solid network engineer. Did I get it checked out
by the network guys at work? You betcha. Have I eliminated all risk? No.
Have I eliminated "affordable risk?" yes. Since then I have created a
DMZ at home (again not necessarily the most solid in the world), but at
least it has the following effects:

My VOIP telephone line is directly in to the DMZ - that just saves a
hop.
My in home wireless network is just that - in home. The NAT router that
protects it has everything I can think of disabled.
I have access to a couple of servers when I am traveling (both in the
DMZ) so that I can access important files and test development web
sites. There is in theory no public access. In practive, of course it is
wide open - that's why we have DMZs

I also have personal firewalls on all computers whether they travel or
not. Why? Because I want to block outbound activities. I rarely see
anything inbound that is blocked, but I do like the ability of my PFWs
to detect outbound activities and make me confirm/deny access. That is
just good hygeine. Oh btw that firewall monitors inbound email too, so
it becomes a first level virus protector. Real virus protection kicks in
behind that.

Now what could the broadband providers do:

First off, they could incorporate NAT into the DOCSIS or other compliant
cable modems/DSL Modems. Make sure that the NAT router is configured so
that incoming ports are all blocked. Yes that makes it hard for gaming,
so there needs to be an extra capability so that gamers have to
explicitly (at a fee?) get the features opened. That is only a start of
course, because as soon as you do that then there are going to be
vulnerabilities. However, the likelihood of infection/spewing of packets
is reduced somewhat.

Second, in the acceptable use policy for high speed connections, require
a "licence" of some kind. We have licenses/permits for our cars, our
dogs, our burglar alarms, for going fishing,..... Why not for broadband.
Actually I can see many reasons both to do it and not to do it, so this
is clearly an area where debate is reasonable.

Third monitor the bandwidth used (ratios on inbound/outbound) for
example. Actual numbers might be better. For example, at my DMZ router,
it reports the following this morning:

Up time 23:50 (just less than 1 day)
Bytes TX 40,612,318
Bytes RX 370,212,922

These numbers are surprisingly large. However I do run Groove at home
and a lot of data is shared with people all over the world, so the TX
isn't terribly surprising. The RX is monstrous though.

Next stat since power on, the DMZ router has recognized 513 alerts -
mostly ping requests from other Comcast users. Now that would be an
interesting set of cluse if Comcast itself were able to do anything
about it. Lots of Pings (against home machines) are usually indicative
of some kind of problem (yeah, preaching to the choir, I know), so in
this combined modem/router, I could envisage some stats gathering and
reporting on usage - especially things that are somewhat suspicious. Of
course the line is fine between privacy, acceptable use, and risk. The
whole approach does need to be thought through pretty carefully.

I now spend time talking with friends, local groups (Church, city or
whatever) describing the risks. Some people even act on them. Some
people ask for help cleaning up their home systems - especially to
remove pop-ups, improve spam handling and keep porn away from the kids.
What they often don't realize is that the actions they have taken
(downloading gator or hotbar) have caused precisely the effects that
they are trying to guard against. So much of my time spent delousing is
running the cleanup tools (ad-aware, pest patrol, taskinfo to see what's
running), enabling firewalls, recommending that people buy firewalls,
instilling a "use the grc tools" discipline and generally doing what I
can to keep the computers relatively clean. At approximately 3 hours per
computer, I am not making as much headway as I would like!

We therefoe have got to encourage the industry (especially the
responsible leading players) to have things configured by default to
make life safe. Then unsafe behavior becomes a choice rather than a
default.

Sorry for the length of this rant, but I wanted to point out that there
are responsible things happening but more is needed on the part of
vendors.

Remember that what most people seem to want is a gourmet meal with the
ease of a TV dinner! So superb service with no effort. That is not going
to change, so make it hard to do bad things and easy to do good things
and (maybe) we are on the right road.

Thanks for your patience (if you got this far!)

Chris

Michael Dillon said.....

The TOS/AUP for most residential broadband connections already allows the ISP to shut off service or do anything they want to the customer without prior notice. It has been this way for at least 3 or 4 years, since the advent of @Home. Take a look at the TOS/AUP for Comcast, Shaw Cable, MSN DSL or similar...

I know they CAN, but the issue is do they have the mechanisms and
operational capabilities of actually doing so? I would like to see my
cable provider making it hard to do some of the things I do. Not because
I should not be doing them, but those same holes that I exploit
(hopefully in a benign fashion) can be used with malicious intent.

By saying, "If you want to use our service then you must deply this kind
of modem/router" at least makes their insistence explicit. Currently
there is more arm waving than actual adherence to security policy. Thus
we have many poorly configured Windows boxes accessing the internet (and
the WWW) in manners which are to the detriment of everyone else.

IMHO, all consumer network access should be behind NAT.

However, the real solutions is (and unfortunately to the detriment
of many 3rd party software companies) for operating system
companies such as Microsoft to realize a system level firewall
is no longer something to be "added on" or configured later.
Systems need to be shipped completely locked down (incoming
*and* outgoing IP ports), and there should be an API for
applications to request permission to access a particular port or
listen on a particular port (invoking a user dialog).

As for plug-in "workgroup" networking (the main reason why
everything is open by default), when you create a Workgroup,
it should require a key for that workgroup and enable shared-key
IPSEC.

Currently Windows 2000 can be configured to be extremely secure
without any additional software. Unfortunately you must have a
*lot* of clue to configure the Machine and IP security policies it
provides.

    Adam

IMHO, all consumer network access should be behind NAT.

Unfortuantely there are enough protocols and applications
which don't work well behind a NAT that deploying this on
a large scale is not practical. Most gamers require incoming
connections. These are people who willing to pay for bandwidth
so attempting to put them in the "nat only" box will not work.
Also what about folks who need to VPN in to their office
(either via PPTP or IPSEC)? How would you take care of that
situation?

However, the real solutions is (and unfortunately to the detriment
of many 3rd party software companies) for operating system
companies such as Microsoft to realize a system level firewall
is no longer something to be "added on" or configured later.
Systems need to be shipped completely locked down (incoming
*and* outgoing IP ports), and there should be an API for
applications to request permission to access a particular port or
listen on a particular port (invoking a user dialog).

Unfortunately something like this would make the PC close to
useless which is not the intent of the software provider. Thus
you see everything open, security be damned.

As for plug-in "workgroup" networking (the main reason why
everything is open by default), when you create a Workgroup,
it should require a key for that workgroup and enable shared-key
IPSEC.

And joe user will understand this because.....

Currently Windows 2000 can be configured to be extremely secure
without any additional software. Unfortunately you must have a
*lot* of clue to configure the Machine and IP security policies it
provides.

And there lies your problem (among other places)....

bye,
ken emery

IMHO, all consumer network access should be behind NAT.

-snip-

As for plug-in "workgroup" networking (the main reason why
everything is open by default), when you create a Workgroup,
it should require a key for that workgroup and enable shared-key
IPSEC.

  These two requirements are mutually exclusive outside
of a LAN environment, and if you're on a LAN, why require IPSEC?

  Filtering or NAT do not protect you from bad implementation
or bad protocol design. Penalizing users that need (and will pay)
for reasonably accessible two way communication is not the answer,
and never will be.

  --msa

Adam Selene wrote:

IMHO, all consumer network access should be behind NAT.

First of all, this would block way too many uses that currently actually sell
the consumer network connections. "I recommend my competition to do this"

Secondly, it�s very hard, if impossible to come up with a NAT device which
could translate a significant amount of bandwidth. Coming up with one to put
just a single large DSLAM behind is tricky. (OC-12 level of bandwidth)

NAT devices which do OC12 or near don�t come cheap either. This is
(fortunately) not a cost you can sink to the customer as added value.
"Because we lack clue and technology, we just block you for anything and
make you pay for it".

However, the real solutions is (and unfortunately to the detriment
of many 3rd party software companies) for operating system
companies such as Microsoft to realize a system level firewall
is no longer something to be "added on" or configured later. Systems need to be shipped completely locked down (incoming *and* outgoing IP ports), and there should be an API for applications to request permission to access a particular port or listen on a particular port (invoking a user dialog).

Don�t underestimate the painfully slow rate of change in widely deployed systems.
There is a lot of software out there which dates back 15 years or more. Can you
afford to wait even five?

Hardly any of the issues we see today would go away if such an API would be enforced
on the applications because the issues are due to the legitimate applications legitimately
talking to the network with permission.

As for plug-in "workgroup" networking (the main reason why
everything is open by default), when you create a Workgroup, it should require a key for that workgroup and enable shared-key IPSEC.

This is not a bad idea at all. Make sure to save a copy of this message in case
somebody tried to patent this.

Currently Windows 2000 can be configured to be extremely secure without any additional software. Unfortunately you must have a *lot* of clue to configure the Machine and IP security policies it provides.

The box should have a sticker "needs a resident computer mechanic" :slight_smile:

Pete

Also what about folks who need to VPN in to their office
(either via PPTP or IPSEC)? How would you take care of that
situation?

IPSEC works over NATs just fine.

Alex

NAT at the end of OC12 sounds hideous indeed. That's why I would prefer
to see it as part of the modem in the house/business. I am sure (by
guesswork and not by statistics) that a very large number of users would
need relatively simple and secure systems. I guess this because of the
way I see a lot of equipment being used in the groups I talk to. Does
that mean that "one size fits all?" No of course not. Just in the same
way that one car type fits all. If it did, wouldn't Skodas be looking
great right about now?!

Of course from an ISP or other provider's point of view,
uniformity/standardization allows costs to be driven downwards. So in
order to keep costs handled, a non-customizable service is the order of
the day.

By making the NAT router a part of the cable modem at least there is a
lesser chance that a large number of people who want a simple network
connection will have any trouble at all.

Perhaps posting a security bond would be an interesting way of
overcoming some behaviors. General society appears to have strong
financial motivations ("look what I can get for free (theft) by
downloading music", etc.) Well make the standard service cheap, and add
the premium features by control of the NAT router inside the modem from
the support center. Remember that access is a privilege not a right. Of
course as soon as you attempt to control a box from outside, that is
throwing down the gauntlet to the malcommunity. So the NATRouter/Modem
combo would have to be a bit clever. That of course may drive cost
up......

As people who inhabit the network space, I think we do have some
responsibilities to encourage the directions that service provider
choose. If this isn't a good idea, what is? If we assume the following
then we are forced to think broadly:

Most PCs that people buy are configured too broadly with too many
services open and are thus vulnerable.
Most people do not want to mess with keeping their systems safe (for a
variety of reasons).
Most people have become accustomed to relatively inexpensive access
Most people have "brothers-in-law" who know a bit about computers and
can royally screw things up!
Most people know a "really bright 12 year old" who can do "very clever
things with the computer that I can't understand"
Many people assume facility with some terminology and fast typing to be
indicators of knowledge and responsibility.
Many people do the computing equivalent of throwing trash out of the car
window - i.e. not taking any responsibility for polluting the
environment.

These sociological phenomena demand that those who provide the services
provide them responsibly or face the consequences. Sadly the
consequences are societal in impact and don't just affect the providers.

How much benefit would we get if we were to reduce the number of
computers that could possibly be infected with something by 50%, 75%?
How much benefit could we get by knowing which networks were potentially
vulnerable - because they chose to open things up.

I realize that we have a long way to go to get security. It is a bit
like when cars first came out - we could/would drive anywhere.
Eventually we agreed that we, in a given country, would drive on a
particular side of the road. There is no obviously good reason why it
should be one side or the other (as successful drivers in the UK and the
US would agree!), but pick one. Once that happened, then some of the
chaos disappeared.

There is a (possibly true) story that when telephone adoption rates were
analyzed in the 1930s, predictions were that every person in the US
would have to be a telephone operator to keep up with the manual
connecting of calls through plug-switchboards. The expected cross-over
was sometime in the 1950s. Well, with the advent of Subscriber Trunk
Dialing we are all telephone operators today! I see the same things
happening in the computing world, we are all going to have to be network
operators and sesames at some point! Sadly those interfaces are not as
easy and standard as the familiar phone keypad!

Chris

Didn't susan ask for this topic to move off-list? Anybody (no...not
Merit) care to step up and create a nanog-issues list where such
discussions can continue unmolested when the nanog topic police declare an
important topic off-topic?

I can understand how some operators might not want to hang out with the
masses in spam-l or spam-tools, or waste their time with the noise and
kooks in nanae. But these are some pretty serious problems and if we
can't come up with solutions soon, the internet is pretty much totally
screwed.

See more below....

Secondly, it�s very hard, if impossible to come up with a NAT device which
could translate a significant amount of bandwidth. Coming up with one to put
just a single large DSLAM behind is tricky. (OC-12 level of bandwidth)

So do the NAT closer to the edge. If you're providing DSL, do many of
your customers use DSL modems plugged into their PCs (USB, PCI)?, or are
you selling/leasing them DSL routers? In the very beginning, we either
sold or gave PCI or USB DSL modems to our customers, but those were
usually a PITA to support due to problems with windows, driver issues,
hardware becoming unsupported when customers upgraded to the next version
of windows, etc. Now, we only hook up DSL customers using DSL routers,
and all the DSL routers we've ever used can do NAT, so there'd be no need
to try to do NAT at the DSL agg router.

I suspect we could selectively do NAT or not for dial-up customers on our
access-servers...though I'm not sure how the very large (like AS5400,
AS5800) units would fare trying to do NAT for several hundred dial-up
sessions.

But why all this talk of NAT? Even if we all universally deployed it on
monday, it wouldn't solve the problem. All it would do is keep the
spammer/hackers from turning grandma's PC into a web server/proxy. She
can still catch tuesday's email virus which will cause her PC to hang out
in some IRC channel or monitor some web page, and be remotely controlled
for the purpose of sending spam, participating in DDoS floods...and now
things just got much harder to track down. When you get complaints that
a.b.c.d is participating in some kind of attack, how do you tell which of
the dozens or hundreds of customers NAT'd to that IP is
responsible/infected?

Unfortuantely there are enough protocols and applications
which don't work well behind a NAT that deploying this on
a large scale is not practical.

It already is deployed upon a large scale. When I had @Home
in Seattle (one of the first subscribers), I had a 10.x address.
Here in Costa Rica, broadband (cable modem) connections for
the entire country are behind NAT.

Also what about folks who need to VPN in to their office
(either via PPTP or IPSEC)? How would you take care of that
situation?

I use IPSEC and it works fine behind NAT.

Unfortunately something like this would make the PC close to
useless which is not the intent of the software provider. Thus
you see everything open, security be damned.

No. You default open the common and popular internet ports for
outbound, and 90% of users never use anything else.

As for plug-in "workgroup" networking (the main reason why
everything is open by default), when you create a Workgroup,
it should require a key for that workgroup and enable shared-key
IPSEC.

And joe user will understand this because.....

That's the point, he doesn't have to. A "workgroup" becomes a
name + a key/phassphrase instead of just a name. What that
accomplishes is completely hidden.

    Adam

Yes, it does work, on a small scale. However what if your neighbor
wants to IPSEC to the same place (say you work at the same place).
If both of you are NAT'd from the same IP address trying to IPSEC
to the same IP address? I don't believe things will work in this
instance.

bye,
ken emery

Penalizing users that need (and will pay) for reasonably
accessible two way communication is not the answer,
and never will be.

By all means, make a non-NAT IP address a optional premium
service, and hope those that request it are sophisticated enought
to secure their machine.

    Adam

Adam Selene wrote:

By all means, make a non-NAT IP address a optional premium
service, and hope those that request it are sophisticated enought
to secure their machine.

NAT is more expensive to produce, so it should be an optional premium service,
and that seems to be more and more the case.

Pete

NAT is more expensive to produce, so it should be an optional
premium service, and that seems to be more and more the case.

Not necessarily when you consider the cost (in bandwidth,
network reliability and support staff) imposed by worms and kiddies
from other networks scanning your IP space for unsecured machines.

That's not even to mention the cost imposed by compromised systems.
Even if NAT only reduces compromised systems by 20%, that's a
cost savings.

Given that most edge hardware supports NAT, the additional cost
is nominal.

Getting IP space allocation is not without cost either.

    Adam

PS. Is this off-topic for NANOG? If so, I apologize. Given my networks
are repeatedly the victim of distributed DoS attacks from compromised
machines on other networks, it seemed relevant to me.

Adam Selene wrote:

NAT is more expensive to produce, so it should be an optional premium service, and that seems to be more and more the case.
   
Not necessarily when you consider the cost (in bandwidth,
network reliability and support staff) imposed by worms and kiddies
from other networks scanning your IP space for unsecured machines.

NAT boxes are quite unreliable, specially large ones. If you say "put 100000 small ones instead",
that really sounds a support nightmare. And you can filter without having NAT.
(a long time ago NAT was thought to be a security mechanism, that has fortunately
mostly died out)

That's not even to mention the cost imposed by compromised systems.
Even if NAT only reduces compromised systems by 20%, that's a
cost savings.

For the price of a large NAT box, you can buy better security mitigation products
which would allow you to get the wilful spammers, trojaned machines, etc. which
are not saved by your magic box.

Given that most edge hardware supports NAT, the additional cost
is nominal.

My operational experience tells quite a different story.

Getting IP space allocation is not without cost either.

That�s nothing compared to the people complaining about their applications
not working because you want to break their packets.

Pete

why not? We use it here, works fine (with certificates for auth).

   tschuess
             Stefan

Stefan Mink wrote:

> > I use IPSEC and it works fine behind NAT.
>
> Yes, it does work, on a small scale. However what if your neighbor
> wants to IPSEC to the same place (say you work at the same place).
> If both of you are NAT'd from the same IP address trying to IPSEC
> to the same IP address? I don't believe things will work in this
> instance.

why not? We use it here, works fine (with certificates for auth).

OK, let's do this one more time. Many-to-one NAT of a many-to-one ESP VPN
does not work. (Period)

Why? There is no way for the NAT device to map the ESP packets to the
nodes it "hides." You say, "The SPI field is perfect for maintaining
a translation table!" It would be accept for one very big problem. IPsec
is a peer-to-peer protocol. Either side may renegotiate the SAs at any
time. While using IKE[0], the SPI passes the NAT device in the _encrypted_
payloads. The NAT device never sees the SPI until the ESP starts flowing.
Also, keep in mind the SPI is _not_ symmetric.

So, now we have two machines behind a NAT device, and both want to have an
ESP VPN to the same machine. What does the NAT device do when it receives
an ESP packet from the exterior end of the ESP VPN tunnel? How does it
decide which of the internal ends to send it to? The SPI has nothing to
do with the outgoing SPIs (if it even has seen any outgoing ESP yet). It
cannot pull the SPI out of the IKE. You can try timing, if it's a new
SPI, try sending it to the last one that had a IKE conversation, but that
is a guess, what happens if two happen to negotiate at once? And if you
guess wrong, things do not fail and recover for the VPN players.

So, you cannot NAT ESP in the general case. Thus we have all of the rather
grotesque kludges of wrapping the ESP in another transport layer of UDP or
TCP so that the NAT devices have some port numbers to play with. If your
IPsec VPN works through NAT, the NATer is making some assumptions (usually
it only will support a single IPsec end point behind it which solves the
"who do I send the ESP to" problem) or your VPN software has a Draft
or vendor kludge to wrap the IPsec in something more NAT friendly.

Note again that "NAT" above implies "many-to-one NAT." This problem
disappears in a one-to-one NAT configuration where only authentication and
integrity issues, which can be dealt with within IPsec, come into play.

If someone has figured out a way around this, I would love to hear about
it.

[0] The fact you don't need to use IKE to set up SAs makes the problem
even more intractable. A NAT device would have to know of every possible
way to configure SPIs.

From what I've seen it depends on whether the NAT has specific support for IPSEC, and if that support includes support for multiple clients. The NAT box has to keep track of the mapping. I've seen NATs priced based on how many VPN clients they support at a time.

See http://www.dslreports.com/faq/4638