RE: 69/8...this sucks

Mailman List Mirror (Read Only)
1

We don't need the adminstrative headache of ICANN/ARIN/RIRs on this. Someone could just do it with a private ASN and advertise the route with an arbitrarily null routed next-hop.

That doesn't solve the problem of bad filters on firewalls.

The problem is lots of books/webpages/templates/etc. say filter bogons. People not smart enough to understand the responsibilities of doing so implement it and forget it. Instead of trying to beat up on the large numbers of people who lack sufficient clue, why isn't the pressure turned to the authors that are irresponsibly and blindly recommending the wide spread use of these filters? I would think we would have more success targeting the people authoring this stuff. There are at most hundreds of authors. There is at least thousands of twits...

Funny the media gets all excited about BGP security and dDos attacks against a root nameserver yet no one ever seems to mention the real scalability issues like that we can't allocate large parts of the net because many network operators aren't bright enough to update filters.

Frank

2

We don't need the adminstrative headache of ICANN/ARIN/RIRs on this.
Someone could just do it with a private ASN and advertise the route with
an arbitrarily null routed next-hop.

That's a non-solution that will never happen. How many networks are going
to trust joe somebody to inject null routes into their backbone? Will
UUNet/Sprint/C&W/Level3/etc. trust me or Rob to tell them what's a bogon
and what's not? I really doubt it. They might have an easier time
trusting their local RIR, but I wouldn't be surprised if they didn't.

I realize this sort of thing worked early on with the RBL, but that was
for a different purpose. For those who took the RBL via BGP, I suspect
the benefit of blocking spammers from their networks outweighed the risk
of RBL abuse and people trusted Vixie to be objective and honest.

That doesn't solve the problem of bad filters on firewalls.

Several people pointed that out earlier. Botched / outdated firewall
configs may be a bigger problem than BGP filters. For a glimpse at why,
see
http://groups.google.com/groups?q=69.0.0.0%2F8&ie=UTF-8&oe=UTF-8&hl=en&btnG=Google+Search

The problem is lots of books/webpages/templates/etc. say filter bogons.
People not smart enough to understand the responsibilities of doing so
implement it and forget it. Instead of trying to beat up on the large

Worse is that there are pages and pages full of links to usenet posts with
these outdated bogon filters. Books and web pages can be updated. The
usenet archive isn't going away and won't be revised. People who don't
know any better are going to continue to misconfigure bogon filters
indefinitely unless something is done to periodically whack some sense
into them.

Funny the media gets all excited about BGP security and dDos attacks
against a root nameserver yet no one ever seems to mention the real
scalability issues like that we can't allocate large parts of the net
because many network operators aren't bright enough to update filters.

I know some writers watch nanog for potential stories. Wake up guys, this
should be one...if not for the news value "ARIN gives out unusable IPs,
future of the Net in question", then at least for the public service value
of getting the word out that bogon filters need to be maintained and kept
up to date or they do more harm than good.

3

I know some writers watch nanog for potential stories. Wake up guys, this
should be one...if not for the news value "ARIN gives out unusable IPs,
future of the Net in question", then at least for the public service value
of getting the word out that bogon filters need to be maintained and kept
up to date or they do more harm than good.

And, please in said story, try not to use the term bogon or at least define
it. The term "bogon" gets many funny looks, even from people that have
firewalls filtering them. :slight_smile:

-Jack

4

That's a non-solution that will never happen. How many networks are going
to trust joe somebody to inject null routes into their backbone? Will
UUNet/Sprint/C&W/Level3/etc. trust me or Rob to tell them what's a bogon
and what's not? I really doubt it. They might have an easier time
trusting their local RIR, but I wouldn't be surprised if they didn't.

Well... I am pretty sure Tier1 backbones are up-to-date on the bogon
filters :slight_smile:

As we've already discussed, it's really the smaller networks with outdated
bogons or with admins who don't know what they are doing..

Several people pointed that out earlier. Botched / outdated firewall
configs may be a bigger problem than BGP filters. For a glimpse at why,
see
http://groups.google.com/groups?q=69.0.0.0%2F8&ie=UTF-8&oe=UTF-8&hl=en&btnG=Google+Search

Yup..

I know some writers watch nanog for potential stories. Wake up guys, this
should be one...if not for the news value "ARIN gives out unusable IPs,
future of the Net in question", then at least for the public service value
of getting the word out that bogon filters need to be maintained and kept
up to date or they do more harm than good.

No kidding..

I think we are just going around the circle/preaching to the choir on the
same topic here.. Is this like what... 3rd time we are discussing
this whole 69/8 issue :-D? Really, someone needs to get out this 69/8
issue on the press.. Just a thought.. heh

-hc

5

Monday, March 10, 2003, 7:44:43 PM, you wrote:

Well... I am pretty sure Tier1 backbones are up-to-date on the bogon
filters :slight_smile:
As we've already discussed, it's really the smaller networks with outdated
bogons or with admins who don't know what they are doing..

Bingo. No silly bgp feed will fix this problem. The problem is
all of the small customer networks that have been setup where the
admin at the time installed a slick firewall using what was then
current information and then walked away.

I only see three ways to deal with this issue:

1. Contact each customer net that we find that is filtering on
outdated information. I'm sure only the operators that have been
assigned 69/8 space will start doing this (and have), since we are in
fact responding to customer complaints. This process should be
complete in say, oh, ten years or so. That should give us enough time
to track them all down.

Oh while we are at that, we might want to contact every operator of
websites that are displaying "sample" firewalls using ipchains,
iptables or ipfw that show 69/8 as a bogon network. We'll need to get
them to change those webpages to show correct information. I mean,
why have that information out there so some other clueless admin can
simply start a fresh problem for us. I figure a couple of years to
fix this too.

2. Find a way to break all of those customers networks that filter
69/8 so that the response time to fix it is much less than the time
to contact each and every operator. The only way to do that is to
move something like the root servers into this space. Yes it's crazy
but it's the only way to break smaller networks. But once joe sixpack
wonders why he can't get to Yahoo this morning and calls his
consultant, the problem would be resolved a lot faster than it will
take us to track them down and do option 1.

3. Have us 69/8 address assignees simply live with the problem and
stop complaining in forums such as this. We're the ones dealing with
the end user complaints about lost connectivity to sites once we've
renumbering a link into this range. This goes back to option number
1, we'll simply bite the bullet and live with the problem and fix them
as we find them.

I'll admit, I run a small network and was quite happy to receive my
first ARIN assignment some months ago. I wasn't so happy to find out
that once I renumbered our internal office workstations into this
range I had complaints from other employees about sites they could not
reach (starting with *.ca.gov). I haven't even put one customer net
into this new range yet and I've already reacted to a couple of dozen
problems that less than 20 employees have found. I'm honestly scared
to death about renumbering all of my customers now.

I think we are just going around the circle/preaching to the choir on the
same topic here.. Is this like what... 3rd time we are discussing
this whole 69/8 issue :-D? Really, someone needs to get out this 69/8
issue on the press.. Just a thought.. heh

I had an email sent to me from a writer from circleid.com (Joe
Baptista) back in late December regarding this issue when the problem
first popped up on Nanog. As far as I can remember he was going to
write up an article on this situation. I have no idea what became of
that.

Regards,

Joe Boyce

6

Look, there's no quick fix solution here. It's going to take real
effort and real work. However, the _REASON_ all those pages reference
sample bogon filters is because there isn't a global bogon filter
that is dynamically updated available. If there was, and people were
aware of it, they'd use it. (At least a significant percentage would).

As such, is a BGP feed a panacea? No. Is it a step in the right direction?
Yes. Will it solve the problem by itself? No. Will it improve the situation?
Yes. Moving the root servers into that space may expidite solving the problem,
but at a _VERY_ significant cost. Moving the GTLD servers might make a little
more sense (at least then, you aren't requireing _EVERYONE_ to update their
hint files), but I still don't think that's a good idea.

Others have suggested that it needs to be available in LDAP. Some have
suggested DNS. As far as I'm concerned, the same servers or some group
of servers could easily be set up to publish the authoritative BOGON list
via DNS, BGP, LDAP, HTTP(XML), FTP, and possibly other protocols.
Getting bogged down in the protocol isn't helpful. Finding a way to make
an authoritative global BOGON list (Note: BOGONS are the UNALLOCATED/UNASSIGNED/
RESERVED/INVALID _LARGE_ blocks, _NOT_ every little hole in the allocation
space) that is dynamically updated _IS_ the most practical solution for the
long haul.

Renumbering multiple global resources every time an RIR starts issuing from a
new /8 isn't feasible.

Publishing the data over the net is.

Owen

7

So, someone feel free to smack me if I'm mentioning something which has
been discussed already (there isn't enough masochism in the world to make
me read this entire thread), buttttt...

How exactly is a BGP feed of bogons useful in any way shape form of
fashion? It doesn't prevent people from announcing more specifics, it
doesn't do anything about source address bogons, it can't be used to
packet filter... How exactly would it do anything other than simply not
having the route at all?

If and when some vendor adds support for taking the routes from a bgp feed
and using them in a packet filter, sign me up. Until then, I must be
missing something.

8

Look, there's no quick fix solution here.

so let's see how much of a kludge we can make to show how clever
we are.

randy

9

How about if we all chip in to hire a bunch of out of work consultants to fly to the NOCs of the various backbones who are being boneheaded to educate them with a clue-by-four?

Alec

10

To a degree the problem is ability to reach proper persons. I'd like to be
able to enter as# or ip and immediatly get email for a tech who knows what
to do. Radb is supposed to provide some of these functionalities, so does
ip whois, so does dns whois. Usually one of these will get you what you
need or if nothing else, youd'd look at AS traceroute and contact the
upstream.

Reality is we do have hierchical structure in ip & as assignments/allocations:
IANA->RIR->LIR->ISP->END-USER but currect information exchange is only
possible at one level (i.e RIR should know how to contact LIR, ISP should
know how to contact END-USER). A lot smaller hierchy is with AS numbers -
IANA->RIR->END-USER. I guess I forgot about all this in my proposal but
I'll be sure to clarify that when new assignment is received ARIN should
notify not only their IP subscriber members and end-users (ip assignments)
customers but also all those listed as contacts for ASNs (removing duplicate
emails gathered from all the sources, of course).

Unfortunetly ASN contact information is one of the least "maintained" as
far as ARIN data goes. And too bad... In my opinion fairly good way to
solve the problem would be to make sure that ASN contact info is up to
date for all RIRs and when new global assignments are made than IANA
makes the announcement and RIRs pass it along to their AS contacts and as
backup through longer ip path. I'm fairly certain if info on who to
contact was up to date at RIRs, the reachibility of this would be well
over 99% and number of blackholes for users of new ip block would be very
small.

11

so let's see how much of a kludge we can make to show how clever
we are.

Hey, I already came up with the slashdot idea.

Seriously though, somewhere there is a popular site that is non-profit in
nature that would trade say a month of free access for the hassle of being
put into a widely-blocked block.

Like anything else, until end users complain, nothing happens.

Charles

12

Excellent point...but then, what to do?

Have we given up and decided that addressing the 69/8 (and similar, future
issues) is a social problem that can't be fixed via technical means?

Are you ok with a solution of patiently waiting for some sort of critical
mass to occur with each new /8 that gets allocated? Sooner or later,
enough content will be in 69/8 (and other commonly filtered /8s) that
people will be forced to fix their filters. But is that the only way?

And would your answer change if you were one of the first networks to be
assigned space in the new range?

Andy

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Andy Dills 301-682-9972
Xecunet, LLC www.xecu.net
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Dialup * Webhosting * E-Commerce * High-Speed Access

13

I guess that emperor is a little naked after all :slight_smile:

Without applying hard-coded bogon filters to your peers (to prevent
receiving longer prefixes in bogon space), it is essentially useless.
http://www.cymru.com/Documents/secure-bgp-template.html lists a nice
template. But then we're back right where we started, may as well just
have a static ACL...unless you can't afford the ACL hit, in which case
filtering announcements from your peers and routing everything bogon into
a traffic sink would be a great solution.

We're all filtering announcements from our peers anyway, right? :slight_smile:

Andy

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Andy Dills 301-682-9972
Xecunet, LLC www.xecu.net
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Dialup * Webhosting * E-Commerce * High-Speed Access

14

Charles Sprickman wrote:

Seriously though, somewhere there is a popular site that is non-profit in
nature that would trade say a month of free access for the hassle of being
put into a widely-blocked block.

The suggestion of putting Yahoo or Google on a 69/8 IP led me to this idea:

Google could put their *beta* sites on a 69/8 IP, without causing them (Google) much Internet reachability/connectivity harm, and benefiting the Internet at large considerably.

Set up a page (hopefully linked from www.google.com) that lists all of Google's present beta sites. On this page, inform the user that the beta sites are hosted on "newly allocated IP addresses" and that if the said user can't reach the beta sites, it most likely means that their ISP/Company is improperly filtering these newly valid IP addresses, Instruct these affected users to contact their IPS's support desk or their company's IS department and alert them that they need to update their IP filter set to avoid filtering newly released and valid IPs. Then also link to a site such as <http://www.cymru.com/Bogons/> which explains bogon filters, shows how to find the latest lists, and educates the filter-clueless on how to subscribe to appropriate announcement lists to become aware of updates/changes in what IPs can be safely filtered. Google could also explain that they are doing this to help the Internet community fix this problem, and perhaps explain why it is a problem. They would get tons of good press which would help advertise Google and their beta projects.

Froogle is a very kewl site that gets better by the day (thanks guys, I use it all the time!), and I bet it also gets more traffic by the day. This would be a good way for Google to get free publicity for Froogle and other beta sites, and get big Internet community "good guy" points for helping fix the 69/8 bogon filter problem, without outright breaking the highly popular Google websearch site itself.

Is there anyone from Google lurking here on nanog?

jc

(Googling on: google beta, I discovered that Google itself went beta in February of 1999, just 4 years ago. My, how time flies!)

15

(Note to Mr. Dill, this is not intended to pick on you specifically,
it's just a convenient place to butt in)

JESUS H CHRIST ENOUGH ALREADY... Please stop with the hairbrained ideas to
put random things in 69/8 space. These goals are mutually exclusive. You
can't put important stuff on broken IPs, and you can't fix broken IPs by
putting unimportant stuff on them. No one is going to move all of the root
servers to try and fix a couple outdated filters, and no one who is still
running outdated filters is going to notice it because they can't reach
Google beta sites.

These are not just bad ideas, they are STUPID ideas. What happened to the
days when, before people posted to mailing lists, they thought "will this
make me look like an idiot in front of engineers across the entire
planet"? This is quickly not only becoming one of the most all-time
useless threads ever, but it is continuing to repell the useful people who
can no longer stand to read NANOG because of crap like this.

Listen, I have space in 69/8, and it is NOT an epidemic. Back when 64/8
was opened up it destroyed a beautiful 64/3 filter on unallocated space,
and yet somehow we all made it through just fine. The people who are
stupid enough to filter IPs without a plan on keeping those filters up to
date deserve their connectivity problems. Maybe next time they'll give
consideration to whether they actually need unallocated bogon filters on
every last linux server. :slight_smile:

16

And I'd like to add I agree.

The problems were wide at first, they have definitely dropped off.

Everyone important, is reachable now it seems.

I can think of maybe one or two small islands who might still be unreachable
but hey if I lived in the troopics I'd be outside with an umbrella drink
too, not changing filters.

Bottom line is that the only way to fix this problem is to move in to the
space, use the space, and contact people when its broken. Nanog although
perhaps not the best place for this, has helped in this goal for me at least
4 or 5 times and its fixed for everyone not just me. With all of us
pounding away the problems clear quickly.

17

Richard A Steenbergen wrote:

Charles Sprickman wrote:

Seriously though, somewhere there is a popular site that is non-profit in
nature that would trade say a month of free access for the hassle of being
put into a widely-blocked block.

The suggestion of putting Yahoo or Google on a 69/8 IP led me to this idea:

Google could put their *beta* sites on a 69/8 IP, without causing them (Google) much Internet reachability/connectivity harm, and benefiting the Internet at large considerably.

(Note to Mr. Dill, this is not intended to pick on you specifically, it's just a convenient place to butt in)

Ahem. It's _MS._ Dill, thank you.

Maybe next time you will stop and think "will this make me look like a sexist idiot in front of engineers across the entire planet"? before posting to a mailing list. (If the shoe fits, wear it.)

JESUS H CHRIST ENOUGH ALREADY... Please stop with the hairbrained ideas to put random things in 69/8 space. These goals are mutually exclusive. You
can't put important stuff on broken IPs, and you can't fix broken IPs by
putting unimportant stuff on them.

Sure you can. You just need content unimportant enough that no one (the end users on a network that is still blocking 69/8, AND the networks that put up the sacrificial target host on a 69/8 IP) is truly hurt if the connection fails, but important enough that the failure will lead to the broken networks being fixed and clue being distributed.

> no one who is still

running outdated filters is going to notice it because they can't reach Google beta sites.

I'm suggesting that Google explain why they are doing this on a page linked off their homepage. If this is done, people ARE going to notice, and ARE going to find out why. When it is widely publicised, it WILL be noticed even more.

These are not just bad ideas, they are STUPID ideas.

Where is your bright suggestion?

Listen, I have space in 69/8, and it is NOT an epidemic.

So how are you solving your 69/8 filtering/connectivity problems?

Back when 64/8 was opened up it destroyed a beautiful 64/3 filter on unallocated space, and yet somehow we all made it through just fine. The people who are stupid enough to filter IPs without a plan on keeping those filters up to date deserve their connectivity problems.

OK, I'm confused. I thought that the connectivity problem was, by and large, endured by the 69/8 IP users, and not on the networks with out-of-date bogon filters. Please elaborate on how this problem is really a connectivity problem for the networks with the bad filters, and how they are experiencing and then fixing this problem. Because from all reports here, it's obvious to ME that these networks are totally unaware of the issue because it is NOT creating a problem for them!

jc

p.s. Please don't cc me on replies, or on replies to replies, etc. I get the list email just fine and I don't need more than one copy of any given email. Really.

18

Ahem. It's _MS._ Dill, thank you.

Woops, my apologies _MS._ Dill. The JC is ambiguous.

Maybe next time you will stop and think "will this make me look like a
sexist idiot in front of engineers across the entire planet"? before
posting to a mailing list. (If the shoe fits, wear it.)

Sexist? Now that's just absurd. I took a guess and lost, big deal. If
anything, that proves my opinion of your idea has nothing to do with your
gender. Get over it.

p.s. Please don't cc me on replies, or on replies to replies, etc. I
get the list email just fine and I don't need more than one copy of any
given email. Really.

I believe you'll find reply-all is commonly used, get over it. Really.

19

[...]
> (Note to Mr. Dill, this is not intended to pick on you specifically,
> it's just a convenient place to butt in)

Ahem. It's _MS._ Dill, thank you.

Please post with a gender-specific name if you want to take offense
when mis-identified.

Sure you can. You just need content unimportant enough that no one
(the end users on a network that is still blocking 69/8, AND the
networks that put up the sacrificial target host on a 69/8 IP) is
truly hurt if the connection fails, but important enough that the
failure will lead to the broken networks being fixed and clue being
distributed.

How do I configure my routers and web servers for that?

I'm suggesting that Google explain why they are doing this on a page
linked off their homepage. If this is done, people ARE going to
notice, and ARE going to find out why. When it is widely
publicised, it WILL be noticed even more.

Last I checked, Google was a for-profit business, not a charity house.
I'm not sure how doing something that will make them look dumb, and
cost them in valuable ad revenue, etc is in their best interests.
Perhaps you could fill me in here.

p.s. Please don't cc me on replies, or on replies to replies, etc.

We have seen time after time that the propagation delays on the NANOG
list, most likely resultant from sub-optimal postfix/majordomo
configuration and/or an overloaded box, make it unsuitable for
realtime communications. With this in mind, I have taken the liberty
of cc'ing you in my reply, despite your request to the contrary.

If duplicate messages cluttering your inbox are causing you much
grief, prehaps it's time to read up on message filtering using
procmail, formail, and friends.

Regards,
-a

20

Are you ok with a solution of patiently waiting for some sort of critical
mass to occur with each new /8 that gets allocated? Sooner or later,
enough content will be in 69/8 (and other commonly filtered /8s) that
people will be forced to fix their filters. But is that the only way?

And would your answer change if you were one of the first networks to be
assigned space in the new range?

I might point out that it can even be worse. The IP addressing I was using
belonged to a provider that I'd used for many, many years. They were pulling
out and sending customers elsewhere, so I bit the bullet and pulled up the
numbers to get my first ARIN assigned addresses. So now I have a /18 that
will be at 90% utilization by the end of the month. I can't delay, as I
can't request more IP addresses until I release the old networks back to the
provider. In essence, I was just forced to screw all my customers. At the
next meeting, might someone kindly mention to ARIN that "initial" requests,
especially renumbers, should not be issued space that is less than a year
off the bogon list?

More than anything, this is what has pissed me off. After the renumber, I'll
only have 69/8 space, which means all critical services such as my mail,
dns, and web servers will all be affected. I hear it now. "I didn't receive
mail from so and so!" I check the logs and don't see an established
connection to my server. So, is the problem that the far mail server lost
the message, the user emailed the wrong place, or my new IP addresses
weren't accessible by the far mail server or the dns servers that it uses?
In addition, sometimes the problem is that my user just needs to put the
crack pipe down. I just don't feel comfortable with this last one anymore,
though. I can't be sure it's the crack. It could be the IPs. How do I know?

-Jack