NANOG36-NOTES 2006.02.15 talks 5-end Lightning talks, closing notes

(they weren’t kidding about lightning!! ^_^;; )

2006.02.15 Lightning Talks:
Infrastructure (DNS and Routing) Security -
Status and Update by Sandra Murphy

Need for Speed: What’s next after 10GE?
by Mike Hughes

A Brief Look at Some DNS Query Data
by John Kristoff

The impact of fiber access to ISP backbones in .jp
by Kenjiro Cho

New Network Monitoring Interest Group
by Mike Caudill

Understanding the Network-Level Behavior of Spammers
by Nick Feamster (presented by Randy Bush)

12:20-12:30 Closing Remarks
Steve Feldman, CNET, Susan Harris, Merit

Reload your agenda for the slides!!

Fun with gnuplot, DNS query data, John Kristoff
X asis, source port of client query to DNS server;
Y axis, how many times that port was used.
Looking at recursive server for an institution
open to inside and outside on 2005.11.22
starting at 1024, lots of clients use that port,
then declining to the right; 1025 is most popular
port; wraps at 5000, windows starts over.
to the right, UNIX boxes start with high ports.
Port 137, windows stuff, all bogus windows lookups
Port 5353, is multicast DNS, MACs use it, also bogus
Some very interesting outliers, either misconfigured
or poorly thought out OS/stacks.
Graphs are similar at different institutions, and at
large ISPs.
If you take out the external queries, points below
1024 (except 53) seem to be machines behind PAT boxes.
Port 1900 is plug and play port, so windows can’t use
it, so it’s a low outlier.
external queries show more outliers in low range.
looking at PTR queries internally; no elbow at 1025.
5353 standout is still there, multicast PTR queries,
all bogus.
MX queries, same thing.
AAAA stuff, not many outliers, very clean; possibly
bogus, though.
A windows box trying to contact IRC server (neutered
bot box); keep using same source port over again until
firewall/virus software moved it.
UNIX box used port range constantly across the range,
more normal (trojaned box)
Normal UNIX box shows more normal rows of different
looking at source ports, what other useful info and
patterns can you start to discern? Look at TTL, dest
ports, all sorts of fun you can start to discover.

Sandra Murphy
sandy at sandy at
DNS and routing security
DNSsec is live, sweden has signed top level zone,
RIPE signing reverse zones, some reverse delegations.
open working grope, dnssec deployment initiative
focused on deployment issues, active mailing list, regular
organizes workshops at conferences, etc.
screenshot of the site; has roadmaps, working group
signups, mailing lists, operator guidelines, links
to NIST, etc., events, and actions.
DNSSEC-tools project
create tools/patches for web browsers and such.
current release is v0.9 from 2/10/2006
Firefox 1.5RPM to check DNS sec records back
Shot of tools being released…
zonesigner tool is how you sign and maintain a signed
Some very detailed documents on how you sign and
maintain a signed zone, as well as mailing lists.
sourceforge link for dnssec-tools bundle
Securing the routing infrastructure:
big problem, no traction on deployable solutions
3 workshops with a wide net of interested parties.
operators, iSP, access, content providers, vendors, security
DHS hosted, anxious to find a solution
Operators’ emphasis
a strong call from the operators for an authenticated
list of authorized prefix originations (accurate, complete
respond to customr requests to route prefixes
useful in debugging routing difficulties
NEW ARIN policy suggestion
new field in address templates (direct and subdelegations)
for list of permitted ASes
inhereits self-discipline of completign form (IRR entries
aren’t always done)
inherits scrutiny of ARIN process on creation
ARIN is authority for who is allocated prefixes
Any IRR would have to check prefix with RIR
Authentication and currency in IRRs
authentication IRR objects
RIR run IRRS have internal access to authentication for
prefix holders
non-RIR run IRRS would have to find a way to get that
authentication from the RIRs
samee is true for RIR IRR objec referring to nonmember
Currency for IRR objects
reclaimed resources have to result in IRR purges
why not a TTL in IRR objects? Handles non-RIR IRRs
This solicits requests and feedback. Try the DNSSec tools,
try signing a zone, see how it works. Try the client
system that does the DNSsec validation.
Participate in ARIN ppml list on routing security, etc.

Mike Hughes, what’s next after 10GE
mike at
Channels geoff huston for scary graph.
curve of traffic growth. By end of 2006,
he’ll be at 150Gb; if he takes last 3
months, he’ll be at 300Gb in one metro.
where is it coming from?
ADSL2, Wimax, FTTx, skype, voip, p2p, etc.
fewer people with bigger pipes.
think back to seattle
chap from force10 came and asked what do you
want, 40g or 100g?
we can do 40g now
expensive at oc768
cheap at 4x10GE
can’t we just do 8x10GE
rate limit/transfer cap users
implment QoS
thorttle p2p apps
either doesn’t scale, isn’t an option, is costly and complex
We either build and scale, or spend money to not scale.
It’s easier to overprovide, actually.
Gary Bachula, VP Internet2
Research came to conclusion that it was far more cost
effective to simply provide more bandwidth.
We already need something faster than 10GE and 40GE
we’re already building 8x10GE link agg bundles on a single
spans anyhow.
common engineering sense says that your backbone has
to be some multiple larger than your largest customer
Selling 10G transit means backbone needs to be multiples
of that!
Your vendor needs you!
Probably–even if they don’t realize it yet!
Stand up Ted Seely!
Some vendors are saying the next ethernet standard is
5 years out. Too late!
Apparently, the IEEE 802.3 HSSG isn’t convinced that it
needs to start working on the next ethernet standard
is it only going to happen if we drive it?
answer seems to be yes! So let’s start beating up
our vendors!!

Kenjiro Cho
Impact of fiber access to ISP backbones in .jp
yes, we DO need 100G!
residential broadband
21 million broadband subcriber
15 mllion for DSL
3 million for CATv
4 million fo rFTTH
100mb bidir fiber is 40USD/month
4% of heavy hitters account for 75% of inbound volume
fiber users account for 86% of inbound volume
DSL is only 14%
no clear boundry between heavy hitters and normal users
data set
sampled netflow data from japanese ISP
ratio of ifber and DSL unique users in dataset
heavy-hitters denote users who send more than 2.5GB/day.
heavy hitters statistically follows power law
up to 200GB/day, 19Mbps sustained!
no clear boundry between heavy-hitters and normal users
lines at 2.5GB/day and the top 4% heavy hitters
4% in total uses, 10% in fiber, 2% in DSL
CDF of traffic volume of heavy-hitters
top 4% use 75% of inbound, and 60% of outbound
correlation of inbound and outbound volumes per user
fiber and DSL graphs
2 clusters one below unity line, another in high volume
more heavy hitters in fiber, more lightweight users in DSL
no differences between DSL and fiber except heavy hitters.
fairly constant in heavy hitter usage
fiber peak is 80% of combined peak. inbound much larger
for heavy hitters, reversed for others?
83% is TCP dynamic ports
RBB home users, DOM, other domestic, INTL
both ends are clssified by commercial geo-IP dbs
62% of residential traffic is user-to-user
90% is inside Japan among RBB and DOM
possible language, cultural barriers
p2p super-nodes among bandwidth rich domestic fiber users
count peer numbers for 50th percentile traffic
expected 2 types; downloads, video (few streams),
other with MANY peers (p2p).
but couldn’t find such a split.
we tend to attribute the skews to divide between heavy hitters
and rst of users
buthere are diverese and widespread heavy hitters
heavy-hitters are no longer exceptional extremes
which came first–start on DSL, become heavy hitter, then
move to fiber?
or start with fiber, and then find uses for the fiber?
is this specific to japan?
need to find faster links, re-evaluate prices!!

Mike Caudill
mcaudill at
FIRST forum of incident response and security team
some special interest groups
Vendor SIG
CVSS SIG common vunlernabilty scoring system.
Abuse SIG
Network Monitoring SIG
it’s a members-only grope, so SIGs are just focus
groups within SIG
Abuse SIG, formed out of ECOAT (european)
aims to further the cooperation of internet abuse
fighting teams of network/information service providers,
and jointly produce tangible results that will benefit
its constituency
Network Monitoring SIG
to discuss and collaborate on various issues, such as sensor
detection methodology, common rule-sets for detection,
data exchange formats
2006 Conference
June 25-30 2006 in Baltimore, Maryland
first-chair at
Still getting some chairs for the SIGs, just getting

Randy Bush, IIJ
Spamming with BGP spectrum agility
Airudh Ramachandran, Nick Feamster
two domains instrumented with mailavenger on same network
sinkhole domain 1
continuous spam collection since aug 2004
no real email addresses–sink everything
10 million+ pieces of spam
sinkhole domain 2
monitors BGP as path and traceroute back to source upon
receipt of every source.
spamming techniques
mostly botnets, of course
DNS hijack of CanC to get botnet topology and geophgraphy
Correlation with Bobax victims
from georgia tech botnet sinkhole
distance in IP space of client IP from mx record
coordinate, low-banwidht sending
BGP spectrum agility
LOG IP address of SMTP relays
join with BGP route advertisments seen at network
where pam trap is col-located
/8’s are being announced. 4678 21562 8717
they bring up the aggregate, send spam from inside
the empty holes in the space!
82.00/8; hit you with spam, bring it back down.
Why such big prefixes?
Flexibility: client IPs can be scattered throughout
dark space within a large /8
same sender usually returns with different IP address
Visibility: route typically won’t be filtered (nice and short)
Low dampening on the /8s, so they make ideal spam sources.
They’re using REAL /8s, not the bogons, so they escape those
IP addresses are widely distributed across the /8 space
IP addresses typically used once only
60-80% use…
evidence that it’s working: only about half of the IPs
spamming from short-lived BGP are listed in any blacklist.
mail to feamster within domain
for more info
Length of short-lived BGP epochs
10% of spam received is coming from short-lived
announcements, then plateus and hits the sharp
curve of…something?

Steve Feldman wraps up with his closing words:
many thanks to Brokaw and Yahoo! for hosting, and
the party Monday night, thanks to Merit and the
program committees, and steering committee.
No Venue for next meeting yet, so keep your eyes
on the website!
Late may, early June.

Susan Harris from Merit for her closing words.
Thanks from Merit to Steve Feldman (PC chair)

And Thanks to the Dallas Yahoo! Team
Brokaw Price
Brian, Vicki, Mike, Todd, Raj, Brad, Sharon,

Meeting stats
Attendance: 340 (515 in LA, 458 in Seattle)
Only 6% women
NAPS: 11
Colleges, Universites, 14

Thanks to all the people from Merit who helped
behind the scenes, thanks to all the presenters,
and we’ll see you in Spring (somewhere!).

Meeting adjourns at 1216 hours Central Standard Time
to the sounds of Jerry Lee Lewis on the piano.