Big Temporary Networks

My best friend just got back from Chicon 7 last week, this year's World
Science Fiction Convention. He tells me that the networking at the con hotel,
the Chicago Hyatt, was miserable, whether wired or wireless... and that Sprint
4G wasn't much better.

I'm talking to the people who will probably be, in 2015, running the first
Worldcon I can practically drive to, in Orlando, at -- I think -- the Disney
World Resort. I've told them how critical the issue is for this market; they,
predictably, replied "We look forward to your patch". :-}

I know without a doubt that this is a problem NANOG PCs deal with 3 times a
year; is there any collected wisdom on the web already about how this has
been dealt with, that I can pore over? Pointers to good archive threads?

If not, do any of the people who've already done have 5 minutes to chime in
on what they did and what they learned?

Cheers,
-- jra

[..]

If not, do any of the people who've already done have 5 minutes to chime in
on what they did and what they learned?

You might want to go through the network presentations given for IETF,
NANOG/ARIN and last but definitely not least: CCC congress + camps.

eg:
http://events.ccc.de/camp/2007/Fahrplan/events/2043.en.html

Typically though it requires people who have done it before with proper
equipment to get a network up and running properly :wink:

Greets,
Jeroen

I did a hack a thon a few months back in Palo Alto a few blocks down
from PAIX. I used 6 of the Xirrius high density access points. About a
1000 attendees scattered over about 1/2 city block. 6 access points
was overkill.

Doing the same for a film festival here in a couple of weeks as well.

-mike

I know someone who did Interop's networking for a number of years and does it for various non-Worldcon conventions. His short summary was to stage and label and debug and test extensively beforehand, even if the reassembly might introduce more bugs in the field.

George William Herbert

I have not done any that size/duration but I have done some where the scale is 1000s of attendees over a long weekend event, with small budgets.

You'll need a beefy NAT box. Linux with Xeon CPU and 4GB RAM minimum. Run your DNS resolver and DHCP here, unless you have hardware to spare.

Set your DCHP lease time to 1 hour so you don't have an address tied up for someone who stopped in for 15 minutes three days ago.

If you don't have any sort of WiFi controller, name the APs differently. People are really pretty good about picking the AP with the best signal strength.

Configure and test your equipment before you get to the venue because you will be running around tryiong to find the electrician to turn on the breakers you need, and they forgot about.

Change the default passwords on the APs. I did a lot of these for maker/hacker crowds, and there's great fun to be had in advertising rude SSID names.

Bandwidth. Lots of Bandwidth.

--Chris

Excellent advice.

But how do you load a wireless network and an uplink with 12-14k attachments
for testing purposes? I can see how to test the uplink, but testing the WLAN
seems ... well, next to impossible, to me, which is why I'm querying the list.

:slight_smile:

Cheers,
-- jra

Excellent advice.

But how do you load a wireless network and an uplink with 12-14k attachments for testing purposes? I can see how to test the uplink, but testing the WLAN seems ... well, next to impossible, to me, which is why I'm querying the list.

:slight_smile:

Cheers,
-- jra

I'm not sure if this is obvious for this list or not, but with your WiFi nodes, a good practice for that kind of density is more nodes, lower power. Keep the client connection load per AP as low as possible to improve overall performance. Jacking up the power in a small area like that will just step on the adjacent APs and cause issues.

Dylan

I'm surprised (well, perhaps I'm not) that no one's chimed in about the defcon network, and the effort they go to each year. Here's some basic information:

http://www.defconnetworking.org/

Defcon is often described as the world's most hostile network, and it does have some interesting problems, including extra efforts to keep the wireless side up, and useful. Considering the foolishness that goes on in the background, it's very stable.

I do wish that they had more immediately useful information in that site up above, but it's still got some interesting data points.

You'll need a beefy NAT box. Linux with Xeon CPU and 4GB RAM minimum.

Or not. The CCC presentation is showing *real* Internet for everyone, unless I'm very much mistaken...

Regards,
Tim.

No NAT was involved there indeed. Typically conferences can get a
temporary prefix from their local RIR for conference-alike setups.

Of course that does require one to arrange uplinks who will announce
that prefix, a friendly LIR etc etc etc. Thus this all boils down on how
large your setup will be and how good your want your network to perform.

Greets,
Jeroen

WLAN in large conferences certainly is a challenge. You basically want
to get as many people on 5GHz as possible due to more available
channels. 2.4GHz becomes quite noisy. Also, configuring your access
points for high density helps. This means disabling the lowest data
rates. You also don't want to run full Tx power. Basically this will
ensure high data rates and quicker handoff to a nearer AP when roaming.
You don't want a client that is far away from an AP connecting to it at
a 1 Megabit data rate tying up the radio. This also is key in high
density seating open floorplan office situations.

Sean Lazar

The 2015 WorldCon site selection is contested. There is a group
seeking selection for the Disney Coronado Spring Resort in Florida but
also competing groups seeking Spokane, Washington, and Helsinki,
Finland.

Thanks,
Donald

We have been using Unifi (a Ubiquiti WIFI product) for local conventions
and festivals. The product is fairly cheap, robust, and their access
points have very good range. We have deployed it at several commercial
businesses as well with great success. The deployment is very easy. We
run the controller on a VM at our NOC, but you can also run it locally at
the event as well.

Besides this, we have a fairly beefy box that handles DNS and DHCP and
basic firewalling.

Josh

Chris Scribbled:

You'll need a beefy NAT box. Linux with Xeon CPU and 4GB RAM minimum.

Or not. The CCC presentation is showing *real* Internet for everyone, unless I'm very much mistaken...

If you know of an ISP in Central Texas that can deploy a 10Mbit plus connection along with a /22 of v4 address space for a 1 day event, please let me know. TWCable has been pretty easy to work with for special events, but I'd be really surprised to see them be able to do that.

--Chris

We have been using Unifi (a Ubiquiti WIFI product) for local conventions and festivals. The product is fairly cheap, robust, and their access points have very good range. We have deployed it at several commercial businesses as well with great success. The deployment is very easy. We run the controller on a VM at our NOC, but you can also run it locally at the event as well.

Besides this, we have a fairly beefy box that handles DNS and DHCP and basic firewalling.

Josh

The UniFi line is hard to beat for the price. The controller software is free and the base access points are <$100, in fact you can get a 3-pack for <$200. Deployment of the APs would be 95% of the work. Configuration via the software would take minutes.

Dylan

I'm not sure if this is obvious for this list or not, but with your WiFi nodes, a good practice for that kind of density is more nodes, lower power. Keep the client connection load per AP as low as possible to improve overall performance. Jacking up the power in a small area like that will just step on the adjacent APs and cause issues.

++;

An enterprisey AP flock that perhaps even can talk to eachother about
power levels is a must.

At all possible cost, avoid login or encryption for the wireless. Captive
portals suck, especially if they try to be clever and keep an eye on the
link-state to each client. Tablets and smartphones turn their radios off
to conserve battery, and that means having to login all the time.

While things have become much better, doing 802.1x on conference wireless
probably is a bit daring. OTOH eduroam does it all over Europe.

Get lots of IP addresses. A /16 probably still can be borrowed for this
kind of event. I know RIPE had rules and addresses for this kind of use
a couple years ago, at least.

And get v6.

Do not NAT. When all those people want to do social networking to the same
furry BBS while also frequenting three social app sites simultaneously
you are going to get Issues if you NAT. So don't. (Keep in mind that the
5-tuple for each TCP connection more often will become a 3-tuple if the
demographic of the user base is skewed towards a focus group and NAT is
in use. )

Lots of IP adresses will also enable you to set sensible DHCP lease
times on the failover-connected (because they are, right?) DHCP
servers. Nothing is so detrimental to connectivity experience as lost
leases from either crashed DHCP servers or short lease times.
Be very thorough and careful in setting DHCP up. It'll pay off.

Have DNS resolvers locally. Unbound is good. As is BIND.

It might be a good idea to have reverse DNS delegation set up,
perhaps via the BIND $GENERATE directive; just something like
wireless-node-47-11.world.con will do.

Make sure that the whois contacts for the address block are proper.

Try setting some monitoring up; it is good to be able to keep an eye on
client count per AP etc. This is also much easier if the wireless solution
is enterprisey.

I knew about Spokane; I wasn't aware of Helsinki.

Thanks, though, for clarifying *which* Disney resort it is; I wasn't at
Chicon, and don't have the bid details in front of me.

Cheers,
-- jra

Have you had to/been able to haul in your own bandwidth to feed it? What
class? (Real DS3/OC1/OC3, FiOS/HFC, something else?)

Cheers,
-- jra

Yes, we backhaul our own bandwidth to it; either using Cambium or Ubiquiti
unlicensed 5Ghz backhauls. Depending on the distance and type of backhaul,
we can get 50-150mbps to the event.

Josh

From: "Måns Nilsson" <mansaxel@besserwisser.org>

04:05:41PM +0000 Quoting Dylan Bouterse (dylan@corp.power1.com):
> I'm not sure if this is obvious for this list or not, but with your
> WiFi nodes, a good practice for that kind of density is more nodes,
> lower power. Keep the client connection load per AP as low as
> possible to improve overall performance. Jacking up the power in a
> small area like that will just step on the adjacent APs and cause
> issues.

It was. :slight_smile: Of course, the propery may (read: probably does) have its
own conference areas and residential floors wifi, and those may or may
not be V-WLAN capable.

An enterprisey AP flock that perhaps even can talk to eachother about
power levels is a must.

At all possible cost, avoid login or encryption for the wireless.

Yes, and no.

Captive portals suck, especially if they try to be clever and keep an eye on
the link-state to each client. Tablets and smartphones turn their radios
off to conserve battery, and that means having to login all the time.

My plan is to have 3 VWLANs:

worldcon-guests, which will have one-time captive portal; I want the
controller to remember the MAC address everywhere, all week

worldcon-dealers, no captive portal (for credit card and other embedded
machines), and

worldcon-staff, which may have some relaxed outbound security compared to
the other networks.

(For example, I have no problems blocking outbound port 25 and redirecting
recursive DNS -- though I do want a system that permits me to whitelist
MACs on request. But I would do those on the guest and dealer nets, and
not on the staff one.)

While things have become much better, doing 802.1x on conference
wireless probably is a bit daring. OTOH eduroam does it all over Europe.

If I did try to do that, it would probably only be on the staff network;
it's a much more contrained environment.

Get lots of IP addresses. A /16 probably still can be borrowed for
this kind of event. I know RIPE had rules and addresses for this kind of
use a couple years ago, at least.

Indeed? I did not see that coming. Hell, perhaps Interop could be talked
into loaning me a /16. :slight_smile:

And get v6.

Yeah, I assumed that, though it will be interesting to see how much play
it actually gets; these are SF geeks, not networking geeks.

Do not NAT. When all those people want to do social networking to the
same
furry BBS while also frequenting three social app sites simultaneously
you are going to get Issues if you NAT. So don't. (Keep in mind that
the
5-tuple for each TCP connection more often will become a 3-tuple if
the
demographic of the user base is skewed towards a focus group and NAT
is
in use. )

This, right here, is the kind of gritty advice that brought me to ask
this question in the first place. You're right; NAT is Right Out;
forget what I said earlier. :slight_smile:

Lots of IP adresses will also enable you to set sensible DHCP lease
times on the failover-connected (because they are, right?) DHCP
servers. Nothing is so detrimental to connectivity experience as lost
leases from either crashed DHCP servers or short lease times.
Be very thorough and careful in setting DHCP up. It'll pay off.

Oh yeah. I'm fond of leases as short as 30 minutes, though if I have
a /16, I won't care as much.

Have DNS resolvers locally. Unbound is good. As is BIND.

Yep, with lots of RAM on the boxes.

It might be a good idea to have reverse DNS delegation set up,
perhaps via the BIND $GENERATE directive; just something like
wireless-node-47-11.world.con will do.

Hmmm.

Make sure that the whois contacts for the address block are proper.

Well, I do have 3 years to plan. :slight_smile:

Try setting some monitoring up; it is good to be able to keep an eye
on client count per AP etc. This is also much easier if the wireless
solution is enterprisey.

I was planning on having a NOC, yes, albeit small.

Very nice, Måns; thanks.

Cheers,
-- jra