[HFC] pooling modems in layer2

Mailman List Mirror (Read Only)
1

Hello

I think it's kind of an isp secret but I would be curious how do people distribute modems to pools before they would even reach the actual IP network so on layer2:

http://dl.packetstormsecurity.net/papers/evaluation/docsis/Service_Distribution.jpg

For this I would like to get some clarification because I do not work in the telco industry. As I can figure out of the docsis, cablelabs documents. The CMTS device is connected to the coax segments through fiber. Therefore one could say that the "modem facing" side is a fiber optic interface but it's not 1000 Base-FX, not a regular Ethernet over fiber. It sends signals through a broad range of frequencies.

So what I would like to accomplish to provide a different pool of dhcp servers, which provides different config file, tod server, router, dns etc. infos to the modems but to do all this in Layer2.

I don't have hands on experience with CMTS-es but I would think that they are able to pool clients by MACs and able to send eg 500 clients to DHCP server1 and the other 1500 to DHCP server2 before they would even get an IP, so I talking of pure layer2 here!

Let's say if the CMTS device does not support this, what are the other options for routing layer2 traffic coming out of the CMTS? If I would know more about the device I would say that put a linuxbox after it (on the ISP facing nic) and mark the packets going out with arptables/ebtables then send them out of different nics to different dhcp servers.

Any suggestions are welcome.

2

Hello

I think it's kind of an isp secret but I would be curious how do
people distribute modems to pools before they would even reach the
actual IP network so on layer2:

http://dl.packetstormsecurity.net/papers/evaluation/docsis/Service_Distribution.jpg

For this I would like to get some clarification because I do not work
in the telco industry. As I can figure out of the docsis, cablelabs
documents. The CMTS device is connected to the coax segments through
fiber. Therefore one could say that the "modem facing" side is a fiber
optic interface but it's not 1000 Base-FX, not a regular Ethernet over
fiber. It sends signals through a broad range of frequencies.

Sounds about right to me.

So what I would like to accomplish to provide a different pool of dhcp
servers, which provides different config file, tod server, router, dns
etc. infos to the modems but to do all this in Layer2.

Why? Do you have a bunch of cable modems and a CMTS? If so, does the documentation not cover this? Or are you trying to hack your cable modem/cable provider?

I don't have hands on experience with CMTS-es but I would think that
they are able to pool clients by MACs and able to send eg 500 clients
to DHCP server1 and the other 1500 to DHCP server2 before they would
even get an IP, so I talking of pure layer2 here!

Let's say if the CMTS device does not support this, what are the other
options for routing layer2 traffic coming out of the CMTS?

Um. Probably via RADIUS and via VLAN assignment?

  If I would

know more about the device I would say that put a linuxbox after it
(on the ISP facing nic) and mark the packets going out with
arptables/ebtables then send them out of different nics to different
dhcp servers.

Most likely they just use VLANs. This rack of CMTS gear is on port 22 of the agg switch, vlan 2 and ip helper is set for vlan 2 to the desired dhcp server (which is most likely an HA floating IP if not a full blown VIP etc).

3

Scott Helms
Vice President of Technology
ZCorum
(678) 507-5000

4

"Toney Mareo" <halflife4@gmx.com> writes:

Hello

I think it's kind of an isp secret but I would be curious how do
people distribute modems to pools before they would even reach the
actual IP network so on layer2:

http://dl.packetstormsecurity.net/papers/evaluation/docsis/Service_Distribution.jpg

Nobody does CMTRI anymore. That illustration is over a decade and a
half old, which is part of what's confusing you. The scheme there is
that they use a dialup modem for the upstream and a cablemodem for the
downstream.

For this I would like to get some clarification because I do not work in the telco industry.

If you're interested in how CMTRI works for historical reasons, the spec is here:

http://www.cablelabs.com/wp-content/uploads/specdocs/SP-CMTRI-I01-970804.pdf

As I can figure out of the docsis, cablelabs documents. The CMTS
device is connected to the coax segments through fiber. Therefore
one could say that the "modem facing" side is a fiber optic
interface but it's not 1000 Base-FX, not a regular Ethernet over
fiber. It sends signals through a broad range of frequencies.

It sends signals over RF (i.e. truly "broadband"). The RF happens to
be on a laser-lit fiber instead of a piece of coax (until it hits the
fiber node and gets turned into coax cable). There are Ethernet MAC
addresses in there if you look at the right layer, but the DOCSIS data
rides as a "program" atop a J.83 single program transport stream on a
QAM64 or QAM256 modulated RF signal. It's just like a digital TV
program and occupies the same frequency space - but 0x1FFE is the
well-known PID that means "DOCSIS data".

The upstream channels are comparatively low (under 80 MHz) and the
downstream channels are comparatively high (over 80 MHz to 800-1000
MHz depending on the system). Splitting them out is accomplished with
bidirectional high and low pass filters called "diplexers".

So what I would like to accomplish to provide a different pool of
dhcp servers, which provides different config file, tod server,
router, dns etc. infos to the modems but to do all this in Layer2.

I don't have hands on experience with CMTS-es but I would think that
they are able to pool clients by MACs and able to send eg 500
clients to DHCP server1 and the other 1500 to DHCP server2 before
they would even get an IP, so I talking of pure layer2 here!

There are multiple ways to approach this. You need a consultant who
is well-versed in the care and feeding of DOCSIS edge networks to walk
through your options with you so that you don't find yourself in a
painful technical place.

Let's say if the CMTS device does not support this, what are the
other options for routing layer2 traffic coming out of the CMTS?

I don't recommend PPPoE. :slight_smile:

If I would know more about the device I would say that put a
linuxbox after it (on the ISP facing nic) and mark the packets going
out with arptables/ebtables then send them out of different nics to
different dhcp servers.

Any suggestions are welcome.

You might start by sharing a high level overview of what it is that
you're trying to accomplish. If it's simply sandboxing people who
haven't paid their bills, there are well-known ways to do that. If
it's business services over DOCSIS, there are likewise ways to do
that.

-r

5

The upstream channels are comparatively low (under 80 MHz) and the
downstream channels are comparatively high (over 80 MHz to 800-1000
MHz depending on the system). Splitting them out is accomplished with
bidirectional high and low pass filters called "diplexers".

The upstream spectrum is (at the moment) is 5-42 MHz in the US, though most
people don't use below 20 MHz and often avoid 26-28 MHz because of
interference.

> Let's say if the CMTS device does not support this, what are the
> other options for routing layer2 traffic coming out of the CMTS?

I don't recommend PPPoE. :slight_smile:

PPPoE not supported on any of the DOCSIS 3.0 certified CMTSs except the
Cisco UBR and then it must also be the termination point for the PPPoE
session, though it can be part of a L2TP (LAC<--LNS) handoff to another
device that can handle the PPPoE termination. I will certainly agree
that's its not a good technology for DOCSIS systems.

> If I would know more about the device I would say that put a
> linuxbox after it (on the ISP facing nic) and mark the packets going
> out with arptables/ebtables then send them out of different nics to
> different dhcp servers.
>
> Any suggestions are welcome.

You might start by sharing a high level overview of what it is that
you're trying to accomplish. If it's simply sandboxing people who
haven't paid their bills, there are well-known ways to do that. If
it's business services over DOCSIS, there are likewise ways to do
that.

Nailed it here.

6

Hello

Thanks for the responses, I think it clarified a lot and I already started reading this CM-SP-L2VPN-I13-140403.pdf documentation.

What I need here is that existing clients are sent through ISP1 currently and I would like to add ISP2 for future clients without interfering anything with the current operations. Then later on move the old clients over to ISP2 as well.

As I see it, this can only be done on the CMTS device not after it unless it's possible to relay packets from the cable side with their original HFC macs through the CMTS.

Yes indeed I do not want to setup failover or balance DHCP servers, but I want to move every new subscriber to a different pool which gets directed to a different DHCP server which then finally able to provide the modems with ips and other settings to be able to go out on ISP2.

On Tue, Aug 12, 2014 at 10:23 AM, Toney Mareo <halflife4@gmx.com> wrote:Hello

I think it's kind of an isp secret but I would be curious how do people distribute modems to pools before they would even reach the actual IP network so on layer2:

http://dl.packetstormsecurity.net/papers/evaluation/docsis/Service_Distribution.jpg

Certainly not secret, DOCSIS is a very well documented protocol with most of the information being publicly available.

For this I would like to get some clarification because I do not work in the telco industry. As I can figure out of the docsis, cablelabs documents. The CMTS device is connected to the coax segments through fiber. Therefore one could say that the "modem facing" side is a fiber optic interface but it's not 1000 Base-FX, not a regular Ethernet over fiber. It sends signals through a broad range of frequencies.

While fiber is commonly used in cable plants as part of a HFC network its completely transparent from a protocol standpoint the entire communication is over RF. D3 and older uses QAM modulation and the downstream runs over "normal" 6 MHz channels which are the same as TV channels.

So what I would like to accomplish to provide a different pool of dhcp servers, which provides different config file, tod server, router, dns etc. infos to the modems but to do all this in Layer2.

Why? The operator is the only one who can tell the CMTS which DHCP server(s) to send traffic to and modern CMTSs do that as an IP relay and passes its IP address as the GIADDR.

Because I advise the operator, you would think they are expert on the CMTS? Think again, I'm not an expert either but at least I learning.

I don't have hands on experience with CMTS-es but I would think that they are able to pool clients by MACs and able to send eg 500 clients to DHCP server1 and the other 1500 to DHCP server2 before they would even get an IP, so I talking of pure layer2 here!

Not exactly, first in nearly all cases the DHCP communication is an IP unicast rather than a layer 2 broadcast. Second, the way that the DHCP server is selected is normally based on the type of device so that modems get a specific GIADDR, CPE (PCs, routers behind modems, etc) get another one, and often the EMTA gets a third. It might be possible to do that off a count of devices, but if so it will be more of a load balancing scenario rather than these specific 500 CMs get this DHCP server. It is possible to do open access in a DOCSIS system, but its very difficult and involves creating filters in both the CMTS and CM configurations.

Let's say if the CMTS device does not support this, what are the other options for routing layer2 traffic coming out of the CMTS? If I would know more about the device I would say that put a linuxbox after it (on the ISP facing nic) and mark the packets going out with arptables/ebtables then send them out of different nics to different dhcp servers.

It doesn't really work that way, but the closest thing is a "soft" tunnel that gets used for things like transparent LAN services, carrier WiFi, and a few other use cases.

http://www.cablelabs.com/wp-content/uploads/specdocs/CM-SP-L2VPN-I09-100611.pdf

Any suggestions are welcome.

7

Toney,

Depending on which DHCP server software you're using, its probably easier
to do this kind of move with it rather than trying to build layer 2
tunnels.

Since each modem MAC is added (usually) to the DHCP server you can simply
run two different server instances and with the original server instance
handing out ISP1 IP information and the second one handing out ISP2
addresses and info. The only gotcha is that you have to make sure your
DHCP servers won't NAK unknown clients, but this is how most of the
conversions I've been involved with are done.

Scott Helms
Vice President of Technology
ZCorum
(678) 507-5000