I am not sure if this is a easy question to answer. But I am wondering what ISP’s do for their residential and business customers for designing POP’s that they usually access to get theur traffic into a given ISP and beyond. Is it usually a L1/L2 connection from the CE to the last mile POP? Or L2 even within the last mile POP. Do you just have POP’s delegated to residential users and a separate POP for business users. Or is it done on a geographical basis. So for this region of City-A we manage both residential and business customers at this same POP.
L3 switches that can handle a reasonable number of routes/VLANs/MACs and lots of bandwidth are so cheap that I'm fond of pushing L3 fairly deep into the access network with them in many cases. Not much benefit to that if you prefer centralized BRAS/BNG style boxes with all the bells and whistles to take the traffic management away from your last-mile gear, though. So you need access gear with its own traffic management capabilities and potentially L2 filtering of higher level traffic (DHCP snooping, ARP/ND inspection, RA guard, TCP/UDP port blocking, etc.) and that may limit your options or force you to terminate fewer customers at a PoP than you'd like to stay within the capabilities of a typical L3 switch product.
I've never been overly fond of the Ma' Bell style designs with humongous routers in centralized areas and L2-only haul out to the last-mile termination. The failure modes of such systems often result in hilariously large outages that are super visible publicly and put egg on peoples' face. A neighborhood being down is a little easier to manage from a customer relations POV, I think, and it's easy to make that happen with distributed L3 termination.
I've also found it easier to handle multiple backhaul paths at L3 than L2 since spanning tree is such a pain in the butt, but E-RPS/G.8032, if you get switches that support it, can also be very handy.
There are some smaller, somewhat cost-effective full-touch routers that can help bridge the gap between those two options, though. Juniper's MX104 and the Cisco ASR1k series are some reasonable options for that, but it'll definitely cost more than a cheap L3 switch for a given amount of bandwidth.
I do like to separate SMB and Resi traffic, but it's mostly for customer service reasons rather than technical reasons. That separation rarely entails separate equipment but rather just VLANs and PCPs, IP subnets, etc.
Now if you want to sell DIA type services where you can offer full BGP tables, MPLS interconnection, etc., that's another matter. A need for IPv4 CGNAT may, as well, but things like 464XLAT, lw4o6, MAP, etc. can fix that up if you're willing to put some extra requirements on your CPE/RG.
If you're in a position where you want to or have to offer competitors access to your network to sell service directly to customers, that's also going to potentially really change the situation.
I am not sure if this is a easy question to answer.
Actually,trivial to answer: "It depends". Often due to "hysterical raisins".
even within the last mile POP. Do you just have POP's delegated to
residential users and a separate POP for business users. Or is it done on a
geographical basis. So for this region of City-A we manage both residential
and business customers at this same POP.
How well is servicing both out of one POP working for you? If what you have in
City A is working for you, your business plan, and your customers, don't change it 
Some companies may want 2 POPs because one area of the city is highly
commercial/industrial and all the home eyeball networks are on the other side
of town. Or they're DSL providers in a not densely packed town, and needed two
POPs to get all the customers inside the cable foot limit for sane DSL. Or they
had their residential POP already up and running, and then acquired a business
ISP that already had a POP. Or they designed it based on what dark fiber or
coller was already in conduits or up on poles. I'm sure that at least one DSL
provider ended up with two POPs due to the headaches of trying to get one POP
past the incumbent, and there's probably somebody who ended up with one POP
because it was impossible to set up 2 with the incumbent...
I've never been overly fond of the Ma' Bell style designs with
humongous routers in centralized areas and L2-only haul out to the
last-mile termination. The failure modes of such systems often result
in hilariously large outages that are super visible publicly and put
egg on peoples' face. A neighborhood being down is a little easier to
manage from a customer relations POV, I think, and it's easy to make
that happen with distributed L3 termination.
We don't do Consumer services, but for our Enterprise customers, we run
IP/MPLS all the way into the Access and deliver services directly off
those devices. Like you, we don't like centralizing services for the
very same reasons that you state.
That said, I've often considered different architectures if we did
provide Consumer services - from centralized BNG's on a per-region or
per-town basis, as well as de-centralized BNG's on smaller routers (back
when the MX80 had just launched, but obviously not fit-for-purpose in
2019). Ultimately, I can't find a feasible way to deliver Consumer
services scalably and inexpensively in a de-centralized model. But, I
suppose, given the nature of the product and the ARPU, reasonable
centralization for such customers is not a bad thing.
There are some smaller, somewhat cost-effective full-touch routers
that can help bridge the gap between those two options, though.
Juniper's MX104 and the Cisco ASR1k series are some reasonable options
for that, but it'll definitely cost more than a cheap L3 switch for a
given amount of bandwidth.
Our poison is the Cisco ASR920 and Juniper MX204. I am yet to find any
other platforms with the size, density, capability and price for full
IP/MPLS capability in the Access.
I do like to separate SMB and Resi traffic, but it's mostly for
customer service reasons rather than technical reasons. That
separation rarely entails separate equipment but rather just VLANs and
PCPs, IP subnets, etc.
Many years ago, I did consider running both Consumer and Enterprise
traffic on one router - and for purposes of pride, I'm sure the major
vendors would like to boast that they could allow you to do this. But in
practice, it's probably a bad idea... BNG's have too many moving parts,
and for some platforms, there is actually special code optimized for BNG
deployments that may have an impact on traditional Enterprise or Service
Provider customers.
So I would separate BNG's from regular edge routers.
Now if you want to sell DIA type services where you can offer full BGP
tables, MPLS interconnection, etc., that's another matter. A need for
IPv4 CGNAT may, as well, but things like 464XLAT, lw4o6, MAP, etc. can
fix that up if you're willing to put some extra requirements on your
CPE/RG.
We do all this in the Access on our ASR920's and MX204's (once we start
deploying them).
If you're in a position where you want to or have to offer competitors
access to your network to sell service directly to customers, that's
also going to potentially really change the situation.
Why? Chances are they will require Ethernet access between their
customer and their head-end, which is a typical scenario.
Mark.
I do like to separate SMB and Resi traffic, but it's mostly for
customer service reasons rather than technical reasons. That
separation rarely entails separate equipment but rather just VLANs and
PCPs, IP subnets, etc.Many years ago, I did consider running both Consumer and Enterprise
traffic on one router - and for purposes of pride, I'm sure the major
vendors would like to boast that they could allow you to do this. But in
practice, it's probably a bad idea... BNG's have too many moving parts,
and for some platforms, there is actually special code optimized for BNG
deployments that may have an impact on traditional Enterprise or Service
Provider customers.So I would separate BNG's from regular edge routers.
Enterprise DIA is a whole different beast. For sure, that stays separate at least for now. Some of the forthcoming PON technologies have so much bandwidth that it may become attractive to start merging them at the access layer for smaller customers, and then I guess we'll have to see what the best way to handle L3 termination on that is.
If anything, just ensuring that the (often) separate tech teams have the proper access to it and knowledge of what the others are doing might be a bit of an issue.
If you're in a position where you want to or have to offer competitors
access to your network to sell service directly to customers, that's
also going to potentially really change the situation.Why? Chances are they will require Ethernet access between their
customer and their head-end, which is a typical scenario.
I'm thinking that, if you push L3 termination all the way out to the last access node (FTTN DSLAM being the obvious one here), you may then lack a decent way to haul pure Ethernet back to their head-end. If your L3 termination also supports MPLS, or Q-in-Q, you're probably fine. The latter might negate the potential advantages of distributed L3 from a routing POV by forcing you to again run STP or similar.
If you're doing L3 termination a bit more centralized, even if not with big behemoths on a "one per super-metro" basis, this may not be a problem at all. HFC and FTTx PONs might end up being like that inherently just because of the nature of the plant and tech that runs on it.
We run direct fiber connections to each house and business and terminate them on the same switches. Our switches are housed in small "huts" that are dispersed throughout the city and each handle a specific area then the huts are all connected in a ring. It really comes down to what your geography looks like.
Aaron
Good for you. None of this PON splitter nonsense.
Miles Fidelman
I've always felt PON is a tool for people who don't know how to design a proper network.
Aaron
Why is that?
I always thought PON was a technology that reduced the number of active ports, thus altering the port cost per subscriber significantly by not actually needing dedicated ports.
It also significantly reduces the requirement to distribute active equipment into the field while massively reducing the feeder fibre requirement. Point to point has its place to be sure, but mass market FTTH is not viable without PON's economics.
My statement was meant to be tongue in cheek. We deliver 1G to the home free of charge and make our money on the 10,40 and 100G connections. We haven't been able to deliver those capacities over PON so we've never really taken it seriously. As with everything else, you're use case and economics may vary.
Aaron
We do 1 gig over pon (gpon)...Calix E7 (olt)
Yes, it's my understanding, and I agree with previous post response, that PON is for using 1 fiber strand to a home (bidir , different wavelengths for xmt and rcv) and then I believe it even gets prism'd (however the heck they do it) into a 1/32 split or something like that so that you don't have to run direct fibers from every home back to the CO....
...AND, in a rural area, geez, those are loooonnnnggg fiber runs.... so a pon cabinet in the field helps greatly
Yes, 2.4g down and 1.2 g up is a concern when you've sold (oversubscribed) more bw than that
We are concerned and looking for ways to overcome this and keep up with subscriber bw demands all the time ... fun and job secure
-Aaron ....another Aaron
I want to work in a ISP environment and all the input here has helped. Thanks!
For a lot of us, PONs are a way of life and may not even have any 100G capable devices in our network, muchless enough to make our money on. While you may be so “lucky” to “never really take it seriously”, it is supporting hundreds of thousands, if not millions, of homes in the US.
PON is the lifeblood if many rural communities. I’m luckily to have a healthy mix of PON and AE operations since I’m located next to cities. But I’ve met cooperatives in the middle of no where with super low density where it’s 6 people + 2 donkeys on staff. AE would never work there, but PONs allow them cheap and available broadband options.
Unless someone wants to give enough funding to run AE to people’s homes, PONs will continue to allow many communities to have more than cellular internet access options, if that.
This email has been sent from my phone. Please excuse any brevity, typos, or lack of formality.
PON and AE both have their strengths and weakness and make sense for different deployment scenarios. My biggest problem with PON is that it seems some operators build their fiber plant for PON for all deployment cases and then it's extremely hard to back out of it and switch to AE. If you have AE you can switch to PON fairly easily, but not the other way around if you've put splitters in the manholes.
My assumption is that you'd be running full IP/MPLS all the way into the
Access. In that case, what I'm saying is that you can run EoMPLS to
deliver the service.
Mark.
PON in my view is well suited for residential distribution and use profiles. 10G/XG-PON at 10gig/2.5gig is a pretty serious residential connection and even 2.5/1 is pretty great for residential 1/1 symmetric service.
That said, I would in urban environments not recommend designing for GPON physical cable plan - go AE on your cabling. Play with PON if you want more headaches here with little redeeming features IMO. Instead, design rings/meshes, and think redundant/diverse path and entry/distro. There’s a reason telco standards work. These days there’s little reason to separate residential vs commercial traffic, it’s all packets at our scale. Our core is agnostic and switches anything we throw it at hardware speed, and it’s HA (min 2 core routers in every POP - even some customer buildings have diverse/redundant fiber entry from us now, back to multiple $alldayallnightjob POPs no less, in some cases to meet regulatory minimum standards compliance. All of our DCs are built this way. Fact is, if you want a network to be fast as hell, and never ever go down, think redundant everything with diversity.
That said, for rural distribution, especially cheap aerial residential services in far flung locations - there’s literally nothing finer and faster and more cost effective than GPON - which is HUGELY important for reaching the final 15 MILLION Americans that do not have broadband internet connections at all.
For those people, GPON can be nothing short of utterly life transforming.
PON in urban areas absolutely makes sense. Maybe less in a high rise area, where each building can have a small building wide network of its own. But it in areas with single family homes PON is king.
Our POPs can have up to 10 000 customers each. All on a single 96 fiber strand cable leading into the POP building. We have extra ducts, but nothing that would allow us to change that to a point to point network. That would require 100x that 96 fiber cable.
With extra ducts it would be possible to rebuild from PON to point to point. But it would require massive investments. Basically you would have to invest all that we saved by building PON. For starters, you would have to have many more POPs.
And yes, there are splitters in the hand holes. This is not what stops you from rebuilding from PON. It is the fact that we never paid for extra fiber. The backbone in a sub area is typically build with a 24 fiber strand cable. Because fibers are not free and are actually quite expensive as the number of fibers grow and the distances get longer. We can do a few point to point connections, for example if we need to deliver a commercial service or for our own needs (to connect POPs etc).
We are not big on commercial services. But if we were, I would use WDM splitters for that. Or the long awaited 10G PON if that ever arrives and turns out at a price point that works.
Regards,
Baldur
I should probably have mentioned that in this sense I view “urban” as exclusive to “single family homes” - meaning I’m talking about high density modern urban with under grounding requirements - and high rise residential towers.
We are the opposite, we are presently enterprise, midsize, and exotic-small business only, and have no residential arm or support structure (or SLA expectations, or standards or lack thereof) of a residential connection.
-Ben.
My preference, for the home, would be Active-E. But I do understand the economics that may support PON, and my position on that has softened over the years. My service provider delivers their FTTH service to me via PON, and for the most part, it’s been all good.
That said, I was particularly impressed with what CDE Lightband did in Clarksville, Tennessee, where they deployed their FTTH network with Active-E using Brocade to over 60,000 subscribers:
If I had to build a consumer broadband network and had the budget (and owned the fibre) to do so, I’d definitely always choose Active-E:
In South Africa, we have an access network operator that uses Active-E primarily to deliver their service, making it perhaps the only FTTH provider not using PON to do this. I find this quite fascinating.
Mark.