DWDM Mux/Demux using 40G Optics

We are building a 40G metro ring using 40-Gigabit Ethernet QSFP+
Transceivers. Specifically, we are using Juniper JNP-QSFP-40G-LR4. This is
a QSFP+ Transceiver with a LC duplex head. We only have one pair of single
mode dark fibers around the ring. Our distance between nodes around the
ring are all less than 10KM, so we can use standard optics.

We go out of one JNP-QSFP-40G-LR4 and into another JNP-QSFP-40G-LR4. There
are no passive muxes involved. This is working great for 40G.

My understanding is a JNP-QSFP-40G-LR4 is really a transceiver with a CWDM
mux built into it. The spec sheet shows it sends 4 10G channels:

https://www.juniper.net/documentation/en_US/release-independent/junos/topics/reference/specifications/optical-interface-qfx-support.html

Lane wavelength
Lane 0–1264.5 nm through 1277.5 nm
Lane 1–1284.5 nm through 1297.5 nm
Lane 2–1304.5 nm through 1317.5 nm
Lane 3–1324.5 nm through 1337.5 nm

This setup is working fine, but now we want to do more than 40G around the
ring. To my knowledge there are no other 40G QSFP+ transceivers that use
four other channel/lanes than the ones already being used, so they only way
to go higher than 40G is to stack 10G or 100G channels ontop of the fiber
pair using a passive mux.

100G is too expensive for the time being, so we are looking to add 10G
channels to a ring that already have one 40G channel using the QSFP+.

I was reading this tutorial, and it mentions "there is a 1310 nm port
integrated in a 40 channels DWDM Mux/Demux system. The 1310nm added port is
a Wide Band Optic port (WBO) added to other specific DWDM wavelengths in a
module. When we run out of all channels in a DWDM Mux/Demux system, we can
add the extra optics via this 1310nm port."
http://www.fs.com/upgrade-to-500g-with-40ch-dwdm-mux-demux-system-aid-493.html

What I can't seem to understand is they are mentioning that this 1310 port
can pass QSFP+ signals, so it sounds like its really a 1270nm through
1330nm port? Is this what they mean by Wide Band Optic port (WBO)?

We don't need 40 10G channels plus a 40G for a total of 440G. More than
likely we are looking at a 8 channel mux/demux, and 1 40G port for a total
of 120G.

I don't care if we do CWDM vs DWDM, but I assume it will be hard to find a
CWDM mux that has one LC dupluex input for 1270nm through 1330nm channels?

Maybe I should just ditch the 40G QSFP+ optics and use all 10G optics, but
the switches I am using have 48 10G SFP+ ports and 6 QSFP+ ports built in.
I know there are 40G breakout cables, but the whole point of 40G is to
aggregate VLAN/circuits.

Has anyone done this before?

[snip]

Maybe I should just ditch the 40G QSFP+ optics and use all 10G optics,
but the switches I am using have 48 10G SFP+ ports and 6 QSFP+ ports
built in. I know there are 40G breakout cables, but the whole point of
40G is to aggregate VLAN/circuits.

Depending on switch vendor, it might be possible to create real 40GE interface
also from 4 consecutive 10G SFP+ ports. Then you can use standard 10G CWDM/DWDM
SFP+ optics and still benefit from wire-speed 40G uplinks.

Note that this is *not* etherchannel/LAG, but true HW-based 40GE mode.

    M.

Answers in-line below.

Faisal Imtiaz
Snappy Internet & Telecom
7266 SW 48 Street
Miami, FL 33155
Tel: 305 663 5518 x 232

Help-desk: (305)663-5518 Option 2 or Email: Support@Snappytelecom.net

From: "Colton Conor" <colton.conor@gmail.com>
To: "nanog list" <nanog@nanog.org>
Sent: Monday, June 19, 2017 2:26:55 PM
Subject: DWDM Mux/Demux using 40G Optics

We are building a 40G metro ring using 40-Gigabit Ethernet QSFP+
Transceivers. Specifically, we are using Juniper JNP-QSFP-40G-LR4. This is
a QSFP+ Transceiver with a LC duplex head. We only have one pair of single
mode dark fibers around the ring. Our distance between nodes around the
ring are all less than 10KM, so we can use standard optics.

We go out of one JNP-QSFP-40G-LR4 and into another JNP-QSFP-40G-LR4. There
are no passive muxes involved. This is working great for 40G.

My understanding is a JNP-QSFP-40G-LR4 is really a transceiver with a CWDM
mux built into it. The spec sheet shows it sends 4 10G channels:

https://www.juniper.net/documentation/en_US/release-independent/junos/topics/reference/specifications/optical-interface-qfx-support.html

Lane wavelength
Lane 0–1264.5 nm through 1277.5 nm
Lane 1–1284.5 nm through 1297.5 nm
Lane 2–1304.5 nm through 1317.5 nm
Lane 3–1324.5 nm through 1337.5 nm

YES this is correct and goes for pretty much all of the SMF QSFP+ 40g Optics. (they utilize 4 cwdm channels)

This setup is working fine, but now we want to do more than 40G around the
ring. To my knowledge there are no other 40G QSFP+ transceivers that use
four other channel/lanes than the ones already being used, so they only way
to go higher than 40G is to stack 10G or 100G channels ontop of the fiber
pair using a passive mux.

Typically you would stack 40G + 10g Channels or 100G+10g Channels
(100g optics would be using 4 channels as well).

100G is too expensive for the time being, so we are looking to add 10G
channels to a ring that already have one 40G channel using the QSFP+.

Yep, that would be the cost effective way to do it.

I was reading this tutorial, and it mentions "there is a 1310 nm port
integrated in a 40 channels DWDM Mux/Demux system. The 1310nm added port is
a Wide Band Optic port (WBO) added to other specific DWDM wavelengths in a
module. When we run out of all channels in a DWDM Mux/Demux system, we can
add the extra optics via this 1310nm port."
http://www.fs.com/upgrade-to-500g-with-40ch-dwdm-mux-demux-system-aid-493.html

What I can't seem to understand is they are mentioning that this 1310 port
can pass QSFP+ signals, so it sounds like its really a 1270nm through
1330nm port? Is this what they mean by Wide Band Optic port (WBO)?

Yes that would be correct, 1310nm is simple nomenclature when used with 40g/100g QSFP+ SMF optics

We don't need 40 10G channels plus a 40G for a total of 440G. More than
likely we are looking at a 8 channel mux/demux, and 1 40G port for a total
of 120G.

I don't care if we do CWDM vs DWDM, but I assume it will be hard to find a
CWDM mux that has one LC dupluex input for 1270nm through 1330nm channels?

If you look at the CWDM Muxes (8 or 9 channel) you will notice a common configuration of

    Upgrade Port (expansion port) + 1450 or 1470 to 1610nm

    in the DWDM muxes you will see them listed as # of Port + 1310 pass thru channel.

These are exactly what you are looking for ..... :slight_smile:

Maybe I should just ditch the 40G QSFP+ optics and use all 10G optics, but
the switches I am using have 48 10G SFP+ ports and 6 QSFP+ ports built in.
I know there are 40G breakout cables, but the whole point of 40G is to
aggregate VLAN/circuits.

Has anyone done this before?

Am in the process of lighting a number of locations in this manner...

Faisal,

How would he inject his current 4x10 40g into the mux which is currently on a single LC cable?

Luke Guillory
Network Operations Manager

Tel: 985.536.1212
Fax: 985.536.0300
Email: lguillory@reservetele.com

Reserve Telecommunications
100 RTC Dr
Reserve, LA 70084

Verify pass-through frequencies for the 1310 (or equivalent) for the passive mux in question. This would only work for a single channel.

Bench test of the system, with the muxes...

sorry for the large pictures :slight_smile:

Faisal Imtiaz
Snappy Internet & Telecom
7266 SW 48 Street
Miami, FL 33155
Tel: 305 663 5518 x 232

Help-desk: (305)663-5518 Option 2 or Email: Support@Snappytelecom.net

I guess that is the real question. Besides the client ports that are
clearly identified by channel number on Muxes, what channels can the
special ports handle?

http://www.fs.com/products/43723.html It has 4 special service port options:

1. Expansion Port (Based on what I am seeing, I think this would be to
stack another mux if you needed more channels. So I assume it allows all
channels to be added besides the client channels?)
2. Monitor Port (I think this is just a tap that you would hook a monitor
up to, and be able to see all channels coming through with a meter. I
assume not a good idea to add/drop channels through this port)?
3. 1310nm Port (Labeled as 1310, but clearly allows more than just 1310
since tutorial is saying it supports QSFP+ which is 1270 - 1330 nm, so what
range does it really support or is there no a range?)
4. 1550nm Port (Labeled as 1550nm, but I wonder if its like the 1330nm?)

Would you recommend a monitor port on every mux you buy?

Answers in-line ...

Faisal Imtiaz
Snappy Internet & Telecom
7266 SW 48 Street
Miami, FL 33155
Tel: 305 663 5518 x 232

Help-desk: (305)663-5518 Option 2 or Email: Support@Snappytelecom.net

From: "Colton Conor" <colton.conor@gmail.com>
To: "Mike Hammett" <nanog@ics-il.net>
Cc: "Luke Guillory" <lguillory@reservetele.com>, "nanog list" <nanog@nanog.org>,
"Faisal Imtiaz" <faisal@snappytelecom.net>
Sent: Monday, June 19, 2017 3:30:37 PM
Subject: Re: DWDM Mux/Demux using 40G Optics

I guess that is the real question. Besides the client ports that are clearly
identified by channel number on Muxes, what channels can the special ports
handle?
http://www.fs.com/products/43723.html It has 4 special service port options:

1. Expansion Port (Based on what I am seeing, I think this would be to stack
another mux if you needed more channels. So I assume it allows all channels to
be added besides the client channels?)

Exactly... this is basically a pass thru port, i.e. what is not getting mux/demux should get passed thru (keep the insertion loss in mind).

2. Monitor Port (I think this is just a tap that you would hook a monitor up to,
and be able to see all channels coming through with a meter. I assume not a
good idea to add/drop channels through this port)?

I don't use this port, but supposedly it will pass a fraction 5% of the light from the main port so that it can be monitored. May be someone else can offer some practical use for this port.

3. 1310nm Port (Labeled as 1310, but clearly allows more than just 1310 since
tutorial is saying it supports QSFP+ which is 1270 - 1330 nm, so what range
does it really support or is there no a range?)

Not sure about the range question, but this is the port for having the 40g/100g QSFP+ pass thru

4. 1550nm Port (Labeled as 1550nm, but I wonder if its like the 1330nm?)

I have not had the need to explore this in detail, but from my initial understanding, this can be used for ZR (long range optics) and or to stack a DWDM Mux

Would you recommend a monitor port on every mux you buy?

As I shared above, I don't.

And how would he pass his current 40g through that mux? Unless I'm misreading your email which I took as he can use his current setup along with a 40g 1310, though I'm thinking you're saying he can use 1310 40g with colored up 10gs alongside of it.

Let me try to explain better...

All Single Mode Fiber 40g Optics are using 4 cwdm channels ...

If you use a cwdm mux/demux with and expansion port and it is only mux/de-muxing 1450 to 1610 (i.e. not using the 1270-1330) you can use the expansion port to connect the 40g Optics

If you have a CWDM or DWDM Mux, with a specific 1310 pass thru port (Wide-band etc... check the specs) then you can plug the 40g Optics on to that port and it will pass the 4 channels thru it.

e.g. with the cwdm mux (see picture in previous post) ..you end up with 1x40g (lane) + 8 or 9 10g (cwdm lanes).

Regards.

Faisal Imtiaz
Snappy Internet & Telecom
7266 SW 48 Street
Miami, FL 33155
Tel: 305 663 5518 x 232

Help-desk: (305)663-5518 Option 2 or Email: Support@Snappytelecom.net

Verify the wavelengths passed by the 1310 port. Verify the wavelengths used by his existing 40G optic. Plug the 40G optic into the 1310 port.

Gotcha, figured I misread it. Sorry it's Monday.

I tried to send some pictures, but looks like the message got stuck for moderator.

Here is a link to pictures what Colton is trying to accomplish.... (my bench test :slight_smile: )

https://1drv.ms/a/s!Ar2zoQlxIvI1gdV9tYj96YUDWElu6w

Regards.

Faisal Imtiaz
Snappy Internet & Telecom
7266 SW 48 Street
Miami, FL 33155
Tel: 305 663 5518 x 232

Help-desk: (305)663-5518 Option 2 or Email: Support@Snappytelecom.net

Pics came in on my side, that's when I figured you were saying 1 40g plus 10s.

Mike,

Have any suggestion on a meter for CWDM and DWDM that is low cost?

Thanks for the answers. From the sounds of it, no one knows the real
difference between the expansion port, 1310 port, and 1550 port. For real
world applications, I would assume the monitor port would be to plug in a
handheld meter, and see which channels are coming through that node without
breaking the ring. Not sure if their would be a monitor port for both
directions is you were using a OADM?

I'd imagine they vary based on vendor, so you'd have to check with the specific vendor in terms of absolute technical specifications.

A 1310 and 1550 port only allow those channels plus or minus some, manufacturer dependent.
An expansion port passes everything not used by that device.

Some manufacturers are even configurable pre-order, so you could get exactly what you needed (other than multiple 40G channels).

My understanding is that ports on the client side would have cut filters for that color, having 1310 and 1550 are nice since you can hang gear you already have off the mux for OOB and so on. I don’t think I have any CWDM with expansion ports to test with.

We use the following for simple testing, is it there and what’s the power.

http://solid-optics.com/tools/power-meter/so-osa-cwdm-18ch-id1685.html

Luke Guillory
Network Operations Manager

        [cid:image300878.JPG@eb7e400b.44984f6e] <http://www.rtconline.com>

Tel: 985.536.1212
Fax: 985.536.0300
Email: lguillory@reservetele.com
Web: www.rtconline.com

        Reserve Telecommunications
100 RTC Dr
Reserve, LA 70084

Disclaimer:
The information transmitted, including attachments, is intended only for the person(s) or entity to which it is addressed and may contain confidential and/or privileged material which should not disseminate, distribute or be copied. Please notify Luke Guillory immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. Luke Guillory therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission.

I guess that makes sense. The plus or minus some is the question. FS is
claiming their 1310 port support QSFP+, which is 1270, 1290, 1310, and 1330
combined. I understand you can us 1310, but I am still scratching my head
as to how they all one minus and two above 1310 to work. Of course they
don't have any datasheets to show the range either.

I think you will find the "monitor" port is most likely to be used for "lawful" intercept by unnamed government entities.