Fiber at Home? Fiber Wall Plate? How to/Practicality/Cost Questions.

[TheNatural]

I've been planning my entry into the home lab world with some serious help from this site and I have some networking questions regarding fiber in the home (some questions for home office network and some for home lab). My current "server room" (will also be home lab) shares a wall with my home office. Last time I renovated the house I ran cat5e everywhere for GbE. With ever increasing file sizes I want MOAR SPEED, but I'm a fiber n00b and 10GbE over cat6, etc, doesn't sound very exciting.

The most important and most immediate thing I want is a fast connection between my storage server and my workstation. Straight line distance is about 3m; probably double that for a cable run through the wall.
-Is a ConnectX2 a reasonable NIC for this? I'm cost conscious and the SFP+ 10Gb options seem the most affordbale on eBay
-Can I use "good fiber" in the wall so I don't have to replace that cable to upgrade to 40Gb or 100Gb in the future? What would that "good fiber" be called? All I've found so far is recommendations to buy "OM3" for home lab use but I don't know what the other options or reasons for them are.
-How do I actually do the connections at the wall plates? Adding keystone jacks to the ends of cat5e was easy to figure out, but I feel like fiber is not so simple to deal with?
-Aliexpress has some fiber optic wall plates for only a couple dollars - is there a reason not to cheap out on these? If so, do you have a recommendation on where to get better wall plates? (I'm in Canada)
-Should I leave the GbE connection from the storage server to the switch that services the home and create another subnet for the point to point fiber connection between server and workstation? Or is there a better way to do what I want to do?

Now for the lab! I want to mess around with fiber in the lab because curiosity.
-Fiber switches are friggin expensive, it seems. I also don't know much (anything) about them. Do you have a recommendation for a switch to learn with?
-Is it possible to have a switch that I can use with both SFP+ 10Gb NICs and QSFP 40Gb NICs?
-Reading about infiniband made me curious. How does it interact with the other stuff I've been talking about? Can you use a fiber ethernet network and infiniband together or is that a stupid question?

Sorry that turned into such a read. Thanks in advance!

 

[PigLover]

Fiber keystone plugs are dead easy...easier thn the ones you used for cat5/6.

Only hard part is that you need to pull terminated fiber and the LC connector is a pain to pull. But three meters through hollow wall shouldn't be too hard.

Cx2 cards are good. Cx3 are usually not that much more. Either works. They have been my go-to card for 10gbe on Linux and Windows.

Sent from my SM-G950U using Tapatalk

 

[whitey]

Plenty of dedicated 10G/40G switches out there, few that I know of w/ it on one chassis/frame hence why I went w/ a Juniper EX4300.

Unless your doing HPC work, don't mess w/ IB IMHO, the ones that do/can bridge ethernet/IB fabric worlds are insanely expensive and it certainly gets away from the KISS principle. I went down that rabbit hole and while it was fun to learn in relation to what I do at work i ended up taking the ethernet route and just getting a 40GbE switch.

 

[TheNatural]

Fiber keystone plugs are dead easy...easier thn the ones you used for cat5/6.

Only hard part is that you need to pull terminated fiber and the LC connector is a pain to pull. But three meters through hollow wall shouldn't be too hard.

Cx2 cards are good. Cx3 are usually not that much more. Either works. They have been my go-to card for 10gbe on Linux and Windows.

Sent from my SM-G950U using Tapatalk

Thanks! I actually have some of the drywall torn down in that room anyway so it'll be a piece of cake to pull the fiber before I patch it back up.

Am I correct that I would measure the exact length I need, then order a fiber cable with ends already on it to plug into a keystone? My understanding is that terminating fiber requires advanced tools and knowledge that make it not practical to do myself?

Plenty of dedicated 10G/40G switches out there, few that I know of w/ it on one chassis/frame hence why I went w/ a Juniper EX4300.

Unless your doing HPC work, don't mess w/ IB IMHO, the ones that do/can bridge ethernet/IB fabric worlds are insanely expensive and it certainly gets away from the KISS principle. I went down that rabbit hole and while it was fun to learn in relation to what I do at work i ended up taking the ethernet route and just getting a 40GbE switch.

Thanks. I'm curious about the HPC world and I would like to learn more about it and IB, but I'll leave that for the more distant future.

 

[PigLover]

You definitely don't want to try to terminate fibers, so yes - buy terminated cables to measure. Don't cut it too close. Better to be a bit long than a bit short. You can always coil a few feet at the end. You'll be glad for the slack if you ever need to make changes later.

 

[BackupProphet]

Isn't it better to just pull a single LC connector and use BiDi transceivers? That is what I'm planning to do soon.

 

[Mike Bailey]

Apologies in advance for formatting, I am typing on a train

Do yourself a favor or and buy multiple fiber pairs.

Pick up OFNP/Plenum rated, UPC (unpolished connector), single mode fiber with LC connectors.


Why? Plenum is the standard required for in-laws installations. UPC connectors are cheaper than APC and you don't need to worry about dB losses over 3 meters. The upc connectors will be less finicky as well. Single Mode has the capability to go all the way up to 100G on a single pair. Future proofed.

Get multiple strands run between the two walls. You never know what you'll need and the fiber is dirt cheap compared to the time invested getting it pulled.

Fiber: 5m (16ft) LC UPC to LC UPC Duplex 2.0mm OFNP 9/125 Single Mode Fiber Patch Cable | FS.COM
Transceivers: Custom & OEM 10G SFP+ Transceiver Modules | FS.COM


You can go on their transceiver page and pick the custom option. They'll code the transceiver to support whatever switch or NIC vendor you have. You'll want the SR (short reach) transceivers.

Try to terminate all strands between the two walls. Yes that would likely mean a 6-port keystone on each side. Again, the extra labor and material cost is insignificant compared to going back a 2nd time. Keep the white caps on the keystone jacks on both sides unless you are using the ports. Dirt of ANY kind will destroy the connectors. Do not drag the bare connectors on the floor. You'll destroy them.

If you're feeling fancy, you can buy BiDi optics from FS that will let you run 1 connection PER individual strand. That doubles your capacity if you feel so inclined.

Don't bother buying multi mode fiber. It may be cheaper now, but we're talking about literally a couple of dollars. Single mode has been future proof for quite a while now, whereas were up to our 4th revision on multi mode (OM4).

Good luck!

 

[TheNatural]

Apologies in advance for formatting, I am typing on a train

Do yourself a favor or and buy multiple fiber pairs.

Pick up OFNP/Plenum rated, UPC (unpolished connector), single mode fiber with LC connectors.


Why? Plenum is the standard required for in-laws installations. UPC connectors are cheaper than APC and you don't need to worry about dB losses over 3 meters. The upc connectors will be less finicky as well. Single Mode has the capability to go all the way up to 100G on a single pair. Future proofed.

Get multiple strands run between the two walls. You never know what you'll need and the fiber is dirt cheap compared to the time invested getting it pulled.

Fiber: 5m (16ft) LC UPC to LC UPC Duplex 2.0mm OFNP 9/125 Single Mode Fiber Patch Cable | FS.COM
Transceivers: Custom & OEM 10G SFP+ Transceiver Modules | FS.COM


You can go on their transceiver page and pick the custom option. They'll code the transceiver to support whatever switch or NIC vendor you have. You'll want the SR (short reach) transceivers.

Try to terminate all strands between the two walls. Yes that would likely mean a 6-port keystone on each side. Again, the extra labor and material cost is insignificant compared to going back a 2nd time. Keep the white caps on the keystone jacks on both sides unless you are using the ports. Dirt of ANY kind will destroy the connectors. Do not drag the bare connectors on the floor. You'll destroy them.

If you're feeling fancy, you can buy BiDi optics from FS that will let you run 1 connection PER individual strand. That doubles your capacity if you feel so inclined.

Don't bother buying multi mode fiber. It may be cheaper now, but we're talking about literally a couple of dollars. Single mode has been future proof for quite a while now, whereas were up to our 4th revision on multi mode (OM4).

Good luck!

This is fantastically useful information. You've cleared up a lot of the fiber and connector stuff I was struggling with. Thank you!

Is FS.com the go-to vendor?

Edit: Do I have to worry about compatibility using SMF with SR transceivers? My previous reading suggested that SMF was intended for long range and MMF was intended for short range.

 

[tullnd]

Here's my suggestion...you really want fiber for one of two scenarios. 1) between switches. 2) general interconnect.

I recently was deploying ethernet in my home that I just purchased. I did a wall mount small rack setup in the basement and ran ethernet from there to the first floor, using CAT6. Most of my runs are short enough there that they can support 10Gb in the future. However, to get to my second floor, I'd need to run multiple bundles of CAT6 all the way to the attic(through first and second floor) and then drop back into each various room. I decided instead to just do one thin bundle of fiber and locate a second switch in a closet upstairs.

I purchased two sets of pre-terminated fiber in both single mode and multi mode(OM4). OM3 or OM4, really isn't a big deal. For me, the cost difference was almost nothing, so I did OM4 as I'm not approaching the distance limits of either. I now have both options and a backup of each one and it was not costly at all.

I did mine via "dump" faceplates, rather than keystones. I figured the keystone interconnect is just one more failure point, so I'd rather just go straight to the switch. If I was doing a single PC, I may consider a keystone...but honestly...we're very early in the 10Gb ethernet game, so they may get the cost/heat/efficiency under control soon to make 10Gb ethernet to be more intriguing over fiber soon.

I personally wouldn't bother with keystones and would just drop the cables in for inter-switch in most scenarios. If I did want a dedicated run, I'd probably also just do one of those "dump" type faceplates again and put it low and behind a desk or something where people won't see the cable coming out.

Also...there's a comment above refferring to UPC to "unpolished" vs APC. Both UPC and APC are polished. UPC is just polished "flat", while APC is at an angle. Generally, APC is used for video transmission for technical reasons. Data interconnects will almost always use UPC. The rare exception is data equipment that interacts with video equipment(think IPTV equipment connecting to an OLT or injecting RF signal as an alternate wavelength in WDM).

 

[Tom5051]

Fibre is only useful in runs that are longer than Ethernet can handle due to the properties of copper vs resistance. Electricity travels at the speed of light so no gains for short runs. Spend your money on better hardware.

 

[BackupProphet]

Fiber has plenty of gains for short runs, the biggest are latency and that is uses less power. Others are cost, flexibility and potential upgrade path up to 400G on same cable.

 

[i386]

-Is a ConnectX2 a reasonable NIC for this? I'm cost conscious and the SFP+ 10Gb options seem the most affordbale on eBay

Awesome cheap cards, great support for third party transceivers.

-Can I use "good fiber" in the wall so I don't have to replace that cable to upgrade to 40Gb or 100Gb in the future? What would that "good fiber" be called? All I've found so far is recommendations to buy "OM3" for home lab use but I don't know what the other options or reasons for them are.

OM3 or OM4 are categories for multi mode fibers. The higher the number the better it supports longer distances and higher bandwidth.

-Should I leave the GbE connection from the storage server to the switch that services the home and create another subnet for the point to point fiber connection between server and workstation? Or is there a better way to do what I want to do?

If you can afford a 10gbe switch connect the server and your workstation with 10gbe to the switch. It's very like that you will see an improvement in transfer speeds between your server and clients without changig the storage in your server.

-Fiber switches are friggin expensive, it seems. I also don't know much (anything) about them. Do you have a recommendation for a switch to learn with?

There are different options for different prices. For home use (24x7, no dedicated server room) I would get quiet switches like the TP-Link T1700G-28TQ (24 1gbe ports, 4x sfp+ 10gbe ports) or a Juniper 4300 switch like @whitey has in his signature.
For a homelab and learning network related stuff I would buy used enterprise switches like the LB6M or for 10gbe + 40gbe Arista 7050 seriesswitches.

-Is it possible to have a switch that I can use with both SFP+ 10Gb NICs and QSFP 40Gb NICs?

Yes, check the Juniper 4300 series (they are quiet compared to enterprise switches) or the Arista 7050 switches (these are loud enterprise switches)

-Reading about infiniband made me curious. How does it interact with the other stuff I've been talking about? Can you use a fiber ethernet network and infiniband together or is that a stupid question?

It can work, but like @whitey said it's expensive and introduces complexity:

Thanks. I'm curious about the HPC world and I would like to learn more about it and IB, but I'll leave that for the more distant future.

IB is pretty interesting, but I would use it only for point-to-point links at home.

Isn't it better to just pull a single LC connector and use BiDi transceivers? That is what I'm planning to do soon.

BiDi transceiver for 40gbe or faster are damn expensive right now ($300 for 40gbe lc vs. $49 for 40gbe mpo @ fs.com)

Don't bother buying multi mode fiber. It may be cheaper now, but we're talking about literally a couple of dollars. Single mode has been future proof for quite a while now, whereas were up to our 4th revision on multi mode (OM4)
...
You can go on their transceiver page and pick the custom option. They'll code the transceiver to support whatever switch or NIC vendor you have. You'll want the SR (short reach) transceivers.

SR = Short Reach = 850nm over multimode fibers or did I miss something?

 

[TheNatural]

SR = Short Reach = 850nm over multimode fibers or did I miss something?

This is also what I thought... although I've noted my extreme lack of proficiency on the topic. A google search for "SR over SMF" and the like didn't help me much. If single mode is superior in future proofing, maybe I should just spend the extra for LR transceivers so I don't have to pull new fiber in the future?

Thanks for the rest of the comments too, @i386

 

[wildchild]

And the latest firmware the unfi 10G switch would be an option too, especially when running a UTP/gbic scenario and unifi ap's

 

[Tom5051]

LR over short distance are not a good idea. Pretty sure it won't work.

 

[Tom5051]

Fiber has plenty of gains for short runs, the biggest are latency and that is uses less power. Others are cost, flexibility and potential upgrade path up to 400G on same cable.

Over copper distances, fibre tends to have more latency actually. The packets have to be turned into light and back again electronically. This causes more latency, not reduce it.
Cost vs benefit, fibre is many times more expensive than copper. Technically you could upgrade 3 times with copper for the same fibre and that is assuming that the current fibre technology doesn't go the way Ethernet did with Cat 6 cable.
Remember everyone installing CAT 6 to future proof? ahahaha FAIL

 

[BackupProphet]

No, it is still faster for shorter distances. 10Gb SFP+ single port = cheaper than dirt

 

[TheNatural]

Okay... I can't find any info on single mode fiber with SR transceivers, so I think I'm going to give up on that and go with the OM4. Is this diagram a reasonable plan? Wall cable is a little longer than needed so I'll have quite a bit of slack; I'll probably put a cable loop in the wall or something. Keystone jacks are "LC to LC multimode duplex"

 

[Evan]

That will work fine

 

[cookiesowns]

LR over short distance are not a good idea. Pretty sure it won't work.

While it's recommended to have some sort of an attenuation, I run 2M ( 6.6 feet )short OS2 patch cables on LR optics with 0 issues. It's only when you run ER or even longer distance laser optics where doing short distances might cause premature burn out.