Help hooking it all together

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

smithse79

Active Member
Sep 17, 2014
205
39
28
44
I’m not really sure where to put this, but since I’m looking for specific advice about networking as part of it, I figured here was as good a place as any.

Current Storage: I am in the process of upgrading my storage solution. Currently, I have a SansDigital 4 bay eSATA enclosure connected to a Highpoint Rocket RAID 622. In it, there are 3 WD20EARS Green Drives that have been set to not park the heads using WDIDLE3 in RAID5. This setup has been running almost 24/7/365 for nearly 4 years. It was originally in my dekstop then moved to a 1U server once that was purchased.

Current Servers:
  1. I have a Dell CS24-TY with Dual L5520s and 36GB of RAM (22GB not allocated to VMs). This machine is a Hyper-V host running Server 2012 R2 and 3 VM's that run constantly. This currently houses the Highpoint 622 and enclosure
    1. DC, DHCP, DNS
    2. File Server, Plex, Torrent
    3. Git (not really necessary, but no real reason to take it down either)
  2. I also have a Dell CS24-SC with Dual L5420s and 8GB of RAM (RAM can be bought cheaply right now). This computer is running CentOS 6.5 and is currently used primarily for encoding DVDs and Blurays. I have also dabbled in compiling Android/Cyanogenmod with it.
  3. I have purchased a Dell J23 JBOD enclosure. I was able to get this for $50+$38 shipping and then found drive trays at $25 for 15 trays.

My initial plan was to get an HBA and install it into the linux box and run ZFS on Linux and directly connect the 2 computers via their second GbE ports. By trade I am a Windows sysadmin but do some backup linux admin at work as well. ZFS seems to be something that I need to be learning as our storage pools at work get larger and larger. Also the advantages ZFS gives over traditional RAID are very appealing to me.

Now, I find myself wavering. I've found out that my CS24-TY has a mezzainne slot for a 10Gb fiber card. This gives me a lttle bit of flexibility that I didn't have before as both machines only have one PCIe slot. I could put a SAS HBA into the CS24-TY and use Windows Storage Spaces, get the mezzanine fiber channel card, and a fiber channel card for the CS24-SC and direct conect the 2 servers so that I would the be able to directly access the shared drive when needed for encoding (something a 1Gb link would be too slow for).

However, this is where my knowledge gets a bit fuzzy. I've never worked with fiber before and finding the mezzanine card has not been easy. I have found this: Dell 10GB Fiber Channel Dual Port Host Bus Adapter Jpynn | eBay and others of the same part number fairly readily, but they are listed as HBAs. Does this mean that they are for fiber channel storage only, or can they be used for networking? I would imagine that I'd be able to find something that they can connect to on the other end readily as well or am I wrong there?

Another question I find myself asking is Storage Spaces vs ZFS. I have never used either but ZFS is obviously the more mature product. ZFS seems like it is more tailored to what I would like to be doing, but obviously, I'm far more comfortable in Windows. SS may be the smarter long term solution for me as well since I have made a career out of Windows and need to learn its ways.

And last for now. I have been looking at the Dell H200E as far as the HBA. It seems to fit best with my hardware as-is. Everything I have is Dell so I don't forsee any compatibility problems, The J23 is 3GbSAS, the H200 is 6Gb so no slow-downs there. I may have to flash it to IT mode but there are walk-throughs here for that if needed. Any better solutions?

Thanks for reading and any insights
 

TuxDude

Well-Known Member
Sep 17, 2011
616
338
63
I hate trying to find out information about what options are available from Dell for normal servers, and Google is telling me that your CS24-TY is a custom thing so I'm not even going to try any further to find out what options would be available for your mez slot.

But I can confirm for you that FiberChannel is a block storage protocol, you cannot do TCP/IP networking over it. While you theoretically could put FC HBA's into both boxes, direct-connect them, and configure one of the HBA's to run in target mode it would still not really accomplish what you are trying to do. If 10GbE over fiber cable is what you want (I'm pretty sure it is), then you are looking for 10GBASE-SR, which will operate over relatively cheap multimode cables - stay away from OM1, but OM2 will be fine for 10G for around up to about 80m length, OM3 will get you to 300m distance, OM4 to 400m.
 

smithse79

Active Member
Sep 17, 2014
205
39
28
44
I hate trying to find out information about what options are available from Dell for normal servers, and Google is telling me that your CS24-TY is a custom thing so I'm not even going to try any further to find out what options would be available for your mez slot.

But I can confirm for you that FiberChannel is a block storage protocol, you cannot do TCP/IP networking over it. While you theoretically could put FC HBA's into both boxes, direct-connect them, and configure one of the HBA's to run in target mode it would still not really accomplish what you are trying to do. If 10GbE over fiber cable is what you want (I'm pretty sure it is), then you are looking for 10GBASE-SR, which will operate over relatively cheap multimode cables - stay away from OM1, but OM2 will be fine for 10G for around up to about 80m length, OM3 will get you to 300m distance, OM4 to 400m.

Yeah, it's almost impossible to find out what mezz cards will work in these things. These have been sold by the thousands so I was hoping someone here might have one and maybe know for certain though. If I can't get a 10GbE mezzanine card then that whole idea goes out the window anyway. These servers are 1U and only have a single expansion slot and one of them will have to have a SAS HBA for the JBOD.
 

smithse79

Active Member
Sep 17, 2014
205
39
28
44
You can get this.

85M9R Dell PowerEdge C2100 C1100 MEZZANINE SAS RAID Controller | eBay

I have it in my cs24-ty. It would free up your pcie slot. It is a LSI SAS2008 card that you can do raid or just JBOD. The price is a little high but invaluable in freeing up that pcie slot.
I've seen that one as well but from what I understand, that goes in a forward Mezzanine slot and is for the 4 3.5" hot-swap bays on the unit itself. I'm attaching an external JBOD and will need external SAS ports.
 

markarr

Active Member
Oct 31, 2013
421
122
43

smithse79

Active Member
Sep 17, 2014
205
39
28
44
That thought has crossed my mind, but I'm stuck on the fiber card frankly. The small bit of literature I can find from Dell on it calls it a 10GbE LAN card but doesn't have any part numbers. I'm half tempted to get one, stick it in there and see what it's detected as.
 

markarr

Active Member
Oct 31, 2013
421
122
43
Do you know if you can pass these through in ESXi? It's just a special form PCIe slot isn't it?
Yes you can. I have passed that device through to a vm on esxi it shows up in the inventory as an lsi 2008 card.
 

mrkrad

Well-Known Member
Oct 13, 2012
1,244
52
48
However, this is where my knowledge gets a bit fuzzy. I've never worked with fiber before and finding the mezzanine card has not been easy. I have found this: Dell 10GB Fiber Channel Dual Port Host Bus Adapter Jpynn | eBay and others of the same part number fairly readily, but they are listed as HBAs. Does this mean that they are for fiber channel storage only, or can they be used for networking? I would imagine that I'd be able to find something that they can connect to on the other end readily as well or am I wrong there?
Fiber channel does not run at 10GBPS! only 2,4,8,16GBPS so this is either an CNA or FCoE specific adapter.

I'd stick with 10gbps ethernet to kiss.
 

smithse79

Active Member
Sep 17, 2014
205
39
28
44
Fiber channel does not run at 10GBPS! only 2,4,8,16GBPS so this is either an CNA or FCoE specific adapter.

I'd stick with 10gbps ethernet to kiss.
Another reason that makes me think the Ebay sellers have it mis-labeled as an HBA when it's actually 10GbE Fiber
 

TuxDude

Well-Known Member
Sep 17, 2011
616
338
63
Fiber channel does not run at 10GBPS! only 2,4,8,16GBPS so this is either an CNA or FCoE specific adapter.

I'd stick with 10gbps ethernet to kiss.
It's far from common, but there is 10G FC out there. I've contemplated getting a few SFP's at that speed for when my new 16G Brocade switches arrive at work - that would then be the fastest link I could run for the connection over to the 10G Nexus 5000's to link to our FCoE environment. Though its probably not worth the cost to replace all the 8G SFPs that we already have working to existing Brocade 300 switches.
 

Aliv Cherevko

New Member
Jul 3, 2015
15
0
1
You can get this.

85M9R Dell PowerEdge C2100 C1100 MEZZANINE SAS RAID Controller | eBay

I have it in my cs24-ty. It would free up your pcie slot. It is a LSI SAS2008 card that you can do raid or just JBOD. The price is a little high but invaluable in freeing up that pcie slot.
Am I correct understanding, that 85M9R is working in DELL C1100, which is CS24-TY - i.e. Quanta built server from Dell Cloud Services line? I have one such CS24-TY (i.e. DELL DCS) and I need a RAID or HBA card to hook SAS drives in it. I will greatly appreciate your reply. Thank you.
 

RyC

Active Member
Oct 17, 2013
359
88
28
Am I correct understanding, that 85M9R is working in DELL C1100, which is CS24-TY - i.e. Quanta built server from Dell Cloud Services line? I have one such CS24-TY (i.e. DELL DCS) and I need a RAID or HBA card to hook SAS drives in it. I will greatly appreciate your reply. Thank you.
You should check if the mezzanine slot is physically there (I've never seen it missing) but yes, I have the 85M9R in a CS24-TY/C1100 and it works just like an LSI 2008 card.
 

Aliv Cherevko

New Member
Jul 3, 2015
15
0
1
You should check if the mezzanine slot is physically there (I've never seen it missing) but yes, I have the 85M9R in a CS24-TY/C1100 and it works just like an LSI 2008 card.
Well, it seems like I have mezzanine card, I think that is a slop I marked with a red rectangle on the first phoho. Also adding some more photos to clearly identify my CS24-TY system.
 

Attachments

Aliv Cherevko

New Member
Jul 3, 2015
15
0
1
Okay, and if we assume that the card will fit into mezz port - what exactly the cables I need to plug-in 4 SAS hard drives via an existing backplane of CS24-TY?

And finally, if taking into account that I have no plans to use something in mezz and PCI-E ports simultaneously, may be there is some cheaper and better from terms of performance RAID controllers available on the market?
 
Last edited:

Aliv Cherevko

New Member
Jul 3, 2015
15
0
1

Thank you for your prompt reply. Could you please give me another suggestion on what else can I use in PCI-E slot of my CS24-TY as RAID or HBA card (to then use ZFS) with a performance in mind? I.e. is there any better card within the same budget as 85M9R mentioned above.

Does this one make sense?
NEW Dell / LSI MegaRAID SAS 9265-8i PCI-E 6Gb/s RAID Controller w/Both Brackets

Thank you!
 

markarr

Active Member
Oct 31, 2013
421
122
43
If your not going to use SSD then there will be no difference between the two cards.