Help with motherboard and CPU for new FreeNAS build

Rand__

Well-Known Member
Mar 6, 2014
4,947
1,017
113
1610549520391.png
(Note thats the board version with SFP+ ports)

Blue = onboard SATA controller
Red = onboard SAS controller
Black = onboard NVME controller (2 Oculink ports)
 
  • Like
Reactions: zack$ and itronin

itronin

Well-Known Member
Nov 24, 2018
537
311
63
Denver, Colorado
so based on what I'm seeing :

And assuming you have a Rosewill 4xxx series with 3 x 4 Hot swap cages...

Storage options:

2 SATA ports with SATADOM support (2 drives total)
2 SFF-8087 so you'll need 1 or 2 x SFF-8087 to SATA forward breakout cables (8 drives total)
2 SFF-8643 so you'll need 1 or
2 x SFF-8643 to SATA forward breakout cables (8 drives total)
2 Oculink ports means you can have 2 NVME drives plus the M.2 80.

Natively 18 SATA / SAS mix of drives
Plus 3 NVME ports
Plus 10Gbe SFP..

I really like this board. Very JOAT.

FWIW @rskoss you might want to go ahead and put in a post with all the parts you have to date and your full intentions/use case. You might get a better set of hints/options/suggestions - then again could be information overload. There could be some synergy in selections and would avoid the oh you want an LSI 92xx HBA when you don't need one! :)

If we keep asking questions piecemeal you could end up with a bad decision in the mix! Just saying...
 
  • Like
Reactions: Markess

rskoss

Member
Oct 9, 2020
115
17
18

zack$

Active Member
Aug 16, 2018
523
198
43
Yes.

In addition to connecting your HBA directly to the sata drives you could also attach them to a backplane, expander or hdd bay. Just depends what you have in mind.
 

rskoss

Member
Oct 9, 2020
115
17
18
View attachment 17121
(Note thats the board version with SFP+ ports)

Blue = onboard SATA controller
Red = onboard SAS controller
Black = onboard NVME controller (2 Oculink ports)
It's true that a picture speaks a thousand words.

You didn't circle the orange ports - the ones labeled S-SATA0 and S_SATA-1.

What is connected right now:
M.2 has ProxMox installed on it and it boots. The server is up and running with nothing installed yet except ProxMox.
I have a P3605 in the left most PCI-E slot in the above picture. This will hold my VM's.

CAT6 cables going from the IPMI port and from a 10Gbe port to my networks switch - working fine. IPMI is so sweet.

I have SATA cables going from each of those orange colored ports in the picture to SATA drives. That takes care of two of the drives.
I have a SFF-8087 to SATA Forward Breakout cables going from each of the blue circled ports to 8 SATA drives (4 each). That brings us up to 10 drives.

And this morning I learned about the SAS port, the red circled ports above. I ordered 2 SFF-8643 to 4xSATA cables this morning.

First and foremost, I need TrueNAS up and running. Primary use case. I'll have more VM's, but that's all secondary to TrueNAS.

I owe dinner/drinks to all y'all.
 
  • Like
Reactions: zack$

Markess

Well-Known Member
May 19, 2018
574
261
63
Your MB actually supports a minimum of 18 sata drives without the need for an add-on card. Are you using your onboard 3008 HBA for anything?
Okay, I feel kinda silly for missing that :oops:. Well, @rskoss at least now if you need an HBA card, you'll know how to spot a fake!
 

rskoss

Member
Oct 9, 2020
115
17
18
Well this isn't good.

I described how I have the drives connected at the moment. I was exploring the ProxMox interface and I didn't see the hard disks. The M2 and the P3605 is listed under Disks, but not a single hard drive.

When I inserted the drives, a blue light appeared above each drive. So there is some kind of electrical connection.
 

Rand__

Well-Known Member
Mar 6, 2014
4,947
1,017
113
Wrong cable or sata ports not active in bios?
Although forward breakout should be correct...
 

Markess

Well-Known Member
May 19, 2018
574
261
63
Well this isn't good.

I described how I have the drives connected at the moment. I was exploring the ProxMox interface and I didn't see the hard disks. The M2 and the P3605 is listed under Disks, but not a single hard drive.

When I inserted the drives, a blue light appeared above each drive. So there is some kind of electrical connection.
May be a dumb question, and probably only applies to the onboard LSI 3008, but what mode are the controllers in (i.e. IR vs IT)? I know that for some controllers in IR (RAID) mode, a drive not assigned to an array won't appear to the OS. I honestly can't say for the 3008 though. I have only one (also integrated on a motherboard) and its always been in IT mode.

If you've got drives connected to the 3008 already, can you access the controller during boot (Ctrl+C I think) and see if it shows the drives connected? For the SATA drives, are they visible in the storage configuration page in the BIOS/UEFI?
 

rskoss

Member
Oct 9, 2020
115
17
18
There are two menu items in the BIOS: PCH SATA Configuration and PCH sSATA Configuration. All ports under both menu items say "Not Installed".

I guess the only thing to do at this point is to turn it off and unplug and reseat each plug. The 8643 should arrive today.
 

Rand__

Well-Known Member
Mar 6, 2014
4,947
1,017
113
Hm maybe the wrong cable after all?
Either that or power issues

You will see when you plug a drive into the 8643 ports (assuming you got the right cables there).
Else you could swap one of the drives with an onboard sata port (not OS) to see if that one is visible in bios/proxmox
 

zack$

Active Member
Aug 16, 2018
523
198
43
In addition to what @Rand__ said, my 2 cents:

Confirming that you have 8087 to sata break out cables.

Ensure that the 8087 cables are inserted the right way into the MB ports. Some 8087 cables have a locking mechanism. If yours does, ensure that they will lock when placed into the MB ports.
 

rskoss

Member
Oct 9, 2020
115
17
18
In addition to what @Rand__ said, my 2 cents:

Confirming that you have 8087 to sata break out cables.

Ensure that the 8087 cables are inserted the right way into the MB ports. Some 8087 cables have a locking mechanism. If yours does, ensure that they will lock when placed into the MB ports.
These are the cables: https://www.amazon.com/gp/product/B018YHS8BS/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1

The cable appears to be able to be inserted in either direction on the motherboard. The only locking mechanism if you want to call it that is the clip that clips to the connector housing.

See next post.
 

rskoss

Member
Oct 9, 2020
115
17
18
Progress, but this is flaky. First, I only have 4 drives at the moment. 10 more are due here on Friday. Honestly, I didn't think I'd get this far to justify spending $thousands on drives.

I removed all HD connections. I used one of the SFF-8087 to SATA breakout cables to connect to the 4 drives. 2 of them appeared in the Proxmox interface. I reseated the cables on the two drives that didn't appear and that didn't do anything. I then removed the 2 drives and reinserted them. One more drive appeared. I removed and reinserted the drive that wasn't showing up and it showed up.

So now I'm seeing all 4 drives, but this seems to be held together with spit and bandaids.
 

itronin

Well-Known Member
Nov 24, 2018
537
311
63
Denver, Colorado
These are the cables: https://www.amazon.com/gp/product/B018YHS8BS/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1

The cable appears to be able to be inserted in either direction on the motherboard. The only locking mechanism if you want to call it that is the clip that clips to the connector housing.

See next post.
I've had good luck with the cable matters and cable creations "low cost" SFF-8087 to SFF-8088 cables. It may seem like the SFF-8087 can be inserted either way but there is only one way that is correct. The end of the cable should "latch" and you should not easily be able to pull it out without pressing on the metal lever/tab. There should be either two holes or two indentations on the SFF-8087 receiver (motherboard side) that correspond to the flanges I have highlighted in the picture.

Screen Shot 2021-01-14 at 7.54.30 AM.png