Time to replace my DIY Freenas

BoredSysadmin

Not affiliated with Maxell
Mar 2, 2019
850
320
63
One more question - I assume I'd need an HBA to connect front 12 drives?
So 2x LSI 9210-8i ($66 for both) or 1x LSI SAS 9201-16I - $120-130
If I go with DL380p single proc, how many PCIe slots would be usable? Could I still use the 10gig network, internal and external SAS cards?
 

BeTeP

Well-Known Member
Mar 23, 2019
573
371
63
What makes you think that these SATIT2 controllers have higher specs than the $100 ones
Just going by the part numbers and my records - 0995122-10 vs 0995122-05
The 0995122-05 I bought before had v1 CPU and 32Gb DOM. Not sure about memory size though - the seller had it removed.
 

BeTeP

Well-Known Member
Mar 23, 2019
573
371
63
I assume I'd need an HBA to connect front 12 drives?
As I mentioned before the controller has a SAS2308 onboard with 4 lanes connected to the external port and 4 lanes going to the expander and the backplane. No need for another HBA. And you can even connect another DAS to the external port.
 

BoredSysadmin

Not affiliated with Maxell
Mar 2, 2019
850
320
63
As I mentioned before the controller has a SAS2308 onboard with 4 lanes connected to the external port and 4 lanes going to the expander and the backplane. No need for another HBA. And you can even connect another DAS to the external port.
I meant with DL380p. I have doubts that BSD-based Truenas Core will be able to install all necessary drivers on StorSimple controller(s).
 

Terry Wallace

PsyOps SysOp
Aug 13, 2018
186
115
43
Central Time Zone
ok, so this would do?
To expand on the previous post.
The 380e series (efficient series) takes the E5-24XX line of cpus. Has less ram slots (16) lower power usage.
The 380p series (performance series) takes the E5-26xx line of cpus. Has more Ram slots (24) higher power usage.

As you indicated that the power usage and heat was a factor I steered you towards the 380e series. I use both at work the 380e for filers and ceph clusters the 380ps for VM and compute servers in a hyper-converged cluster.

BeTeP replaces the drive cages and if you good with hardware its not that hard to do. However I prefer buying the mostly assembled as Its guarantee to arrive working (assuming you buy from a good ebay seller) Where as if you buy parts and put in the CPU, ram etc and it doesn't boot.. your left holding the bag guessing which parts are bad and the 30day clock running on returns.

the 380p's will be louder than the 380e's but have almost double the ram capacity (I usually run 128gig in my 380e's and its cheap)
Anyway thats my thoughts. I think you'll be happy with either one and sounds like BeTeP and I can both help you out with any questions getting it up and running.

Terry

p.s. I run TrueNas on most all of filers without any issues. all of the 380 series has 3 pci slots minimum with 1 cpu and 6 pci slots with both cpu's
that are empty to use. The onboard nics (usually a 4x1 gig or 2x10gig) don't count in that number.
I always turn off the onboard raid controller in bios. And run a lsi card straight to the drive backplane (usualy using the cables that are already inside)
Then I stick a nvme slot card in to run as cache and a second (dual m2. sata drive card) in the other slot (run mirrored 256g samsung sata m2's as the boot device)

Low Power - 64gig 4 1 gig nics and 3 open pci slots.

p.p.s Best price I have found for drive trays (if your not buying in bulk 100 packs)

or if you want all 12 trays.
 
Last edited:
  • Like
Reactions: BoredSysadmin

BoredSysadmin

Not affiliated with Maxell
Mar 2, 2019
850
320
63
To expand on the previous post.
The 380e series (efficient series) takes the E5-24XX line of cpus. Has less ram slots (16) lower power usage.
The 380p series (performance series) takes the E5-26xx line of cpus. Has more Ram slots (24) higher power usage.

As you indicated that the power usage and heat was a factor I steered you towards the 380e series. I use both at work the 380e for filers and ceph clusters the 380ps for VM and compute servers in a hyper-converged cluster.

BeTeP replaces the drive cages and if you good with hardware its not that hard to do. However I prefer buying the mostly assembled as Its guarantee to arrive working (assuming you buy from a good ebay seller) Where as if you buy parts and put in the CPU, ram etc and it doesn't boot.. your left holding the bag guessing which parts are bad and the 30day clock running on returns.

the 380p's will be louder than the 380e's but have almost double the ram capacity (I usually run 128gig in my 380e's and its cheap)
Anyway thats my thoughts. I think you'll be happy with either one and sounds like BeTeP and I can both help you out with any questions getting it up and running.

Terry

p.s. I run TrueNas on most all of filers without any issues. all of the 380 series has 3 pci slots minimum with 1 cpu and 6 pci slots with both cpu's
that are empty to use. The onboard nics (usually a 4x1 gig or 2x10gig) don't count in that number.
I always turn off the onboard raid controller in bios. And run a lsi card straight to the drive backplane (usualy using the cables that are already inside)
Then I stick a nvme slot card in to run as cache and a second (dual m2. sata drive card) in the other slot (run mirrored 256g samsung sata m2's as the boot device)
You're correct. Power efficiency is a concern, thus probably DL380e is the right choice. I won't mind replacing the disk cages as it is a cheap option.
128GB RAM is way more than plenty for my Freenas, in fact, even the current 32GB is somewhat of an overkill.

Sorry if it's a stupid question, which LSI controller would you use to run the 12 disks in front? Do I need a dual 8i or a single 16i?

The onboard nic of the DL380e series, I assume something like this could give me 10gig sfp+, not using the 3 available PCIe slots?
 

Terry Wallace

PsyOps SysOp
Aug 13, 2018
186
115
43
Central Time Zone
You're correct. Power efficiency is a concern, thus probably DL380e is the right choice. I won't mind replacing the disk cages as it is a cheap option.
128GB RAM is way more than plenty for my Freenas, in fact, even the current 32GB is somewhat of an overkill.

Sorry if it's a stupid question, which LSI controller would you use to run the 12 disks in front? Do I need a dual 8i or a single 16i?

The onboard nic of the DL380e series, I assume something like this could give me 10gig sfp+, not using the 3 available PCIe slots?
Yes and if you ask the people at sonic they will usually swap the 1x4 for the 2x10 if they have it in stock (well they did for me but I buy alot from them :)

the 12 drive cages has a 12 port backplane that has 2 8087 connectors. you run 2 8087 cables to your 8i card and you should be good. (I generally buy the 25drive sff version as we stuff them with 1tb ssd's) But work pays for that :)

also added a few links to my post #25
 
Last edited:

BeTeP

Well-Known Member
Mar 23, 2019
573
371
63
Oh. The DL380p comes with PMC based P420i RAID controller which (after firmware update) supports the pass-through mode. You can use that or just get an HP H220. Up to you.
 
Last edited:
  • Like
Reactions: Terry Wallace

Terry Wallace

PsyOps SysOp
Aug 13, 2018
186
115
43
Central Time Zone
Oh. The DL380p comes with PMC based P410i RAID controller which (after firmware update) supports the pass-through mode. You can use that or just get an HP H220. Up to you.
Correct.. Have you had good luck with that passthrough mode.. I have had FreeNas (a couple of years back) drop drives out on the onboard passthrough. The H220 IT mode has been my goto card for the 380 servers.. rock solid.
 

BeTeP

Well-Known Member
Mar 23, 2019
573
371
63
To expand on the previous post.
The 380e series (efficient series) takes the E5-24XX line of cpus. Has less ram slots (16) lower power usage.
The 380p series (performance series) takes the E5-26xx line of cpus. Has more Ram slots (24) higher power usage.
380e is not inherently more efficient than 380p. Sure in a maxed out config the 380p would draw more power. But the difference between power draw of similarly configured -p and -e systems is negligent. When buying new it did not make much sense to overpay (more than twice IIRC) for the -p if you were not going to run in a performance oriented config. But when buying used - the price difference between the two is like $20. It's well worth it to pay that little extra to get the -p even you are going to run it with a single low power CPU.

The one obvious benefit is the number of PCIe lanes - 40 vs 24.
 

BoredSysadmin

Not affiliated with Maxell
Mar 2, 2019
850
320
63
I've found a bit more details on DL380e vs p and it seems like "e" doesn't stand for Efficiency, but for Entry
Especially useful is the attached DOC file.
The doc mentioned that 380e comes standard with 4x1gig LOM, while 2x10gig LOM is only shown as an option for DL380p?
I am also looking forward to saving money and not purchase an 8i sas controller, but instead to reflash "free" onboard P420i.
@BeTeP above mentioned E5-2628Lv2 is going to provide me plenty of power and decent efficiency. It is a 70W TPD build on 22nm.

I am starting to think more and more that DL380p is a better way to go for me.
 
Last edited:

BeTeP

Well-Known Member
Mar 23, 2019
573
371
63
Have you had good luck with that passthrough mode.. I have had FreeNas (a couple of years back) drop drives out on the onboard passthrough
I have only run it on a test system (very light use) for a few months before I retired it. It ran reasonably well with the latest v8.32 firmware.

There was a typo in my original post - the RAID controller in Gen8 is P420i (not P410i).
 

Terry Wallace

PsyOps SysOp
Aug 13, 2018
186
115
43
Central Time Zone
I've found a bit more details on DL380e vs p and it seems like "e" doesn't stand for Efficiency, but for Entry
Especially useful is the attached DOC file.
The doc mentioned that 380e comes standard with 4x1gig LOM, while 2x10gig LOM is only shown as an option for DL380p?
I am also looking forward to saving money and not purchase an 8i sas controller, but instead to reflash "free" onboard P410i.
@BeTeP above mentioned E5-2628Lv2 is going to provide me plenty of power and decent efficiency. It is a 70W TPD build on 22nm.

I am starting to think more and more that DL380p is a better way to go for me.
Your right I was going off memory.. it was labeled entry not efficiency.. :)
and the 380e the 1x4LOM is part of the motherboard and not swappable. the 380p is where its an add on module that you can swap for the 2x10 or 2x40.
I use the mellanox connect x3's for my cluster networking and the 1 gig for just acessing the UI on proxmox.
This is one of my clusters.. all running on hp 380s' ...used... :)

1614272676716.png
 

BoredSysadmin

Not affiliated with Maxell
Mar 2, 2019
850
320
63
Your right I was going off memory.. it was labeled entry not efficiency.. :)
and the 380e the 1x4LOM is part of the motherboard and not swappable. the 380p is where its an add on module that you can swap for the 2x10 or 2x40.
I use the mellanox connect x3's for my cluster networking and the 1 gig for just acessing the UI on proxmox.
This is one of my clusters.. all running on hp 380s' ...used... :)

View attachment 17664
My home lab cluster is nowhere near as impressive, but my work clusters are close to what you have - I have x7 such nodes in one cluster and the 20 cores Gold R CPUs in the other 7 node cluster. all 12 SSDs are 12G sas 3.84TB.

1614274951468.png
 
Last edited:
  • Like
Reactions: Terry Wallace

BoredSysadmin

Not affiliated with Maxell
Mar 2, 2019
850
320
63
I searched, but I can't seem to find the answer - does E5-2628L V2 requires using the Low-Voltage memory DIMMs or not?
Doesn't seems so, but figured I'd ask to doublecheck
 

BoredSysadmin

Not affiliated with Maxell
Mar 2, 2019
850
320
63
I also heard back from Sonic and they estimate it at $235 with items on my wish list - Which seems like a very good deal to me.
I already have some ddr3 RDIMMs and ILO keys for ILO4 advanced seem to be available online.
Wish list:

DL380p G8
12 LFF with 12 Disk trays
1x E5-2628LV2 cpu with heatsink
HP FlexFabric with 2x10gig ports
2x PSUs and all stock fans included.
No memory
No HD
No rail kit
 
Last edited:

BeTeP

Well-Known Member
Mar 23, 2019
573
371
63
Personally I always get both heatsinks for all my dual socket systems even if I initially plan to use them with just a single CPU. Plans have tendency to change. Also ask them about H220 - if the price is right I would recommend getting one as well.
 

BoredSysadmin

Not affiliated with Maxell
Mar 2, 2019
850
320
63
Personally I always get both heatsinks for all my dual socket systems even if I initially plan to use them with just a single CPU. Plans have tendency to change. Also ask them about H220 - if the price is right I would recommend getting one as well.
Is $48 is the right price for H220 to add this build? I assume this is in case P420i doesn't quite work right in Freenas?