Home Server Build
- Thread starter Visseroth
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
I will try something like this in my next supermicro chassis with an expander backplane. Two ports to raid controller and one port with breakout cable to the rear bays (sas ssds for cachecade
)I get the concept. I think I'm understanding.
EL1 can allow for multiple connections to one HBA (whatever that stands for)
1 cable support 4 lanes which is why you can connect 4 drives in a break out cable. Each lane support 6Gb. So 1 cable with 4 lanes gives you a theoretical 24Gb (SAS2). 2 connected cables at 4 lanes each can give you a theoretical 2 x 4 = 8 lanes x 6Gb = 48Gbps theoretical.
EL2 allows for cross connect between two different servers as long as you are running SAS drives that support that kind of topology. This would be typically used for hi-redundancy. One failed or offline machine wouldn't take the pool offline while still acting like a EL1 backplane but EL1 time 2 on the same board. Basically EL1 for each machine. Each machine talking to shared drives.
Now according to some of my reading last night (happen to find another thread on this forum about the x10 and SAS). SAS 3 Controllers don't play well with SAS2 backplanes. Likely a driver related issue. So I'm educating that in best practice it is good to match your HBA to your backplane. There by eliminating the possibility of controller/driver compatibility issues.
In this supposed driver related issue, wouldn't this also be a issue if you connected your SAS3 HBA to a SAS3 backplane which is then connected to a SAS2 backplane? If so the best practice would likely be to use your SAS3 controller for your SAS3 Backplane and connect a SAS2 controller to your SAS2 Backplane or would that create some kind of a driver conflict in the FreeBSD environment?
BTW, thanks a TON guys! For all the information and not getting ticked when I don't seemingly understand the answer.
There are forums I've been on where I'll just get ignored because I should apparently know the answer. Likely because the thread has been posted about before though I didn't find it. I typically will search for answers before I ask questions because usually someone else has already asked the question and received a answer.
EL1 can allow for multiple connections to one HBA (whatever that stands for)
1 cable support 4 lanes which is why you can connect 4 drives in a break out cable. Each lane support 6Gb. So 1 cable with 4 lanes gives you a theoretical 24Gb (SAS2). 2 connected cables at 4 lanes each can give you a theoretical 2 x 4 = 8 lanes x 6Gb = 48Gbps theoretical.
EL2 allows for cross connect between two different servers as long as you are running SAS drives that support that kind of topology. This would be typically used for hi-redundancy. One failed or offline machine wouldn't take the pool offline while still acting like a EL1 backplane but EL1 time 2 on the same board. Basically EL1 for each machine. Each machine talking to shared drives.
Now according to some of my reading last night (happen to find another thread on this forum about the x10 and SAS). SAS 3 Controllers don't play well with SAS2 backplanes. Likely a driver related issue. So I'm educating that in best practice it is good to match your HBA to your backplane. There by eliminating the possibility of controller/driver compatibility issues.
In this supposed driver related issue, wouldn't this also be a issue if you connected your SAS3 HBA to a SAS3 backplane which is then connected to a SAS2 backplane? If so the best practice would likely be to use your SAS3 controller for your SAS3 Backplane and connect a SAS2 controller to your SAS2 Backplane or would that create some kind of a driver conflict in the FreeBSD environment?
BTW, thanks a TON guys! For all the information and not getting ticked when I don't seemingly understand the answer.
There are forums I've been on where I'll just get ignored because I should apparently know the answer. Likely because the thread has been posted about before though I didn't find it. I typically will search for answers before I ask questions because usually someone else has already asked the question and received a answer.
SuperMicro HBAs to BackPlane questionsLink please ?
Posts 15 and 16
Well that's good to know. That thread I found about the driver issue was fairly old, so maybe it's been fixed?
I will have a chance to test.
Well Mr PigLover was kind enough to sell me a MB CPU combo and let me barrow some RAM to do the initial tests, set up the board, ect and I'm liking what I'm seeing....
Here's a pic of it benched with a static clean bag underneath it to help keep it safe from being pressed against the desk and what ever might slip under it....
Here's a pic of it off but power applied. You'll see by the Watt Meter it draws 4W with power applied. Granted some of that could be power supply too but I'm betting it's mostly the IPMI...
Here are pictures of startup. You'll see the power consumption jumps around between 50 and 100W...
Here you see I've temporarily loaded Windows onto a SSD for testing purposes. All the updates done so the updates don't try and run in the background while doing tests.
Here it is idling at a nice cool ~30C in a 23C room...
And consuming about 57W...
At full load, all CPUs busy (but not the RAM) you can see about 150W consumption and a peak temp of 53C in a 23C room...
Two hours later I came back after watching a movie (Thor Ragnarök) and the peak temp on any given core is 55C, average peak temp... 53.875C.
The active SuperMicro CPU heatsink was a singing!!! After killing the stress test for the evening the CPU fan settled down. It's nice that the fans are active but sing when they need to, so I guess once in use I shouldn't give them much reason too unless called upon but I'll at least know the heatsink can handle the heat and the CPU can handle the job.
Anyhow, given that these are likely the core temps, not the thermal junction temp and the tCase max is 72.1C, it's safe to say this heatsink suites this Xeon E5-2630V3 CPU.
My plan now, thanks to the VERY kind advice and people on this forum is to get a 64GB RDIMM kit (any recommendations appreciated), the BPN-SAS3-826EL1 backplane for the back of the server, the BPN-SAS2-846EL1 for the front of the server. Move my most active drives to the back and the backup/spare drives to the front and breath some new life into my FreeNAS server, hopefully with a idle power consumption of about 250W (Will check when implemented).
With 8 cores and 16 possible threads and while still being upgradeable, this should be a very fine server for a while.
Again, thank you gents, and as always, suggestions, comments, ect always welcome from this kind open minded crew!
Here's a pic of it benched with a static clean bag underneath it to help keep it safe from being pressed against the desk and what ever might slip under it....
Here's a pic of it off but power applied. You'll see by the Watt Meter it draws 4W with power applied. Granted some of that could be power supply too but I'm betting it's mostly the IPMI...
Here are pictures of startup. You'll see the power consumption jumps around between 50 and 100W...
Here you see I've temporarily loaded Windows onto a SSD for testing purposes. All the updates done so the updates don't try and run in the background while doing tests.
Here it is idling at a nice cool ~30C in a 23C room...
And consuming about 57W...
At full load, all CPUs busy (but not the RAM) you can see about 150W consumption and a peak temp of 53C in a 23C room...
Two hours later I came back after watching a movie (Thor Ragnarök) and the peak temp on any given core is 55C, average peak temp... 53.875C.
The active SuperMicro CPU heatsink was a singing!!! After killing the stress test for the evening the CPU fan settled down. It's nice that the fans are active but sing when they need to, so I guess once in use I shouldn't give them much reason too unless called upon but I'll at least know the heatsink can handle the heat and the CPU can handle the job.
Anyhow, given that these are likely the core temps, not the thermal junction temp and the tCase max is 72.1C, it's safe to say this heatsink suites this Xeon E5-2630V3 CPU.
My plan now, thanks to the VERY kind advice and people on this forum is to get a 64GB RDIMM kit (any recommendations appreciated), the BPN-SAS3-826EL1 backplane for the back of the server, the BPN-SAS2-846EL1 for the front of the server. Move my most active drives to the back and the backup/spare drives to the front and breath some new life into my FreeNAS server, hopefully with a idle power consumption of about 250W (Will check when implemented).
With 8 cores and 16 possible threads and while still being upgradeable, this should be a very fine server for a while.
Again, thank you gents, and as always, suggestions, comments, ect always welcome from this kind open minded crew!
Doh! I do recall briefly thinking about that for a moment. That 6Gb x4 lanes or even 8 lanes is more than enough to saturate a 10Gb NIC and no I don't currently plan on using SSDs until 2TB and 4TB come down in price. WAY DOWN.
So yep, good call! I'll save the $$ until I move to a 2.5" drive bay server later when the price of SSDs make it worth the move!
Thanks for pointing that out!
So yep, good call! I'll save the $$ until I move to a 2.5" drive bay server later when the price of SSDs make it worth the move!
Thanks for pointing that out!
Well rebuilt the server yesterday. Power consumption is down by about 100W to around 260W or so and the inside of the server is much cleaner, cooler and quieter, idling at about 30C.
So now I'm running the X10SRH with 2 SAS2EL1 expanders.
Here's the before (After I removed the 5 LSI 9211-8i cards)
Here's the final install and running!
AGAIN! Thank you EVERYONE for all the help!!!!! I'm digging my new server!
So now I'm running the X10SRH with 2 SAS2EL1 expanders.
Here's the before (After I removed the 5 LSI 9211-8i cards)
Here's the final install and running!
AGAIN! Thank you EVERYONE for all the help!!!!! I'm digging my new server!