SC113MTQ. 1U short depth with 8 2.5" hotswaps + an LSI9300 IT mode that I have on the shelf. For now. This will likely be the first of 5 CEPH OSD nodes in a small cluster. I plan to do some simple performance testing and may go down-scale to the 2C/4T model for the rest of the cluster (OSD really isn't an intensive use case).Evan said:@PigLover what case will you be using ?
Rack cases are plenty, but small desktop cases (needed as in apartment and no chance for a rack) for flex-atx seem few and far between.
Yes they are. And no it doesn't. They are not silent - or really even close to silent - but the MB holds them at about 5k RPM and they give a nice dull hum at that speed.Paul Bommel said:@PigLover
Are these Supermicros 65L4-Fans?
If the case is closed and in the rack, does it sound like a jet engine?
Last night I just received and built up an X10SDV-8C-TLNF4 with a Samsung 950 Pro M.2, and 64GB ECC RDIMM's and I think I may be having the same or at least a related issue. When starting up it takes a long time, maybe about 45-60 seconds or so to get get to the screen where I can enter the BIOS, from there on to Kubuntu 15.10 desktop takes almost another 120 seconds. Once in Kubuntu it is giving me "eno1", "eno2", "rename4", and "rename5" for my NIC's. Just FYI I did a EFI install and also have an nVidia GT640 video card installed. Please, keep us posted on your progress on this as these slow boots are a bit demoralizing. Any help you can provide would be greatly appreciated! Thank you in advance!!A couple of notes on the X10SDV-TP8F:
I loaded Ubuntu 16.04 daily last night. Came up perfectly and recognized all 8 NICs (2x i210s, 4x i350s and 2x 10Gbe). There was an annoyingly long pause by the installer at "detecting devices" - I estimated 90 seconds. Don't really understand that but I'm speculating that it was trying to figure out which NIC had a live internet connection.
Since the manual isn't published yet i had to guess on the NIC order. Appears that the leftmost two (facing from the rear) are the i210s with port 0 on the bottom (typical for SM). The next four are the i350s. The 10Gbe are sorta obvious as they are SFP+.
I did notice an oddity with the new Linux NIC renaming scheme. Firstly, it only seems to permit 4 "on MB" NICs, named "eno1", eno2", "eno3" and "en04". The other for get named "rename1", "rename2", etc. It was designed to "ensure" stable naming across reboots - but it doesn't. The names shift around across reboots, probably because of probe-response time randomness, which makes it impossible to get a clean startup across boots. For now I disabled the "new" naming and went back to MAC-based binding to eth[0-7].
I also successfully set up booting to a ZFS mirror rpool. This is still not supported directly in the installer (boo) and I had to follow the method published for 15.10. But it does appear to work. We'll see if it will still be stable after a kernel update.
Idle or under (heavy) load?Yes they are. And no it doesn't. They are not silent - or really even close to silent - but the MB holds them at about 5k RPM and they give a nice dull hum at that speed.
They never seem to spin up no matter what I do. Did a prime59 burn on it and the fans just sat at ~5k rpm the whole time. At 35W I just don't think it every generates enough heat to need more than a slow breeze across the heatsink.Idle or under (heavy) load?
You can go back to legacy device naming pretty easily:Last night I just received and built up an X10SDV-8C-TLNF4 with a Samsung 950 Pro M.2, and 64GB ECC RDIMM's and I think I may be having the same or at least a related issue. When starting up it takes a long time, maybe about 45-60 seconds or so to get get to the screen where I can enter the BIOS, from there on to Kubuntu 15.10 desktop takes almost another 120 seconds. Once in Kubuntu it is giving me "eno1", "eno2", "rename4", and "rename5" for my NIC's. Just FYI I did a EFI install and also have an nVidia GT640 video card installed. Please, keep us posted on your progress on this as these slow boots are a bit demoralizing. Any help you can provide would be greatly appreciated! Thank you in advance!!
I was just about to ask you ... Thank you and have a great weekend!BTW - I also have a trouble ticket open with SM asking them to consider fixing their BIOS so that it returns unique SMBIOS type-41 records for each NIC. This would be the correct fix.
Thank you for that! Pardon my ignorance, won't this only affect things after posting, between the Supermicro Logo and reaching the desktop. My underlying issue seems to between hitting the power button and reaching the Supermicro Logo. That is taking 45-60 seconds. My current bios is "1.0c", but I just saw that they have released "1.1". That may address the problem?You can go back to legacy device naming pretty easily:
Edit /etc/default/grub and change the boot arguments to this:
GRUB_CMDLINE_LINUX="net.ifnames=0 biosdevname=0"Then run "update-grub" and reboot. You'll be back to NICs named "eth0", "eth1". With your board they should be stable across boot because the 1Gbe NICs always seem to init faster than their 10Gbe brothers. On my board I still get an ugly race between the i210 and i350 1Gbe NICs.
You fix that by writing a udev rule /etc/udev/rules.d/70-persistent-net.rules and force the names by MAC address.
hmm, "5k rpm" is still to much.They never seem to spin up no matter what I do. Did a prime59 burn on it and the fans just sat at ~5k rpm the whole time. At 35W I just don't think it every generates enough heat to need more than a slow breeze across the heatsink.
Hoping it's soon along with the IPMI fix.Got the engineering build BIOS loaded and tested. Seems to fix the Ethernet identification issue.
Using "dmidecode" confirmed that the on-board ethernet ports are enumerated corrected (on-die I210s #1 & #2, on-die 10Gbe #3 & #4, and off-die i350s #5, #6, #7 & #8).
Removed the boot arguments and rebooted Xenial - NICs named eno1..eno8 just like you'd expect.
I've asked SM when they expect to publish the 1.0a BIOS. No answer yet.
Good to hear! While not mission critical for me, it'd nonetheless be nice to have this fix for my board as well.Got the engineering build BIOS loaded and tested. Seems to fix the Ethernet identification issue.
Using "dmidecode" confirmed that the on-board ethernet ports are enumerated corrected (on-die I210s #1 & #2, on-die 10Gbe #3 & #4, and off-die i350s #5, #6, #7 & #8).
Removed the boot arguments and rebooted Xenial - NICs named eno1..eno8 just like you'd expect.
I've asked SM when they expect to publish the 1.0a BIOS. No answer yet.