The first C6100 arrived yesterday night with one Infiniband QDR mez card. The unit was well packaged and apart from a few very light scuffs (probably racking and de-racking related) looked to be in perfect condition. All the parts appeared to be as expected, all the cables were neat and tidy and even all the black shield pieces were there.
First impressions.
1. The top removable panel it pretty thin and flimsy. A little surprised at this as the Supermicto chasses and even the Norco units seem to be quite a bit stronger. On the other hand, how strong does it need to be
.
Top tip: Top middle where the removable 'lid' meets the top front of the chassis is a screw. This needs to be removed before the top 'lid' section can be slid off. Took me a while to spot it ...
2. The drive trays are terribly cheap. Again, compared to Norco or Supermicro they are just cheap. Easy to bend, just two 'rails' to attach the drive to with no bottom (no chance of mounting a 2.5" drive without an adapter).
3. The rails that came with it seem to be for racks with round mounting holes. Mine has square holes and the rails just do not seem to be able to fit. This is disappointing but not a big problem.
4. As other have said, this unit is noisy, very noisy. I will be doing the fan mod PigLover has detailed in his thread
here. It also does not help that one or more of my fans are pulsing.
5. The motherboard sleds are secured with a single screw for each. Again, it took a few moments to work out why the sleds were not coming out until I finally found these.
On a more positive note, all CPUs recognised, all ram looking good with the option of doubling to 48GB per node with more 4GB modules (only half the ram slots are populated).
Infiniband.
I also received DBAs Infiniband DDR switch which is in great condition. I am not sure how I will rack it yet as it is around four times the depth of my HP 1810-24G and heavy all the way (weight is distributed along its full depth). A set of ears for the rear of the switch (the front came with ears) and a mid point rail in the rack would probably do the job very well but I currently have neither. Plugging in my cable from the C6100 Infiniband Mez card to the switch seemed to go well with a green light appearing on that port but I am not yet in a position to test anything (nothing else connected to the switch and no OS on any of the C6100 nodes yet). Looking good so far though.
IPMI.
Due to issues that have been reported after setting static IP addresses for the BMC I left it on DHCP but then needed to find a way of working out which IP had been assigned. I found that Supermicros IPMIView utility will detect the BMCs when doing a scan and enable login, sensor readings and power on/off etc but not any of the graceful reboots or console. This is not much of an issue as when you have the IP address you can just
HTTPS://[ipaddress] and there is a web interface for management. The only issue now is that I can only seem to find two of the nodes. I suspect it is a switch config issue but unfortunately we had a power cut the night before and I cannot now locate the IP address to manage the Procurve (not default or any of the two addresses I have assigned in the past). I will probably need to reset the switch but that is no big deal.
Flirting with storage appliances.
I wanted to give ZFS a go so thought I would try Nexenta Community Edition. i downloaded the image (fast download) and burnt it to a DVD. I had a Seagate ES 500GB drive I recently purchased from a friend second hand so put it in the tray and booted up. It booted as expected, started the install and then after selecting the 500GB drive I hit the first problem. "FDisk cannot write partition layout to the drive". The installer then become unresponsive and for all intents and purposes hung. I pulled the drive, took it out of the sled and slotted it in to my trayless hotswap bay on my PC case. The issue was that the drive had been used in a Mac and had a 200MB EFI partition. Had to spin up Diskpart and remove it, put ti back int he sled and then reboot the node and start the installer again. This time it went through to the end, displayed the new hostname etc details but then appeared to hang again at the "press ok to reboot" screen. This is not instilling my with the greatest confidence. I will continue with it tonight.
One other thing I has seen is that the drive activity lights seem to be very weak. I can barely make out that the drive is being used. Anyone else see this ?.
RB