Cisco C240-M3 drives AND tray/caddies - Help my storage

[Harry P. Nyce]

Need at least a half dozen 2.5" drive caddies for the C240, as well as trying to find a deal on 128GB RAM. Currently running: 8GB DDR3L 1600MHz PC3L-12800R RDIMM (Registered) Samsung M393B1K70DH0-YK0 or 16GB version would be rad.

Also am beginning the process learning and trying to figure out and expand my storage needs. Currently testing ESXi with three 600GB 10k rpm Seagate SAS drives in RAID 0, but want at least a pair of SSDs (Samsung 1625, Intel 3510, Samsung PM863) plus more ultra cheap spinning rust.

I know so little about storage options, that was my first time installing ESXi. Not sure if I need, or want virtualized FreeNAS... But an all in one homelab solution sound ideal to get started. Workload: Plex, Nextcloud with only half dozen users to start, Redmine, pfSense, Win2016, Ubuntu Server w/ Docker.

Thank you for this great community, and for anyone who can offer tips, criticisms, advice, or even pro-tips. I'm concerned parts won't work with my aging workhorse, should I buy new to ensure compatibility?

 

[frogtech]

800-35052-01 | eBay

If no one here has any trays.

 

[Harry P. Nyce]

Thank you kindly. Apparently i'm going to have to spend 15$ per for drive caddies. I did realize there'd be some disadvantages to owning a Cisco machine. Appreciate the response.

800-35052-01 | eBay

If no one here has any trays.

 

[Harry P. Nyce]

800-35052-01 Bump. Please help me track down some drive caddies! Appears it'll cost more for drives than it will the trays.

 

[frogtech]

Caddies aren't really super cheap when you buy them individually in bulk anyway, even for companies like dell or hp, since they're meant to be sold with the drives. The only cheap ones I know of are really supermicro and the older style HP ones for 2.5".

 

[Churchill]

I'd make an offer on that guy on ebay for 8 trays (buy extras) and see what he can do.

For 2.5" hard drives any 10k SAS hard drive will work without a hitch, search the forums here and you'll see that there are tons of people selling 2.5" drives on here. I bought 30 600GB hard drives on here just by browsing and hitting up old threads to see if these guys still got drives available.

 

[Harry P. Nyce]

Awesome, thanks for the pro-tips, guys. Truly appreciate it. Obviously i've got a lot to learn. I'd be happy to purchase them with drives, but guess that'd mean paying brand new enterprise prices, eh? Guess i should just bite the bullet so i can move on with these projects. Kind of floundering with the storage options, unsure of how i should even proceed. Still trying to get at least 2 SSDs, as well. A friend picked up a few 850 EVOs for his R720xd, but i wanted to go the used enterprise route, i'm trying to learn and play with all those features i've missed out on in the consumer realm.

I was worried about drive compatibility, have absolutely no idea if the Cisco would be fussy in any way. I believe he lost some monitoring by going with consumer ssds, but otherwise i think it's working out. 4 x 500GB in RAID10 (i think?).

 

[whitey]

@Harry P. Nyce why don't you shoot for a pair of Intel s3700 400-800gb or hitachi/hgst husmm/hussl drives in same capacity to dip your toes, that's my recommendation anyways, pickup a couple $10-15 cisco trays and GO!

 

[Harry P. Nyce]

Thank you for the advice, i've obviously been floundering a bit. Trays and hardware compatibility (both VMware & Cisco UCS) have been a bit daunting, as well as trying to figure out my storage options. I just want to get some redundant fast storage so i can get started. Exactly what you are describing sounds perfect. Was looking at some 960GB drives, but even a pair of 400 would surely suffice. This is my setup:

24 2.5 inch (63.5 mm) SAS or SATA hard drives (HDDs) or solid state drives (SSDs) with the 24-drive backplane server configuration​

And these are listed in the specsheet as compatible drives (didn't list the 12Gbps, as i'll assume they're pricey):

6 Gbps Drives

1. UCS-SD800G0KS2-EP 800 GB 2.5 inch Enterprise Performance SAS SSD (Samsung 1625) SAS 800 GB
2. UCS-SD400G0KS2-EP 400 GB 2.5 inch Enterprise Performance SAS SSD (Samsung 1625) SAS 400 GB
3. UCS-SD200G0KS2-EP 200 GB 2.5 inch Enterprise Performance SAS SSD (Samsung 1625) SAS 200 GB
4. UCS-SD480GBKS4-EV 480 GB 2.5 inch Enterprise Value 6G SATA SSD (Intel 3510) SATA 480 GB
5. UCS-SD120GBKS4-EV 120 GB 2.5 inch Enterprise Value 6G SATA SSD (Intel 3510) SATA 120 GB
6. UCS-SD960GBKS4-EV 960 GB 2.5 inch Enterprise Value 6G SATA SSD (Samsung PM863) SATA 960 GB
7. UCS-SD400G0KA2-G 400 GB SATA 2.5 inch Enterprise Value SSD SATA 400 GB
8. UCS-SD100G0KA2-G 100 GB SATA 2.5 inch Enterprise Value SSD SATA 100 GB

EDIT: Have the LSI 9266-8i RAID controller, which is presently configured with 3 x 600GB Seagate 10K.5 SAS disks in a basic RAID 0 for initial testing purposes. Learning as much as i can before destroying the array and hopefully redoing it. I'm wondering if i should be taking advantage of the FlexFlash internal dual SD card reader, not sure what i'd put on 16/32GB SD cards, i'm booting ESXi 6.5 from a 32GB Sandisk USB device which has worked out well.

(Also, fixed formatting.)

 

[whitey]

You certainly don't have to use what is listed on the vendors compat list as far as drives, for reference I've been scooping up sas3 husmm's over the last 6 months or so for $250 for 800GB model, $140 for 400GB model, and $70 for 200GB models and using them in a variety of ways on ZFS AIO builds...things freaking chew up I/O!

Just sayin', keep an eye out and stalk ebay/make offers/be patient.

 

[mstrzyze]

This is not compatibility list

It's what you can order from cisco when you buy server / spare
Anything will work on sas / sas2 / sas3
Ive used even SanDisk 120g sata and 4tb sata Seagate..

 

[mstrzyze]

And regarding sd/usb boot
With sd you can have raid 1 for boot (it's m3 use 8 or 16 GB sd)
With usb you have one booting device..

 

[Harry P. Nyce]

I guess i don't have any use for RAID1 ESXi boot device, was just curious if a 16GB sd card RAID1 volume would be sufficient for running a FreeNAS VM or something. I have tons to learn, but even beginning to comprehend reasonable storage options for single node homelab all-in-one has been a chore for me.

Appreciate you guys schooling me. It's so valuable to have outside input to help redirect me down the proper path when i'm going astray. Thank you very much.

 

[mstrzyze]

What nic you have? ( In riser)

Nice is that you can have many vnic's ( defined in cimc) and os see that as independent

 

[Harry P. Nyce]

I don't have any fancy VIC, it was either removed before this machine was sold, or i'm not sure what could have happened with it.

It came with the LSI 9266-8i, as well as nVidia Quadro 2000 (x3) and the slot where i expect the VIC (the NIC in riser i believe you are asking about) is missing. Well, not missing, as there's rear face-plate covers over the other two potential expansion slots on the rear, above the redundant PSUs.

I have the onboard Gigabit (x4) Ethernet, had to look up the exact specs:

Embedded quad-port Intel i350 PCIe-based Gigabit Ethernet controller, supporting the following:​

■ Pre-Execution Boot (PXE boot)
■ iSCSI boot
■ Checksum and segmentation offload
■ NIC teaming

Expansion slots Five PCIe slots (on two riser cards)​

■ Riser 1 (PCIe slots 1, 2, and 3)
■ Riser 2 (PCIe slots 4 and 5)

Internal storage devices: Drives are installed into front-panel drive bays that provide hot-pluggable access.​

■ Small Form Factor (SFF) drives. The server can hold up to: • 24 2.5 inch (63.5 mm) SAS or SATA hard drives (HDDs) or solid state drives (SSDs) with the 24-drive backplane server configuration​

 

[Harry P. Nyce]

So if i wanted to purchase a pair of intel DC S3XXX drives for my upcoming home-prod project(s) where would be the best resource to go about that to find decent deals? Can these drives be had for $0.50 / GB anywhere? I'm looking for some durable SSDs that'll continue to grow with me as i learn more and attempt caching and/or storage tiers.

$300 for 240GB S3510 seems pretty steep: https://www.newegg.com/Product/Product.aspx?Item=0D9-0060-00017
$384 for 480GB S3610: https://www.newegg.com/Product/Product.aspx?Item=N82E16820167291

Please help me do this better. I understand i'll be paying a premium for datacenter drives, but i feel liek i'm missing something. I also realize my needs are meager, but am justifying these initials costs as a form of continuing education. I hope two SSDs in RAID1 will suffice as my initial VM host until i outgrow it, however long that may be.

Will eventually spin up a separate box for mass storage / media to support the 3.5" disks i have on hand, but for now i'm focused on getting my single node hyperconverged ESXi host started with modern hardware upgrades.

 

[whitey]

Ebay my friend, think I already advised thats where most of us get used ent class gear 'on the cheap'...either that or fs sub-forum here, btw you can and should do MUCH better that .50cents per gb on used ssd, hell sometimes even get new for less and no worry abt used for the most part, ent class devs have insane pbw endurance ratings and high end nand

 

[Harry P. Nyce]

Ebay my friend, think I already advised thats where most of us get used ent class gear 'on the cheap'...either that or fs sub-forum here, btw you can and should do MUCH better that .50cents per gb on used ssd, hell sometimes even get new for less and no worry abt used for the most part, ent class devs have insane pbw endurance ratings and high end nand

Thanks for reinforcing the Fleabay thing. I'm always hesitant to proceed on there, as i've had terrible experiences in the past. I was shopping around for used intel data center SSDs and found an old S3500 that said it had 319TB (yes Tera, not Giga) of data written to it.

My actual use case and needs are meager by comparison to any enterprise environment, but i'm looking to make an investment that i'll be able to use and learn from over the coming year(s) and don't want to have to worry terribly about reliability in half a year. Lacking redundancy, or fail-over systems in a homelab environment i am attempting to hedge my bets by cobbling together what i perceive to be "reliable" hardware.

I've glanced at NewEgg and Amazon both for new drives, where else should i be looking for these new SSDs at 50 cents per GB? Thanks you, thanks you, thanks you to everyone who has (and continues to) contribute!

EDIT: I'd pay $150 for these (or comparable) in a heartbeat, but am concerned with HP's firmware in my Cisco server -- 480GB Samsung PM1633 SAS 3.0‐12Gb/s 2.5" 15mm SFF‐8680 SSD MZILS 480 HCGR ‐ 000 | eBay

 

[whitey]

Please do yourself/your wallet a favor and re-consider used market if budget is of concern. The drives I recommended are extremely overzealous in their PBW catagories, S3700/S3710, HUSSL, HUSMM drives all fall in the 9PBW-35PBW (yes petabytes written) realms and there is no way you will EVER wear them out...I expect my SSD'd to outlive me quite possibly to be more blunt.

EDIT: S3500/S3510 are in nowhere near in the same realms of same class devices (performance or PBW) compared to the ones I listed. That said I DO recommend OEM Hitachi branded drives over a vendor for sure but a simple sg_format back to 512bytes almost always certainly fixes those most stubborn drives.

 

[Harry P. Nyce]

Please do yourself/your wallet a favor and re-consider used market if budget is of concern. The drives I recommended are extremely overzealous in their PBW catagories, S3700/S3710, HUSSL, HUSMM drives all fall in the 9PBW-35PBW (yes petabytes written) realms and there is no way you will EVER wear them out...I expect my SSD'd to outlive me quite possibly to be more blunt.

EDIT: S3500/S3510 are in nowhere near in the same realms of same class devices (performance or PBW) compared to the ones I listed. That said I DO recommend OEM Hitachi branded drives over a vendor for sure but a simple sg_format back to 512bytes almost always certainly fixes those most stubborn drives.

If it wasn't already obvious, i have zero experience with much of the enterprise level gear -- hence the desire to learn and grow. I've supported various Dell PE machines over the years, but that's about it. A coworker is using 850 EVOs (x4) in his R720xd, which would probably suffice for my needs, but i'm willing to spend a little extra so i don't have to wonder or worry about rebuilding my primary flash array a year from now.

So... seeing a complete sale for S3700 on eBay with 319TB of data written to it is nothing to be concerned with? From what little knowledge i've gathered thus far, the Intel drives appear to be the most durable, while Samsung is difficult to compete with when it comes to performance -- would that be a correct, general sentiment?

Provantage is having a sale on these: PROVANTAGE: Samsung MZ-7LM480NE Enterprise SSD PM863a SATA 480GB for Business (or forty dollars more for the write intensive SM863a version.) Every bit of advice you are sharing is wildly valuable to me. I've been lacking direction so began initial testing and learning with the spinning rust i have in place, but looking to add solid state disks in the very near future.

EDIT: Should i even be considering any SAS SSDs, or is that an unrealistic proposition in a homelab with a shoestring budget? 12Gbps sounded expensive, so i've mostly avoided that category.