Recent content by Cixelyn

  1. Cixelyn

    Rare Epyc 7B12 CPUs PSB Unlocked. 64 CORE SOLD

    Sounds good -- will DM in a bit! Though I wasn't really planning on getting two...
  2. Cixelyn

    3090 driver handicap?

    We managed to find a deal on blower 3090s so I guess I'll have a chance to revisit our earlier benchmarks now that CUDA 11.2 has been out for a while.
  3. Cixelyn

    3090 driver handicap?

    From page 13 of the user manual: The switch is actually mandatory since you need the ConnectX-6 cards directly linked to the GPU on the same PCIe Root in order for GPUDirect RDMA to work for large multi-node deployments. You'll notice that each GPU also has a direct path to one of the NICs on...
  4. Cixelyn

    3090 driver handicap?

    I could definitely see an A100 having higher latency in specific cases; especially if the pinning isn't right, and data has to traverse both a PCIe Switch as well as an Infinity Fabric link before hitting the target GPU. Just due to the nature of 3090 systems, they're much more likely to be...
  5. Cixelyn

    Supermicro E300-9D-4CN8TP Review A 4x 10GbE and 4x 1GbE Server

    Yeah I think these boards need a fair bit of cooling, even moreso than the normal servesr. We had a E300-9D-4C8TP that slowly died on us (likely due to heat death). One of the 10Gbase-T NICs failed first after about a year. And then a few months later the entire motherboard died. Ended up...
  6. Cixelyn

    3090 driver handicap?

    Yeah in our experience the scaling breakpoints are: - 1 GPU: easiest, you don't need to write any special model or data parallel code and stuff "just works" as long as you have enough VRAM. - 4 GPUs: no longer fits in a single EATX chassis _and_ you've maxed out an entire 15A standard USA...
  7. Cixelyn

    3090 driver handicap?

    The Original TF-based SG2 repo is what we were using to test, config-f with batchsize 32. Chassis fans are all maxed. Rack also has a rear door with a giant maxed active fan. We have one system w/ pairs nvlinked, and one without, but it didn't make a big enough difference for our particular...
  8. Cixelyn

    3090 driver handicap?

    lol I wish. The stock situation hasn't improved at all in the past 4 months.The whole parts shortage + scalper situation is driving me nuts; currently trying to get our hands on AMD 5000 series for workstation builds and can't :confused: Wasn't in consideration due to chassis constraints. Our...
  9. Cixelyn

    3090 driver handicap?

    We've had issues attempting to use 3090s in distributed training. I tweeted some graphs here. Not sure if this is driver handicaping or not. Granted this was in October of last year, so the driver situation might have improved a bit, but at the time the downclocking issues was severe enough...
  10. Cixelyn

    Sound proofe Server Rack?

    We use a UCoustic 9210i in our office, and we're pretty happy with the sound dampening. Instead of our office sounding previously sounding like a torrent of screeching harpies, it now just sounds like there's a large always-on commercial AC unit in the corner. Can't do infinite sound...
  11. Cixelyn

    Any interest in forecast planning type content?

    Yeah, honestly I think these projection articles are the best content. It's one thing to regurgitate benchmark numbers, but it's another to actually editorialize a little bit and draw some projections and conclusions. Especially given your industry experience -- I find this sort of content...
  12. Cixelyn

    Rack newbie - cage nut hell

    At least in the US, it's much more difficult to get M6 in the average hardware store vs. 10-32 or 12-24. On one hand, metric is better down with imperial, but on the other hand the convenience of being able to get 10-32 on a moment's notice from the mom-and-pop hardware store down the street is...
  13. Cixelyn

    NVIDIA Acquires Cumulus Networks

    Yeah hmm unfortunately gonna guess that Onyx is going to be killed in favor of Cumulus in a year or two. I liked it too, but it doesn't make sense to fund development of two separate switch OSes esp. when Mellanox already has a SKU + deep integration for Cumulus-based Spectrum switches.
  14. Cixelyn

    RJ45 Cable tester that can certified copper cable to 10Gbps

    Ooh, that video is nice. Please report back with thoughts if you end up purchasing one of the new POE Microscanners! :D