Search results

  1. Cixelyn

    Rare Epyc 7B12 CPUs PSB Unlocked. 64 CORE SOLD

    Sounds good -- will DM in a bit! Though I wasn't really planning on getting two...
  2. Cixelyn

    3090 driver handicap?

    We managed to find a deal on blower 3090s so I guess I'll have a chance to revisit our earlier benchmarks now that CUDA 11.2 has been out for a while.
  3. Cixelyn

    3090 driver handicap?

    From page 13 of the user manual: The switch is actually mandatory since you need the ConnectX-6 cards directly linked to the GPU on the same PCIe Root in order for GPUDirect RDMA to work for large multi-node deployments. You'll notice that each GPU also has a direct path to one of the NICs on...
  4. Cixelyn

    3090 driver handicap?

    I could definitely see an A100 having higher latency in specific cases; especially if the pinning isn't right, and data has to traverse both a PCIe Switch as well as an Infinity Fabric link before hitting the target GPU. Just due to the nature of 3090 systems, they're much more likely to be...
  5. Cixelyn

    Supermicro E300-9D-4CN8TP Review A 4x 10GbE and 4x 1GbE Server

    Yeah I think these boards need a fair bit of cooling, even moreso than the normal servesr. We had a E300-9D-4C8TP that slowly died on us (likely due to heat death). One of the 10Gbase-T NICs failed first after about a year. And then a few months later the entire motherboard died. Ended up...
  6. Cixelyn

    3090 driver handicap?

    Yeah in our experience the scaling breakpoints are: - 1 GPU: easiest, you don't need to write any special model or data parallel code and stuff "just works" as long as you have enough VRAM. - 4 GPUs: no longer fits in a single EATX chassis _and_ you've maxed out an entire 15A standard USA...
  7. Cixelyn

    3090 driver handicap?

    The Original TF-based SG2 repo is what we were using to test, config-f with batchsize 32. Chassis fans are all maxed. Rack also has a rear door with a giant maxed active fan. We have one system w/ pairs nvlinked, and one without, but it didn't make a big enough difference for our particular...
  8. Cixelyn

    3090 driver handicap?

    lol I wish. The stock situation hasn't improved at all in the past 4 months.The whole parts shortage + scalper situation is driving me nuts; currently trying to get our hands on AMD 5000 series for workstation builds and can't :confused: Wasn't in consideration due to chassis constraints. Our...
  9. Cixelyn

    3090 driver handicap?

    We've had issues attempting to use 3090s in distributed training. I tweeted some graphs here. Not sure if this is driver handicaping or not. Granted this was in October of last year, so the driver situation might have improved a bit, but at the time the downclocking issues was severe enough...
  10. Cixelyn

    Sound proofe Server Rack?

    We use a UCoustic 9210i in our office, and we're pretty happy with the sound dampening. Instead of our office sounding previously sounding like a torrent of screeching harpies, it now just sounds like there's a large always-on commercial AC unit in the corner. Can't do infinite sound...
  11. Cixelyn

    Any interest in forecast planning type content?

    Yeah, honestly I think these projection articles are the best content. It's one thing to regurgitate benchmark numbers, but it's another to actually editorialize a little bit and draw some projections and conclusions. Especially given your industry experience -- I find this sort of content...
  12. Cixelyn

    Rack newbie - cage nut hell

    At least in the US, it's much more difficult to get M6 in the average hardware store vs. 10-32 or 12-24. On one hand, metric is better down with imperial, but on the other hand the convenience of being able to get 10-32 on a moment's notice from the mom-and-pop hardware store down the street is...
  13. Cixelyn

    NVIDIA Acquires Cumulus Networks

    Yeah hmm unfortunately gonna guess that Onyx is going to be killed in favor of Cumulus in a year or two. I liked it too, but it doesn't make sense to fund development of two separate switch OSes esp. when Mellanox already has a SKU + deep integration for Cumulus-based Spectrum switches.
  14. Cixelyn

    RJ45 Cable tester that can certified copper cable to 10Gbps

    Ooh, that video is nice. Please report back with thoughts if you end up purchasing one of the new POE Microscanners! :D
  15. Cixelyn

    RJ45 Cable tester that can certified copper cable to 10Gbps

    has anyone used one of the new PoE Microscanners? would be nice to hear some real world experience -- it's hard to find reviews online.
  16. Cixelyn

    Choosing a server/chassis for GPU workload

    Judging from the service manual diagram, the tops of the GPU seem very close to the outside edge of the case. You might want to ask for the GPU QVL and see if there are any GPUs w/ top power connectors in that list; it might be a pretty tight fit.
  17. Cixelyn

    Choosing a server/chassis for GPU workload

    We have a G242-Z10 as well as several ESC8000 G4/10Gs in production w/ dual-width consumer stuff. If cost is a huge concern, then increasing the GPU density is definitely recommended as the cost you're trying to optimize is total system cost per GPU. I would definitely recommend skipping the...
  18. Cixelyn

    10 Intel NUC's + 5 Nvidia Jetson TK1's Mini PC 1u Single Board Computer Array

    Yeah I've been so incredibly tempted to buy this because "it's so cool! and only $250 or so! Have to reason with myself pretty hard though to not take up even more space in the lab (read: electronics trash heap) My rationale goes like this: if rabb.it were being started today, my guess is that...
  19. Cixelyn

    10 Intel NUC's + 5 Nvidia Jetson TK1's Mini PC 1u Single Board Computer Array

    Found this image over at Rabbit and WebRTC: An Interview With Philippe Clavel • BlogGeek.me. Assuming that they didn't overhaul the architecture significantly between 2014 and when they built those racks, my guess is the NUCs are probably the Rabbitcasts (since they have the HDMI dongles to...