Broken probability of data-centre GPUs
- Thread starter blang
- Start date
-
- Tags
- a100 data center
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
the best public data I know of is the OPT-175B training log and it's rough metaseq/projects/OPT/chronicles/OPT175B_Logbook.pdf at main · facebookresearch/metaseq
my understanding is that it's not so much the GPUs failing as transient communications errors causing collective operations to hang
my understanding is that it's not so much the GPUs failing as transient communications errors causing collective operations to hang
