GPU communication bandwidth test on SuperMicro 4029GP-TRT2

Discussion in 'DIY Server and Workstation Builds' started by matthew_z, Aug 27, 2018.

  1. matthew_z

    matthew_z New Member

    Joined:
    Aug 25, 2018
    Messages:
    3
    Likes Received:
    0
    This 4029GP-TRT2 server consists of 10 x 1080Ti + 2x Xeon Silver 4114. I suppose TRT2 is a double-root since the the official site did not say it is a single root complex.
    It seems that in the uni-p2pDiabled mode, the bandwidth is much better than the double-root version (4028GR-TR), and is close to the single root version (TR2).

    However, somehow I cannot make the P2P=enabled work. The program hangs when P2P=enabled. Maybe I didn't install CUDA or NCCL correctly.

    4029GP-TRT2:
    upload_2018-8-28_14-5-53.png






    4028GR-TR bandwidth result from https://www.servethehome.com/single-root-or-dual-root-for-deep-learning-gpu-to-gpu-systems/

    upload_2018-8-28_14-16-52.png
    4028GR-TR2

    upload_2018-8-28_14-16-44.png
     

    Attached Files:

    #1
Similar Threads: communication bandwidth
Forum Title Date
DIY Server and Workstation Builds Expanders, dual ports and bandwidth OH MY Oct 6, 2017

Share This Page