I am building a budget server to run AI and I have no experience running AI software. I'm thinking starting with Llama LLM, but would like to get into making AI pictures and videos as well plus who knows what else once I learn more about this. I am just getting into this and have not received the hardware yet but it is ordered. I'm just gathering information so I know how to get started when it gets here.
System specs:
Dual E5 2686 V4 (32 cores, 72 threads total)
128GB ECC RAM
2TB Gen 4 NVME SSD
(4) 1TB SATA SSDs in RAID 0
(4) Tesla P40 24Gb cards (uses the GP102 chip, same as the Titan XP and 1080TI)
I'm planning to run this headless and remote into it. This is just for tinkering at home and I'm not worried if it isn't the fastest system in the world.
What would be the best OS?
What drivers are the best to use with the Tesla P40 cards?
Any other thoughts on this setup, or suggestions?
Do I need to use NV link on the cards in order to use all the VRAM?
I am thinking of using bifurcation and running each card on 8 PCIE gen 3 lanes, Do you think that would cause a bottleneck?
System specs:
Dual E5 2686 V4 (32 cores, 72 threads total)
128GB ECC RAM
2TB Gen 4 NVME SSD
(4) 1TB SATA SSDs in RAID 0
(4) Tesla P40 24Gb cards (uses the GP102 chip, same as the Titan XP and 1080TI)
I'm planning to run this headless and remote into it. This is just for tinkering at home and I'm not worried if it isn't the fastest system in the world.
What would be the best OS?
What drivers are the best to use with the Tesla P40 cards?
Any other thoughts on this setup, or suggestions?
Do I need to use NV link on the cards in order to use all the VRAM?
I am thinking of using bifurcation and running each card on 8 PCIE gen 3 lanes, Do you think that would cause a bottleneck?