CPU importance in LLM usage in a machine with a good GPU ?.

[RimBlock]

As per the title...

I am spec'ing a machine for running (not training - I have another machine for that) a LLM and have a GPU for the heavy lifting but have having issues finding information on how much budget should be put on the CPU for this type of system.

Does the CPU just handle normal system tasks (disk, network OS standard processes etc) or is it more involved with running a LLM even if a GPU is already involved.

What would be a good choice of CPU for a mid-range locally hosted private chatbot. Low(ish) TDP would be good.

Thanks

 

[CyklonDX]

when running llama in docker *(with 4 pinned cores) i'm getting around 5-15% cpu usage for 40-80% of gpu utilization.
Its important to note that fast cpu will produce better results if you have powerful gpu - something has to feed work for the gpu.

 

[Patriot]

high clocks, low core counts, keeps the gpu fed just fine. Yes the cpu has impact, its a cpu light workload, so clocks matter the most.
For an Epyc setup, the F (frequency) optimized chips are best. but id imagine a regular ryzen would be dandy.

 

[RimBlock]

Great. Thanks for the feedback.