Hey Folks,
I’m new to AI hardware and am having trouble working out what the best option for me to get started with is. I’m interested in training some small models (problem sets that previously I would have called more ML than LLM applicable), learning more about AI hardware deployment models and experimenting a bit with local inferencing for coding and testing.
the catch is that I’ve only got $500ish I want to throw at this right now. I’m not worried about longevity - if I get traction I’ll throw more dollars at it later.
I’ve got two nodes I can put cards into, each with a single x16 3.0 slot, 40 e5v4 cores and 256gb RAM. They both have 2x40gbe links with RoCE.
I’m new to AI hardware and am having trouble working out what the best option for me to get started with is. I’m interested in training some small models (problem sets that previously I would have called more ML than LLM applicable), learning more about AI hardware deployment models and experimenting a bit with local inferencing for coding and testing.
the catch is that I’ve only got $500ish I want to throw at this right now. I’m not worried about longevity - if I get traction I’ll throw more dollars at it later.
I’ve got two nodes I can put cards into, each with a single x16 3.0 slot, 40 e5v4 cores and 256gb RAM. They both have 2x40gbe links with RoCE.
