Funny enough, when the alternative is spending $10k-15k, yes I will do that. But I am a developer myself and I enjoy tinkering with things.
Intel Max 1550 ES (QZFN) — 128GB HBM2e on the desktop: feasibility, oneAPI/llama.cpp backend status, carrier board options
- Thread starter Raineypenguin
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
Same, that's why I got a Intel max 1100, plus that most of it is opensource makes it a huge bonus. But Pytorch-XPU still supports the Intel max cards
Ended up purchasing the "ES" 1550 on eBay, will update later once I get it. It seems the seller was using Intel arc or battle mage drivers from what the screen shot shows.
Have you received it? I'm sure Claude or similar AI harness could help you set it up once you've connected it, presumably with OAM to PCIE adapter card
Got to go pick it up from the post office.
But I've done a lot of thinking, researching and talking with Claude and Gemini. We are pretty sure it will work with the drivers I currently have since I'm running an Intel max 1100 ES/QS already.
As an update, the Gaudi 2 does indeed work with the OAM to PCIE adapter. Obviously though, the RoCE does not as there are no QFSP ports on the adapter board. Additionally I discovered that these adapter boards are "leaked from NVIDIA" and have NVIDIA branding on them, so I think they might be engineering samples. I am surprised that no factory in China has started making these yet.
TY fren, i'm very interested in gaudi 2 myself, did anyone manage to run llamacpp with these guys? I mean it is cheap vs NV and clearly there has to be a reason why it's that cheap. Is it big pain to run any of recently released models on these? qwen, nemotron, ornith whatnot?
it seems tolerable, you get updates once a quarter GitHub - vllm-project/vllm-gaudi: Community maintained hardware plugin for vLLM on Intel Gaudi but I am not sure how much longer the Gaudi division of Intel will continue to exist.
the model compatibility matrix is poor Validated Models - vLLM Hardware Plugin for Intel® Gaudi® and somehow even makes the old ROCm look impressive
the model compatibility matrix is poor Validated Models - vLLM Hardware Plugin for Intel® Gaudi® and somehow even makes the old ROCm look impressive
Do some (old) models work? Yes. Do they need to be optimized by your favorite LLM? Also yes. This is not for your average joe unfortunately, and until more people get their hands on them I think performant open source kernels for the latest models will be hard to find. However with the rise of intelligent agentic coding, I think you can get it working for most models. Is it better then spending $15k (or is it now $20k) on a RTX 6000 pro? Absolutely. Anything is better than that highway robbery.
Yeah i looked at vllm-gaudi and it wasnt very promising at the first glance. Maybe it'll get better soon? Idk. But gaudi 2 is really cheap. I honestly have no deep or even vague understanding of how exactly LLMs work tbh i'm rather new to all this and been running mostly small-medium sized recent moes with CPU only inference so llama-server is all i know 
Not a factory, but zesty_close_hat_1020 on reddit makes them, he makes sxm5 adapters and nvlink bridges as well for his adapter boards.
But, anyway, I got the 1550, but haven't gotten a oam to pcie board yet, and I need a meanwell 1000watt 48v psu, because according to what I've found when the 1550 comes up it tests for voltage spike stability and goes up to 800 watts.
It's also takes some work to get models working on the Intel max cards, but still worth it in my opinion. Openvino does not work with them from the start, need some modifications to get them to work, and it is limited to what models it can use. Pytorch is simpler to get models running in my opinion and there isn't really any performance difference, but the models I've tried in pytorch either don't work on openvino or weren't compatible from the start.
I bought from him, so I can confirm that the board works. However he does not make them, they are NVIDIA engineering samples. If you ask he might be able to sell you a 12V to 48V step up converter on a custom PCB so you can use an ATX PSU.
Do you know where to buy any? If I can get my hands on some I’d be happy to work on it.
I joined the Gaudi2 journey by the way. Purchased oam to pcie boards from goofish(xianyu) which is chineese ebay.
Also with the 54v psu's. I'll try it and share some review when I get it guys!
Also with the 54v psu's. I'll try it and share some review when I get it guys!
我跟所有在出售 1550 ES 的人聊过了，他们自己都在寻求技术支持，也有可能是那些原来的卖家已经消失了
Ah understand, appreciate that information. I guess I'll document everything I can about mine as I work on it.
@jo2305
hi buddy, could you confirm you mean you've tested that OAM to PCIe adapter with Gaudi2, but not A100?
have you tried to run vLLM-Gaudi with the Gaudi2 and this adapter?
Seller said he only tested it with A100.
if indeed you've used it with Gaudi2 then guess all OAM 1.1 card should be compatible with it including A100/MI250/Gaudi2?
