EU [WTB] Storage and low end recent gpus

[luckylinux]

The Mi50 32gb cannot be found anywhere anymore unfortuntely.

The Mi50 16gb can be found relatively easy at around 100 EUR though, but I quickly read that the BIOS is not so widely available in case you need to reflash.

Not a super good Deal but I guess for a small Setup it might be "OK".

 

[luckylinux]

PIO has the AMD Radeon Instinct MI50 Accelerator Vega 20 16GB HBM2 PCIe 4.0 x16 listed at 99 EUR

AMD Radeon Instinct MI50 Accelerator 16GB HBM2 PCIe 4.0 x16 Vega 20 - Piospartslap

Radeon Instinct MI50 Accelerator mit Vega 20 GPU, 16GB HBM2 Speicher und PCIe 4.0 x16 Schnittstelle. Hochleistungs-GPU für Rechenzentren, KI u

www.piospartslap.de

They just accepted my offer of 65 EUR / Piece (including 19% German VAT) for 5 Pieces.

32GB would have been better of course, but this is what can be purchased right now ...

I don't really use huge Models yet, so I guess this will do fine for a while .

 

[thulle]

@luckylinux huh, didn't know these had started to show up cheaply. Did a check on alibaba, and the 32GB card is available there.

But on top of the 26 fp16TFLOPs MI50 there's also the 2 year later model MI100 @ 184 fp16TFLOPs.
€1400 in quantities of 5, but €140 in quantities of 2500+.. group buy?

 

[luckylinux]

@luckylinux huh, didn't know these had started to show up cheaply. Did a check on alibaba, and the 32GB card is available there.

But on top of the 26 fp16TFLOPs MI50 there's also the 2 year later model MI100 @ 184 fp16TFLOPs.
€1400 in quantities of 5, but €140 in quantities of 2500+.. group buy?

How much do you trust Alibaba ? At least PIO you get 1 Year Warranty for used Goods I believe (EU Regulation). Although yeah, 16GB isn't ideal by any means. But it's cheap until maybe something better comes by . Since I am cheap I don't even know exactly where to install them though yet. Maybe 1-2 here, 1-2 there, on some cheap MB + CPU, powered Ad-Hoc, etc.

Is that 1400 EUR / 5 Pieces or 1400 EUR / Piece if you buy 5 ?

 

[iraqigeek]

How much do you trust Alibaba ? At least PIO you get 1 Year Warranty for used Goods I believe (EU Regulation).

Personally, I trust a reputable seller anywhere regardless of location.

I have purchased 17 Mi50s from Alibaba and haven't had any issues. They are solid state devices, not even a fan. Those cards are built for the data center and like any other enterprise/data-center product, they are very reliable. PIO, or any other shop, wouldn't be selling anything cheap if they had any meaningful failure rate, otherwise they'd go out of business.

And TBH, even if one or two die, considering I got them for €135/card delivered, it's still a very good deal.

 

[luckylinux]

Personally, I trust a reputable seller anywhere regardless of location.

I have purchased 17 Mi50s from Alibaba and haven't had any issues. They are solid state devices, not even a fan. Those cards are built for the data center and like any other enterprise/data-center product, they are very reliable. PIO, or any other shop, wouldn't be selling anything cheap if they had any meaningful failure rate, otherwise they'd go out of business.

And TBH, even if one or two die, considering I got them for €135/card delivered, it's still a very good deal.

Yeah, you got them back then .

Right now Price on eBay is around 300 EUR / Piece for the 32GB Model, that's why I was tempted by and got 5 Pieces of the 16GB one at 65 EUR / Piece from PIO Website. I'm tempted to buy another 5 of these actually ...

Not sure if anybody has other Alibaba Sellers to recommend for the 32GB Version for a similar Price you got it. It seems Price shot up in the last Month or so.

 

[iraqigeek]

Right now Price on eBay is around 300 EUR / Piece for the 32GB Model, that's why I was tempted by and got 5 Pieces of the 16GB one at 65 EUR / Piece from PIO Website. I'm tempted to buy another 5 of these actually ...

Not sure if anybody has other Alibaba Sellers to recommend for the 32GB Version for a similar Price you got it. It seems Price shot up in the last Month or so.

Just go to alibaba.com and search for Mi50 32GB. That's how I bought mine. I searched now, and there are a couple of sellers selling the 32GB model for €165 plus shipping. I'd just message them asking for a quote for whatever number of cards you want (the more the better), and see what they say. Like I said before, shipping is not linear. So, the more you buy the cheaper shipping is. Also ask for DDP shipping (Deliver Duty Paid). it's more expensive, but saves a lot of hassle and clearance fees.

That is literally how I got my cards. I just messaged a few sellers and ordered with the one who replied answering all my questions and was the most helpful.

 

[luckylinux]

Just go to alibaba.com and search for Mi50 32GB. That's how I bought mine. I searched now, and there are a couple of sellers selling the 32GB model for €165 plus shipping. I'd just message them asking for a quote for whatever number of cards you want (the more the better), and see what they say. Like I said before, shipping is not linear. So, the more you buy the cheaper shipping is. Also ask for DDP shipping (Deliver Duty Paid). it's more expensive, but saves a lot of hassle and clearance fees.

That is literally how I got my cards. I just messaged a few sellers and ordered with the one who replied answering all my questions and was the most helpful.

How did you pay ? Paypal with Buyer Protection or ?

These 165 EUR you mention, would that be including Taxes/VAT (DDP) or without ?

The 16GB for 65 EUR seemed like an OK Deal, but maybe I should NOT have done that after all if the 32GB is just slightly more than double the Price.

EDIT 1: Today alone look like approx. 100 Pieces of the 16GB Model were sold on PIO Website . Only 83 remaining. I wonder if I should order another 5 Pieces there .

 

[redblood]

I messaged some of them. Real prices haven’t been updated so some say it’s out of stock other say it’s 250 usd

 

[iraqigeek]

How did you pay ? Paypal with Buyer Protection or ?

Yes, paypal. Regular paypal payments always include buyer protection, AFAIK. Alibaba also holds payment in escrow for some time. I haven't checked the details of the escrow service because I treat it the same way I treat ebay or local classifieds: talk to the seller and gauge their cooperation and honesty beforehand. I've been buying 2nd hand all over the world for well over 20 years now, bought literally hundreds of items for myself, family, and friends, and an count on one hand the number of times I had any sort of issue.

These 165 EUR you mention, would that be including Taxes/VAT (DDP) or without ?

Prices on the website are always without shipping and taxes. I ask for a quote for X cards with DDP shipping to my address (Germany) and taxes, and then work the price per unit.

It really doesn't cost anything to write a message to a few sellers. You can even copy-paste the same message to multiple sellers. It also doesn't cost anything to try to negotiate the price. The worst thing they can say is: no discount.

The 16GB for 65 EUR seemed like an OK Deal, but maybe I should NOT have done that after all if the 32GB is just slightly more than double the Price.

EDIT 1: Today alone look like approx. 100 Pieces of the 16GB Model were sold on PIO Website . Only 83 remaining. I wonder if I should order another 5 Pieces there .

My issue with the 16GB model is density. Even with six cards, that's 96GB VRAM. Not bad, but not great either. With the 32GB cards, those six cards get you 192GB VRAM, which opens the door to running a lot bigger and more interesting models.

The cards idle between 15-22W whether they're 16 or 32GB, and whether they have a model loaded or not. So, there's also that, but at 65€ it'll be a while before the cost of power comes into play.

I have Gemma 27B Q8, Qwen 3 Coder 30B-A3B Q8, and Orpheus-Kartoffel 3B F16 loaded on four cards, and can still load gpt-oss-120b on the remaining two cards plus the 16GB left on the one running Kartoffel. Or, I can load Qwen 3 235B on five cards with lots of context and still have room for 30B or lower model on the sixth card.

 

[iraqigeek]

I messaged some of them. Real prices haven’t been updated so some say it’s out of stock other say it’s 250 usd

Yeah, most of the stock has already been sold out.

I've been trying to tell people to jump on those cards for two months now. They were already great value for LLMs before llama.cpp and (forked) vllm support, but so many were stuck on this "out of support" notion. By the time the news about better support in llama.cpp and vllm spread out, it was already too late.

 

[luckylinux]

My issue with the 16GB model is density. Even with six cards, that's 96GB VRAM. Not bad, but not great either. With the 32GB cards, those six cards get you 192GB VRAM, which opens the door to running a lot bigger and more interesting models.

The cards idle between 15-22W whether they're 16 or 32GB, and whether they have a model loaded or not. So, there's also that, but at 65€ it'll be a while before the cost of power comes into play.

I have Gemma 27B Q8, Qwen 3 Coder 30B-A3B Q8, and Orpheus-Kartoffel 3B F16 loaded on four cards, and can still load gpt-oss-120b on the remaining two cards plus the 16GB left on the one running Kartoffel. Or, I can load Qwen 3 235B on five cards with lots of context and still have room for 30B or lower model on the sixth card.

True, I guess to get started the 16GB could be of some use, but of course when I will need to do some more advanced Stuff, I'll surely appreciated the 32GB Model.

I can't really find a cheap Server Motherboard right now. The cheapest I currently have, avoiding dual Socket Motherboards, would be the Supermicro X10SRi-F (or X10SRL-F, unsure which one I have, I'd have to check) where I'd need to cut off the plastic End of the PCIe Slots, and without Risers I'd have 3xGPUs (with 1 Riser and a supporting Chassis, that would be 4xGPUs).

Still not sure how you manage to stretch 6x of them in one Motherboard. Surely with Risers, but with an external Enclosure or what ?

IIRC you suggested the Supermicro X11 LGA3647 single Socket Motherboard (X11SPL-F maybe ?), but prices are still very high for that .

I got a single socket EPYC in proprietary Form Factor, so I'd need a PCIe x16 to external Chassis conversion "Riser" to use that (probably Bifurcation is NOT supported, thus it needs some costly active Type).

The cheap alternative otherwise is 2xGPUs per System with a Supermicro X10SLL-F/X10SLM-F which I have plenty or potentially a X11SSL-F/X11SSM-F which I have a few of.

 

[luckylinux]

Yeah, most of the stock has already been sold out.

I've been trying to tell people to jump on those cards for two months now. They were already great value for LLMs before llama.cpp and (forked) vllm support, but so many were stuck on this "out of support" notion. By the time the news about better support in llama.cpp and vllm spread out, it was already too late.

I think you told me to contact you if I wanted the contact of the Seller, then Life got in the Way and something else had higher Priority for me .

Now it's too late indeed .

Maybe better to grab another 5x16GB while I can still place an Order with the existing Quote and at least grab those so I'll be somewhat OK for now ...

 

[luckylinux]

Oh well, just placed another Order for 5 Pieces, so I'll have 10x16GB in total for around 720 EUR shipped.

I'll probably regret it soon, but given that the other alternatives are currently much more expensive, I guess they will better do for now .

EDIT 1: I'm almost eating my Nails ... a few Months ago PIO counter-offered the 32GB for 180 EUR and I refused . Well, cannot go back in Time, would be nice though .

 

[iraqigeek]

Still not sure how you manage to stretch 6x of them in one Motherboard. Surely with Risers, but with an external Enclosure or what ?

Just go back to the first page of this thread for a picture
There's one riser for the 5th and sixth card.

IIRC you suggested the Supermicro X11 LGA3647 single Socket Motherboard (X11SPL-F maybe ?), but prices are still very high for that .

I got a single socket EPYC in proprietary Form Factor, so I'd need a PCIe x16 to external Chassis conversion "Riser" to use that (probably Bifurcation is NOT supported, thus it needs some costly active Type).

The cheap alternative otherwise is 2xGPUs per System with a Supermicro X10SLL-F/X10SLM-F which I have plenty or potentially a X11SSL-F/X11SSM-F which I have a few of.

Single vs dual socket is only relevant if you don't have enough VRAM and plan to offload to system RAM, which I guess is your situation now with the 16GB cards. Otherwise, X11DPG-QT or X10DRX. I have a few of the latter that I plan to sell, but I first need to test them. Both are BIG, SSI-MEB with a couple of screws in a slightly different location.

I'd avoid splitting the cards across systems now that you're stuck with 16GB cards. 32 GB VRAM won't get you far. llama.cpp still has some bugs around AMD cards when doing (their fake) tensor parallelism, so you can only split across layers.

The Mi50 isn't bad, but isn't great either with dense models. I get 16t/s on Gemma 27B Q8 with less than 1k context. gpt-oss-120b, OTOH, runs at 40t/s, and Qween3 Coder 30B Q8 at ~46t/s, both with over 10k context. Prompt prompt processing for Gemma is ~150t/s, while gpt-oss-120b and Qwen 3 30B are ~400t/s.

 

[iraqigeek]

Oh well, just placed another Order for 5 Pieces, so I'll have 10x16GB in total for around 720 EUR shipped.

I'll probably regret it soon, but given that the other alternatives are currently much more expensive, I guess they will better do for now .

You won't regret it. Wait until those sell out and they'll be worth double what you paid. So, worst case scenario you can sell the ones you don't need for a nice profit

Why do you think I have 17 32GB Mi50?
I plan to use ten, and sell the other seven to recover my cost.

EDIT 1: I'm almost eating my Nails ... a few Months ago PIO counter-offered the 32GB for 180 EUR and I refused . Well, cannot go back in Time, would be nice though .

Now you know!
There'll always be a next time, and when that happen you'll know to get more cards than you know what to do with.

I also loaded on over 2TB of ECC DDR4 over the past year. Now they're selling for 4-5x what I paid. The internet also told me I was wasting my money, yet here we are.

 

[luckylinux]

Just go back to the first page of this thread for a picture
There's one riser for the 5th and sixth card.

I think 6 GPUs in one Server is a bit ... tight. I'm not even sure I can easily fit 4 of them in one Chassis !

I saw some 120mm Fan 3D Printed Adapters to 2xGPU or 3xGPU that looked promising. You used a 80mm Fan. Do you have the 3D Model to 3D print (for the Adapter) ?

Single vs dual socket is only relevant if you don't have enough VRAM and plan to offload to system RAM, which I guess is your situation now with the 16GB cards.

I don't really plan to offload the Model to the CPU. I planned on having the Model split across multiple GPUs.

There might be some confusing Discussions online on how GPUs directly communicate between each other over PCIe. I would have expected such Traffic to go through the CPU though. Or is it some kind of DMA that is supported over PCIe (from which other Technologies like NIC Offload RDMA/ROCE also benefit) ?

Otherwise, X11DPG-QT or X10DRX. I have a few of the latter that I plan to sell, but I first need to test them. Both are BIG, SSI-MEB with a couple of screws in a slightly different location.

Coincidentally about the X10DRX (I was originally thinking of getting a X9DRX for a flash Server) I bought a second-hand Chassis that should fit it: Phanteks Enthoo Pro II Server Edition (has 11 PCI/PCIe Slots).

Phanteks Enthoo Pro II – Server Edition | TG

The Enthoo Pro 2 Server Edition is designed to bring server-grade capabilities in a full-tower chassis. This lowers the entry into server systems for semi-professional users and businesses with a very powerful solution.

phanteks.com

But again, why wouldn't a Dual Socket Board work if a Model needs to split across multiple GPUs ?

I'd avoid splitting the cards across systems now that you're stuck with 16GB cards. 32 GB VRAM won't get you far. llama.cpp still has some bugs around AMD cards when doing (their fake) tensor parallelism, so you can only split across layers.

The Mi50 isn't bad, but isn't great either with dense models. I get 16t/s on Gemma 27B Q8 with less than 1k context. gpt-oss-120b, OTOH, runs at 40t/s, and Qween3 Coder 30B Q8 at ~46t/s, both with over 10k context. Prompt prompt processing for Gemma is ~150t/s, while gpt-oss-120b and Qwen 3 30B are ~400t/s.

So Ollama cannot split the Model VRAM across multiple GPUs ?

What would be the consequence and difference (split across layers vs split across cards) ?

 

[luckylinux]

You won't regret it. Wait until those sell out and they'll be worth double what you paid. So, worst case scenario you can sell the ones you don't need for a nice profit

Maybe I should buy another 5 after all then .

Why do you think I have 17 32GB Mi50?
I plan to use ten, and sell the other seven to recover my cost.

I thought you wanted to use them, not to be a Scalper .

Now you know!
There'll always be a next time, and when that happen you'll know to get more cards than you know what to do with.

The Question is when and be aware of the great Deals, which sometimes go crazy fast .

 

[MSameer]

EDIT 1: I'm almost eating my Nails ... a few Months ago PIO counter-offered the 32GB for 180 EUR and I refused . Well, cannot go back in Time, would be nice though .

There is a seller on Aliexpress selling 2 MI 50 32GB cards for 300 euros (with all discounts applied). With our beloved 25.5% VAT that should be 189 euros per card.

Only problem is shop is from Feb this year. 69 positive reviews and 4.9 rating but too new to trust :/

 

[iraqigeek]

I thought you wanted to use them, not to be a Scalper .

This is a very expensive hobby, and I kept telling everyone I could to get them while they were cheap. It's not that bad to recoup some of my costs.

The Question is when and be aware of the great Deals, which sometimes go crazy fast .

When is anytime there is hardware that has potential. The Mi50 has 32GB VRAM and more FP32 compute (and much much much more Fp16 compute ) than the P40.

How to be aware? Keep an eye on the STH forums and on r/LocalLLaMA on reddit. I found about the Mi50 on LocalLLaMA.