Search results

Well, that's probably for the best - it's hard to justify the hardware and energy usage for LLMs in 2026 if you don't have a secondary use for it.

Welcome to the short and succint guide to the HP Elite t755 thin client (Also known by HP internal codename Puffin) So why should you care (as a home labber)? It's a large-ish thin client, roughly in the dimensions of 210mm W x 210mm H x 50 mm D, ~2.20 Liters, released in Q4 2023, GA in Q1...

WANg submitted a new resource: Introduction to the HP t755 Elite Thin client - decent bang for the buck if you can score one - It's the HP replacement for the t740. And you know I am going to write this eventually... Read more about this resource...

That burns both ways - either the stuff you bought were so cheap that you don't mind it being e-waste (because they essentially are), or you have secondary uses for them besides running unadulterated punchline spigots LLMs. If it's green field spending at 2500-3000, well, don't pay today's...

Well, I bought an Oculink-to-PCIe card for the N5 Air and had no issues with Re-Bar on that port with the V620 (same 16GT/sec bandwidth constraint), so I don't think the resource allocation issue is port specific per-se. That being said, I don't know if the ministry of finance will be okay with...

The Mac Studio M5 Ultra won't be out until late September 2026, and that's 5500 to start (30/64) with 96GB of RAM/1TB SSD, an extra 1300 for 36/80, and an extra 4000 for 256GB of RAM. Now imagine buying 3 of those 256GB nodes (or wait until 2027 for the 512GB nodes) and wire them up for RDMA...

Eh....if GPU is your thing, the V620s are listing for 5-600, but in actuality some vendors will take offers for 380-420, depending on interest. That being said, smart money always look for a plan-B instead of going for the same exits as the others. Have you considered a V340 or a Tesla P40...

...if it's me and I am starting things clean without repurposing things in mind? Something like this Macbook Pro M2 Max/64/512GB - although I'll check the usual secondary marketplace to scour for a deal (that one is viable but not great). It has the 38 core GPU to give you decent (not leading...

Yeah, I know. Got mine in mid-July. ' I am waiting for Tarlin to officially announce the Intel motherboard collab set.

Eh, the underlying OS rarely has any influence on how things run, unless the OS makes it easier to install updated AMD RADV or ROCm drivers...which won't for OpenSuSE (AMD itself gravitate towards Fedora or Ubuntu/Debian for the most part), and as for RAM....eeeh, it depends on how much you are...

...not quite, since you are not running a docker container on proxmox, you are running an LXC container based on a converted docker image, and because of that, you lose some benefits of docker (the ability to switch/rebase OS/userland images while retaining the underlying data since docker has...

Technique 2 (the silly way) - Getting LocalAI to run on Proxmox 9.1 and above Okay, so the second and admittedly sillier method of running an LLM is to run it inside a docker container, and joy of joys, you can now run docker containers as LXC images on Proxmox 9.1 and above...so we will take...

Well, I feel like a lot of the localAI documentation assumes some knowledge about how LLMs are supposed to work on your hardware, and that they really didn't bake in some tools that will make it easier to troubleshoot. Alright, guess I should probably write something that can help out...I...

*ugh* - the more I play with localAI, the less I want to spend time getting it to function. It assumes the following: a) You are running it on your own hardware and will never ever let anything from the outside call to it/use it b) You like docker and wants to run a docker engine on your own...

Yeah, you’ll need to make sure that nothing on the host locked up access to the GPU before the docker container goes up - and even then, you can’t do ROCm (not supported for mainline ROCm due to its gfx900 based iGPU) and I am not sure if theRock (the non-prod, testing only YMMV version of ROCm)...

well, those Macs aren’t exactly the AI inferencing monsters that some will hype it to be - plus on the base model M4s and M5s, even if the prompt rates are good they are still tokenwalking on LPDDR5X so it's not nearly as fast as you think (I think my M4 Mini is roughly the same on token...

Don't forget the Oculink port. You also lose the Oculink port of the N5 Air/Pro. The deeper you look, the less likely you'll favor the value prop of the N5 Max versus its S1 Max mini PC cousin.

Yeah, same here - I am going to toss an Oculink board into the N5 Air and move the V620 there to see if BAR resizing still works or not. I like the idea of dropping an extra DEG01 dock onto my N5 Air to test different cards, but we shall see. Right now I want to see if BAR resizing is...

Were you trying to get Oculink working on the N5 pro using both its built-in Oculink port and on a 3rd party Oculink card installed on that PCIe slot? Yeah, probably no bueno - the slot will need more BAR space allocated and I am not sure if Minisforum expects you to use more than a single...

That might be a BIOS/firmware feature/issue on the N5 Pro…my Air had resizable bar set on day 1 (take a look at my post on the other thread on how to check whether resizable BAR is turned on for Oculink or not). That being said, re-BAR is different between Oculink (really just PCIe on a...