I'm deciding a little late to finally start to learn more about this AI thing over the next year. A little late to the party with how high vram gpus skyrocketed and the RAMpocalypse so that makes it more expensive than I would like because i'm still slowly working thru college. (yes years later for those who saw me post things awhile back, covid era set things back) So learning at minimum cost is the goal. Two years ago I was interested as a curiosity for video generation, now I have other reasons to learn as well.
I DONT need to run a "production system" right now, I do need enough function to learn how things work, so that when I can afford to upgrade things to be used on some real world project i'll have the skills to make things operate. It's okay if it's a bit slow or uses lesser hardware as long as everything learned transfers to better stuff.
I currently have two systems that I think are the most suitable for AI learning - an HP Z6 G4 and a Dell Rack 7920, both supporting 1st and 2nd gen Xeon Scalable and capable of a whole bunch of now less affordable DDR4 ram, both PCIe 3.0. Either or both can be used for this if there's any difference, i'll also be playing with Proxmox, relearning Linux, and some other things. The Z6 is already a dual cpu, the Dell only a single but it looks like adding a 2nd cpu is an easy add on like adding a few fans and a fan duct. Using Optane is an option on either AFAIK.
I expect to be buying certain ex-datacenter gpus ANYWAYS for other tech projects, mostly having larger VRAM for cheap to run Davinci Resolve on - Tesla K80 (aware it's not supported on newer Resolve just older), M40, P40, maybe an AMD Mi50, and although the Teslas are old and the AMD will be less compatible i'm wondering if there's anything at all runnable, learnable, or useful on any of them. It's hardware i'll be having for other reasons - i'm not buying it to run AI.
What I hope to actually get for AI will be 1-4x V100 cards which there's a chinese hack to enable NVLink for. My understanding is the V100 at least is usable, esp in larger VRAM pools with NVlink, although it may not be no longer supported by the latest AI stuff coming out. Which is fine if that drives the price down some since i'm probably not buying that for at least 6 months right now.
Specific things I hope to learn will involve selfhosted AI video generation, selfhosted AI music algorithms (some can use 24gb or more of VRAM tho I wont have dual 16gb V100 NVlinked for awhile to run the "more" part), LLM stuff specifically with retrieval augmented generation (though I don't know much beyond the concept yet), and AI type automation/the Claude type things. (nothing with access to my bank account or critical passwords though ) I'm aware of options like renting someone else's GPU online and just transfering files to/from those sessions and experiments much over 24gb VRAM or 32gb for awhile might involve that but first I have to know how to run it local and i'm still stuck at the relearning linux command line phase for awhile. Giving myself a year to try to get back up to speed.
Suggested next steps, places to look for understanding, or other questions about what i'm trying to do? I put this under processors not sure where it most belonged/thread can be moved if that's not the best place for it.
I DONT need to run a "production system" right now, I do need enough function to learn how things work, so that when I can afford to upgrade things to be used on some real world project i'll have the skills to make things operate. It's okay if it's a bit slow or uses lesser hardware as long as everything learned transfers to better stuff.
I currently have two systems that I think are the most suitable for AI learning - an HP Z6 G4 and a Dell Rack 7920, both supporting 1st and 2nd gen Xeon Scalable and capable of a whole bunch of now less affordable DDR4 ram, both PCIe 3.0. Either or both can be used for this if there's any difference, i'll also be playing with Proxmox, relearning Linux, and some other things. The Z6 is already a dual cpu, the Dell only a single but it looks like adding a 2nd cpu is an easy add on like adding a few fans and a fan duct. Using Optane is an option on either AFAIK.
I expect to be buying certain ex-datacenter gpus ANYWAYS for other tech projects, mostly having larger VRAM for cheap to run Davinci Resolve on - Tesla K80 (aware it's not supported on newer Resolve just older), M40, P40, maybe an AMD Mi50, and although the Teslas are old and the AMD will be less compatible i'm wondering if there's anything at all runnable, learnable, or useful on any of them. It's hardware i'll be having for other reasons - i'm not buying it to run AI.
What I hope to actually get for AI will be 1-4x V100 cards which there's a chinese hack to enable NVLink for. My understanding is the V100 at least is usable, esp in larger VRAM pools with NVlink, although it may not be no longer supported by the latest AI stuff coming out. Which is fine if that drives the price down some since i'm probably not buying that for at least 6 months right now.
Specific things I hope to learn will involve selfhosted AI video generation, selfhosted AI music algorithms (some can use 24gb or more of VRAM tho I wont have dual 16gb V100 NVlinked for awhile to run the "more" part), LLM stuff specifically with retrieval augmented generation (though I don't know much beyond the concept yet), and AI type automation/the Claude type things. (nothing with access to my bank account or critical passwords though
) I'm aware of options like renting someone else's GPU online and just transfering files to/from those sessions and experiments much over 24gb VRAM or 32gb for awhile might involve that but first I have to know how to run it local and i'm still stuck at the relearning linux command line phase for awhile. Giving myself a year to try to get back up to speed.Suggested next steps, places to look for understanding, or other questions about what i'm trying to do? I put this under processors not sure where it most belonged/thread can be moved if that's not the best place for it.