what type of AI audiovisual software is out there? (& hardware needs)

[Twice_Shy]

Hopefully this doesn't seem to broad or clueless, but let me try to say or ask it better:

#1 I am trying to research and better understand alot of the AI software that is available now, or is coming out/in ongoing development... specifically for any audiovisual use.. (picture generation like DALL-E, creating video clips, replacing one voice with a sampled model, etc) Not looking for language models or text or that, JUST things that are for playing with and experimenting with sound, images, maybe generated music, being pretty broad.

#2 People on servethehome seems to be pushing the limits in alot of ways so i'm assuming i'll find interesting things here - that said if there is a "go to" forum or website for this specifically feel free to suggest it.

#3 This is in reality a hardware requirements question. Because i'm curious what kind of hardware is needed to start playing with all of these software tools, including things that may be a bit outside the range of normal consumer gpu's (ie require some specific card that can be bought used, maybe a tesla k80, maybe tensor cores, i dont even know whats out there) and i'm not even sure what exists to get up to speed. What is even out there that I can play around with, if I have the right hardware setup?

Specifically referring to client side or 'home use' not some online service esp not for pay because i'm not doing this for pay. Just for fun and learning at the moment.

 

[unwind-protect]

Consumer NVidia cards are fine, but you need more GPU memory than for gaming. Which card depends on your budget of course. I use a 2080ti, which is one of the cheapest ways to get 11 GB.

 

[Twice_Shy]

I was curious whether everything was Nvidia only, or if it also can run on AMD, or if anything needs like those google tensor unit/other external coprocessor type cards if those can even be gotten on the secondhand market. I don't know if everything runs under CUDA or something else, and whether it would be possible to work on programming experiments starting at the lowest oldest compatible card (which I think is that tesla k80) which would be compatible and have it work on everything newer right up thru grace hopper.

 

[unwind-protect]

Keep in mind that AMD allows only some consumer cards to be used for compute. Unless I knew that my software runs on it and needs more RAM than I could afford in an NVidia card I wouldn't risk AMD.

 

[CyklonDX]

#1 there are plenty of software suites, most easy to set up would be stable diffusion - it also has plenty of webui/ui setups which makes it quite easy to get into. Nice audio models aren't public,

#3
P40 would be a decent starting option. (a lot of vram, and has power states)
Another option good cheap option would be getting 2080/Ti and getting memory upgraded on them to 44-48GB (there are few hacks on aliexpress selling upgraded 20 series cards.) V100 is overpriced for its actual performance - you shouldn't buy them / they loose to A4000 in performance.
Performance wise, you should get A4000 16G its great - the cooling is on the weak side - and homelab A5000 24G is the best.


on AMD side of things, its ROCm; and whatever is being supported on that suite.
Atm everything from Radeon 6600 upwards to 7900 XTX is supported. (HIP SDK is only supported from 6800 and upwards)
Overall support/ease is at this time better on nvidia cards, but... amd also has its niches where things work better there.

 

[bayleyw]

3060 if you feel brave and know your workload fits in 12GB (most image generation work does with a little finagling), 3090 if you don't feel brave. P40 and P100 are for language model enthusiasts who need all the memory they can get, you'll rarely see the benefit of 24GB for image generation and Pascal has no tensor cores.