You could start by planning out what you would want next to GPU-heavy server(s) and spec out that hardware's needs to scope out your colo requirements. And have a ball-park budget in mind because you will need to haggle with everyone and be willing to enter into multi-year contracts to get decent pricing.
Patrick touched on some of them:
- router / firewall / vpn server?
- 1G managed switch for mgmt ports on servers, colo up-link etc
- switch for high-speed interconnect
- file server for training and model data etc
- utility / vm server for non gpu things?
- PDUs (un-managed ones often can be included in the NRC & MRC of your cabinet, but manged ones you can source on your own)
If you end up with a colo facility near you, much of that you can build up over time and add hardware as you go. If you determine what your base needs are so most all GPU related data is contained within the rack, you won't need to provision a lot of excess Internet bandwidth to start. You can direct connect the GPU server and file server high-speed network ports then add a switch later for example.
Then with your numbers for:
- power needs A+B redundant 208V 30A
- bandwidth (50Mb commited on aburstable, or 100M fixed port, etc)
- Size of IPv4 block (and IPv6 if you can handle it at home)
- If fractional cabinets are offered that meet your power needs
Keep in mind that if you do go with less than a full cabinet, you will probably need horizontal PDUs, so expect to use up 2U+ for each (4U for an A+B redundant power feed).
For bandwidth, with what you describe I would try and stick to facilities that can offer you bandwidth directly from a blend of providers they manage. The other style of site is "carrier neutral" and the colo will sell you a cross connect to a transit provider who you have to do a seperate contract with for IP & Bandwidth. That is more hassle than you probably want to deal with for your first colo setup. Also if they start asking you about Transfer or even advertise things that way on their site, don't bother with them. You should be able to get a price for 50Mb commit on a 100M or 1G port and know how much overages cost if it is a burstable connection. Find out the providers they use to make up their blend, or look the provider up in peering DB to find out yourself. You can also as who is on-net in the building for the future if you want to get bandwidth directly from a provider.
You won't need a lot of IPs if you are basically extending your current home lab to the colo - you only need one to setup a tunnel between your current lab and the colo and keep everything else in the rack on private IPs. A $50 EdgeRouter-X on each end would get you started. You can setup a tunnel between the 2 points, and on the colo side offer up VPN connections so you can "dial-in" to the rack and do work within from anywhere, plus provide DHCP and DNS services for the equipment in the rack.
Then just google for colocation and the city you want to start looking in and call around with your power and bandwidth needs. You will quickly run into the 3 or 4 colo directory sites that aggregate info by location (DataCenterMap, CloudAndColo etc). Talk to some folks to get numbers and play them off each other, then see if you can get a tour of one or 2 (they should have skeleton staff 24x7 even in the COVID era).
If you also get a price from a provider 1-2h away in a less popular city or across a state line you can get an idea of what the bulk power costs are they they have to pay and see if you can save a few thousand per year by putting your equipment a bit father from home to see if it is worth the drive.
If you have a target area you are looking in that you can share, people may have suggestions on sites to try or to avoid.
Good luck and have fun with it! Now is a great time of year to do this if you are quick - sales guys want to add another client to their books before EOY, so take advantage of that.