[FS][US-NJ] Supermicro 1029GQ-TVRT w/ 4× V100 32GB SXM2 ($3,900) | Foxconn Mt. Collins 2× Ampere Altra Q80-30, 24× U.2 ($2,400) - local pickup

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
W

werewolf191

New Member
Feb 10, 2025
28
8
3
Terms
  • Local pickup only (near nj 35, middletown monmouth county). I don't have suitable boxes for these, so no shipping.
  • Cash or cash equivalent at pickup.
  • I'm unable to lift these systems myself, so the buyer will need to carry them out.
  • I'm generally home and flexible on pickup times.
  • Both are priced well below comparable configurations I've seen, but reasonable offers are welcome.
Server 1: Supermicro SYS-1029GQ-TVRT (1U), 4× Tesla V100 32GB SXM2 - $3,600 OBO
  • 4× NVIDIA Tesla V100 SXM2 32GB (128GB HBM2 total), fully connected NVLink topology
  • Two of the four are the LS variant, currently capped at ~163W; the other two are full 300W. The power limit on the LS cards can be raised.
  • 2× Intel Xeon Gold 6230 (40 cores / 80 threads total)
  • 2× Mellanox ConnectX-5 VPI (100GbE / EDR InfiniBand)
  • Dual onboard 10GbE
  • 2× 256GB SATA SSDs for OS, Ubuntu 24.04 installed
  • Storage expansion: 1× M.2, 2× U.2 NVMe
  • Dual 2000W PSUs
Testing: 10-minute gpu-burn across all four GPUs. All four completed with 0 errors and 0 ECC errors, max observed temperature ~68°C. Results are in the photos. Photos and test results:
http://imgur.com/a/e1UuCZV
Server 2: Foxconn Mt. Collins (2U), dual Ampere Altra, 160 cores, 24× U.2 NVMe - $2,200 OBO
  • 2× Ampere Altra Q80-30 (80 cores each, 160 ARM64 cores total)
  • 64GB DDR4-3200 (4× 16GB); platform supports up to 16 DIMMs / 4TB per socket
  • 1TB NVMe SSD
  • 24× U.2 NVMe bays, cabled and ready for drives
  • PCIe Gen4 expansion with support for multiple FHFL accelerator/GPU cards
  • Broadcom NetXtreme-E OCP 3.0, 4× 25GbE SFP28
  • 2× Intel I350 1GbE
  • Dual 2000W PSUs
An unusual platform that would suit ARM64 development, Kubernetes/container labs, compile farms, virtualization, or all-flash storage. Photos:
http://imgur.com/a/J2BsUEf
PM me if interested. Thanks!
 
Last edited:
  • Like
Reactions: TangTasteTester
W

werewolf191

New Member
Feb 10, 2025
28
8
3
TonyArrr said:
Aww man, that Foxconn would be perfect if we could arrange shipping :eek: ah well, good luck!
Click to expand...
Thank you. Yeah it is such a beast... Just that i have no use anymore in the lab as i am winding down a lot... I wish I could ship but not now... May be possible in couple of months though....
 
S

SnJ9MX

Active Member
Jul 18, 2019
221
199
43
let me help you out a bit with an interesting repo - GitHub - 1CatAI/1Cat-vLLM: V100 / SM70-focused vLLM engineering fork for modern LLM inference.. it is more or less a way to run nvfp4 quantized models on v100

I came across that and bought a 8x V100 32GB server off ebay. it runs these local models very well. I of course had Claude further optimize, mine is a bit above the published numbers but looks like the repo is making huge progress on optimizing these older GPUs.

4x V100 32GB is plenty good for dense models under say 70B params or so... and it can run the larger at smaller context.

Code: 
hey claude, I just got a v100 server - 4x 32GB v100. you can connect to it at ssh user@10.1.2.3. clone this repo and get qwen3.8-27b or qwen3.8-flash-next nvfp4 brought up on it please.
I have high hopes for qwen4 coming out soon. same arch as flash-next, so should be relatively drop in ready

edit: the repo it looking more AI slop tastic compared to last time I actually looked at the readme but the underlying code/contents is solid
 
  • Like
Reactions: werewolf191
W

werewolf191

New Member
Feb 10, 2025
28
8
3
SnJ9MX said:
let me help you out a bit with an interesting repo - GitHub - 1CatAI/1Cat-vLLM: V100 / SM70-focused vLLM engineering fork for modern LLM inference.. it is more or less a way to run nvfp4 quantized models on v100

I came across that and bought a 8x V100 32GB server off ebay. it runs these local models very well. I of course had Claude further optimize, mine is a bit above the published numbers but looks like the repo is making huge progress on optimizing these older GPUs.

4x V100 32GB is plenty good for dense models under say 70B params or so... and it can run the larger at smaller context.

Code: 
hey claude, I just got a v100 server - 4x 32GB v100. you can connect to it at ssh user@10.1.2.3. clone this repo and get qwen3.8-27b or qwen3.8-flash-next nvfp4 brought up on it please.
I have high hopes for qwen4 coming out soon. same arch as flash-next, so should be relatively drop in ready

edit: the repo it looking more AI slop tastic compared to last time I actually looked at the readme but the underlying code/contents is solid
Click to expand...
Thanks for the pointer, appreciate it. I am actually already on 1Cat-vLLM with this box and agree, it gives the V100s a real second life for inference. NVFP4 on 4x 32GB is surprisingly usable.

My main objective is LoRA fine-tuning rather than serving though. I've been experimenting with a few things there, including some of the newer MoE LoRA approaches to see what learns well at FP16, since there is no native bf16 on Volta.

Honest reason for the sale is I just have too much gear. Running and managing multiple GPU generations side by side became an overhead than it's worth. So I am consolidating.

Still available if anyone local wants it.
 
S

SnJ9MX

Active Member
Jul 18, 2019
221
199
43
werewolf191 said:
Thanks for the pointer, appreciate it. I am actually already on 1Cat-vLLM with this box and agree, it gives the V100s a real second life for inference. NVFP4 on 4x 32GB is surprisingly usable.

My main objective is LoRA fine-tuning rather than serving though. I've been experimenting with a few things there, including some of the newer MoE LoRA approaches to see what learns well at FP16, since there is no native bf16 on Volta.

Honest reason for the sale is I just have too much gear. Running and managing multiple GPU generations side by side became an overhead than it's worth. So I am consolidating.

Still available if anyone local wants it.
Click to expand...
excellent, yes sorry I should've been more clear. I posted the link to the repo so others could see that these v100s still have life left in hopes they would be more inclined to purchase your server!
 
  • Like
Reactions: werewolf191
W

werewolf191

New Member
Feb 10, 2025
28
8
3
SnJ9MX said:
excellent, yes sorry I should've been more clear. I posted the link to the repo so others could see that these v100s still have life left in hopes they would be more inclined to purchase your server!
Click to expand...
No need to apologize at all, that was a genuinely helpful post and I appreciate you putting it in the thread. You're right that these V100s have plenty of life left, and the fork proves it. Hopefully it helps someone local see the value. I am also flexible on pricing, so anyone interested could feel free to PM me with an offer. Thanks again!
 
W

werewolf191

New Member
Feb 10, 2025
28
8
3
Added evidence that the 1029GQ-TVRT with 4× V100 32GB SXM2 is still a very capable inference server: 1Cat-vLLM (V100-optimized vLLM fork) serving Qwen3.8-27B NVFP4 with DFlash2 speculative decoding at TP4 sustains 210–235 tok/s single-request (224.6 median) and ~278 tok/s aggregate at 4 concurrent requests. Numbers are end-to-end including prefill, 512-token outputs. Full NVLink mesh between all four GPUs20261012_114400_1.jpg
 
  • Like
Reactions: TangTasteTester
You must log in or register to reply here.
Top Bottom