Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
I think i have exactly this one, and can sell for cheaper. I PMd you about if interested.
Eh, well, I have 10/40 cards already filling half a drawer, so buying might not be a thing in the near future...
That's interesting. I had an impression you wouldn't directly install that into t740 without an external PCIe ribbon extender. I bought such an extender from AliE, but it won't work with t740. It works with t620 plus though, so i am not sure. I have no clue why Mellanox ConnectX3 VPI isn't detected when using such an extender on t740, while it does with t620 plus.
If it's "safe" using it directly inside, i will test that, that would be ideal ofc. I was a bit concerned about the heat since i will be using both ports of the card with RoCE 40Gbit/s.
And there is still hope for my CEPH cluster plans!
If it's "safe" using it directly inside, i will test that, that would be ideal ofc. I was a bit concerned about the heat since i will be using both ports of the card with RoCE 40Gbit/s.
And there is still hope for my CEPH cluster plans!
lol i know what you mean... i have a box full of copper stuff, 1g, 10g, and sfp/+ too.
@fossxplorer how long is your extender? is it not detected at all or maybe you boot it up with module already inserted? maybe power draw issue? idk for how much these simple x16 extenders rated.
I'm curious about the data bus for this M.2A+E slot. If I add a third hard drive here;
1. Does it need to be NVME or Sata?
2. Will it be at full speed like the other drives, or partial speed?
Thanks!
It's PCIe 2.0x2. It won't work for NVMe, and if it's SATA you'll need a SATA board, and a way to inject power to the drive. And you'll need to figure out where you can physically fit one inside the chassis.
Bit of a thread necro, but I just wanted to let you know I have SR-IOV working just fine in an M920q running Proxmox, and an AOC-STGN-I2S (82599ES)
so…not a Ryzen embedded, not on an HP, and not really relevant to the topic at hand…? Thanks, I guess?Bit of a thread necro, but I just wanted to let you know I have SR-IOV working just fine in an M920q running Proxmox, and an AOC-STGN-I2S (82599ES)
Yeup @zer0sum , and that only had to do with creating VF NICs. Standard SR-IOV based passthrough works fine (i.e. you can pass through a device directly to a VM/LXC no issues). This is usually down the individual port level and generally is not limited by IOMMU groups (or at least I haven't hit that wall, yet).
The BIOS/BAR issue appears when you try to create a virtual functions to share a device with multiple hosts at the same time (i.e. create multiple VF NICs/PCIeoF initiators to use the card as a switch/exceed the OVS 25Gbps speed limit). I know at a spec level this also theorictically possible for a SAS controller, but I don't know if any vendors have implemented this in their drivers/cards.
@WANg have had a chance to order in one of the new Alder Lake TMM nodes (either Lenovo or HP if they have one)?
I haven't had a chance to order one of the Ryzen Lenovos to see if I could replicate your success with the HP on the Thinkcenter Tiny boxes.
The BIOS/BAR issue appears when you try to create a virtual functions to share a device with multiple hosts at the same time (i.e. create multiple VF NICs/PCIeoF initiators to use the card as a switch/exceed the OVS 25Gbps speed limit). I know at a spec level this also theorictically possible for a SAS controller, but I don't know if any vendors have implemented this in their drivers/cards.
@WANg have had a chance to order in one of the new Alder Lake TMM nodes (either Lenovo or HP if they have one)?
I haven't had a chance to order one of the Ryzen Lenovos to see if I could replicate your success with the HP on the Thinkcenter Tiny boxes.
Last edited:
I am talking about full VF's working with an M920q + 82599 + Proxmox
I believe it should work with a T740 as well, but I don't have one handy to test it unfortunately
I just so happen to own a Ryzen 2400G with a Gigabyte motherboard (a B450M-DS3H), I can enable ARI on that, it actually works, I tested with a Chelsio T520 and its unusually strict cxgb4 driver, and it let me create VFs with no problems. If I'm not mistaken, this was introduced in one of the more recent BIOS updates. So it's probably HP's BIOS not supporting ARI.
@zer0sum can you spit out the BIOS revision and the Intel driver version you're using, when you have a second? When I get back to the house I'll spit out mine as well.
I didn't even know you could create VFs on the 82599 generation, I thought is was only added in the 500 series generation. Even the PCIe Gen 2 doesn't matter that much since the I/O block isn't shared with anyone else, so the M.2 should still run at full speed.
I have a 500 series card as well sitting on my desk so I'm going to try that again and see what happens.
If it works then that means that the Lenovo Tiny series of boxes are pretty much on par with the HP ones as far as viable VM cluster nodes.
I didn't even know you could create VFs on the 82599 generation, I thought is was only added in the 500 series generation. Even the PCIe Gen 2 doesn't matter that much since the I/O block isn't shared with anyone else, so the M.2 should still run at full speed.
I have a 500 series card as well sitting on my desk so I'm going to try that again and see what happens.
If it works then that means that the Lenovo Tiny series of boxes are pretty much on par with the HP ones as far as viable VM cluster nodes.
Well, I don't have an Alder lake TMM, not really planning to get one since I barely have space for all of my tech toys. Maybe when I upgrade my Framework to an Alder Lake i5 I'll see if I can get ARI working using the m.2 A+E/MiniPCIe and my i350 dual port card.
Does anyone know what the small external port bracket is called? I'm trying to find one so I can install an external eSATA port. Is it something that is custom to this chasis?
Last edited:
The option port? Yeah, it's specific to the chassis and only come in limited options - blank, Serial port or external wifi/BT antenna connectors. it's roughly the same dimension as most M2 to MiniPCIe adapter cards, so you can totally do something like this to make it happen.
Yes, I just installed one of these and connected to my Promax JBOD box with no issues. I actually get slightly faster transfer rates than the initial PCIe card that I had installed.
IO CREST 2 Ports SATA III 6gb/s to M.2 NGFF A Key + E Key A+E Key 22x30 2230 mm JMicron JMB582
Bought this t740 unit to be a Plex server and then learned that AMD GPUs cannot reliably be passed through on Linux Plex...
So now I have the PCIe slot open for another GPU.
This has been a long journey, hope I'm not setting myself up for another roadblock with this approach!!