Qotom Denverton fanless system with 4 SFP+

[Caennanu]

So...
Today i was tinkering to try and replace the edgerouter with the qotom.
During troubleshooting off my PfSense install on why internet wasn't working. I ran into an issue.
It seems that when i reboot proxmox / pfsense and there's an SFP in the cage, the interface fails to show up entirely, pfsense fails to boot and hangs with an interface alteration and if i want to configure vlans etc...
Also notice that the cage LED's don't even light up anymore.

Is this a proxmox thing? was working earlier

---
So i booted straight into PfSense (yay iommu passthrough), the exact same issue occurs.
When i populate a cage, the interface disappears completely on reboot.
Defective cages?

 

[Caennanu]

I bought the Qotom Q20331G9 1U Rackmount version. It shipped in 2 days from China which is insane. I installed PfSense on it and it was running like butter for about 2 hours. Then it died. Anyone else have problems or did I just get a dud? I've been working with the seller through Amazon but they keep asking me what seem to be irrelevant questions. Also the communication is delayed as most of their business hours are when I'm sleeping. I'm hoping they will send me a new board to cut down on shipping but we'll see.

Did it die completely, or partially?
Cause i seem to have ended up with dead SFP cages. (even tho i've only used <1.2w rated sfp's)

 

[R3Z3N]

You can install proxmox via the console however unless ypu already have a network, installing/setting up pfsense will be harder as you can't use the hardware console for vms/lxc. However you can setup another computer to directly connect via eth to you'd proxmox bridge like one would do via cossover eth.

Hopefully you also have a managed switch w console port....

 

[Caennanu]

You can install proxmox via the console however unless ypu already have a network, installing/setting up pfsense will be harder as you can't use the hardware console for vms/lxc. However you can setup another computer to directly connect via eth to you'd proxmox bridge like one would do via cossover eth.

Hopefully you also have a managed switch w console port....

If that is in reply to me. Installing PfSense / Proxmox isn't an issue (at this moment), in that regard its all about configuration that i'm lacking some knowledge with.

The thing i'm experiencing now, is that the SFP interfaces completely vanish when i boot with an SFP in the cage, and when i boot without an SFP in the cage, they won't come online when i insert them while pfsense is running, status leds don't do anything, while they did before.

--- edit

When putting in an old multimode duplex SFP, that one does show the laser giving signal and doesn't fail the interface at boot. (unfortunately have no way to test this....). Could these be a problem? https://www.fs.com/de-en/products/11802.html (mine are configured for ubiquiti btw)

--- edit

So... considering the other conclusion, i decided to steal a DAC cable from my server. And whaddayaknowit, the interfaces come up. da f....?

 

[sko]

mine are configured for ubiquiti btw

Use transceivers for intel. Intel has always been picky about what transceivers they support.

Also I'm not sure if the Atom integrated NICs actually support other than rated (10G) speeds. If they do, maybe you also have to set/force the interface speed via sysctl or ifconfig.

 

[Caennanu]

Use transceivers for intel. Intel has always been picky about what transceivers they support.

Also I'm not sure if the Atom integrated NICs actually support other than rated (10G) speeds. If they do, maybe you also have to set/force the interface speed via sysctl or ifconfig.

So... don't use FS ones? cause the FS DAC cable works, (no idea at what speeds tho, not been able to test yet)

And got a suggestion for intel ones, with the linked specs? 1000base 20km bidi lc/upc 1310tx with 1490rx?

 

[sko]

So... don't use FS ones?

FS also offers them encoded for intel

 

[Caennanu]

FS also offers them encoded for intel

well yes, just thought it had to be non fs.
ordered! :d (wish i knew someone with a firmware box, to just flash the 5 i have...)

 

[sko]

wish i knew someone with a firmware box, to just flash the 5 i have...

FS offers to reflash their transceivers. Just write your sales rep or their general support. However, for small quantities of transceivers the shipping costs might add up to more than just ordering new ones...

 

[Caennanu]

FS offers to reflash their transceivers. Just write your sales rep or their general support. However, for small quantities of transceivers the shipping costs might add up to more than just ordering new ones...

haha yeah. 20 euro to get a new one shipped. shipping back and forth my 5 is gonna cost the same.
And why i have 5, i can't remember, i only needed 1..

 

[Arekx]

Could anyone describe how bottom fan works/behaves in Q20331G9 1U system?

When machine is powered on does fan start immediately and runs with constant speed that doesn't depend on system temperature? Or maybe it behaves differently?

 

[Caennanu]

Could anyone describe how bottom fan works/behaves in Q20331G9 1U system?

My experience so far. Its an always on fan but also has a curve. The range off that curve is t that big tho. So its not disturbing or anything and i had it on my desk for a couple of days.

I would compare it with an elite desk mini at 30% fan speed

 

[krakonos]

Hi!

I'm looking into a new router solution and this ticks all the boxes. However, I wonder how do the models compare with IDLE power consumption? I saw some numbers around 20W+. Is that accurate? Specifically, I'd be interested if there is a difference in the models Q20311G9/Q20321G9/Q20332G9, since they differ mostly in the SOC.

There is little chance I will be able to get anything over 2Gbit/s internet connectivity, so in theory the 2.5G would be good enough - I would opt for the slowest one Q20311G9 if the idle power would be significantly better. Otherwise, I think I'd go for the Q20332G9 to have the 10Gbit SFPs and a bit more power in case I want to repurpose it later on. I already have a separate home server solution that I don't want to mix into the router for reliability and ability to tinker with it a bit more, while keeping the connectivity going.

 

[Zaf9670]

Hi!

I'm looking into a new router solution and this ticks all the boxes. However, I wonder how do the models compare with IDLE power consumption? I saw some numbers around 20W+. Is that accurate? Specifically, I'd be interested if there is a difference in the models Q20311G9/Q20321G9/Q20332G9, since they differ mostly in the SOC.

There is little chance I will be able to get anything over 2Gbit/s internet connectivity, so in theory the 2.5G would be good enough - I would opt for the slowest one Q20311G9 if the idle power would be significantly better. Otherwise, I think I'd go for the Q20332G9 to have the 10Gbit SFPs and a bit more power in case I want to repurpose it later on. I already have a separate home server solution that I don't want to mix into the router for reliability and ability to tinker with it a bit more, while keeping the connectivity going.

I'm not sure which is the lowest end model they included without looking but Patrick had a slide on his C3000 Refresh article here.

The lowest end R model added QAT which "should" work in PFSense (and maybe OPNSense? not 100% sure there). But the R did have a small TDP increase. However TDP does not equal power draw so I can't speak to how much efficiency is lost/gained with the included AC adapter. I just got mine but I'm waiting on the RAM to get it kicked off.

 

[blunden]

I'm not sure which is the lowest end model they included without looking but Patrick had a slide on his C3000 Refresh article here.

The lowest end R model added QAT which "should" work in PFSense (and maybe OPNSense? not 100% sure there). But the R did have a small TDP increase. However TDP does not equal power draw so I can't speak to how much efficiency is lost/gained with the included AC adapter. I just got mine but I'm waiting on the RAM to get it kicked off.

QAT on the Atom C3000 series isn't necessarily very useful since its cipher support is limited. It depends on your use case, but one should not assume that it will be used at all. For users of IPSec and OpenVPN with a supported algorithm it can be useful though.

QAT is a paid feature on pfSense and free on OPNsense as far as I know, but I haven't checked since mine is running VyOS.

EDIT: The non-R models have QAT too. There are almost no differences between them except for the extra 200 MHz.

 

[Hard2Reach]

I'm not sure which is the lowest end model they included without looking but Patrick had a slide on his C3000 Refresh article here.

The lowest end R model added QAT which "should" work in PFSense (and maybe OPNSense? not 100% sure there). But the R did have a small TDP increase. However TDP does not equal power draw so I can't speak to how much efficiency is lost/gained with the included AC adapter. I just got mine but I'm waiting on the RAM to get it kicked off.

Qotom with C3758Rs are now quite hard to find since Patrick's review drove the demand more than the supply. I too got lucked out finding one since I delayed purchasing one when it was available.

 

[R3Z3N]

^There are a few 5019A Supermicro builds with the C3758 on ebay right now...I have one I am debating on selling as I have been moving from C3000 to D1540/1s or D1700. It's

 

[Zaf9670]

QAT on the Atom C3000 series isn't necessarily very useful since its cipher support is limited. It depends on your use case, but one should not assume that it will be used at all. For users of IPSec and OpenVPN with a supported algorithm it can be useful though.

QAT is a paid feature on pfSense and free on OPNsense as far as I know, but I haven't checked since mine is running VyOS.

I forgot about the Plus drama saga. I may still purchase it though. I believe OPNSense is having issues porting to the latest FreeBSD so they're slipping behind on security (not that I'm a huge target but still).

Qotom with C3758Rs are now quite hard to find since Patrick's review drove the demand more than the supply. I too got lucked out finding one since I delayed purchasing one when it was available.

Yeah I realized I didn't get the R model off Amazon just recently but probably not a huge deal. The 200Mhz won't hurt me in the long run I'm sure.

 

[krakonos]

I'm not sure which is the lowest end model they included without looking but Patrick had a slide on his C3000 Refresh article here.

The lowest end R model added QAT which "should" work in PFSense (and maybe OPNSense? not 100% sure there). But the R did have a small TDP increase. However TDP does not equal power draw so I can't speak to how much efficiency is lost/gained with the included AC adapter. I just got mine but I'm waiting on the RAM to get it kicked off.

Well, in my experience TDP doesn't matter for IDLE power draw too much or at all. It comes down to how power management is done on the board/in BIOS and if there are extra peripherals. Ie. how much do the extra features draw when unused (10Gbit, SAS, maybe something else?). I'd love if I could get to around 10W IDLE or so, which can be done with my current homelab setup with Lenovo M700 (but lacks the faster networking and rackmount - which is why I'm not running it as a router too).

There also is a variant with just the 2.5Gbit NICs and J4125. I'm tempted to buy all and give it a try, but with shipping and import taxes it's starting to be a rather expensive experiment.

 

[blunden]

Qotom with C3758Rs are now quite hard to find since Patrick's review drove the demand more than the supply. I too got lucked out finding one since I delayed purchasing one when it was available.

Before me and others started contacting Qotom about the 3758R, they had basically stopped producing them since Qotom didn't think the price difference between the C3758 and C3758R made sense. Apparently they could source the C3758 quite a bit cheaper.

That's what they told me at least. That's probably still true. They might make a batch of them once in while to satisfy demand if they deem it to be large enough.

I forgot about the Plus drama saga. I may still purchase it though. I believe OPNSense is having issues porting to the latest FreeBSD so they're slipping behind on security (not that I'm a huge target but still).

That's a bit suprising since OPNsense used to be faster at porting to newer FreeBSD versions, especially for the Community Edition releases. What you say appears to be true though, based on some quick googling.

Yeah I realized I didn't get the R model off Amazon just recently but probably not a huge deal. The 200Mhz won't hurt me in the long run I'm sure.

Besides 200 MHz extra clock speed, they are basically identical.