CWWK/Topton/... Nxxx quad NIC router

[cmmh]

I have the same device and a very similar setup (using PFSense instead of OpenSense). I started out on Proxmox 8.1 and was also experiencing hickups like yours, yet only twice in the last three weeks. Still, this is an issue I would desperately like to see resolved.

Not sure how to troubleshoot this problem. I never seen any errors in the host log, usually I don't have time to tinker and get into the guest OS and see if it has errors. I'm going to try to wire in a serial-port host connection so I can access the guest (OPNsense) on my gui-less console PC.

 

[void.devel]

Tzvia, thank you for your reply.

I just bought a 12v8a power supply, running memtest86 as we speak. Also, none of the power supplies (original or the new one) feels hot, both of them are just warm during memtest operation.
I also reapplied a generous amount of new thermal paste.

I've seen the forum thread you mentioned, I'm just too afraid to load a custom BIOS so it will be the last thing I'll do when all else fails. My current BIOS date is 11/24/2023 so its a relatively new version. However I might end up updating it since it provides extra functionality/options, we'll see ..

Funny thing is, yesterday right after my previous post, I decided to install esxi 8 on a usb stick, installed a couple of cheap 1tb m2, connected cable to ETH0 and moved there a bunch of VMs. No errors, at all. I tried updating all VMs, run a bunch of stuff, copied close to 1tb (the vmdks), did some network testing, no problems/errors.

If memtest finishes with no errors for 4 passes, I'll fire up esxi again and copy some of my 'production' VMs that I use on a daily basis and see what happens. I'll probably run a stress test on a couple of VMs, see how it goes ... however I'll need at least a couple of weeks of error free operation before I feel any kind of confidence ...

Could it be a problem with memtest itself ? I tried both memtest86+ v6 and memtest86 v10.6 (v6 returned way more errors than v10 for some reason). Also tried running both from usb stick and external usb cdrom, same results.

So my current state is : 32gb crucial ram, 2x 1tb cheap m2, new thermal paste, new power supply, connected only one ethernet cable to ETH0, bottom case cover screwed in place and running memtest86 v10.6. At the moment RAM temp is 72c/161f (room temp is 17c/62f), device feels slightly hot to the touch (nothing alarming), PSU feels barely warm, power consumption is at ~25W.

On a side note, this device is a little beast, I really hope I can get to work properly. ESXI 8 runs flawlessly (didnt even have to inject nic drivers), no CPU complaints, web ui feels very snappy and cpu power is more than I expected. All I need it for is to run pfsense, one windows 2022 vm (backup ad/dhcp/dns/etc) and three linux VMs but it feels it can do even more ... low power consumption, tiny size, 6x 2.5 nics, its perfect for my needs.

EDIT : 101 mins in and memtest started returning errors ... RAM temp is at 85c/185f ...

 

[askewpiste]

Tzvia, thank you for your reply.

I just bought a 12v8a power supply, running memtest86 as we speak. Also, none of the power supplies (original or the new one) feels hot, both of them are just warm during memtest operation.
I also reapplied a generous amount of new thermal paste.

I've seen the forum thread you mentioned, I'm just too afraid to load a custom BIOS so it will be the last thing I'll do when all else fails. My current BIOS date is 11/24/2023 so its a relatively new version. However I might end up updating it since it provides extra functionality/options, we'll see ..

Funny thing is, yesterday right after my previous post, I decided to install esxi 8 on a usb stick, installed a couple of cheap 1tb m2, connected cable to ETH0 and moved there a bunch of VMs. No errors, at all. I tried updating all VMs, run a bunch of stuff, copied close to 1tb (the vmdks), did some network testing, no problems/errors.

If memtest finishes with no errors for 4 passes, I'll fire up esxi again and copy some of my 'production' VMs that I use on a daily basis and see what happens. I'll probably run a stress test on a couple of VMs, see how it goes ... however I'll need at least a couple of weeks of error free operation before I feel any kind of confidence ...

Could it be a problem with memtest itself ? I tried both memtest86+ v6 and memtest86 v10.6 (v6 returned way more errors than v10 for some reason). Also tried running both from usb stick and external usb cdrom, same results.

So my current state is : 32gb crucial ram, 2x 1tb cheap m2, new thermal paste, new power supply, connected only one ethernet cable to ETH0, bottom case cover screwed in place and running memtest86 v10.6. At the moment RAM temp is 72c/161f (room temp is 17c/62f), device feels slightly hot to the touch (nothing alarming), PSU feels barely warm, power consumption is at ~25W.

On a side note, this device is a little beast, I really hope I can get to work properly. ESXI 8 runs flawlessly (didnt even have to inject nic drivers), no CPU complaints, web ui feels very snappy and cpu power is more than I expected. All I need it for is to run pfsense, one windows 2022 vm (backup ad/dhcp/dns/etc) and three linux VMs but it feels it can do even more ... low power consumption, tiny size, 6x 2.5 nics, its perfect for my needs.

EDIT : 101 mins in and memtest started returning errors ... RAM temp is at 85c/185f ...

Sorry you are still having issues. My unit sits about at 35-42C for the CPU while under low load running opnsense. When I did my memory test with the Kingston stick my temps only got to about 62C. If you were to keep the RAM cooler, would it pass the test?

 

[fta]

I have the Pentium 8505 version,

Then they provided only a 5a power supply.

For the 8505, a quality 60W supply is enough. I use a 60W with my 8505 with no issues. When you step up to the 1235U you need a 90W. The 60W they supply with the unit, though, may or may not be enough. Even if it's a "name brand", a lot of times the guts of it are swapped with junk. My opinion is to never trust the power supplies they ship with these boxes.

 

[askewpiste]

For the 8505, a quality 60W supply is enough. I use a 60W with my 8505 with no issues. When you step up to the 1235U you need a 90W. The 60W they supply with the unit, though, may or may not be enough. Even if it's a "name brand", a lot of times the guts of it are swapped with junk. My opinion is to never trust the power supplies they ship with these boxes.

On that note I think I will order a replacement.

 

[void.devel]

Hi askewpiste,

I reapplied the thermal paste, removed the m2s, removed the bottom and side covers, removed the ethernet cable and rerun the test with a desktop fan blowing on top of it. But even if that works it would be an unacceptable solution, an almost naked device that gets overheated in 16c/60f room temperature would be completely useless.

A previous post suggested trying out a custom BIOS, however it is not for my exact device and could brick it I guess. The best I can find in CWWKs support site is 先锋版N100-N200-i3-N305-BIOS.iso which I *assume* covers my device, it is however older than my current version.

I've contacted CWWK but do not really expect any kind of help from them (I knew that before buying the device).

This is getting really annoying, I'm down ~600 euros so far (device, ram, couple cheap m2, heatsinks, thermal paste x2, new power supply) and even more on other equipment I was supposed to attach to it (mainly a couple of 2.5/10gbps switches and a custom nas).

I found a 32GB version of the RAM you mentioned earlier but that is another 130 euros, I'd buy it if I was sure it would work ... but the device 'feels' extremely unstable and I am unable to accurately pinpoint the source of the problem ... sometimes it takes an hour for memtest to return an error, other times I get 10+ errors in the first 10 minutes (all that without changing anything AT ALL, not even moving the device an inch, room temperature is pretty stable during the day).

I should just admit that I made a mistake purchasing this and move on but damnit its easy to get obsessed trying to figure this out

 

[askewpiste]

I've asked on chat if they can provide an unlocked BIOS, I'm not even sure which is the correct one for the n100 6 lan version.

I've sent my Crucial RAM back however I am not sure if I am likely to see a refund and if they will accept it. It could be that the RAM is actually defective? Hard to know unless you have some spare sticks laying around to test and some other known good devices.

 

[chrisro]

I bought from amazon a kingnowy minipc - the 2 Port i226-V 2.5G LAN Fanless Mini PC Intel N305 barebone and it overheats with a Samsung 990 PRO and 1 x Crucial 48 GB SODIMM. The memory sensor jumped to 125 degrees Celsius in the latest memtest after only about 10 minutes (although there were no errors, I stopped the test). The case overheats pretty bad, it's probably the first mini pc on which you can cook an omelette or fried eggs
The mini pc has the following dimensions 162 mm x 100 mm x 36 mm ( 6.378 inches x 3.937 inches x 1.417 inches ) with metal or aluminium case.
What cooler do I have to put in order to have the memory run at most 65 - 70 degrees Celsius under sustained load ? I am planning to remove the bottom of the minipc and put a cooler (Noctua or other) instead. What would be the best cooling solution for this minipc?

 

[zeroflow]

Hi,

I've got my N100 unit a few weeks ago and have been battling with repeating issues where all interfaces would stop responding BUT seem to keep some existing connections alive. A reboot fixes the issue at first, but after some time (13h, 12h, 62h) connection is lost again.
I've looked into existing issues, but other than the issues with EEE (guru3d.com) I did not find any more reports.

Hardware

• CWWK 12TH GENERATION N SERIES 8-CORE NEW MEMBER AFFORDABLE VERSION
• 8 GB Crucial RAM (CT8G52C42S5)
• 128 GB Patriot P300 SSD
• pfSense CE 2.7.2-RELEASE

My setup consists of two pfsense boxes in two locations:

• J4125 Topton at "Main" - 192.168.128.0/24, Modem 192.168.100.1
• N100 (this one) at "House" - 192.168.4.0/24, Modem 192.168.8.1
• Linked by wireguard tunnel: 172.16.16.1 for Main, 172.16.16.10 for House. Main has a public IP

From what I was able to diagnose:

• LAN side (192.168.4.1) gets unreachable (WebUI, Ping) and stops routing data to the internet
• pfSense can also not ping devices on LAN like AP on 192.168.4.5
• WAN also reports connection loss (Gateway Monitor: Modem IP 192.168.8.1)
• Wireguard Tunnel still established
  • WebUI accessible, but started to degrade (Jan 15 08:27:25 in log ) until no longer accessible (Jan 15 09:57:28)
  • Main Wireguard still kept showing successful handshakes until unit was physically turned off (Jan 15 19:46:27)

I've uploaded the system log from the last issue, the connection loss started at Jan 14 23:15:08 where my monitoring could no longer ping 8.8.8.8
Gateway monitoring actions for WAN were deactivated because I suspected pfSense was killing WAN because monitoring stopped working.

pfSense System Log

pfSense System Log. GitHub Gist: instantly share code, notes, and snippets.

gist.github.com

At the moment, it has been removed from active duty and moved into a lab setup to investigate this issue.
Due to the long times before issues occur, this is rather tedious to debug. So I'm looking for any ideas / help.

Steps / Investigations I've identified so far:
* Reproduce in lab (currently running)
* Disable hardware checksum offload
* Reset BIOS, Reinstall pfSense from scratch

Thanks in advance for your ideas!

 

[devast]

it overheats with a Samsung 990 PRO and 1 x Crucial 48 GB SODIMM

I don't know where you got the bright idea, to insert a high end, high power consumption nvme ssd into a mini chassis of a passive device, that's already on the brink of overheating without any ssd. The first thing would be, to remove that ssd from there, and insert something that's very low power. My x86-p5 came with an external 5v usb fan and a bracket included in the package. If you did not receive that, try getting one from the seller. With that fan, and a low power ssd, it should not have any overheating issues.

 

[zeroflow]

I have the CWWK N100 version with 4 ports. I'm passing my 2.5Gbit NICs through to a Proxmox VM running OPNsense. I've developed a problem where the passed-through ports don't work properly after 48 hours or so. Always happens at a low-traffic time at night. Rebooting the OPNsense VM doesn't fix the problem; the host has to be rebooted too. I'm using a usb-dongle to provide the proxmox host ethernet, and that is always up and online.

I've reached out to CWWK to see if there is a later bios revision, but they sent the bios revision I already have. This problem seemed to develop sometime after upgrading to Proxmox 8.1. Just throwing this out there for anyone who might have any insight or observed something related.

I can only add that I also have this problem as posted above. Did you get any other feedback from CWWK?

 

[dogderecho]

I was thinking about throwing a Coral TPU in there. As far as i understand, I could use the M-Key variant in any of the 5 nvme slots - or the E-Key one (see specs in image) - which has 2 TPUs - directly in the wifi slot after removing the nvme adapter.
But coral has this asteriks on its page and now I don't know if it will work. I found out that the wifi slot is 1 of the 9 Gen3 pcie lanes - while the TPU speccs specifically request 2 Gen2 lanes.
So, considering their disclaimer and "most manufacturers" I'm probably out of luck here?

View attachment 33546

So I've got one of the Coral Ekey PCIe devices and it's detected by Proxmox and works as expected with Unraid.
That being said, I've spent the last 2 weeks pulling my hair out trying to get Proxmox to pass it to a VM, and I'm facing the exact same issue as

@Esports_are_for_losers @nense

You can share your proxmox or BIOS configurations. When trying to pass PCI to VM I have an error
can't change power state from D3cold to D0

@mack Did you ever find a solution for this? I've update my grub.cfg to
GRUB_CMDLINE_LINUX_DEFAULT="quiet pcie_port_pm=off intel_iommu=on iommu=pt vfio_iommu_type1.allow_unsafe_interrupts=1 kvm.ignore_msrs=1 pcie_aspm=off pci=noaer"
and blacklisted the drivers and regardless what I've done, when I try to start the VM, the log reports:
Proxmox kernel: vfio-pci 0000:08:00.0: not ready 1023ms after bus reset; waiting multiple times, and then finally Proxmox kernel: vfio-pci 0000:08:00.0: Unable to change power state from D3cold to D0, device inaccessible

If anyone has any ideas, that would be appreciated

 

[Stovar]

For the 8505, a quality 60W supply is enough. I use a 60W with my 8505 with no issues. When you step up to the 1235U you need a 90W. The 60W they supply with the unit, though, may or may not be enough. Even if it's a "name brand", a lot of times the guts of it are swapped with junk. My opinion is to never trust the power supplies they ship with these boxes.

On that note I think I will order a replacement.

Which power supply would you recommend for these mini routers (N100+), any links?

 

[Stovar]

Hi askewpiste,

I reapplied the thermal paste, removed the m2s, removed the bottom and side covers, removed the ethernet cable and rerun the test with a desktop fan blowing on top of it. But even if that works it would be an unacceptable solution, an almost naked device that gets overheated in 16c/60f room temperature would be completely useless.

A previous post suggested trying out a custom BIOS, however it is not for my exact device and could brick it I guess. The best I can find in CWWKs support site is 先锋版N100-N200-i3-N305-BIOS.iso which I *assume* covers my device, it is however older than my current version.

I've contacted CWWK but do not really expect any kind of help from them (I knew that before buying the device).

This is getting really annoying, I'm down ~600 euros so far (device, ram, couple cheap m2, heatsinks, thermal paste x2, new power supply) and even more on other equipment I was supposed to attach to it (mainly a couple of 2.5/10gbps switches and a custom nas).

I found a 32GB version of the RAM you mentioned earlier but that is another 130 euros, I'd buy it if I was sure it would work ... but the device 'feels' extremely unstable and I am unable to accurately pinpoint the source of the problem ... sometimes it takes an hour for memtest to return an error, other times I get 10+ errors in the first 10 minutes (all that without changing anything AT ALL, not even moving the device an inch, room temperature is pretty stable during the day).

I should just admit that I made a mistake purchasing this and move on but damnit its easy to get obsessed trying to figure this out

Apart from the good advice from devast, finding the replacing the hot components is always the real fix imo, but a noctua or similar 120mm case fan could be used on just one side of the mini pc (top part with nvme, ram etc) as long as there are vents so cool air can be pushed over those components it helps out a great deal.

Rather then trying to go excessive with new hardware and coolers, try just a single 120mm fan first. You could even get 2 x 120mm case fans and strap them on top and bottom (with cable ties even) and push any hot air away from the unit and test temps and stability.

Last thing you want to do is waste more money and time, these mini pcs are fantastic but sometimes there can be problems as you can read in the last several pages, some have gone back to good old Lenova mini pcs or desktop mini itx mainboards and separate pci cards etc so it can be a trial and error situation.

I always find cwwk the best for support and quality goods, but sure you do have to find something that works best for yourself.

 

[Dostring]

I can only add that I also have this problem as posted above. Did you get any other feedback from CWWK?

On the other hand, I have the same device and setup (two NICs passed through to OPNsense and one shared for proxmox and VMs) and my current uptime is 15 days. I've rebooted proxmox before that because of BIOS tinkering that went nowhere. Can't help you guys but just saying that it works for me.

 

[askewpiste]

Which power supply would you recommend for these mini routers (N100+), any links?

This is the one fta recommended. Haven't purchased yet but is likely the one I will order - just seeing if there is anything else worth adding to get the free shipping! https://www.digikey.co.uk/en/products/detail/globtek-inc/TR9CE5000LCP-N-R6B/8598040

 

[zeroflow]

I've rebooted proxmox before that because of BIOS tinkering that went nowhere. Can't help you guys but just saying that it works for me.

Could you elaborate? I think I also went trough the whole BIOS trying to enable power saving features. What did you have to revert?

 

[nva]

Does anyone have problem with these unit crashing with Proxmox. It runs Opnsense bare metal without problem but it appears to have problem with Proxmox with minimal work load.

 

[zeroflow]

Does anyone have problem with these unit crashing with Proxmox. It runs Opnsense bare metal without problem but it appears to have problem with Proxmox with minimal work load.

cmmh, OPNsense on Proxmox: https://forums.servethehome.com/ind...xxx-quad-nic-router.39685/page-81#post-407685
me, pfSense native: https://forums.servethehome.com/ind...xxx-quad-nic-router.39685/page-82#post-407853

So yeah, there seems to be an issue with idling.
In both cmmh and my case, the unit does not crash but rather network stops working.

 

[romx]

What 4-pin fan connector does the motherboard use? I'd like to use my own fan but suspect I need to add a new connector to it.

In my BK-1264NP-4L-5G is a JST 1.25 4-pin connector (marked as SYSFAN1), compatible with standard pwm fan pinout (GND-12V-TACH-PWM)