CWWK/Topton/... Nxxx quad NIC router

[Patrick]

So 6 Sata 3.0 2.5" drives kind of hints at a NAS system with 1 NVMe rated at x1 only, not at x4.

We have the new CWWK N100 NAS motherboard as well. I think that is more of their NAS solution:

• N100
• 6 SATA ports
• 2x M.2
• 4x 2.5GbE
• 2x Internal USB
• mITX form factor

There are still some quirks with that one like memory training on the first boot takes MINUTES, but from a hardware perspective, it is better than I would have thought. We had the Demonboard and the J6413 version of the NAS motherboard, and I think they are going in the right direction with these.

We are just super backed up. I sent Jeff Geerling a note from the studio last night saying just on the video side, we have four new switches, four new mini PCs, three servers, and four motherboards/ CPUs, all of which I want to do videos on that are already in progress. We also are doing the Intel Emerald Rapids piece for later this week.

Realistically, the Scottsdale studio is about 1.5 months behind schedule, so we are behind in content production.

We also need to hire 1-3 folks to help get content out.

 

[SBMe]

.... We also need to hire 1-3 folks to help get content out.

Thanks for all the info. Sounds like your correct. You all are just a little backed up
Saved that NAS-N6005 board for the future.

 

[thepsyborg]

Some incorrect links @Patrick

Mod Edit: Got it.

 

[ToniCipriani]

Are there any Proxmox users here who have CW-ADLN-6L six-port N305? I installed Kingston FURY Impact KF556S40IB-32 32GB memory and two Samsung 990 Pro NVMes. I did full memory tests twice - all without errors. N305 is cold when idle (~26-34 C). BIOS 5.27 08/23/2023

FreeBSD 14 works great - no complaints at all.
Windows 11 works great - no complaints at all.
VMware ESXi 8.0u2 works great - no complaints at all.

But I'm having problems with all versions of Proxmox that I've tested. With all versions, spontaneous freezes occur either on the host OS in Proxmox, or bugs and freezes in guest systems (Ubuntu 23.10 and 22.04.03 LTS). I'm using ZFS RAID1 mode, ashift 12.

I've tried several different versions:
Proxmox 8.0.3 with 6.2.16-3 kernel
Proxmox 8.0.3 with 6.2.16-12 kernel
Proxmox 8.1.3 with 6.5.11-4 kernel
Proxmox 8.1.3 with 6.5.11-6 kernel.
And others. Latest version hangs up just without VMs/containers at all, just fresh install. Tried to disable memory ballooning, KSM, tried updating Intel's microcode. No success at all.

Some host system errors:
proxmox kernel: BUG: unable to handle page fault for address: ffff985b1221e5c3
proxmox kernel: BUG: unable to handle page fault for address: ffffff5db9a6daba
proxmox kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 1595s! [kworker/0:2:571]
proxmox kernel: WARNING: CPU: 7 PID: 992 at arch/x86/kernel/fpu/xstate.c:1465 xfd_validate_state+0x62/0x80

Guest systems sometimes freeze and display various errors, sometimes simply do not respond without errors, including I cannot turn them off.

I also tried connecting a laboratory power supply instead of the standard one - without results.

Everything can work for several hours without failures, or it can freeze several times in one hour.

upd: this bug happened while writing this post, just clean system without VMs at all.

Spoiler: Screenshot

View attachment 33000

I have it, with Proxmox 8 running my home network with an OPNSense VM. Runs great, no hiccups or anything, been using it for months.

 

[MBehr]

Are there any Proxmox users here who have CW-ADLN-6L six-port N305? I installed Kingston FURY Impact KF556S40IB-32 32GB memory and two Samsung 990 Pro NVMes. I did full memory tests twice - all without errors. N305 is cold when idle (~26-34 C). BIOS 5.27 08/23/202

I have a CWI3N3056L23080031 no other label on MB, BIOS shows:

CW-ADLN-6L Ver: 11.1
08/23/2023 22:22:03

Running proxmox fully updated (started with 7.02 install, then updated kernel, and finally did the latest update to 8.1.3 running IOMM and VT-d so I can pass through the NICs to pfsense+ and pi-hole. No problems so far...I haven't made any major changes in a while, and it's been pretty stable - tho I do get and intermittent (twice so far) problem where traffic is not passed through (via rulesets), from external sources....so still investigating and collecting logs. Otherwise no dead in the water problems or longterm stability.,

 

[greenkiwi]

@MBehr - Did you have to start with 7.02? Or could you just have installed starting with 8.1.3?

Just looking to set this up and didn't know if I needed to start with 7 and update.

 

[Dostring]

Sound exactly like my setup and upcoming usecase for the N100

You could try to play with "powertop" a little:
Here is a good explanation how to lower power consumption when running Unraid on bare metal - but I'd say you could also try this for Proxmox.
Maybe the OPNsense VM prevents the system from getting in one of the lower C-states and therefor the systems draws more.

I played a little w/ powertop. After applying --auto-tune it bricked the whole thing and I needed a hard reset. Which seemed to restore the "bad" settings I had. Anyway, I've returned to the Becks post about his final setup and was surprised that most of the settings in OPNsense are missing Power governor in proxmox as well. I've made some progress nonetheless by turning on Power Savings in OPNsense. That got me from 12 W to 9W idle (while proxmox alone was 8W). There are some cpu spikes while the router is doing nothing still but it is a progress.

 

[nitrosont]

Yesterday my N100 4-port arrived.
The package content is more or less equivalent to the one from Mercantides

I just got red USB ports - I guess CWWK ran out of blue ones

Havent tested anything yet but I'm satisfied with the build quality. Also it seems there is no gap between the copper block and the heatsink/case. At least nothing visible. Idk if that always the case, but mine also has a aluminium block to transfer the heat from the four NICs to the heatsink/case.

The M.2-expander-card is screwed in at all four corners to the MB - seems pretty solid.

 

[nitrosont]

I played a little w/ powertop. After applying --auto-tune it bricked the whole thing and I needed a hard reset. Which seemed to restore the "bad" settings I had. Anyway, I've returned to the Becks post about his final setup and was surprised that most of the settings in OPNsense are missing Power governor in proxmox as well. I've made some progress nonetheless by turning on Power Savings in OPNsense. That got me from 12 W to 9W idle (while proxmox alone was 8W). There are some cpu spikes while the router is doing nothing still but it is a progress.

Oh, sorry to hear that you unit crashed with those settings!
But cool, that the idle went down! Keep us updated, please!

I'll start tweaking with my unit in the upcoming days as well.

 

[nitrosont]

By the way: did someone find out, what the little switch labeled "ATX1" does?

 

[spongi]

Are there any Proxmox users here who have CW-ADLN-6L six-port N305? I installed Kingston FURY Impact KF556S40IB-32 32GB memory and two Samsung 990 Pro NVMes. I did full memory tests twice - all without errors. N305 is cold when idle (~26-34 C). BIOS 5.27 08/23/2023

FreeBSD 14 works great - no complaints at all.
Windows 11 works great - no complaints at all.
VMware ESXi 8.0u2 works great - no complaints at all.

But I'm having problems with all versions of Proxmox that I've tested. With all versions, spontaneous freezes occur either on the host OS in Proxmox, or bugs and freezes in guest systems (Ubuntu 23.10 and 22.04.03 LTS). I'm using ZFS RAID1 mode, ashift 12.

I've tried several different versions:
Proxmox 8.0.3 with 6.2.16-3 kernel
Proxmox 8.0.3 with 6.2.16-12 kernel
Proxmox 8.1.3 with 6.5.11-4 kernel
Proxmox 8.1.3 with 6.5.11-6 kernel.
And others. Latest version hangs up just without VMs/containers at all, just fresh install. Tried to disable memory ballooning, KSM, tried updating Intel's microcode. No success at all.

Some host system errors:
proxmox kernel: BUG: unable to handle page fault for address: ffff985b1221e5c3
proxmox kernel: BUG: unable to handle page fault for address: ffffff5db9a6daba
proxmox kernel: watchdog: BUG: soft lockup - CPU#0 stuck for 1595s! [kworker/0:2:571]
proxmox kernel: WARNING: CPU: 7 PID: 992 at arch/x86/kernel/fpu/xstate.c:1465 xfd_validate_state+0x62/0x80

Guest systems sometimes freeze and display various errors, sometimes simply do not respond without errors, including I cannot turn them off.

I also tried connecting a laboratory power supply instead of the standard one - without results.

Everything can work for several hours without failures, or it can freeze several times in one hour.

upd: this bug happened while writing this post, just clean system without VMs at all.

Spoiler: Screenshot

View attachment 33000

I had the same problem (page fault) a few months ago and my solution was to use an usb-c ssd instead of the usb-3 hdd (withouth external power supply) for making the backups (error always happened during backups). But I also opened a thread in the proxmox forum and there I got a reply just yesterday, mentioning that the user found out, that on his unit, the on-die RAM ECC chip (which seems to be mandatory for DDR5) was disabled and since enabling it, the problem didn't occur again.

 

[nitrosont]

I had the same problem (page fault) a few months ago and my solution was to use an usb-c ssd instead of the usb-3 hdd (withouth external power supply) for making the backups (error always happened during backups). But I also opened a thread in the proxmox forum and there I got a reply just yesterday, mentioning that the user found out, that on his unit, the on-die RAM ECC chip (which seems to be mandatory for DDR5) was disabled and since enabling it, the problem didn't occur again.

Could you link to the thread in the Proxmox forum?
How do we enable the on-die RAM ECC chip?`

 

[spongi]

Could you link to the thread in the Proxmox forum?
How do we enable the on-die RAM ECC chip?`

somewhere in the uefi settings

pve freezes during backup job

Hello everyone! I'm using one of these CWWK/Topton/... Nxxx quad NIC router devices as my pve host. Two days ago I have migrated from a unit with the N100 to the exact same unit only with the N305 CPU. This means that I have just swapped out the two NVME drives, SATA OS drive and RAM. My daily...

forum.proxmox.com

 

[kliguin]

Regarding PVE crashed I got some strange issue, woke up without internet because my OPNsense router is virtualized. N305 HUNSN V2. The NFS storage is as designed, the NAS switches off at night.

How is the key pressed when everybody sleeps ;-) One thing I can think off is the Leicke powersupply not holding up during the backup...its 60watts 1:1 match with the original one.



Dec 12 04:05:14 pve pvestatd[2773687]: storage 'backup-remote' is not online
Dec 12 04:05:17 pve pvestatd[2773687]: storage 'NFS-HDD' is not online
Dec 12 04:05:17 pve pvestatd[2773687]: status update time (5.228 seconds)
Dec 12 04:05:18 pve kernel: nfs: server 172.16.1.2 not responding, timed out
Dec 12 04:05:23 pve kernel: nfs: server 172.16.1.2 not responding, timed out
Dec 12 04:05:26 pve pvestatd[2773687]: storage 'NFS-HDD' is not online
Dec 12 04:05:29 pve pvestatd[2773687]: storage 'backup-remote' is not online
Dec 12 04:05:30 pve pvestatd[2773687]: status update time (7.296 seconds)
Dec 12 04:05:34 pve kernel: nfs: server 172.16.1.2 not responding, timed out
Dec 12 04:05:39 pve pvestatd[2773687]: storage 'backup-remote' is not online
Dec 12 04:05:41 pve systemd-logind[3981267]: Power key pressed short.
Dec 12 04:05:41 pve systemd-logind[3981267]: Powering off...
Dec 12 04:05:41 pve systemd-logind[3981267]: System is powering down.
Dec 12 04:05:41 pve systemd[1]: 102.scope: Deactivated successfully.
Dec 12 04:05:41 pve systemd[1]: Stopped 102.scope.
Dec 12 04:05:41 pve systemd[1]: 102.scope: Consumed 2h 4min 7.542s CPU time.
Dec 12 04:05:41 pve systemd[1]: 103.scope: Deactivated successfully.
Dec 12 04:05:41 pve systemd[1]: Stopped 103.scope.
Dec 12 04:05:41 pve systemd[1]: 103.scope: Consumed 16h 53min 55.535s CPU time.
Dec 12 04:05:41 pve systemd[1]: 105.scope: Deactivated successfully.
Dec 12 04:05:41 pve systemd[1]: Stopped 105.scope.
Dec 12 04:05:41 pve systemd[1]: 105.scope: Consumed 13h 59min 48.772s CPU time.
Dec 12 04:05:41 pve systemd[1]: 107.scope: Deactivated successfully.
Dec 12 04:05:41 pve systemd[1]: Stopped 107.scope.
Dec 12 04:05:41 pve systemd[1]: 107.scope: Consumed 3d 2h 54min 30.537s CPU time.
Dec 12 04:05:41 pve systemd[1]: 110.scope: Deactivated successfully.
Dec 12 04:05:41 pve systemd[1]: Stopped 110.scope.

 

[batagy]

Hi All,

I'm new to this forum. I purchased CW-X86-P5(N100) Mini PC from Aliexpress.
This model:

CWWK X86 P5 Super Mini Router Twelfth generation Intel N100 DDR5 4800MHz Firewall PC 2x i226-V 2.5G LAN Send two SATA cables

Parameters/Model Model: CW-X86-P5(N100)/CW-X86-P5(i3-N305) CPU: Intel N100/Intel i3-N305 Main frequency: 6M Cache, Up to 3.40GHz Cores: 4 (Threads: 4) Network card: Intel* i226-V Memory: SO-DIMM DDR5 Storage: M.2 NVMe 2242/2280 Graphics card: Intel UHD Graphics 24EU Display: HDMI2.0 4K @60Hz...

cwwk.net

I was not aware of this topic and hence didn't know that ordering is much cheaper directly from CWWK, but my purchase happened now, doesn't matter. Glad that I found this topic.

My default BIOS included with the PC was :
Core Version: 5.27
Build Date and Time: 07/19/2023
Version (at bottom): I didn't check unfortunately, but based on pigr8's post here, it was probably 2.22.1289.

Now, after I received the package, I asked the Ali seller to provide me latest BIOS. He sent me a file CW-ADLNT-1C2L(VER_F1).zip, which had file modification times 08/02/2023, so I thought it maybe newer then existing BIOS. The seller just said this is latest but didn't provide me which version it was .

I prepared the bios update with FPT based on guides, and I wanted to backup the existing BIOS first (command "fpt.efi -D MyOriginalBios.bin"). But, since I copied all the files from CW-ADLNT-1C2L(VER_F1).zip, it included also STARTUP.NSH, so that resulted that during bootup it started to flash the bios without my intervention. So it was not backed up.

And it turned out the BIOS that the seller sent is actually older than it was. Now I have:
Core Version: 5.27
Build Date and Time: 05/30/2023
Version (at bottom): 2.22.1287

Do you have the 2.22.1289 or any newer BIOS for CW-X86-P5(N100)?
Can these download from somewhere?

I found 畅网官方网盘 but that doesn't contain BIOS for CW-X86-P5.

Thanks in advance!
György

Hi,

Does anybody can share me the 2.22.1289 BIOS or any newer BIOS , for CW-X86-P5(N100)?

Thanks in advance!

 

[MBehr]

@MBehr - Did you have to start with 7.02? Or could you just have installed starting with 8.1.3?

Just looking to set this up and didn't know if I needed to start with 7 and update.

Started with 7.02 so the NICs were recognized = they've been updated in 8.x - so not necessary anymore.

 

[ToniCipriani]

@MBehr - Did you have to start with 7.02? Or could you just have installed starting with 8.1.3?

Just looking to set this up and didn't know if I needed to start with 7 and update.

On my 6 NIC unit I started straight up with 8.0.2 (I think), then updated to 8.0.3, didn't need to start with 7.

Haven't updated to 8.1.3 yet though.

 

[Maor]

I have narrowed down the problem to enabeling APSM option in the bios.
Be aware KC3000 does not support ASPM on CWWK v2 4port version. If you want the lowest power consumption in idle, get some other ssd.
I will be getting crucial p5 plus as a replacement.

The KC3000 is fine, it works well.

The problem is with the PCIe ports 2-4 that do not work with ASPM enabled. The PCIe#1 works with ASPM enabled on the4x M.2 board.
I hope CWWK can fix it with a bios update or new revision of the 4xM.2 board.

@tibbe @RubberChicken does your work when you enable ASPM for the SSDs?

 

[kliguin]

Anyone knows what this switch does?

 

[nitrosont]

Anyone knows what this switch does?
View attachment 33198

Exactly my question!