CWWK/Topton/... Nxxx quad NIC router

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Dostring

New Member
Dec 1, 2023
23
9
3
wg easy is the way to go in my opinion. Sometimes I had issues with the wg plugin of opnsense, and it wasn't really stable all the time and for every version. I dropped this idea, instead I just run a small lxc container in Proxmox, with wg easy as container. I also love wg easy for providing QR codes, which makes adding mobile phones very easy.
I had a CT w/ docker and wg-easy already but thought it would be better integration in OPNsense w/ plugin and wg interface. In the end I did as you suggested, kept the CT and just configured port forwarding. It took me two days because of changing IP of my docker CT (it didn't identify by MAC) and some misconfiguration on the OPNsense side. However, it is working "perfectly splendid" now. As you say, QR code config is so convenient that even if it is only for my phone it is worth it!

Regarding IPv6 I did as you recommended as well. I am still beginner in networking and it would be a unnecessary complication. I turned it off.

I've also turned off Unbound that was on by default and every logging option I could find. I still have ~130k writes/s which I think is the reason my NVMe drive is running hot. I've ran out of options what to turn off :) Even after reboot I see no change. I've allocated all 4 cpu cores to OPNsense. They are mostly idle and spent most of their time in C10. But every so often they wake up and I see some C6 states and 28% cpu usage for a second (had 7% w/ one core allocated before). Since it is not doing much I might lower the CPU count again and see if my power consumption improves somewhat. I am at 12W idle now while I had 9W before the OPNsense was doing anything. I know it is not worth my time but I want to try to improve it anyway.
 
  • Like
Reactions: lightsword

metlrise

Member
Nov 17, 2021
18
27
13
Here is a collections of links that I've used

Proxmox VE Tools -> Proxmox VE Post Install
----should help with the writes on the NVME, the main culprit is High Availability in Promox

Used: Node temperature view

Not used: Nag screen deactivation, because the subscription nag screen is already in the first link


Tested in Win10 even if the tutorial is for Win11, and it works, the downside.... more heat due to the igpu utilization in Windows and after each Proxmox kernel update if have to recompile. In theory it should work also for Jellyfin
 
  • Like
Reactions: Stovar and Dostring

buz

Member
Nov 5, 2023
30
18
8
I followed the same instruction from page 4. I am using Unraid, my power consumption with only USB flash and an SD card and only using LAN0 is about 6W. I have LAN1-3 all disabled completely. I have the Crucial 48GB memory stick.
Any usb peripheral or video output adds watts.
With 2 SSDs I am at 11W mark.
How did you disable the LAN ports?
 

buz

Member
Nov 5, 2023
30
18
8
In the Bios, PCIe settings. Disable one or more lanes and check what is no more working. This can be a NIC or a nvme drive.
I suspected it will end up that way... I presume lanes 1-4 are the main NVMe (seeing that they are bonded in the BIOS of mine), so guess I'll start at the higher ones :)

The other thing that bother me is SSD temperatures. I got a Crucial P3 and sensors show it goes up to the low 70C...
 
  • Like
Reactions: SBMe

kliguin

Member
Nov 22, 2022
59
45
18
Memory issues, please help and report ;-) thanks in advance.

I have the 305 V2 from HUNSN and another local got the same motherboard from TopTon. Both of them have issues with the Crucial 48G RAM module CT48G56C46S5. It al looks stable for 9 days and then out of the blue the units crashes during PVE backups from the high speed 4x NVMe to the lower speed 1x NVMe. The crashes are always during the backup window, call it the high load window. There is an original fan on the bottom and a big fan on top, heat could not be a real issue....at least not from chassis perspective, might be specific memory heat?

Did some tests with memtest and this memory fails sometimes, with a fan it fails less and I got green passes. Noticed the crashes it is still failing, so I'm looking for others who experience the same? Any one having the same V2 box (picture below, check USB ports) N305 and also memory errors.

Or no memory errors and the N305 V2 please report too, including your so-dimm model ;-)

Many thanks, its hard to troubleshoot this.

1703331291655.png


Temps during a "the" backup job which crashed the unit at night, now by day simulated to get it into the crash state.


Code:
root@pve:~# sensors
nvme-pci-0600
Adapter: PCI adapter
Composite:    +48.9°C  (low  = -273.1°C, high = +81.8°C)
                       (crit = +84.8°C)
Sensor 1:     +48.9°C  (low  = -273.1°C, high = +65261.8°C)
Sensor 2:     +56.9°C  (low  = -273.1°C, high = +65261.8°C)

acpitz-acpi-0
Adapter: ACPI interface
temp1:        +27.8°C  (crit = +119.0°C)

coretemp-isa-0000
Adapter: ISA adapter
Package id 0:  +48.0°C  (high = +105.0°C, crit = +105.0°C)
Core 0:        +37.0°C  (high = +105.0°C, crit = +105.0°C)
Core 1:        +37.0°C  (high = +105.0°C, crit = +105.0°C)
Core 2:        +37.0°C  (high = +105.0°C, crit = +105.0°C)
Core 3:        +37.0°C  (high = +105.0°C, crit = +105.0°C)
Core 4:        +48.0°C  (high = +105.0°C, crit = +105.0°C)
Core 5:        +48.0°C  (high = +105.0°C, crit = +105.0°C)
Core 6:        +48.0°C  (high = +105.0°C, crit = +105.0°C)
Core 7:        +48.0°C  (high = +105.0°C, crit = +105.0°C)

nvme-pci-0100
Adapter: PCI adapter
Composite:    +46.9°C  (low  = -273.1°C, high = +81.8°C)
                       (crit = +84.8°C)
Sensor 1:     +46.9°C  (low  = -273.1°C, high = +65261.8°C)
Sensor 2:     +62.9°C  (low  = -273.1°C, high = +65261.8°C)
 
  • Like
Reactions: TvdH and SBMe

kliguin

Member
Nov 22, 2022
59
45
18
EDIT: I replaced the RAM and it appears to be working fine now.

I purchased an N100 from Topton on AliExpress from this link. I heard they used Crucial for RAM/SSD, so I purchased the 16GB RAM / 512GB SSD option for a total of only $248. I loaded Proxmox with two VMs--OPNsense and HAOS. CPU usage was in the teens or less, memory usage was low. Thermals were unusually high and the device was very hot to the touch.

It worked for a couple days, then it began crashing once a day. I ran memtest86 on the RAM and it passed the first test, then rebooted during the second test, and passed the third test. I also tested the SSD with CrystalDiskInfo which received a healthy result. As far as I can tell, the SSD and RAM are in fine working condition, although they are both from companies I've never heard of (SK hynix ram and Huadisk SSD).

I noticed the thermals were at 81c during the memtest, so I assumed the issue was overheating and placed a small box fan next to the unit and left it there overnight. The box was cool to the touch and thermals were down to ~40c at idle (previously in the 70s at idle). When I woke up, the device had crashed and again was unresponsive. The thermals were still at 40c right before the crash.

I attempted to reinstall Proxmox, however it then booted into the UEFI Interactive Shell after install. Tried the install again and it crashed immediately. I tried to boot to the BIOS but it crashed immediately.

At this point I believe there is likely a fault in the CPU or motherboard. I was going to try replacing the SSD and/or RAM, but based on the test results that seems like a futile effort. If anyone has any ideas here I'd love to hear them. I'm pretty sure the box is just dead, but if there's a way to fix it that would be the ideal solution.

At this point my plan is to open a return and refund request on AliExpress due to the item being defective and receive a full refund. There doesn't appear to be a return and replace option as far as I can tell, and at this point I'd likely prefer to get one from CWWK, even though I got a huge discount ordering on Black Friday. I have never returned anything on AliExpress before and have been dreading this process after reading numerous awful experiences here and on reddit.

Does anyone have any advice on how to go through the return process on AliExpress? Will they send me a prepaid shipping label? It seems like the return shipping is often where things get complicated.

EDIT: I replaced the RAM and it appears to be working fine now.

View attachment 33231View attachment 33232View attachment 33233View attachment 33234View attachment 33235

Goodmorning see my last post also RAM issues with the V2 board, only diff is you have a N100 and I have an N305. Could you please share which RAM you swapped for which RAM? Could you also tag me in the future, because I thought it was solved and 9 days later it crashed again. I'm really thinking: is it the board or the memory.
 

kliguin

Member
Nov 22, 2022
59
45
18
CT48G56C46S5 48G is not compatible with the N305 V2 with 8 USB ports.
CT32G48C40S5 32G is perfectly compatible with the N305 V2 with 8 USB ports.

Proven on a local forum in my home country with 4 different boxes and 4 different 48G memory modules. The new N305 V2 with 8 USB ports (2x 3.x USB and 6x 2.0USB) is not compatible with the 48G module of Crucial.

Yes it boot, and you are able to run proxmox but then out of the blue the box start crashing in the upcoming 10 days. Did some extensive test, and 3 other users, and all of us getting memory errors when running the box for several hours with the 48G SO-DIMM CT48G56C46S5. Memory temps reaching out to 137degrees Celsius...and causing issues.

Quite hard to diagnose, because the box need to reach his maximum temperatures, even with the bottom fan it crashes during the backup window while it is saving the VM's to another disk. For now the strange crashes I had with the 48G module are gone with the 32G module. Let's monitor for the upcoming days.

1703590803894.png
 
Last edited:

shimian

New Member
Dec 19, 2023
8
0
1
Is it safe right now to buy one of these N100 CWWK boxes with the 4x NVMe support if we do truly want to use all 4 NVMe ports? I want to build a hybrid NAS/virt box using one. Some of the earlier posts aren't super promising about the NVMe actually working correctly.
 

splifingate

Member
Oct 7, 2023
45
36
18
CT48G56C46S5 48G is not compatible with the N305 V2 with 8 USB ports.
CT32G48C40S5 32G is perfectly compatible with the N305 V2 with 8 USB ports.

Proven on a local forum in my home country with 4 different boxes and 4 different 48G memory modules. The new N305 V2 with 8 USB ports (2x 3.x USB and 6x 2.0USB) is not compatible with the 48G module of Crucial.

Yes it boot, and you are able to run proxmox but then out of the blue the box start crashing in the upcoming 10 days. Did some extensive test, and 3 other users, and all of us getting memory errors when running the box for several hours with the 48G SO-DIMM CT48G56C46S5. Memory temps reaching out to 137degrees Celsius...and causing issues.

Quite hard to diagnose, because the box need to reach his maximum temperatures, even with the bottom fan it crashes during the backup window while it is saving the VM's to another disk. For now the strange crashes I had with the 48G module are gone with the 32G module. Let's monitor for the upcoming days.

View attachment 33354
Hmm.

I am *extremely* new to the Prox Eco (for what that's worth), but if you could give me a rough set of steps-to-recreate, I would be willing (I have the time).

I have not seen temps >~42C, but I have a Noctua 12x12 sitting on top.

----CWWK-C; n305; AL-4L-V2; 3TB/48GiB (Crucial CT48G56C46S5.M16B1)
 
  • Like
Reactions: lightsword and SBMe

tibbe

New Member
Mar 17, 2022
24
6
3
Is it safe right now to buy one of these N100 CWWK boxes with the 4x NVMe support if we do truly want to use all 4 NVMe ports? I want to build a hybrid NAS/virt box using one. Some of the earlier posts aren't super promising about the NVMe actually working correctly.
I have the n305 and running out with 5 nvmes works fine for me.
 

kliguin

Member
Nov 22, 2022
59
45
18
Is it safe right now to buy one of these N100 CWWK boxes with the 4x NVMe support if we do truly want to use all 4 NVMe ports? I want to build a hybrid NAS/virt box using one. Some of the earlier posts aren't super promising about the NVMe actually working correctly.
The only concern I read about is the for the NVMe slots 2,3,4 are not working with ASPM power saving. Thats because they re-use a WiFi PCIe slot which somehow works different.
 
  • Like
Reactions: Stovar

kliguin

Member
Nov 22, 2022
59
45
18
Hmm.

I am *extremely* new to the Prox Eco (for what that's worth), but if you could give me a rough set of steps-to-recreate, I would be willing (I have the time).

I have not seen temps >~42C, but I have a Noctua 12x12 sitting on top.

----CWWK-C; n305; AL-4L-V2; 3TB/48GiB (Crucial CT48G56C46S5.M16B1)
It's quite easy, make a bootable USB stick with the memory test below. This memory test is way better than the embedded one from Proxmox (the blue screen). With all the multicore load this memory test below gives, its easier to detect problems.

Did you experience any strange shutdowns? Without the real life outages, I would have never noticed this memory issues with this module. Its that the box switches off during the backup window of proxmox where it is zipping and copying files between both NVMe's

closed source memtest86
 
Last edited:
  • Like
Reactions: TvdH

splifingate

Member
Oct 7, 2023
45
36
18
It's quite easy, make a bootable USB stick with the memory test below. This memory test is way better than the embedded one from Proxmox (the blue screen). With all the multicore load this memory test below gives, its easier to detect problems.
I've run the official memtest before. Not used the Prox one

Did you experience any strange shutdowns? Without the real life outages, I would have never noticed this memory issues with this module. Its that the box switches off during the backup window of proxmox where it is zipping and copying files between both NVMe's
No strange shutdowns. Just getting Prox running without pulling my remaining hair out has been the challenge (hardware takes it fine, but I just don't "get" it, yet (let-alone getting things to a point where I can do backups)).

I just have the h-board with two nvme's

closed source memtest86
Yes, that's the one.
 

splifingate

Member
Oct 7, 2023
45
36
18
I'm not getting this, where is the H located? Dit you run the tests for 4 passes, until you noticed the green passmark/finshed sign?

It's just an h-shaped break-out board (came with the unit) that moves the A+E slot over, and enables another 2280 nvme m.2 (@PCIe3-x1), leaving the main nvme m.2 (@PCIe3-x4) un-blocked.). My example, here (did subsequently switch from the Inland's to the SkH P31s):


It seems that I am (unfortunately) highly allergic to Proxmox, so I can only expose myself to it for short periods of time before I go batsh* crazy . . . I am seeking other pathways (sp. libvirt, et al.).