CWWK/Topton/... Nxxx quad NIC router

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Mike9474593

Active Member
Aug 5, 2022
120
87
28
Who's going to test the new N100 version with 8x 2.5G or 4x 2.5G and 2x SPF+ NIC?
cwwk.net
aliexpress

I have doubts about the PCIe lanes. PCI x4, 4x NIC and a M.2 1x port and all lanes are gone. How about the SATA, USB and mPCIe (USB only according aliexpress)?
And the Intel x520 is PCIe 2.0, so no full bandwidth with 2 SFP+ on PCIe x4...
Without any weird PCIe switch/sharing/transformation there will be some bottlenecks for sure.
 
Last edited:
  • Wow
Reactions: Stovar

Napalm

New Member
Sep 8, 2021
28
5
3
I've been on the same quest for over a year and I held off because I knew a new processor update was coming. Glad I did.

However one thing I've seen more than once is not to virtualize the router/gateway regardless of the config. Run it bare metal. Many of the newer router OS's allow for snapshot or other backup internal services so you don't need the hypervisor. Likewise they are meant to run headless and without the virtualization layer you have more capacity. Most of the router OS's also allow for running additions in the ecosystem especially the linux based IPfire. But supposedly PF and OPN sense do as well I've not used either.

Meanwhile on the number of ports thing. Again supposedly withthe right NIC the hardware off load of NAT and other Switch/router features between the NIC's eases the processor workload and isn't an issue. But I've also heard a number of opinions that say use just the 2 ports for the router/gateway and run a stand alone switch. Which is interesting when you consider the big business router boxes like Ubiquity and TP link etc actually act that way. Router on one end, and an internal switch on the other.- but it's all in the box.

I'm on the fence as I don't know enough to know better I really only need 5 ports in total, and getting a 6 port box is tempting from a pure cost savings vs getting a new 2.5gb switch also. Then it's still 200 ish for the router box and another 80-140 for a switch of some flavor vs just 200 ish for a 6 port box. The more I've thought about it the more I think I want the 2 port device and a stand alone switch. But regardless what I get I'm running it bare metal.
 

Napalm

New Member
Sep 8, 2021
28
5
3
Who's going to test the new N100 version with 8x 2.5G or 4x 2.5G and 2x SPF+ NIC?
cwwk.net
aliexpress

I have doubts about the PCIe lanes. PCI x4, 4x NIC and a M.2 1x port and all lanes are gone. How about the SATA, USB and mPCIe (USB only according aliexpress)?
And the Intel x520 is PCIe 2.0, so no full bandwidth with 2 SFP+ on PCIe x4...
Without any weird PCIe switch/sharing/transformation there will be some bottlenecks for sure.
there is only 9 pci lanes right but they are gen 4 speed if I recall. so one lane for the 4, 2.5 nics, and one lane for each SFP - covers 3 of the 9 lanes. then one lane for a M.2 NVME, and one lane for the other ports to share. gets to 5 if looking at the 4, 2.5 2 sfp model. which is interesting.

Maybe I'm thinking about it wrong.
 
  • Like
Reactions: Stovar

BlueLineSwinger

Active Member
Mar 11, 2013
188
75
28
there is only 9 pci lanes right but they are gen 4 speed if I recall. so one lane for the 4, 2.5 nics, and one lane for each SFP - covers 3 of the 9 lanes. then one lane for a M.2 NVME, and one lane for the other ports to share. gets to 5 if looking at the 4, 2.5 2 sfp model. which is interesting.

Maybe I'm thinking about it wrong.

The N100/300/etc. are PCIe3, not 4 unfortunately. Wouldn't really matter, however, given the way these systems are designed.

Typical design for these boards is that each 2.5 Gb NIC gets its own PCIe x1 (that's just how the I226 is built). So that's four taken up there. The m.2 slot brings it up to five. That leaves at most x4 for the 10 Gb NIC card.

The 10 Gb NICs are probably this controller/card, which is PCIe2 x8. Since it's connected to a x4 slot, the bandwidth is cut in half and throughput will be significantly bottlenecked if you wish to use both SFP+ ports at 10 Gb.

I like the idea of this box, but it looks like a less than optimal design. A newer controller that supports PCIe3 (e.g., the X550 or something similar for SPF+) would have been a much better, though almost certainly pricier, choice. Unfortunately the design means their proprietary format is required, standard PCIe cards won't fit.
 
  • Like
Reactions: Stovar

MaZy

New Member
Mar 17, 2024
10
1
3
Hi guys,

The Topton C/N100 unit with Crucial CT32G56C46S5 memory is failing the memtest.

The Kingston KF548S38IB-32 is set to arrive next week, and I will provide an update on whether it works with the Kingston memory.

I'm using a Meanwell 65W adapter, so I believe the issue is not related to the power supply.
 

Attachments

dangar27

New Member
Mar 11, 2024
3
6
3
[...]

4 NIC
Best for a virtualized firewall. eth0 as LAN, eth1 as WAN, using PCIE pass through. eth2 as a virtual bridge for all the other proxmox VMs, and eth3 as a static IP for proxmox management. This is what I'm leaning towards, but wanted to make sure my understanding was right.
[...]

6+ NIC
This is where I'm at a loss. As mentioned in the 4 NIC case, ideally I'd be able to use additional NICs as additional LAN ports. In the case of a 6+ NIC case, I could pass through eth4 and eth5 to pfsense and create a virutal bridge in pfsense rather than in proxmox. This avoids double-processing all LAN traffic, but all traffic on eth4 and eth5 would still need to be CPU processed. So I'm back to getting an external switch to increase the number of LAN ports, which would leave eth4 and eth5 unused.
[...]
Regardless to the number of NICs, you may also consider something else. This CW-AL-4L-V2.0 N305 has 4 NICs *and* the first nvme is PCIe3 x4. And it has 2x usb 3.0. Even if you are not considering right now to use the device as a nas, you may eventually along the line. Note that the 6L version nvme is only x1.

I got the 4l version due to this reason and I am testing it this days.
 

KLMR

New Member
Jun 25, 2022
16
2
3
My two cents for the CWWK i3-N305 version.
I've tried CT24G56C46S5 24GB @4800MTs module and works fine after +48h of memtest on the N305 v2.

Now I'm trying at 5600MTs (16,8GB/s aparently) and see how it works.

About the FAN.
The PWM cable came with a weird configuration and fan didn't work. I had to check out everty signal because connecting a fan reseted the device. The reason is how the dongle is cabled (dongle on the right, signal on the left!), which is wrong!
PWM_Fan_Cable.jpg

The proper order should be, from top to bottom:
1 GND (dongle black)
2 12V (dongle red)
3 TACHO (dongle yellow)
4 PWM (dongle blue)
you can repin the connector as needed with needle or similar. Check out if you have the same problem.

BTW I'm trying to find the IOMMU options to separate the groups to passthrough two NICs do you know where are those options in the bios menu?
Aparently after updating grub and setting it up my IOMMU grups appeared correclty, first all hadrware appeared as being at same group (-1).
 
Last edited:
  • Like
Reactions: Stovar
Sep 10, 2015
53
42
18
68
Hi guys,

The Topton C/N100 unit with Crucial CT32G56C46S5 memory is failing the memtest.

The Kingston KF548S38IB-32 is set to arrive next week, and I will provide an update on whether it works with the Kingston memory.

I'm using a Meanwell 65W adapter, so I believe the issue is not related to the power supply.
Try blowing a fan on the CT32G56C46S5. Might be overheating? Most people here tend to pick the 4800MHz version instead of 5600MHz
 

MaZy

New Member
Mar 17, 2024
10
1
3
Try blowing a fan on the CT32G56C46S5. Might be overheating? Most people here tend to pick the 4800MHz version instead of 5600MHz
I thought about that, but I want to keep it with fully passive cooling, which is why I opted for the N100 instead of the i3-N305. I have a J4125 box from Topton, and it has been working perfectly for many years in this mode. If the RAM cannot work without active cooling, then it is not suitable for me.

I bought 5600MHz RAM because it was priced the same as 4800MHz. I thought that if the 5600 chips operated at a lower frequency, it would increase reliability (reduce heating, extend service life, etc.). Apparently, that was my mistake.

Let's see how it performs with Kingston.
 

dss

New Member
Mar 28, 2024
3
0
1
After I boot into the USB I get 1.nsh is not a recognized as an Internal or external command.
I had that trouble. I think it’s because the BIOS update is on fs1 rather than fs0. Some instructions say to take the NVME out which might solve it.

An alternative is to run the following when you get to the prompt

Code:
FS1:
CD EFI
1.nsh
 

tusk9541

Member
Nov 23, 2022
59
72
18
I got one of those little "M1" boxes from a brand called mrroute, on Amazon but it appears it's shipped from China anyway. They're also available on aliexpress from different stores including Topton and Kingovy but all of them are more expensive.

I've searched in the thread but aside from someone asking about them, there's still not much info that I can tell. Nobody got em?

They're a bit more expensive than the normal bigger fanless boxes but I was looking for LPDDR5 and a more minimal setup to see if I can shave several watts compared to my N5105 Opnsense setup. I'll be looking to mod it with a fanless heatsink which doesn't seem too hard.
 

userbob7659

New Member
Apr 20, 2024
2
0
1
I had that trouble. I think it’s because the BIOS update is on fs1 rather than fs0. Some instructions say to take the NVME out which might solve it.

An alternative is to run the following when you get to the prompt

Code:
FS1:
CD EFI
1.nsh
Thank you dss that worked after I removed the SSD!
 

MaZy

New Member
Mar 17, 2024
10
1
3
Hi guys,

Does anyone know if it's possible to limit NVMe speed in the BIOS somehow? Or to limit the active PCIe lanes for the main NVMe storage? I'm just checking if I can do this for a slight reduction in speed, which should result in less heating, respectively.

Modern drives are excessively fast for my needs. Currently, I have a SATA drive that operates at 500 MB/sec, and that's enough for me.
 

pigr8

Member
Jul 13, 2017
95
96
18
there is expansion board, but I cannot find it how to order it, most likely it depends also on the board version v2....v3


and


all in one, without a case
i've ordered a N305 replacement board with the 4 nvme expansion, it should arrive next week.. i already have a V3 with a N100 and the link speed is x4 so the board does split them to pcie 3.0 x1 each.. can't wait to test as a nas.
 

idle_user

Member
Jun 24, 2023
76
57
18
Hi guys,

Does anyone know if it's possible to limit NVMe speed in the BIOS somehow? Or to limit the active PCIe lanes for the main NVMe storage? I'm just checking if I can do this for a slight reduction in speed, which should result in less heating, respectively.

Modern drives are excessively fast for my needs. Currently, I have a SATA drive that operates at 500 MB/sec, and that's enough for me.
You do know just because a drive has a high theoretical speed, it doesn't mean it's constantly running at that speed and generating heat right?

r/w only happen when needed. And the PCIe lane will already be limited to the speed of the connected device.


Plus, the math probably wouldn't checkout anyways. I would assume a device would generate more heat the longer the r/w operation takes. So purposely slowing down a link speed would result in longer operating times and result in more heat
 

MaZy

New Member
Mar 17, 2024
10
1
3
You do know just because a drive has a high theoretical speed, it doesn't mean it's constantly running at that speed and generating heat right?

r/w only happen when needed. And the PCIe lane will already be limited to the speed of the connected device.


Plus, the math probably wouldn't checkout anyways. I would assume a device would generate more heat the longer the r/w operation takes. So purposely slowing down a link speed would result in longer operating times and result in more heat
Let me disagree with you.

Please do some basic tests with dd, and you will see that drives generate a lot less heat when operating at a slow speed.
Yes, you are right that heat is produced only when I/O operations are performed, and this is what I want to avoid. I want to avoid these kinds of sudden spikes with temperatures warming up to 90 degrees Celsius. With a limit of 2GB/sec, for instance, an SSD drive will generate little heat and will have time to cool on its own

The same is true for the CPU/GPU frequency
 

idle_user

Member
Jun 24, 2023
76
57
18
Let me disagree with you.

Please do some basic tests with dd, and you will see that drives generate a lot less heat when operating at a slow speed.
Yes, you are right that heat is produced only when I/O operations are performed, and this is what I want to avoid. I want to avoid these kinds of sudden spikes with temperatures warming up to 90 degrees Celsius. With a limit of 2GB/sec, for instance, an SSD drive will generate little heat and will have time to cool on its own

The same is true for the CPU/GPU frequency
If you're getting sudden 90c spikes, it sounds like you have a heat dissipation issue.

Have you repasted and checked for good contact?
 
  • Like
Reactions: slidermike

MaZy

New Member
Mar 17, 2024
10
1
3
If you're getting sudden 90c spikes, it sounds like you have a heat dissipation issue.

Have you repasted and checked for good contact?
Are you serious? We're discussing NVMe storage. What do you mean by 'good contact'? My question was whether it's possible to reduce load peaks for SSD storage to achieve more stable temperatures. I don't require, nor do I have, workloads that operate at the 4-5-6GB/s speeds modern SSDs offer.

I simply want to avoid overheating the SSDs when making backups or writing large files