CWWK/Topton/... Nxxx quad NIC router

[Mike9474593]

Who's going to test the new N100 version with 8x 2.5G or 4x 2.5G and 2x SPF+ NIC?
cwwk.net
aliexpress

I have doubts about the PCIe lanes. PCI x4, 4x NIC and a M.2 1x port and all lanes are gone. How about the SATA, USB and mPCIe (USB only according aliexpress)?
And the Intel x520 is PCIe 2.0, so no full bandwidth with 2 SFP+ on PCIe x4...
Without any weird PCIe switch/sharing/transformation there will be some bottlenecks for sure.

 

[Napalm]

I've been on the same quest for over a year and I held off because I knew a new processor update was coming. Glad I did.

However one thing I've seen more than once is not to virtualize the router/gateway regardless of the config. Run it bare metal. Many of the newer router OS's allow for snapshot or other backup internal services so you don't need the hypervisor. Likewise they are meant to run headless and without the virtualization layer you have more capacity. Most of the router OS's also allow for running additions in the ecosystem especially the linux based IPfire. But supposedly PF and OPN sense do as well I've not used either.

Meanwhile on the number of ports thing. Again supposedly withthe right NIC the hardware off load of NAT and other Switch/router features between the NIC's eases the processor workload and isn't an issue. But I've also heard a number of opinions that say use just the 2 ports for the router/gateway and run a stand alone switch. Which is interesting when you consider the big business router boxes like Ubiquity and TP link etc actually act that way. Router on one end, and an internal switch on the other.- but it's all in the box.

I'm on the fence as I don't know enough to know better I really only need 5 ports in total, and getting a 6 port box is tempting from a pure cost savings vs getting a new 2.5gb switch also. Then it's still 200 ish for the router box and another 80-140 for a switch of some flavor vs just 200 ish for a 6 port box. The more I've thought about it the more I think I want the 2 port device and a stand alone switch. But regardless what I get I'm running it bare metal.

 

[Napalm]

Who's going to test the new N100 version with 8x 2.5G or 4x 2.5G and 2x SPF+ NIC?
cwwk.net
aliexpress

I have doubts about the PCIe lanes. PCI x4, 4x NIC and a M.2 1x port and all lanes are gone. How about the SATA, USB and mPCIe (USB only according aliexpress)?
And the Intel x520 is PCIe 2.0, so no full bandwidth with 2 SFP+ on PCIe x4...
Without any weird PCIe switch/sharing/transformation there will be some bottlenecks for sure.

there is only 9 pci lanes right but they are gen 4 speed if I recall. so one lane for the 4, 2.5 nics, and one lane for each SFP - covers 3 of the 9 lanes. then one lane for a M.2 NVME, and one lane for the other ports to share. gets to 5 if looking at the 4, 2.5 2 sfp model. which is interesting.

Maybe I'm thinking about it wrong.

 

[BlueLineSwinger]

there is only 9 pci lanes right but they are gen 4 speed if I recall. so one lane for the 4, 2.5 nics, and one lane for each SFP - covers 3 of the 9 lanes. then one lane for a M.2 NVME, and one lane for the other ports to share. gets to 5 if looking at the 4, 2.5 2 sfp model. which is interesting.

Maybe I'm thinking about it wrong.

The N100/300/etc. are PCIe3, not 4 unfortunately. Wouldn't really matter, however, given the way these systems are designed.

Typical design for these boards is that each 2.5 Gb NIC gets its own PCIe x1 (that's just how the I226 is built). So that's four taken up there. The m.2 slot brings it up to five. That leaves at most x4 for the 10 Gb NIC card.

The 10 Gb NICs are probably this controller/card, which is PCIe2 x8. Since it's connected to a x4 slot, the bandwidth is cut in half and throughput will be significantly bottlenecked if you wish to use both SFP+ ports at 10 Gb.

I like the idea of this box, but it looks like a less than optimal design. A newer controller that supports PCIe3 (e.g., the X550 or something similar for SPF+) would have been a much better, though almost certainly pricier, choice. Unfortunately the design means their proprietary format is required, standard PCIe cards won't fit.

 

[MaZy]

Hi guys,

The Topton C/N100 unit with Crucial CT32G56C46S5 memory is failing the memtest.

The Kingston KF548S38IB-32 is set to arrive next week, and I will provide an update on whether it works with the Kingston memory.

I'm using a Meanwell 65W adapter, so I believe the issue is not related to the power supply.

 

[dangar27]

[...]

4 NIC
Best for a virtualized firewall. eth0 as LAN, eth1 as WAN, using PCIE pass through. eth2 as a virtual bridge for all the other proxmox VMs, and eth3 as a static IP for proxmox management. This is what I'm leaning towards, but wanted to make sure my understanding was right.
[...]

6+ NIC
This is where I'm at a loss. As mentioned in the 4 NIC case, ideally I'd be able to use additional NICs as additional LAN ports. In the case of a 6+ NIC case, I could pass through eth4 and eth5 to pfsense and create a virutal bridge in pfsense rather than in proxmox. This avoids double-processing all LAN traffic, but all traffic on eth4 and eth5 would still need to be CPU processed. So I'm back to getting an external switch to increase the number of LAN ports, which would leave eth4 and eth5 unused.
[...]

Regardless to the number of NICs, you may also consider something else. This CW-AL-4L-V2.0 N305 has 4 NICs *and* the first nvme is PCIe3 x4. And it has 2x usb 3.0. Even if you are not considering right now to use the device as a nas, you may eventually along the line. Note that the 6L version nvme is only x1.

I got the 4l version due to this reason and I am testing it this days.

 

[KLMR]

My two cents for the CWWK i3-N305 version.
I've tried CT24G56C46S5 24GB @4800MTs module and works fine after +48h of memtest on the N305 v2.

Now I'm trying at 5600MTs (16,8GB/s aparently) and see how it works.

About the FAN.
The PWM cable came with a weird configuration and fan didn't work. I had to check out everty signal because connecting a fan reseted the device. The reason is how the dongle is cabled (dongle on the right, signal on the left!), which is wrong!


The proper order should be, from top to bottom:
1 GND (dongle black)
2 12V (dongle red)
3 TACHO (dongle yellow)
4 PWM (dongle blue)
you can repin the connector as needed with needle or similar. Check out if you have the same problem.

BTW I'm trying to find the IOMMU options to separate the groups to passthrough two NICs do you know where are those options in the bios menu?
Aparently after updating grub and setting it up my IOMMU grups appeared correclty, first all hadrware appeared as being at same group (-1).

 

[Ayugai Michiko]

Hi guys,

The Topton C/N100 unit with Crucial CT32G56C46S5 memory is failing the memtest.

The Kingston KF548S38IB-32 is set to arrive next week, and I will provide an update on whether it works with the Kingston memory.

I'm using a Meanwell 65W adapter, so I believe the issue is not related to the power supply.

Try blowing a fan on the CT32G56C46S5. Might be overheating? Most people here tend to pick the 4800MHz version instead of 5600MHz

 

[MaZy]

Try blowing a fan on the CT32G56C46S5. Might be overheating? Most people here tend to pick the 4800MHz version instead of 5600MHz

I thought about that, but I want to keep it with fully passive cooling, which is why I opted for the N100 instead of the i3-N305. I have a J4125 box from Topton, and it has been working perfectly for many years in this mode. If the RAM cannot work without active cooling, then it is not suitable for me.

I bought 5600MHz RAM because it was priced the same as 4800MHz. I thought that if the 5600 chips operated at a lower frequency, it would increase reliability (reduce heating, extend service life, etc.). Apparently, that was my mistake.

Let's see how it performs with Kingston.

 

[userbob7659]

Hi all when updating my Toptop N100 routers bios with the instructions provide from Thys post https://forums.servethehome.com/ind...xxx-quad-nic-router.39685/page-75#post-405132
After I boot into the USB I get 1.nsh is not a recognized as an Internal or external command. Has anyone else come across this when updating a Bk-1264np-4l ?

 

[dss]

After I boot into the USB I get 1.nsh is not a recognized as an Internal or external command.

I had that trouble. I think it’s because the BIOS update is on fs1 rather than fs0. Some instructions say to take the NVME out which might solve it.

An alternative is to run the following when you get to the prompt

FS1:
CD EFI
1.nsh

 

[tusk9541]

I got one of those little "M1" boxes from a brand called mrroute, on Amazon but it appears it's shipped from China anyway. They're also available on aliexpress from different stores including Topton and Kingovy but all of them are more expensive.

I've searched in the thread but aside from someone asking about them, there's still not much info that I can tell. Nobody got em?

They're a bit more expensive than the normal bigger fanless boxes but I was looking for LPDDR5 and a more minimal setup to see if I can shave several watts compared to my N5105 Opnsense setup. I'll be looking to mod it with a fanless heatsink which doesn't seem too hard.

 

[idle_user]

I got one of those little "M1" boxes from a brand called mrroute, on Amazon but it appears it's shipped from China anyway.

This is why you look at the "sold by" info on Amazon. Usually if it's not Prime, it's not sold or shipped by Amazon.

 

[userbob7659]

I had that trouble. I think it’s because the BIOS update is on fs1 rather than fs0. Some instructions say to take the NVME out which might solve it.

An alternative is to run the following when you get to the prompt

FS1:
CD EFI
1.nsh

Thank you dss that worked after I removed the SSD!

 

[MaZy]

Hi guys,

Does anyone know if it's possible to limit NVMe speed in the BIOS somehow? Or to limit the active PCIe lanes for the main NVMe storage? I'm just checking if I can do this for a slight reduction in speed, which should result in less heating, respectively.

Modern drives are excessively fast for my needs. Currently, I have a SATA drive that operates at 500 MB/sec, and that's enough for me.

 

[pigr8]

there is expansion board, but I cannot find it how to order it, most likely it depends also on the board version v2....v3

and

all in one, without a case

X86 P5 development board 4*M2 NVMe expansion board 12th generation Intel N100 N200 i3-N305 DDR5 4800MHz firewall PC 2x i226-V 2.5G network card

cwwk

cwwk.net

i've ordered a N305 replacement board with the 4 nvme expansion, it should arrive next week.. i already have a V3 with a N100 and the link speed is x4 so the board does split them to pcie 3.0 x1 each.. can't wait to test as a nas.

 

[idle_user]

Hi guys,

Does anyone know if it's possible to limit NVMe speed in the BIOS somehow? Or to limit the active PCIe lanes for the main NVMe storage? I'm just checking if I can do this for a slight reduction in speed, which should result in less heating, respectively.

Modern drives are excessively fast for my needs. Currently, I have a SATA drive that operates at 500 MB/sec, and that's enough for me.

You do know just because a drive has a high theoretical speed, it doesn't mean it's constantly running at that speed and generating heat right?

r/w only happen when needed. And the PCIe lane will already be limited to the speed of the connected device.


Plus, the math probably wouldn't checkout anyways. I would assume a device would generate more heat the longer the r/w operation takes. So purposely slowing down a link speed would result in longer operating times and result in more heat

 

[MaZy]

You do know just because a drive has a high theoretical speed, it doesn't mean it's constantly running at that speed and generating heat right?

r/w only happen when needed. And the PCIe lane will already be limited to the speed of the connected device.


Plus, the math probably wouldn't checkout anyways. I would assume a device would generate more heat the longer the r/w operation takes. So purposely slowing down a link speed would result in longer operating times and result in more heat

Let me disagree with you.

Please do some basic tests with dd, and you will see that drives generate a lot less heat when operating at a slow speed.
Yes, you are right that heat is produced only when I/O operations are performed, and this is what I want to avoid. I want to avoid these kinds of sudden spikes with temperatures warming up to 90 degrees Celsius. With a limit of 2GB/sec, for instance, an SSD drive will generate little heat and will have time to cool on its own

The same is true for the CPU/GPU frequency

 

[idle_user]

Let me disagree with you.

Please do some basic tests with dd, and you will see that drives generate a lot less heat when operating at a slow speed.
Yes, you are right that heat is produced only when I/O operations are performed, and this is what I want to avoid. I want to avoid these kinds of sudden spikes with temperatures warming up to 90 degrees Celsius. With a limit of 2GB/sec, for instance, an SSD drive will generate little heat and will have time to cool on its own

The same is true for the CPU/GPU frequency

If you're getting sudden 90c spikes, it sounds like you have a heat dissipation issue.

Have you repasted and checked for good contact?

 

[MaZy]

If you're getting sudden 90c spikes, it sounds like you have a heat dissipation issue.

Have you repasted and checked for good contact?

Are you serious? We're discussing NVMe storage. What do you mean by 'good contact'? My question was whether it's possible to reduce load peaks for SSD storage to achieve more stable temperatures. I don't require, nor do I have, workloads that operate at the 4-5-6GB/s speeds modern SSDs offer.

I simply want to avoid overheating the SSDs when making backups or writing large files