ASUSTOR FLASHSTOR NAS with 6x/12x M.2 SSDs, 2.5/10 GbE

[icecrash]

I saw an article today posted on CNX Software about an ASUSTOR FLASHTOR NAS with either 6x or 12x M.2 bays and with up to 10 GbE (for the 12x one). This looks really interesting. It's similar to the idea I had in:

https://forums.servethehome.com/index.php?threads/fanless-all-ssd-4-bay-nas.39673/

about making a NAS from one of those fanless router boxes.

Some Amazon links:
6 bay
12 bay

Couple of caveats:
* It has a fan
* N5105 is limited to 8x PCIe Gen 3 lanes (1 GB/s per lane), so bandwidth may be limited. Although, it should still be enough to max out a 10 GbE line.

 

[zunder1990]

That is a bummer that the there is no sfp slot, I really dont like 10gb based copper.

 

[oneplane]

I thought QNAP or IEI or some blend of those had something very similar to this. I've looked for it, but so far I've only come across this beauty (which is something totally different ):Mustang-200-C-8G-R10. I remember the QNAP one having a similar form factor but older PCIe generation.

Edit: looks like their variant was more oriented around HTPC type applications: HS-453DX-8G-US HS-453DX-8G-US 4-Bay 10GB NAS 8GB RAM which only had 2x M.2 and 4x SATA 2.5", so pretty much designed for bulk that 'previous generation' storage methods. In a way it might not actually matter that much considering you're not going to have much luck pushing many lanes of PCIe bandwidth over a 10G link.

 

[mrpasc]

Qnap has had several „toys“.
TBS-453A and it’s successor TBS-464 came instantly to my mind… But not sure if they ever made it to market

 

[Patrick]

We have the 6-bay Asustor. Pushing to get something up soon-ish. Bryan did photos, B-roll, and some setup today. I am going to play with it more tomorrow.

 

[Patrick]

Ok I think this was a bad idea.

There are things like volume creation that are largely single threaded so even with NVMe SSDs, this is VERY slow.

The other thing is that with 1TB SSDs in RAID 5 that is only about ~4.5TB usable.

$450 for the NAS
$350 for the Crucial NVMe SSDs

So about $800 for 4.5TB of SSD storage limited by 2x 2.5GbE ports. Doubling the capacity would only be another $300 or so so $1100 for 9TB. Going to 4TB would be $1800 for 18-19TB using Crucial P3 Plus drives. Crucial P3's would be $1650. For only 4.5TB usable, it is almost better to just buy two 4TB NVMe SSDs and put them in an existing system to make a NAS from it.



Here is the back side of the unit. Jeff Geerling (the Raspberry Pi guy) has the 12-bay. These are populated with M.2 slots on his. The 3 unpopulated pads are PCIe switches in his, still with the ASMedia ASM1480 MUX chips all over.


Drives installed


I have this unit sitting on my desk next to me. The fan is not bad at all.

 

[geerlingguy]

Jeff Geerling (the Raspberry Pi guy) has the 12-bay. These are populated with M.2 slots on his. The 3 unpopulated pads are PCIe switches in his, still with the ASMedia ASM1480 MUX chips all over.

Indeed—for me, the main question is (with either model, but more specifically the 12 bay unit): can it saturate the 10 GbE port continuously, and does it have good low-latency access with reasonable storage configurations?

E.g. my use case is an edit NAS with 4x 1 TB NVMe drives (decent ones that are $99-150 per right now). So total system cost around $1000, which is quite a bit more than buying used gear (or even a mini PC) and a PCIe to NVMe card that bifurcates the lanes, and dropping FreeNAS or something like that on it.

The ability to throw in more NVMe drives is nice, though for me it's all about the latency—maybe the single 10 GbE port isn't enough or maybe it's perfect for the bandwidth required. Haven't done any testing mostly because I threw some drives in last night and ran into a weird issue in initialization.

I was a little surprised by the heat the thing pumps out through the side (the fan is nice and quiet, owing to it's large size), and saw it pulling 17.7W at idle without any drives installed. I would've thought the N5105 draws less than that...?

I haven't cracked open the bottom board to see what other chip is under a large heat sink (looks like maybe it's the 10G NIC... I saw a realtek chip for 7.1 audio, too, Realtek ALC888S.

I plugged in an HDMI display, but it shows the TianoCore splash screen logo, then a blinking cursor. I didn't know if it would be like other NASes with HDMI, where it displays a simple UI for media playback. But since I couldn't fully initialize the NAS, I'm not sure if there's more or not.

 

[geerlingguy]

And a picture of the populated 12-bay board:

 

[Patrick]

Indeed—for me, the main question is (with either model, but more specifically the 12 bay unit): can it saturate the 10 GbE port continuously, and does it have good low-latency access with reasonable storage configurations?

A wild JeffyG appears!

N5105 has a 10W TDP and can go over that. You also have power consumption with the SODIMM. Each drive is a few watts.

Also, I just put 2x 32GB SODIMMs in and it worked. 64GB with the Intel N5105.


A crash happened, so back to 4GB.

 

[Patrick]

Copied some photos/ footage over SMB3 to the HP Elite Mini 600 G9 with the 10Gbase-T adapter. Not too bad at all with the only tuning being checking the SMB multi-channel box in Asustor's web GUI.

 

[geerlingguy]

That's... not bad at all! So you just had a single 10G adapter on the Elite Mini, and that went into a switch with both 2.5G ports plugged in? (And do you have a 10G card in that box, or is there some 10 Gbps USB adaptor I haven't seen before?

 

[oneplane]

I'm curious what type of storage management tech it is actually doing in the software. Is it just mdadm raid, or *fancy* LVM+MD? Or does it even does btrfs or ZFS-on-Linux?

 

[geerlingguy]

I'm curious what type of storage management tech it is actually doing in the software. Is it just mdadm raid, or *fancy* LVM+MD? Or does it even does btrfs or ZFS-on-Linux?

It's running ADM (ASUSTOR Data Manager), a custom fork of some tiny Linux distro, and it uses mdadm for standard RAID, or Btrfs optionally. No ZFS support at this time.

 

[Patrick]

That's... not bad at all! So you just had a single 10G adapter on the Elite Mini, and that went into a switch with both 2.5G ports plugged in? (And do you have a 10G card in that box, or is there some 10 Gbps USB adaptor I haven't seen before?

It is a 10G adapter in the HP.

@oneplane mdraid and you can use btrfs. No ZoL

 

[oneplane]

Interesting. Essentially it does the same kind of data durability and availability as practically any consumer NAS, but in this configuration it should theoretically be much faster than your average standalone NAS (both in terms of latency and bandwidth).

While I'm not familiar with ADM, it does seem to support the standard swath of filing protocols including NFS and SFTP, perhaps those get closer to the interface speeds than SMB does (depending on what samba version they are running under the hood). It might also simply be a limitation of the PCIe hardware configuration and no amount of throwing SSDs at the box makes it go any faster (but at last the expansion capabilities are pretty sweet -- even if just for bulk storage).

 

[geektech]

I wonder if this NAS would support Synology port Xpenology DSM.

Still seems pricey, it might be cheaper to DIY a smaller MoBO with NVMe PcI e cards and custom flash Xpenology DSM on it.

 

[oldpenguin]

Something doesn't really add up for me.
N5105 with 8x PCIe lanes - single 10GBe (4 lanes, unless shoddified to 2 lanes), 4-6 lanes remaining for 16 m.2 units that usually need x4 each. Is my math broken at this hour, are all m2's SATA or remaining 4-6 lanes are MUXed for whatever outcome may be? Not to mention it also has USB 3.2 too.

 

[PigLover]

Something doesn't really add up for me.
N5105 with 8x PCIe lanes - single 10GBe (4 lanes, unless shoddified to 2 lanes), 4-6 lanes remaining for 16 m.2 units that usually need x4 each. Is my math broken at this hour, are all m2's SATA or remaining 4-6 lanes are MUXed for whatever outcome may be? Not to mention it also has USB 3.2 too.

They are using PCIe switches (ASM1480 apparently) to share the PCIe lanes across all of the various consumers (M.2 drives, 10gbe NIC, etc). Without a good block diagram it is not real clear exactly how they do this distribution (e.g., how many lanes are exposed to each device and what is each device sharing/muxing with).

 

[Patrick]

Something doesn't really add up for me.
N5105 with 8x PCIe lanes - single 10GBe (4 lanes, unless shoddified to 2 lanes), 4-6 lanes remaining for 16 m.2 units that usually need x4 each. Is my math broken at this hour, are all m2's SATA or remaining 4-6 lanes are MUXed for whatever outcome may be? Not to mention it also has USB 3.2 too.

1. The 6x M.2 version uses the ASM1480 PCIe MUX chips.
2. The 12x M.2 version uses ASMedia switch chips as well.

M.2 drives really only need x1 to connect. The Supermicro SYS-111C-NR we reviewed today only has a x2 to each M.2 SSD since they are just boot SSD slots.

You are correct, that if everything was at full speed the 12-bay would usually need:

• 12 x4 lanes for the M.2 SSDs
• 1 x4 (or x2) for the 10Gbase-T
• So 52x PCIe lanes total

The other way to think about it is that really all Asustor is trying to do is to fill the network pipe and have more drives for capacity.

 

[PigLover]

..M.2 drives really only need x1 to connect...

In the 12 port version wouldn't the M.2 drives need at least x2 each to saturate the 10gbe network port?