[ebay BO] RMS-200/8G PCI-e NVRAM Accelerator 360$

Monoman · Jun 8, 2019

FINALLY got mine installed on my proxmox host.

dual e5 2697v2
256gb ram
24x 200gb s3700 (off expander) to sas3 hba
RMS-200 8gb

Here's pveperf on just the drive.

Code:

root@pve1:/mnt# pveperf .

CPU BOGOMIPS:      259232.64
REGEX/SECOND:      1545679
HD SIZE:           7.80 GB (/dev/nvme0n1p1)
BUFFERED READS:    1723.78 MB/sec
AVERAGE SEEK TIME: 0.02 ms
FSYNCS/SECOND:     6827.35
DNS EXT:           43.92 ms
DNS INT:           57.11 ms (m0n0mana.local.lan)

Here's pve perv on my array with the RMS-200 with sync=standard

Code:

root@pve1:~# zfs set sync=standard s3700_pool
root@pve1:~# zfs get sync s3700_pool
NAME        PROPERTY  VALUE     SOURCE
s3700_pool  sync      standard  local
root@pve1:~# pveperf /s3700_pool/dir/
CPU BOGOMIPS:      259232.64
REGEX/SECOND:      1535681
HD SIZE:           2322.21 GB (s3700_pool)
FSYNCS/SECOND:     3805.59
DNS EXT:           42.72 ms
DNS INT:           58.03 ms (m0n0mana.local.lan)

Here's pve perv on my array with the RMS-200 with sync=always

Code:

root@pve1:~# zfs set sync=always s3700_pool
root@pve1:~# zfs get sync s3700_pool
NAME        PROPERTY  VALUE     SOURCE
s3700_pool  sync      always    local
root@pve1:~# pveperf /s3700_pool/dir/
CPU BOGOMIPS:      259232.64
REGEX/SECOND:      1555953
HD SIZE:           2322.21 GB (s3700_pool)
FSYNCS/SECOND:     2578.80
DNS EXT:           43.10 ms
DNS INT:           54.00 ms (m0n0mana.local.lan)

Monoman · Jun 8, 2019

P.S. It's quite a bit faster than when I used my optane (don't have the numbers off hand)

rshakin · Jun 11, 2019

Very nice reaults... I wish I had that many drives

i386 · Aug 15, 2019

I got 3 of these devices for ~73£ each on ebay

Some windows diskspd numbers:

Code:

diskspd -b4K -c6G -d120 -L -o8 -r -Sh -t4 -w20 I:\testfile.dat

Code:

Input parameters:

   timespan:   1
   -------------
   duration: 120s
   warm up time: 5s
   cool down time: 0s
   measuring latency
   random seed: 0
   path: 'I:\testfile.dat'
       think time: 0ms
       burst size: 0
       software cache disabled
       hardware write cache disabled, writethrough on
       performing mix test (read/write ratio: 80/20)
       block size: 4096
       using random I/O (alignment: 4096)
       number of outstanding I/O operations: 8
       thread stride size: 0
       threads per file: 4
       using I/O Completion Ports
       IO priority: normal

Code:

CPU |  Usage |  User  |  Kernel |  Idle
-------------------------------------------
   0|  99.96%|  13.66%|   86.30%|   0.04%
   1| 100.00%|  10.25%|   89.75%|   0.00%
   2|  99.97%|   8.78%|   91.20%|   0.03%
   3|  99.96%|   9.10%|   90.86%|   0.04%
   4|  29.57%|  19.64%|    9.93%|  70.43%
   5|  29.66%|  21.95%|    7.71%|  70.34%
   6|  33.55%|  24.04%|    9.52%|  66.45%
   7|  30.63%|  22.43%|    8.19%|  69.38%
-------------------------------------------
avg.|  65.41%|  16.23%|   49.18%|  34.59%

Total IO
thread |       bytes     |     I/Os     |    MiB/s   |  I/O per s |  AvgLat  | LatStdDev |  file
-----------------------------------------------------------------------------------------------------
     0 |     25525678080 |      6231855 |     202.86 |   51932.12 |    0.134 |     0.237 | I:\testfile.dat (6144MiB)
     1 |     27306102784 |      6666529 |     217.01 |   55554.40 |    0.126 |     0.173 | I:\testfile.dat (6144MiB)
     2 |     30074277888 |      7342353 |     239.01 |   61186.27 |    0.114 |     0.153 | I:\testfile.dat (6144MiB)
     3 |     30585831424 |      7467244 |     243.07 |   62227.03 |    0.112 |     0.162 | I:\testfile.dat (6144MiB)
-----------------------------------------------------------------------------------------------------
total:      113491890176 |     27707981 |     901.95 |  230899.82 |    0.121 |     0.182

Read IO
thread |       bytes     |     I/Os     |    MiB/s   |  I/O per s |  AvgLat  | LatStdDev |  file
-----------------------------------------------------------------------------------------------------
     0 |     20423725056 |      4986261 |     162.31 |   41552.17 |    0.134 |     0.239 | I:\testfile.dat (6144MiB)
     1 |     21842378752 |      5332612 |     173.59 |   44438.43 |    0.125 |     0.171 | I:\testfile.dat (6144MiB)
     2 |     24058986496 |      5873776 |     191.20 |   48948.13 |    0.114 |     0.151 | I:\testfile.dat (6144MiB)
     3 |     24477962240 |      5976065 |     194.53 |   49800.54 |    0.112 |     0.158 | I:\testfile.dat (6144MiB)
-----------------------------------------------------------------------------------------------------
total:       90803052544 |     22168714 |     721.64 |  184739.27 |    0.121 |     0.181

Write IO
thread |       bytes     |     I/Os     |    MiB/s   |  I/O per s |  AvgLat  | LatStdDev |  file
-----------------------------------------------------------------------------------------------------
     0 |      5101953024 |      1245594 |      40.55 |   10379.95 |    0.135 |     0.229 | I:\testfile.dat (6144MiB)
     1 |      5463724032 |      1333917 |      43.42 |   11115.97 |    0.127 |     0.182 | I:\testfile.dat (6144MiB)
     2 |      6015291392 |      1468577 |      47.81 |   12238.14 |    0.115 |     0.163 | I:\testfile.dat (6144MiB)
     3 |      6107869184 |      1491179 |      48.54 |   12426.49 |    0.113 |     0.175 | I:\testfile.dat (6144MiB)
-----------------------------------------------------------------------------------------------------
total:       22688837632 |      5539267 |     180.31 |   46160.55 |    0.122 |     0.187



total:
  %-ile |  Read (ms) | Write (ms) | Total (ms)
----------------------------------------------
    min |      0.065 |      0.076 |      0.065
   25th |      0.104 |      0.105 |      0.104
   50th |      0.106 |      0.107 |      0.106
   75th |      0.112 |      0.113 |      0.112
   90th |      0.129 |      0.131 |      0.129
   95th |      0.158 |      0.160 |      0.159
   99th |      0.294 |      0.299 |      0.294
3-nines |      1.925 |      1.928 |      1.926
4-nines |      7.371 |      7.465 |      7.400
5-nines |     19.236 |     20.042 |     19.319
6-nines |     30.519 |     42.370 |     42.162
7-nines |     48.470 |     48.533 |     48.496
8-nines |     48.497 |     48.533 |     48.533
9-nines |     48.497 |     48.533 |     48.533
    max |     48.497 |     48.533 |     48.533

Javik50k · Dec 5, 2019

Does anybody know, can this be used for caching under windows? These are cheap, and have nice performance as a cache - 8GB and about 1m IOPS. Planning to get one, but i'm afraid this will be useless piece of hardware (will use it in my Windows NAS anyway, if it'll fail in workstation).

i386 · Dec 5, 2019

These are "regular" nvme devices, you can use them as cache in storage spaces or other software raid implementations. Or as a small storage device for high performance (if your application needs less than 8gb space).

Javik50k · Dec 6, 2019

i386 said:
These are "regular" nvme devices, you can use them as cache in storage spaces or other software raid implementations. Or as a small storage device for high performance (if your application needs less than 8gb space).

Thank you! That's exactly i want to do - cache my disk system. Maybe i should start new thread, but what about this card? PMC NVDRAM PCA-00364-03-C NVRAM Card TCA-00364-04-C | eBay

That is Microsemi NV-1616 really and it is NVMe compatible.

i386 · Dec 6, 2019

You will need the additional flash Module for powerloss protection with the microsemi Card.
In nvme Mode IT should Work Like a normal Block device. The microsemi Card also Supports a dma Mode which allows Up to 10kk iops, but requires os and Application compatility.

Javik50k · Dec 6, 2019

Seems i'll need only a supercap. This listing include flash memory module. Also where can i find specs for this? Microsemi wants login.

MiniKnight · Dec 6, 2019

@Javik50k new thread! I want to track what you find.

Javik50k · Dec 6, 2019

MiniKnight said:
@Javik50k new thread! I want to track what you find.

I'm awaiting seller's answer to my questions. Maybe it's defective or something else bad. I'll let you ppl know if and when i'll buy it. If everything will be ok, i promise, i'll post test results.

Upd: here - https://forums.servethehome.com/index.php?threads/pmc-pca-00364-03-c-nvram-accelerator.26652/

i386 · Dec 17, 2019

Flashtec 16gb Prototype:
https://forums.servethehome.com/ind...364-03-c-nvram-accelerator.26652/#post-247192

vs.

RMS-200/8G

CPU: Xeon E5 1620 v3
OS: Server 2019

Javik50k · Dec 17, 2019

Remember, i'm using prototype version. I don't know what differ my card from retail. Firmware may be incomplete and buggy. But i will try to reflash it to latest NV1616 firmware. If only i'll be able to get that firmware at all. Microsemi is very elusive.

ky56 · Dec 1, 2021

How loud is the fan? Does it get very hot? I'm trying to balance performance while keeping my build as quiet as possible.
Would it be possible to take the cover off and cool with external fan?

i386 · Dec 1, 2021

I put one in a fujitsu workstation (with just two fans) at work, the fan of the rms-200 was audible from ~6meters away...

zack$ · Dec 2, 2021

ky56 said:
How loud is the fan? Does it get very hot? I'm trying to balance performance while keeping my build as quiet as possible.
Would it be possible to take the cover off and cool with external fan?

It does get very hot....Had it in a 2U with a modified 80mm fan wall and it kept getting so hot that it would lock up..Tried removing the internal fan and placing 60mm fans on it but that didn't help at all. The 60mm fans were placed alongside the entire card.

Ended up having to put it in a bigger case with 120/140mm fans to kept it cool and conserve on the noise.

These things need a lot of airflow.

clusty · Nov 29, 2024

Monoman said:

RMS200/8gb

my FIO results for 100% read

Code:

fio --filename=/dev/nvme1n1 --direct=1 --rw=randrw --refill_buffers --norandommap --randrepeat=0 --ioengine=libaio --bs=4k --rwmixread=100 --iodepth=16 --numjobs=16 --runtime=60 --group_reporting --name=4ktest

Code:

4ktest: (groupid=0, jobs=16): err= 0: pid=26217: Wed May  1 14:02:21 2019
   read: IOPS=1075k, BW=4199MiB/s (4403MB/s)(128GiB/31187msec)
    slat (nsec): min=1360, max=44117k, avg=2873.53, stdev=45201.30
    clat (usec): min=13, max=47185, avg=232.32, stdev=185.59
     lat (usec): min=16, max=47188, avg=235.54, stdev=190.99
    clat percentiles (usec):
     |  1.00th=[  139],  5.00th=[  151], 10.00th=[  153], 20.00th=[  163],
     | 30.00th=[  233], 40.00th=[  251], 50.00th=[  255], 60.00th=[  258],
     | 70.00th=[  258], 80.00th=[  260], 90.00th=[  262], 95.00th=[  265],
     | 99.00th=[  273], 99.50th=[  289], 99.90th=[  465], 99.95th=[  627],
     | 99.99th=[ 3032]
   bw (  KiB/s): min=218400, max=276024, per=6.30%, avg=270689.97, stdev=4749.32, samples=984
   iops        : min=54600, max=69006, avg=67672.44, stdev=1187.32, samples=984
  lat (usec)   : 20=0.01%, 50=0.01%, 100=0.15%, 250=36.53%, 500=63.23%
  lat (usec)   : 750=0.05%, 1000=0.01%
  lat (msec)   : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
  cpu          : usr=10.97%, sys=33.34%, ctx=17386180, majf=0, minf=193
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwt: total=33521664,0,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=4199MiB/s (4403MB/s), 4199MiB/s-4199MiB/s (4403MB/s-4403MB/s), io=128GiB (137GB), run=31187-31187msec

Disk stats (read/write):
  nvme1n1: ios=33511793/0, merge=0/0, ticks=7615258/0, in_queue=0, util=99.73%

my FIO results for 100% write

Code:

fio --filename=/dev/nvme1n1 --direct=1 --rw=randrw --refill_buffers --norandommap --randrepeat=0 --ioengine=libaio --bs=4k --rwmixread=0 --iodepth=16 --numjobs=16 --runtime=60 --group_reporting --name=4ktest

Code:

4ktest: (groupid=0, jobs=16): err= 0: pid=26639: Wed May  1 14:03:49 2019
  write: IOPS=1196k, BW=4672MiB/s (4899MB/s)(128GiB/28025msec)
    slat (nsec): min=1380, max=33235k, avg=2907.74, stdev=37783.53
    clat (usec): min=13, max=33387, avg=207.07, stdev=155.30
     lat (usec): min=17, max=33393, avg=210.34, stdev=159.82
    clat percentiles (usec):
     |  1.00th=[  147],  5.00th=[  163], 10.00th=[  172], 20.00th=[  186],
     | 30.00th=[  200], 40.00th=[  206], 50.00th=[  212], 60.00th=[  215],
     | 70.00th=[  219], 80.00th=[  223], 90.00th=[  227], 95.00th=[  233],
     | 99.00th=[  253], 99.50th=[  281], 99.90th=[  465], 99.95th=[  627],
     | 99.99th=[ 3228]
   bw (  KiB/s): min=287752, max=308584, per=6.31%, avg=302014.60, stdev=2526.42, samples=880
   iops        : min=71938, max=77146, avg=75503.64, stdev=631.60, samples=880
  lat (usec)   : 20=0.01%, 50=0.01%, 100=0.12%, 250=98.79%, 500=1.01%
  lat (usec)   : 750=0.04%, 1000=0.01%
  lat (msec)   : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
  cpu          : usr=19.55%, sys=35.26%, ctx=13866882, majf=0, minf=166
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwt: total=0,33521664,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
  WRITE: bw=4672MiB/s (4899MB/s), 4672MiB/s-4672MiB/s (4899MB/s-4899MB/s), io=128GiB (137GB), run=28025-28025msec

Disk stats (read/write):
  nvme1n1: ios=258/33496332, merge=0/0, ticks=19/6686278, in_queue=0, util=99.68%

A bit of a necropost I know...
I just got mine and i am not able to get more than 2.5GB/s in write?
Also, what is the noise level on this thing ? My fan is always running and being quite loud: maybe bearings start dying.

ky56 · Nov 29, 2024

clusty said:
A bit of a necropost I know...
I just got mine and i am not able to get more than 2.5GB/s in write?
Also, what is the noise level on this thing ? My fan is always running and being quite loud: maybe bearings start dying.

It might be because it's overheating. Check lmsensors output for nvme temperature. "sensors" command.

I think that fan is always ruined by the time any of us get it. Mine seem to fall out and jam up due to the bearings being that worn out.

If quiet is your goal the best thing to do is to is start by removing all but the 3 screws that hold the heatsink on to the PCB. Then deshroud the card by carefully wriggling out the insides making sure you don't break the two nylon supports that hold the capacitor bank in place. Then carefully cover the capacitor bank with kapton tape. Next desolder or cut the fan wires close to the PCB. Use one brass washer on the long screw and ~7 washers on the short screw to space out the original screws for the pcie bracket and if you're using it in a full height system find another bracket with the same hole spacing. I used one off a spare card I didn't need. Make sure one of the system fans is pointing at the card and you should be good to go.

The card is designed with a shroud inorder to protect the temperature sensitive capacitor bank. It uses a TEC to cool the capacitors which is why is ok and best to cover them. My advice on removing the shroud assumes you have good cooling and the air that reaches it is not very hot. The original intent of the shroud is to shield the capacitor bank from the hot air that would be at the back of a blade server after the air has been heated by everything like the CPU cooler.

You could awlays replace the fan instead but it would still be noisy and that's unacceptable for me however if you would rather do that I believe I found the replacement fan on Aliexpress. I didn't buy or try this so if you do let me know if it worked out.

NEW YOUNG LIN DFB400712H 4CM Ball bearing Ultra thin 4007 cooling fan

clusty · Nov 29, 2024

Thanks.

It was steady around 44-45c so not crazy hot.
I opened it up to take a peek and saw just what you described

I was planning on putting a noctua fan. Only issue is that it’s 10mm deep instead of 7: I imagine I can “lose” 3mm as the PCI boards are not stacked fully on top of each other. If this fails, will go your route and cut the wires and deshrouding it.

Some capacitors explode when they get old: wonder if the shroud does not also protect nearby boards

the original fan feels very noisy ( even new ). Spec mentions 7k rpm and 30db. The noctuas can be throttled down to 2k I think with 5db noise

ky56 · Nov 29, 2024

clusty said:
Thanks.

It was steady around 44-45c so not crazy hot.
I opened it up to take a peek and saw just what you described
I was planning on putting a noctua fan. Only issue is that it’s 10mm deep instead of 7: I imagine I can “lose” 3mm as the PCI boards are not stacked fully on top of each other. If this fails, will go your route and cut the wires and deshrouding it.

Some capacitors explode when they get old: wonder if the shroud does not also protect nearby boards

the original fan feels very noisy ( even new ). Spec mentions 7k rpm and 30db. The noctuas can be throttled down to 2k I think with 5db noise

I considered a noctua fan and ran in to the 7 vs 10mm fan problem. I have have 6 of the 7 PCIe slots with cards so suffice to say I really needed it to fit in one card slot.

Good point on the capacitors. I really need to try and track down the monitoring/config software. I think someone on this forum previous talked about or linked to it but I couldn't find a working link.

[ebay BO] RMS-200/8G PCI-e NVRAM Accelerator 360$

Active Member

Active Member

New Member

Well-Known Member

Member

Well-Known Member

Member

Well-Known Member

Member

Well-Known Member

Member

Well-Known Member

Member

New Member

Well-Known Member

Well-Known Member

New Member

New Member

New Member

New Member