[ebay BO] RMS-200/8G PCI-e NVRAM Accelerator 360$

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Monoman

Active Member
Oct 16, 2013
433
190
43
FINALLY got mine installed on my proxmox host.

dual e5 2697v2
256gb ram
24x 200gb s3700 (off expander) to sas3 hba
RMS-200 8gb

Here's pveperf on just the drive.
Code:
root@pve1:/mnt# pveperf .

CPU BOGOMIPS:      259232.64
REGEX/SECOND:      1545679
HD SIZE:           7.80 GB (/dev/nvme0n1p1)
BUFFERED READS:    1723.78 MB/sec
AVERAGE SEEK TIME: 0.02 ms
FSYNCS/SECOND:     6827.35
DNS EXT:           43.92 ms
DNS INT:           57.11 ms (m0n0mana.local.lan)

Here's pve perv on my array with the RMS-200 with sync=standard
Code:
root@pve1:~# zfs set sync=standard s3700_pool
root@pve1:~# zfs get sync s3700_pool
NAME        PROPERTY  VALUE     SOURCE
s3700_pool  sync      standard  local
root@pve1:~# pveperf /s3700_pool/dir/
CPU BOGOMIPS:      259232.64
REGEX/SECOND:      1535681
HD SIZE:           2322.21 GB (s3700_pool)
FSYNCS/SECOND:     3805.59
DNS EXT:           42.72 ms
DNS INT:           58.03 ms (m0n0mana.local.lan)

Here's pve perv on my array with the RMS-200 with sync=always
Code:
root@pve1:~# zfs set sync=always s3700_pool
root@pve1:~# zfs get sync s3700_pool
NAME        PROPERTY  VALUE     SOURCE
s3700_pool  sync      always    local
root@pve1:~# pveperf /s3700_pool/dir/
CPU BOGOMIPS:      259232.64
REGEX/SECOND:      1555953
HD SIZE:           2322.21 GB (s3700_pool)
FSYNCS/SECOND:     2578.80
DNS EXT:           43.10 ms
DNS INT:           54.00 ms (m0n0mana.local.lan)
 
  • Like
Reactions: Patrick

Monoman

Active Member
Oct 16, 2013
433
190
43
P.S. It's quite a bit faster than when I used my optane (don't have the numbers off hand)
 
  • Like
Reactions: rshakin

i386

Well-Known Member
Mar 18, 2016
4,588
1,737
113
35
Germany
I got 3 of these devices for ~73£ each on ebay :D

Some windows diskspd numbers:
Code:
diskspd -b4K -c6G -d120 -L -o8 -r -Sh -t4 -w20 I:\testfile.dat
Code:
Input parameters:

   timespan:   1
   -------------
   duration: 120s
   warm up time: 5s
   cool down time: 0s
   measuring latency
   random seed: 0
   path: 'I:\testfile.dat'
       think time: 0ms
       burst size: 0
       software cache disabled
       hardware write cache disabled, writethrough on
       performing mix test (read/write ratio: 80/20)
       block size: 4096
       using random I/O (alignment: 4096)
       number of outstanding I/O operations: 8
       thread stride size: 0
       threads per file: 4
       using I/O Completion Ports
       IO priority: normal
Code:
CPU |  Usage |  User  |  Kernel |  Idle
-------------------------------------------
   0|  99.96%|  13.66%|   86.30%|   0.04%
   1| 100.00%|  10.25%|   89.75%|   0.00%
   2|  99.97%|   8.78%|   91.20%|   0.03%
   3|  99.96%|   9.10%|   90.86%|   0.04%
   4|  29.57%|  19.64%|    9.93%|  70.43%
   5|  29.66%|  21.95%|    7.71%|  70.34%
   6|  33.55%|  24.04%|    9.52%|  66.45%
   7|  30.63%|  22.43%|    8.19%|  69.38%
-------------------------------------------
avg.|  65.41%|  16.23%|   49.18%|  34.59%

Total IO
thread |       bytes     |     I/Os     |    MiB/s   |  I/O per s |  AvgLat  | LatStdDev |  file
-----------------------------------------------------------------------------------------------------
     0 |     25525678080 |      6231855 |     202.86 |   51932.12 |    0.134 |     0.237 | I:\testfile.dat (6144MiB)
     1 |     27306102784 |      6666529 |     217.01 |   55554.40 |    0.126 |     0.173 | I:\testfile.dat (6144MiB)
     2 |     30074277888 |      7342353 |     239.01 |   61186.27 |    0.114 |     0.153 | I:\testfile.dat (6144MiB)
     3 |     30585831424 |      7467244 |     243.07 |   62227.03 |    0.112 |     0.162 | I:\testfile.dat (6144MiB)
-----------------------------------------------------------------------------------------------------
total:      113491890176 |     27707981 |     901.95 |  230899.82 |    0.121 |     0.182

Read IO
thread |       bytes     |     I/Os     |    MiB/s   |  I/O per s |  AvgLat  | LatStdDev |  file
-----------------------------------------------------------------------------------------------------
     0 |     20423725056 |      4986261 |     162.31 |   41552.17 |    0.134 |     0.239 | I:\testfile.dat (6144MiB)
     1 |     21842378752 |      5332612 |     173.59 |   44438.43 |    0.125 |     0.171 | I:\testfile.dat (6144MiB)
     2 |     24058986496 |      5873776 |     191.20 |   48948.13 |    0.114 |     0.151 | I:\testfile.dat (6144MiB)
     3 |     24477962240 |      5976065 |     194.53 |   49800.54 |    0.112 |     0.158 | I:\testfile.dat (6144MiB)
-----------------------------------------------------------------------------------------------------
total:       90803052544 |     22168714 |     721.64 |  184739.27 |    0.121 |     0.181

Write IO
thread |       bytes     |     I/Os     |    MiB/s   |  I/O per s |  AvgLat  | LatStdDev |  file
-----------------------------------------------------------------------------------------------------
     0 |      5101953024 |      1245594 |      40.55 |   10379.95 |    0.135 |     0.229 | I:\testfile.dat (6144MiB)
     1 |      5463724032 |      1333917 |      43.42 |   11115.97 |    0.127 |     0.182 | I:\testfile.dat (6144MiB)
     2 |      6015291392 |      1468577 |      47.81 |   12238.14 |    0.115 |     0.163 | I:\testfile.dat (6144MiB)
     3 |      6107869184 |      1491179 |      48.54 |   12426.49 |    0.113 |     0.175 | I:\testfile.dat (6144MiB)
-----------------------------------------------------------------------------------------------------
total:       22688837632 |      5539267 |     180.31 |   46160.55 |    0.122 |     0.187



total:
  %-ile |  Read (ms) | Write (ms) | Total (ms)
----------------------------------------------
    min |      0.065 |      0.076 |      0.065
   25th |      0.104 |      0.105 |      0.104
   50th |      0.106 |      0.107 |      0.106
   75th |      0.112 |      0.113 |      0.112
   90th |      0.129 |      0.131 |      0.129
   95th |      0.158 |      0.160 |      0.159
   99th |      0.294 |      0.299 |      0.294
3-nines |      1.925 |      1.928 |      1.926
4-nines |      7.371 |      7.465 |      7.400
5-nines |     19.236 |     20.042 |     19.319
6-nines |     30.519 |     42.370 |     42.162
7-nines |     48.470 |     48.533 |     48.496
8-nines |     48.497 |     48.533 |     48.533
9-nines |     48.497 |     48.533 |     48.533
    max |     48.497 |     48.533 |     48.533
 
  • Like
Reactions: abq and MiniKnight

Javik50k

Member
Oct 11, 2018
52
40
18
Does anybody know, can this be used for caching under windows? These are cheap, and have nice performance as a cache - 8GB and about 1m IOPS. Planning to get one, but i'm afraid this will be useless piece of hardware (will use it in my Windows NAS anyway, if it'll fail in workstation).
 
  • Like
Reactions: Samir

i386

Well-Known Member
Mar 18, 2016
4,588
1,737
113
35
Germany
These are "regular" nvme devices, you can use them as cache in storage spaces or other software raid implementations. Or as a small storage device for high performance (if your application needs less than 8gb space).
 
  • Like
Reactions: Samir

Javik50k

Member
Oct 11, 2018
52
40
18
These are "regular" nvme devices, you can use them as cache in storage spaces or other software raid implementations. Or as a small storage device for high performance (if your application needs less than 8gb space).
Thank you! That's exactly i want to do - cache my disk system. Maybe i should start new thread, but what about this card? PMC NVDRAM PCA-00364-03-C NVRAM Card TCA-00364-04-C | eBay

That is Microsemi NV-1616 really and it is NVMe compatible.
 
  • Like
Reactions: Samir

i386

Well-Known Member
Mar 18, 2016
4,588
1,737
113
35
Germany
You will need the additional flash Module for powerloss protection with the microsemi Card.
In nvme Mode IT should Work Like a normal Block device. The microsemi Card also Supports a dma Mode which allows Up to 10kk iops, but requires os and Application compatility.
 
  • Like
Reactions: Samir

Javik50k

Member
Oct 11, 2018
52
40
18
Seems i'll need only a supercap. This listing include flash memory module. Also where can i find specs for this? Microsemi wants login.
 
  • Like
Reactions: Samir

Javik50k

Member
Oct 11, 2018
52
40
18
Last edited:
  • Like
Reactions: Patrick and Samir

Javik50k

Member
Oct 11, 2018
52
40
18
Remember, i'm using prototype version. I don't know what differ my card from retail. Firmware may be incomplete and buggy. But i will try to reflash it to latest NV1616 firmware. If only i'll be able to get that firmware at all. Microsemi is very elusive.
 

ky56

New Member
Jan 5, 2021
21
3
3
How loud is the fan? Does it get very hot? I'm trying to balance performance while keeping my build as quiet as possible.
Would it be possible to take the cover off and cool with external fan?
 

i386

Well-Known Member
Mar 18, 2016
4,588
1,737
113
35
Germany
I put one in a fujitsu workstation (with just two fans) at work, the fan of the rms-200 was audible from ~6meters away...
 

zack$

Well-Known Member
Aug 16, 2018
738
383
63
How loud is the fan? Does it get very hot? I'm trying to balance performance while keeping my build as quiet as possible.
Would it be possible to take the cover off and cool with external fan?
It does get very hot....Had it in a 2U with a modified 80mm fan wall and it kept getting so hot that it would lock up..Tried removing the internal fan and placing 60mm fans on it but that didn't help at all. The 60mm fans were placed alongside the entire card.

Ended up having to put it in a bigger case with 120/140mm fans to kept it cool and conserve on the noise.

These things need a lot of airflow.
 

clusty

New Member
Nov 29, 2024
5
2
3
RMS200/8gb

my FIO results for 100% read
Code:
fio --filename=/dev/nvme1n1 --direct=1 --rw=randrw --refill_buffers --norandommap --randrepeat=0 --ioengine=libaio --bs=4k --rwmixread=100 --iodepth=16 --numjobs=16 --runtime=60 --group_reporting --name=4ktest
Code:
4ktest: (groupid=0, jobs=16): err= 0: pid=26217: Wed May  1 14:02:21 2019
   read: IOPS=1075k, BW=4199MiB/s (4403MB/s)(128GiB/31187msec)
    slat (nsec): min=1360, max=44117k, avg=2873.53, stdev=45201.30
    clat (usec): min=13, max=47185, avg=232.32, stdev=185.59
     lat (usec): min=16, max=47188, avg=235.54, stdev=190.99
    clat percentiles (usec):
     |  1.00th=[  139],  5.00th=[  151], 10.00th=[  153], 20.00th=[  163],
     | 30.00th=[  233], 40.00th=[  251], 50.00th=[  255], 60.00th=[  258],
     | 70.00th=[  258], 80.00th=[  260], 90.00th=[  262], 95.00th=[  265],
     | 99.00th=[  273], 99.50th=[  289], 99.90th=[  465], 99.95th=[  627],
     | 99.99th=[ 3032]
   bw (  KiB/s): min=218400, max=276024, per=6.30%, avg=270689.97, stdev=4749.32, samples=984
   iops        : min=54600, max=69006, avg=67672.44, stdev=1187.32, samples=984
  lat (usec)   : 20=0.01%, 50=0.01%, 100=0.15%, 250=36.53%, 500=63.23%
  lat (usec)   : 750=0.05%, 1000=0.01%
  lat (msec)   : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
  cpu          : usr=10.97%, sys=33.34%, ctx=17386180, majf=0, minf=193
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwt: total=33521664,0,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=4199MiB/s (4403MB/s), 4199MiB/s-4199MiB/s (4403MB/s-4403MB/s), io=128GiB (137GB), run=31187-31187msec

Disk stats (read/write):
  nvme1n1: ios=33511793/0, merge=0/0, ticks=7615258/0, in_queue=0, util=99.73%
my FIO results for 100% write
Code:
fio --filename=/dev/nvme1n1 --direct=1 --rw=randrw --refill_buffers --norandommap --randrepeat=0 --ioengine=libaio --bs=4k --rwmixread=0 --iodepth=16 --numjobs=16 --runtime=60 --group_reporting --name=4ktest
Code:
4ktest: (groupid=0, jobs=16): err= 0: pid=26639: Wed May  1 14:03:49 2019
  write: IOPS=1196k, BW=4672MiB/s (4899MB/s)(128GiB/28025msec)
    slat (nsec): min=1380, max=33235k, avg=2907.74, stdev=37783.53
    clat (usec): min=13, max=33387, avg=207.07, stdev=155.30
     lat (usec): min=17, max=33393, avg=210.34, stdev=159.82
    clat percentiles (usec):
     |  1.00th=[  147],  5.00th=[  163], 10.00th=[  172], 20.00th=[  186],
     | 30.00th=[  200], 40.00th=[  206], 50.00th=[  212], 60.00th=[  215],
     | 70.00th=[  219], 80.00th=[  223], 90.00th=[  227], 95.00th=[  233],
     | 99.00th=[  253], 99.50th=[  281], 99.90th=[  465], 99.95th=[  627],
     | 99.99th=[ 3228]
   bw (  KiB/s): min=287752, max=308584, per=6.31%, avg=302014.60, stdev=2526.42, samples=880
   iops        : min=71938, max=77146, avg=75503.64, stdev=631.60, samples=880
  lat (usec)   : 20=0.01%, 50=0.01%, 100=0.12%, 250=98.79%, 500=1.01%
  lat (usec)   : 750=0.04%, 1000=0.01%
  lat (msec)   : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
  cpu          : usr=19.55%, sys=35.26%, ctx=13866882, majf=0, minf=166
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwt: total=0,33521664,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
  WRITE: bw=4672MiB/s (4899MB/s), 4672MiB/s-4672MiB/s (4899MB/s-4899MB/s), io=128GiB (137GB), run=28025-28025msec

Disk stats (read/write):
  nvme1n1: ios=258/33496332, merge=0/0, ticks=19/6686278, in_queue=0, util=99.68%
A bit of a necropost I know...
I just got mine and i am not able to get more than 2.5GB/s in write?
Also, what is the noise level on this thing ? My fan is always running and being quite loud: maybe bearings start dying.
 

ky56

New Member
Jan 5, 2021
21
3
3
A bit of a necropost I know...
I just got mine and i am not able to get more than 2.5GB/s in write?
Also, what is the noise level on this thing ? My fan is always running and being quite loud: maybe bearings start dying.
It might be because it's overheating. Check lmsensors output for nvme temperature. "sensors" command.

I think that fan is always ruined by the time any of us get it. Mine seem to fall out and jam up due to the bearings being that worn out.

If quiet is your goal the best thing to do is to is start by removing all but the 3 screws that hold the heatsink on to the PCB. Then deshroud the card by carefully wriggling out the insides making sure you don't break the two nylon supports that hold the capacitor bank in place. Then carefully cover the capacitor bank with kapton tape. Next desolder or cut the fan wires close to the PCB. Use one brass washer on the long screw and ~7 washers on the short screw to space out the original screws for the pcie bracket and if you're using it in a full height system find another bracket with the same hole spacing. I used one off a spare card I didn't need. Make sure one of the system fans is pointing at the card and you should be good to go.

The card is designed with a shroud inorder to protect the temperature sensitive capacitor bank. It uses a TEC to cool the capacitors which is why is ok and best to cover them. My advice on removing the shroud assumes you have good cooling and the air that reaches it is not very hot. The original intent of the shroud is to shield the capacitor bank from the hot air that would be at the back of a blade server after the air has been heated by everything like the CPU cooler.

You could awlays replace the fan instead but it would still be noisy and that's unacceptable for me however if you would rather do that I believe I found the replacement fan on Aliexpress. I didn't buy or try this so if you do let me know if it worked out.

NEW YOUNG LIN DFB400712H 4CM Ball bearing Ultra thin 4007 cooling fan
 
  • Like
Reactions: abq and nexox

clusty

New Member
Nov 29, 2024
5
2
3
Thanks.

It was steady around 44-45c so not crazy hot.
I opened it up to take a peek and saw just what you described :)
I was planning on putting a noctua fan. Only issue is that it’s 10mm deep instead of 7: I imagine I can “lose” 3mm as the PCI boards are not stacked fully on top of each other. If this fails, will go your route and cut the wires and deshrouding it.

Some capacitors explode when they get old: wonder if the shroud does not also protect nearby boards :p

the original fan feels very noisy ( even new ). Spec mentions 7k rpm and 30db. The noctuas can be throttled down to 2k I think with 5db noise
 
  • Like
Reactions: ky56

ky56

New Member
Jan 5, 2021
21
3
3
Thanks.

It was steady around 44-45c so not crazy hot.
I opened it up to take a peek and saw just what you described :)
I was planning on putting a noctua fan. Only issue is that it’s 10mm deep instead of 7: I imagine I can “lose” 3mm as the PCI boards are not stacked fully on top of each other. If this fails, will go your route and cut the wires and deshrouding it.

Some capacitors explode when they get old: wonder if the shroud does not also protect nearby boards :p

the original fan feels very noisy ( even new ). Spec mentions 7k rpm and 30db. The noctuas can be throttled down to 2k I think with 5db noise
I considered a noctua fan and ran in to the 7 vs 10mm fan problem. I have have 6 of the 7 PCIe slots with cards so suffice to say I really needed it to fit in one card slot.

Good point on the capacitors. I really need to try and track down the monitoring/config software. I think someone on this forum previous talked about or linked to it but I couldn't find a working link.