tl;dr - Disk usage is 100% in Windows 10 Task Manager, System memory shoots up, whenever doing disk-intensive write operations that are well within the capabilities of the RAID in question.
First a bit of background: We have an 8-disk RAID 5 that's inside of a motion picture scanner's host PC. The RAID controller is a Highpoint RocketRaid 2720, which we've used for many years. We were able to consistently get about 1200MB/sec when testing using the AJA Disk Speed utility, which is designed to simulate the kinds of files we work with (image sequences, one per frame, at various resolutions and bit depths). For many years this system ran fine, under Windows 7.
Last year we upgraded the scanner, and with that came a new PC. It runs Win10. When we got the system, I set up a new RAID, because we kept the old PC around since there were many jobs on that PC's array. The new system uses the Highpoint card. At first I used an LSI card, but it was overheating due to the dual GPUs in the new PC. The Highpoint is much smaller and simpler, and doesn't overheat. In any case, we had an issue with some drives where the scanner would get ahead of the speed at which the disks could write. Normally the scanner software understands this, and stops. Instead, the operating system's memory usage (according to the Task Manager) starts to shoot up when the disk array is saturated. It gets to 85%, the software stops scanning, and then we have to wait a long time for the data to be written from system memory to the drive.
The scanner starts to capture again, but now, the max speed the disks are writing at is much lower - under 100MB/s - so the only solution is to slow the scanner *way* down to reduce the throughput significantly. We thought the problem was with the drives. When the scanner was upgraded, I purchased some Seagate Terascale disks, which I'd never used before. The scanner manufacturer and I assumed that the problem was that the big caches in these disks were filling up, and that the actual native write speed is probably pretty low, which accounted for the fluctuations we were seeing.
Fast forward to a couple weeks ago, and one of those disks failed. I decided to use the opportunity to replace them all with faster disks (8x Toshiba N300 NAS drives, which we've used successfully in other arrays). Everything seemed good. I was able to get about 1350MB/s on my tests, and when scanning the problem went away.
Then I started having the same issue as described above, last week. I found that sometimes rebooting the PC fixed the problem, but it eventually came back. Today, it happened immediately after a reboot. I'm doing a scan that only requires about 450MB/s throughput, which isn't all that much, and this array should easily handle that. But it's not.
So the question now is - is this a problem with the RAID card? With Windows 10?
We have 8-disk RAIDs all over the office that we pound on daily, that don't show this issue, but most of our machines are either Windows 7, MacPro, or Linux boxes. We only have a couple Win10 machines (which I absolutely hate, for a variety of reasons).
Any ideas?
First a bit of background: We have an 8-disk RAID 5 that's inside of a motion picture scanner's host PC. The RAID controller is a Highpoint RocketRaid 2720, which we've used for many years. We were able to consistently get about 1200MB/sec when testing using the AJA Disk Speed utility, which is designed to simulate the kinds of files we work with (image sequences, one per frame, at various resolutions and bit depths). For many years this system ran fine, under Windows 7.
Last year we upgraded the scanner, and with that came a new PC. It runs Win10. When we got the system, I set up a new RAID, because we kept the old PC around since there were many jobs on that PC's array. The new system uses the Highpoint card. At first I used an LSI card, but it was overheating due to the dual GPUs in the new PC. The Highpoint is much smaller and simpler, and doesn't overheat. In any case, we had an issue with some drives where the scanner would get ahead of the speed at which the disks could write. Normally the scanner software understands this, and stops. Instead, the operating system's memory usage (according to the Task Manager) starts to shoot up when the disk array is saturated. It gets to 85%, the software stops scanning, and then we have to wait a long time for the data to be written from system memory to the drive.
The scanner starts to capture again, but now, the max speed the disks are writing at is much lower - under 100MB/s - so the only solution is to slow the scanner *way* down to reduce the throughput significantly. We thought the problem was with the drives. When the scanner was upgraded, I purchased some Seagate Terascale disks, which I'd never used before. The scanner manufacturer and I assumed that the problem was that the big caches in these disks were filling up, and that the actual native write speed is probably pretty low, which accounted for the fluctuations we were seeing.
Fast forward to a couple weeks ago, and one of those disks failed. I decided to use the opportunity to replace them all with faster disks (8x Toshiba N300 NAS drives, which we've used successfully in other arrays). Everything seemed good. I was able to get about 1350MB/s on my tests, and when scanning the problem went away.
Then I started having the same issue as described above, last week. I found that sometimes rebooting the PC fixed the problem, but it eventually came back. Today, it happened immediately after a reboot. I'm doing a scan that only requires about 450MB/s throughput, which isn't all that much, and this array should easily handle that. But it's not.
So the question now is - is this a problem with the RAID card? With Windows 10?
We have 8-disk RAIDs all over the office that we pound on daily, that don't show this issue, but most of our machines are either Windows 7, MacPro, or Linux boxes. We only have a couple Win10 machines (which I absolutely hate, for a variety of reasons).
Any ideas?