LSI 9260-8i - poor random read/write performance with SSDs?

mrkrad

Well-Known Member
Oct 13, 2012
1,244
52
48
Return the 840 pro and use the proper SSD like server GRADE S3500.

Or just use the intel raid.

You will find your speed either way. Patrick is the owner of this website, you should read the front pages and find the article. He writes excellent articles and you can learn a lot from them.

the HBA should be used in IT mode only.

If you need IR - buy Megaraid controllers.

I bet you would get better performance in IT mode with software raid-0 almost as good as software raid-0 with intel ports :)
 

iq100

Member
Jun 5, 2012
66
3
8
MiniKnight wrote>"... I don't have the link handy but thought it was a forum post here? Also iirc it was resolved using iometer? "

I think this was the link: http://www.servethehome.com/lsi-sas-2308-hba-ssd-windows-server-2012-turn-write-cache/
Patrick's only reference to IOMeter was>"... I did want to point out that IOMeter managed to provide almost 17K on QD1 and over 40K on QD4 sequential 4K write tests so the above seems correct. ..."
---------------------------------------------------------------------------------------------------
Interesting quotes are:
"... flashed to IT mode, for the results displayed below. IR mode showed very similar results. ...". That contradicts mrkrad "... the HBA should be used in IT mode only ..." ??
" 840 Pro Anvil 4K Writes (I assume these are QD=1) went from approx. 2MByes/sec to approx. 66MBytes/sec by checking Anvil's>Setting>Write Through box.

My 2MBbyte/sec score does not change whether Anvil's Write Through box is checked or unchecked.
Question: Why should Write Through be so much faster. I thought the Write Back is the conceptually faster option, as it would allow Samsung's internal 512MB Ram and controller to do its own thing and hopefully overlap writes using its multichannel write capability?
--------------------------------------------------------------------------------------------------
mrkad wrote>"... Return the 840 pro and use the proper SSD like server GRADE S3500 ..."

That is not a possibility. Note than anandtech measured its 4KB Random Write as 47.49MBytes/sec.
AnandTech | Intel SSD DC S3500 Review (480GB): Part 1 - Print View
---------------------------------
mrkrad also wrote>"... I bet you would get better performance in IT mode with software raid-0 almost as good as software raid-0 with intel ports ..."

That would mean using Microsoft Disk Manager's dynamic/striped disk capability for my two Samsung 840Pros. BUT THAT WOULD MEAN NOT BOOTABLE. Unless someone can provide how to make Window Disk Manager dynamic/striped raid0 bootable, that is a deal breaker?

Also, I suspect the LSI sata3 provided raido sequential read rate would drop from approx. 1GByte/sec to 512MByte/sec, if I abandoned the LSI 9212 4i43 HBA and used Dell T110 sata2 3gbps for Windows Disk Manager raid0 using dynamic/striped volume.
--------------------------------------------------------------------------------------------------
mrkrad also wrote>"... the HBA should be used in IT mode only.

According to Patrick article linked above "... IR mode showed very similar results ..."
--------------------------------------------------------------------------------------------------
I am waiting for LSI to deliver on their promise to provide an ECN that fixes what LSI admitted via email is a defect effecting LSI HBAs in regard to 4K, QD=1, Writes.
If anyone as any ideas why Anvil write through setting made a substantial difference for Patricks testing, but no difference for my P16 IR Mode LSI 9212 4i4e HBA, I am all ears.

Thanks to STH for sometimes being the cutting edge on these things :).
 
Last edited:

mrkrad

Well-Known Member
Oct 13, 2012
1,244
52
48
so yeah you notice how many people (google samsung 840 raid-0) use the intel software raid? t
 

littlejojo

New Member
Jul 7, 2013
14
0
0
Hello Mr. iq100
Just FYI:

I just have LSI 9212-4i4e HBA card,
my system is: HP Z800, 24.Gb Ram, Nvidia K2000

(1) I connect single Samsung 840 Pro 512 Gb into
port-5 of the HBA as bootable with window7 ultimate NO RAID.
(2) connect second Samsung 840 Pro 256 Gb into port-4 of HBA
again NO-RAID

--> goto window manager I see check mark Enable write caching on the device
I'll be able to toggle (disable/enable) this option
--> Sorry I don't know how to attack image so you can see it
--> talking about benchmark: for samsung 840 pro they both similar
number:
Read Write
Seq 524.47 MB/s 365.38 MB/s
4K 19.59 MB/s 2.35 MB/s
4K-64 Thrd 212.12 MB/s 20.48 MB/s
Acc.time 0.065 ms 1.661 ms
Score: 284 59
487

Best Regards
 

iq100

Member
Jun 5, 2012
66
3
8
mobilenvidia wrote>"You can probably toggle the Cache on/off on older Firmware, LSI disabled it in more recent ones."

How far back do I need to go? P15, P14, P13, or even earlier? Which do you suggest?
All my 9212 4i4e testing has been using LSI P16 IR mode. IR Firmware, BIOS, and Windows 7 Driver downloaded from lsi.com.

As we both know, my LSI HBA has no RAM resource on the PCI-e card. So no ability for PCI-e card resident cache.
You are saying if I use an earlier firmware, and/or BIOS, and/or driver, that LSI BIOS (control-C at boot screen) will have a setting to toggle Cache (meaning what Windows Device Manager Policies Tab calls "write-caching", aka write-back)?
Or, do I use Windows Device Manager to toggle its write-caching box, which with current LSI P16, Windows Device Manager Policies Tab screen says "This device does not allow its write-caching setting to be changed"
Or do I use LSI MSM screens to do that. MSM Logical Tab now shows writes-through and provides no ability to change this?

If you could provide a step by step, starting with which LSI package (Pnn) to use? Do I change firmware, BIOS, and driver? All three?
Do I use LSI BIOS, or MSM, or Windows Device Manger to "toggle the Cache on/off"?
Thanks!
 
Last edited:

iq100

Member
Jun 5, 2012
66
3
8
littlejoho wrote>"... 4K 19.59 MB/s 2.35 MB/s ... "

My two Samsung 840 Pro 256GB, LSI raid0, 4K Write score in Anvil and AS SSD is also approx. 2.35 MB/s.
LSI technical support, Atlanta, confirms they get similar "abysmal 4K Write" scores with Seagate Enterprise SSDs and has requested an LSI engineering change.
I'll let everyone know when I get the official LSI engineering change to test.
----------------------------
littlejoho also wrote>"... Sorry I don't know how to attack image so you can see it ..."

I use free dropbox.com. I upload the image to dropbox and dropbox provides a link which I paste to STH.
There is an icon above this reply box that says "insert image" and provide a URL box, but that never cause image to appear directly??
Anyone here know how to post an inline image into this reply box?
 
Last edited:

littlejojo

New Member
Jul 7, 2013
14
0
0
Hello iq100
thanks for your info, whenever you have a LSI official fix the random write back issue then please
let us know. Some how there is a conflict between HBA firmware controller and SSD controller, but
for now as long us my PC is running better than legacy hard disk drive for my video editing, I'm ok
with that, but a fix if excellent to boot up SSD as Samsung Spec describred.

Have a good day
 

mobilenvidia

Moderator
Sep 25, 2011
1,767
60
48
New Zealand
MegaCLi version changelog
8.04.08 - Suggested
-------
Fixes:
- Changed the default disk cache policy from "Unchanged" to "Disabled," when
creating a VD in the VD property setting for both HDDs and SSDs.
LSI are determined to disable Disk Cache (not to be confused with write back!)
 
Last edited:

iq100

Member
Jun 5, 2012
66
3
8
mobilenvidia wrote>"... LSI are determined to disable Disk Cache (not to be confused with write back!) ..."

First, this would only apply to LSI cards that have PCIe resident RAM, like the 9260, not the 9212-4i4e. Is that correct mobilenvidia?
Second, why do you think LSI is determined to disable (PCIe resident) Disk Cache? You are the man around here with long history of following these things in detail, so I would appreciate your understanding of these things :).
 
Last edited:

mrkrad

Well-Known Member
Oct 13, 2012
1,244
52
48
iq100 - you notice this happen around the time they bought sandforce, and sandforce drives are not affected?

hmmm...

I wish I could tell you more but i've had samsung consumer engineering with lsi engineering on 3 way call over this. But the details are restricted information sorry.

Hints are the best I can tell you.
 

mobilenvidia

Moderator
Sep 25, 2011
1,767
60
48
New Zealand
mobilenvidia wrote>"... LSI are determined to disable Disk Cache (not to be confused with write back!) ..."

First, this would only apply to LSI cards that have PCIe resident RAM, like the 9260, not the 9212-4i4e. Is that correct mobilenvidia?
No you are still getting this wrong
Write back can only be achieved with Caching controllers ie 9260/9280 etc etc
LSI HBA's 9200/9211/9212 etc etc has no cache and CAN'T do Write back !!
Both controller can only control the Disk cache (cachw on the drive not the LSI controller)
It's the Disk cache LSI is now defaulting to disabled, it used to be 'No change'


Second, why do you think LSI is determined to disable (PCIe resident) Disk Cache? You are the man around here with long history of following these things in detail, so I would appreciate your understanding of these things :).
I can only assume as i have no non Sandforce SSD, that the cache's are getting much smarter.
LSI's algorithms might be getting screwed up with the drive probably outsmarting it
LSI's answer with any issue is to just disable it, look at active drive spin down.
Marketing probably also has a role to play, why make the 840 pro make you own products llook slow, not good for sales.
 

mrkrad

Well-Known Member
Oct 13, 2012
1,244
52
48
The intel datacenter drives are affected by this as well (3700) - since sandforce drives are not affected. this would be a great way to make everyone else drive look like crap and their own chipset look awesome.

That would be silly of them.

however say you have a drive that is not qualified (840 pro was pulled and just this month added back as approved). If the drive was dropping from the array all the time on LSI but not on PMC-adaptec, would that not make your raid card look bad? IT sure would.

The 840 PRO is really not worth the trouble any more. the 830 was a solid drive, it kicked ass. The 840 pro had serious issues out of the door. Serious cachecade problems due to GC.

With Controller cache disabled patrol scan/consistency check would cause massive latency issues on the 9266. It is documented that the only solution to this problem is to enable controller cache. the M1015 is not stable in esxi for this reason. I would never use it with esxi. The 840 pro was unstable with factory OP unlike the 830. It would lockup and require a powerdown to work again. We had to move to 30% OP to get stable 840 pro.

With the intel S3500, the 384gb usable for 840 pro 512gb is not the best deal anymore. The S3500 has more usable per dollar and tantalum capacitor for power loss flush.

As I have said before - the 840 pro is not a good solution with LSI. It requires unacceptable levels of tuning (megascu/etc) to work reliably. The samsung 830 was flawless AS-IS.

If you ask me - would I use the m1015 with esxi? NO.

If you ask me - would I use the 840 pro with LSI? NO.

If you ask me - would I use a cheaper raid controller such as HP P420/1gb FBWC versus LSI 9271, I would say yes [I am hp shop].

I don't blame the chipset, HP has made a fantastic HP Smartarray out of the LSI2308 chipset, so I would say it is a shit firmware and driver.

I have 8 servers with LSI megaraid and 840 PRO - they are solid but took much work. I have 6 servers with P420/1gb FBWC with hard drives (DL180/SE) and they take no work.

Knowing what I know now. I would have ZERO LSI controllers, and ZERO 840 pro ssd if i had to purchase the same today.

You have two problematic devices. The LSI controller and the 840 pro.

Are they defective hardware? Nope.

Are they defective firmware/driver? yup.

Whose fault is it? Does not matter LSI points finger at samsung. Samsung points finger at LSI.

Time is money, why do you spend so much time to many shit work with shit? I have no idea but it is a fantastic waste of money.

Why did I make my megaraid work with 840 pro? Because I bought all of the hardware. The same week they pulled support for 840 pro from LSI. You cannot return 40 SSD after you open them.

Doesn't matter any more - I think based on this forum you can deduce 840 pro is a fail for server. LSI is a fail in general.

Sad since the old combination of LSI 9260 and samsung 830 was bulletproof.
 

iq100

Member
Jun 5, 2012
66
3
8
mobilenvidia wrote"... Both controller can only control the Disk cache (cachw on the drive not the LSI controller) "
'Both' means LSI 9260/9280 'megaraid cards' which have pcie resident RAM?

Some data points:
1- With the Samsung 840 Pro directly connected to Dell T110 server motherboard 3gbps sata2, what Windows 7 Device Manager disk policy tab calls "Enable write caching on the device" is functional.
When enabled, both Anvil and AS SSD show 4K write score of approx. 50 MByte/sec.
When not enabled, the 4K write score becomes approx. 2 MByte/sec

2- With two Samsung 840 Pro connected to two of the LSI 9212 4i4e 6gbps sata3 ports as a raid0 volume, the Anvil and AS SSD scores approx. 2 MByte/sec, just like in 1 above with Enable write caching on the device unchecked (disabled).
MSM 12.05.0300 has two terms of art.
First, MSM>Physical Tab>SSD Disk Setting is "Enabled". MSM GUI offers to change to Disabled, but upon exit from MSM and relaunch of MSM the SSD Disk Setting reverts back to Enabled.
Second MSM>Logical Tab>Write Policy is "Write Through" and MSM offers no ability to change it to Write Back.

Whatever you want to call it, Microsoft has the ability to enable/disable what Microsoft calls 'Enable write caching on the device'.
This is in fact what most refer to as write back, meaning the host does NOT wait for each 4K Write command to be completed by the Samsung 840 Pro subsystem. Samsung 840 Pro has its on 512 MByte Ram and multichannel ability to write to more than one NAND die at one time.
Write through is what most refer to when the host waits for the Samsung 840 Pro to tell the host that each single 4K Write command has been completed before the host system will send the next 4K write command.

mobilenvidia also wrote>"... LSI HBA's 9200/9211/9212 etc etc has no cache and CAN'T do Write back !!"
If you are implying that the Samsung 840 Pro has no theoretical ability to use a write back scheme when connected to LSI HBA PCIe cards (which have no PCIe resident RAM), like the 9212-4i4e, then I respectfully think you are misinformed. Clearly the 4K Write 50 MByte scores Anvil and AS SSD show for the Samsung 840 SSD connected directly to the motherboard PCH show that write back is achievable. LSI HBA should be able to use the Samsung 840 Pro in write back mode just like Intel PCH can.

Here are the MSM screen shots (may be hard to read, but I am certain you know what they are)
http://img713.imageshack.us/img713/3701/9kmx.jpg

I received a phone call from LSI tech support manager today, stating LSI is able to duplicate the defect with multiple LSI HBAs and multiple SSDs. He promised to send me beta software to test to see if LSI has fixed this. I will post when this happens.
 
Last edited:

iq100

Member
Jun 5, 2012
66
3
8
mrkrad, thanks for sharing your experience and thoughts :).

Do you thing that a HP P410/420 will work in a Dell PowerEdge T110 slot, without flashing?
Or, will it have to be first flashed, so it is seen as a LSI or Dell Perc controller?
Remember my environment is Window 7 Pro, 64bit, and my goal is bootable two Samsung 840 Pro raid0 volume.
 
Last edited:

mrkrad

Well-Known Member
Oct 13, 2012
1,244
52
48
Does your motherboard have intel ICH RAID? If so connect each one up, setup raid-0 and install operating system on it.

I have used this method for many desktop/low end servers to increase the speed since sata-2 is the limiting factor you can get double the performance of sata-2 which is good setup.

Which model do you have?

1. Set the bios to ACHI/RAID or RAID . Set the bios to BIOS not UEFI!!

2. during bootup go into raid mode CTRL-R and create raid-0

3. install windows 7 pro

4. Enjoy.

** Key point is to use BIOS mode to create the raid!! DO NOT USE UEFI mode.

I've been using this for years with maxtor 320gb and adata 50gb because sata-2 is not fast enough but with raid-0 sata2 is very fast.
 

iq100

Member
Jun 5, 2012
66
3
8
mrkrad wrote>"... Does your motherboard have intel ICH RAID? If so connect each one up, setup raid-0 and install operating system on it ..."

Thanks for your suggestion. The Dell PowerEdge Server T110 (google it for official specs) PCH is Intel Series 5/3400. It is 3gbps sata2. Four sata connectors used for hard drives. Yes, two Samsung 840 Pro,raid0 on sata2, using Intel RST (which may or may not work for PCH Series 5 chipset) would probably give me 500 MByte/sec sequential. But connecting to LSI 9212 4i4e now is giving approx. 1 GByte/sec sequential.

You never answered the question regarding HP P410 PCIe 6gbps controller. Will that work in Dell servers. My LSI 9212-4i4e is immediately detected and works. Would I have to flash the P410 to a LSI or Dell Perc 200 for it to be acceptable to Dell server BIOS? Or would installing the HP driver, BIOS, and firmware work in a Dell Server environment? If you know? Thanks.
 
Last edited:

mrkrad

Well-Known Member
Oct 13, 2012
1,244
52
48
I do not have any evidence that this would work. I do not have the time to try this.

Isn't the Perc s300 just a M1015 in IT mode?

I think your choice is 1 Gbyte/sec sequential with poor random i/o versus 500MB/sec sequential with good random i/o is an easy choice. I'll take random over linear all day long. The samsung 840 pro won't sustain that peak speed for long without serious overprovisioning anyways. Steady state will drop down in speed. There is a reason why folks still use hard drives for linear speed, it costs a lot less to go fast consistently.

8 15K SAS drives in raid-10 will run 1GBYTE/SEC linear forever - but you will find the ssd after a certain period of time will degrade greatly to unacceptable speeds. Try it run a benchmark for 1 hour writing and see if you can keep that 1GBYTE/sec speed! :)

What about the PERC S300? That is a fully supported software raid sas controller (AKA M1015) - Only the very brave will cross oem :)

My M5014 works fine in both the HP and DELL with 840 pro :) very cheap when I bought them for $66 used.
 

lunadesign

Member
Aug 7, 2013
109
7
18
Holy hijacked threads Batman! :)

I was away for a while and was surprised to find all this activity. I wasn't sure if I should start a new thread but since this was originally my thread, I figured it would be okay to jump back in.

Testing Update

I was able to borrow a 9271-8i and re-ran my tests and found that this card performed substantially better than the 9260-8i. However, it still lagged onboard somewhat in random reads and random writes.

4K Random Read:
- 9260-8i is 43-73% less IOPS than onboard
- 9271-8i is 0-24% less IOPS than onboard

4K Random Write:
- 9260-8i is 41-63% less IOPS than onboard
- 9271-8i is 0-30% less IOPS than onboard

In the process, I *think* I discovered a PCI Express compatibility problem between the 9260 and the Supermicro system's PCIe 2.0 and 3.0 slots. I noticed that when I ran HD Tune Pro's read test, the speed is almost perfectly constant throughout the test when using the onboard controller or the 9271. With the 9260, the graphed line was pretty jaggy and generally lower than on the others. A PCI Express incompatibility might explain why the 9260 was somewhat faster on an much older X48-based system.

I'm still not sure why the 9271 is slower than onboard as the processor isn't doing any complicated RAID calculations in a single drive RAID 0.

I'm curious if Adaptec and/or Areca cards have a similar performance lag.....
 

mobilenvidia

Moderator
Sep 25, 2011
1,767
60
48
New Zealand
When you leave for a month the rats come in and take over :)

RAID0, a multi Ghz CPU on a Mobo can do this faster than a few 100 Mhz ROC
As RAID0 requires no real processing, just a matter of storing data in a stripe across drives

HW controllers like the SAS9260 are for enterprise use ie SAS drives which are slower but more reliable.
Mobo controller can't do SAS
Mobo controller can't do more than 2x SATA3 (Z87 can do 6) off the controller for RAID
Mobo controller has no HW caching
Mobo controller has no error checking correcting
Mobo controller can't do RAID6, or spanned RAID5 (ie 50/60)

Try and take a Mobo created RAID set to another computer, LSI RAID will work on all LSI RAID cards.