Should be sticky: Samsung 840 and 840 pro are not LSI megaraid compatible

Status
Not open for further replies.

mrkrad

Well-Known Member
Oct 13, 2012
1,244
52
48
LSI will not support the 840 or 840 pro with LSI megaraid, 9260 through 9271 .

The problem manifests itself in two specific drives, oddly these two drives are both competitors to a company called Sandforce.

Samsung 840/840 pro - arguably the best ssd from a stable company (sorry OCZ but your reputation is tarnished, great drive the vector is).

Intel DC3700.

Odd how these two competitors are the biggest threat to sandforce, but Intel is a huge LSI reseller no less.

So - What's the problem?

1. Controller will not let you alter the Disk Cache policy. So if it is off , the massive 512meg cache on the samsung 840 pro (large sized) is not enabled*

2. Strange performance - typically when building a raid set using CTRL-H the raid will perform on AS SSD 4K WRITE at 1-2MB/s taking 7 minutes to complete that portion of the test.

3. Using the 5.6 firmware,MSM, and Drivers (all dated Mar 02 2013) with windows 2008 R2 we could replicate this. We consulted Webhostingtalk and lowendtalk and this is a known problem manifesting the two drive makers' product. LSI knew about this in the last set of drivers and did not update anything on the Mar 02 2013 release of firmware and driver.

4. Controller resets to RA randomly on reboot.Why the heck would I set the SSD to Read Ahead, that's silly and against the fastpath guidelines. More than that gets reset but I won't say more.

5. Creating the raid volume via MSM however may not cause this problem (WTF!). Benchmarks after creating a volume with the 9266 as secondary and the 9260 as primary show normal performance. The 9266 is about twice as fast as the 9260 without fastpath. Fastpath doesn't really seem to work that hot with 2 drives (4 drive raid-10). More on this**
.
6. You must cold shutdown this controller between changes. Firmware, settings. Shutdown and pull the power plug for a minute. Seriously. Everytime.

7. Why would this affect the DC3700 drive as well? it's on the list of approved. Watch it fall off the list soon. LSI we have google and we can go back in time to retrieve your older approved lists. the 840 pro was on the list a few months ago.

The symptoms are quite simple. Flakey. The AS SSD 4K write taking 7 minutes is slower than a 5400 rpm drive. The drive lights are lit solid. This can be replicated on the 9260 by "ENABLE DISK CACHE POLICY: DISABLE" - but the option is greyed out on the 9266. Both controllers use the same 5.6 driver, and 5.6 MSM. Matter of fact, doesn't the 9260 and 9266 share the same firmware nowadays? I don't know that answer.

The symptom also presents as "BLOCK SSD WRITE CACHE CHANGE: YES" when these drives are present. That's a pretty straight up obvious message (storecli show all, or megacli adpaLInfo ) - this message pretty much goes with the "Enable Disk Cache Policy" being greyed out on the MSM.

** We applied fastpath/cc 2 trial key to the card and it then would allow change of policy on the card. however you should not have to buy this key?

* the 9260 had a dead battery, the 9266 had no CV module, but the cache offload (cachevault) was enabled in advanced software options.

So really, LSI, why don't you go back and fix your firmware and controllers. The folks are webhostingtalk are probably some of your larger customers. The original firmware that Samsung 840/840 pro had did have GC issues, but was replaced a month or so ago with new firmware. The DC3700 should have no such problems, especially since it has super capacitor.

The megaraid controllers are pretty interesting. If you build a raid-1 or 1+0 you can kind of see what's going on. It's like it is tuning itself based on the type of reads, Queue Depth, latency, and linear/random, to use the drives as read ahead. Most RAID 1+0 systems will read from 4 drives, if you build a 4 drive raid. It seems that LSI is dynamically choosing to read from 2 , 3 [blip], 3 , or 4 drives based on the block size, type of read, and read/write mix. Unless I have the read/write thing backwards in which case its reading from all 4 but writing to 2,3, or 4 disks. You can replicate this by building a raid-10 and watching ATTO (QD4,QD10), AS SSD, and CDM - all popular benchmarks. This is very different from every other raid manufacturer and I wonder if it part of their secret sauce or part of their problem?

For reference for my above opinion, I believe LSI needs to work on their drivers (win2k8R2) and firmware. We noticed you could change the C: or E: drive cache policy in windows with the 9266. The 9260 would refuse when I touched this. Likewise the P400,P410,P420 HP [latter two are PMC Adaptec] would refuse to allow us to change these settings. This is very normal since the raid card should be in control of these settings.

Back to the reference:
9260 w/fastpath is just as fast as 9266 without, and quite honestly it is stable with the 840 pro/840 and cheap refurb/used.
PMC-Adaptec PM8020 based HP P420 - I got a deal on these, nobody wants them because they have tiny heatsinks and overheat quickly and go offline, nothing a little arctic goop and a creative copped bridge or active cooling can't fix [idle the P420 is 58C, the 9266 idles at 68C in the same slot on a dl380 G7 - put that in your pipe and smoke it].

So the P420, a rather new card, not designed for the G7, with 0% read-ahead, and 100% write back outruns the 9266. I suspect with more drives it would outrun it further. It can enable the drive write cache on the 840/840 pro. It does not mind that I'm running the non-hp drives. I am sure the 1GB supercap cache helps with writes, but quite honestly, at the end of the day it really is which controller is consistent and performs well.

Folks might notice the 71605e (raid 1/10/1e and job at same time) or 71605h (hba only) use a newer version of this chip. It has 16 ports per card standard. It requires no fastpath to go faster because it has 16 ports - 16 840 pro's will go faster than the 9266/9271 w/fastpath. The Adaptec(PMC) cards also support JBOD+RAID at the same time on the raid. And Cachekade on their expensive 71605q. [The hp P420 enables raid-6/60 and cacheKAde 1.0 read only with the SAAP 2.0 key].

There are other options but all know that pretty much HP uses PMC-adaptec in their servers now, everyone else on the planet uses LSI.

I hope someone else reads this, perhaps someone at LSI, and definitely anyone buying samsung 840/840 pro's and an expensive Megaraid card to save them the time and trouble i've wasted.

I am not the only one. google "840 pro 9266 webhostingtalk" or "840 pro cachecade webhostingtalk" or look around here. Folks are having to cross-flash their LSI branded cards to PERC to get them to perform? How odd is that?.

As always, have a nice day!
 

supermacro

New Member
Aug 31, 2012
101
1
0
LSI purchased Sandforce early last year. You think this has something to do with it? I mean Samsung manufactures their own controller for the 840 drives and LSI should have no interests in supporting anything but Sandforce or degrade the others in some sense.
 

TheBay

New Member
Feb 25, 2013
220
1
0
UK
I was going to say LSI purchased sandforce, ironically it's always made me wonder that the reason Hitachi drives work so well on RAID or HBA cards is because they use LSI chipsets.
 

Toddh

Member
Jan 30, 2013
120
8
18
I will not use Adaptec cards any more. I have had too many problems with them over the years. Had a nightmare last year that went on for over a month and 2 different cards, they sent a replacement that had the exact same issue. Their support kept blaming different parts and wanted me to replace each one, cables, hds, ps, backplane etc. I pulled the card and put in an LSI and have not had so much as a peep since. Everything was the same but the Adaptec Raid controller.

I put the Adaptec card in a different server and it did the same thing. I surmised it must have been a firmware/driver issue but Adaptec refused to take any responsibility. Either that or I got 2 bad cards with the same issue - unlikely but not impossible.

.
 

supermacro

New Member
Aug 31, 2012
101
1
0
1. Controller will not let you alter the Disk Cache policy. So if it is off , the massive 512meg cache on the samsung 840 pro (large sized) is not enabled*
I was able to enable disk cache policy with my 9260-4i. Both in the WebBIOS & MSM.

Disk: 4X Samsung 840 Pro 512GB
RAID: 9260-4i - RAID 10

 
Last edited:

supermacro

New Member
Aug 31, 2012
101
1
0
Again, was able to enable disk cache policy with Intel RS25DB080 (same as 9270-8i) in both WebBIOS & MSM.

 

mrkrad

Well-Known Member
Oct 13, 2012
1,244
52
48
Run megacli alIinfo and see if you see this line:

/opt/lsi/MegaCLI # ./MegaCli -adpallinfo -a0 |more

or

MegaCli -adpallinfo -a0

it's the dump all info command for the adapter. Please let me know what you find.



"Block SSD Write Disk Cache Change: Yes "
 
Last edited:

mobilenvidia

Moderator
Sep 25, 2011
1,767
59
48
New Zealand
Just tried the below and I can toggle the ability to enable disable the ability to change the 'Disk Cache'
BTW you can still change it via Device manager

Grab Megascu as you mentioned elsewhere.
Command prompt as Administrator

megascu -adpsettings -write -f mfc.ini -a0

Edit mfc.ini
Look for 'blockSSDWriteCacheChange = 1 # 1=don't allow SSD write cache setting to be changed'
Set this to zero and save
Now

megascu -adpsettings -read -f mfc.ini -a0

reboot
Done

You can also change other settings you were talking about elsewhere to your hearts content.

Like I said before I can toggle the ability to change the Disk Cache change setting.
It should work for you also.
 

mrkrad

Well-Known Member
Oct 13, 2012
1,244
52
48
Interesting. I wonder then if something was factory set wrong - perhaps this one setting. Why on earth would you want to do that? It would not discriminate against the 840/840pro? This almost seems like a factory defect since the term itself "blockSSDWriteCacheCahnge" is talking about all SSD!

Would you share your mfc.ini (minus any personal data via PM? I can share mine and we see what's what.) I want to make sure my lsi 9266-8i has the same features/settings yours does as you seem to be having more luck than I am.

Whilst this doesn't change the 840/840 PRO issue being withdrawn from LSI megaraid support (seems around jan-feb based on google cache of the pdf file) - I think I might have gotten bum cards, or cards that someone else got to first? I seem to be the only one with this enable disk cache policy problem. grrr.
 

klree

Member
Mar 28, 2013
58
0
6
For the SSD Cache setting, I guess it is made to disable such that the SSD Cache is always on.
otherwise there won't be such item "blockSSDWriteCacheChange" in the mfc.ini
 

klree

Member
Mar 28, 2013
58
0
6
or may be 9260 is too old to LSI and not dare to update the mfc and so everything is there
 

mrkrad

Well-Known Member
Oct 13, 2012
1,244
52
48
We're talking 9266 here. The 9260 works fine. The greyed out option means "unchanged". I wonder if this sticks to the raid RIS? Create a raid with the option on, where is it stored? disks? Controller? Both?

I've noticed if you don't init the ssd with 9266 it will see old data from other raid cards/configs. Perhaps the lack of init, is causing naughty bits to be read off the drive as configuration?

*mind blown*
 

mobilenvidia

Moderator
Sep 25, 2011
1,767
59
48
New Zealand
Did you try the edit mfc.ini above ?

Can you change the Disk Cache setting in Windows device manger ?

No change means that the 9266 won't change it, but you should still be able to elsewhere.

Here is my now highly modified mfc.ini I use on the 9266 and M5016

Use AT YOUR OWN RISK !!!

Code:
# MegaSAS CLI utility generated ini file for MFC programming.

subVendorId=0x1000              # sub-vendor ID for this board (Readonly)
subDeviceId=0x9266              # sub-vendor ID for this board (Readonly)
OEM = 1                        # OEM Name of controller (Readonly)
SUBOEM = 0x06                  # SUBOEM ID of controller (Readonly)
sasAddress = 50:06:05:b0:03:86:c2:d0                 # controller SAS address
phyPolarity = 0                # polarity for each possible PHY (bitmap...1=inverted) - used with phyPolaritySplit
phyPolaritySplit = 0           # polarity for TX/RX pair is different (bitmap...0=RX/TX same, 1=RX/TX different; phyPolarity.x=RX)
backGroundRate = 30            # default rebuild rate for this card (0-100)
stripeSize = 7                 # default LD strip size (per DDF: 4=8K, 5=16K, 7=64K, 11=1MB, etc.)
flushTime = 4                  # default cache flush time in seconds
writeBack = 1                  # default LD is WriteBack (0=WT, 1=WB, 2=Adaptive)
readAhead = 2                  # default LD read-ahead policy (0=none, 1=RA, 2=Adaptive)
cacheWhenBbuBad = 1            # default LD setting for enabling WB when BBU bad - 1=enable WB even when BBU bad
cachedIo = 0                   # default LD is cached
smartMode = 0                  # default smart mode (0=mode 6, 1=disable, 2=mode 2)
alarmDisable = 0               # default alarm to disabled if set
coercion = 2                   # 0=none, 1=128M, 2=1G, 3=reserved
zcrConfig = 0                  # ZCR configuration data (0=unknown, 1=RADIOS, 16-31=IDSEL for ZCR)
dirtyLedShowsDriveActivity = 1 # 1=change dirty LED to show drive activity
biosContinueOnError = 1        # 1=continue BIOS boot on error - default
spindownMode = 3               # drive spindown mode on shutdown (0=none, 1=internal only, 2=external only, 3=all)
allowedDeviceTypes = 0         # 0=SAS/SATA mix, 1=no SATA(SAS only), 2=no SAS(SATA only)
allowMixInEnclosure = 1        # allow SAS/SATA mix within an enclosure
allowMixInLD = 1               # allow SAS/SATA HDD mix within an LD
allowSataInCluster = 1         # allow SATA drives to be used in a cluster configuration
allowSSDMixInLD = 1            # allow SAS/SATA SSD mix within an LD
allowMixSSDHDDInLD = 1         # allow SSD/HDD mix within an LD
maxChainedEnclosures = 16       # maximum number of enclosures that may be chained
disableCtrlR = 1               # 1=disable Ctrl-R configuration utility, 0=enable Ctrl-R utility
enableWebBios = 1              # 1=enable WebBIOS configuration utility, 0=disable WebBIOS utility
directPdMapping = 0            # 1=enable direct PD mapping, 0=persistent PD mapping
biosEnumerateLds = 1           # 1=BIOS to display LDs at POST
restoreHotSpareOnInsertion = 0 # HSP kicks in, insert the failed drive, it becomes HSP
exposeEnclosureDevices = 1     # 1=device driver should expose enclosure devices, 0=hide enclosure devices
maintainPdFailHistory = 0      # 1=enable tracking of bad PDs across reboot; will also show failed LED status for missing bad drives
disablePuncturing = 0          # 1=Don't puncture LBA's in PD's. 0=okay to puncture. This is for SAS drives
zeroBasedEnclEnumeration = 0   # 1=Enclosure enumeration start with Zero. 0=Enclosure enumeration start with One.
disableBootCLI = 1             # 0=enable pre boot CLI tool, 1=disable pre boot CLI.
quadPortConnectorMap = 0       # Bitmap defining connector map for 8 quad-ports, 0=Internal connector/unused, 1=External connector/SAS MUX control.
driveActivityLed = 1           # 0=No LED for drive activity, 1=Show all drive activity.
disableAutoDetectBackplane = 0 # 0=SGPIO/i2c SEP enabled, 1=disable auto detect SGPIO,2=disable i2c SEP auto detect,3=disable both auto detect.
enableLedHeaders = 1           # 0=No LED for drive faults, 1=enable LED for drive faults.
useFdeOnly = 0                 # 0=FDE and controller encryption (if HW supports) is allowed, 1=Only support FDE encryption, disallow controller encryption.
delayPOST = 0                  # intentional delay to be introduced during FW POST. Possible coded values: 0 to 7,which gives a value of 0,15,30,45,60,75,90,105 seconds
enableCrashDump = 0            # 1=enable collection of system core in event of a controller crash
disableOnlineCtrlReset = 0     # 1=disable online controller resets, 0=allow online controller resets
enableLDBBM = 1                # 1=enable LD Bad block management
allowUnCertifiedHDDs = 1       # 1=allow, 0=block, 2=Warn & allow
treatR1EAsR10 = 0              # 1=treat single span R1E as R10
maxLdsPerArray = 1             # 0=16 LDs per array, 1=64 LDs per array
disablePowerSavings = 0        # [0 = 0] All bits are 0=enable all power saving options, All bits are one=disable all power saving options , 
                                # bit-0=Don't spin down unconfigured drives, bit-1=Don't spin down Hot spares, bit-2=Don't auto spin down configured drives, 
                                # bit-3=Apply settings to all drives - don't allow individual PD/LD configuration, bit-4 =Disallow MAX power saving option for LDs - only T10 power conditions are to be used, 
                                # bit-5=Don't use cached writes for spun down VDs, bit-6=Provide option to schedule disable power savings at controller level 
spinDownTime = 30			   # Default spin down time in minutes
snapVDSpace = 3                # % of source LD to be reserved for snapshot in snap VD
autoSnapVDSpace = 8            # % of source LD to be reserved for auto snapshot in snap VD
viewSpace = 4                  # snapshot writeable VIEWs capacity in 202010f src VD capacity. 0 - READ only
failPdOnSMARTer = 0			   # 1=fail a PD that's reached SMART failure threshold, if it doesn't take an LD offline
nonRevertibleSpares = 0		   # 1=hot spares are not revertible
enableJBOD = 1                 # 1=enable JBOD mode, 0=disable JBOD mode by default
detectCMETimer = 60     	   #Time in seconds taken to detect CME. 15-60 are valid values. 
ttyLogInFlash = 1              # 1=tty logs are maintained in flash
autoEnhancedImport = 1         # 1=import foreign configuration automatically, at boot
breakMirrorRAIDSupport = 0     # 0=supported for all redundant RAID level - R1, R10 (PRL1 & PRL11), and triple mirror R6, 1=single span R1 (PRL1), 2=All R1 and R10 combinations (PRL1 & PRL11)
disableJoinMirror = 0          # 1=join LD mirror operation is not supported
enablePI = 1                   # 1=PI is enabled
preventPIImport = 1            # 1=Prevent PI Import is enabled
enableShieldState = 1          # 1=Shield state array PD feature is enabled
enableDriveWCEforRebuild = 1   # 1=Enable drive write cache during rebuild
enableEmergencySpare = 1       # 1=Enable Emergency Spare feature
useGlobalSparesForEmergency = 1# 1=use global spares for emergency feature
useUnconfGoodForEmergency = 1  # 1=use unconfigured good drives for emergency feature
blockSSDWriteCacheChange = 0   # 1=don't allow SSD write cache setting to be changed
disableHII = 0            	   # 1=disable HII, 0=enable HII
SMARTerEnabled = 1			   # 1=SMARTer enabled for HDDs (copyback on SMART error)
SSDSMARTerEnabled = 1		   # 1=SMARTer enabled for SSDs (copyback on SMART error)
Vendor/OEM settings can't be changed this way !!

Here is my 'storeCLI show all' with all mods in affect:
Code:
Controller = 0
Status = Success
Description = None


Basics :
======
Controller = 0
Model = LSI MegaRAID SAS 9266-8i
Serial Number = SR136P0821
Current Controller Date/Time = 04/07/2013, 09:57:00
Current System Date/time = 04/07/2013, 09:57:00
SAS Address = 500605b00386c2d0
Mfg Date = 09/06/11
Rework Date = 00/00/00
Revision No = 091


Version :
=======
Firmware Package Build = 23.12.0.0011
Firmware Version = 3.240.05-2282
Bios Version = 5.38.00_4.12.05.00_0x05270000
NVDATA Version = 2.1304.03-0117
Boot Block Version = 2.05.00.00-0010
Bootloader Version = 07.26.13.219
Driver Name = megasas2.sys
Driver Version = 6.506.02.00


Bus :
===
Vendor Id = 0x1000
Device Id = 0x5B
SubVendor Id = 0x1000
SubDevice Id = 0x9266
Host Interface = PCIE
Device Interface = SAS-6G
Bus Number = 1
Device Number = 0
Function Number = 0


Pending Images in Flash :
=======================
Image name = No pending images


Status :
======
Controller Status = OK
Memory Correctable Errors = 0
Memory Uncorrectable Errors = 0
ECC Bucket Count = 0
Any Offline VD Cache Preserved = No
BBU Status = NA
Support PD Firmware Download = No
Lock Key Assigned = No
Failed to get lock key on bootup = No
Lock key has not been backed up = No
Bios was not detected during boot = Yes
Controller must be rebooted to complete security operation = No
A rollback operation is in progress = No
At least one PFK exists in NVRAM = No
SSC Policy is WB = Yes
Controller has booted into safe mode = No


Supported Adapter Operations :
============================
Rebuild Rate = Yes
CC Rate = Yes
BGI Rate  = Yes
Reconstruct Rate = Yes
Patrol Read Rate = Yes
Alarm Control = Yes
Cluster Support = No
BBU  = Yes
Spanning = Yes
Dedicated Hot Spare = Yes
Revertible Hot Spares = Yes
Foreign Config Import = Yes
Self Diagnostic = Yes
Allow Mixed Redundancy on Array = No
Global Hot Spares = Yes
Deny SCSI Passthrough = No
Deny SMP Passthrough = No
Deny STP Passthrough = No
Support more than 8 Phys = Yes
FW and Event Time in GMT = No
Support Enhanced Foreign Import = Yes
Support Enclosure Enumeration = Yes
Support Allowed Operations = Yes
Abort CC on Error = Yes
Support Multipath = Yes
Support Odd & Even Drive count in RAID1E = No
Support Security = Yes
Support Config Page Model = Yes
Support the OCE without adding drives = Yes
support EKM = Yes
Snapshot Enabled = Yes
Support PFK = Yes
Support PI = Yes
Support LDPI Type1 = No
Support LDPI Type2 = No
Support LDPI Type3 = No
Support Ld BBM Info = No
Support Shield State = Yes
Block SSD Write Disk Cache Change = No
Support Suspend Resume BG ops = Yes
Support Emergency Spares = Yes
Support Set Link Speed = Yes
Support Boot Time PFK Change = No
Support JBOD = Yes
Disable Online PFK Change = No
Support Perf Tuning = Yes
Support SSD PatrolRead = Yes
Real Time Scheduler = Yes
Support Reset Now = Yes
Support Emulated Drives = Yes
Headless Mode = Yes
Dedicated HotSpares Limited = No


Supported PD Operations :
=======================
Force Online = Yes
Force Offline = Yes
Force Rebuild = Yes
Deny Force Failed = No
Deny Force Good/Bad = No
Deny Missing Replace = No
Deny Clear = No
Deny Locate = No
Support Power State = Yes
Set Power State For Cfg = No
Support T10 Power State = No
Support Temperature = Yes


Supported VD Operations :
=======================
Read Policy = Yes
Write Policy = Yes
IO Policy = Yes
Access Policy = Yes
Disk Cache Policy = Yes
Reconstruction = Yes
Deny Locate = No
Deny CC = No
Allow Ctrl Encryption = No
Enable LDBBM = Yes
Support FastPath = Yes
Performance Metrics = Yes
Power Savings = No
Support Powersave Max With Cache = No
Support Breakmirror = Yes
Support SSC WriteBack = Yes
Support SSC Association = Yes


Advanced Software Option :
========================

----------------------------------------------------------
Adv S/W Opt                 Time Remaining           Mode 
----------------------------------------------------------
MegaRAID FastPath           Unlimited                -    
MegaRAID Recovery           Unlimited                -    
MegaRAID CacheCade Pro 2.0  Unlimited                -    
MegaRAID SafeStore          Unlimited                -    
MegaRAID RAID6              Unlimited                -    
MegaRAID RAID5              Unlimited                -    
Cache Offload               Unlimited (Unsupported)  -    
----------------------------------------------------------

Safe ID =  TMF9XQJ2PG2KCFRV4BEFGP15B8IML822L8DTKIAZ

HwCfg :
=====
ChipRevision =  C1
BatteryFRU = N/A
Front End Port Count = 0
Backend Port Count = 8
BBU = Absent
Alarm = Disable
Serial Debugger = Present
NVRAM Size = 32KB
Flash Size = 16MB
On Board Memory Size = 1024MB
On Board Expander = Absent
Temperature Sensor for ROC = Present
Temperature Sensor for Controller = Absent
Current Size of CacheCade (GB) = 54
Current Size of FW Cache (MB) = 793
ROC temperature(Degree Celcius) = 42


Policies :
========

Policies Table :
==============

------------------------------------------------
Policy                          Current Default 
------------------------------------------------
Predictive Fail Poll Interval   300 sec         
Interrupt Throttle Active Count 16              
Interrupt Throttle Completion   50 us           
Rebuild Rate                    30 %    30%     
PR Rate                         30 %    30%     
BGI Rate                        30 %    30%     
Check Consistency Rate          30 %    30%     
Reconstruction Rate             30 %    30%     
Cache Flush Interval            4s              
------------------------------------------------

Flush Time(Default) = 4s
Drive Coercion Mode = 1GB
Auto Rebuild = On
Battery Warning = Off
ECC Bucket Size = 15
ECC Bucket Leak Rate (hrs) = 24
Restore HotSpare on Insertion = Off
Expose Enclosure Devices = On
Maintain PD Fail History = Off
Reorder Host Requests = On
Auto detect BackPlane = SGPIO/i2c SEP
Load Balance Mode = Auto
Security Key Assigned = Off
Disable Online Controller Reset = Off
Use drive activity for locate = Off


Boot :
====
BIOS Enumerate VDs = 1
Stop BIOS on Error = On
Delay during POST = 0
Spin Down Mode = all
Enable Ctrl-R = Yes
Enable Web BIOS = Yes
Enable PreBoot CLI = No
Enable BIOS = Yes
Max Drives to Spinup at One Time = 2
Maximum number of direct attached drives to spin up in 1 min = 10
Delay Among Spinup Groups (sec) = 12
Allow Boot with Preserved Cache = On


Defaults :
========
Phy Polarity = 0
Phy PolaritySplit = 0
Strip Size = 64kB
Write Policy = WB
Read Policy = Adaptive
Cache When BBU Bad = On
Cached IO = Off
VD PowerSave Policy = Maximum
Default spin down time (mins) = 30
Coercion Mode = 1 GB
ZCR Config = Unknown
Max Chained Enclosures = 16
Direct PD Mapping = No
Restore Hot Spare on Insertion = No
Expose Enclosure Devices = Yes
Maintain PD Fail History = No
Zero Based Enclosure Enumeration = No
Disable Puncturing = No
EnableLDBBM = Yes
Un-Certified Hard Disk Drives = Allow
SMART Mode = Mode 6
Enable LED Header = Yes
LED Show Drive Activity = Yes
Dirty LED Shows Drive Activity = Yes
EnableCrashDump = No
Disable Online Controller Reset = No
Treat Single span R1E as R10 = No
Power Saving option = Enable
TTY Log In Flash = Yes
Auto Enhanced Import = Yes
BreakMirror RAID Support = No
Disable Join Mirror = No
Enable Shield State = Yes
Time taken to detect CME = 60 sec


Capabilities :
============
Supported Drives = SAS, SATA
RAID Level Supported = RAID0, RAID1, RAID5, RAID6, RAID00, RAID10, RAID50, 
RAID60, PRL 11, PRL 11 with spanning, SRL 3 supported, 
PRL11-RLQ0 DDF layout with no span, PRL11-RLQ0 DDF layout with span
Enable JBOD = Yes
Mix in Enclosure = Allowed
Mix of SAS/SATA of HDD type in VD = Allowed
Mix of SAS/SATA of SSD type in VD = Allowed
Mix of SSD/HDD in VD = Allowed
SAS Disable = No
Max Arms Per VD = 32
Max Spans Per VD = 8
Max Arrays = 128
Max VD per array = 64
Max Number of VDs = 64
Max Parallel Commands = 1008
Max SGE Count = 60
Max Data Transfer Size = 8192 sectors
Max Strips PerIO = 42
Max Configurable CacheCade Size = 512
Min Strip Size = 8 KB
Max Strip Size = 1.0 MB


Scheduled Tasks :
===============
Consistency Check Reoccurrence = 168 hrs
Next Consistency check launch = 04/13/2013, 02:00:00
Patrol Read Reoccurrence = 168 hrs
Next Patrol Read launch = 04/13/2013, 02:00:00
Battery learn Reoccurrence = NA
Next Battery Learn = NA
OEMID = LSI

Drive Groups = 2

TOPOLOGY :
========

-------------------------------------------------------------------------
DG Arr Row EID:Slot DID Type  State BT      Size PDC  PI SED DS3  FSpace 
-------------------------------------------------------------------------
 0 -   -   -        -   RAID5 Optl  Y   9.089 TB dsbl N  N   dflt N      
 0 0   -   -        -   RAID5 Optl  Y   9.089 TB dsbl N  N   dflt N      
 0 0   0   252:0    13  DRIVE Onln  N   1.817 TB dsbl N  N   dflt -      
 0 0   1   252:1    11  DRIVE Onln  N   1.817 TB dsbl N  N   dflt -      
 0 0   2   252:4    15  DRIVE Onln  N   1.817 TB dsbl N  N   dflt -      
 0 0   3   252:5    14  DRIVE Onln  N   1.817 TB dsbl N  N   dflt -      
 0 0   4   252:6    12  DRIVE Onln  N   1.817 TB dsbl N  N   dflt -      
 0 0   5   252:7    16  DRIVE Onln  N   1.817 TB dsbl N  N   dflt -      
 1 -   -   -        -   Cac0  Optl  N  54.947 GB dflt N  N   none N      
 1 0   -   -        -   RAID0 Optl  N  54.947 GB dflt N  N   none N      
 1 0   0   252:3    17  DRIVE Onln  N  54.947 GB dflt N  N   none -      
-------------------------------------------------------------------------

DG=Disk Group Index|Arr=Array Index|Row=Row Index|EID=Enclosure Device ID
DID=Device ID|Type=Drive Type|Onln=Online|Rbld=Rebuild|Dgrd=Degraded
Pdgd=Partially degraded|Offln=Offline|BT=Background Task Active
PDC=PD Cache|PI=Protection Info|SED=Self Encrypting Drive|Frgn=Foreign
DS3=Dimmer Switch 3|dflt=Default|Msng=Missing|FSpace=Free Space Present

Virtual Drives = 2

VD LIST :
=======

----------------------------------------------------------------------
DG/VD TYPE  State Access Consist Cache  Cac sCC      Size Name        
----------------------------------------------------------------------
0/0   RAID5 Optl  RW     No      RaAWBD RW  ON   9.089 TB             
1/1   Cac0  Optl  RW     Yes     NRWBD  -   ON  54.947 GB CacheCade_1 
----------------------------------------------------------------------

Cac=CacheCade|Rec=Recovery|OfLn=OffLine|Pdgd=Partially Degraded|dgrd=Degraded
Optl=Optimal|RO=Read Only|RW=Read Write|B=Blocked|Consist=Consistent|
Ra=Read Ahead Adaptive|R=Read Ahead Always|NR=No Read Ahead|WB=WriteBack|
AWB=Always WriteBack|WT=WriteThrough|C=Cached IO|D=Direct IO|sCC=Scheduled
Check Consistency

Physical Drives = 8

PD LIST :
=======

-------------------------------------------------------------------------------
EID:Slt DID State DG      Size Intf Med SED PI SeSz Model                   Sp 
-------------------------------------------------------------------------------
252:0    13 Onln   0  1.817 TB SATA HDD N   N  512B Hitachi HDS722020ALA330 U  
252:1    11 Onln   0  1.817 TB SATA HDD N   N  512B Hitachi HDS722020ALA330 U  
252:2    18 JBOD   - 54.947 GB SATA SSD N   N  512B OCZ-SOLID3              U  
252:3    17 Onln   1 54.947 GB SATA SSD N   N  512B OCZ-SOLID3              U  
252:4    15 Onln   0  1.817 TB SATA HDD N   N  512B Hitachi HDS722020ALA330 U  
252:5    14 Onln   0  1.817 TB SATA HDD N   N  512B Hitachi HDS722020ALA330 U  
252:6    12 Onln   0  1.817 TB SATA HDD N   N  512B Hitachi HDS722020ALA330 U  
252:7    16 Onln   0  1.817 TB SATA HDD N   N  512B Hitachi HDS722020ALA330 U  
-------------------------------------------------------------------------------

EID-Enclosure Device ID|Slt-Slot No.|DID-Device ID|DG-DriveGroup
DHS-Dedicated Hot Spare|UGood-Unconfigured Good|GHS-Global Hotspare
UBad-Unconfigured Bad|Onln-Online|Offln-Offline|Intf-Interface
Med-Media Type|SED-Self Encryptive Drive|PI-Protection Info
SeSz-Sector Size|Sp-Spun|U-Up|D-Down|T-Transition|F-Foreign
I have since also enabled a more features using MegaSCU and StorCLi
I may make a FW flash utility, that does all the enhancements

[UPDATE] also added Loadshare = Auto
 
Last edited:

0egp8

New Member
Apr 9, 2013
11
0
0
Did you try the edit mfc.ini above ?

Can you change the Disk Cache setting in Windows device manger ?

No change means that the 9266 won't change it, but you should still be able to elsewhere.

Here is my now highly modified mfc.ini I use on the 9266 and M5016

Use AT YOUR OWN RISK !!!

Code:
# MegaSAS CLI utility generated ini file for MFC programming.

subVendorId=0x1000              # sub-vendor ID for this board (Readonly)
subDeviceId=0x9266              # sub-vendor ID for this board (Readonly)
OEM = 1                        # OEM Name of controller (Readonly)
SUBOEM = 0x06                  # SUBOEM ID of controller (Readonly)
sasAddress = 50:06:05:b0:03:86:c2:d0                 # controller SAS address
phyPolarity = 0                # polarity for each possible PHY (bitmap...1=inverted) - used with phyPolaritySplit
phyPolaritySplit = 0           # polarity for TX/RX pair is different (bitmap...0=RX/TX same, 1=RX/TX different; phyPolarity.x=RX)
backGroundRate = 30            # default rebuild rate for this card (0-100)
stripeSize = 7                 # default LD strip size (per DDF: 4=8K, 5=16K, 7=64K, 11=1MB, etc.)
flushTime = 4                  # default cache flush time in seconds
writeBack = 1                  # default LD is WriteBack (0=WT, 1=WB, 2=Adaptive)
readAhead = 2                  # default LD read-ahead policy (0=none, 1=RA, 2=Adaptive)
cacheWhenBbuBad = 1            # default LD setting for enabling WB when BBU bad - 1=enable WB even when BBU bad
cachedIo = 0                   # default LD is cached
smartMode = 0                  # default smart mode (0=mode 6, 1=disable, 2=mode 2)
alarmDisable = 0               # default alarm to disabled if set
coercion = 2                   # 0=none, 1=128M, 2=1G, 3=reserved
zcrConfig = 0                  # ZCR configuration data (0=unknown, 1=RADIOS, 16-31=IDSEL for ZCR)
dirtyLedShowsDriveActivity = 1 # 1=change dirty LED to show drive activity
biosContinueOnError = 1        # 1=continue BIOS boot on error - default
spindownMode = 3               # drive spindown mode on shutdown (0=none, 1=internal only, 2=external only, 3=all)
allowedDeviceTypes = 0         # 0=SAS/SATA mix, 1=no SATA(SAS only), 2=no SAS(SATA only)
allowMixInEnclosure = 1        # allow SAS/SATA mix within an enclosure
allowMixInLD = 1               # allow SAS/SATA HDD mix within an LD
allowSataInCluster = 1         # allow SATA drives to be used in a cluster configuration
allowSSDMixInLD = 1            # allow SAS/SATA SSD mix within an LD
allowMixSSDHDDInLD = 1         # allow SSD/HDD mix within an LD
maxChainedEnclosures = 16       # maximum number of enclosures that may be chained
disableCtrlR = 1               # 1=disable Ctrl-R configuration utility, 0=enable Ctrl-R utility
enableWebBios = 1              # 1=enable WebBIOS configuration utility, 0=disable WebBIOS utility
directPdMapping = 0            # 1=enable direct PD mapping, 0=persistent PD mapping
biosEnumerateLds = 1           # 1=BIOS to display LDs at POST
restoreHotSpareOnInsertion = 0 # HSP kicks in, insert the failed drive, it becomes HSP
exposeEnclosureDevices = 1     # 1=device driver should expose enclosure devices, 0=hide enclosure devices
maintainPdFailHistory = 0      # 1=enable tracking of bad PDs across reboot; will also show failed LED status for missing bad drives
disablePuncturing = 0          # 1=Don't puncture LBA's in PD's. 0=okay to puncture. This is for SAS drives
zeroBasedEnclEnumeration = 0   # 1=Enclosure enumeration start with Zero. 0=Enclosure enumeration start with One.
disableBootCLI = 1             # 0=enable pre boot CLI tool, 1=disable pre boot CLI.
quadPortConnectorMap = 0       # Bitmap defining connector map for 8 quad-ports, 0=Internal connector/unused, 1=External connector/SAS MUX control.
driveActivityLed = 1           # 0=No LED for drive activity, 1=Show all drive activity.
disableAutoDetectBackplane = 0 # 0=SGPIO/i2c SEP enabled, 1=disable auto detect SGPIO,2=disable i2c SEP auto detect,3=disable both auto detect.
enableLedHeaders = 1           # 0=No LED for drive faults, 1=enable LED for drive faults.
useFdeOnly = 0                 # 0=FDE and controller encryption (if HW supports) is allowed, 1=Only support FDE encryption, disallow controller encryption.
delayPOST = 0                  # intentional delay to be introduced during FW POST. Possible coded values: 0 to 7,which gives a value of 0,15,30,45,60,75,90,105 seconds
enableCrashDump = 0            # 1=enable collection of system core in event of a controller crash
disableOnlineCtrlReset = 0     # 1=disable online controller resets, 0=allow online controller resets
enableLDBBM = 1                # 1=enable LD Bad block management
allowUnCertifiedHDDs = 1       # 1=allow, 0=block, 2=Warn & allow
treatR1EAsR10 = 0              # 1=treat single span R1E as R10
maxLdsPerArray = 1             # 0=16 LDs per array, 1=64 LDs per array
disablePowerSavings = 0        # [0 = 0] All bits are 0=enable all power saving options, All bits are one=disable all power saving options , 
                                # bit-0=Don't spin down unconfigured drives, bit-1=Don't spin down Hot spares, bit-2=Don't auto spin down configured drives, 
                                # bit-3=Apply settings to all drives - don't allow individual PD/LD configuration, bit-4 =Disallow MAX power saving option for LDs - only T10 power conditions are to be used, 
                                # bit-5=Don't use cached writes for spun down VDs, bit-6=Provide option to schedule disable power savings at controller level 
spinDownTime = 30			   # Default spin down time in minutes
snapVDSpace = 3                # % of source LD to be reserved for snapshot in snap VD
autoSnapVDSpace = 8            # % of source LD to be reserved for auto snapshot in snap VD
viewSpace = 4                  # snapshot writeable VIEWs capacity in 202010f src VD capacity. 0 - READ only
failPdOnSMARTer = 0			   # 1=fail a PD that's reached SMART failure threshold, if it doesn't take an LD offline
nonRevertibleSpares = 0		   # 1=hot spares are not revertible
enableJBOD = 1                 # 1=enable JBOD mode, 0=disable JBOD mode by default
detectCMETimer = 60     	   #Time in seconds taken to detect CME. 15-60 are valid values. 
ttyLogInFlash = 1              # 1=tty logs are maintained in flash
autoEnhancedImport = 1         # 1=import foreign configuration automatically, at boot
breakMirrorRAIDSupport = 0     # 0=supported for all redundant RAID level - R1, R10 (PRL1 & PRL11), and triple mirror R6, 1=single span R1 (PRL1), 2=All R1 and R10 combinations (PRL1 & PRL11)
disableJoinMirror = 0          # 1=join LD mirror operation is not supported
enablePI = 1                   # 1=PI is enabled
preventPIImport = 1            # 1=Prevent PI Import is enabled
enableShieldState = 1          # 1=Shield state array PD feature is enabled
enableDriveWCEforRebuild = 1   # 1=Enable drive write cache during rebuild
enableEmergencySpare = 1       # 1=Enable Emergency Spare feature
useGlobalSparesForEmergency = 1# 1=use global spares for emergency feature
useUnconfGoodForEmergency = 1  # 1=use unconfigured good drives for emergency feature
blockSSDWriteCacheChange = 0   # 1=don't allow SSD write cache setting to be changed
disableHII = 0            	   # 1=disable HII, 0=enable HII
SMARTerEnabled = 1			   # 1=SMARTer enabled for HDDs (copyback on SMART error)
SSDSMARTerEnabled = 1		   # 1=SMARTer enabled for SSDs (copyback on SMART error)
Vendor/OEM settings can't be changed this way !!

Here is my 'storeCLI show all' with all mods in affect:
Code:
Controller = 0
Status = Success
Description = None


Basics :
======
Controller = 0
Model = LSI MegaRAID SAS 9266-8i
Serial Number = SR136P0821
Current Controller Date/Time = 04/07/2013, 09:57:00
Current System Date/time = 04/07/2013, 09:57:00
SAS Address = 500605b00386c2d0
Mfg Date = 09/06/11
Rework Date = 00/00/00
Revision No = 091


Version :
=======
Firmware Package Build = 23.12.0.0011
Firmware Version = 3.240.05-2282
Bios Version = 5.38.00_4.12.05.00_0x05270000
NVDATA Version = 2.1304.03-0117
Boot Block Version = 2.05.00.00-0010
Bootloader Version = 07.26.13.219
Driver Name = megasas2.sys
Driver Version = 6.506.02.00


Bus :
===
Vendor Id = 0x1000
Device Id = 0x5B
SubVendor Id = 0x1000
SubDevice Id = 0x9266
Host Interface = PCIE
Device Interface = SAS-6G
Bus Number = 1
Device Number = 0
Function Number = 0


Pending Images in Flash :
=======================
Image name = No pending images


Status :
======
Controller Status = OK
Memory Correctable Errors = 0
Memory Uncorrectable Errors = 0
ECC Bucket Count = 0
Any Offline VD Cache Preserved = No
BBU Status = NA
Support PD Firmware Download = No
Lock Key Assigned = No
Failed to get lock key on bootup = No
Lock key has not been backed up = No
Bios was not detected during boot = Yes
Controller must be rebooted to complete security operation = No
A rollback operation is in progress = No
At least one PFK exists in NVRAM = No
SSC Policy is WB = Yes
Controller has booted into safe mode = No


Supported Adapter Operations :
============================
Rebuild Rate = Yes
CC Rate = Yes
BGI Rate  = Yes
Reconstruct Rate = Yes
Patrol Read Rate = Yes
Alarm Control = Yes
Cluster Support = No
BBU  = Yes
Spanning = Yes
Dedicated Hot Spare = Yes
Revertible Hot Spares = Yes
Foreign Config Import = Yes
Self Diagnostic = Yes
Allow Mixed Redundancy on Array = No
Global Hot Spares = Yes
Deny SCSI Passthrough = No
Deny SMP Passthrough = No
Deny STP Passthrough = No
Support more than 8 Phys = Yes
FW and Event Time in GMT = No
Support Enhanced Foreign Import = Yes
Support Enclosure Enumeration = Yes
Support Allowed Operations = Yes
Abort CC on Error = Yes
Support Multipath = Yes
Support Odd & Even Drive count in RAID1E = No
Support Security = Yes
Support Config Page Model = Yes
Support the OCE without adding drives = Yes
support EKM = Yes
Snapshot Enabled = Yes
Support PFK = Yes
Support PI = Yes
Support LDPI Type1 = No
Support LDPI Type2 = No
Support LDPI Type3 = No
Support Ld BBM Info = No
Support Shield State = Yes
Block SSD Write Disk Cache Change = No
Support Suspend Resume BG ops = Yes
Support Emergency Spares = Yes
Support Set Link Speed = Yes
Support Boot Time PFK Change = No
Support JBOD = Yes
Disable Online PFK Change = No
Support Perf Tuning = Yes
Support SSD PatrolRead = Yes
Real Time Scheduler = Yes
Support Reset Now = Yes
Support Emulated Drives = Yes
Headless Mode = Yes
Dedicated HotSpares Limited = No


Supported PD Operations :
=======================
Force Online = Yes
Force Offline = Yes
Force Rebuild = Yes
Deny Force Failed = No
Deny Force Good/Bad = No
Deny Missing Replace = No
Deny Clear = No
Deny Locate = No
Support Power State = Yes
Set Power State For Cfg = No
Support T10 Power State = No
Support Temperature = Yes


Supported VD Operations :
=======================
Read Policy = Yes
Write Policy = Yes
IO Policy = Yes
Access Policy = Yes
Disk Cache Policy = Yes
Reconstruction = Yes
Deny Locate = No
Deny CC = No
Allow Ctrl Encryption = No
Enable LDBBM = Yes
Support FastPath = Yes
Performance Metrics = Yes
Power Savings = No
Support Powersave Max With Cache = No
Support Breakmirror = Yes
Support SSC WriteBack = Yes
Support SSC Association = Yes


Advanced Software Option :
========================

----------------------------------------------------------
Adv S/W Opt                 Time Remaining           Mode 
----------------------------------------------------------
MegaRAID FastPath           Unlimited                -    
MegaRAID Recovery           Unlimited                -    
MegaRAID CacheCade Pro 2.0  Unlimited                -    
MegaRAID SafeStore          Unlimited                -    
MegaRAID RAID6              Unlimited                -    
MegaRAID RAID5              Unlimited                -    
Cache Offload               Unlimited (Unsupported)  -    
----------------------------------------------------------

Safe ID =  TMF9XQJ2PG2KCFRV4BEFGP15B8IML822L8DTKIAZ

HwCfg :
=====
ChipRevision =  C1
BatteryFRU = N/A
Front End Port Count = 0
Backend Port Count = 8
BBU = Absent
Alarm = Disable
Serial Debugger = Present
NVRAM Size = 32KB
Flash Size = 16MB
On Board Memory Size = 1024MB
On Board Expander = Absent
Temperature Sensor for ROC = Present
Temperature Sensor for Controller = Absent
Current Size of CacheCade (GB) = 54
Current Size of FW Cache (MB) = 793
ROC temperature(Degree Celcius) = 42


Policies :
========

Policies Table :
==============

------------------------------------------------
Policy                          Current Default 
------------------------------------------------
Predictive Fail Poll Interval   300 sec         
Interrupt Throttle Active Count 16              
Interrupt Throttle Completion   50 us           
Rebuild Rate                    30 %    30%     
PR Rate                         30 %    30%     
BGI Rate                        30 %    30%     
Check Consistency Rate          30 %    30%     
Reconstruction Rate             30 %    30%     
Cache Flush Interval            4s              
------------------------------------------------

Flush Time(Default) = 4s
Drive Coercion Mode = 1GB
Auto Rebuild = On
Battery Warning = Off
ECC Bucket Size = 15
ECC Bucket Leak Rate (hrs) = 24
Restore HotSpare on Insertion = Off
Expose Enclosure Devices = On
Maintain PD Fail History = Off
Reorder Host Requests = On
Auto detect BackPlane = SGPIO/i2c SEP
Load Balance Mode = Auto
Security Key Assigned = Off
Disable Online Controller Reset = Off
Use drive activity for locate = Off


Boot :
====
BIOS Enumerate VDs = 1
Stop BIOS on Error = On
Delay during POST = 0
Spin Down Mode = all
Enable Ctrl-R = Yes
Enable Web BIOS = Yes
Enable PreBoot CLI = No
Enable BIOS = Yes
Max Drives to Spinup at One Time = 2
Maximum number of direct attached drives to spin up in 1 min = 10
Delay Among Spinup Groups (sec) = 12
Allow Boot with Preserved Cache = On


Defaults :
========
Phy Polarity = 0
Phy PolaritySplit = 0
Strip Size = 64kB
Write Policy = WB
Read Policy = Adaptive
Cache When BBU Bad = On
Cached IO = Off
VD PowerSave Policy = Maximum
Default spin down time (mins) = 30
Coercion Mode = 1 GB
ZCR Config = Unknown
Max Chained Enclosures = 16
Direct PD Mapping = No
Restore Hot Spare on Insertion = No
Expose Enclosure Devices = Yes
Maintain PD Fail History = No
Zero Based Enclosure Enumeration = No
Disable Puncturing = No
EnableLDBBM = Yes
Un-Certified Hard Disk Drives = Allow
SMART Mode = Mode 6
Enable LED Header = Yes
LED Show Drive Activity = Yes
Dirty LED Shows Drive Activity = Yes
EnableCrashDump = No
Disable Online Controller Reset = No
Treat Single span R1E as R10 = No
Power Saving option = Enable
TTY Log In Flash = Yes
Auto Enhanced Import = Yes
BreakMirror RAID Support = No
Disable Join Mirror = No
Enable Shield State = Yes
Time taken to detect CME = 60 sec


Capabilities :
============
Supported Drives = SAS, SATA
RAID Level Supported = RAID0, RAID1, RAID5, RAID6, RAID00, RAID10, RAID50, 
RAID60, PRL 11, PRL 11 with spanning, SRL 3 supported, 
PRL11-RLQ0 DDF layout with no span, PRL11-RLQ0 DDF layout with span
Enable JBOD = Yes
Mix in Enclosure = Allowed
Mix of SAS/SATA of HDD type in VD = Allowed
Mix of SAS/SATA of SSD type in VD = Allowed
Mix of SSD/HDD in VD = Allowed
SAS Disable = No
Max Arms Per VD = 32
Max Spans Per VD = 8
Max Arrays = 128
Max VD per array = 64
Max Number of VDs = 64
Max Parallel Commands = 1008
Max SGE Count = 60
Max Data Transfer Size = 8192 sectors
Max Strips PerIO = 42
Max Configurable CacheCade Size = 512
Min Strip Size = 8 KB
Max Strip Size = 1.0 MB


Scheduled Tasks :
===============
Consistency Check Reoccurrence = 168 hrs
Next Consistency check launch = 04/13/2013, 02:00:00
Patrol Read Reoccurrence = 168 hrs
Next Patrol Read launch = 04/13/2013, 02:00:00
Battery learn Reoccurrence = NA
Next Battery Learn = NA
OEMID = LSI

Drive Groups = 2

TOPOLOGY :
========

-------------------------------------------------------------------------
DG Arr Row EID:Slot DID Type  State BT      Size PDC  PI SED DS3  FSpace 
-------------------------------------------------------------------------
 0 -   -   -        -   RAID5 Optl  Y   9.089 TB dsbl N  N   dflt N      
 0 0   -   -        -   RAID5 Optl  Y   9.089 TB dsbl N  N   dflt N      
 0 0   0   252:0    13  DRIVE Onln  N   1.817 TB dsbl N  N   dflt -      
 0 0   1   252:1    11  DRIVE Onln  N   1.817 TB dsbl N  N   dflt -      
 0 0   2   252:4    15  DRIVE Onln  N   1.817 TB dsbl N  N   dflt -      
 0 0   3   252:5    14  DRIVE Onln  N   1.817 TB dsbl N  N   dflt -      
 0 0   4   252:6    12  DRIVE Onln  N   1.817 TB dsbl N  N   dflt -      
 0 0   5   252:7    16  DRIVE Onln  N   1.817 TB dsbl N  N   dflt -      
 1 -   -   -        -   Cac0  Optl  N  54.947 GB dflt N  N   none N      
 1 0   -   -        -   RAID0 Optl  N  54.947 GB dflt N  N   none N      
 1 0   0   252:3    17  DRIVE Onln  N  54.947 GB dflt N  N   none -      
-------------------------------------------------------------------------

DG=Disk Group Index|Arr=Array Index|Row=Row Index|EID=Enclosure Device ID
DID=Device ID|Type=Drive Type|Onln=Online|Rbld=Rebuild|Dgrd=Degraded
Pdgd=Partially degraded|Offln=Offline|BT=Background Task Active
PDC=PD Cache|PI=Protection Info|SED=Self Encrypting Drive|Frgn=Foreign
DS3=Dimmer Switch 3|dflt=Default|Msng=Missing|FSpace=Free Space Present

Virtual Drives = 2

VD LIST :
=======

----------------------------------------------------------------------
DG/VD TYPE  State Access Consist Cache  Cac sCC      Size Name        
----------------------------------------------------------------------
0/0   RAID5 Optl  RW     No      RaAWBD RW  ON   9.089 TB             
1/1   Cac0  Optl  RW     Yes     NRWBD  -   ON  54.947 GB CacheCade_1 
----------------------------------------------------------------------

Cac=CacheCade|Rec=Recovery|OfLn=OffLine|Pdgd=Partially Degraded|dgrd=Degraded
Optl=Optimal|RO=Read Only|RW=Read Write|B=Blocked|Consist=Consistent|
Ra=Read Ahead Adaptive|R=Read Ahead Always|NR=No Read Ahead|WB=WriteBack|
AWB=Always WriteBack|WT=WriteThrough|C=Cached IO|D=Direct IO|sCC=Scheduled
Check Consistency

Physical Drives = 8

PD LIST :
=======

-------------------------------------------------------------------------------
EID:Slt DID State DG      Size Intf Med SED PI SeSz Model                   Sp 
-------------------------------------------------------------------------------
252:0    13 Onln   0  1.817 TB SATA HDD N   N  512B Hitachi HDS722020ALA330 U  
252:1    11 Onln   0  1.817 TB SATA HDD N   N  512B Hitachi HDS722020ALA330 U  
252:2    18 JBOD   - 54.947 GB SATA SSD N   N  512B OCZ-SOLID3              U  
252:3    17 Onln   1 54.947 GB SATA SSD N   N  512B OCZ-SOLID3              U  
252:4    15 Onln   0  1.817 TB SATA HDD N   N  512B Hitachi HDS722020ALA330 U  
252:5    14 Onln   0  1.817 TB SATA HDD N   N  512B Hitachi HDS722020ALA330 U  
252:6    12 Onln   0  1.817 TB SATA HDD N   N  512B Hitachi HDS722020ALA330 U  
252:7    16 Onln   0  1.817 TB SATA HDD N   N  512B Hitachi HDS722020ALA330 U  
-------------------------------------------------------------------------------

EID-Enclosure Device ID|Slt-Slot No.|DID-Device ID|DG-DriveGroup
DHS-Dedicated Hot Spare|UGood-Unconfigured Good|GHS-Global Hotspare
UBad-Unconfigured Bad|Onln-Online|Offln-Offline|Intf-Interface
Med-Media Type|SED-Self Encryptive Drive|PI-Protection Info
SeSz-Sector Size|Sp-Spun|U-Up|D-Down|T-Transition|F-Foreign
I have since also enabled a more features using MegaSCU and StorCLi
I may make a FW flash utility, that does all the enhancements

[UPDATE] also added Loadshare = Auto
WebHostingTalk member here. Thanks for the advice, I just tried the following:

1) Ran MegaSCU -adpsettings -write -f mfc.ini -a0
2) Edited the mfc.ini file, setting "blockSSDWriteCacheChange" to 0 instead of 1.
3) Ran MegaSCU -adpsettings -read -f mfc.ini -a0
4) Rebooted
5) Disk cache policy could then be changed (through MegaCLI, MegaPCLI, or StorCLI)

I ran CrystalDiskMark after enabling disk cache policy, and 4k write speeds were MUCH better.

HOWEVER.

I ran IOMeter with the following settings:

Transfer Size Request: 8 KB
Read: 80%
Random: 80%
Outstanding I/Os: 64
Number of Workers: 24
Maximum Disk Size: 1 GB
Run Time: 10 MIN

HBA was an LSI 9271-8iCC with one VD [6 256 GB Samsung 840 Pros in RAID 0]. IOPS were around 90,000+ initially. Dropped with time, but that was expected with no TRIM.

About 4 minutes into the test, HBA starts beeping. Shortly after was BSOD. After reboot, go into WebBIOS and one of the drives was missing.

It may have just been a bad SSD, I hadn't used them all that much until then. Though I can't discount that the workaround may have bricked the drive. Until more people are willing to test this and determine whether this occurrence was a fluke, USE AT YOUR OWN DISCRETION.
 

mobilenvidia

Moderator
Sep 25, 2011
1,767
59
48
New Zealand
All you did was enable the 840 cache via the 9271
The default 9271 setting with the 840 is 'No Change'
You can still change this in device manager I think as 'No change' mean if its on leave it on and the opposite also applies.

Now is the missing 840 visible on a onboard SATA port ?

Also save to the TTY log somewhere so you can have a closer look at what happened.
When the alarm went off there would have been a log entry on why.
 

0egp8

New Member
Apr 9, 2013
11
0
0
Also save to the TTY log somewhere so you can have a closer look at what happened.
When the alarm went off there would have been a log entry on why.
There are a few events that occurred repeatedly. Label them as follows.

Event #1:
Code:
Class: Warning
Locale: Physical Drive
Device ID: 0xa
CDB Length: 16
CDB Data:
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
Sense Length: 0
Sense Data:
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Description: Command timeout on PD 0a(e0xfc/s1) Path 4433221102000000
Event #2:
Code:
Class: Warning
Locale: Physical Drive
Device ID: 0xa
Error: 0x3
Description: PD 0a(e0xfc/s1) Path 4433221102000000 reset (Type 03)
Event #3
Code:
Class: Informational
Locale: Physical Drive
Device ID: 0xa
CDB Length: 10
CDB Data:
28 0 1 66 3d 3 0 0 10 0 0 0 0 0 0 0
Sense Length: 18
Sense Data:
70 0 2 0 0 0 0 a 0 0 0 0 4 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Description: Unexpected Sense: PD 0a(e0xfc/s1) Path 4433221102000000
Event #4:
Code:
Class: Warning
Locale: Physical Drive
Device ID: 0xa
Error: 0xf0
Description: Error on PD 0a(e0xfc/s1) (Error f0)
Event #5:
Code:
Class: Informational
Locale: Physical Drive
Device ID: 0xa
Previous State: 24
New State: 144
Description: State change on PD 0a(e0xfc/s1) from ONLINE(18) to SHIELD(90)
Event #6
Code:
Class: Informational
Locale: Physical Drive
Device ID: 0xa
CDB Length: 6
CDB Data:
1d 20 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Sense Length: 18
Sense Data:
70 0 2 0 0 0 0 a 0 0 0 0 4 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Description: Unexpected Sense: PD 0a(e0xfc/s1) Path 4433221102000000
Event #7:
Code:
Class: Critical
Locale: Physical Drive
Device ID: 0xa
Description: Diagnostics failed for PD 0a(e0xfc/s1)
Event #8:
Code:
Class: Informational
Locale: Physical Drive
Device ID: 0xa
Previous State: 144
New State: 17
Description: State change on PD 0a(e0xfc/s1) from SHIELD(90) to FAILED(11)
Events occurred in this order:

1,2,1,2,3,1,2,3,1,2,4,5,2,6,2,7,8

Now is the missing 840 visible on a onboard SATA port ?
It just switched over to unconfigured bad, re-inserted it into the array and no BSOD yet.

Just realized I need to update the firmware on the 9271-8iCC. I'm going to do that, check if the disk cache is enabled afterward, and try IOMeter again.
 
Last edited:

0egp8

New Member
Apr 9, 2013
11
0
0
UPDATE

Flashing latest HBA firmware didn't fix the problem, the error still occurred running the same IOMeter test at the same settings. CrystalDiskMark benches were noticeably higher though for 4k QD32, so the newer firmware did something for performance.

It's VERY similar to the event sequence as before, with the following changes:

1) In all events, Device ID 0xa changes to 0x9, s1 changes to s2, and Path 4433221102000000 changes to 4433221101000000 (i.e. a different drive failed)
2) Event #3 CDB data 28 0 1 66 3d 3 0 0 10 0 0 0 0 0 0 0 changes to 28 0 1 68 8 ed 0 0 10 0 0 0 0 0 0 0

Events occurred in this order:

1,2,1,2,3,1,2,3,1,2,4,5,2,2,7,8

That is, event #6 was skipped this time.

I should point out that flashing new firmware did NOT change the disk cache policy back to disabled. It remained enabled, but became unchangeable in MSM and MegaCLI. I'm going to attempt changing the policy back to disabled after using MegaSCU, and running the test once again. If the test doesn't result in the error after disabling, it would suggest enabling disk cache policy increases instability.
 

0egp8

New Member
Apr 9, 2013
11
0
0
FINAL UPDATE

Just disabled disk cache, ran IOMeter tests again. Not a single disk error.

Which is a shame too, considering it docked random 4k write QD32 by more than 90% to ~40MB/s and reduced IOMeter results to 13,420 IOPS.

Looks like all we can do is wait for LSI to release new firmware.
 
Status
Not open for further replies.