weird m1015 and intel res2sv240 issue

[jbates58]

hi all

i have purchased a couple of intel res2sv240 after every ine saying that they are the bee's knee's of expanders, (alongsode others like chenbro and HP). and i have an issue with them.

when i plugged them in, they decided to ruin 7 of my 14 drive flexraid system, and i have lost a pissload of data from it.

and during my attempts to recover my data (mostly un-successfull) i have managed to recover some, and moved it off my drives, and have formatted them so that i can start fresh; now for the weird issue.

the m1015 has 2 ports, 0 & 1. if i plug expander 1 into port 0, i get my drives to show and work in windows. if i plug expander 1 into port 1, i get my drives to show and work in windows. if i plug expander 2 into port 0, i get my drives to show and work in windows. if i plug expander 2 into port 1, i get my drives to show and work in windows.

HOWEVER... if i plug expander 1 & 2 into ports 0 & 1, i get absolutly nothing in windows. I have also tried to daisy chain the expanders together, and i only get 2 of the drives to show.

the m1015 is freshly flashed to P16 F/W. and the system worked well before the expanders.

the fact that if i use them one at a time, it works, and when i try to use them both it dosent is weird.

i have also tried this with 2 different M1015's and 2 different motherboards (a 1366 Gigabyte & a 775 Asus) aswell as trying them under Windows 7, 8.1 and ubuntu. i still get the same results irrespective of the OS, or Motherboard.

anyone able to shed some light on what i might be doing wrong?

Jason

 

[PigLover]

Have you looked to see what the M1015 'sees' in its BIOS with both expanders and drives plugged in? Does it see all the drives?

 

[sfbayzfs]

Yes, if you flashed it without a BIOS, put the BIOS on, at least for now, it is a great troubleshooting tool. Also, newer firmware versions claim to improve expander handling, so try P19 firmware, or P20 firmware since you are running Windows (P20 doesn't work well on any other OS, it seems) if you are prepared to upgrade to the P20 drivers to match, but try P19 first for safety. When you did get 2 drives to show up with daisy-chained expanders, which expander and ports were the drives on? Also, were you double-cabling your daisy-chain, or single? How many total drives are you trying to attach, and what model(s) are they? It could be a multipath topology discovery error. Try using different ports on each expander as the uplink port to see if it makes a difference.

I have been running a RE2SV240 on a 9201-8i for years under Linux with either P14 or P15 firmware and 20 Hitachi 4TB 5400RPM drives for years with no trouble, but I did get a second spare RE2SV240 with the recent sale, and I am setting up a new larger system now, so I could at least test it with similar firmware. My expander is connected to the LSI card through the "wrong" port according to the manual, but it works fine.

 

[rubylaser]

I'm running a m1015 (P19 firmware) + RE2SV240 with both m1015 ports ran into the expander connected to (16) 4TB disks in Ubuntu 14.04 w/SnapRAID. I haven't seen issues like this with at all. But, I am not running two expanders off on m1015.

Also, what version of FlexRAID are running? Having a disk not available due to cabling/HBA/Expander shouldn't cause data loss, if you can get the disks hooked back up and recognized again.

 

[jbates58]

Have you looked to see what the M1015 'sees' in its BIOS with both expanders and drives plugged in? Does it see all the drives?

The m1015 is in IT mode. I will flash it to IR mode thisarfternoon. Was wanting it to work in IT so that boot times are reduced etc...

Yes, if you flashed it without a BIOS, put the BIOS on, at least for now, it is a great troubleshooting tool. Also, newer firmware versions claim to improve expander handling, so try P19 firmware, or P20 firmware since you are running Windows (P20 doesn't work well on any other OS, it seems) if you are prepared to upgrade to the P20 drivers to match, but try P19 first for safety. When you did get 2 drives to show up with daisy-chained expanders, which expander and ports were the drives on? Also, were you double-cabling your daisy-chain, or single? How many total drives are you trying to attach, and what model(s) are they? It could be a multipath topology discovery error. Try using different ports on each expander as the uplink port to see if it makes a difference.

I have been running a RE2SV240 on a 9201-8i for years under Linux with either P14 or P15 firmware and 20 Hitachi 4TB 5400RPM drives for years with no trouble, but I did get a second spare RE2SV240 with the recent sale, and I am setting up a new larger system now, so I could at least test it with similar firmware. My expander is connected to the LSI card through the "wrong" port according to the manual, but it works fine.

The drives are mostly WD 3tb greens (have wdidled them to 300s), there is also a few 6tb wd reds and a few 2,3,4tb seagate green equivilents.

I went with p16 as on the freenas forums, its suposedly the most stable (note im not using freenas, just the only place with a large user base and info)

the only drives that show are the ones in the first expander, its as if the second expander is set wrong. But there is no config or even jumper settings to use.

I'm running a m1015 (P19 firmware) + RE2SV240 with both m1015 ports ran into the expander connected to (16) 4TB disks in Ubuntu 14.04 w/SnapRAID. I haven't seen issues like this with at all. But, I am not running two expanders off on m1015.

Also, what version of FlexRAID are running? Having a disk not available due to cabling/HBA/Expander shouldn't cause data loss, if you can get the disks hooked back up and recognized again.

Latest flexraid. And the drives missing wasnt the issue. Somehow the issues with the expanders deleated the partition tables and no matter what i did i couldnt recover all the data. I have saved the important stuff. All i have realy lost is some media and games. Nothing to important. And i have a complete off site backup of that aswell.

Jason

 

[jbates58]

Have you looked to see what the M1015 'sees' in its BIOS with both expanders and drives plugged in? Does it see all the drives?

The m1015 is in IT mode. I will flash it to IR mode thisarfternoon. Was wanting it to work in IT so that boot times are reduced etc...

Yes, if you flashed it without a BIOS, put the BIOS on, at least for now, it is a great troubleshooting tool. Also, newer firmware versions claim to improve expander handling, so try P19 firmware, or P20 firmware since you are running Windows (P20 doesn't work well on any other OS, it seems) if you are prepared to upgrade to the P20 drivers to match, but try P19 first for safety. When you did get 2 drives to show up with daisy-chained expanders, which expander and ports were the drives on? Also, were you double-cabling your daisy-chain, or single? How many total drives are you trying to attach, and what model(s) are they? It could be a multipath topology discovery error. Try using different ports on each expander as the uplink port to see if it makes a difference.

I have been running a RE2SV240 on a 9201-8i for years under Linux with either P14 or P15 firmware and 20 Hitachi 4TB 5400RPM drives for years with no trouble, but I did get a second spare RE2SV240 with the recent sale, and I am setting up a new larger system now, so I could at least test it with similar firmware. My expander is connected to the LSI card through the "wrong" port according to the manual, but it works fine.

The drives are mostly WD 3tb greens (have wdidled them to 300s), there is also a few 6tb wd reds and a few 2,3,4tb seagate green equivilents.

I went with p16 as on the freenas forums, its suposedly the most stable (note im not using freenas, just the only place with a large user base and info)

the only drives that show are the ones in the first expander, its as if the second expander is set wrong. But there is no config or even jumper settings to use.

I'm running a m1015 (P19 firmware) + RE2SV240 with both m1015 ports ran into the expander connected to (16) 4TB disks in Ubuntu 14.04 w/SnapRAID. I haven't seen issues like this with at all. But, I am not running two expanders off on m1015.

Also, what version of FlexRAID are running? Having a disk not available due to cabling/HBA/Expander shouldn't cause data loss, if you can get the disks hooked back up and recognized again.

Latest flexraid. And the drives missing wasnt the issue. Somehow the issues with the expanders deleated the partition tables and no matter what i did i couldnt recover all the data. I have saved the important stuff. All i have realy lost is some media and games. Nothing to important. And i have a complete off site backup of that aswell.

Jason

 

[PigLover]

You don't need to flash it to IR mode, but you may need to flash the BIOS back onto it if you followed one of the guides that say 'you don't need the bios in IT mode'.

Just flash the bios and see what it says...

 

[jbates58]

ok, i flashed the bios onto the m1015, aswell as updated it to the latest P20 F/W.

and its still not working.

i can get the expanders seen individually in the bios, in either port. but if i try them both together, then its a no go, i only get 1 to show. and if i expand it to see the drives, i have 4 plugged into the cable. and it will sometimes display 3 of the 4 drives.

it seems no matter what i do, it just wont work.

what can i try next? im kinda stumped.

Jason

 

[jbates58]

if i daisy chain them together, i get this error upon boot

SAS discovery error 0x00000002 on adapter0, port1

i have a picture of my setup but i cant seem to post it. do i need to have access to imbed an image in a post? i click the insert pic button, and paste the url. but nothing happens.

the url fo the pic is

 

[PigLover]

When you flashed the card to IT mode did you remember to set the SAS address on the card? The first step of the flash instructions zeros the flash area, then you flash the card to IT mode. Some of the instructions forget the step where you reset the SAS address of the card, or some of them just say to set it to all zeros. This won't work in your config with two backplanes.

Follow the instructions here to set the flash address. You really don't need to re-do all of it - just find the sticker on the card with the address and the step that sets the SAS address.

• Turn machine off grab the SAS address of the card, it’s on the back on a green sticker (ie 500605B0xxxxxxxx)
• sas2flsh -o -sasadd 500605bxxxxxxxxx (x= numbers for SAS address)

Also - since you are using Windows - you probably don't want to use P20. Get P19 or earlier.

 

[jbates58]

yes, i have flashed the sas address from the back of the card. i do that everytime i re-flash it.

Jason

 

[jbates58]

have flashed it back to P19, still having the same issues. can someone on here confirm that the m1015 will work with more than 1 expander attached (wether directly or daisy chained)

Jason

 

[PigLover]

Yes. M1015 works well with multiple expanders, both as daisy chain or connected to each 8087 port.

I think you have something else going on here. Perhaps a problem with the sas address on one or both expanders.

 

[rubylaser]

Just tested two of my Intel expanders daisy chained off an M1015 and they work for me as well.

 

[jbates58]

Yes. M1015 works well with multiple expanders, both as daisy chain or connected to each 8087 port.

I think you have something else going on here. Perhaps a problem with the sas address on one or both expanders.

I will check both of them when i get home. How can i change the address if that is the issue?

I will also check that it is on the latest f/w

Jason.

 

[jbates58]

ok, i downloaded the latest intel FW for the expanders. one was already on the latest (254.13), but i flashed it anyway and the other one was on (254.06) acording to the flash update tool.


annnnddd....

it still made no difference.

i have found the stickers on the cards with what im sure is the sas address, and they are both different.

and im now out of ideas.

Jason

 

[PigLover]

Put them onto the expander one at a time (the way it works). Go into the M1015 BIOS and look at the "SAS Topology". Check the SAS address that the expander card reports and make sure it looks valid (that it matches the sticker).

Oh - and just in case - swap the two cables you've been using to connect the expanders just in case you've been chasing a bad cable the whole time.

 

[jbates58]

Put them onto the expander one at a time (the way it works). Go into the M1015 BIOS and look at the "SAS Topology". Check the SAS address that the expander card reports and make sure it looks valid (that it matches the sticker).

Oh - and just in case - swap the two cables you've been using to connect the expanders just in case you've been chasing a bad cable the whole time.

i have tried that, but i cant seem to get and info about the expander in the bios. i press the keys (alt+d i think) to get more details, but i dosent load anything on the expanders.if i have a drive, then iget alot more info aboit it like serial etc... but nothing with the expanders. and i have been swapping the 2 cables that i have here around through the whole process.

is there another way to get the info on the expander sas address?

Jason

 

[PigLover]

It's beginning to sound like either your M1015 or one (or both) of the expanders may be faulty.

 

[jbates58]

Its just odd that they work individually.