3ware 9750 problems

[viper14]

Hi everyone I'm currently building a new file server with a 3ware 9750-8i card, using sff-8087 to a chenbro 28 port sas expander, and from there 5 sff-8087 cables toa norco 4220 case, i have 10 WD20EADS drives (older model with TLER support), in RAID6 (14.55TB usable) I've enabled TLER R/W for 7 sec on every drive, I've also used wdidle3 to disable the head parking after 8 sec of idle, so took a day or so to build the array and now that its up and running my server OS is on a 80gb SSD drive (centos 5.5), kernel 2.6, I've used the XFS file system on the drive I've tried multiple ways even just a basic mkfs.xfs /dev/sdb1 no matter what every time i do any type of disk I/O on the raid set, it randomly drops every disk i check the 3DM2 web service and it shows the array as INOPERABLE, soon as i reboot the array goes into verify mode and within 2 min its back to OK, not sure whats goin on or if anyone has any idea on what tit might be I've tried 3ware support and i need to send them some logs but it doesn't seem like they will be much help since I'm running the WD green drives and there not on the supported list. has anyone ever seen anything like this happen, not sure how to fix this problem or even determine whats wrong.

Thanks

 

[ddrueding]

Wow. I haven't heard of that before, but I only have experience with the older 3Ware RAID cards.

 

[Bozo]

Did the RAID array initialize?
Try running a 'verify' from the 3Ware web access program

 

[viper14]

yup the RAID array initialized with out a problem i sent a emial and log files off to 3ware and this was there response

"You have some major power cycle issues with these drives. Note that these are desktop drives, not enterprise drives. This type of behavior is common with drives not designed for use with RAID controllers. I recommend upgrading to enterprise class drives."

so not sure what i can do to fix the problem, its weird i also have another raid5 set of 6 1TB WD green drives and it dosnt have any problems at all, i know these drives goto sleep after 8 sec of idle so i used teh wdidle3 util onteh drives to dissable but it seems there still a problem with the wd20eads drives, the drives are fine just sitting but its as soon as any I/O hits the drive they all drop.

 

[ddrueding]

Doesn't look like the utility is compatible with those drives?

This firmware modifies the behavior of the drive to wait longer before positioning the heads in their park position and turning off unnecessary electronics. This utility is designed to upgrade the firmware of the following hard drives: WD1000FYPS-01ZKB0, WD7500AYPS-01ZKB0, WD7501AYPS-01ZKB0.

 

[MaxBurn]

I agree, those aren't the drives you want to be using for this. Having to enable TLER and use wdidle3 should be a sign. Can you return or swap the drives?

If you can't you might have better luck ditching that 3ware, converting to a soft RAID situation while using a AOC-SASLP-MV8 instead.

 

[Handruin]

Would it help to increase the TLER to a longer timeout on the drives? Is the TLER configurable on the 3WARE card? It does seem odd that the array can go through the entire initialize without dropping any drives, but once complete it easily drops all the drives.

 

[ddrueding]

During the initialize, the drives are active enough to not cause issues. As soon the the drive is "ready", the drives will sleep. The RAID card doesn't realize this until you try to access something. At that point the drives take much longer to respond, or give some "not ready" message, and the RAID card dumps them from the array.

What happens if you constantly access the drives (stream video? copy files?) from the moment it is initialized? If they never sleep for 8 seconds, you should be fine

 

[Handruin]

That makes sense and which is why I was wondering if it's possible to tell the 3WARE controller to wait longer than the time it takes the drives to wake up before pronouncing it dead/errored.

 

[viper14]

During the initialize, the drives are active enough to not cause issues. As soon the the drive is "ready", the drives will sleep. The RAID card doesn't realize this until you try to access something. At that point the drives take much longer to respond, or give some "not ready" message, and the RAID card dumps them from the array.

What happens if you constantly access the drives (stream video? copy files?) from the moment it is initialized? If they never sleep for 8 seconds, you should be fine

will this is where the problem is when it array is idle theirs no problems or copying small files no problem, soon as its under any heavy I/O sometimes i can get 4-5gb copied every drive drops out of the array, doesn't make sense to me because i have the exact same 1TB versions of the same WD20EADS and they don't act the same at all no problems at all i have 6 in RAID5 an have filled 4TB in 2 days without any problems, anyway since these problems i took out the WD20EADS (2tb) drives and have replaced them with Hitachi 2TB disks which are on the supported list for the 3ware 9750 card, the array is still building should be done in a hour or 2 been going for 5 so far. I'll post my results tomorrow. since I'm doing this all over again does anyone have any recommendations to make.xfs for settings? The raid is built in 64K block size with 10 2TB disks in raid6 if anyone knows what optimal make.xfs settings to use could you let me know.

Thanks

 

[ddrueding]

No suggestions on the make.xfs, I don't know that much on the Linux side.

Getting drives that are on their HCL seems like an unnecessary move until you have problems. It is always nice to be able to hold their feet to the fire.

 

[blakerwry]

I think the default mkfs settings should be fine. Usually the mount options (like noatime) make a larger performance impact. XFS has some pretty good defaults on both sides.

One thing to be aware of with WD drives is that the model number is a conglomeration of the drive's interface, cache size, capacity, and spindle speed and does not uniquely identify a specific drive. e.g. any 7200RPM SATA2 drive with 32MB cache and 2TB capacity will have the same model number, even though WD may have used 10 different configurations through the lifetime of that model number.

If you purchased the drives > 6 months apart, there's a good chance that they are different drives, even if the model numbers are the same.

 

[kideternal]

My first thought when 3ware said power-issues is: how many watts is your power supply? (I run 1000W in my NORCO 4020.)

FWIW, I've used multiple 20EADS drives with my old 3ware 9650-8LPML without any of them dropping. (Though the EADS did seem to fail "fairly often", so I've since switched to EURS with no failures in over a year. *knocks wood*)

If it's not the power supply, perhaps try rebuilding an 8-drive array without the expander for testing purposes?

Also, I'm contemplating a 9750-24i4e, so I'm curious which motherboard are you using? (It's gets difficult to pick one that has a full PCIe2x8 slot that's not shared with the videocard.)

If you figure out a solution, please update this thread so we can learn from your experience.

 

[kideternal]

Further poking around the web indicates there were driver problems initially with the 9690 dropping drives attached to expanders due to timing issues until a driver update fixed them. Perhaps the 9750 has a similar issue that needs patching? Something to bring up with support?

 

[ChatsworthRed]

Dropping drives

Did you ever resolve this issue?
I am having major problems with regular drives dropping which then takes 2 days to rebuild the array.

 

[MaxBurn]

In another thread here I fixed my dropping disk by replacing a bad cable. Don't overlook the basics.