New BackBlaze Server build online

[ddrueding]

These guys offer online storage very cheaply, and they release the entire hardware spec of their servers to the public. Their new one does 135TB in a 4U server for $7400 (http://blog.backblaze.com/2011/07/20/petabytes-on-a-budget-v2-0revealing-more-secrets/).

An interesting quote from someone who would know:


We are constantly looking at new hard drives, evaluating them for reliability and power consumption. The Hitachi 3TB drive (Hitachi Deskstar 5K3000 HDS5C3030ALA630) is our current favorite for both its low power demand and astounding reliability. The Western Digital and Seagate equivalents we tested saw much higher rates of popping out of RAID arrays and drive failure. Even the Western Digital Enterprise Hard Drives had the same high failure rates. The Hitachi drives, on the other hand, perform wonderfully.

[time]

Definitive. And also depressing.

[Mercutio]

That does not surprise me even a tiny little bit, based on my experiences with Hitachi drives in RAID generally, and with recent Seagate 5k drives in various types of arrays.

The Hitachi drives are also the cheapest 3TB drives available right now. So that's working for them, too.

[ddrueding]

There are more bits in there as well.

1. They run their servers blank for a few days due to high infant mortality rates.
2. Higher temps in the middle of racks don't cause more failures.
3. 1% annual failure is currently being observed with the HDS5C3030ALA630 drives.

[Handruin]

Their blog is blocked from work, but I was curious if their build also includes any data related to performance of the storage server? Having 135TB in a 4U space is fun and great, but if it takes an eternity to access it...that's not so great.

[Mercutio]

If you're building RAID5s or JBODs of consumer 5k SATA drives, I/O performance probably isn't on the top of the list.

[ddrueding]

The only performance metric provided was that they could easily saturate a GbE link.

[Mercutio]

... which you can do with a fast single drive

[ddrueding]

Didn't say it was much, just that that is all they provided. Would make a hell of a home media server; I'm just passing 12TB used at the moment.

[Mercutio]

I can tell you that with my setup I absolutely give not a single crap about I/O performance. As long as it's fast enough to play video files without stuttering, I don't care. It's just kind of missing the point of having that collection of drives.

[Handruin]

If you're building RAID5s or JBODs of consumer 5k SATA drives, I/O performance probably isn't on the top of the list.

Agreed, but to a certain limit. At some point one must look at the density of storage and respectfully ask if it can be accessed in a reasonable amount of time. When there is more storage space in a single box, I assume more users will have their data stored in that location.

Figuring the typical consumer of BackBlaze is on an ISP, how many customers can you realistically fit onto a 135TB box with a single 1-gig connection based on the average user size and access rate. It sounds like the GigE is the limitation so far since they're saturating it. I was curious how it performed and the next question is where does peak access begin and how do they over-come that portion next?

[ddrueding]

They said in the article that saturating the NIC only happens in bench tests and that, in production, a server is filled in 25 days. They also stated that they have far more writes than reads. I wouldn't be surprised if they had a caching server full of SSDs on the front end that simply keeps the storage servers grinding away.

[CougTek]

Figuring the typical consumer of BackBlaze is on an ISP, how many customers can you realistically fit onto a 135TB box with a single 1-gig connection based on the average user size and access rate.
Their 135TB model has a motherboard with two gigabit ports on it. Still easy to saturate, but better than a single port.

[Mercutio]

Why would they build a machine like that and not use something that supports multiple NICs?

[ddrueding]

The throughput must not be a limitation for them.

[LunarMist]

They must be replacing dead drives every week in that setup.

[ddrueding]

10 a week.

[Handruin]

The throughput must not be a limitation for them.

Sounds like it to me also. I wasn't meaning to derail the thread, it was more of a curiosity thing for this type of bulk storage.

[LunarMist]

Did you see all those red cases? I'd go blind working that room all day. It's like the fire vehicles in the 1960s.

[CougTek]

I would only go blind if they were pink. I already see red.

[Mercutio]

I already see red.

The one and only time I was angry enough to actually see red, I suspect I was on the ragged edge of passing out. I felt completely disembodied. I hurt a couple people. They both deserved it, but having that experience was enough for me to make a huge change in my personality.

[LunarMist]

I would only go blind if they were pink.

Depending on the shade of pink it would not be as annoying as red.


I already see red.

Financial humor?

[timwhit]

Is anyone using BackBlaze's services? Looks interesting.

[timwhit]

No Linux support. Won't work for me.

[Handruin]

Is anyone using BackBlaze's services? Looks interesting.

I didn't care for their lack of configuration when I evaluated it. Their service basically backed up everything and anything it felt was important and I couldn't get it to backup the items I wanted.