Building a Storage Server Thread

[Buck]

Sadist.

Sorry, I never viewed you as a love object.

 

[Fushigi]

(Fushigi's AS/400 monsters don't qualify for "small servers" - no offense)

None taken. We have 1200 users doing OLTP-type workloads; that's a far cry from a small business/home server doing file/print/media handling. Although OS/400 does have media streaming capabilities, robust file & print processing, and better security than practically any platform out there, the price tag is definitely beyond the reach of this group or our intended audience.

I've no problem with Merc taking the lead on the article and designing a storage server best suited to some particluar task.

 

[Mercutio]

Gah! Another sadist.

 

[blakerwry]

But then he'll have to give that crucial RAM to himself.... merry christmas.

 

[mubs]

Ha ha, that's funny, Blake!

 

[ddrueding]

David,
Where are you getting the SP1614C for $136? Best price I've found for it is still over $150.

newegg.com here

 

[ddrueding]

Doesn't everyone need an exchange and SQL server at home (in addition to IIS)? Use it as a firewall as well (with ISA) using your 2 NICs...

Exchange, not at all. I've never used it. Every time I hear about Exchange, it seems to be mummyfied in "unsecured" bindings.

I have nothing but good experiences with it, very easy to admin., and provides more features than the competition. Quite possibly more than this expirament requires.

MySQL 4.0x works well enough not to wonder about SQL Server.

I wouldn't use Microsoft's software for security (firewall) unless I have no other choice.

Another product I'm quite happy with. The most intuitive interface since Firewall-1.

Thermalright SLK-900-A: $37
VANTEC's Tornado (92mm): $13

IMO, this heatsink/fan combo is way more noisy than necessary. I wouldn't overclock a CPU sitting inside a server box, so this cooling solution wouldn't make it on my list. Plus, that heatsink is quite heavy. It's not a problem for an enthusiast box, but for a server where reliability should be paramount, a heatsink within the CPU manufacturer's weight limit should be prefered.

I like this combo as it keeps the CPU within about 4 degrees of ambient. It is incredibly loud (and I haven't looked at alternatives) but it's effectiveness is unquestioned. Re: overweight, not by much, and in the chassis I listed, the motherboard is lying flat with the heatsink ontop of it...quite safe.

Also, this thing is incredibly loud. The tornado aside, the chassis fans that Antec uses (4 of them) move @ 5000RPM and the fans in the supermicro enclosures @ 4800. Swapping the tornado for a panaflo would do virtually nothing to the noise levels here.

 

[ddrueding]

What you're building looks to me a lot more like a workstation than an actual server to me, SBS aside.

I don't know where you're getting that impression (besides the fact that my desktop has the same mobo/CPU). I selected the motherboard for it's onboard SATA Raid, GigE and support of less expensive "new" CPUs. Unfortunatly, this combo isn't available with onboard video or 64-bit PCI, but still saves some bucks to get what I was after.

At least it fits the "low end" fairly well...unless you want to enter EPIA land....

 

[Mercutio]

You'll have to forgive me a little. To me, the only excuse for buying an nforce2 is the high-quality onboard sound.

 

[Mercutio]

Ha ha, that's funny, Blake!

Yeah, me celebrating xmas IS pretty amusing. Usually I don't have to work too hard to find some poor schlub who doesn't mind paying me some nice "holiday" rate to work the 25th of December. I get get a paycheck for doing something that's usually fairly mindless - although a few years ago I did ISP tech support, which sucked - and they get time with their belligerent and numerous spawn.

 

[CougTek]

You'll have to forgive me a little. To me, the only excuse for buying an nforce2 is the high-quality onboard sound.

I don't see an nForce2-based motherboard with only dual-phase VRM inside a server either. It's a nice SOHO board, both IMO, it has no place inside a server.

 

[ddrueding]

I don't see an nForce2-based motherboard with only dual-phase VRM inside a server either. It's a nice SOHO board, both IMO, it has no place inside a server.

VRM is another feature that is nice and expensive. I thought we were looking at a SOHO server?


IMO, some things are worth the added cost:

-True hardware RAID
-SATA with hotswap enclosures
-Seperate controller for OS drives

And some come at too high a price for an "entry level" file server:

-64/66 PCI or PCI-X
-Dual CPU
-High-end VRM
-Intel in general (Xeon, PIV, Itanium especially...what else is a new part from intel?"

Some of the not-so-important bits that were included in my build were:

-Support for 400Mhz Bus
-Dual-Channel PC2700 RAM
-Dual Ethernet

 

[Mercutio]

I disagree that advanced expansion isn't worth the expense. If we're talking about Gbit networking, 32/33 PCI is actually a bottleneck. If we're talking about enterprise-class disk controllers, PCI is actually a bottleneck. Especially if it's sharing a bus with a Gbit NIC.

I also disagree that Intel isn't compelling in this space. In fact, it's AMD that's struggling here. My P3-based server is lightweight enough to get away without a 500W PSU. The i875's CSA makes it the only compelling choice for using a desktop board, and a P4 Serverworks board is the best of all worlds as far as I can tell. AMD brings nothing to the mix (power-hungry CPUs - like the P4 isn't, I know, no advanced PCI, no CSA-equivalent). There aren't enough boards with the kinds of high-end features we're looking for in AMD-land, and those there are don't look like good choices for a storage appliance.

Also, even though we're talking about a lot of dual CPU system boards here, absolutely no one is talking about using two CPUs.

So, back to David's suggested config: Why SATA? I've stated a case against it, above, so what's making it worthwhile to you?
Also, why bother with hotswap? I'm still in SOHO-land here, and I can't see much in terms of justificatiion for not taking the 10 minutes of downtime it might take to down my server and replace a failed disk. Mission critical home server stuff?

 

[ddrueding]

So, back to David's suggested config: Why SATA? I've stated a case against it, above, so what's making it worthwhile to you?
Also, why bother with hotswap? I'm still in SOHO-land here, and I can't see much in terms of justificatiion for not taking the 10 minutes of downtime it might take to down my server and replace a failed disk. Mission critical home server stuff?

Well, it depends (again) on what we're looking for. If this is just a file server, than putting some extra cash towards the storage components seems appropriate. Having a server in a headless configuration makes powering down/up the system more difficult. Howerver, I wouldn't mind having a client "pull the tray with the red light on and drop in the spare" (thus the disk enclosures). Those have hardware indicators for activity and failed drives. Those enclosures also provide great ventilation for all the drives.

Why SATA? Because those enclosures don't come in PATA, and I couldn't find a case I liked that supported 10+ 3.5" drives on it's own with sufficent ventilation.

The stuff I'm listing here has WAY more cooling capacity than is required, but after 5 years of sitting in a closet it will still have enough.

I feel much less strongly about the mobo/cpu issue than about the drive/enclosure/controller issue.

 

[ddrueding]

Perhaps this would be a more effective use of our time if we set the requirements for the 3 levels and went from there:

Base level: 1TB+ redundant storage, 100Mbps ethernet, $2000 (hardware only)

Mid-Level: 2TB+ Redundant storage (expandable using current drives to 3TB+), GigE*, $4000 (hardware only)

High-End: 4TB+ Redundant storage (expandable using current drives to 6TB+) , 2+ GigE* connections, $10000

*Must account for and work around bus limitations




comments?

 

[ddrueding]

Proposal for base level:

VIA EPIA-ME6000 (600Mhz, Fanless): $123
1x 256MB PC2700: $76
3Ware Escalade 8506-8: $525
8x Samsung SATA 160GB: $1088
ANTEC SX1040BII with 400 Watt PS: $89

$1901 for 1.12TB of RAID5

 

[ddrueding]

Provided that PS can handle it, of course. And switching to PATA would save ~$300 and not affect much in this application.

 

[blakerwry]

what do the I/O benchmarks look like with the C3? I would bet a slower pII or PIII would be faster... a current Duron w/ M-ATX mobo cheaper and faster as well.

 

[Mercutio]

I see no reason to move away from the P3 config I mentioned earlier, in part because I'm unsure of the wisdom in installing a CPU that can't be replaced without replacing the whole board.. Plus I'm a little more comfortable with SuperMicro's engineering and, hey, 64bit PCI.

That 600MHz EPIA chip is IIRC in the same ballpark with a P2-400, maybe a little slower.

 

[ddrueding]

what do the I/O benchmarks look like with the C3? I would bet a slower pII or PIII would be faster... a current Duron w/ M-ATX mobo cheaper and faster as well.

Just about anything would be faster, but cheaper? This thing has everything integrated for $123. Tack on the added reliability bonus of not requireing a CPU fan; this means any fan failures would not instantly kill the system.

 

[ddrueding]

I see no reason to move away from the P3 config I mentioned earlier, in part because I'm unsure of the wisdom in installing a CPU that can't be replaced without replacing the whole board.. Plus I'm a little more comfortable with SuperMicro's engineering and, hey, 64bit PCI.

That 600MHz EPIA chip is IIRC in the same ballpark with a P2-400, maybe a little slower.

Just curious, which P3 bard are you talking about? And where is it available?

 

[Mercutio]

The SuperMicro P3TDLE. It has the distinction of being the only SuperMicro board I've ever actually used.

I can buy it locally for $275 but a lot of the usual suspects on Pricewatch (Axion, PartsPC, PC Nation, carry them.)

 

[mubs]

Just curious, which P3 bard are you talking about?

So Intel's into poetry now?

 

[blakerwry]

ddrueding, I dont know if you realize this, but any pII or pIII can be run passivley.

Also, you can get decent motherbaords for about $60 with onboard video, audio, and LAN Pop in a $48 athlon XP and you have a much faster, cheaper system that will fit in a normal case and is more upgradable.

Even on silentPCreview.com the VIA C3 loses to intel and AMD alot of the time... it just doesn't perform as good.

 

[Bozo]

Using an old/slow processor is going to slow things down. Even file transfer over the network. I ran some quick test to see the difference in file trans fers from PC to PC.
The recieving PC (server) is an Intel 865 MB/P4-2.6Ghz/800Mhz FSB/512Meg dual channel memory box. The network card is an 3Com 3c905c (this network card was used in all the test boxes)
I copied the i386 files from a Win2k cd to all the test boxes and then copied the file to the Server. (~300meg)
Box #1- Intel d845 MB/P4-2.0Ghz/400Mhz FSB/512meg ram Tranasfer time=1minute,53seconds
Box #2=Intel D865MB/P4-2.4Ghz/800Mhz FSB/512Meg RAM-dual channel
Transfer time=1minute,23seconds.
Box #3=Intel BX chipset MB/P2-300Mhz/66Mhz FSB/256Meg RAM Transfer Time=I stopped timming after 10 Minutes!

Two servers that I just built contained the following:
Intel D865GBFL MB $103
Intel P4-2.6Ghz/800FSB $175
Antec SX1040BII case $89
Sony CD $16
4-WD360JD SATA $468
3Ware 8500-4 $343
2-Crucial 512 MEG RAM $188
Floppy drive $13
Total $1395
This is a file server running RAID5 with ~100GB of storage space. Need more space? Substitute the WD1200JD hard drives for about the same price and end up with 360GB of storage.
Even adding the OS (Win Server 2003) and the price is still under $2000.

Bozo :mrgrn:

 

[Bozo]

One other thing. All the test boxes were running a WD400JB hard drives. Box #3 had a Promise Ultra 100 card in it to let the hard drive run in ATA100 mode.

Bozo :mrgrn:

 

[Mercutio]

Are you sure what you've got there isn't being limited by something else, Bozo? Your 3rd machine is substantially different from the others after all.

 

[Bozo]

My point was that you need a high performance setup even if it is only a file server. Using a P3 or celeron with a slow buss speed will bog down everything, even if you are on a GB network.
This is not the first time I've run these kind of test. Fast CPUs, fast FSB speeds, and lots of memory always do better across a network.

The test boxes are all in my cube and all are connected to a D-Link switch. The network card was the same in each box. The hard drives used were all WD400JBs. Box #3 had a Promise Ultra 100 controller card in it so the hard drive could run at ATA100.

The only difference was the MB, CPU, and memory.

The two servers I built are not "live" yet and are still in the setup stage. As soon as I get the chance I would like to give thses servers the same network test. I would expect the same kind of performance as box #2.
Not too shabby for $2000.

Bozo :mrgrn:

 

[Mercutio]

So I just tried the \i386 xfer with my Celeron 433 Laptop (also 66MHz bus, 256MB RAM, 10GB crappy laptop hard disk), over 802.11A. Took ~3 minutes 45 seconds.

Over 802.11A. On a 4000rpm hard disk. In a laptop that wasn't plugged in.

 

[ddrueding]

So I just tried the \i386 xfer with my Celeron 433 Laptop (also 66MHz bus, 256MB RAM, 10GB crappy laptop hard disk), over 802.11A. Took ~3 minutes 45 seconds.

Over 802.11A. On a 4000rpm hard disk. In a laptop that wasn't plugged in.

I got mine down to 1 minute 5 sec by turning my laptop off (didn't unpluc it, tho....

Can different motherboards support a differnt number of interupts on their PCI bus?

 

[ddrueding]

Can different motherboards support a differnt number of interupts on their PCI bus?

Assuming same bus speed, of course.....

 

[Mercutio]

I'm not positive but I think interupt requests depend on bus clock speed more than anything.

I have some kernels of goodness for an article. I spent a couple hours on an outline this weekend.

 

[Mercutio]

Building a Storage Server


In this day and age, an 80GB hard disk is more than most computer users can fill, but at the same time, there are some in the world for whom even 800GB is not enough.

800GB? That’s around 133 DVDs. It’s 10,000 hours of reasonably high-quality MP3 music, or 1100 CD-ROM image files, or perhaps a million digital pictures. It’s 70 hours of DV for a camcorder junky.

In short, it’s not enough for someone with a serious interest in digital media.

Desktop hard disks are available in sizes up to 300GB, but even at that, there’s a limit to what one can, or what one might want to do with an everyday desktop PC, particularly since many of these media hobbyists have two, three or even four PCs scattered about already.

In the business world, the problem of massive general storage needs is solved with a File Server – a general-use computer with a complex network-aware operating system dedicated almost exclusively to making storage available to hosts on a network. Even better, for some, is the Network Attached Storage device, a zero-maintenance “black box”, neither computer nor appliance but somewhere in between.

The new-found needs of the storage-starved media hobbyist, it seems, fall somewhere in between.

 

[CougTek]

Good intro, looking forward for the rest of the article.

 

[Mercutio]

I do have another storage server idea, one that might be appealing to budget-conscious types.

$150 P3 500, a lease-return machine, or maybe that Celeron 300A you've been ignoring for the last two years. Add a $30 Promise Fasttrack, four $60 80GB drives and a linux install, and for all of $240, you have 320GB of storage.

Which is a REALLY nice $/GB ratio.

I just got a bunch of the Gateway E4200s from one of our old classrooms. They're slimline desktops and they make NO NOISE. None. If you're three feet away, the only way you'd know they're on is to see the lights on the front or feel the air from the PSU (or if the CD-ROM was running, I guess).

What a perfect little storage appliance! And four of 'em - $1000, overmatch our 1GB needs very well.

 

[Mercutio]

That should be $270, if you've already got the PC. You guys are supposed to correct my math!

 

[blakerwry]

i like the intro as well... what about some of the devices mentioned on SR a few days ago (appliance NAS devices for the masses by D-link, NetDisk, and a few others)?

For 80-160GB they are somewhat compelling.... for more storage you'll need a real file server.


I think it would be fine to include the pII-pIII lease-return machine since they are easily available at some electronics stores (the local ones) and ebay. I was actually thinking of taking up an offer(via an email advertiesement I got from a company I've done business with before) on an old leased machine.. about $50 for a slow pIII with NIC, sound, vid, CD, smallish HDD.... They have these kinds of offers all the time.

 

[ddrueding]

I like the intro as well, what capacity are you planning on hitting?

 

[Mercutio]

Approximately 1TB disregarding RAID config.

I'm building a digital audio workstation this weekend (if I feel up to it. My foot is #$%#ing killing me) that's going to use a Promise TX4000 controller and 4 SATA 7k250s, on a Gigabyte 8I875.

College kids and their money (he wouldn't spring for the 3ware).

 

[Mercutio]

That's the 13G0255 - 250GB version.

Anyway, it'd be nice for some pictures.