<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML
><HEAD
><TITLE
>What To Optimize</TITLE
><META
NAME="GENERATOR"
CONTENT="Modular DocBook HTML Stylesheet Version 1.7"><LINK
REL="HOME"
TITLE="The Unix Hardware Buyer HOWTO"
HREF="index.html"><LINK
REL="PREVIOUS"
TITLE="Buying the Basics"
HREF="basics.html"><LINK
REL="NEXT"
TITLE="But What If I'm Economizing?"
HREF="economizing.html"></HEAD
><BODY
CLASS="sect1"
BGCOLOR="#FFFFFF"
TEXT="#000000"
LINK="#0000FF"
VLINK="#840084"
ALINK="#0000FF"
><DIV
CLASS="NAVHEADER"
><TABLE
SUMMARY="Header navigation table"
WIDTH="100%"
BORDER="0"
CELLPADDING="0"
CELLSPACING="0"
><TR
><TH
COLSPAN="3"
ALIGN="center"
>The Unix Hardware Buyer HOWTO</TH
></TR
><TR
><TD
WIDTH="10%"
ALIGN="left"
VALIGN="bottom"
><A
HREF="basics.html"
ACCESSKEY="P"
>Prev</A
></TD
><TD
WIDTH="80%"
ALIGN="center"
VALIGN="bottom"
></TD
><TD
WIDTH="10%"
ALIGN="right"
VALIGN="bottom"
><A
HREF="economizing.html"
ACCESSKEY="N"
>Next</A
></TD
></TR
></TABLE
><HR
ALIGN="LEFT"
WIDTH="100%"></DIV
><DIV
CLASS="sect1"
><H1
CLASS="sect1"
><A
NAME="optimize"
></A
>4. What To Optimize</H1
><DIV
CLASS="sect2"
><H2
CLASS="sect2"
><A
NAME="AEN378"
></A
>4.1. First, add more memory</H2
><P
>Max out your memory. Having lots of free memory will improve your
virtual-memory performance (and Unix takes advantage of extra memory more
effectively than Windows does). Fortunately, with RAM as cheap as it is
now, a gigabyte or three is unlikely to bust your budget even if you're
economizing.</P
></DIV
><DIV
CLASS="sect2"
><H2
CLASS="sect2"
><A
NAME="AEN381"
></A
>4.2. Bus and Disk speeds</H2
><P
>Most people think of the processor as the most important choice in
specifying any kind of personal-computer system. But for typical job loads
under Linux, the processor type is nearly a red herring — it's far
more important to specify a capable system bus and disk I/O subsystem. If
you don't believe this, you may find it enlightening to keep
<SPAN
CLASS="citerefentry"
><SPAN
CLASS="refentrytitle"
>top</SPAN
>(1)</SPAN
>
running for a while as you use your machine. Notice how seldom the CPU
idle percentage drops below 90%!</P
><P
>It's true that after people upgrade their motherboards they often do
report a throughput increase. But this is often more due to other changes
that go with the processor upgrade, such as improved cache memory or an
increase in the clocking speed of the system's front-side bus (enabling
data to get in and out of the processor faster).</P
><P
>If you're buying for Linux on a fixed budget, it makes sense to trade
away some excess processor clocks to get a faster bus and disk subsystem.
If you're building a monster hot-rod, go ahead and buy that fastest
available processor — but once you've gotten past that gearhead
desire for big numbers, pay careful attention to bus speeds and your disk
subsystem, because that's where you'll get the serious performance wins.
The gap between processor speed and I/O subsystem throughput has only
widened in the last five years.</P
><P
>How does it translate into a recipe in 2007? Like this; if
you're building a hot rod,</P
><P
></P
><UL
><LI
><P
><EM
>Do</EM
> buy a machine with the fastest
available "front-side" (e.g. processor-to-memory) bus.</P
></LI
><LI
><P
><EM
>Do</EM
> get a high-speed SCSI controller
and the fastest SCSI disks you can get your hands on.</P
></LI
></UL
><P
>If you're economizing, you can back down on these. But in trading
away SCSI for SATA your reliability (expected time before failure) will
drop. We'll cover this in more detail in the next section.</P
></DIV
><DIV
CLASS="sect2"
><H2
CLASS="sect2"
><A
NAME="diskwars"
></A
>4.3. Disk Wars: SATA vs. SCSI</H2
><P
>For the fastest disks you can find, pay close attention to
average seek and latency time. The former is an average time
required to seek to any track; the latter is the maximum time
required for any sector on a track to come under the heads, and is
a function of the disk's rotation speed.</P
><P
>Of these, average seek time is <EM
>much</EM
> more
important. When you're running Linux or any other virtual-memory operating
system, a one millisecond faster seek time can make a really substantial
difference in system throughput. Back when PC processors were slow enough
for the comparison to be possible (and I was running System V Unix), it was
easily worth as much as a 30MHz increment in processor speed. Today the
corresponding figure would probably be as much as 300MHz!</P
><P
>The manufacturers themselves avoid running up seek time on the
larger-capacity drives by stacking platters vertically rather than
increasing the platter size. Thus, seek time (which is proportional
to the platter radius and head-motion speed) tends to be constant across
different capacities in the same product line. This is good because it
means you don't have to worry about a capacity-vs.-speed tradeoff.</P
><P
>Disks of less than 40GB capacity simply aren't being manufactured
anymore; there's no margin in them. Our spies tell us that all major disk
makers retooled their lines a while back to produce 9GB unit platters,
which are simply being stacked 2N per spindle to produce ranges of drives
with roughly 18GB increments of capacity. </P
><P
>Average drive latency is inversely proportional to the disk's
rotational speed. For years, most disks spun at 3600 rpm; most disks now
spin at 7,200 or 10,000rpm, and high-end disks at 15,000 rpm. These
fast-spin disks run extremely hot; cooling is becoming a critical
constraint in drive design.</P
><P
>Another basic decision is SATA vs. SCSI (the older IDE and EIDE buses
are obsolete). Either kind of disk costs about the same, but the premium
for a SCSI card varies all over the lot, partly because of price
differences between VLB and PCI SCSI cards and especially because many
motherboard vendors bundle a SATA chipset right on the system board. SCSI
gives you better speed and throughput and loads the processor less, a win
for larger disks and an especially significant consideration in a
multi-user environment; also it's more expandable. You can have at most four
SATA devices on a single controller. SCSI permits up to 7 (15 for wide
SCSI).</P
><P
>Admittedly, the case for SCSI has eroded a bit since 2001; the new
generation of SATA drives is very fast, and controller cards now normally
feature a channel per drive and DMA (Direct Memory Access), so that some of
of the multi-user contention problems that used to dog IDE have diminished.
At 10KRPM and below, SATA is as good as SCSI now (a painful admission for an
old-time IDE-hater like me), but at the 15KRPM high end SCSI still
rules.</P
><P
>Of course, SATA is cheaper. Many motherboards have SATA right on board
now; if not, you'll pay maybe $15 for a SATA adapter board, as opposed
to $200+ for the leading SCSI controller. Also, the cheap SCSI
cabling most vendors ship can be flaky. You have to use expensive
high-class cables for consistently good results. See <A
HREF="optimize.html#sutton"
>Mark Sutton's horror story</A
>.</P
><DIV
CLASS="sect3"
><H3
CLASS="sect3"
><A
NAME="AEN410"
></A
>4.3.1. Advantages of SCSI</H3
><P
>For starters, SCSI is still at least 10%-15% faster than IDE/ATAPI
running flat out. Like Windows, SATA I is layered over a pile of ancestral
designs (ST-506 and IDE) that's antiquated and prone to failure under
stress. For example, on the Tyan K7 motherboards, there are known
data-corruption problems with the ATA controller in the presence of various
DMA-using bus-mastering cards.</P
><P
>SCSI, on the other hand, was designed from the beginning to scale up
well to high-speed, high-throughput systems. Because it's perceived as a
<SPAN
CLASS="QUOTE"
>"professional"</SPAN
> choice, SCSI peripherals are generally better
engineered than IDE/ATAPI equivalents, and new high-performing drive
technologies tend to become available in SCSI first. You'll pay a few
dollars more, but for Linux the cost is well repaid in increased throughput
and reliability.</P
><P
>The one aspect of SCSI that often gets overlooked is that it is a
true multitasking interface, thanks to the
<SPAN
CLASS="QUOTE"
>"disconnect/reconnect"</SPAN
> sequence that almost all SCSI hardware
implements. With disconnect/reconnect, if a target device has to perform
some kind of time-consuming mechanical operation (e.g., a seek in the case
of a disk or a medium position operation in the case of a tape drive) the
device will release control of the SCSI bus and allow it to be used for
some other operation. IDE/ATAPI has no such capability and is often
responsible for a system stall while a disk, CD-drive or tape drive seeks
to the desired medium position.</P
><P
>(Incidentally, SCSI performance can sometimes be improved by setting
the ID of the most frequently used disk drive as high as possible.
The SCSI priority pecking order is such that devices with higher ID's
get first crack at the bus when arbitration occurs during the
selection phase.)</P
><P
>Rick's comments from 2001 are still apposite: "They call me a SCSI
bigot. So be it — but my hardware keeps being future-proof, I don't have
to run bizarre emulation layers to address CDRs, I never run low on IRQs or
resort to IRQ-sharing (on account of 3-4 ATA controllers each needing one,
plus special adapters for scanners, etc.), all my hard drives have
hardware-level hot-fix, all my hard disk/CD/tape/etc. devices support a
stable standard rather than this month's cheap extension kludge, and I
don't have to worry about adverse interactions at the hardware or driver
levels from mixing ATA and SCSI."</P
><P
>The cutting edge in SCSI devices is ultra wide LVD
(low-voltage-differential) SCSI drives with 320MB/sec transfer speed,
running over a 68-pin cable (this is twice as fast as the LVD-160 drives we
used last time around). Vendors often call LVD drives "SCSI-3", which is
incorrect as most of these devices don't have built-in support for the
entire SCSI-3 protocol, and it would be overdesign if they did (the extra
commands are designed for use with CD and multimedia devices).</P
><P
>Fast ultra LVD is a bit more expensive to support than the older
versions of SCSI (for which key words are "single-ended", describing the
electrical interface, and "narrow", describing the width of data transfers
over the older-style 50-pin connector). Thus, you're likely to find it
only on hard drives that are physically capable of doing high-speed
data access off their media; slower devices such as tapes and CD drives
are normally still built with the narrow single-ended variant.</P
><P
>The LVD-160 standard defines the SCSI bus, not the drive itself.
Therefore, when used with a single hard drive in a lightly loaded system
(e.g., a Linux machine supporting only one user) LVD-160 will have only a
marginal effect on system performance. This is because a single hard drive
running flat out will use only about 15-20 percent of the available
bandwidth, as current drive technology can manage no more than about 28-30
MB/sec off the platters, less if a time consuming seek is involved. This
rate could be higher, of course, if a read request was pending and the
drive had cached the desired data. Where the LVD-160 bandwidth really
becomes advantageous is in implementations of multiple drives (e.g., RAID
5) and/or when activities produce the frequent issue of drive access
commands. The latter condition would be common in any environment that
supports a lot of users.</P
><P
>Current SCSI drives are not quite fast enough to flood more than half
the SCSI bus bandwidth, so you can have at least two drives on a single bus
pumping full speed without using it up. In reality, you don't keep drives
running full speed all the time, so you should be able to have 3-4 drives
on a bus before you really start feeling bandwidth crunch.</P
></DIV
><DIV
CLASS="sect3"
><H3
CLASS="sect3"
><A
NAME="scsi_terms"
></A
>4.3.2. SCSI Terminology</H3
><P
>The following, by Ashok Singhal
<ashoks@duckjibe.eng.sun.com> of Sun Microsystems with additions
by your humble editor, is a valiant attempt to demystify SCSI
terminology.</P
><P
>The terms <SPAN
CLASS="QUOTE"
>"SCSI"</SPAN
>, <SPAN
CLASS="QUOTE"
>"SCSI-2"</SPAN
>, and
<SPAN
CLASS="QUOTE"
>"SCSI-3"</SPAN
> refer to three different specifications. Each
specification has a number of options. Many of these options are
independent of each other. I like to think of the main options (there are
others that I'll skip over because I don't know enough about them to talk
about them on the net) by classifying them into five categories:</P
><DIV
CLASS="sect4"
><H4
CLASS="sect4"
><A
NAME="AEN430"
></A
>4.3.2.1. Logical: SCSI-1, SCSI-2, SCSI-3</H4
><P
>This refers to the commands that the controllers understand. You'll
no longer see SCSI-1 in new hardware. SCSI-3 is a superset of SCSI-2
including commands intended for CD-R and streaming multimedia
devices.</P
></DIV
><DIV
CLASS="sect4"
><H4
CLASS="sect4"
><A
NAME="AEN433"
></A
>4.3.2.2. Electrical Interface</H4
><P
></P
><UL
><LI
><P
>single-ended (max cable length 6 meters)</P
></LI
><LI
><P
>differential (max cable length 25 meters)</P
></LI
></UL
><P
>This option is independent of command set, speed, and path width.
Differential is less common but allows higher transfer speeds, better noise
immunity and longer cables. It's rare in SCSI-1 controllers.</P
><P
>You will normally see single-ended SCSI controllers on
low-speed devices such as tapes and CD drives, and differential
SCSI on hard drives (look for the specification LVD which means
"low-voltage differential").</P
><P
>Nowadays most controllers support both electrical interfaces, but if
you mix LVD with single-ended on the same chain, the whole chain will fall
back to single-ended (and possibly halve the speed of the faster
devices).</P
></DIV
><DIV
CLASS="sect4"
><H4
CLASS="sect4"
><A
NAME="AEN443"
></A
>4.3.2.3. Handshake</H4
><P
></P
><UL
><LI
><P
>Asynchronous (acknowledge each word (8, 16 or 32 bits) transferred.</P
></LI
><LI
><P
>Synchronous (multiple-word transfers permitted between ACKS).</P
></LI
></UL
><P
>Synchronous is faster. This mode is negotiated between controller
and device; modes may be mixed on the same bus. This is independent
of command set, data width, and electrical interface.</P
></DIV
><DIV
CLASS="sect4"
><H4
CLASS="sect4"
><A
NAME="AEN451"
></A
>4.3.2.4. Synchronous Speed (does not apply for asynchronous option)</H4
><P
>Normal transfer speed is 5 megabytes/sec. The <SPAN
CLASS="QUOTE"
>"fast"</SPAN
>
option (10 mb/sec) is defined only in SCSI-2 and SCSI-3. Fast-20 (or
<SPAN
CLASS="QUOTE"
>"Ultra"</SPAN
>) is 20 mb/sec; Fast-40 (or "Ultra-2") is 40MB/sec.
The fast options basically defines shorter timing parameters such as the
assertion period and hold time.</P
><P
>The parameters of the synchronous transfer are negotiated
between each target and initiator so different speed transfers
can occur over the same bus.</P
></DIV
><DIV
CLASS="sect4"
><H4
CLASS="sect4"
><A
NAME="AEN457"
></A
>4.3.2.5. Path width</H4
><P
>The standard SCSI data path is 8 bits wide. The <SPAN
CLASS="QUOTE"
>"wide"</SPAN
>
option exploits a 16- or 32-bit data path (uses 68-pin rather than 50-pin
data cables). You also get 4-bit rather than 3-bit device IDs, so you can
have up to 16 devices. The wide option doubles or quadruples your transfer
rate, so for example a fast-20/wide SCSI link using 16 bits transfers
40mb/sec.</P
><P
>What are those <SPAN
CLASS="QUOTE"
>"LUN"</SPAN
> numbers you see when you boot up?
Think of them as sub-addresses on the SCSI bus. Most SCSI devices have
only one <SPAN
CLASS="QUOTE"
>"logical"</SPAN
> device inside them, thus they're LUN zero.
Some SCSI devices can, however, present more than one separate logical unit
to the bus master, with different LUNs (0 through 7). The only context in
which you'll normally use LUNs is with CD-ROM juke boxes. Some have been
marketed that offer up to 7 CD-ROMS with one read head. These use the LUN
to differentiate which disk to select.</P
><P
>(There's history behind this. Back in the days of EISA, drives were
supposed to be under the control of a separate SCSI controller, which
could handle up to 7 such devices (15 for wide SCSI). These drives
were to be the Logical Units; hence the LUN, or Logical Unit Number.
Then, up to 7 of these SCSI controllers would be run by the controller
that we today consider the SCSI controller. In practice, hardware
cost dropped so rapidly, and capability increased so rapidly, it
became more logical to embed the controller on the drive.)</P
></DIV
></DIV
><DIV
CLASS="sect3"
><H3
CLASS="sect3"
><A
NAME="AEN465"
></A
>4.3.3. Avoiding Pitfalls</H3
><P
>Here are a couple of rules and heuristics to follow:</P
><P
>Rule 1: Total SCSI cable length (both external and internal devices) must
not exceed six meters. For modern Ultra SCSI (with its higher speed)
cut that to three feet!</P
><P
>It's probably not a good idea to cable 20MB/s or faster SCSI devices
externally at all. If you must, one of our informants advises using a
Granite Digital <SPAN
CLASS="QUOTE"
>"perfect impedance"</SPAN
> teflon cable (or equivalent);
these cables basically provide a near-perfect electrical environment
for a decent price, and can be ordered in custom configurations if
needed.</P
><P
>A common error is to forget the length of the ribbon cable used
for internal devices when adding external ones (that is, devices
chained to the SCSI board's external connector).</P
><P
>Rule 2: Both ends of the bus have to be electrically terminated. </P
><P
>On older devices this is done with removable resistor packs
— typically 8-pin-inline widgets, yellow or blue, that are
plugged into a plastic connector somewhere near the edge of the PCB
board on your device. Peripherals commonly come with resistor packs
plugged in; you must <EM
>remove</EM
> the packs on all
devices except the two end ones in the physical chain.</P
><P
>Newer devices advertised as having "internal termination" have a
jumper or switch on the PCB board that enables termination. These
devices are preferable, because the resistor packs are easy to lose or
damage.</P
><P
>Rule 3: No more than seven devices per chain (fifteen for Wide
SCSI).</P
><P
>There are eight SCSI IDs per controller. The controller reserves ID 7
or 15, so your devices can use IDs 0 through 6 (or 0 through 14,
wide). No two devices can share an ID; if this happens by accident,
neither will work.</P
><P
>The conventional ID assignments are: Primary hard disk = ID 0,
Secondary hard disk = ID 1, Tape = ID 2. Some Unixes (notably SCO)
have these wired in. You select a device's ID with jumpers on the PCB
or a thumbwheel.</P
><P
>SCSI IDs are completely independent of physical device chain
position.</P
><P
>Heuristic A: You'll have fewer hassles if all your cables are made by
the same outfit. (This is due to impedence reflections from minor
mismatches. You can get situations where cable A will work with B,
cable B will work with C, but A and C aren't happy together. It's
also non-commutative. The fact that `computer to A to B' works
doesn't mean that `computer to B to A' will work.</P
><P
>Heuristic B. Beware Cheap SCSI Cables!</P
><P
>Mark Sutton tells the following instructive horror story in a
note dated 5 Apr 1997: </P
><P
>I recently added an additional SCSI hard drive to my home
machine. I bought an OEM packaged Quantum Fireball 2 gig SCSI drive
(meaning, I bought a drive in shrinkwrap, without so much as mounting
hardware or a manual. Thank God for Quantum's web page or I would
have had no idea how to disable termination or set the SCSI ID on this
sucker. Anyway, I digress...). I stuck the drive in an external
mounting kit that I found in a pile of discarded computer parts at
work and my that boss said I could have. (All 5 of my internal bays
were full of devices.)</P
><P
>Anyway, I had my drive, and my external SCSI mounting kit, I
needed a cable.</P
><P
>I went into my friendly local CompUSA in search of a SCSI cable,
and side-by-side, on two hooks, were two "identical" SCSI cables.
Both were 3 feet. Both had Centronics to Centronics connectors, both
were made by the same manufacturer. They had slightly different model
numbers. One was $16.00, one was $30.00. Of course, I bought the $16
cable.</P
><P
>Bad, I say, bad <EM
>bad</EM
> mistake. I hooked this
sucker up like so: </P
><TABLE
BORDER="0"
BGCOLOR="#E0E0E0"
WIDTH="100%"
><TR
><TD
><FONT
COLOR="#000000"
><PRE
CLASS="screen"
> +--------+ +-------+ +-----------+ +-------+
|Internal|---|Adaptec|-----|New Quantum|-----|UMAX |
|Devices | |1542CF | ^ | Disk | ^ |Scanner|
+--------+ +-------+ | +-----------+ | +-------+
| |
New $16 cable Cable that came
with scanner.
</PRE
></FONT
></TD
></TR
></TABLE
><P
>Shortly after booting, I found that data all over my old internal hard
drive was being hosed. This was happening in DOS as well as in Linux.
Any disk access on either disk was hosing data on both disks, attempts
to scan were resulting in corrupted scans *and* hosing files on the
hard disks. By the time I finished swapping cables around, and
checking terminations and settings, I had to restore both Linux and
DOS from backups. </P
><P
>I went back to CompUSA, exchanged the $16 cable for the $30 one, hooked
it up and had no more problems. </P
><P
>I carefully examined the cables and discovered that the $30 cable
contained 24 individual twisted pairs. Each data line was twisted
with a ground line. The $16 cable was 24 data wires with one overall
grounded shield. Yet, both of these cables (from the same
<EM
>manufacturer</EM
>) were being sold as SCSI cables! </P
><P
>You get what you pay for. </P
><P
>(Another correspondent guesses that the cheap cable probably
said <SPAN
CLASS="QUOTE"
>"Macintosh"</SPAN
> on it. The Mac connector is missing most of its
ground pins.)</P
></DIV
><DIV
CLASS="sect3"
><H3
CLASS="sect3"
><A
NAME="more_disks"
></A
>4.3.4. More Resources</H3
><P
>There's a USENET
<A
HREF="http://www.cis.ohio-state.edu/hypertext/faq/usenet/scsi-faq/top.html"
TARGET="_top"
> SCSI FAQ</A
>. Also see the home page of the
<A
HREF="http://www.t10.org"
TARGET="_top"
>T10
committee</A
> that writes SCSI standards.</P
><P
>There is a large searchable database of hard disk and controller
information at the <A
HREF="http://www.pc-disk.de/pcdisk.htm"
TARGET="_top"
>PC DISK
Hardware Database</A
>.</P
></DIV
></DIV
><DIV
CLASS="sect2"
><H2
CLASS="sect2"
><A
NAME="iotune"
></A
>4.4. Tuning Your I/O Subsystem</H2
><P
><EM
>(This section comes to us courtesy of Perry The Cynic,
<perry@sutr.cynic.org>; it was written in 1998. My own experience
agrees pretty completely with his. I have revised the numbers in it since
to reflect more recent developments.)</EM
></P
><P
>Building a good I/O subsystem boils down to two major points:
<EM
>pick matched components</EM
> so you don't over-build any
piece without benefit, and <EM
>construct the whole pipe such that
it can feed what your OS/application combo needs</EM
>.</P
><P
>It's important to recognize that <SPAN
CLASS="QUOTE"
>"balance"</SPAN
> is with
respect to not only a particular processor/memory subsystem, but also to a
particular OS and application mix. A Unix server machine running the whole
TCP/IP server suite has radically different I/O requirements than a
video-editing workstation. For the <SPAN
CLASS="QUOTE"
>"big boys"</SPAN
> a good
consultant will sample the I/O mix (by reading existing system performance
logs or taking new measurements) and figure out how big the I/O system
needs to be to satisfy that app mix. This is not something your typical
Linux buyer will want to do; for one, the application mix is not static and
will change over time. So what you'll do instead is design an I/O subsystem
that is internally matched and provides maximum potential I/O performance
for the money you're willing to spend. Then you look at the price points
and compare them with those for the memory subsystem. That's the most
important trade-off inside the box.</P
><P
>So the job now is to design and buy an I/O subsystem that is well
matched to provide the best bang for your buck. The two major performance
numbers for disk I/O are latency and bandwidth. Latency is how long a
program has to wait to get a little piece of random data it asked for.
Bandwidth is how much contiguous data can be sent to/from the disk once
you've done the first piece. Latency is measured in milliseconds (ms);
bandwidth in megabytes per second (MB/s). Obviously, a third number of
interest is how big all of your disks are together (how much storage you've
got), in Gigabytes (GB).</P
><P
>Within a rather big envelope, minimizing latency is the cat's meow.
Every millisecond you shave off effective latency will make your system
feel significantly faster. Bandwidth, on the other hand, only helps you
if you suck a big chunk of contiguous data off the disk, which happens
rarely to most programs. You have to keep bandwidth in mind to avoid
mis-matching pieces, because (obviously) the lowest usable bandwidth in
a pipe constrains everything else.</P
><P
>I'm going to ignore IDE. IDE is no good for multi-processing systems,
period. You may use an IDE CD-ROM if you don't care about its
performance, but if you care about your I/O performance, go SCSI.
(Beware that if you mix an IDE CD-ROM with SCSI drives under Linux,
you'll have to run a SCSI emulation layer that is a bit flaky.)</P
><P
>Let's look at the disks first. Whenever you seriously look at a
disk, <EM
>get its data sheet</EM
>. Every reputable
manufacturer has them on their website; just read off the product code
and follow the bouncing lights. Beware of numbers (`<12ms fast!')
you may see in ads; these folks often look for the lowest/highest
numbers on the data sheet and stick them into the ad copy. Not
dishonest (usually), but ignorant.</P
><P
>What you need to find out for a disk is:</P
><P
></P
><OL
TYPE="1"
><LI
><P
>What kind of SCSI interface does it have? Look for
"fast", "ultra", and "wide". Ignore disks that say "fiber"
(this is a specialty physical layer not appropriate for the insides
of small computers). Note that you'll often find the same disk with
different interfaces.</P
></LI
><LI
><P
>What is the "typical seek" time (ms)? Make sure
you get "typical", not "track-to-track" or "maximum" or some other
measure (these don't relate in obvious ways, due to things like
head-settling time).</P
></LI
><LI
><P
>What is the rotational speed? This is typically
4500, 5400, 7200, 10000, or 15000 rpm (rotations per minute). Also look
for "rotational latency" (in ms). (In a pinch, average rotational
latency is approx. 30000/rpm in milliseconds.)</P
></LI
><LI
><P
>What is the ‘media transfer rate’ or speed (in
MB/s)? Many disks will have a range of numbers (say,
7.2-10.8MB/s). Don't confuse this with the "interface transfer rate"
which is always a round number (10 or 20 or 40MB/s) and is the speed of
the SCSI bus itself.</P
></LI
></OL
><P
>These numbers will let you do apple-with-apples comparisons of disks.
Beware that they will differ on different-size models of the same disk;
typically, bigger disks have slower seek times.</P
><P
>Now what does it all mean? Bandwidth first: the `media transfer rate'
is how much data you can, under ideal conditions, get off the disk per
second. This is a function mostly of rotation speed; the faster the
disk rotates, the more data passes under the heads per time unit. This
constrains the sustained bandwidth of <EM
>this disk</EM
>.</P
><P
>More interestingly, your effective latency is the sum of typical seek
time and rotational latency. So for a disk with 8.5ms seek time and 4ms
rotational latency, you can expect to spend about 12.5ms between the
moment the disk `wants' to read your data and the moment when it
actually starts reading it. This is the one number you are trying to
make small. Thus, you're looking for a disk with low seek times and
high rotation (RPM) rates.</P
><P
>For comparison purposes, the first hard drive I ever bought was a
20MB drive with 65ms seek time and about 3000RPM rotation. A floppy drive
has about 100-200ms seek time. A CD-ROM drive can be anywhere between 120ms
(fast) and 400ms (slow). The best IDE harddrives have about 10-12ms and
5400 rpm. The best SCSI harddrive I know (the Fujitsu MAM) runs
3.9ms/15000rpm.</P
><P
>Fast, big drives are expensive. Really big drives are very
expensive. Really fast drives are pretty expensive. On the other end,
really slow, small drives are cheap but not cost effective, because it
doesn't cost any less to make the cases, ship the drives, and sell
them.</P
><P
>In between is a ‘sweet spot’ where moving in either
direction (cheaper or more expensive) will cost you more than you get out
of it. The sweet spot moves (towards better value) with time. Right now
(early 2004), it's about at 36GB drives, 6ms, 10000rpm, ultra2 SCSI. If you
can make the effort, go to your local computer superstore and write down a
dozen or so drives they sell ‘naked’. (If they don't sell at
least a dozen hard drives naked, find yourself a better store. Use the Web,
Luke!) Plot cost against size, seek and rotational speed, and it will
usually become pretty obvious which ones to get for your budget.</P
><P
>Do look for specials in stores; many superstores buy overstock from
manufacturers. If this is near the sweet spot, it's often
surprisingly cheaper than comparable drives. Just make sure you
understand the warranty procedures.</P
><P
>Note that if you need a lot of capacity, you may be better off with
two (or more) drives than a single, bigger one. Not only can it be cheaper
but you end up with two separate head assemblies that move independently,
which can cut down on latency quite a bit (see below).</P
><P
>Once you've decided which kind of drive(s) you want, you must decide
how to distribute them over one or more SCSI buses. Yes, you
<EM
>may</EM
> want more than one SCSI bus. (My current desktop
machine has three.) Essentially, the trick is to make sure that all the
disks on one bus, talking at the same time, don't exceed the capacity of
that bus. At this time, I can't recommend anything but an Ultra/Wide SCSI
controller. This means that the attached SCSI bus can transfer data at up
to 40MB/s for an Ultra/Wide disk, 20MB/s for an Ultra/narrow disk, and
10MB/s for a `fast SCSI' disk. These numbers allow you do do your math: an
8MB/s disk will eat an entire bus on its own if it's ‘fast’
(10MB/s). Three 6MB/s ultra/narrow disks fit onto one bus
(3x6=18MB/s<20MB/s), but just barely. Two ultra/wide Cheetahs (12.8MB/s)
will share an (ultra/wide) bus (25.6<40), but they would collide on an
ultra/narrow bus, and any one Cheetah would be bandwidth constrained on a
(non-ultra) `fast' bus (12.8 > 10).</P
><P
>If you find that you need two SCSI buses, you can go with ‘dual
channel’ versions of many popular SCSI controller cards (including
the Adaptec). These are simply two controllers on one card (thus taking
only one PCI slot). This is cheaper and more compact than two cards;
however, on some motherboards with more than 3 PCI slots, using two cards
may be somewhat faster (ask me what a PCI bridge is :-).</P
><P
>SCSI performance can sometimes be improved by setting the ID of the
most frequently used disk drive as high as possible. The SCSI priority
pecking order is such that devices with higher ID's get first crack at the
bus when arbitration occurs during the selection phase.</P
><P
>How do you deal with slow SCSI devices — CD-ROMS, scanners, tape
drives, etc.? If you stick these onto a SCSI bus with fast disks,
they will slow down things a bit. You can either accept that (as in <SPAN
CLASS="QUOTE"
>"I
hardly ever use my scanner anyway"</SPAN
>), or stick them onto a separate
SCSI bus off a cheap controller card. Or you can (try to) get an ATA
version to stick onto that inevitable IDE interface on your
motherboard. The same logic applies to disks you won't normally use,
such as removables for data exchange.</P
><P
>If you find yourself at the high end of the bandwidth game, be aware
that the theoretical maximum of the PCI bus itself is 132MB/s. That
means that a dual ultra/wide SCSI controller (2x40MB/s) can fill more
than half of the PCI bus's bandwidth, and it is not advised to add
another fast controller to that mix. As it is, your device driver
better be well written, or your entire system will melt down (figuratively
speaking).</P
><P
>Incidentally, all of the numbers I used are ‘optimal’
bandwidth numbers. The real scoop is usually somewhere between 50-70% of
nominal, but things tend to cancel out — the drives don't quite
transfer as fast as they might, but the SCSI bus has overhead too, as does
the controller card.</P
><P
>Whether you have a single disk or multiple ones, on one or several
SCSI buses, you should give careful thought to their partition layout.
Given a set of disks and controllers, this is the most crucial
performance decision you'll make.</P
><P
>A partition is a contiguous group of sectors on the disk. Partitioning
typically starts at the outside and proceeds inwards. All partitions
on one disk share a single head assembly. That means that if you try
to overlap I/O on the first and last partition of a disk, the heads
must move full stroke back and forth over the disk, which can
radically increase seek time delays. A partition that is in the
middle of a partition stack is likely to have best seek performance,
since at worst the heads only have to move half-way to get there (and
they're likely to be around the area anyway).</P
><P
>Whenever possible, split partitions that compete onto different
disks. For example, /usr and the swap should be on different disks if
at all possible (unless you have outrageous amounts of RAM).</P
><P
>Another wrinkle is that most modern disks use `zone sectoring'. The
upshot is that outside partitions will have higher bandwidth than inner
ones (there is more data under the heads per revolution). So if you
need a work area for data streaming (say, a CD-R pre-image to record),
it should go on an outside (early numbered) partition of a
fast-rotating disk. Conversely, it's a good convention to put
rarely-used, performance-uncritical partitions on the inside (last).</P
><P
>Another note concerns SCSI mode pages. Each (modern) SCSI disk has a
small part of its disk (or a dedicated EEPROM) reserved for persistent
configuration information. These parameters are called ‘mode
pages’, for the mechanism (in the SCSI protocol) for accessing
them. Mode page parameters determine, among others, how the disk will
write-cache, what forms of error recovery it uses, how its RAM cache is
organized, etc. Very few configuration utilities allow access to mode page
parameters (I use FWB Toolkit on a Mac — it's simply the best tool I know
for that task), and the settings are usually factory preset for, uh,
Windows 95 environments with marginal hardware and single-user operation.
Particularly the cache organization and disconnect/reconnect pages can make
a tremendous difference in actual performance. Unfortunately there's really
no easy lunch here - if you set mode page parameters wrong, you can screw
up your data in ways you won't notice until months later, so this is
definitely `no playing with the pushebuttons' territory.</P
><P
>Ah yes, caches. There are three major points where you could cache I/O
buffers: the OS, the SCSI controller, and the on-disk controller.
Intelligent OS caching is by far the biggest win, for many reasons. RAM
caches on SCSI controller cards are pretty pointless these days; you
shouldn't pay extra for them, and experiment with disabling them if
you're into tinkering.</P
><P
>RAM caches on the drives themselves are a mixed bag. At moderate size
(1-2MB), they are a potential big win for Windows 95/98, because
Windows has stupid VM and I/O drivers. If you run a true multi-tasking
OS like Linux, having unified RAM caches on the disk is a significant
loss, since the overlapping I/O threads kick each other out of the
cache, and the disk ends up performing work for nothing.</P
><P
>Most high-performance disks can be reconfigured (using mode page
parameters, see above) to have `segmented' caches (sort of like a
set-associative memory cache). With that configured properly, the RAM
caches can be a moderate win, not because caching is so great on the
disk (it's much better in the OS), but because it allows the disk
controller more flexibility to reschedule its I/O request queue. You
won't really notice it unless you routinely have >2 I/O requests
pending at the SCSI level. The conventional wisdom (try it both ways)
applies.</P
><P
>And finally I <EM
>do</EM
> have to make a
disclaimer. Much of the stuff above is shameless simplification. In
reality, high-performance SCSI disks are very complicated
beasties. They run little mini-operating systems that are most
comfortable if they have 10-20 I/O requests pending <EM
>at the
same time</EM
>. Under those circumstances, the amortized global
latencies are much reduced, though any single request may experience
<EM
>longer</EM
> latencies than if it were the only one
pending. The only really valid analysis are stochastic-process models,
which we <EM
>really</EM
> don't want to get into
here. :-)</P
></DIV
></DIV
><DIV
CLASS="NAVFOOTER"
><HR
ALIGN="LEFT"
WIDTH="100%"><TABLE
SUMMARY="Footer navigation table"
WIDTH="100%"
BORDER="0"
CELLPADDING="0"
CELLSPACING="0"
><TR
><TD
WIDTH="33%"
ALIGN="left"
VALIGN="top"
><A
HREF="basics.html"
ACCESSKEY="P"
>Prev</A
></TD
><TD
WIDTH="34%"
ALIGN="center"
VALIGN="top"
><A
HREF="index.html"
ACCESSKEY="H"
>Home</A
></TD
><TD
WIDTH="33%"
ALIGN="right"
VALIGN="top"
><A
HREF="economizing.html"
ACCESSKEY="N"
>Next</A
></TD
></TR
><TR
><TD
WIDTH="33%"
ALIGN="left"
VALIGN="top"
>Buying the Basics</TD
><TD
WIDTH="34%"
ALIGN="center"
VALIGN="top"
> </TD
><TD
WIDTH="33%"
ALIGN="right"
VALIGN="top"
>But What If I'm Economizing?</TD
></TR
></TABLE
></DIV
></BODY
></HTML
>
