<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN"> <HTML> <HEAD> <META NAME="GENERATOR" CONTENT="SGML-Tools 1.0.9"> <TITLE>Linux Benchmarking HOWTO: Introduction </TITLE> <LINK HREF="Benchmarking-HOWTO-2.html" REL=next> <LINK HREF="Benchmarking-HOWTO.html#toc1" REL=contents> </HEAD> <BODY> <A HREF="Benchmarking-HOWTO-2.html">Next</A> Previous <A HREF="Benchmarking-HOWTO.html#toc1">Contents</A> <HR> <H2><A NAME="s1">1. Introduction </A></H2> <P> <P><EM>"What we cannot speak about we must pass over in silence."</EM> <BLOCKQUOTE> <EM>Ludwig Wittgenstein (1889-1951), Austrian philosopher </EM> </BLOCKQUOTE> <P>Benchmarking means <B>measuring</B> the speed with which a computer system will execute a computing task, in a way that will allow comparison between different hard/software combinations. It <B>does not</B> involve user-friendliness, aesthetic or ergonomic considerations or any other subjective judgment. <P>Benchmarking is a tedious, repetitive task, and takes attention to details. Very often the results are not what one would expect, and subject to interpretation (which actually may be the most important part of a benchmarking procedure). <P>Finally, benchmarking deals with facts and figures, not opinion or approximation. <H2><A NAME="ss1.1">1.1 Why is benchmarking so important ? </A> </H2> <P> <P>Apart from the reasons pointed out in the BogoMips Mini-HOWTO (section 7, paragraph 2), one occasionally is confronted with a limited budget and/or minimum performance requirements while putting together a Linux box. In other words, when confronted with the following questions: <UL> <LI>How do I maximize performance within a given budget ? </LI> <LI>How do I minimize costs for a required minimum performance level ? </LI> <LI>How do I obtain the best performance/cost ratio (within a given budget or given performance requirements)? </LI> </UL> <P>one will have to examine, compare and/or produce benchmarks. Minimizing costs with no performance requirements usually involves putting together a machine with leftover parts (that old 386SX-16 box lying around in the garage will do fine) and does not require benchmarks, and maximizing performance with no cost ceiling is not a realistic situation (unless one is willing to put a Cray box in his/her living room - the leather-covered power supplies around it look nice, don't they ?). <P>Benchmarking per se is senseless, a waste of time and money; it is only meaningful as part of a decision process, i.e. if one has to make a choice between two or more alternatives. <P>Usually another parameter in the decision process is <B>cost</B>, but it could be availability, service, reliability, strategic considerations or any other rational, measurable characteristic of a computer system. When comparing the performance of different Linux kernel versions, for example, <B>stability</B> is almost always more important than speed. <H2><A NAME="ss1.2">1.2 Invalid benchmarking considerations</A> </H2> <P> <P>Very often read in newsgroups and mailing lists, unfortunately: <OL> <LI>Reputation of manufacturer (unmeasurable and meaningless). </LI> <LI>Market share of manufacturer (meaningless and irrelevant). </LI> <LI>Irrational parameters (for example, superstition or prejudice: would you buy a processor labeled 131313ZAP and painted pink ?) </LI> <LI>Perceived value (meaningless, unmeasurable and irrational). </LI> <LI>Amount of marketing hype: this one is the worst, I guess. I personally am fed up with the "XXX inside" or "kkkkkws compatible" logos (now the "aaaaaPowered" has joined the band - what next ?). IMHO, the billions of dollars spent on such campaigns would be better used by research teams on the design of new, faster, (cheaper :-) bug-free processors. No amount of marketing hype will remove a floating-point bug in the FPU of the brand-new processor you just plugged in your motherboard, but an exchange against a redesigned processor will. </LI> <LI>"You get what you pay for" opinions are just that: opinions. Give me the facts, please. </LI> </OL> <HR> <A HREF="Benchmarking-HOWTO-2.html">Next</A> Previous <A HREF="Benchmarking-HOWTO.html#toc1">Contents</A> </BODY> </HTML>