<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /><title>Traffic Shaping/Control</title><link rel="stylesheet" href="html.css" type="text/css" /><meta name="generator" content="DocBook XSL Stylesheets V1.73.2" /></head><body><div class="article" lang="en" xml:lang="en"><div class="titlepage"><div><div><h2 class="title"><a id="id257527"></a>Traffic Shaping/Control</h2></div><div><div class="authorgroup"><div class="author"><h3 class="author"><span class="firstname">Tom</span> <span class="surname">Eastep</span></h3></div><div class="author"><h3 class="author"><span class="firstname">Arne</span> <span class="surname">Bernin</span></h3></div></div></div><div><p class="copyright">Copyright © 2001-2008 Thomas M. Eastep</p></div><div><p class="copyright">Copyright © 2005 Arne Bernin & Thomas M. Eastep</p></div><div><div class="legalnotice"><a id="id257928"></a><p>Permission is granted to copy, distribute and/or modify this
document under the terms of the GNU Free Documentation License, Version
1.2 or any later version published by the Free Software Foundation; with
no Invariant Sections, with no Front-Cover, and with no Back-Cover
Texts. A copy of the license is included in the section entitled
“<span class="quote"><a class="ulink" href="GnuCopyright.htm" target="_self">GNU Free Documentation
License</a></span>”.</p></div></div><div><p class="pubdate">2008/12/15</p></div></div><hr /></div><div class="toc"><p><b>Table of Contents</b></p><dl><dt><span class="section"><a href="#Intro">Introduction</a></span></dt><dt><span class="section"><a href="#LinuxTC">Linux traffic shaping and control</a></span></dt><dt><span class="section"><a href="#Kernel">Linux Kernel Configuration</a></span></dt><dt><span class="section"><a href="#Shorewall">Enable TC support in Shorewall</a></span></dt><dt><span class="section"><a href="#Builtin">Using builtin traffic shaping/control</a></span></dt><dd><dl><dt><span class="section"><a href="#tcdevices">/etc/shorewall/tcdevices</a></span></dt><dt><span class="section"><a href="#tcclasses">/etc/shorewall/tcclasses</a></span></dt><dt><span class="section"><a href="#tcrules">/etc/shorewall/tcrules</a></span></dt><dt><span class="section"><a href="#ppp">ppp devices</a></span></dt><dt><span class="section"><a href="#Real">Real life examples</a></span></dt><dd><dl><dt><span class="section"><a href="#Wondershaper">Configuration to replace Wondershaper</a></span></dt><dt><span class="section"><a href="#simiple">A simple setup</a></span></dt></dl></dd></dl></dd><dt><span class="section"><a href="#Xen">A Warning to Xen Users</a></span></dt><dt><span class="section"><a href="#Downloads">Shaping Download Traffic</a></span></dt><dt><span class="section"><a href="#IFB">Intermediate Frame Block (IFB) Devices</a></span></dt><dd><dl><dt><span class="section"><a href="#tcfilters">/etc/shorewall/tcfilters</a></span></dt></dl></dd><dt><span class="section"><a href="#show">Understanding the output of 'shorewall show tc'</a></span></dt><dt><span class="section"><a href="#External">Using your own tc script</a></span></dt><dd><dl><dt><span class="section"><a href="#owntcstart">Replacing builtin tcstart file</a></span></dt><dt><span class="section"><a href="#Start">Traffic control outside Shorewall</a></span></dt></dl></dd><dt><span class="section"><a href="#Testing">Testing Tools</a></span></dt></dl></div><div class="important" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Important</h3><p>Traffic shaping is complex and the Shorewall community is not well
equipped to answer traffic shaping questions. So if you are the type of
person who needs "insert tab A into slot B" instructions for everything
that you do, then please don't try to implement traffic shaping using
Shorewall. You will just frustrate yourself and we won't be able to help
you.</p></div><div class="warning" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Warning</h3><p>Said another way, reading just Shorewall documentation is not going
to give you enough background to use this material.</p><p>At a minimum, you will need to refer to at least the following
additional information:</p><div class="itemizedlist"><ul type="disc"><li><p>The LARTC HOWTO: <a class="ulink" href="http://www.lartc.org" target="_self">http://www.lartc.org</a></p></li><li><p>The HTB User's Guide: <a class="ulink" href="http://luxik.cdi.cz/~devik/qos/htb/manual/userg.htm" target="_self">http://luxik.cdi.cz/~devik/qos/htb/manual/userg.htm</a></p></li><li><p>Some of the documents listed at <a class="ulink" href="http://www.netfilter.org/documentation/index.html#documentation-howto" target="_self">http://www.netfilter.org/documentation/index.html#documentation-howto</a>.
The tutorial by Oskar Andreasson is particularly good.</p></li><li><p>The output of <span class="command"><strong>man iptables</strong></span></p></li></ul></div></div><div class="section" lang="en" xml:lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a id="Intro"></a>Introduction</h2></div></div></div><p>Starting with Version 2.5.5, Shorewall has builtin support for
traffic shaping and control. Before this version, the support was quite
limited. You were able to use your own tcstart script (and you still are),
but besides the tcrules file it was not possible to define classes or
queuing disciplines inside the Shorewall config files.</p><p>The support for traffic shaping and control still does not cover all
options available (and especially all algorithms that can be used to queue
traffic) in the Linux kernel but it should fit most needs. If you are
using your own script for traffic control and you still want to use it in
the future, you will find information on how to do this, <a class="link" href="#owntcstart" title="Replacing builtin tcstart file"> later in this document</a>. But for this to work,
you will also need to enable traffic shaping in the kernel and Shorewall
as covered by the next sections.</p></div><div class="section" lang="en" xml:lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a id="LinuxTC"></a>Linux traffic shaping and control</h2></div></div></div><p>This section gives a brief introduction of how controlling traffic
with the Linux kernel works. Although this might be enough for configuring
it in the Shorewall configuration files, we strongly recommend that you
take a deeper look into the <a class="ulink" href="http://lartc.org/howto/" target="_self">Linux
Advanced Routing and Shaping HOWTO</a>. At the time of writing this,
the current version is 1.0.0.</p><p>Since kernel 2.2 Linux has extensive support for controlling
traffic. You can define different algorithms that are used to queue the
traffic before it leaves an interface. The standard one is called pfifo
and is (as the name suggests) of the type First In First out. This means,
that it does not shape anything, if you have a connection that eats up all
your bandwidth, this queuing algorithm will not stop it from doing
so.</p><p>For Shorewall traffic shaping we use two algorithms, one is called
HTB (Hierarchical Token Bucket) and SFQ (Stochastic Fairness Queuing). SFQ
is easy to explain: it just tries to track your connections (tcp or udp
streams) and balances the traffic between them. This normally works well.
HTB allows you to define a set of classes, and you can put the traffic you
want into these classes. You can define minimum and maximum bandwidth
settings for those classes and order them hierarchically (the less
prioritized classes only get bandwidth if the more important have what they
need). Shorewall builtin traffic shaping allows you to define these
classes (and their bandwidth limits), and it uses SFQ inside these classes
to make sure, that different data streams are handled equally.</p><p><span class="bold"><strong>If you are running Shorewall-shell or if you
are running Shorewall-perl 4.1.5 or earlier:</strong></span></p><div class="blockquote"><blockquote class="blockquote"><p><span class="bold"><strong>You can only shape outgoing traffic. The
reason for this is simple, the packets were already received by your
network card before you can decide what to do with them</strong></span>. So
the only choice would be to drop them which normally makes no sense
(since you received the packet already, it went through the possible
bottleneck (the incoming connection). The next possible bottleneck
might come if the packet leaves on another interface, so this will be
the place where queuing might occur. So, defining queues for incoming
packets is not very useful, you just want to have it forwarded to the
outgoing interface as fast as possible.</p><p>There is one exception, though. Limiting incoming traffic to a
value a bit slower than your actual line speed will avoid queuing on
the other end of that connection. This is mostly useful if you don't
have access to traffic control on the other side and if this other
side has a faster network connection than you do (the line speed
between the systems is the bottleneck, e.g. a DSL or Cable Modem
connection to your provider's router, the router itself is normally
connected to a much faster backbone). So, if you drop packets that are
coming in too fast, the underlying protocol might recognize this and
slow down the connection. TCP has a builtin mechanism for this, UDP
has not (but the protocol over UDP might recognize it , if there is
any).</p><p>The reason why queuing is bad in these cases is, that you might
have packets which need to be prioritized over others, e.g. VoIP or ssh.
For this type of connections it is important that packets arrive in a
certain amount of time. For others like HTTP downloads, it does not
really matter if it takes a few seconds more.</p><p>If you have a large queue on the other side and the router there
does not care about QoS or the QoS bits are not set properly, your
important packets will go into the same queue as your less
time critical download packets which will result in a large
delay.</p></blockquote></div><p><span class="bold"><strong>If you are running Shorewall-perl 4.1.6 or
later:</strong></span></p><div class="blockquote"><blockquote class="blockquote"><p>You can shape incoming traffic through use of an
<em class="firstterm">Intermediate Frame Block</em> (IFB) device. <a class="link" href="#IFB" title="Intermediate Frame Block (IFB) Devices">See below</a>. <span class="bold"><strong>But beware:
using an IFB can result in queues building up both at your ISPs router
and at your own.</strong></span></p></blockquote></div><p><span class="bold"><strong>This is not to say that you cannot shape
download traffic, regardless of which Shorewall release you are
running</strong></span>.</p><div class="blockquote"><blockquote class="blockquote"><p>If you wish to shape downloads, you can always configure traffic
shaping on your firewall's local interface. An example appears <a class="link" href="#Downloads" title="Shaping Download Traffic">below</a>.</p><p>Again, however, <span class="bold"><strong>this can result in queues
building up both at your ISPs router and at your own</strong></span>.</p></blockquote></div><p>You shape and control outgoing traffic by assigning the traffic to
<em class="firstterm">classes</em>. Each class is associated with exactly one
network interface and has a number of attributes:</p><div class="orderedlist"><ol type="1"><li><p>PRIORITY - Used to give preference to one class over another
when selecting a packet to send. The priority is a numeric value with
1 being the highest priority, 2 being the next highest, and so
on.</p></li><li><p>RATE - The minimum bandwidth this class should get, when the
traffic load rises. Classes with a higher priority (lower PRIORITY
value) are served even if there are others that have a guaranteed
bandwidth but have a lower priority (higher PRIORITY value).</p></li><li><p>CEIL - The maximum bandwidth the class is allowed to use when
the link is idle.</p></li><li><p>MARK - Netfilter has a facility for
<em class="firstterm">marking</em> packets. Packet marks have a numeric
value which is limited in Shorewall to the values 1-255. You assign
packet marks to different types of traffic using entries in the
<code class="filename">/etc/shorewall/tcrules</code> file.</p></li></ol></div><p>One class for each interface must be designated as the
<em class="firstterm">default class</em>. This is the class to which unmarked
traffic (packets to which you have not assigned a mark value in
<code class="filename">/etc/shorewall/tcrules</code>) is assigned.</p><p>Netfilter also supports a mark value on each connection. You can
assign connection mark values in
<code class="filename">/etc/shorewall/tcrules</code>, you can copy the current
packet's mark to the connection mark (SAVE), or you can copy the
connection mark value to the current packet's mark (RESTORE). For more
information, see<a class="ulink" href="PacketMarking.html" target="_self"> this
article</a>.</p></div><div class="section" lang="en" xml:lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a id="Kernel"></a>Linux Kernel Configuration</h2></div></div></div><p>You will need at least kernel 2.4.18 for this to work, please take a
look at the following screenshot for what settings you need to enable. For
builtin support, you need the HTB scheduler, the Ingress scheduler, the
PRIO pseudoscheduler and SFQ queue. The other scheduler or queue
algorithms are not needed.</p><p>This screen shot shows how I configured QoS in a 2.6.16
Kernel:</p><div align="center"><img src="images/traffic_shaping2.6.png" align="middle" /></div><p>And here's my recommendation for a 2.6.21 kernel:</p><div align="center"><img src="images/traffic_shaping2.6.21.png" align="middle" /></div></div><div class="section" lang="en" xml:lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a id="Shorewall"></a>Enable TC support in Shorewall</h2></div></div></div><p>You need this support whether you use the builtin support or whether
you provide your own tcstart script.</p><p>To enable the builtin traffic shaping and control in Shorewall, you
have to do the following:</p><div class="itemizedlist"><ul type="disc"><li><p>Set <span class="bold"><strong>TC_ENABLED</strong></span> to "<span class="bold"><strong>Internal</strong></span>" in /etc/shorewall/shorewall.conf.
Setting <span class="bold"><strong>TC_ENABLED=Yes</strong></span> causes
Shorewall to look for an external tcstart file (See <a class="link" href="#tcstart">a later section</a> for details).</p></li><li><p>Setting <span class="bold"><strong>CLEAR_TC</strong></span> parameter in
/etc/shorewall/shorewall.conf to <span class="bold"><strong>Yes</strong></span>
will clear the traffic shaping configuration during Shorewall
[re]start and Shorewall stop. This is normally what you want when
using the builtin support (and also if you use your own tcstart
script)</p></li><li><p>The other steps that follow depend on whether you use your own
script or the builtin solution. They will be explained in the
following sections.</p></li></ul></div></div><div class="section" lang="en" xml:lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a id="Builtin"></a>Using builtin traffic shaping/control</h2></div></div></div><p>Shorewall's builtin traffic shaping feature provides a thin layer on
top of the ingress qdesc, HTB and SFQ. That translation layer allows you
to:</p><div class="itemizedlist"><ul type="disc"><li><p>Define HTB classes using Shorewall-style column-oriented
configuration files.</p></li><li><p>Integrate the reloading of your traffic shaping configuration
with the reloading of your packet-filtering and marking
configuration.</p></li><li><p>Assign traffic to HTB classes by TOS value.</p></li><li><p>Assign outgoing TCP ACK packets to an HTB class.</p></li><li><p>Assign traffic to HTB classes based on packet mark value.</p></li></ul></div><div class="warning" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Warning</h3><p>Shorewall's builtin traffic shaping feature is limited to ten (10)
devices.</p></div><p>Those few features are really all that builtin traffic
shaping/control provides; consequently, you need to understand HTB and
Linux traffic shaping as well as Netfilter packet marking in order to use
the facility. Again, please see the links at top of this article.</p><p>For defining bandwidths (for either devices or classes) please use
kbit or kbps (for Kilobytes per second) and make sure there is <span class="bold"><strong>NO</strong></span> space between the number and the unit (it is
100kbit <span class="bold"><strong>not</strong></span> 100 kbit). Using mbit, mbps
or a raw number (which means bytes) could be used, but note that only
integer numbers are supported (0.5 is <span class="bold"><strong>not
valid</strong></span>).</p><p><span class="bold"><strong>To properly configure the settings for your
devices you need to find out the real up- and downstream rates you
have</strong></span>. This is especially the case, if you are using a DSL
connection or one of another type that do not have a guaranteed bandwidth.
Don't trust the values your provider tells you for this; especially
measuring the real download speed is important! There are several online
tools that help you find out; search for "dsl speed test" on google (For
Germany you can use <a class="ulink" href="http://www.speedcheck.arcor.de/cgi-bin/speedcheck.cgi" target="_self">arcor speed
check</a>). Be sure to choose a test located near you.</p><div class="section" lang="en" xml:lang="en"><div class="titlepage"><div><div><h3 class="title"><a id="tcdevices"></a>/etc/shorewall/tcdevices</h3></div></div></div><p>This file allows you to define the incoming and outgoing bandwidth
for the devices you want traffic shaping to be enabled. That means, if
you want to use traffic shaping for a device, you have to define it
here.</p><p>Columns in the file are as follows:</p><div class="itemizedlist"><ul type="disc"><li><p>INTERFACE - Name of interface. Each interface may be listed
only once in this file. You may NOT specify the name of an alias
(e.g., eth0:0) here; see <a class="ulink" href="FAQ.htm#faq18" target="_self">FAQ #18</a>.
You man NOT specify wildcards here, e.g. if you have multiple ppp
interfaces, you need to put them all in here! With Shorewall
versions prior to 3.0.8 and 3.2.0 Beta 8, the device named in this
column must exist at the time that Shorewall is started, restarted
or refreshed. Beginning with Shorewall 3.0.8 and 3.2.0 Beta 8,
Shorewall will determine if the device exists and will only
configure the device if it does exist. If it doesn't exist, the
following warning is issued:</p><p><span class="bold"><strong>WARNING: Device <device name> not
found -- traffic-shaping configuration skipped</strong></span></p><p>Shorewall assigns a sequential <em class="firstterm">interface
number</em> to each interface (the first entry in
<code class="filename">/etc/shorewall/tcdevices</code> is interface 1, the
second is interface 2 and so on) Beginning with Shorewall-perl
4.1.6, you can explicitly specify the interface number by prefixing
the interface name with the number and a colon (":"). Example:
1:eth0.</p></li><li><p>IN-BANDWIDTH - The incoming Bandwidth of that interface.
Please note that when you use this column, you are not traffic
shaping incoming traffic, as the traffic is already received before
you could do so. This Column allows you to define the maximum
traffic allowed for this interface in total, if the rate is
exceeded, the excess packets are dropped. You want this mainly if
you have a DSL or Cable Connection to avoid queuing at your
providers side. If you don't want any traffic to be dropped set this
to a value faster than your interface maximum rate (or to 0 (zero),
if you are running Shorewall 3.2.6 or later).</p><p>To determine the optimum value for this setting, we recommend
that you start by setting it significantly below your measured
download bandwidth (20% or so). While downloading, measure the
<span class="emphasis"><em>ping</em></span> response time from the firewall to the
upstream router as you gradually increase the setting.The optimal
setting is at the point beyond which the <span class="emphasis"><em>ping</em></span>
time increases sharply as you increase the setting.</p></li><li><p>OUT-BANDWIDTH - Specify the outgoing bandwidth of that
interface. This is the maximum speed your connection can handle. It
is also the speed you can refer as "full" if you define the tc
classes. Outgoing traffic above this rate will be dropped.</p></li><li><p>OPTIONS (Added in Shorewall-perl 4.1.4) — A comma-separated
list of options from the following list:</p><div class="variablelist"><dl><dt><span class="term">classify</span></dt><dd><p>If specified, classification of traffic into the various
classes is done by CLASSIFY entries in
<code class="filename">/etc/shorewall/tcrules</code> or by entries in
<code class="filename">/etc/shorewall/tcfilters</code>. No MARK value
will be associated with classes on this interface.</p></dd></dl></div></li><li><p>REDIRECTED INTERFACES (Added in Shorewall-perl 4.1.6) —
Entries are appropriate in this column only if the device in the
INTERFACE column names a <a class="link" href="#IFB" title="Intermediate Frame Block (IFB) Devices">Intermediate Frame
Block (IFB)</a>. It lists the physical interfaces that will have
their input shaped using classes defined on the IFB. Neither the IFB
nor any of the interfaces listed in this column may have an
IN-BANDWIDTH specified. You may specify zero (0) or a dash ("-:) in
the IN-BANDWIDTH column.</p><p>IFB devices automatically get the <span class="bold"><strong>classify</strong></span> option.</p></li></ul></div><div class="example"><a id="Example0"></a><p class="title"><b>Example 1. </b></p><div class="example-contents"><p>Suppose you are using PPP over Ethernet (DSL) and ppp0 is the
interface for this. The device has an outgoing bandwidth of 500kbit
and an incoming bandwidth of 6000kbit</p><pre class="programlisting">#INTERFACE IN-BANDWITH OUT-BANDWIDTH
ppp0 6000kbit 500kbit</pre></div></div><br class="example-break" /></div><div class="section" lang="en" xml:lang="en"><div class="titlepage"><div><div><h3 class="title"><a id="tcclasses"></a>/etc/shorewall/tcclasses</h3></div></div></div><p>This file allows you to define the actual classes that are used to
split the outgoing traffic.</p><div class="itemizedlist"><ul type="disc"><li><p>INTERFACE - Name of interface. Users of Shorewall-perl 4.1.6
or later may also specify the interface number. Must match the name
(or number) of an interface with an entry in
<code class="filename">/etc/shorewall/tcdevices</code>. If the interface has
the <span class="bold"><strong>classify</strong></span> option in
<code class="filename">/etc/shorewall/tcdevices</code>, then the interface
name or number must be followed by a colon and a <em class="firstterm">class
number</em>. Examples: eth0:1, 4:9. Class numbers must be
unique for a given interface.</p></li><li><p>MARK - The mark value which is an integer in the range 1-255.
You define these marks in the tcrules file, marking the traffic you
want to go into the queuing classes defined in here. You can use
the same marks for different Interfaces. You must specify "-' in
this column if the device specified in the INTERFACE column has the
<span class="bold"><strong>classify</strong></span> option in
<code class="filename">/etc/shorewall/tcdevices</code>.</p></li><li><p>RATE - The minimum bandwidth this class should get, when the
traffic load rises. Please note that first the classes which equal
or a lesser priority value are served even if there are others that
have a guaranteed bandwidth but a lower priority. <span class="bold"><strong>If the sum of the RATEs for all classes assigned to an
INTERFACE exceed that interfaces's OUT-BANDWIDTH, then the
OUT-BANDWIDTH limit will not be honored.</strong></span></p></li><li><p>CEIL - The maximum bandwidth this class is allowed to use when
the link is idle. Useful if you have traffic which can get full
speed when more important services (e.g. interactive like ssh) are
not used. You can use the value "full" in here for setting the
maximum bandwidth to the defined output bandwidth of that
interface.</p></li><li><p>PRIORITY - you have to define a priority for the class.
packets in a class with a higher priority (=lesser value) are
handled before less prioritized ones. You can just define the mark
value here also, if you are increasing the mark values with lesser
priority.</p></li><li><p>OPTIONS - A comma-separated list of options including the
following:</p><div class="itemizedlist"><ul type="circle"><li><p>default - this is the default class for that interface
where all traffic should go, that is not classified
otherwise.</p><div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Note</h3><p>defining default for exactly <span class="bold"><strong>one</strong></span> class per interface is
mandatory!</p></div></li><li><p>tos-<tosname> - this lets you define a filter for
the given <tosname> which lets you define a value of the
Type Of Service bits in the ip package which causes the package
to go in this class. Please note, that this filter overrides all
mark settings, so if you define a tos filter for a class all
traffic having that mark will go in it regardless of the mark on
the package. You can use the following for this option:
tos-minimize-delay (16) tos-maximize-throughput (8)
tos-maximize-reliability (4) tos-minimize-cost (2)
tos-normal-service (0)</p><div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Note</h3><p>Each of this options is only valid for <span class="bold"><strong>one</strong></span> class per interface.</p></div></li><li><p>tcp-ack - if defined causes an tc filter to be created
that puts all tcp ack packets on that interface that have an
size of <=64 Bytes to go in this class. This is useful for
speeding up downloads. Please note that the size of the ack
packets is limited to 64 bytes as some applications (p2p for
example) use to make every package an ack package which would
cause them all into here. We want only packets WITHOUT payload
to match, so the size limit. Bigger packets just take their
normal way into the classes.</p><div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Note</h3><p>This option is only valid for <span class="bold"><strong>class</strong></span> per interface.</p></div></li></ul></div></li></ul></div></div><div class="section" lang="en" xml:lang="en"><div class="titlepage"><div><div><h3 class="title"><a id="tcrules"></a>/etc/shorewall/tcrules</h3></div></div></div><div class="important" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Important</h3><p>Unlike rules in the <a class="ulink" href="manpages/shorewall-rules.html" target="_self">shorewall-rules</a>(5) file,
evaluation of rules in this file will continue after a match. So the
final mark for each packet will be the one assigned by the LAST tcrule
that matches.</p></div><p>The fwmark classifier provides a convenient way to classify
packets for traffic shaping. The “<span class="quote">/etc/shorewall/tcrules</span>”
file is used for specifying these marks in a tabular fashion. For an
in-depth look at the packet marking facility in Netfilter/Shorewall,
please see <a class="ulink" href="PacketMarking.html" target="_self">this article</a>.</p><p>Normally, packet marking occurs in the PREROUTING chain before any
address rewriting takes place. This makes it impossible to mark inbound
packets based on their destination address when SNAT or Masquerading are
being used. You can cause packet marking to occur in the FORWARD chain
by using the MARK_IN_FORWARD_CHAIN option in shorewall.conf.</p><p>Columns in the file are as follows:</p><div class="itemizedlist"><ul type="disc"><li><p>MARK or CLASSIFY - MARK specifies the mark value is to be
assigned in case of a match. This is an integer in the range 1-255.
This value may be optionally followed by “<span class="quote">:</span>” and either
“<span class="quote">F</span>”, “<span class="quote">P</span>” or "T" to designate that the
marking will occur in the FORWARD, PREROUTING or POSTROUTING chains
respectively. If this additional specification is omitted, the chain
used to mark packets will be determined as follows:</p><div class="itemizedlist"><ul type="circle"><li><p>If the SOURCE is
$FW[:<<span class="emphasis"><em>address</em></span>>], then the rule is
inserted in the OUTPUT chain.</p></li><li><p>Otherwise, the chain is determined by the setting of the
MARK_IN_FORWARD_CHAIN option in shorewall.conf.</p></li></ul></div><div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Note</h3><p>The "T" qualifier was added in Shorewall version 3.3.6 and
is not available in earlier versions.</p></div><p>Normally, the mark is applied to the packet. If you follow the
mark value with ":" and "C", then the mark is applied to the
connection. "C" can be combined with "F", "P" or "T" to designate
that the connection should be marked in a particular chain (e.g.,
"CF", "CP", "CT").</p><p>There are additional special values available:</p><div class="orderedlist"><ol type="a"><li><p><span class="bold"><strong>RESTORE</strong></span>[/<span class="emphasis"><em>mask</em></span>] --
restore the packet's mark from the connection's mark using the
supplied mask if any. Your kernel and iptables must include
CONNMARK support.</p><p>As above, may be followed by <span class="bold"><strong>:P</strong></span>, <span class="bold"><strong>:F</strong></span>
or <span class="bold"><strong>:T</strong></span>.</p></li><li><p><span class="bold"><strong>SAVE</strong></span>[/<span class="emphasis"><em>mask</em></span>] -- save
the packet's mark to the connection's mark using the supplied
mask if any. Your kernel and iptables must include CONNMARK
support.</p><p>As above, may be followed by <span class="bold"><strong>:P</strong></span>, <span class="bold"><strong>:F</strong></span>
or <span class="bold"><strong>:T</strong></span>.</p></li><li><p><span class="bold"><strong>CONTINUE</strong></span> Don't process
any more marking rules in the table.</p><p>As above, may be followed by <span class="bold"><strong>:P</strong></span>, <span class="bold"><strong>:F</strong></span>
or <span class="bold"><strong>:T</strong></span>.</p></li><li><p><span class="bold"><strong>COMMENT</strong></span> (Added in
Shorewall version 3.3.3) -- the rest of the line will be
attached as a comment to the Netfilter rule(s) generated by the
following entries. The comment will appear delimited by "/* ...
*/" in the output of <span class="command"><strong>shorewall show
mangle</strong></span></p><p>To stop the comment from being attached to further rules,
simply include COMMENT on a line by itself.</p></li></ol></div><p>To use CLASSIFY, your kernel and iptables must include
CLASSIFY target support. In that case, this column contains a
classification (classid) of the form <major>:<minor>
where <major> and <minor> are integers. Corresponds to
the 'class' specification in these traffic shaping modules:</p><table class="simplelist" border="0" summary="Simple list"><tr><td>atm</td></tr><tr><td>cbq</td></tr><tr><td>dsmark</td></tr><tr><td>pfifo_fast</td></tr><tr><td>htb</td></tr><tr><td>prio</td></tr></table><p>With Shorewall versions prior to 3.2.3, classify rules are
always placed in the POSTROUTING chain. Beginning with Shorewall
3.2.3, classification occurs in the POSTROUTING chain <span class="bold"><strong>except</strong></span> when the SOURCE contains
$FW[:<<span class="emphasis"><em>address</em></span>>] in which case, the
classify action takes place in the OUTPUT chain. When used with the
builtin traffic shaper, the <major> class is the interface
number and the <minor> class is either a) the MARK value of
the class preceded by the number "1" (MARK value 1 is <minor>
class 11, MARK value 22 is <minor> class 122, and so on) or b)
The class number (if the <span class="bold"><strong>classify</strong></span>
option was specified in for the interface
<code class="filename">/etc/shorewall/interfaces</code>)</p></li><li><p>SOURCE - Source of the packet. A comma-separated list of
interface names, IP addresses, MAC addresses and/or subnets for
packets being routed through a common path. List elements may also
consist of an interface name followed by ":" and an address (e.g.,
eth1:192.168.1.0/24). For example, all packets for connections
masqueraded to eth0 from other interfaces can be matched in a single
rule with several alternative SOURCE criteria. However, a connection
whose packets gets to eth0 in a different way, e.g., direct from the
firewall itself, needs a different rule.</p><p>Accordingly, use $FW in its own separate rule for packets
originating on the firewall. In such a rule, the MARK column may NOT
specify either ":P" or ":F" because marking for firewall-originated
packets always occurs in the OUTPUT chain.</p><p>MAC addresses must be prefixed with "~" and use "-" as a
separator.</p><p>Example: ~00-A0-C9-15-39-78</p></li><li><p>DEST - Destination of the packet. Comma separated list of IP
addresses and/or subnets. If your kernel and iptables include
iprange match support, IP address ranges are also allowed. List
elements may also consist of an interface name followed by ":" and
an address (e.g., eth1:192.168.1.0/24). If the MARK column
specifies a classification of the form <major>:<minor>
then this column may also contain an interface name.</p></li><li><p>PROTO - Protocol - Must be "tcp", "udp", "icmp", "ipp2p",
"ipp2p:udp", "ipp2p:all" a number, or "all". "ipp2p" requires ipp2p
match support in your kernel and iptables.</p></li><li><p>PORT(S) - Destination Ports. A comma-separated list of Port
names (from /etc/services), port numbers or port ranges; if the
protocol is "icmp", this column is interpreted as the destination
icmp-type(s).</p><p>If the protocol is ipp2p, this column is interpreted as an
ipp2p option without the leading "--" (example "bit" for
bit-torrent). If no PORT is given, "ipp2p" is assumed.</p><p>This column is ignored if PROTOCOL = all but must be entered
if any of the following field is supplied. In that case, it is
suggested that this field contain "-"</p></li><li><p>CLIENT PORT(S) - (Optional) Port(s) used by the client. If
omitted, any source port is acceptable. Specified as a
comma-separate list of port names, port numbers or port
ranges.</p></li><li><p>USER/GROUP (Added in Shorewall version 1.4.10) - (Optional)
This column may only be non-empty if the SOURCE is the firewall
itself. When this column is non-empty, the rule applies only if the
program generating the output is running under the effective user
and/or group. It may contain :</p><p>[!][<user name or number>]:[<group name or
number>][+<program name>]</p><p>The colon is optional when specifying only a user.</p><p>Examples:</p><pre class="programlisting">joe #program must be run by joe
:kids #program must be run by a member of the 'kids' group
!:kids #program must not be run by a member of the 'kids' group
+upnpd #program named upnpd (This feature was removed from Netfilter in kernel version 2.6.14).</pre></li><li><p>TEST - Defines a test on the existing packet or connection
mark. The rule will match only if the test returns true. Tests have
the format [!]<value>[/<mask>][:C]</p><p>Where:</p><table class="simplelist" border="0" summary="Simple list"><tr><td>! Inverts the test (not equal)</td></tr><tr><td><value> Value of the packet or connection
mark.</td></tr><tr><td><mask> A mask to be applied to the mark before
testing</td></tr><tr><td>:C Designates a connection mark. If omitted, the packet
mark's value is tested.</td></tr></table></li><li><p>LENGTH (Optional, added in Shorewall version 3.2.0) Packet
Length - This field, if present, allows you to match the length of a
packet against a specific value or range of values. A range is
specified in the form <min>:<max> where either
<min> or <max> (but not both) may be omitted. If
<min> is omitted, then 0 is assumed; if <max> is
omitted, than any packet that is <min> or longer will
match.</p><p>You must have iptables length support for this to work. If you
let it empty or place an "-" here, no length match will be
done.</p><p>Examples: 1024, 64:1500, :100</p></li><li><p>TOS (Optional, added in Shorewall version 3.2.0 Beta 6) Type
of Service. Either a standard name, or a numeric value to
match.</p><div class="blockquote"><blockquote class="blockquote"><table class="simplelist" border="0" summary="Simple list"><tr><td>Minimize-Delay (16)</td></tr><tr><td>Maximize-Throughput (8)</td></tr><tr><td>Maximize-Reliability (4)</td></tr><tr><td>Minimize-Cost (2)</td></tr><tr><td>Normal-Service (0)</td></tr></table></blockquote></div></li><li><p>HELPER (Optional, added in Shorewall version 4.2.0 Beta 2).
Names one of the Netfilter protocol helper modules such as
<span class="emphasis"><em>ftp</em></span>, <span class="emphasis"><em>sip</em></span>,
<span class="emphasis"><em>amanda</em></span>, etc.</p></li></ul></div><div class="example"><a id="Example1"></a><p class="title"><b>Example 2. </b></p><div class="example-contents"><p>All packets arriving on eth1 should be marked with 1. All
packets arriving on eth2 and eth3 should be marked with 2. All packets
originating on the firewall itself should be marked with 3.</p><pre class="programlisting">#MARK SOURCE DESTINATION PROTOCOL PORT(S)
1 eth1 0.0.0.0/0 all
2 eth2 0.0.0.0/0 all
2 eth3 0.0.0.0/0 all
3 $FW 0.0.0.0/0 all</pre></div></div><br class="example-break" /><div class="example"><a id="Example2"></a><p class="title"><b>Example 3. </b></p><div class="example-contents"><p>All GRE (protocol 47) packets not originating on the firewall
and destined for 155.186.235.151 should be marked with 12.</p><pre class="programlisting">#MARK SOURCE DESTINATION PROTOCOL PORT(S)
12 0.0.0.0/0 155.182.235.151 47</pre></div></div><br class="example-break" /><div class="example"><a id="Example3"></a><p class="title"><b>Example 4. </b></p><div class="example-contents"><p>All SSH request packets originating in 192.168.1.0/24 and
destined for 155.186.235.151 should be marked with 22.</p><pre class="programlisting">#MARK SOURCE DESTINATION PROTOCOL PORT(S)
22 192.168.1.0/24 155.182.235.151 tcp 22</pre></div></div><br class="example-break" /><div class="example"><a id="Example4"></a><p class="title"><b>Example 5. </b></p><div class="example-contents"><p>All SSH packets packets going out of the first device in in
/etc/shorewall/tcdevices should be assigned to the class with mark
value 10.</p><pre class="programlisting">#MARK SOURCE DESTINATION PROTOCOL PORT(S) CLIENT
# PORT(S)
1:110 0.0.0.0/0 0.0.0.0/0 tcp 22
1:110 0.0.0.0/0 0.0.0.0/0 tcp - 22</pre></div></div><br class="example-break" /><div class="example"><a id="Example5"></a><p class="title"><b>Example 6. </b></p><div class="example-contents"><p>Mark all ICMP echo traffic with packet mark 1. Mark all peer to
peer traffic with packet mark 4.</p><p>This is a little more complex than otherwise expected. Since the
ipp2p module is unable to determine all packets in a connection are
P2P packets, we mark the entire connection as P2P if any of the
packets are determined to match. We assume packet/connection mark 0 to
means unclassified.</p><pre class="programlisting">#MARK SOURCE DESTINATION PROTOCOL PORT(S) CLIENT USER/ TEST
# PORT(S) GROUP
1 0.0.0.0/0 0.0.0.0/0 icmp echo-request
1 0.0.0.0/0 0.0.0.0/0 icmp echo-reply
RESTORE 0.0.0.0/0 0.0.0.0/0 all - - - 0
CONTINUE 0.0.0.0/0 0.0.0.0/0 all - - - !0
4 0.0.0.0/0 0.0.0.0/0 ipp2p:all
SAVE 0.0.0.0/0 0.0.0.0/0 all - - - !0</pre><p>The last four rules can be translated as:</p><div class="blockquote"><blockquote class="blockquote"><p>"If a packet hasn't been classified (packet mark is 0), copy
the connection mark to the packet mark. If the packet mark is set,
we're done. If the packet is P2P, set the packet mark to 4. If the
packet mark has been set, save it to the connection mark."</p></blockquote></div></div></div><br class="example-break" /><div class="example"><a id="id300044"></a><p class="title"><b>Example 7. </b></p><div class="example-contents"><p>Mark all forwarded VOIP connections with connection mark 1 and
ensure that all VOIP packets also receive that mark (assumes that
nf_conntrack_sip is loaded and that Shorewall-perl 4.2.0 or later is
being used).</p><pre class="programlisting">#MARK SOURCE DESTINATION PROTOCOL PORT(S) CLIENT USER/ TEST CONNBYTES TOS HELPER
# PORT(S) GROUP
RESTORE 0.0.0.0/0 0.0.0.0/0 all - - - 0
CONTINUE 0.0.0.0/0 0.0.0.0/0 all - - - !0
1 0.0.0.0/0 0.0.0.0/0 all - - - - - - sip
SAVE 0.0.0.0/0 0.0.0.0/0 all - - - !0</pre></div></div><br class="example-break" /></div><div class="section" lang="en" xml:lang="en"><div class="titlepage"><div><div><h3 class="title"><a id="ppp"></a>ppp devices</h3></div></div></div><p>If you use ppp/pppoe/pppoa) to connect to your Internet provider
and you use traffic shaping you need to restart shorewall traffic
shaping. The reason for this is, that if the ppp connection gets
restarted (and it usually does this at least daily), all
“<span class="quote">tc</span>” filters/qdiscs related to that interface are
deleted.</p><p>The easiest way to achieve this, is just to restart shorewall once
the link is up. To achieve this add a small executable script
to“<span class="quote">/etc/ppp/ip-up.d</span>”.</p><pre class="programlisting">#! /bin/sh
/sbin/shorewall refresh</pre></div><div class="section" lang="en" xml:lang="en"><div class="titlepage"><div><div><h3 class="title"><a id="Real"></a>Real life examples</h3></div></div></div><div class="section" lang="en" xml:lang="en"><div class="titlepage"><div><div><h4 class="title"><a id="Wondershaper"></a>Configuration to replace Wondershaper</h4></div></div></div><p>You are able to fully replace the wondershaper script by using
the buitin traffic control.You can find example configuration files at
<a class="ulink" href="http://www1.shorewall.net/pub/shorewall/Samples/tc4shorewall/" target="_self">"http://www1.shorewall.net/pub/shorewall/Samples/tc4shorewall/</a>.
Please note that they are just examples and need to be adjusted to
work for you. In this example it is assumed that your interface for
your Internet connection is ppp0 (for DSL), if you use another
connection type, you have to change it. You also need to change the
settings in the tcdevices.wondershaper file to reflect your line
speed. The relevant lines of the config files follow here. Please note
that this is just a 1:1 replacement doing exactly what wondershaper
should do. You are free to change it...</p><div class="section" lang="en" xml:lang="en"><div class="titlepage"><div><div><h5 class="title"><a id="realtcd"></a>tcdevices file</h5></div></div></div><pre class="programlisting">#INTERFACE IN-BANDWITH OUT-BANDWIDTH
ppp0 5000kbit 500kbit</pre></div><div class="section" lang="en" xml:lang="en"><div class="titlepage"><div><div><h5 class="title"><a id="realtcc"></a>tcclasses file</h5></div></div></div><pre class="programlisting">#INTERFACE MARK RATE CEIL PRIORITY OPTIONS
ppp0 1 5*full/10 full 1 tcp-ack,tos-minimize-delay
ppp0 2 3*full/10 9*full/10 2 default
ppp0 3 2*full/10 8*full/10 2</pre></div><div class="section" lang="en" xml:lang="en"><div class="titlepage"><div><div><h5 class="title"><a id="realtcr"></a>tcrules file</h5></div></div></div><pre class="programlisting">#MARK SOURCE DEST PROTO PORT(S) CLIENT USER
# PORT(S)
1:F 0.0.0.0/0 0.0.0.0/0 icmp echo-request
1:F 0.0.0.0/0 0.0.0.0/0 icmp echo-reply
# mark traffic which should have a lower priority with a 3:
# mldonkey
3 0.0.0.0/0 0.0.0.0/0 udp - 4666</pre><p>Wondershaper allows you to define a set of hosts and/or ports
you want to classify as low priority. To achieve this , you have to
add these hosts to tcrules and set the mark to 3 (true if you use
the example configuration files).</p></div><div class="section" lang="en" xml:lang="en"><div class="titlepage"><div><div><h5 class="title"><a id="lowpro"></a>Setting hosts to low priority</h5></div></div></div><p>lets assume the following settings from your old wondershaper
script (don't assume these example values are really useful, they
are only used for demonstrating ;-):</p><pre class="programlisting">
# low priority OUTGOING traffic - you can leave this blank if you want
# low priority source netmasks
NOPRIOHOSTSRC="192.168.1.128/25 192.168.3.28"
# low priority destination netmasks
NOPRIOHOSTDST=60.0.0.0/24
# low priority source ports
NOPRIOPORTSRC="6662 6663"
# low priority destination ports
NOPRIOPORTDST="6662 6663" </pre><p>This would result in the following additional settings to the
tcrules file:</p><pre class="programlisting">3 192.168.1.128/25 0.0.0.0/0 all
3 192.168.3.28 0.0.0.0/0 all
3 0.0.0.0/0 60.0.0.0/24 all
3 0.0.0.0/0 0.0.0.0/0 udp 6662,6663
3 0.0.0.0/0 0.0.0.0/0 udp - 6662,6663
3 0.0.0.0/0 0.0.0.0/0 tcp 6662,6663
3 0.0.0.0/0 0.0.0.0/0 tcp - 6662,6663</pre></div></div><div class="section" lang="en" xml:lang="en"><div class="titlepage"><div><div><h4 class="title"><a id="simiple"></a>A simple setup</h4></div></div></div><p>This is a simple setup for people sharing an Internet connection
and using different computers for this. It just basically shapes
between 2 hosts which have the ip addresses 192.168.2.23 and
192.168.2.42</p><div class="section" lang="en" xml:lang="en"><div class="titlepage"><div><div><h5 class="title"><a id="simpletcd"></a>tcdevices file</h5></div></div></div><pre class="programlisting">#INTERFACE IN-BANDWITH OUT-BANDWIDTH
ppp0 6000kbit 700kbit</pre><p>We have 6mbit down and 700kbit upstream.</p></div><div class="section" lang="en" xml:lang="en"><div class="titlepage"><div><div><h5 class="title"><a id="simpletcc"></a>tcclasses file</h5></div></div></div><pre class="programlisting">#INTERFACE MARK RATE CEIL PRIORITY OPTIONS
ppp0 1 10kbit 50kbit 1 tcp-ack,tos-minimize-delay
ppp0 2 300kbit full 2
ppp0 3 300kbit full 2
ppp0 4 90kbit 200kbit 3 default</pre><p>We add a class for tcp ack packets with highest priority, so
that downloads are fast. The following 2 classes share most of the
bandwidth between the 2 hosts, if the connection is idle, they may
use full speed. As the hosts should be treated equally they have the
same priority. The last class is for the remaining traffic.</p></div><div class="section" lang="en" xml:lang="en"><div class="titlepage"><div><div><h5 class="title"><a id="simpletcr"></a>tcrules file</h5></div></div></div><pre class="programlisting">#MARK SOURCE DEST PROTO PORT(S) CLIENT USER
# PORT(S)
1:F 0.0.0.0/0 0.0.0.0/0 icmp echo-request
1:F 0.0.0.0/0 0.0.0.0/0 icmp echo-reply
2:F 192.168.2.23 0.0.0.0/0 all
3:F 192.168.2.42 0.0.0.0/0 all</pre><p>We mark icmp ping and replies so they will go into the fast
interactive class and set a mark for each host.</p></div></div></div></div><div class="section" lang="en" xml:lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a id="Xen"></a>A Warning to Xen Users</h2></div></div></div><p>If you are running traffic shaping in your dom0 and traffic shaping
doesn't seem to be limiting outgoing traffic properly, it may be due to
"checksum offloading" in your domU(s). Check the output of "shorewall show
tc". Here's an excerpt from the output of that command:</p><pre class="programlisting">class htb 1:130 parent 1:1 leaf 130: prio 3 quantum 1500 rate 76000bit <span class="bold"><strong>ceil 230000bit</strong></span> burst 1537b/8 mpu 0b overhead 0b cburst 1614b/8 mpu 0b overhead 0b level 0
Sent 559018700 bytes 75324 pkt (dropped 0, overlimits 0 requeues 0)
<span class="bold"><strong>rate 299288bit</strong></span> 3pps backlog 0b 0p requeues 0
lended: 53963 borrowed: 21361 <span class="bold"><strong>giants: 90174</strong></span>
tokens: -26688 ctokens: -14783</pre><p>There are two obvious problems in the above output:</p><div class="orderedlist"><ol type="1"><li><p>The rate (299288) is considerably larger than the ceiling
(230000).</p></li><li><p>There are a large number (90174) of giants reported.</p></li></ol></div><p>This problem will be corrected by disabling "checksum offloading" in
your domU(s) using the <span class="command"><strong>ethtool</strong></span> utility. See the <a class="ulink" href="XenMyWay-Routed.html" target="_self">one of the Xen articles</a> for
instructions.</p></div><div class="section" lang="en" xml:lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a id="Downloads"></a>Shaping Download Traffic</h2></div></div></div><p>As stated at the outset, traffic shaping works on traffic being sent
by the firewall. Download traffic from the Internet to local hosts is sent
by the firewall over a local interface. So it follows that if you want to
shape such traffic, you must configure shaping on the local
interface.</p><p>Shaping of download traffic is most straightforward when there are
only two interface. That way, traffic leaving the local interface falls
into only two broad categories:</p><div class="itemizedlist"><ul type="disc"><li><p>Traffic being forwarded from the Internet</p></li><li><p>Traffic that originated on the firewall itself</p></li></ul></div><p>In general, you will want to shape the forwarded traffic and leave
the local traffic unrestricted.</p><p>Extending the <a class="link" href="#simiple" title="A simple setup">simple example</a>
above:</p><p><code class="filename">/etc/shorewall/tcdevices</code>:</p><pre class="programlisting">#INTERFACE IN-BANDWITH OUT-BANDWIDTH
ppp0 6000kbit 700kbit
eth1 - 100mbit</pre><p>/etc/shorewall/tcclasses:</p><pre class="programlisting">#INTERFACE MARK RATE CEIL PRIORITY OPTIONS
ppp0 1 10kbit 50kbit 1 tcp-ack,tos-minimize-delay
ppp0 2 300kbit full 2
ppp0 3 300kbit full 2
ppp0 4 90kbit 200kbit 3 default
eth0 1 100kbit 500kbit 1 tcp-ack
eth0 2 3mbit 6mbit 2
eth0 3 3mbit 6mbit 3
eth0 4 94mbit full default #for local traffic</pre><p>/etc/shorewall/tcrules:</p><pre class="programlisting">#MARK SOURCE DEST PROTO PORT(S) CLIENT USER
# PORT(S)
1:F 0.0.0.0/0 0.0.0.0/0 icmp echo-request
1:F 0.0.0.0/0 0.0.0.0/0 icmp echo-reply
2:F 192.168.2.23 0.0.0.0/0 all
3:F 192.168.2.42 0.0.0.0/0 all
2:F ppp0 192.168.2.23 all
3:F ppp0 192.168.2.42 all</pre></div><div class="section" lang="en" xml:lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a id="IFB"></a>Intermediate Frame Block (IFB) Devices</h2></div></div></div><p>Beginning with Shorewall 4.1.6, Shorewall-perl includes support for
IFBs. The principles behind an IFB is fairly simple:</p><div class="itemizedlist"><ul type="disc"><li><p>It looks like a network interface although it is never given an
IPv4 configuration.</p></li><li><p>Because it is a network interface, queuing disciplines can be
associated with an IFB.</p></li></ul></div><p>The magic of an IFB comes in the fact that a filter may be defined
on a real network interface such that each packet that arrives on that
interface is queued for the IFB! In that way, the IFB provides a means for
shaping input traffic.</p><p>To use an IFB, you must have IFB support in your kernel
(configuration option CONFIG_IFB). Assuming that you have a modular
kernel, the name of the IFB module is 'ifb' and may be loaded using the
command <span class="command"><strong>modprobe ifb</strong></span> (if you have modprobe installed)
or <span class="command"><strong>insmod /path/to/module/ifb</strong></span>.</p><p>By default, two IFB devices (ifb0 and ifb1) are created. You can
control that using the numifbs option (e.g., <span class="command"><strong>modprobe ifb
numifbs=1</strong></span>).</p><p>To create a single IFB when Shorewall starts, place the following
two commands in <code class="filename">/etc/shorewall/init</code>:</p><pre class="programlisting"><span class="command"><strong>modprobe ifb numifbs=1
ip link set ifb0 up</strong></span></pre><p>Entries in <code class="filename">/etc/shorewall/tcrules</code> have no
effect on shaping traffic through an IFB. To allow classification of such
traffic, the /etc/shorewall/tcfilters file has been added. Entries in that
file create <a class="ulink" href="http://b42.cz/notes/u32_classifier/" target="_self">u32
classification rules</a>.</p><div class="section" lang="en" xml:lang="en"><div class="titlepage"><div><div><h3 class="title"><a id="tcfilters"></a>/etc/shorewall/tcfilters</h3></div></div></div><p>While this file was created to allow shaping of traffic through an
IFB, the file may be used for general traffic classification as well.
The file is similar to <a class="ulink" href="shorewall-tcrules.html" target="_self">shorewall-tcrules</a>(5) with the
following key exceptions:</p><div class="itemizedlist"><ul type="disc"><li><p>The first match determines the classification, whereas in the
tcrules file, the last match determines the classification.</p></li><li><p>ipsets are not supported</p></li><li><p>DNS Names are not supported</p></li><li><p>filters are applied to packets as they <span class="emphasis"><em>appear on the
wire</em></span>. So incoming packets will not have DNAT applied yet
(the destination IP address will be the external address) and
outgoing packets will have had SNAT applied.</p></li></ul></div><p>The last point warrants elaboration. When looking at traffic being
shaped by an IFB, there are two cases to consider:</p><div class="orderedlist"><ol type="1"><li><p>Requests — packets being sent from remote clients to local
servers. These packets may undergo subsequent DNAT, either as a
result of entries in <code class="filename">/etc/shorewall/nat</code> or as a
result of DNAT or REDIRECT rules.</p><p>Example: <code class="filename">/etc/shorewall/rules</code>:</p><pre class="programlisting">#ACTION SOURCE DEST PROTO DEST SOURCE ORIGINAL
# PORT(S) PORT(S) DEST
DNAT net dmz:192.168.4.5 tcp 80 - 206.124.146.177</pre><p>Requests redirected by this rule will have destination IP
address 206.124.146.177 and destination port 80.</p></li><li><p>Responses — packets being sent from remote servers to local
clients. These packets may undergo subsequent DNAT as a result of
entries in <code class="filename">/etc/shorewall/nat</code> or in
<code class="filename">/etc/shorewall/masq</code>. The packet's destination
IP address will be the external address specified in the
entry.</p><p>Example:
<code class="filename">/etc/shorewall/masq</code>:</p><pre class="programlisting">#INTERFACE SOURCE ADDRESS
eth0 192.168.1.0/24 206.124.146.179</pre><p>HTTP response packets corresponding to requests that fall
under that rule will have destination IP address 206.124.146.179 and
<span class="bold"><strong>source</strong></span> port 80.</p></li></ol></div><p>Columns in the file are as follow. As in all Shorewall
configuration files, a hyphen ("-") may be used to indicate that no
value is supplied in the column.</p><div class="variablelist"><dl><dt><span class="term">CLASS</span></dt><dd><p>The interface name or number followed by a colon (":") and
the class number.</p></dd><dt><span class="term">SOURCE</span></dt><dd><p>SOURCE IP address (host or network). DNS names are not
allowed.</p></dd><dt><span class="term">DEST</span></dt><dd><p>DESTINATION IP address (host or network). DNS names are not
allowed.</p></dd><dt><span class="term">PROTO</span></dt><dd><p>Protocol name or number.</p></dd><dt><span class="term">DEST PORT(S)</span></dt><dd><p>Comma-separated list of destination port names or numbers.
May only be specified if the protocol is TCP, UDP, SCTP or ICMP.
Port ranges are supported except for ICMP.</p></dd><dt><span class="term">SOURCE PORT</span></dt><dd><p>Comma-separated list of source port names or numbers. May
only be specified if the protocol is TCP, UDP or SCTP. Port ranges
are supported.</p></dd></dl></div><p>Example:</p><p>I've used this configuration on my own firewall. The IFB portion
is more for test purposes rather than to serve any well-reasoned QOS
strategy.</p><p><code class="filename">/etc/shorewall/init</code>:</p><pre class="programlisting">qt modprobe ifb numifbs=1
qt ip link set dev ifb0 up</pre><p><code class="filename">/etc/shorewall/tcdevices</code>:</p><pre class="programlisting">#INTERFACE IN-BANDWITH OUT-BANDWIDTH OPTIONS REDIRECTED
# INTERFACES
1:eth0 - 384kbit classify
2:ifb0 - 1300kbit - eth0</pre><p>
<code class="filename">/etc/shorewall/tcclasses</code>:</p><pre class="programlisting">#INTERFACE MARK RATE CEIL PRIORITY OPTIONS
1:110 - 5*full/10 full 1 tcp-ack,tos-minimize-delay
1:120 - 2*full/10 6*full/10 2 default
1:130 - 2*full/10 6*full/10 3
2:110 - 5*full/10 full 1 tcp-ack,tos-minimize-delay
2:120 - 2*full/10 6*full/10 2 default
2:130 - 2*full/10 6*full/10 3</pre><p><code class="filename">/etc/shorewall/tcfilters</code>:</p><pre class="programlisting">#INTERFACE: SOURCE DEST PROTO DEST SOURCE
#CLASS PORT(S) PORT(S)
#
# OUTGOING TRAFFIC
#
1:130 206.124.146.178 - tcp - 49441,49442 #BITTORRENT on wookie
1:110 206.124.146.178 #wookie
1:110 206.124.146.179 #SNAT of internal systems
1:110 206.124.146.180 #Work Laptop
1:110 - - icmp echo-request,echo-reply
1:110 - - icmp echo-reply
1:130 206.124.146.177 - tcp - 873,25 #Bulk Traffic
#
# INCOMING TRAFFIC
#
2:110 - 206.124.146.178 #Wookie
2:110 - 206.124.146.179 #SNAT Responses
2:110 - 206.124.146.180 #Work Laptop
2:130 - 206.124.146.177 tcp 25 #Incoming Email.</pre><p>You can examine the installed filters with the <span class="command"><strong>shorewall
show filters</strong></span> command. What follows shows the output for
<code class="filename">eth0</code> with the filters shown
above. <span class="bold"><strong>Bold font</strong></span> are comments
explaining the rules.</p><pre class="programlisting">gateway:~ # shorewall-lite show filters
Shorewall Lite 4.1.6 Classifiers at gateway - Fri Mar 21 08:06:47 PDT 2008
Device eth1:
Device eth2:
Device eth0:
filter parent 1: protocol ip pref 10 u32
filter parent 1: protocol ip pref 10 u32 <span class="bold"><strong>fh 3:</strong></span> ht divisor 1 <span class="bold"><strong> <========= Start of table 3. parses TCP header</strong></span>
filter parent 1: protocol ip pref 10 u32 <span class="bold"><strong>fh 3</strong></span>::800 order 2048 key ht 3 bkt 0 <span class="bold"><strong>flowid 1:130</strong></span> (rule hit 102 success 0)
match 03690000/ffff0000 at nexthdr+0 (success 0 ) <span class="bold"><strong> <========= SOURCE PORT 873 goes to class 1:130</strong></span>
filter parent 1: protocol ip pref 10 u32 <span class="bold"><strong>fh 2:</strong></span> ht divisor 1 <span class="bold"><strong> <========= Start of table 2. parses ICMP header</strong></span>
filter parent 1: protocol ip pref 10 u32 <span class="bold"><strong>fh 2</strong></span>::800 order 2048 key ht 2 bkt 0 <span class="bold"><strong>flowid 1:110</strong></span> (rule hit 0 success 0)
match 08000000/ff000000 at nexthdr+0 (success 0 ) <span class="bold"><strong> <========= ICMP Type 8 goes to class 1:110</strong></span>
filter parent 1: protocol ip pref 10 u32 <span class="bold"><strong>fh 2</strong></span>::801 order 2049 key ht 2 bkt 0 <span class="bold"><strong>flowid 1:110</strong></span> (rule hit 0 success 0)
match 00000000/ff000000 at nexthdr+0 (success 0 ) <span class="bold"><strong> <========= ICMP Type 0 goes to class 1:110</strong></span>
filter parent 1: protocol ip pref 10 u32 <span class="bold"><strong>fh 1:</strong></span> ht divisor 1 <span class="bold"><strong> <========= Start of table 1. parses TCP header</strong></span>
filter parent 1: protocol ip pref 10 u32 <span class="bold"><strong>fh 1:</strong></span>:800 order 2048 key ht 1 bkt 0 <span class="bold"><strong>flowid 1:130</strong></span> (rule hit 0 success 0)
match c1210000/ffff0000 at nexthdr+0 (success 0 ) <span class="bold"><strong> <========= SOURCE PORT 49441 goes to class 1:130</strong></span>
filter parent 1: protocol ip pref 10 u32 <span class="bold"><strong>fh 1</strong></span>::801 order 2049 key ht 1 bkt 0 <span class="bold"><strong>flowid 1:130</strong></span> (rule hit 0 success 0)
match c1220000/ffff0000 at nexthdr+0 (success 0 ) <span class="bold"><strong> <========= SOURCE PORT 49442 goes to class 1:130</strong></span>
filter parent 1: protocol ip pref 10 u32 <span class="bold"><strong>fh 800:</strong></span> ht divisor 1 <span class="bold"><strong><========= Start of Table 800. Packets start here!</strong></span>
<span class="bold"><strong>=============== The following 2 rules are generated by the class definition in /etc/shorewall/classes ==================</strong></span>
filter parent 1: protocol ip pref 10 u32 <span class="bold"><strong>fh 800:</strong></span>:800 order 2048 key ht 800 bkt 0 <span class="bold"><strong>flowid 1:110</strong></span> (rule hit 2204 success 138)
match 00060000/00ff0000 at 8 (success 396 ) <span class="bold"><strong><========= TCP </strong></span>
match 05000000/0f00ffc0 at 0 (success 250 ) <span class="bold"><strong><========= Header length 20 and Packet Length < 64</strong></span>
match 00100000/00ff0000 at 32 (success 138 ) <span class="bold"><strong><========= ACK</strong></span>
filter parent 1: protocol ip pref 10 u32 <span class="bold"><strong>fh 800:</strong></span>:801 order 2049 key ht 800 bkt 0 <span class="bold"><strong>flowid 1:110</strong></span> (rule hit 2066 success 0)
match 00100000/00100000 at 0 (success 0 ) <span class="bold"><strong><========= Minimize-delay</strong></span><span class="bold"><strong> goes to class 1:110</strong></span>
<span class="bold"><strong> =============== Jump to Table 1 if the matches are met ==================</strong></span>
filter parent 1: protocol ip pref 10 u32 <span class="bold"><strong>fh 800:</strong></span>:802 order 2050 key ht 800 bkt 0 <span class="bold"><strong>link 1:</strong></span> (rule hit 2066 success 0)
match ce7c92b2/ffffffff at 12 (success 1039 ) <span class="bold"><strong><========= SOURCE 206.124.146.178 </strong></span>
match 00060000/00ff0000 at 8 (success 0 ) <span class="bold"><strong><========= PROTO TCP</strong></span>
offset 0f00>>6 at 0 eat
filter parent 1: protocol ip pref 10 u32 <span class="bold"><strong>fh 800:</strong></span>:803 order 2051 key ht 800 bkt 0 <span class="bold"><strong>flowid 1:110</strong></span> (rule hit 2066 success 1039)
match ce7c92b2/ffffffff at 12 (success 1039 ) <span class="bold"><strong><========= SOURCE 206.124.146.178 goes to class 1:110</strong></span>
filter parent 1: protocol ip pref 10 u32 <span class="bold"><strong>fh 800:</strong></span>:804 order 2052 key ht 800 bkt 0 <span class="bold"><strong>flowid 1:110</strong></span> (rule hit 1027 success 132)
match ce7c92b3/ffffffff at 12 (success 132 ) <span class="bold"><strong> <========= SOURCE 206.124.146.179 goes to class 1:110</strong></span>
filter parent 1: protocol ip pref 10 u32 <span class="bold"><strong>fh 800:</strong></span>:805 order 2053 key ht 800 bkt 0 <span class="bold"><strong>flowid 1:110</strong></span> (rule hit 895 success 603)
match ce7c92b4/ffffffff at 12 (success 603 ) <span class="bold"><strong><========= SOURCE 206.124.146.180 goes to class 1:110</strong></span>
<span class="bold"><strong> =============== Jump to Table 2 if the matches are met ==================</strong></span>
filter parent 1: protocol ip pref 10 u32 <span class="bold"><strong>fh 800:</strong></span>:806 order 2054 key ht 800 bkt 0 <span class="bold"><strong>link 2:</strong></span> (rule hit 292 success 0)
match 00010000/00ff0000 at 8 (success 0 ) <span class="bold"><strong><========= PROTO ICMP</strong></span>
offset 0f00>>6 at 0 eat
<span class="bold"><strong> =============== Jump to Table 3 if the matches are met ==================</strong></span>
filter parent 1: protocol ip pref 10 u32 <span class="bold"><strong>fh 800:</strong></span>:807 order 2055 key ht 800 bkt 0 <span class="bold"><strong>link 3:</strong></span> (rule hit 292 success 0)
match ce7c92b1/ffffffff at 12 (success 265 ) <span class="bold"><strong><========= SOURCE 206.124.146.177</strong></span>
match 00060000/00ff0000 at 8 (success 102 ) <span class="bold"><strong><========= PROTO TCP</strong></span>
offset 0f00>>6 at 0 eat </pre></div></div><div class="section" lang="en" xml:lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a id="show"></a>Understanding the output of 'shorewall show tc'</h2></div></div></div><p>The <span class="command"><strong>shorewall show tc</strong></span> (<span class="command"><strong>shorewall-lite
show tc</strong></span>) command displays information about the current state of
traffic shaping. For each device, it executes the following
commands:</p><pre class="programlisting"> echo Device $device:
tc -s -d qdisc show dev $device
echo
tc -s -d class show dev $device
echo </pre><p>So, the traffic-shaping output is generated entirely by the
<span class="command"><strong>tc</strong></span> utility.</p><p>Here's the output of for eth0. The configuration is the one shown in
the preceding section (the output was obtained almost 24 hours later than
the <span class="command"><strong>shorewall show filters</strong></span> output shown above).</p><pre class="programlisting">Device eth0:
<span class="bold"><strong> ============== The primary queuing discipline is HTB (Hierarchical Token Bucket) ==================== </strong></span>
qdisc htb 1: r2q 10 default 120 direct_packets_stat 9 ver 3.17
Sent 2133336743 bytes 4484781 pkt (dropped 198, overlimits 4911403 requeues 21) <span class="bold"><strong><=========== Note the overlimits and dropped counts</strong></span>
rate 0bit 0pps backlog 0b 8p requeues 21
<span class="bold"><strong>============== The ingress filter. If you specify IN-BANDWIDTH, you can see the 'dropped' count here. =========</strong></span>
<span class="bold"><strong>In this case, the packets are being sent to the IFB for shaping</strong></span>
qdisc ingress ffff: ----------------
Sent 4069015112 bytes 4997252 pkt (dropped 0, overlimits 0 requeues 0)
rate 0bit 0pps backlog 0b 0p requeues 0
<span class="bold"><strong>============ Each of the leaf HTB classes has an SFQ qdisc to ensure that each flow gets its turn ============</strong></span>
qdisc sfq 110: parent 1:110 limit 128p quantum 1514b flows 128/1024 perturb 10sec
Sent 613372519 bytes 2870225 pkt (dropped 0, overlimits 0 requeues 6)
rate 0bit 0pps backlog 0b 0p requeues 6
qdisc sfq 120: parent 1:120 limit 128p quantum 1514b flows 128/1024 perturb 10sec
Sent 18434920 bytes 60961 pkt (dropped 0, overlimits 0 requeues 0)
rate 0bit 0pps backlog 0b 0p requeues 0
qdisc sfq 130: parent 1:130 limit 128p quantum 1514b flows 128/1024 perturb 10sec
Sent 1501528722 bytes 1553586 pkt (dropped 198, overlimits 0 requeues 15)
rate 0bit 0pps backlog 11706b 8p requeues 15
<span class="bold"><strong>============= Class 1:110 -- the high-priority class ===========
Note the rate and ceiling calculated from 'full'</strong></span>
class htb <span class="bold"><strong>1:110</strong></span> parent 1:1 leaf 110: prio 1 quantum 4800 <span class="bold"><strong>rate 192000bit ceil 384000bit</strong></span> burst 1695b/8 mpu 0b overhead 0b cburst 1791b/8 mpu 0b overhead 0b level 0
Sent 613372519 bytes 2870225 pkt (dropped 0, overlimits 0 requeues 0)
<span class="bold"><strong>rate 195672bit 28pps backlog 0b 0p</strong></span> requeues 0 <span class="bold"><strong><=========== Note the current rate of traffic. There is no queuing (no packet backlog)</strong></span>
lended: 2758458 borrowed: 111773 giants:
tokens: 46263 ctokens: 35157
<span class="bold"><strong>============== The root class ============</strong></span>
class htb <span class="bold"><strong>1:1 root</strong></span> rate 384000bit ceil 384000bit burst 1791b/8 mpu 0b overhead 0b cburst 1791b/8 mpu 0b overhead 0b level 7
<span class="bold"><strong>Sent 2133276316 bytes 4484785 pkt</strong></span> (dropped 0, overlimits 0 requeues 0) <span class="bold"><strong><==== Total output traffic since last 'restart'</strong></span>
rate 363240bit 45pps backlog 0b 0p requeues 0
lended: 1081936 borrowed: 0 giants: 0
tokens: -52226 ctokens: -52226
<span class="bold"><strong>============= Bulk Class (outgoing rsync, email and bittorrent) ============</strong></span>
class htb 1:130 parent 1:1 leaf 130: prio 3 quantum 1900 rate 76000bit ceil 230000bit burst 1637b/8 mpu 0b overhead 0b cburst 1714b/8 mpu 0b overhead 0b level 0
Sent 1501528722 bytes 1553586 pkt (dropped 198, overlimits 0 requeues 0)
<span class="bold"><strong>rate 162528bit 14pps backlog 0b 8p</strong></span> requeues 0 <span class="bold"><strong><======== Queuing is occurring (8 packet backlog). The rate is still below the ceiling.</strong></span>
lended: 587134 borrowed: 966459 giants: 0 <span class="bold"><strong>During peak activity, the rate tops out at around 231000 (just above ceiling).</strong></span>
tokens: -30919 ctokens: -97657
<span class="bold"><strong>================= Default class (mostly serving web pages) ===============</strong></span>
class htb 1:120 parent 1:1 leaf 120: prio 2 quantum 1900 rate 76000bit ceil 230000bit burst 1637b/8 mpu 0b overhead 0b cburst 1714b/8 mpu 0b overhead 0b level 0
Sent 18434920 bytes 60961 pkt (dropped 0, overlimits 0 requeues 0)
rate 2240bit 2pps backlog 0b 0p requeues 0
lended: 57257 borrowed: 3704 giants: 0
tokens: 156045 ctokens: 54178
</pre></div><div class="section" lang="en" xml:lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a id="External"></a>Using your own tc script</h2></div></div></div><div class="section" lang="en" xml:lang="en"><div class="titlepage"><div><div><h3 class="title"><a id="owntcstart"></a>Replacing builtin tcstart file</h3></div></div></div><p>If you prefer your own tcstart file, just install it in
/etc/shorewall/tcstart.</p><p>In your tcstart script, when you want to run the “<span class="quote">tc</span>”
utility, use the run_tc function supplied by Shorewall if you want tc
errors to stop the firewall.</p><div class="orderedlist"><ol type="1"><li><p>Set TC_ENABLED=Yes and CLEAR_TC=Yes</p></li><li><p>Supply an /etc/shorewall/tcstart script to configure your
traffic shaping rules.</p></li><li><p>Optionally supply an /etc/shorewall/tcclear script to stop
traffic shaping. That is usually unnecessary.</p></li><li><p>If your tcstart script uses the “<span class="quote">fwmark</span>”
classifier, you can mark packets using entries in
/etc/shorewall/tcrules.</p></li></ol></div></div><div class="section" lang="en" xml:lang="en"><div class="titlepage"><div><div><h3 class="title"><a id="Start"></a>Traffic control outside Shorewall</h3></div></div></div><p>To start traffic shaping when you bring up your network
interfaces, you will have to arrange for your traffic shaping
configuration script to be run at that time. How you do that is
distribution dependent and will not be covered here. You then
should:</p><div class="orderedlist"><ol type="1"><li><p>Set TC_ENABLED=No and CLEAR_TC=No</p></li><li><p>If your script uses the “<span class="quote">fwmark</span>” classifier, you
can mark packets using entries in /etc/shorewall/tcrules.</p></li></ol></div></div></div><div class="section" lang="en" xml:lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a id="Testing"></a>Testing Tools</h2></div></div></div><p>At least one Shorewall user has found this tool helpful: <a class="ulink" href="http://e2epi.internet2.edu/network-performance-toolkit.html" target="_self">http://e2epi.internet2.edu/network-performance-toolkit.html</a></p></div></div></body></html>
