<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /><title>Shorewall Troubleshooting Guide</title><link rel="stylesheet" href="html.css" type="text/css" /><meta name="generator" content="DocBook XSL Stylesheets V1.73.2" /></head><body><div class="article" lang="en" xml:lang="en"><div class="titlepage"><div><div><h2 class="title"><a id="usefull_links"></a>Shorewall Troubleshooting Guide</h2></div><div><div class="author"><h3 class="author"><span class="firstname">Tom</span> <span class="surname">Eastep</span></h3></div></div><div><p class="copyright">Copyright © 2001-2007 Thomas M. Eastep</p></div><div><div class="legalnotice"><a id="id289807"></a><p>Permission is granted to copy, distribute and/or modify this
document under the terms of the GNU Free Documentation License, Version
1.2 or any later version published by the Free Software Foundation; with
no Invariant Sections, with no Front-Cover, and with no Back-Cover
Texts. A copy of the license is included in the section entitled
“<span class="quote"><a class="ulink" href="GnuCopyright.htm" target="_self">GNU Free Documentation
License</a></span>”.</p></div></div><div><p class="pubdate">2008/12/15</p></div></div><hr /></div><div class="toc"><p><b>Table of Contents</b></p><dl><dt><span class="section"><a href="#Start">“<span class="quote">shorewall start</span>” and “<span class="quote">shorewall restart</span>”
Errors</a></span></dt><dd><dl><dt><span class="section"><a href="#Start-shell">Shorewall-shell</a></span></dt><dt><span class="section"><a href="#Start-perl">Shorewall-perl</a></span></dt></dl></dd><dt><span class="section"><a href="#Network">Your Network Environment</a></span></dt><dt><span class="section"><a href="#NewDevice">New Device Doesn't Work?</a></span></dt><dt><span class="section"><a href="#Connections">Connection Problems</a></span></dt><dt><span class="section"><a href="#Ping">Ping Problems</a></span></dt><dt><span class="section"><a href="#Other">Some Things to Keep in Mind</a></span></dt><dt><span class="section"><a href="#More">Other Gotchas</a></span></dt><dt><span class="section"><a href="#Support">Still Having Problems?</a></span></dt></dl></div><div class="section" lang="en" xml:lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a id="Start"></a>“<span class="quote">shorewall start</span>” and “<span class="quote">shorewall restart</span>”
Errors</h2></div></div></div><div class="section" lang="en" xml:lang="en"><div class="titlepage"><div><div><h3 class="title"><a id="Start-shell"></a>Shorewall-shell</h3></div></div></div><p>If you use the Shorewall-shell compiler and you receive an error
message when starting or restarting the firewall and you can't determine
the cause. First, if your VERBOSITY setting in shorewall.conf is less
than 2, then try running with a higher verbosity level by using the "-v"
option:</p><div class="blockquote"><blockquote class="blockquote"><pre class="programlisting"><span class="command"><strong>shorewall -vv [re]start</strong></span></pre></blockquote></div><p>That will give you additional progress messages that may make it
clear which entry in which file is generating the error.</p><p>If that didn't help, then do the following:</p><div class="itemizedlist"><ul type="disc"><li><p>Make a note of the error message that you see.</p></li><li><p><span class="command"><strong>shorewall debug start 2>
/tmp/trace</strong></span></p></li><li><p>Look at the <code class="filename">/tmp/trace</code> file and see if
that helps you determine what the problem is. Be sure you find the
place in the log where the error message you saw is generated -- If
you are using Shorewall 1.4.0 or later, you should find the message
near the end of the log.</p></li><li><p>If you still can't determine what's wrong then see the <a class="ulink" href="support.htm" target="_self">support page</a>.</p></li></ul></div><div class="example"><a id="Example1"></a><p class="title"><b>Example 1. Startup Error</b></p><div class="example-contents"><p>During startup, a user sees the following:</p><pre class="programlisting">Adding Common Rules
iptables: No chain/target/match by that name
Terminated</pre><p>A search through the trace for “<span class="quote">No chain/target/match by
that name</span>” turned up the following:</p><pre class="programlisting">+ echo 'Adding Common Rules'
+ add_common_rules
+ run_iptables -A reject -p tcp -j REJECT --reject-with tcp-reset
++ echo -A reject -p tcp -j REJECT --reject-with tcp-reset
++ sed 's/!/! /g'
+ iptables -A reject -p tcp -j REJECT --reject-with tcp-reset
iptables: No chain/target/match by that name
</pre><p>The command that failed was: “<span class="quote"><span class="command"><strong>iptables -A reject
-p tcp -j REJECT --reject-with tcp-reset</strong></span></span>”. In this
case, the user had compiled his own kernel and had forgotten to
include REJECT target support (see <a class="ulink" href="kernel.htm" target="_self">kernel.htm</a>)</p></div></div><br class="example-break" /></div><div class="section" lang="en" xml:lang="en"><div class="titlepage"><div><div><h3 class="title"><a id="Start-perl"></a>Shorewall-perl</h3></div></div></div><p>If the error is detected by the Shorewall-perl compiler, it should
be fairly obvious where the problem was found. Each error message
includes the configuration file name and line number where the error was
detected and often gives the particular item in error. The item is
either enclosed in parentheses or is at the end following a colon
(":").</p><p>Example:</p><pre class="programlisting">gateway:~/test # shorewall restart .
Compiling...
ERROR: Invalid ICMP Type (0/400) : /root/test/rules (line 19)
gateway:~/test # </pre><p>In this case, line 19 in the rules file
specified an invalid ICMP Type (0/400).</p><p>Additional information about the error can be obtained using the
'debug' keyword:</p><pre class="programlisting">gateway:~/test # shorewall debug restart .
Compiling...
ERROR: Invalid ICMP Type (0/400) : /root/test/rules (line 19) at /usr/share/shorewall-perl/Shorewall/Config.pm line 338
Shorewall::Config::fatal_error('Invalid ICMP Type (0/400)') called at /usr/share/shorewall-perl/Shorewall/Chains.pm line 885
Shorewall::Chains::validate_icmp('0/400') called at /usr/share/shorewall-perl/Shorewall/Chains.pm line 949
Shorewall::Chains::do_proto('icmp', '0/400', '-') called at /usr/share/shorewall-perl/Shorewall/Rules.pm line 1055
Shorewall::Rules::process_rule1('ACCEPT', 'loc', 'net', 'icmp', '0/400', '-', '-', '-', '-', ...) called at /usr/share/shorewall-perl/Shorewall/Rules.pm line 1290
Shorewall::Rules::process_rule('ACCEPT', 'loc', 'net', 'icmp', '0/400', '-', '-', '-', '-', ...) called at /usr/share/shorewall-perl/Shorewall/Rules.pm line 1336
Shorewall::Rules::process_rules() called at /usr/share/shorewall-perl/Shorewall/Compiler.pm line 799
Shorewall::Compiler::compiler('/var/lib/shorewall/.restart', '/root/test', 0, 4) called at /usr/share/shorewall-perl/compiler.pl line 86
gateway:~/test # </pre><p>This information is useful to Shorewall
support if you need to <a class="ulink" href="support.html" target="_self">file a problem
report</a>.</p><p>The end of the compile phase is signaled by a message such as the
following:</p><pre class="programlisting">Shorewall configuration compiled to /var/lib/shorewall/.restart</pre><p>Errors
occurring past that point are said to occur at
<em class="firstterm">run-time</em> because they occur during the running of
the compiled firewall script (/var/lib/shorewall/.restart in the case of
the above message).</p><p>One common run-time failure is that the iptables-restore program
encounters an error. This will produce an error such as the
following:</p><pre class="programlisting">...
Restarting Shorewall....
iptables-restore v1.3.6: No chain/target/match by that name
Error occurred at line: 83
Try `iptables-restore -h' or 'iptables-restore --help' for more information.
ERROR: iptables-restore Failed. Input is in /var/lib/shorewall/.iptables-restore-input
Restoring Shorewall...
Shorewall restored from /var/lib/shorewall/restore
Terminated
gateway:~/test # </pre><p>A look at /var/lib/shorewall/restore at line
83 might show something like the following:</p><pre class="programlisting">-A reject -p tcp -j REJECT --reject-with tcp-reset</pre><p>In
this case, the user had compiled his own kernel and had forgotten to
include REJECT target support (see <a class="ulink" href="kernel.htm" target="_self">kernel.htm</a>).</p><p>f you are running Shorewall-perl 4.0.5 or later, you may also
include the word <span class="bold"><strong>debug</strong></span> as the first
argument to the <code class="filename">/sbin/shorewall</code> and
<code class="filename">/sbin/shorewall-lite</code> commands.</p><pre class="programlisting"><span class="command"><strong>shorewall debug restart</strong></span></pre><p>In
most cases, <span class="bold"><strong>debug</strong></span> is a synonym for
<span class="bold"><strong>trace</strong></span>. The exceptions are:</p><div class="itemizedlist"><ul type="disc"><li><p><span class="bold"><strong>debug</strong></span> is ignored by the
Shorewall-perl compiler.</p></li><li><p><span class="bold"><strong>debug</strong></span> causes altered behavior
of scripts generated by the Shorewall-perl compiler. These scripts
normally use<span class="command"><strong> iptables-restore</strong></span> to install the
Netfilter ruleset but with <span class="bold"><strong>debug</strong></span>,
the commands normally passed to <span class="command"><strong>iptables-restore</strong></span>
in its input file are passed individually to
<span class="command"><strong>iptables</strong></span>. This is a diagnostic aid which allows
identifying the individual command that is causing
<span class="command"><strong>iptables-restore</strong></span> to fail; it should be used when
iptables-restore fails when executing a <span class="command"><strong>COMMIT</strong></span>
command.</p></li></ul></div><div class="warning" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Warning</h3><p> The <span class="bold"><strong>debug</strong></span> feature is strictly
for problem analysis. When <span class="bold"><strong>debug</strong></span> is
used:</p><div class="orderedlist"><ol type="1"><li><p>The firewall is made 'wide open' before the rules are
applied.</p></li><li><p>The <code class="filename">routestopped</code> file is not
consulted.</p></li><li><p>The rules are applied in the canonical
<span class="command"><strong>iptables-restore</strong></span> order. So if you need critical
hosts to be always available during start/restart, you may not be
able to use <span class="bold"><strong>debug</strong></span>.</p></li></ol></div></div><p>In other run-time failure cases:</p><div class="itemizedlist"><ul type="disc"><li><p>Make a note of the error message that you see.</p></li><li><p><span class="command"><strong>shorewall debug start 2>
/tmp/trace</strong></span></p></li><li><p>Look at the <code class="filename">/tmp/trace</code> file and see if
that helps you determine what the problem is. Be sure you find the
place in the log where the error message you saw is generated --
you should find the message near the end of the log.</p></li><li><p>If you still can't determine what's wrong then see the
<a class="ulink" href="support.htm" target="_self">support page</a>.</p></li></ul></div></div></div><div class="section" lang="en" xml:lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a id="Network"></a>Your Network Environment</h2></div></div></div><p>Many times when people have problems with Shorewall, the problem is
actually an ill-conceived network setup. Here are several popular
snafus:</p><div class="itemizedlist"><ul type="disc"><li><p>Port Forwarding where client and server are in the same subnet.
See <a class="ulink" href="FAQ.htm#faq2" target="_self">FAQ 2</a>.</p></li><li><p>Trying to test net->loc DNAT rules from inside your firewall.
You must test these rules from <span class="bold"><strong>outside</strong></span> your firewall.</p></li><li><p>Multiple interfaces connected to the same HUB or Switch. Given
the way that the Linux kernel respond to ARP “<span class="quote">who-has</span>”
requests, this type of setup <span class="bold"><strong>does NOT work the
way that you expect it to</strong></span>. You can test using this kind of
configuration if you specify the <span class="bold"><strong>arp_filter</strong></span> option or the <span class="bold"><strong>arp_ignore</strong></span> option in <code class="filename"><a class="ulink" href="manpages/shorewall-interfaces.html" target="_self">/etc/shorewall/interfaces</a></code>
for all interfaces connected to the common hub/switch. <span class="bold"><strong>Using such a setup with a production firewall is strongly
recommended against</strong></span>.</p></li></ul></div></div><div class="section" lang="en" xml:lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a id="NewDevice"></a>New Device Doesn't Work?</h2></div></div></div><p>If you have just added a new device such as VOIP and it doesn't
work, be sure that you have assigned it an IP address in your local
network and that its default gateway has been set to the IP address of
your internal interface. For many of these devices, the simplest solution
is to run a DHCP server; running it on your firewall is fine — be sure to
set the <span class="bold"><strong>dhcp</strong></span> option on your internal
interface in <a class="ulink" href="manpages/shorewall-interfaces.html" target="_self">/etc/shorewall/interfaces</a>.</p></div><div class="section" lang="en" xml:lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a id="Connections"></a>Connection Problems</h2></div></div></div><p>One very important thing to remember is that not all connection
problems are Shorewall configuration problems. If the connection that is
giving you problems is to or from the firewall system or if it doesn't
rely on NAT or Proxy ARP then you can often eliminate Shorewall using a
simple test:</p><div class="itemizedlist"><ul type="disc"><li><p><span class="command"><strong>/sbin/shorewall clear</strong></span></p></li><li><p>Try the connection. If it works then the problem is in your
Shorewall configuration; if the connection still doesn't work then the
problem is not with Shorewall or the way that it is configured.</p></li><li><p>Be sure to <span class="command"><strong>/sbin/shorewall start</strong></span> after the
test.</p></li></ul></div><p>If you still suspect Shorewall and the appropriate policy for the
connection that you are trying to make is ACCEPT, please DO NOT ADD
ADDITIONAL ACCEPT RULES TRYING TO MAKE IT WORK. Such additional rules will
NEVER make it work, they add clutter to your rule set and they represent a
big security hole in the event that you forget to remove them
later.</p><p>I also recommend against setting all of your policies to ACCEPT in
an effort to make something work. That robs you of one of your best
diagnostic tools - the “<span class="quote">Shorewall</span>” messages that Netfilter
will generate when you try to connect in a way that isn't permitted by
your rule set.</p><p>Check your log (“<span class="quote"><span class="command"><strong>/sbin/shorewall show
log</strong></span></span>”). If you don't see Shorewall messages, then your
problem is probably NOT a Shorewall problem. If you DO see packet
messages, it may be an indication that you are missing one or more rules
-- see <a class="ulink" href="FAQ.htm#faq17" target="_self">FAQ 17</a>.</p><p>While you are troubleshooting, it is a good idea to clear two
variables in
<code class="filename"><code class="filename">/etc/shorewall/shorewall.conf</code></code>:</p><pre class="programlisting">LOGRATE=
LOGBURST=""</pre><p>This way, you will see all of the log messages
being generated (be sure to restart shorewall after clearing these
variables).</p><div class="example"><a id="Example2"></a><p class="title"><b>Example 2. Log Message</b></p><div class="example-contents"><pre class="programlisting">Jun 27 15:37:56 gateway kernel: Shorewall:all2all:REJECT:IN=eth2
OUT=eth1 SRC=192.168.2.2
DST=192.168.1.3 LEN=67 TOS=0x00
PREC=0x00 TTL=63 ID=5805 DF
PROTO=UDP SPT=1803 DPT=53 LEN=47</pre><p>Let's look at the important parts of this message:</p><div class="itemizedlist"><ul type="disc"><li><p>all2all:REJECT - This packet was REJECTed out of the all2all
chain -- the packet was rejected under the
“<span class="quote">all</span>”->“<span class="quote">all</span>” REJECT policy (see <a class="ulink" href="FAQ.htm#faq17" target="_self">FAQ 17</a>).</p></li><li><p>IN=eth2 - the packet entered the firewall via eth2</p></li><li><p>OUT=eth1 - if accepted, the packet would be sent on
eth1</p></li><li><p>SRC=192.168.2.2 - the packet was sent by 192.168.2.2</p></li><li><p>DST=192.168.1.3 - the packet is destined for
192.168.1.3</p></li><li><p>PROTO=UDP - UDP Protocol</p></li><li><p>DPT=53 - DNS</p></li></ul></div><p>In this case, 192.168.2.2 was in the “<span class="quote">dmz</span>” zone and
192.168.1.3 is in the “<span class="quote">loc</span>” zone. I was missing the
rule:</p><pre class="programlisting">#ACTION SOURCE DEST PROTO DEST
# PORT(S)
ACCEPT dmz loc udp 53</pre></div></div><br class="example-break" /></div><div class="section" lang="en" xml:lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a id="Ping"></a>Ping Problems</h2></div></div></div><p>Either can't ping when you think you should be able to or are able
to ping when you think that you shouldn't be allowed? Shorewall's
“<span class="quote">Ping</span>” Management is <a class="ulink" href="ping.html" target="_self">described
here</a>. Here are a couple of tips:</p><div class="itemizedlist"><ul type="disc"><li><p>Remember that Shorewall doesn't automatically allow ICMP type 8
(“<span class="quote">ping</span>”) requests to be sent between zones. If you want
pings to be allowed between zones, you need a rule of the form:</p><pre class="programlisting">#ACTION SOURCE DEST PROTO DEST
# PORT(S)
Ping/ACCEPT <span class="emphasis"><em><source zone></em></span> <span class="emphasis"><em><destination zone></em></span></pre><p>The ramifications of this can be subtle. For example, if you
have the following in <code class="filename"><a class="ulink" href="NAT.htm" target="_self">/etc/shorewall/nat</a></code>:</p><pre class="programlisting">#EXTERNAL INTERFACE INTERNAL
10.1.1.2 eth0 130.252.100.18</pre><p>and you ping 130.252.100.18, unless you have allowed icmp type 8
between the zone containing the system you are pinging from and the
zone containing 10.1.1.2, the ping requests will be dropped.</p></li><li><p>Ping requests are subject to logging under your policies. So
ping floods can cause an equally big flood of log messages. To
eliminate these, as the last rule in your /etc/shorewall/rules file
add:</p><pre class="programlisting">#ACTION SOURCE DEST PROTO DEST
# PORT(S)
Ping/DROP net all</pre></li></ul></div></div><div class="section" lang="en" xml:lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a id="Other"></a>Some Things to Keep in Mind</h2></div></div></div><div class="itemizedlist"><ul type="disc"><li><p><span class="bold"><strong>You cannot test your firewall from the
inside</strong></span>. Just because you send requests to your firewall
external IP address does not mean that the request will be associated
with the external interface or the “<span class="quote">net</span>” zone. Any
traffic that you generate from the local network will be associated
with your local interface and will be treated as loc->fw
traffic.</p></li><li><p><span class="bold"><strong>IP addresses are properties of systems,
not of interfaces</strong></span>. It is a mistake to believe that your
firewall is able to forward packets just because you can ping the IP
address of all of the firewall's interfaces from the local network.
The only conclusion you can draw from such pinging success is that the
link between the local system and the firewall works and that you
probably have the local system's default gateway set correctly.</p></li><li><p><span class="bold"><strong>All IP addresses configured on firewall
interfaces are in the $FW (fw) zone</strong></span>. If 192.168.1.254 is
the IP address of your internal interface then you can write
“<span class="quote"><span class="bold"><strong>$FW:192.168.1.254</strong></span></span>” in a
rule but you may not write “<span class="quote"><span class="bold"><strong>loc:192.168.1.254</strong></span></span>”. Similarly, it is
nonsensical to add 192.168.1.254 to the <span class="bold"><strong>loc</strong></span> zone using an entry in
<code class="filename">/etc/shorewall/hosts</code>.</p></li><li><p><span class="bold"><strong>Reply packets do NOT automatically follow
the reverse path of the one taken by the original request</strong></span>.
All packets are routed according to the routing table of the host at
each step of the way. This issue commonly comes up when people install
a Shorewall firewall parallel to an existing gateway and try to use
DNAT through Shorewall without changing the default gateway of the
system receiving the forwarded requests. Requests come in through the
Shorewall firewall where the destination IP address gets rewritten but
replies go out unmodified through the old gateway.</p></li><li><p><span class="bold"><strong>Shorewall itself has no notion of inside
or outside</strong></span>. These concepts are embodied in how Shorewall is
configured.</p></li></ul></div></div><div class="section" lang="en" xml:lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a id="More"></a>Other Gotchas</h2></div></div></div><div class="itemizedlist"><ul type="disc"><li><p>Seeing rejected/dropped packets logged out of the INPUT or
FORWARD chains? This means that:</p><div class="orderedlist"><ol type="1"><li><p>your zone definitions are screwed up and the host that is
sending the packets or the destination host isn't in any zone
(using an <a class="ulink" href="manpages/shorewall-hosts.html" target="_self"><code class="filename">/etc/shorewall/hosts</code></a>
file are you?); or</p></li><li><p>the source and destination hosts are both connected to the
same interface and you don't have a policy or rule for the source
zone to or from the destination zone or you haven't set the
<span class="bold"><strong>routeback</strong></span> option for the
interface in <a class="ulink" href="manpages/shorewall-interfaces.html" target="_self"><code class="filename">/etc/shorewall/interfaces</code></a>.</p></li><li><p>You have connected two firewall interfaces (from different
zones) to the same hub or switch.</p></li></ol></div></li><li><p>If you specify “<span class="quote">routefilter</span>” for an interface, that
interface must be up prior to starting the firewall.</p></li><li><p>Is your routing correct? For example, internal systems usually
need to be configured with their default gateway set to the IP address
of their nearest firewall interface. One often overlooked aspect of
routing is that in order for two hosts to communicate, the routing
between them must be set up <span class="bold"><strong>in both
directions</strong></span>. So when setting up routing between <span class="bold"><strong>A</strong></span> and <span class="bold"><strong>B</strong></span>, be
sure to verify that the route from <span class="bold"><strong>B</strong></span>
back to <span class="bold"><strong>A</strong></span> is defined and
correct.</p></li><li><p>Do you have your kernel properly configured? <a class="ulink" href="kernel.htm" target="_self">Click here to see kernel configuration
information</a>.</p></li></ul></div></div><div class="section" lang="en" xml:lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a id="Support"></a>Still Having Problems?</h2></div></div></div><p>See the <a class="ulink" href="support.htm" target="_self">Shorewall Support
Page</a>.</p></div></div></body></html>
