<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml"> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <script type="text/javascript"> var _gaq = _gaq || []; _gaq.push(['_setAccount', 'UA-3052036-8']); _gaq.push(['_trackPageview']); (function() { var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); })(); </script> <title>Configuring Output — Twiggy</title> <link rel="stylesheet" href="_static/default.css" type="text/css" /> <link rel="stylesheet" href="_static/pygments.css" type="text/css" /> <script type="text/javascript"> var DOCUMENTATION_OPTIONS = { URL_ROOT: '', VERSION: '0.4.4', COLLAPSE_INDEX: false, FILE_SUFFIX: '.html', HAS_SOURCE: true }; </script> <script type="text/javascript" src="_static/jquery.js"></script> <script type="text/javascript" src="_static/underscore.js"></script> <script type="text/javascript" src="_static/doctools.js"></script> <link rel="search" type="application/opensearchdescription+xml" title="Search within Twiggy" href="_static/opensearch.xml"/> <link rel="top" title="Twiggy" href="index.html" /> <link rel="next" title="Reference Guide" href="reference_guide.html" /> <link rel="prev" title="Logging Messages" href="logging.html" /> </head> <body> <div class="related"> <h3>Navigation</h3> <ul> <li class="right" style="margin-right: 10px"> <a href="genindex.html" title="General Index" accesskey="I">index</a></li> <li class="right" > <a href="reference_guide.html" title="Reference Guide" accesskey="N">next</a> |</li> <li class="right" > <a href="logging.html" title="Logging Messages" accesskey="P">previous</a> |</li> <li><a href="index.html">Twiggy v0.4.4 documentation</a> »</li> </ul> </div> <div class="document"> <div class="documentwrapper"> <div class="bodywrapper"> <div class="body"> <div class="section" id="configuring-output"> <h1>Configuring Output<a class="headerlink" href="#configuring-output" title="Permalink to this headline">¶</a></h1> <p>This part discusses how to configure twiggy’s output of messages. You should do this once, near the start of your application’s <tt class="docutils literal"><span class="pre">__main__</span></tt>.</p> <div class="section" id="quick-setup"> <h2>Quick Setup<a class="headerlink" href="#quick-setup" title="Permalink to this headline">¶</a></h2> <p>To quickly configure output, use the <a class="reference internal" href="api.html#twiggy.quickSetup" title="twiggy.quickSetup"><tt class="xref py py-obj docutils literal"><span class="pre">quickSetup</span></tt></a> function. Quick setup is limited to sending all messages to a file or <tt class="docutils literal"><span class="pre">sys.stderr</span></tt>. A timestamp will be prefixed when logging to a file.</p> <dl class="function"> <dt> <tt class="descclassname">twiggy.</tt><tt class="descname">quickSetup</tt><big>(</big><em>min_level=<LogLevel DEBUG></em>, <em>file=None</em>, <em>msg_buffer=0</em><big>)</big></dt> <dd><p>Quickly set up <a class="reference internal" href="api.html#twiggy.emitters" title="twiggy.emitters"><tt class="xref py py-obj docutils literal"><span class="pre">emitters</span></tt></a>.</p> <table class="docutils field-list" frame="void" rules="none"> <col class="field-name" /> <col class="field-body" /> <tbody valign="top"> <tr class="field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first last simple"> <li><strong>min_level</strong> (<em>LogLevel</em>) – lowest message level to cause output</li> <li><strong>file</strong> (<em>string</em>) – filename to log to, or <tt class="docutils literal"><span class="pre">sys.stdout</span></tt>, or <tt class="docutils literal"><span class="pre">sys.stderr</span></tt>. <tt class="xref docutils literal"><span class="pre">None</span></tt> means standard error.</li> <li><strong>msg_buffer</strong> (<em>int</em>) – number of messages to buffer, see <tt class="xref py py-obj docutils literal"><span class="pre">outputs.AsyncOutput.msg_buffer</span></tt></li> </ul> </td> </tr> </tbody> </table> </dd></dl> </div> <div class="section" id="twiggy-setup-py"> <span id="twiggy-setup"></span><h2>twiggy_setup.py<a class="headerlink" href="#twiggy-setup-py" title="Permalink to this headline">¶</a></h2> <p>Twiggy’s output side features modern, loosely coupled design.</p> <p>By convention, your configuration lives in a file in your application called <tt class="docutils literal"><span class="pre">twiggy_setup.py</span></tt>, in a function called <tt class="docutils literal"><span class="pre">twiggy_setup()</span></tt>. You can of course put your configuration elsewhere, but using a separate module makes integration with configuration management systems easy. You should import and run <tt class="docutils literal"><span class="pre">twiggy_setup</span></tt> near the top of your application. It’s particularly important to set up Twiggy <em>before spawning new processes</em>.</p> <p>A <tt class="docutils literal"><span class="pre">twiggy_setup</span></tt> function should create ouputs and use the <a class="reference internal" href="api.html#twiggy.addEmitters" title="twiggy.addEmitters"><tt class="xref py py-func docutils literal"><span class="pre">addEmitters()</span></tt></a> convenience function to link those outputs to the <a class="reference internal" href="api.html#twiggy.log" title="twiggy.log"><tt class="xref py py-data docutils literal"><span class="pre">log</span></tt></a>.</p> <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">twiggy</span> <span class="kn">import</span> <span class="n">addEmitters</span><span class="p">,</span> <span class="n">outputs</span><span class="p">,</span> <span class="n">levels</span><span class="p">,</span> <span class="n">filters</span><span class="p">,</span> <span class="n">formats</span><span class="p">,</span> <span class="n">emitters</span> <span class="c"># import * is also ok</span> <span class="k">def</span> <span class="nf">twiggy_setup</span><span class="p">():</span> <span class="n">alice_output</span> <span class="o">=</span> <span class="n">outputs</span><span class="o">.</span><span class="n">FileOutput</span><span class="p">(</span><span class="s">"alice.log"</span><span class="p">,</span> <span class="n">format</span><span class="o">=</span><span class="n">formats</span><span class="o">.</span><span class="n">line_format</span><span class="p">)</span> <span class="n">bob_output</span> <span class="o">=</span> <span class="n">outputs</span><span class="o">.</span><span class="n">FileOutput</span><span class="p">(</span><span class="s">"bob.log"</span><span class="p">,</span> <span class="n">format</span><span class="o">=</span><span class="n">formats</span><span class="o">.</span><span class="n">line_format</span><span class="p">)</span> <span class="n">addEmitters</span><span class="p">(</span> <span class="c"># (name, min_level, filter, output),</span> <span class="p">(</span><span class="s">"alice"</span><span class="p">,</span> <span class="n">levels</span><span class="o">.</span><span class="n">DEBUG</span><span class="p">,</span> <span class="bp">None</span><span class="p">,</span> <span class="n">alice_output</span><span class="p">),</span> <span class="p">(</span><span class="s">"betty"</span><span class="p">,</span> <span class="n">levels</span><span class="o">.</span><span class="n">INFO</span><span class="p">,</span> <span class="n">filters</span><span class="o">.</span><span class="n">names</span><span class="p">(</span><span class="s">"betty"</span><span class="p">),</span> <span class="n">bob_output</span><span class="p">),</span> <span class="p">(</span><span class="s">"brian.*"</span><span class="p">,</span> <span class="n">levels</span><span class="o">.</span><span class="n">DEBUG</span><span class="p">,</span> <span class="n">filters</span><span class="o">.</span><span class="n">glob_names</span><span class="p">(</span><span class="s">"brian.*"</span><span class="p">),</span> <span class="n">bob_output</span><span class="p">),</span> <span class="p">)</span> <span class="c"># near the top of your __main__</span> <span class="n">twiggy_setup</span><span class="p">()</span> </pre></div> </div> <p><a class="reference internal" href="api.html#twiggy.addEmitters" title="twiggy.addEmitters"><tt class="xref py py-func docutils literal"><span class="pre">addEmitters()</span></tt></a> populates the <a class="reference internal" href="api.html#twiggy.emitters" title="twiggy.emitters"><tt class="xref py py-data docutils literal"><span class="pre">emitters</span></tt></a> dictionary:</p> <div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="nb">sorted</span><span class="p">(</span><span class="n">emitters</span><span class="o">.</span><span class="n">keys</span><span class="p">())</span> <span class="go">['alice', 'betty', 'brian.*']</span> </pre></div> </div> <p>In this example, we create two log destinations: <tt class="docutils literal"><span class="pre">alice.log</span></tt> and <tt class="docutils literal"><span class="pre">bob.log</span></tt>. alice will recieve all messages, and bob will receive two sets of messages:</p> <ul class="simple"> <li>messages with the name field equal to <tt class="docutils literal"><span class="pre">betty</span></tt> and level >= <tt class="docutils literal"><span class="pre">INFO</span></tt></li> <li>messages with the name field glob-matching <tt class="docutils literal"><span class="pre">brian.*</span></tt></li> </ul> <p><a class="reference internal" href="api.html#twiggy.filters.Emitter" title="twiggy.filters.Emitter"><tt class="xref py py-class docutils literal"><span class="pre">Emitters</span></tt></a> can be removed by deleting them from this dict. <a class="reference internal" href="api.html#twiggy.filters.Emitter.filter" title="twiggy.filters.Emitter.filter"><tt class="xref py py-attr docutils literal"><span class="pre">filter</span></tt></a> and <a class="reference internal" href="api.html#twiggy.filters.Emitter.min_level" title="twiggy.filters.Emitter.min_level"><tt class="xref py py-attr docutils literal"><span class="pre">min_level</span></tt></a> may be modified during the running of the application, but outputs <em>cannot</em> be changed. Instead, remove the emitter and re-add it.</p> <div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="c"># bump level</span> <span class="gp">... </span><span class="n">emitters</span><span class="p">[</span><span class="s">'alice'</span><span class="p">]</span><span class="o">.</span><span class="n">min_level</span> <span class="o">=</span> <span class="n">levels</span><span class="o">.</span><span class="n">WARNING</span> <span class="gp">>>> </span><span class="c"># change filter</span> <span class="gp">... </span><span class="n">emitters</span><span class="p">[</span><span class="s">'alice'</span><span class="p">]</span><span class="o">.</span><span class="n">filter</span> <span class="o">=</span> <span class="n">filters</span><span class="o">.</span><span class="n">names</span><span class="p">(</span><span class="s">'alice'</span><span class="p">,</span> <span class="s">'andy'</span><span class="p">)</span> <span class="gp">>>> </span><span class="c"># remove entirely</span> <span class="gp">... </span><span class="k">del</span> <span class="n">emitters</span><span class="p">[</span><span class="s">'alice'</span><span class="p">]</span> </pre></div> </div> <p>We’ll examine the various parts in more detail.</p> </div> <div class="section" id="outputs"> <h2>Outputs<a class="headerlink" href="#outputs" title="Permalink to this headline">¶</a></h2> <p>Outputs are the destinations to which log messages are written (files, databases, etc.). Several <a class="reference internal" href="api.html#module-twiggy.outputs" title="twiggy.outputs"><tt class="xref py py-mod docutils literal"><span class="pre">implementations</span></tt></a> are provided. Once created, outputs cannot be modified. Each output has an associated <a class="reference internal" href="api.html#module-twiggy.formats" title="twiggy.formats"><tt class="xref py py-mod docutils literal"><span class="pre">format</span></tt></a>.</p> <div class="section" id="asynchronous-logging"> <span id="async-logging"></span><h3>Asynchronous Logging<a class="headerlink" href="#asynchronous-logging" title="Permalink to this headline">¶</a></h3> <p>Many outputs can be configured to use a separate, dedicated process to log messages. This is known as <a class="reference internal" href="glossary.html#term-asynchronous-logging"><em class="xref std std-term">asynchronous logging</em></a> and is enabled with the <a class="reference internal" href="api.html#twiggy.outputs.AsyncOutput" title="twiggy.outputs.AsyncOutput"><tt class="xref py py-obj docutils literal"><span class="pre">msg_buffer</span></tt></a> argument. Asynchronous mode dramatically reduces the cost of logging, as expensive formatting and writing operations are moved out of the main thread of control.</p> </div> </div> <div class="section" id="formats"> <h2>Formats<a class="headerlink" href="#formats" title="Permalink to this headline">¶</a></h2> <p><a class="reference internal" href="api.html#module-twiggy.formats" title="twiggy.formats"><tt class="xref py py-mod docutils literal"><span class="pre">Formats</span></tt></a> transform a log message into a form that can be written by an output. The result of formatting is output dependent; for example, an output that posts to an HTTP server may take a format that provides JSON, whereas an output that writes to a file may produce text.</p> <div class="section" id="line-oriented-formatting"> <h3>Line-oriented formatting<a class="headerlink" href="#line-oriented-formatting" title="Permalink to this headline">¶</a></h3> <p><a class="reference internal" href="api.html#twiggy.formats.LineFormat" title="twiggy.formats.LineFormat"><tt class="xref py py-class docutils literal"><span class="pre">LineFormat</span></tt></a> formats messages for text-oriented outputs such as a file or standard error. It uses a <a class="reference internal" href="api.html#twiggy.lib.converter.ConversionTable" title="twiggy.lib.converter.ConversionTable"><tt class="xref py py-obj docutils literal"><span class="pre">ConversionTable</span></tt></a> to stringify the arbitrary fields in a message. To customize, copy the default <a class="reference internal" href="api.html#twiggy.formats.line_format" title="twiggy.formats.line_format"><tt class="xref py py-data docutils literal"><span class="pre">line_format</span></tt></a> and modify:</p> <div class="highlight-python"><div class="highlight"><pre><span class="c"># in your twiggy_setup</span> <span class="kn">import</span> <span class="nn">copy</span> <span class="n">my_format</span> <span class="o">=</span> <span class="n">copy</span><span class="o">.</span><span class="n">copy</span><span class="p">(</span><span class="n">formats</span><span class="o">.</span><span class="n">line_format</span><span class="p">)</span> <span class="n">my_format</span><span class="o">.</span><span class="n">conversion</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">key</span> <span class="o">=</span> <span class="s">'address'</span><span class="p">,</span> <span class="c"># name of the field</span> <span class="n">convertValue</span> <span class="o">=</span> <span class="nb">hex</span><span class="p">,</span> <span class="c"># gets original value</span> <span class="n">convertItem</span> <span class="o">=</span> <span class="s">"{0}={1}"</span><span class="o">.</span><span class="n">format</span><span class="p">,</span> <span class="c"># gets called with: key, converted_value</span> <span class="n">required</span> <span class="o">=</span> <span class="bp">True</span><span class="p">)</span> <span class="c"># output messages with name 'memory' to stderr</span> <span class="n">addEmitters</span><span class="p">((</span><span class="s">'memory'</span><span class="p">,</span> <span class="n">levels</span><span class="o">.</span><span class="n">DEBUG</span><span class="p">,</span> <span class="n">filters</span><span class="o">.</span><span class="n">names</span><span class="p">(</span><span class="s">'memory'</span><span class="p">),</span> <span class="n">outputs</span><span class="o">.</span><span class="n">StreamOutput</span><span class="p">(</span><span class="n">format</span> <span class="o">=</span> <span class="n">my_format</span><span class="p">)))</span> </pre></div> </div> </div> </div> <div class="section" id="filtering-output"> <h2>Filtering Output<a class="headerlink" href="#filtering-output" title="Permalink to this headline">¶</a></h2> <p>The messages output by an emitter are determined by its <a class="reference internal" href="api.html#twiggy.filters.Emitter.min_level" title="twiggy.filters.Emitter.min_level"><tt class="xref py py-attr docutils literal"><span class="pre">min_level</span></tt></a> and filter (a <a class="reference internal" href="api.html#twiggy.filters.filter" title="twiggy.filters.filter"><tt class="xref py py-func docutils literal"><span class="pre">function</span></tt></a> which take a <a class="reference internal" href="api.html#twiggy.message.Message" title="twiggy.message.Message"><tt class="xref py py-class docutils literal"><span class="pre">Message</span></tt></a> and returns bool). These attributes may be changed while the application is running. The <a class="reference internal" href="api.html#twiggy.filters.Emitter.filter" title="twiggy.filters.Emitter.filter"><tt class="xref py py-attr docutils literal"><span class="pre">filter</span></tt></a> attribute of emitters is <a class="reference internal" href="api.html#twiggy.filters.msgFilter" title="twiggy.filters.msgFilter"><tt class="xref py py-obj docutils literal"><span class="pre">intelligent</span></tt></a>; you may assign strings, bools or functions and it will magically do the right thing. Assigning a list indicates that <em>all</em> of the filters must pass for the message to be output.</p> <div class="highlight-python"><div class="highlight"><pre><span class="n">e</span> <span class="o">=</span> <span class="n">emitters</span><span class="p">[</span><span class="s">'memory'</span><span class="p">]</span> <span class="n">e</span><span class="o">.</span><span class="n">min_level</span> <span class="o">=</span> <span class="n">levels</span><span class="o">.</span><span class="n">WARNING</span> <span class="c"># True allows all messages through (None works as well)</span> <span class="n">e</span><span class="o">.</span><span class="n">filter</span> <span class="o">=</span> <span class="bp">True</span> <span class="c"># False blocks all messages</span> <span class="n">e</span><span class="o">.</span><span class="n">filter</span> <span class="o">=</span> <span class="bp">False</span> <span class="c"># Strings are interpreted as regexes (regex objects ok too)</span> <span class="n">e</span><span class="o">.</span><span class="n">filter</span> <span class="o">=</span> <span class="s">"^mem.*y$"</span> <span class="c"># functions are passed the message; return True to emit</span> <span class="n">e</span><span class="o">.</span><span class="n">filter</span> <span class="o">=</span> <span class="k">lambda</span> <span class="n">msg</span><span class="p">:</span> <span class="n">msg</span><span class="o">.</span><span class="n">fields</span><span class="p">[</span><span class="s">'address'</span><span class="p">]</span> <span class="o">></span> <span class="mh">0xDECAF</span> <span class="c"># lists are all()'d</span> <span class="n">e</span><span class="o">.</span><span class="n">filter</span> <span class="o">=</span> <span class="p">[</span><span class="s">"^mem.y$"</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">msg</span><span class="p">:</span> <span class="n">msg</span><span class="o">.</span><span class="n">fields</span><span class="p">[</span><span class="s">'address'</span><span class="p">]</span> <span class="o">></span> <span class="mh">0xDECAF</span><span class="p">]</span> </pre></div> </div> <div class="admonition-see-also admonition seealso"> <p class="first admonition-title">See also</p> <p class="last">Available <a class="reference internal" href="api.html#module-twiggy.filters" title="twiggy.filters"><tt class="xref py py-mod docutils literal"><span class="pre">filters</span></tt></a></p> </div> </div> </div> </div> </div> </div> <div class="sphinxsidebar"> <div class="sphinxsidebarwrapper"> <p class="logo"><a href="index.html"> <img class="logo" src="_static/twig_sidebar.jpg" alt="Logo"/> </a></p> <h3><a href="index.html">Table Of Contents</a></h3> <ul> <li><a class="reference internal" href="#">Configuring Output</a><ul> <li><a class="reference internal" href="#quick-setup">Quick Setup</a></li> <li><a class="reference internal" href="#twiggy-setup-py">twiggy_setup.py</a></li> <li><a class="reference internal" href="#outputs">Outputs</a><ul> <li><a class="reference internal" href="#asynchronous-logging">Asynchronous Logging</a></li> </ul> </li> <li><a class="reference internal" href="#formats">Formats</a><ul> <li><a class="reference internal" href="#line-oriented-formatting">Line-oriented formatting</a></li> </ul> </li> <li><a class="reference internal" href="#filtering-output">Filtering Output</a></li> </ul> </li> </ul> <h4>Previous topic</h4> <p class="topless"><a href="logging.html" title="previous chapter">Logging Messages</a></p> <h4>Next topic</h4> <p class="topless"><a href="reference_guide.html" title="next chapter">Reference Guide</a></p> <h3>This Page</h3> <ul class="this-page-menu"> <li><a href="_sources/configuration.txt" rel="nofollow">Show Source</a></li> </ul> <div id="searchbox" style="display: none"> <h3>Quick search</h3> <form class="search" action="search.html" method="get"> <input type="text" name="q" size="18" /> <input type="submit" value="Go" /> <input type="hidden" name="check_keywords" value="yes" /> <input type="hidden" name="area" value="default" /> </form> <p class="searchtip" style="font-size: 90%"> Enter search terms or a module, class or function name. </p> </div> <script type="text/javascript">$('#searchbox').show(0);</script> </div> </div> <div class="clearer"></div> </div> <div class="related"> <h3>Navigation</h3> <ul> <li class="right" style="margin-right: 10px"> <a href="genindex.html" title="General Index" >index</a></li> <li class="right" > <a href="reference_guide.html" title="Reference Guide" >next</a> |</li> <li class="right" > <a href="logging.html" title="Logging Messages" >previous</a> |</li> <li><a href="index.html">Twiggy v0.4.4 documentation</a> »</li> </ul> </div> <div class="footer"> © Copyright 2010, <a href="http://i.wearpants.org">Peter Fein</a>. Last updated on Jul 10, 2012. Created using <a href="http://sphinx.pocoo.org/">Sphinx</a> 1.0.7. </div> </body> </html>