<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Eventloops and PyZMQ — PyZMQ v2.2.0.1 documentation</title>
<link rel="stylesheet" href="_static/default.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '',
VERSION: '2.2.0.1',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
};
</script>
<script type="text/javascript" src="_static/jquery.js"></script>
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<link rel="shortcut icon" href="_static/zeromq.ico"/>
<link rel="top" title="PyZMQ v2.2.0.1 documentation" href="index.html" />
<link rel="next" title="Asynchronous Logging via PyZMQ" href="logging.html" />
<link rel="prev" title="Devices in PyZMQ" href="devices.html" />
</head>
<body>
<div style="background-color: white; text-align: left; padding: 10px 10px 15px 15px">
<a href="index.html"><img src="_static/logo.png" border="0" alt="PyZMQ Documentation"/></a>
</div>
<div class="related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="genindex.html" title="General Index"
accesskey="I">index</a></li>
<li class="right" >
<a href="py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="right" >
<a href="logging.html" title="Asynchronous Logging via PyZMQ"
accesskey="N">next</a> |</li>
<li class="right" >
<a href="devices.html" title="Devices in PyZMQ"
accesskey="P">previous</a> |</li>
<li><a href="index.html">home</a>| </li>
<li><a href="search.html">search</a>| </li>
<li><a href="api/index.html">API</a> »</li>
</ul>
</div>
<div class="sphinxsidebar">
<div class="sphinxsidebarwrapper">
<h3><a href="index.html">Table Of Contents</a></h3>
<ul>
<li><a class="reference internal" href="#">Eventloops and PyZMQ</a><ul>
<li><a class="reference internal" href="#tornado-ioloop">Tornado IOLoop</a><ul>
<li><a class="reference internal" href="#install"><tt class="docutils literal"><span class="pre">install()</span></tt></a></li>
<li><a class="reference internal" href="#send"><tt class="docutils literal"><span class="pre">send()</span></tt></a></li>
<li><a class="reference internal" href="#on-recv"><tt class="docutils literal"><span class="pre">on_recv()</span></tt></a></li>
<li><a class="reference internal" href="#on-recv-stream"><tt class="docutils literal"><span class="pre">on_recv_stream()</span></tt></a></li>
<li><a class="reference internal" href="#flush"><tt class="docutils literal"><span class="pre">flush()</span></tt></a></li>
</ul>
</li>
<li><a class="reference internal" href="#gevent">gevent</a></li>
</ul>
</li>
</ul>
<h4>Previous topic</h4>
<p class="topless"><a href="devices.html"
title="previous chapter">Devices in PyZMQ</a></p>
<h4>Next topic</h4>
<p class="topless"><a href="logging.html"
title="next chapter">Asynchronous Logging via PyZMQ</a></p>
<h3>This Page</h3>
<ul class="this-page-menu">
<li><a href="_sources/eventloop.txt"
rel="nofollow">Show Source</a></li>
</ul>
<div id="searchbox" style="display: none">
<h3>Quick search</h3>
<form class="search" action="search.html" method="get">
<input type="text" name="q" size="18" />
<input type="submit" value="Go" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
<p class="searchtip" style="font-size: 90%">
Enter search terms or a module, class or function name.
</p>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body">
<div class="section" id="eventloops-and-pyzmq">
<span id="eventloop"></span><h1>Eventloops and PyZMQ<a class="headerlink" href="#eventloops-and-pyzmq" title="Permalink to this headline">¶</a></h1>
<div class="section" id="tornado-ioloop">
<h2>Tornado IOLoop<a class="headerlink" href="#tornado-ioloop" title="Permalink to this headline">¶</a></h2>
<p>Facebook’s <a class="reference external" href="https://github.com/facebook/tornado">Tornado</a> includes an eventloop for handing poll events on filedescriptors and
native sockets. We have included a small part of Tornado (specifically its
<a class="reference internal" href="api/generated/zmq.eventloop.ioloop.html#module-zmq.eventloop.ioloop" title="zmq.eventloop.ioloop"><tt class="xref py py-mod docutils literal"><span class="pre">ioloop</span></tt></a>), and adapted its <tt class="xref py py-class docutils literal"><span class="pre">IOStream</span></tt> class into <a class="reference internal" href="api/generated/zmq.eventloop.zmqstream.html#zmq.eventloop.zmqstream.ZMQStream" title="zmq.eventloop.zmqstream.ZMQStream"><tt class="xref py py-class docutils literal"><span class="pre">ZMQStream</span></tt></a> for
handling poll events on ØMQ sockets. A ZMQStream object works much like a Socket object,
but instead of calling <tt class="xref py py-meth docutils literal"><span class="pre">recv()</span></tt> directly, you register a callback with
<a class="reference internal" href="api/generated/zmq.eventloop.zmqstream.html#zmq.eventloop.zmqstream.ZMQStream.on_recv" title="zmq.eventloop.zmqstream.ZMQStream.on_recv"><tt class="xref py py-meth docutils literal"><span class="pre">on_recv()</span></tt></a>. callbacks can also be registered for send events
with <a class="reference internal" href="api/generated/zmq.eventloop.zmqstream.html#zmq.eventloop.zmqstream.ZMQStream.on_send" title="zmq.eventloop.zmqstream.ZMQStream.on_send"><tt class="xref py py-meth docutils literal"><span class="pre">on_send()</span></tt></a>.</p>
<div class="section" id="install">
<h3><tt class="xref py py-func docutils literal"><span class="pre">install()</span></tt><a class="headerlink" href="#install" title="Permalink to this headline">¶</a></h3>
<p>With PyZMQ’s ioloop, you can use zmq sockets in any tornado application. You must first
install PyZMQ’s <a class="reference internal" href="api/generated/zmq.eventloop.ioloop.html#zmq.eventloop.ioloop.IOLoop" title="zmq.eventloop.ioloop.IOLoop"><tt class="xref py py-class docutils literal"><span class="pre">IOLoop</span></tt></a>, with the <a class="reference internal" href="api/generated/zmq.eventloop.ioloop.html#zmq.eventloop.ioloop.install" title="zmq.eventloop.ioloop.install"><tt class="xref py py-func docutils literal"><span class="pre">ioloop.install()</span></tt></a> function:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">zmq.eventloop</span> <span class="kn">import</span> <span class="n">ioloop</span>
<span class="n">ioloop</span><span class="o">.</span><span class="n">install</span><span class="p">()</span>
</pre></div>
</div>
<p>This sets the global instance of <tt class="xref py py-class docutils literal"><span class="pre">tornado.ioloop.IOLoop</span></tt> with the global instance of
our IOLoop class. The reason this must happen is that tornado objects avoid having to pass
the active IOLoop instance around by having a staticmethod <a class="reference internal" href="api/generated/zmq.eventloop.ioloop.html#zmq.eventloop.ioloop.IOLoop.instance" title="zmq.eventloop.ioloop.IOLoop.instance"><tt class="xref py py-meth docutils literal"><span class="pre">IOLoop.instance()</span></tt></a>, which
always returns the active instance. If PyZMQ’s IOLoop is installed after the first call to
<a class="reference internal" href="api/generated/zmq.eventloop.ioloop.html#zmq.eventloop.ioloop.IOLoop.instance" title="zmq.eventloop.ioloop.IOLoop.instance"><tt class="xref py py-meth docutils literal"><span class="pre">IOLoop.instance()</span></tt></a> (called in almost every tornado object constructor), then it will
raise an <tt class="xref py py-exc docutils literal"><span class="pre">AssertionError</span></tt>, because the global IOLoop instance has already been
created, and proceeding would result in not all objects being associated with the right
IOLoop.</p>
<p>It is possible to use PyZMQ sockets with tornado <em>without</em> calling <a class="reference internal" href="api/generated/zmq.eventloop.ioloop.html#zmq.eventloop.ioloop.install" title="zmq.eventloop.ioloop.install"><tt class="xref py py-func docutils literal"><span class="pre">ioloop.install()</span></tt></a>,
but it is less convenient. First, you must instruct the tornado IOLoop to use the zmq poller:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">tornado.ioloop</span> <span class="kn">import</span> <span class="n">IOLoop</span>
<span class="kn">from</span> <span class="nn">zmq.eventloop.ioloop</span> <span class="kn">import</span> <span class="n">ZMQPoller</span>
<span class="n">loop</span> <span class="o">=</span> <span class="n">IOLoop</span><span class="p">(</span><span class="n">ZMQPoller</span><span class="p">())</span>
</pre></div>
</div>
<p>Then, when you instantiate tornado and ZMQStream objects, you must pass the <cite>io_loop</cite>
argument to ensure that they use this loop, instead of the global instance. You can
install this IOLoop as the global tornado instance, with:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="n">loop</span><span class="o">.</span><span class="n">install</span><span class="p">()</span>
</pre></div>
</div>
<p>but it will <strong>NOT</strong> be the global <em>pyzmq</em> IOLoop instance, so it must still be passed to
your ZMQStream constructors.</p>
</div>
<div class="section" id="send">
<h3><tt class="xref py py-meth docutils literal"><span class="pre">send()</span></tt><a class="headerlink" href="#send" title="Permalink to this headline">¶</a></h3>
<p>ZMQStream objects do have <a class="reference internal" href="api/generated/zmq.eventloop.zmqstream.html#zmq.eventloop.zmqstream.ZMQStream.send" title="zmq.eventloop.zmqstream.ZMQStream.send"><tt class="xref py py-meth docutils literal"><span class="pre">send()</span></tt></a> and <a class="reference internal" href="api/generated/zmq.eventloop.zmqstream.html#zmq.eventloop.zmqstream.ZMQStream.send_multipart" title="zmq.eventloop.zmqstream.ZMQStream.send_multipart"><tt class="xref py py-meth docutils literal"><span class="pre">send_multipart()</span></tt></a>
methods, which behaves the same way as <tt class="xref py py-meth docutils literal"><span class="pre">Socket.send()</span></tt>, but instead of sending right
away, the <a class="reference internal" href="api/generated/zmq.eventloop.ioloop.html#zmq.eventloop.ioloop.IOLoop" title="zmq.eventloop.ioloop.IOLoop"><tt class="xref py py-class docutils literal"><span class="pre">IOLoop</span></tt></a> will wait until socket is able to send (for instance if <tt class="docutils literal"><span class="pre">HWM</span></tt>
is met, or a <tt class="docutils literal"><span class="pre">REQ/REP</span></tt> pattern prohibits sending at a certain point). Messages sent via
send will also be passed to the callback registered with <a class="reference internal" href="api/generated/zmq.eventloop.zmqstream.html#zmq.eventloop.zmqstream.ZMQStream.on_send" title="zmq.eventloop.zmqstream.ZMQStream.on_send"><tt class="xref py py-meth docutils literal"><span class="pre">on_send()</span></tt></a> after
sending.</p>
</div>
<div class="section" id="on-recv">
<h3><tt class="xref py py-meth docutils literal"><span class="pre">on_recv()</span></tt><a class="headerlink" href="#on-recv" title="Permalink to this headline">¶</a></h3>
<p><a class="reference internal" href="api/generated/zmq.eventloop.zmqstream.html#zmq.eventloop.zmqstream.ZMQStream.on_recv" title="zmq.eventloop.zmqstream.ZMQStream.on_recv"><tt class="xref py py-meth docutils literal"><span class="pre">ZMQStream.on_recv()</span></tt></a> is the primary method for using a ZMQStream. It registers a
callback to fire with messages as they are received, which will <em>always</em> be multipart,
even if its length is 1. You can easily use this to build things like an echo socket:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="n">s</span> <span class="o">=</span> <span class="n">ctx</span><span class="o">.</span><span class="n">socket</span><span class="p">(</span><span class="n">zmq</span><span class="o">.</span><span class="n">REP</span><span class="p">)</span>
<span class="n">s</span><span class="o">.</span><span class="n">bind</span><span class="p">(</span><span class="s">'tcp://localhost:12345'</span><span class="p">)</span>
<span class="n">stream</span> <span class="o">=</span> <span class="n">ZMQStream</span><span class="p">(</span><span class="n">s</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">echo</span><span class="p">(</span><span class="n">msg</span><span class="p">):</span>
<span class="n">stream</span><span class="o">.</span><span class="n">send_multipart</span><span class="p">(</span><span class="n">msg</span><span class="p">)</span>
<span class="n">stream</span><span class="o">.</span><span class="n">on_recv</span><span class="p">(</span><span class="n">echo</span><span class="p">)</span>
<span class="n">ioloop</span><span class="o">.</span><span class="n">IOLoop</span><span class="o">.</span><span class="n">instance</span><span class="p">()</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
</pre></div>
</div>
<p>on_recv can also take a <cite>copy</cite> flag, just like <tt class="xref py py-meth docutils literal"><span class="pre">Socket.recv()</span></tt>. If <cite>copy=False</cite>, then
callbacks registered with on_recv will receive tracked <tt class="xref py py-class docutils literal"><span class="pre">Frame</span></tt> objects instead of
bytes.</p>
</div>
<div class="section" id="on-recv-stream">
<h3><tt class="xref py py-meth docutils literal"><span class="pre">on_recv_stream()</span></tt><a class="headerlink" href="#on-recv-stream" title="Permalink to this headline">¶</a></h3>
<p><a class="reference internal" href="api/generated/zmq.eventloop.zmqstream.html#zmq.eventloop.zmqstream.ZMQStream.on_recv_stream" title="zmq.eventloop.zmqstream.ZMQStream.on_recv_stream"><tt class="xref py py-meth docutils literal"><span class="pre">ZMQStream.on_recv_stream()</span></tt></a> is just like on_recv above, but the callback will be
passed both the message and the stream, rather than just the message. This is meant to make
it easier to use a single callback with multiple streams.</p>
<div class="highlight-python"><div class="highlight"><pre><span class="n">s1</span> <span class="o">=</span> <span class="n">ctx</span><span class="o">.</span><span class="n">socket</span><span class="p">(</span><span class="n">zmq</span><span class="o">.</span><span class="n">REP</span><span class="p">)</span>
<span class="n">s1</span><span class="o">.</span><span class="n">bind</span><span class="p">(</span><span class="s">'tcp://localhost:12345'</span><span class="p">)</span>
<span class="n">stream1</span> <span class="o">=</span> <span class="n">ZMQStream</span><span class="p">(</span><span class="n">s1</span><span class="p">)</span>
<span class="n">s2</span> <span class="o">=</span> <span class="n">ctx</span><span class="o">.</span><span class="n">socket</span><span class="p">(</span><span class="n">zmq</span><span class="o">.</span><span class="n">REP</span><span class="p">)</span>
<span class="n">s2</span><span class="o">.</span><span class="n">bind</span><span class="p">(</span><span class="s">'tcp://localhost:54321'</span><span class="p">)</span>
<span class="n">stream2</span> <span class="o">=</span> <span class="n">ZMQStream</span><span class="p">(</span><span class="n">s2</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">echo</span><span class="p">(</span><span class="n">msg</span><span class="p">,</span> <span class="n">stream</span><span class="p">):</span>
<span class="n">stream</span><span class="o">.</span><span class="n">send_multipart</span><span class="p">(</span><span class="n">msg</span><span class="p">)</span>
<span class="n">stream1</span><span class="o">.</span><span class="n">on_recv_stream</span><span class="p">(</span><span class="n">echo</span><span class="p">)</span>
<span class="n">stream2</span><span class="o">.</span><span class="n">on_recv_stream</span><span class="p">(</span><span class="n">echo</span><span class="p">)</span>
<span class="n">ioloop</span><span class="o">.</span><span class="n">IOLoop</span><span class="o">.</span><span class="n">instance</span><span class="p">()</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
</pre></div>
</div>
</div>
<div class="section" id="flush">
<h3><tt class="xref py py-meth docutils literal"><span class="pre">flush()</span></tt><a class="headerlink" href="#flush" title="Permalink to this headline">¶</a></h3>
<p>Sometimes with an eventloop, there can be multiple events ready on a single iteration of
the loop. The <a class="reference internal" href="api/generated/zmq.eventloop.zmqstream.html#zmq.eventloop.zmqstream.ZMQStream.flush" title="zmq.eventloop.zmqstream.ZMQStream.flush"><tt class="xref py py-meth docutils literal"><span class="pre">flush()</span></tt></a> method allows developers to pull messages off of
the queue to enforce some priority over the event loop ordering. flush pulls any pending
events off of the queue. You can specify to flush only recv events, only send events, or
any events, and you can specify a limit for how many events to flush in order to prevent
starvation.</p>
</div>
</div>
<div class="section" id="gevent">
<span id="zmq-green"></span><h2>gevent<a class="headerlink" href="#gevent" title="Permalink to this headline">¶</a></h2>
<p>PyZMQ ≥ 2.2.0.1 ships with a <a class="reference external" href="http://www.gevent.org/">gevent</a> compatible API as <tt class="xref py py-mod docutils literal"><span class="pre">zmq.green</span></tt>.
To use it, simply:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="kn">import</span> <span class="nn">zmq.green</span> <span class="kn">as</span> <span class="nn">zmq</span>
</pre></div>
</div>
<p>Then write your code as normal.</p>
<p>Currently, Socket.send/recv methods and zmq.Poller are gevent-aware.
The tornado-based ZMQStream/IOLoop <em>are not</em> compatible with gevent.</p>
<div class="admonition warning">
<p class="first admonition-title">Warning</p>
<p class="last">There is a <a class="reference external" href="https://github.com/zeromq/pyzmq/issues/229">known issue</a> in gevent ≤ 1.0 or libevent,
which can cause zeromq socket events to be missed.
PyZMQ works around this by adding a timeout so it will not wait forever for gevent to notice events.
The only known solution for this is to use gevent ≥ 1.0, which is currently at 1.0b3,
and does not exhibit this behavior.</p>
</div>
<div class="admonition-see-also admonition seealso">
<p class="first admonition-title">See also</p>
<p class="last">zmq.green examples <a class="reference external" href="https://github.com/zeromq/pyzmq/tree/master/examples/gevent">on GitHub</a>.</p>
</div>
<p><tt class="xref py py-mod docutils literal"><span class="pre">zmq.green</span></tt> is simply <a class="reference external" href="https://github.com/traviscline/gevent_zeromq">gevent_zeromq</a>,
merged into the pyzmq project.</p>
</div>
</div>
</div>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="genindex.html" title="General Index"
>index</a></li>
<li class="right" >
<a href="py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="right" >
<a href="logging.html" title="Asynchronous Logging via PyZMQ"
>next</a> |</li>
<li class="right" >
<a href="devices.html" title="Devices in PyZMQ"
>previous</a> |</li>
<li><a href="index.html">home</a>| </li>
<li><a href="search.html">search</a>| </li>
<li><a href="api/index.html">API</a> »</li>
</ul>
</div>
<div class="footer">
© Copyright 2010-2011, Brian E. Granger & Min Ragan-Kelley.
ØMQ logo © iMatix Corportation, used under the Creative Commons Attribution-Share Alike 3.0 License.
Python logo ™ of the Python Software Foundation, used by Min RK with permission from the Foundation.
Created using <a href="http://sphinx.pocoo.org/">Sphinx</a> 1.0.7.
</div>
</body>
</html>
distrib
>
Mandriva
>
2010.2
>
i586
>
media
>
contrib-backports
>
by-pkgid
>
b9ca70a7484981df9126f0365edf0863
>
files
>
253
