<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /><title>Layered Library Design</title><link rel="stylesheet" href="styles.css" type="text/css" /><meta name="generator" content="DocBook XSL Stylesheets V1.73.2" /><link rel="start" href="index.html" title="Version Control with Subversion" /><link rel="up" href="svn.developer.html" title="Chapter 8. Embedding Subversion" /><link rel="prev" href="svn.developer.html" title="Chapter 8. Embedding Subversion" /><link rel="next" href="svn.developer.insidewc.html" title="Inside the Working Copy Administration Area" /></head><body><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">Layered Library Design</th></tr><tr><td width="20%" align="left"><a accesskey="p" href="svn.developer.html">Prev</a> </td><th width="60%" align="center">Chapter 8. Embedding Subversion</th><td width="20%" align="right"> <a accesskey="n" href="svn.developer.insidewc.html">Next</a></td></tr></table><hr /></div><div class="sect1" lang="en" xml:lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a id="svn.developer.layerlib"></a>Layered Library Design</h2></div></div></div><p>Each of Subversion's core libraries can be said to exist in
one of three main layers—the Repository Layer, the
Repository Access (RA) Layer, or the Client Layer (see <a class="xref" href="svn.intro.whatis.html#svn.intro.architecture.dia-1" title="Figure 1. Subversion's Architecture">Figure 1, “Subversion's Architecture”</a>). We will examine
these layers shortly, but first, let's briefly summarize
Subversion's various libraries. For the sake of consistency, we
will refer to the libraries by their extensionless Unix library
names (libsvn_fs, libsvn_wc, mod_dav_svn, etc.).</p><div class="variablelist"><dl><dt><span class="term">libsvn_client</span></dt><dd><p>Primary interface for client
programs</p></dd><dt><span class="term">libsvn_delta</span></dt><dd><p>Tree and byte-stream differencing
routines</p></dd><dt><span class="term">libsvn_diff</span></dt><dd><p>Contextual differencing and merging
routines</p></dd><dt><span class="term">libsvn_fs</span></dt><dd><p>Filesystem commons and module
loader</p></dd><dt><span class="term">libsvn_fs_base</span></dt><dd><p>The Berkeley DB filesystem
back-end</p></dd><dt><span class="term">libsvn_fs_fs</span></dt><dd><p>The native filesystem (FSFS)
back-end</p></dd><dt><span class="term">libsvn_ra</span></dt><dd><p>Repository Access commons and module
loader</p></dd><dt><span class="term">libsvn_ra_dav</span></dt><dd><p>The WebDAV Repository Access
module</p></dd><dt><span class="term">libsvn_ra_local</span></dt><dd><p>The local Repository Access
module</p></dd><dt><span class="term">libsvn_ra_serf</span></dt><dd><p>Another (experimental) WebDAV Repository
Access module</p></dd><dt><span class="term">libsvn_ra_svn</span></dt><dd><p>The custom protocol Repository Access
module</p></dd><dt><span class="term">libsvn_repos</span></dt><dd><p>Repository interface</p></dd><dt><span class="term">libsvn_subr</span></dt><dd><p>Miscellaneous helpful
subroutines</p></dd><dt><span class="term">libsvn_wc</span></dt><dd><p>The working copy management
library</p></dd><dt><span class="term">mod_authz_svn</span></dt><dd><p>Apache authorization module for Subversion
repositories access via WebDAV</p></dd><dt><span class="term">mod_dav_svn</span></dt><dd><p>Apache module for mapping WebDAV operations to
Subversion ones</p></dd></dl></div><p>The fact that the word “<span class="quote">miscellaneous</span>” only
appears once in the previous list is a good sign. The
Subversion development team is serious about making sure that
functionality lives in the right layer and libraries. Perhaps
the greatest advantage of the modular design is its lack of
complexity from a developer's point of view. As a developer,
you can quickly formulate that kind of “<span class="quote">big
picture</span>” that allows you to pinpoint the location of
certain pieces of functionality with relative ease.</p><p>Another benefit of modularity is the ability to replace a
given module with a whole new library that implements the same
API without affecting the rest of the code base. In some sense,
this happens within Subversion already. The libsvn_ra_dav,
libsvn_ra_local, libsvn_ra_serf, and libsvn_ra_svn libraries
each implement the same interface, all working as plugins to
libsvn_ra. And all four communicate with the Repository
Layer—libsvn_ra_local connects to the repository directly;
the other three do so over a network. The libsvn_fs_base and
libsvn_fs_fs libraries are another pair of libraries that
implement the same functionality in different ways—both
are plugins to the common libsvn_fs library.</p><p>The client itself also highlights the benefits of modularity
in the Subversion design. Subversion's libsvn_client library is
a one-stop shop for most of the functionality necessary for
designing a working Subversion client (see <a class="xref" href="svn.developer.layerlib.html#svn.developer.layerlib.client" title="Client Layer">the section called “Client Layer”</a>). So while the
Subversion distribution provides only the <span class="command"><strong>svn</strong></span>
command-line client program, there are several third-party
programs which provide various forms of graphical client UI.
These GUIs use the same APIs that the stock command-line client
does. This type of modularity has played a large role in the
proliferation of available Subversion clients and IDE
integrations and, by extension, to the tremendous adoption rate
of Subversion itself.</p><div class="sect2" lang="en" xml:lang="en"><div class="titlepage"><div><div><h3 class="title"><a id="svn.developer.layerlib.repos"></a>Repository Layer</h3></div></div></div><p>When referring to Subversion's Repository Layer, we're
generally talking about two basic concepts—the versioned
filesystem implementation (accessed via libsvn_fs, and
supported by its libsvn_fs_base and libsvn_fs_fs plugins), and
the repository logic that wraps it (as implemented in
libsvn_repos). These libraries provide the storage and
reporting mechanisms for the various revisions of your
version-controlled data. This layer is connected to the
Client Layer via the Repository Access Layer, and is, from the
perspective of the Subversion user, the stuff at the
“<span class="quote">other end of the line.</span>”</p><p>The Subversion Filesystem is not a kernel-level filesystem
that one would install in an operating system (like the Linux
ext2 or NTFS), but a virtual filesystem. Rather than storing
“<span class="quote">files</span>” and “<span class="quote">directories</span>” as real
files and directories (as in, the kind you can navigate
through using your favorite shell program), it uses one of two
available abstract storage backends—either a Berkeley DB
database environment, or a flat-file representation. (To
learn more about the two repository back-ends, see <a class="xref" href="svn.reposadmin.planning.html#svn.reposadmin.basics.backends" title="Choosing a Data Store">the section called “Choosing a Data Store”</a>.) There has even
been considerable interest by the development community in
giving future releases of Subversion the ability to use other
back-end database systems, perhaps through a mechanism such as
Open Database Connectivity (ODBC). In fact, Google did
something similar to this before launching the Google Code
Project Hosting service: they announced in mid-2006 that members
of its Open Source team had written a new proprietary
Subversion filesystem plugin which used their ultra-scalable
Bigtable database for its storage.</p><p>The filesystem API exported by libsvn_fs contains the
kinds of functionality you would expect from any other
filesystem API—you can create and remove files and
directories, copy and move them around, modify file contents,
and so on. It also has features that are not quite as common,
such as the ability to add, modify, and remove metadata
(“<span class="quote">properties</span>”) on each file or directory.
Furthermore, the Subversion Filesystem is a versioning
filesystem, which means that as you make changes to your
directory tree, Subversion remembers what your tree looked
like before those changes. And before the previous changes.
And the previous ones. And so on, all the way back through
versioning time to (and just beyond) the moment you first
started adding things to the filesystem.</p><p>All the modifications you make to your tree are done
within the context of a Subversion commit transaction. The
following is a simplified general routine for modifying your
filesystem:</p><div class="orderedlist"><ol type="1"><li><p>Begin a Subversion commit transaction.</p></li><li><p>Make your changes (adds, deletes, property
modifications, etc.).</p></li><li><p>Commit your transaction.</p></li></ol></div><p>Once you have committed your transaction, your filesystem
modifications are permanently stored as historical artifacts.
Each of these cycles generates a single new revision of your
tree, and each revision is forever accessible as an immutable
snapshot of “<span class="quote">the way things were.</span>”</p><div class="sidebar"><p class="title"><b>The Transaction Distraction</b></p><p>The notion of a Subversion transaction can become easily
confused with the transaction support provided by the
underlying database itself, especially given the former's
close proximity to the Berkeley DB database code in
libsvn_fs_base. Both types of transaction exist to provide
atomicity and isolation. In other words, transactions give
you the ability to perform a set of actions in an
all-or-nothing fashion—either all the actions in the
set complete with success, or they all get treated as if
<span class="emphasis"><em>none</em></span> of them ever happened—and in
a way that does not interfere with other processes acting on
the data.</p><p>Database transactions generally encompass small
operations related specifically to the modification of data
in the database itself (such as changing the contents of a
table row). Subversion transactions are larger in scope,
encompassing higher-level operations like making
modifications to a set of files and directories which are
intended to be stored as the next revision of the filesystem
tree. If that isn't confusing enough, consider the fact
that Subversion uses a database transaction during the
creation of a Subversion transaction (so that if the
creation of Subversion transaction fails, the database will
look as if we had never attempted that creation in the first
place)!</p><p>Fortunately for users of the filesystem API, the
transaction support provided by the database system itself
is hidden almost entirely from view (as should be expected
from a properly modularized library scheme). It is only
when you start digging into the implementation of the
filesystem itself that such things become visible (or
interesting).</p></div><p>Most of the functionality provided by the filesystem
interface deals with actions that occur on individual
filesystem paths. That is, from outside of the filesystem, the
primary mechanism for describing and accessing the individual
revisions of files and directories comes through the use of
path strings like <code class="filename">/foo/bar</code>, just as if
you were addressing files and directories through your
favorite shell program. You add new files and directories by
passing their paths-to-be to the right API functions. You
query for information about them by the same mechanism.</p><p>Unlike most filesystems, though, a path alone is not
enough information to identify a file or directory in
Subversion. Think of a directory tree as a two-dimensional
system, where a node's siblings represent a sort of
left-and-right motion, and descending into subdirectories a
downward motion. <a class="xref" href="svn.developer.layerlib.html#svn.developer.layerlib.repos.dia-1" title="Figure 8.1. Files and directories in two dimensions">Figure 8.1, “Files and directories in two dimensions”</a> shows a typical
representation of a tree as exactly that.</p><div class="figure"><a id="svn.developer.layerlib.repos.dia-1"></a><p class="title"><b>Figure 8.1. Files and directories in two dimensions</b></p><div class="figure-contents"><div><img src="images/ch08dia1.png" alt="Files and directories in two dimensions" /></div></div></div><br class="figure-break" /><p>The difference here is that the Subversion filesystem has a
nifty third dimension that most filesystems do not
have—Time!
<sup>[<a id="id409752" href="#ftn.id409752" class="footnote">53</a>]</sup>
In the filesystem interface, nearly every function that has a
<em class="parameter"><code>path</code></em> argument also expects a
<em class="parameter"><code>root</code></em> argument. This
<span class="structname">svn_fs_root_t</span> argument describes
either a revision or a Subversion transaction (which is simply
a revision-in-the-making), and provides that third-dimensional
context needed to understand the difference between
<code class="filename">/foo/bar</code> in revision 32, and the same
path as it exists in revision 98. <a class="xref" href="svn.developer.layerlib.html#svn.developer.layerlib.repos.dia-2" title="Figure 8.2. Versioning time—the third dimension!">Figure 8.2, “Versioning time—the third dimension!”</a> shows revision
history as an added dimension to the Subversion filesystem
universe.</p><div class="figure"><a id="svn.developer.layerlib.repos.dia-2"></a><p class="title"><b>Figure 8.2. Versioning time—the third dimension!</b></p><div class="figure-contents"><div><img src="images/ch08dia2.png" alt="Versioning time—the third dimension!" /></div></div></div><br class="figure-break" /><p>As we mentioned earlier, the libsvn_fs API looks and feels
like any other filesystem, except that it has this wonderful
versioning capability. It was designed to be usable by any
program interested in a versioning filesystem. Not
coincidentally, Subversion itself is interested in that
functionality. But while the filesystem API should be
sufficient for basic file and directory versioning support,
Subversion wants more—and that is where libsvn_repos
comes in.</p><p>The Subversion repository library (libsvn_repos) sits
(logically speaking) atop the libsvn_fs API, providing
additional functionality beyond that of the underlying
versioned filesystem logic. It does not completely wrap each
and every filesystem function—only certain major steps
in the general cycle of filesystem activity are wrapped by the
repository interface. Some of these include the creation and
commit of Subversion transactions, and the modification of
revision properties. These particular events are wrapped by
the repository layer because they have hooks associated with
them. A repository hook system is not strictly related to
implementing a versioning filesystem, so it lives in the
repository wrapper library.</p><p>The hooks mechanism is but one of the reasons for the
abstraction of a separate repository library from the rest of
the filesystem code. The libsvn_repos API provides several
other important utilities to Subversion. These include the
abilities to:</p><div class="itemizedlist"><ul type="disc"><li><p>create, open, destroy, and perform recovery steps on a
Subversion repository and the filesystem included in that
repository.</p></li><li><p>describe the differences between two filesystem
trees.</p></li><li><p>query for the commit log messages associated with all
(or some) of the revisions in which a set of files was
modified in the filesystem.</p></li><li><p>generate a human-readable “<span class="quote">dump</span>” of the
filesystem, a complete representation of the revisions in
the filesystem.</p></li><li><p>parse that dump format, loading the dumped revisions
into a different Subversion repository.</p></li></ul></div><p>As Subversion continues to evolve, the repository library
will grow with the filesystem library to offer increased
functionality and configurable option support.</p></div><div class="sect2" lang="en" xml:lang="en"><div class="titlepage"><div><div><h3 class="title"><a id="svn.developer.layerlib.ra"></a>Repository Access Layer</h3></div></div></div><p>If the Subversion Repository Layer is at “<span class="quote">the other
end of the line</span>”, the Repository Access (RA) Layer is
the line itself. Charged with marshaling data between the
client libraries and the repository, this layer includes the
libsvn_ra module loader library, the RA modules themselves
(which currently includes libsvn_ra_dav, libsvn_ra_local,
libsvn_ra_serf, and libsvn_ra_svn), and any additional
libraries needed by one or more of those RA modules (such as
the mod_dav_svn Apache module or libsvn_ra_svn's server,
<span class="command"><strong>svnserve</strong></span>).</p><p>Since Subversion uses URLs to identify its repository
resources, the protocol portion of the URL scheme (usually
<code class="literal">file://</code>, <code class="literal">http://</code>,
<code class="literal">https://</code>, <code class="literal">svn://</code>, or
<code class="literal">svn+ssh://</code>) is used to determine which RA
module will handle the communications. Each module registers
a list of the protocols it knows how to “<span class="quote">speak</span>”
so that the RA loader can, at runtime, determine which module
to use for the task at hand. You can determine which RA
modules are available to the Subversion command-line client,
and what protocols they claim to support, by running
<span class="command"><strong>svn --version</strong></span>:</p><pre class="screen">
$ svn --version
svn, version 1.4.3 (r23084)
compiled Jan 18 2007, 07:47:40
Copyright (C) 2000-2006 CollabNet.
Subversion is open source software, see http://subversion.tigris.org/
This product includes software developed by CollabNet (http://www.Collab.Net/).
The following repository access (RA) modules are available:
* ra_dav : Module for accessing a repository via WebDAV (DeltaV) protocol.
- handles 'http' scheme
- handles 'https' scheme
* ra_svn : Module for accessing a repository using the svn network protocol.
- handles 'svn' scheme
* ra_local : Module for accessing a repository on local disk.
- handles 'file' scheme
$
</pre><p>The public API exported by the RA Layer contains
functionality necessary for sending and receiving versioned
data to and from the repository. And each of the available RA
plugins is able to perform that task using a specific
protocol—libsvn_ra_dav speaks HTTP/WebDAV (optionally
using SSL encryption) with an Apache HTTP Server that is
running the mod_dav_svn Subversion server module;
libsvn_ra_svn speaks a custom network protocol with the
<span class="command"><strong>svnserve</strong></span> program; and so on.</p><p>And for those who wish to access a Subversion repository
using still another protocol, that is precisely why the
Repository Access Layer is modularized! Developers can simply
write a new library that implements the RA interface on one
side and communicates with the repository on the other. Your
new library can use existing network protocols, or you can
invent your own. You could use inter-process communication
(IPC) calls, or—let's get crazy, shall we?—you
could even implement an email-based protocol. Subversion
supplies the APIs; you supply the creativity.</p></div><div class="sect2" lang="en" xml:lang="en"><div class="titlepage"><div><div><h3 class="title"><a id="svn.developer.layerlib.client"></a>Client Layer</h3></div></div></div><p>On the client side, the Subversion working copy is where
all the action takes place. The bulk of functionality
implemented by the client-side libraries exists for the sole
purpose of managing working copies—directories full of
files and other subdirectories which serve as a sort of local,
editable “<span class="quote">reflection</span>” of one or more repository
locations—and propagating changes to and from the
Repository Access layer.</p><p>Subversion's working copy library, libsvn_wc, is directly
responsible for managing the data in the working copies. To
accomplish this, the library stores administrative information
about each working copy directory within a special
subdirectory. This subdirectory, named
<code class="filename">.svn</code>, is present in each working copy
directory and contains various other files and directories
which record state and provide a private workspace for
administrative action. For those familiar with CVS, this
<code class="filename">.svn</code> subdirectory is similar in purpose
to the <code class="filename">CVS</code> administrative directories
found in CVS working copies. For more information about the
<code class="filename">.svn</code> administrative area, see <a class="xref" href="svn.developer.insidewc.html" title="Inside the Working Copy Administration Area">the section called “Inside the Working Copy Administration Area”</a>in this chapter.</p><p>The Subversion client library, libsvn_client, has the
broadest responsibility; its job is to mingle the
functionality of the working copy library with that of the
Repository Access Layer, and then to provide the highest-level
API to any application that wishes to perform general revision
control actions. For example, the function
<code class="function">svn_client_checkout()</code> takes a URL as an
argument. It passes this URL to the RA layer and opens an
authenticated session with a particular repository. It then
asks the repository for a certain tree, and sends this tree
into the working copy library, which then writes a full
working copy to disk (<code class="filename">.svn</code> directories
and all).</p><p>The client library is designed to be used by any
application. While the Subversion source code includes a
standard command-line client, it should be very easy to write
any number of GUI clients on top of the client library. New
GUIs (or any new client, really) for Subversion need not be
clunky wrappers around the included command-line
client—they have full access via the libsvn_client API
to same functionality, data, and callback mechanisms that the
command-line client uses. In fact, the Subversion source code
tree contains a small C program (which can be found at
<code class="filename">tools/examples/minimal_client.c</code> that
exemplifies how to wield the Subversion API to create a simple
client program</p><div class="sidebar"><p class="title"><b>Binding Directly—A Word About Correctness</b></p><p>Why should your GUI program bind directly with a
libsvn_client instead of acting as a wrapper around a
command-line program? Besides simply being more efficient,
it can be more correct as well. A
command-line program (like the one supplied with Subversion)
that binds to the client library needs to effectively
translate feedback and requested data bits from C types to
some form of human-readable output. This type of
translation can be lossy. That is, the program may not
display all of the information harvested from the API, or
may combine bits of information for compact representation.</p><p>If you wrap such a command-line program with yet another
program, the second program has access only to
already-interpreted (and as we mentioned, likely incomplete)
information, which it must <span class="emphasis"><em>again</em></span>
translate into <span class="emphasis"><em>its</em></span> representation
format. With each layer of wrapping, the integrity of the
original data is potentially tainted more and more, much
like the result of making a copy of a copy (of a copy …)
of a favorite audio or video cassette.</p><p>But the most compelling argument for binding directly to
the APIs instead of wrapping other programs is that the
Subversion project makes compatibility promises regarding
its APIs. Across minor versions of those APIs (such as
between 1.3 and 1.4), no function's prototype will change.
In other words, you aren't forced to update your program's
source code simply because you've upgraded to a new version
of Subversion. Certain functions might be deprecated, but
they still work, and this gives you a buffer of time to
eventually embrace the newer APIs. These kinds of
compatibility promises do not exist for Subversion
command-line program output, which is subject to change from
release to release.</p></div></div><div class="footnotes"><br /><hr width="100" align="left" /><div class="footnote"><p><sup>[<a id="ftn.id409752" href="#id409752" class="para">53</a>] </sup>We understand that this may come as a shock to sci-fi
fans who have long been under the impression that Time was
actually the <span class="emphasis"><em>fourth</em></span> dimension, and we
apologize for any emotional trauma induced by our
assertion of a different theory.</p></div></div></div><div class="navfooter"><hr /><table width="100%" summary="Navigation footer"><tr><td width="40%" align="left"><a accesskey="p" href="svn.developer.html">Prev</a> </td><td width="20%" align="center"><a accesskey="u" href="svn.developer.html">Up</a></td><td width="40%" align="right"> <a accesskey="n" href="svn.developer.insidewc.html">Next</a></td></tr><tr><td width="40%" align="left" valign="top">Chapter 8. Embedding Subversion </td><td width="20%" align="center"><a accesskey="h" href="index.html">Home</a></td><td width="40%" align="right" valign="top"> Inside the Working Copy Administration Area</td></tr></table></div></body></html>
