<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN"> <!--Converted with LaTeX2HTML 2008 (1.71) original version by: Nikos Drakos, CBLU, University of Leeds * revised and updated by: Marcus Hennecke, Ross Moore, Herb Swan * with significant contributions from: Jens Lippmann, Marek Rouchal, Martin Wilck and others --> <HTML> <HEAD> <TITLE>Python Scripting</TITLE> <META NAME="description" CONTENT="Python Scripting"> <META NAME="keywords" CONTENT="misc"> <META NAME="resource-type" CONTENT="document"> <META NAME="distribution" CONTENT="global"> <META NAME="Generator" CONTENT="LaTeX2HTML v2008"> <META HTTP-EQUIV="Content-Style-Type" CONTENT="text/css"> <LINK REL="STYLESHEET" HREF="misc.css"> <LINK REL="next" HREF="Variable_Expansion.html"> <LINK REL="previous" HREF="Contents.html"> <LINK REL="up" HREF="Bacula_Miscellaneous_Guide.html"> <LINK REL="next" HREF="Variable_Expansion.html"> </HEAD> <BODY > <!--Navigation Panel--> <A NAME="tex2html154" HREF="Variable_Expansion.html"> <IMG WIDTH="37" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="next" SRC="next.png"></A> <A NAME="tex2html148" HREF="Bacula_Miscellaneous_Guide.html"> <IMG WIDTH="26" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="up" SRC="up.png"></A> <A NAME="tex2html142" HREF="Contents.html"> <IMG WIDTH="63" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="previous" SRC="prev.png"></A> <A NAME="tex2html150" HREF="Contents.html"> <IMG WIDTH="65" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="contents" SRC="contents.png"></A> <A NAME="tex2html152" HREF="Index.html"> <IMG WIDTH="43" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="index" SRC="index.png"></A> <BR> <B> Next:</B> <A NAME="tex2html155" HREF="Variable_Expansion.html">Variable Expansion</A> <B> Up:</B> <A NAME="tex2html149" HREF="Bacula_Miscellaneous_Guide.html">Bacula Miscellaneous Guide</A> <B> Previous:</B> <A NAME="tex2html143" HREF="Contents.html">Contents</A> <B> <A NAME="tex2html151" HREF="Contents.html">Contents</A></B> <B> <A NAME="tex2html153" HREF="Index.html">Index</A></B> <BR> <BR> <!--End of Navigation Panel--> <!--Table of Child-Links--> <A NAME="CHILD_LINKS"><STRONG>Subsections</STRONG></A> <UL> <LI><A NAME="tex2html156" HREF="Python_Scripting.html#SECTION00210000000000000000">Python Configuration</A> <LI><A NAME="tex2html157" HREF="Python_Scripting.html#SECTION00220000000000000000">Bacula Events</A> <LI><A NAME="tex2html158" HREF="Python_Scripting.html#SECTION00230000000000000000">Python Objects</A> <LI><A NAME="tex2html159" HREF="Python_Scripting.html#SECTION00240000000000000000">Python Console Command</A> <LI><A NAME="tex2html160" HREF="Python_Scripting.html#SECTION00250000000000000000">Debugging Python Scripts</A> <LI><A NAME="tex2html161" HREF="Python_Scripting.html#SECTION00260000000000000000">Python Example</A> </UL> <!--End of Table of Child-Links--> <HR> <H1><A NAME="SECTION00200000000000000000"></A> <A NAME="PythonChapter"></A> <BR> Python Scripting </H1> <A NAME="85"></A> <A NAME="86"></A> <P> You may be asking what Python is and why a scripting language is needed in Bacula. The answer to the first question is that Python is an Object Oriented scripting language with features similar to those found in Perl, but the syntax of the language is much cleaner and simpler. The answer to why have scripting in Bacula is to give the user more control over the whole backup process. Probably the simplest example is when Bacula needs a new Volume name, with a scripting language such as Python, you can generate any name you want, based on the current state of Bacula. <P> <H1><A NAME="SECTION00210000000000000000"> Python Configuration</A> </H1> <A NAME="88"></A> <A NAME="89"></A> <P> Python must be enabled during the configuration process by adding a <code>--</code>with-python, and possibly specifying an alternate directory if your Python is not installed in a standard system location. If you are using RPMs you will need the python-devel package installed. <P> When Python is configured, it becomes an integral part of Bacula and runs in Bacula's address space, so even though it is an interpreted language, it is very efficient. <P> When the Director starts, it looks to see if you have a <B>Scripts Directory</B> Directive defined (normal default <B>/etc/bacula/scripts</B>, if so, it looks in that directory for a file named <B>DirStartUp.py</B>. If it is found, Bacula will pass this file to Python for execution. The <B>Scripts Directory</B> is a new directive that you add to the Director resource of your bacula-dir.conf file. <P> Note: Bacula does not install Python scripts by default because these scripts are for you to program. This means that with a default installation with Python enabled, Bacula will print the following error message: <P> <PRE> 09-Jun 15:14 bacula-dir: ERROR in pythonlib.c:131 Could not import Python script /etc/bacula/scripts/DirStartUp. Python disabled. </PRE> <P> The source code directory <B>examples/python</B> contains sample scripts for DirStartUp.py, SDStartUp.py, and FDStartUp.py that you might want to use as a starting point. Normally, your scripts directory (at least where you store the Python scripts) should be writable by Bacula, because Python will attempt to write a compiled version of the scripts (e.g. DirStartUp.pyc) back to that directory. <P> When starting with the sample scripts, you can delete any part that you will not need, but you should keep all the Bacula Event and Job Event definitions. If you do not want a particular event, simply replace the existing code with a <B>noop = 1</B>. <P> <H1><A NAME="SECTION00220000000000000000"> Bacula Events</A> </H1> <A NAME="99"></A> <A NAME="100"></A> A Bacula event is a point in the Bacula code where Bacula will call a subroutine (actually a method) that you have defined in the Python StartUp script. Events correspond to some significant event such as a Job Start, a Job End, Bacula needs a new Volume Name, ... When your script is called, it will have access to all the Bacula variables specific to the Job (attributes of the Job Object), and it can even call some of the Job methods (subroutines) or set new values in the Job attributes, such as the Priority. You will see below how the events are used. <P> <H1><A NAME="SECTION00230000000000000000"> Python Objects</A> </H1> <A NAME="102"></A> <A NAME="103"></A> <P> There are four Python objects that you will need to work with: <DL> <DT><STRONG>The Bacula Object</STRONG></DT> <DD>The Bacula object is created by the Bacula daemon (the Director in the present case) when the daemon starts. It is available to the Python startup script, <B>DirStartup.py</B>, by importing the Bacula definitions with <B>import bacula</B>. The methods available with this object are described below. <P> </DD> <DT><STRONG>The Bacula Events Class</STRONG></DT> <DD>You create this class in the startup script, and you pass it to the Bacula Object's <B>set_events</B> method. The purpose of the Bacula Events Class is to define what global or daemon events you want to monitor. When one of those events occurs, your Bacula Events Class will be called at the method corresponding to the event. There are currently three events, JobStart, JobEnd, and Exit, which are described in detail below. <P> </DD> <DT><STRONG>The Job Object</STRONG></DT> <DD>When a Job starts, and assuming you have defined a JobStart method in your Bacula Events Class, Bacula will create a Job Object. This object will be passed to the JobStart event. The Job Object has a has good number of read-only members or attributes providing many details of the Job, and it also has a number of writable attributes that allow you to pass information into the Job. These attributes are described below. <P> </DD> <DT><STRONG>The Job Events Class</STRONG></DT> <DD>You create this class in the JobStart method of your Bacula Events class, and it allows you to define which of the possible Job Object events you want to see. You must pass an instance of your Job Events class to the Job Object set_events() method. Normally, you will probably only have one Job Events Class, which will be instantiated for each Job. However, if you wish to see different events in different Jobs, you may have as many Job Events classes as you wish. </DD> </DL> <P> The first thing the startup script must do is to define what global Bacula events (daemon events), it wants to see. This is done by creating a Bacula Events class, instantiating it, then passing it to the <B>set_events</B> method. There are three possible events. <P> <DL> <DT><STRONG>JobStart</STRONG></DT> <DD><A NAME="111"></A> This Python method, if defined, will be called each time a Job is started. The method is passed the class instantiation object as the first argument, and the Bacula Job object as the second argument. The Bacula Job object has several built-in methods, and you can define which ones you want called. If you do not define this method, you will not be able to interact with Bacula jobs. <P> </DD> <DT><STRONG>JobEnd</STRONG></DT> <DD>This Python method, if defined, will be called each time a Job terminates. The method is passed the class instantiation object as the first argument, and the Bacula Job object as the second argument. <P> </DD> <DT><STRONG>Exit</STRONG></DT> <DD>This Python method, if defined, will be called when the Director terminates. The method is passed the class instantiation object as the first argument. </DD> </DL> <P> Access to the Bacula variables and methods is done with: <P> import bacula <P> The following are the read-only attributes provided by the bacula object. <DL> <DT><STRONG>Name</STRONG></DT> <DD> </DD> <DT><STRONG>ConfigFile</STRONG></DT> <DD> </DD> <DT><STRONG>WorkingDir</STRONG></DT> <DD> </DD> <DT><STRONG>Version</STRONG></DT> <DD>string consisting of "Version Build-date" </DD> </DL> <P> A simple definition of the Bacula Events Class might be the following: <P> <PRE> import sys, bacula class BaculaEvents: def JobStart(self, job): ... </PRE> <P> Then to instantiate the class and pass it to Bacula, you would do: <P> <PRE> bacula.set_events(BaculaEvents()) # register Bacula Events wanted </PRE> <P> And at that point, each time a Job is started, your BaculaEvents JobStart method will be called. <P> Now to actually do anything with a Job, you must define which Job events you want to see, and this is done by defining a JobEvents class containing the methods you want called. Each method name corresponds to one of the Job Events that Bacula will generate. <P> A simple Job Events class might look like the following: <P> <PRE> class JobEvents: def NewVolume(self, job): ... </PRE> <P> Here, your JobEvents class method NewVolume will be called each time the Job needs a new Volume name. To actually register the events defined in your class with the Job, you must instantiate the JobEvents class and set it in the Job <B>set_events</B> variable. Note, this is a bit different from how you registered the Bacula events. The registration process must be done in the Bacula JobStart event (your method). So, you would modify Bacula Events (not the Job events) as follows: <P> <PRE> import sys, bacula class BaculaEvents: def JobStart(self, job): events = JobEvents() # create instance of Job class job.set_events(events) # register Job events desired ... </PRE> <P> When a job event is triggered, the appropriate event definition is called in the JobEvents class. This is the means by which your Python script or code gets control. Once it has control, it may read job attributes, or set them. See below for a list of read-only attributes, and those that are writable. <P> In addition, the Bacula <B>job</B> object in the Director has a number of methods (subroutines) that can be called. They are: <DL> <DT><STRONG>set_events</STRONG></DT> <DD>The set_events method takes a single argument, which is the instantiation of the Job Events class that contains the methods that you want called. The method names that will be called must correspond to the Bacula defined events. You may define additional methods but Bacula will not use them. </DD> <DT><STRONG>run</STRONG></DT> <DD>The run method takes a single string argument, which is the run command (same as in the Console) that you want to submit to start a new Job. The value returned by the run method is the JobId of the job that started, or -1 if there was an error. </DD> <DT><STRONG>write</STRONG></DT> <DD>The write method is used to be able to send print output to the Job Report. This will be described later. </DD> <DT><STRONG>cancel</STRONG></DT> <DD>The cancel method takes a single integer argument, which is a JobId. If JobId is found, it will be canceled. </DD> <DT><STRONG>DoesVolumeExist</STRONG></DT> <DD>The DoesVolumeExist method takes a single string argument, which is the Volume name, and returns 1 if the volume exists in the Catalog and 0 if the volume does not exist. </DD> </DL> <P> The following attributes are read/write within the Director for the <B>job</B> object. <P> <DL> <DT><STRONG>Priority</STRONG></DT> <DD>Read or set the Job priority. Note, that setting a Job Priority is effective only before the Job actually starts. </DD> <DT><STRONG>Level</STRONG></DT> <DD>This attribute contains a string representing the Job level, e.g. Full, Differential, Incremental, ... if read. The level can also be set. </DD> </DL> <P> The following read-only attributes are available within the Director for the <B>job</B> object. <P> <DL> <DT><STRONG>Type</STRONG></DT> <DD>This attribute contains a string representing the Job type, e.g. Backup, Restore, Verify, ... </DD> <DT><STRONG>JobId</STRONG></DT> <DD>This attribute contains an integer representing the JobId. </DD> <DT><STRONG>Client</STRONG></DT> <DD>This attribute contains a string with the name of the Client for this job. </DD> <DT><STRONG>NumVols</STRONG></DT> <DD>This attribute contains an integer with the number of Volumes in the Pool being used by the Job. </DD> <DT><STRONG>Pool</STRONG></DT> <DD>This attribute contains a string with the name of the Pool being used by the Job. </DD> <DT><STRONG>Storage</STRONG></DT> <DD>This attribute contains a string with the name of the Storage resource being used by the Job. </DD> <DT><STRONG>Catalog</STRONG></DT> <DD>This attribute contains a string with the name of the Catalog resource being used by the Job. </DD> <DT><STRONG>MediaType</STRONG></DT> <DD>This attribute contains a string with the name of the Media Type associated with the Storage resource being used by the Job. </DD> <DT><STRONG>Job</STRONG></DT> <DD>This attribute contains a string containing the name of the Job resource used by this job (not unique). </DD> <DT><STRONG>JobName</STRONG></DT> <DD>This attribute contains a string representing the full unique Job name. </DD> <DT><STRONG>JobStatus</STRONG></DT> <DD>This attribute contains a single character string representing the current Job status. The status may change during execution of the job. It may take on the following values: <DL> <DT><STRONG>C</STRONG></DT> <DD>Created, not yet running </DD> <DT><STRONG>R</STRONG></DT> <DD>Running </DD> <DT><STRONG>B</STRONG></DT> <DD>Blocked </DD> <DT><STRONG>T</STRONG></DT> <DD>Completed successfully </DD> <DT><STRONG>E</STRONG></DT> <DD>Terminated with errors </DD> <DT><STRONG>e</STRONG></DT> <DD>Non-fatal error </DD> <DT><STRONG>f</STRONG></DT> <DD>Fatal error </DD> <DT><STRONG>D</STRONG></DT> <DD>Verify found differences </DD> <DT><STRONG>A</STRONG></DT> <DD>Canceled by user </DD> <DT><STRONG>F</STRONG></DT> <DD>Waiting for Client </DD> <DT><STRONG>S</STRONG></DT> <DD>Waiting for Storage daemon </DD> <DT><STRONG>m</STRONG></DT> <DD>Waiting for new media </DD> <DT><STRONG>M</STRONG></DT> <DD>Waiting for media mount </DD> <DT><STRONG>s</STRONG></DT> <DD>Waiting for storage resource </DD> <DT><STRONG>j</STRONG></DT> <DD>Waiting for job resource </DD> <DT><STRONG>c</STRONG></DT> <DD>Waiting for client resource </DD> <DT><STRONG>d</STRONG></DT> <DD>Waiting on maximum jobs </DD> <DT><STRONG>t</STRONG></DT> <DD>Waiting on start time </DD> <DT><STRONG>p</STRONG></DT> <DD>Waiting on higher priority jobs </DD> </DL> <P> </DD> <DT><STRONG>Priority</STRONG></DT> <DD>This attribute contains an integer with the priority assigned to the job. </DD> <DT><STRONG>CatalogRes</STRONG></DT> <DD>tuple consisting of (DBName, Address, User, Password, Socket, Port, Database Vendor) taken from the Catalog resource for the Job with the exception of Database Vendor, which is one of the following: MySQL, PostgreSQL, SQLite, Internal, depending on what database you configured. </DD> <DT><STRONG>VolumeName</STRONG></DT> <DD>After a Volume has been purged, this attribute will contain the name of that Volume. At other times, this value may have no meaning. </DD> </DL> <P> The following write-only attributes are available within the Director: <P> <DL> <DT><STRONG>JobReport</STRONG></DT> <DD>Send line to the Job Report. </DD> <DT><STRONG>VolumeName</STRONG></DT> <DD>Set a new Volume name. Valid only during the NewVolume event. </DD> </DL> <P> <H1><A NAME="SECTION00240000000000000000"> Python Console Command</A> </H1> <A NAME="138"></A> <A NAME="139"></A> <P> There is a new Console command named <B>python</B>. It takes a single argument <B>restart</B>. Example: <PRE> python restart </PRE> <P> This command restarts the Python interpreter in the Director. This can be useful when you are modifying the DirStartUp script, because normally Python will cache it, and thus the script will be read one time. <P> <H1><A NAME="SECTION00250000000000000000"> Debugging Python Scripts</A> </H1> <A NAME="145"></A> In general, you debug your Python scripts by using print statements. You can also develop your script or important parts of it as a separate file using the Python interpreter to run it. Once you have it working correctly, you can then call the script from within the Bacula Python script (DirStartUp.py). <P> If you are having problems loading DirStartUp.py, you will probably not get any error messages because Bacula can only print Python error messages after the Python interpreter is started. However, you may be able to see the error messages by starting Bacula in a shell window with the <B>-d1</B> option on the command line. That should cause the Python error messages to be printed in the shell window. <P> If you are getting error messages such as the following when loading DirStartUp.py: <P> <PRE> Traceback (most recent call last): File "/etc/bacula/scripts/DirStartUp.py", line 6, in ? import time, sys, bacula ImportError: /usr/lib/python2.3/lib-dynload/timemodule.so: undefined symbol: PyInt_FromLong bacula-dir: pythonlib.c:134 Python Import error. </PRE> <P> It is because the DirStartUp script is calling a dynamically loaded module (timemodule.so in the above case) that then tries to use Python functions exported from the Python interpreter (in this case PyInt_FromLong). The way Bacula is currently linked with Python does not permit this. The solution to the problem is to put such functions (in this case the import of time into a separate Python script, which will do your calculations and return the values you want. Then call (not import) this script from the Bacula DirStartUp.py script, and it all should work as you expect. <P> <H1><A NAME="SECTION00260000000000000000"> Python Example</A> </H1> <A NAME="150"></A> <A NAME="151"></A> <P> An example script for the Director startup file is provided in <B>examples/python/DirStartup.py</B> as follows: <P> <PRE> # # Bacula Python interface script for the Director # # You must import both sys and bacula import sys, bacula # This is the list of Bacula daemon events that you # can receive. class BaculaEvents(object): def __init__(self): # Called here when a new Bacula Events class is # is created. Normally not used noop = 1 def JobStart(self, job): """ Called here when a new job is started. If you want to do anything with the Job, you must register events you want to receive. """ events = JobEvents() # create instance of Job class events.job = job # save Bacula's job pointer job.set_events(events) # register events desired sys.stderr = events # send error output to Bacula sys.stdout = events # send stdout to Bacula jobid = job.JobId; client = job.Client numvols = job.NumVols job.JobReport="Python Dir JobStart: JobId=%d Client=%s NumVols=%d\n" % (jobid,client,numvols) # Bacula Job is going to terminate def JobEnd(self, job): jobid = job.JobId client = job.Client job.JobReport="Python Dir JobEnd output: JobId=%d Client=%s.\n" % (jobid, client) # Called here when the Bacula daemon is going to exit def Exit(self, job): print "Daemon exiting." bacula.set_events(BaculaEvents()) # register daemon events desired """ These are the Job events that you can receive. """ class JobEvents(object): def __init__(self): # Called here when you instantiate the Job. Not # normally used noop = 1 def JobInit(self, job): # Called when the job is first scheduled noop = 1 def JobRun(self, job): # Called just before running the job after initializing # This is the point to change most Job parameters. # It is equivalent to the JobRunBefore point. noop = 1 def NewVolume(self, job): # Called when Bacula wants a new Volume name. The Volume # name returned, if any, must be stored in job.VolumeName jobid = job.JobId client = job.Client numvol = job.NumVols; print job.CatalogRes job.JobReport = "JobId=%d Client=%s NumVols=%d" % (jobid, client, numvol) job.JobReport="Python before New Volume set for Job.\n" Vol = "TestA-%d" % numvol job.JobReport = "Exists=%d TestA-%d" % (job.DoesVolumeExist(Vol), numvol) job.VolumeName="TestA-%d" % numvol job.JobReport="Python after New Volume set for Job.\n" return 1 def VolumePurged(self, job): # Called when a Volume is purged. The Volume name can be referenced # with job.VolumeName noop = 1 </PRE> <P> <HR> <!--Navigation Panel--> <A NAME="tex2html154" HREF="Variable_Expansion.html"> <IMG WIDTH="37" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="next" SRC="next.png"></A> <A NAME="tex2html148" HREF="Bacula_Miscellaneous_Guide.html"> <IMG WIDTH="26" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="up" SRC="up.png"></A> <A NAME="tex2html142" HREF="Contents.html"> <IMG WIDTH="63" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="previous" SRC="prev.png"></A> <A NAME="tex2html150" HREF="Contents.html"> <IMG WIDTH="65" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="contents" SRC="contents.png"></A> <A NAME="tex2html152" HREF="Index.html"> <IMG WIDTH="43" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="index" SRC="index.png"></A> <BR> <B> Next:</B> <A NAME="tex2html155" HREF="Variable_Expansion.html">Variable Expansion</A> <B> Up:</B> <A NAME="tex2html149" HREF="Bacula_Miscellaneous_Guide.html">Bacula Miscellaneous Guide</A> <B> Previous:</B> <A NAME="tex2html143" HREF="Contents.html">Contents</A> <B> <A NAME="tex2html151" HREF="Contents.html">Contents</A></B> <B> <A NAME="tex2html153" HREF="Index.html">Index</A></B> <!--End of Navigation Panel--> <ADDRESS> 2010-06-14 </ADDRESS> </BODY> </HTML>