Further Streamline BOF Module Deployment During Development

Introduction

In a previous article,
Automate BOF Module Deployment During Development.
I detailed how to install Documentum Business Object
Framework (BOF) Modules using an undocumented Ant task
provided by Documentum’s Application Builder. With the
BOFPackaging Ant task, it is possible to install and update
modules in an automated manner, freeing us from having to
maintain these modules manually through docapps during
development.
While the automated BOFPackaging task represents a
significant improvement over manual deployments, it too can
be inefficient at times. Namely, the BOFPackaging task will
completely re-install all of a module’s Java archive (JAR)
files, including all third party dependencies (e.g., XML
parsers, Log4j, etc.) which are unlikely to have changed.
Often these static libraries can total several megabytes in
size, and the update process will waste several minutes –
even upwards to an hour over a poor network – updating them
unnecessarily. Considering that my fellow developers and I
will re-deploy a module numerous times during development,
the time wasted updating library files that haven’t been
modified quickly adds up. As such, we’re discouraged from
deploying many, small updates which are otherwise more
efficient and less error-prone, and instead will batch
enhancements into larger updates that are more difficult to
isolate, test, and troubleshoot.
To address this problem, I created a generic, commandline
application, FileLoader, which can add new files or update existing files in a
docbase. I use the FileLoader
tool within my Ant build scripts to update only the JAR
files that are likely to have changed while developing my
project’s modules. In this article, I’ll describe how my
FileLoader works and how Documentum makes these efficient
updates possible.
For your reference, links to all source code can be found at
the end of this article.

Review of BOF Object Schema and Folder Structure

Before we dive into the implementation of the FileLoader,
let’s review the object schema and folder structure of a BOF
module.
As of DFC 5.3, Documentum BOF code is stored in the
repository and dynamically loaded by DFC as needed by client
application code. Service Based Objects (SBOs) are hosted in
a single “global repository”, providing the same
functionality to multiple repositories, while Type Based
Objects (TBOs) are hosted and configured in the repository
that uses them. Additionally, Documentum introduced the
concept of a “simple module”, which is a unit of executable
code stored in the repository. Like TBOs, simple modules are
hosted in the repository that uses them.
To support this module framework, a few new object types
were introduced: “dmc_module”, “dmc_jar”, and “dmc_java_library”.
The “dmc_module” type is a subtype of “dm_folder” and represents
a BOF module. A “dmc_module” is comprised of the “dmc_jar” and
“dmc_java_library” objects that are contained within it.
(Technically, “dmc_java_library” objects are associated with
“dmc_module” objects through dm_relation objects, enabling the
Java libraries to be reused across modules. But it’s common
practice for each module to have its own “dmc_java_library”
object linked directly under it within the repository.) The
“dmc_jar” type is a subtype of dm_document and represents
individual JAR files. The “dmc_java_library” type is another
subtype of “dm_folder” and represents the set of “dmc_jar”
objects that are contained within it. A module will
typically include a “dmc_jar” object for the module’s
interfaces, a “dmc_jar” object for the implementation of the
module’s interfaces, and a single “dmc_java_library”
containing all of the module’s runtime dependencies.
Every repository has a System cabinet, which contains a top
level folder named Modules. By convention, modules are
installed under this folder according to their type. For
example, SBOs are installed under “/System/Modules/SBO”, and
TBOs are installed under “/System/Modules/TBO”. For simple
modules, you can reuse existing or create additional
subfolders of “/System/Modules” that match your modules’
types. For example, if the repository uses Java-based
evaluation of validation expressions, the associated modules
would appear under “/System/Modules/Validation”. The hierarchy
of folders under “/System/Modules/” is referred to as the
repository’s module registry, or simply its registry.
The following screenshot is an example of one repository’s
module registry as seen through Webtop.

Figure 1:
Subfolders of Modules representing BOF types.

Please see the “BOF Infrastructure” section of the
Documentum Foundation Classes Development Guide for a full
description of BOF Modules.

Hot Deployments

When any of a module’s JAR files are updated in the
repository, Documentum will automatically update the BOF
object caches on all client machines to include the updated
JAR files, and the DFC BOF
ClassLoader
will redeploy the classes loaded within a client
application’s JVM without requiring an application restart.
FThis hot deploy behavior unlocks even more value from the
FileLoader utility. Using the FileLoader, we can checkin new
versions of a module’s JAR files, and all applications
(e.g., Documentum Administrator, Webtop, Documentum Java
Method Server, and custom web applications) will immediately
pick up the new behavior without any interaction required.
This is a tremendous time saver for developers.
How does the Business Object Framework know when a module’s
JAR files have been updated? In DFC 5.3, they used an early,
internal version of the Aspects feature that is now publicly
available starting with the release of Documentum 6.0. By
inspecting the r_aspect_name repeating attribute on all
“dmc_jar” objects, we see that the Business Object Framework
attaches a change monitor Aspect
(“com.documentum.fc.bof.bootstrap.DfModuleItemChangeMonitor”)
to all JAR files loaded into the repository. When a “dmc_jar”
object is updated, the change monitor Aspect is invoked and
notifies a modification manager
(“com.documentum.fc.bof.util.IDfModificationManager”) of the
change. The modification manager encapsulates a distributed
system that ensures all running DFC client applications are
notified of the change. The BOF ClassLoader within the DFC
client application will then reloads all classes that
originated from the modified JAR file, and the client
application acquires the new behavior.
I should warn that despite this sophisticated framework,
client applications may occasionally fail to pick up the
changes made to a module. When this happens, shut down the
client application, delete the repository’s BOF object cache
on the client machine
(“[install drive]:Documentumcache[DFC version]bof[repository name]”),
and restart the client application. DFC will then
refresh the client machine’s BOF cache upon startup and pick
up the changes made to the module.

The FileLoader Commandline Tool

The FileLoader is a relatively simple commandline tool. You
can use it to load or update a set of documents in a
repository. For each file processed, the FileLoader will
look for a corresponding document in the repository using
the docbase location provided and the name of the file on
the client machine to make a call to
“IDfSession.getObjectByPath(docbaseFilePath)”. If the files
don’t already exist within the target repository, then new
documents will be created and linked into the correct
folder. If the files do exist, then they’ll be saved, minor
versioned (default), or major versioned as desired. For new
documents, it is possible to specify the documents’ object
type and content type as well.
The following table describes the options supported by the FileLoader:

The files to upload should be passed to the FileLoader as arguments
after the last commandline option. A sample usage might look like

C:java .. com.ArgonDigitalgroup.util.FileLoader –docbase=db1 –username=steve –password=password –docbaseFolder=/System/Modules/TBO/bf_custom_doc/ C:/dev/build/bf-tbo-int.jar C:/dev/build/bf-tbo-impl.jar

Finally, for existing files that should be minor- or major-versioned,
the FileLoader follows Documentum’s recommended practice of leveraging
the IDfCheckoutOperation and IDfCheckinOperation.

The code sample below represents FileLoader’s main file processing logic:

/**
 * Uses loginName to establish a session, creates folders as necessary
 * to build up the path in the docbase, and then uploads each file
 * in the collection.
 */
protected void execute() throws Exception {
    IDfSession session = getSession();

    // First create the folders in the path.
    createFolders();

    // Now upload each file, versioning existing content as appropriate
    for (int i=0; i < files.length; i++) {
        // Local file to load.
        File file = createFile(files[i]);
        // Object name.
        String objName = file.getName();
        // Full docbase object path.
        String docbaseFilePath = docbaseFolder + (docbaseFolder.endsWith("/") ? "" : "/") + objName;

        System.out.println();
        System.out.println("        File:  " + file.getAbsolutePath());
        System.out.println(" Object name:  " + objName);
        System.out.println("Docbase path:  " + docbaseFilePath);
        System.out.println();

        IDfSysObject doc = (IDfSysObject) session.getObjectByPath(docbaseFilePath);
        if (doc == null) {
            System.out.println("Creating new file in docbase.");
            doc = (IDfSysObject) session.newObject(objectType);
            doc.setObjectName(objName);
            doc.link(docbaseFolder);
            doc.setContentType(contentType);
            doc.setFile(file.getAbsolutePath());
            doc.save();
        } else {
            System.out.println("Updating file in docbase.");
            checkout(session, doc);
            checkin(session, doc, file);
        }

        System.out.println("Loaded.");
    }
}

To use the FileLoader in your Ant scripts, I suggest creating an
Ant target to encapsulate the java task that will invoke the FileLoader:

<target name="_run-file-loader" depends="compile" description="Updates files in the docbase.">
	<property name="fileloader.optional.args" value="" />

	<java classname="com.ArgonDigitalgroup.util.FileLoader" fork="true" failonerror="true">
		<classpath>
			<path refid="project.class.path" />
		</classpath>

		<arg line="--docbase=${docbase.name} --username=${docbase.username} --password=${docbase.password}" />
		<arg value="--docbaseFolder=${fileloader.location}" />
		<arg value="--version=${fileloader.version}" />
		<arg line="${fileloader.optional.args}" />
		<arg line="${fileloader.files}" />

		<!-- We need the lava.library.path set to pickup the DMCL DLLs -->
		<sysproperty key="java.library.path" value="${documentum.lib.dir}" />
	</java>

</target>

Then to update TBOs, SBOs, and simple modules, simply call the
“_run-file-loader” target with the correct parameters.

Here’s the Ant target included that I’ve written to update this article’s sample TBO:

<target name="update-tbo" depends="init" description="Updates document TBO jar files in the docbase (no 3rd party dependencies).">
	<antcall target="_run-file-loader" inheritRefs="true">
		<param name="fileloader.location" value="/System/Modules/TBO/${tbo.module.name}" />
		<param name="fileloader.version" value="minor" />
		<param name="fileloader.optional.args" value="--contentType=jar" />
		<param name="fileloader.files" value="${tbo.int.jar.file} ${tbo.impl.jar.file}" />
	</antcall>
</target>

Here’s the Ant target included that I’ve written to update this article’s sample SBO:

<target name="update-sbo" description="Updates SBO jar files in the docbase (no 3rd party dependencies).">
	<antcall target="_run-file-loader" inheritRefs="true">
		<param name="fileloader.location" value="/System/Modules/SBO/${sbo.module.name}" />
		<param name="fileloader.version" value="minor" />
		<param name="fileloader.optional.args" value="--contentType=jar" />
		<param name="fileloader.files" value="${sbo.int.jar.file} ${sbo.impl.jar.file}" />
	</antcall>
</target>

Finally, here’s the Ant target included that I’ve written to update this article’s sample simple module:

<target name="update-module" description="Updates module jar file in the docbase (no 3rd party dependencies).">
	<antcall target="_run-file-loader" inheritRefs="true">
		<param name="fileloader.location" value="/System/Modules/Lifecycle/${lifecycle.module.name}" />
		<param name="fileloader.version" value="minor" />
		<param name="fileloader.optional.args" value="--contentType=jar" />
		<param name="fileloader.files" value="${lifecycle.module.jar.file}" />
	</antcall>
</target>

Conclusion

By taking advantage of the cache update and hot deployment features
built into Documentum’s Business Object Framework, the FileLoader
utility can save you valuable minutes each time you update a module
by updating only the JAR files that are likely to have changed.
These minutes multiplied over the entire development team and length
of the release can quickly add up to hours or even days of overall
effort. That’s a significant savings that eliminates most of the
tedium from BOF development and can sometimes make the difference
between project success and failure.

Further Reading

• Article source code – bof_ant_task_samples.zip
• Automate BOF Module Deployment During Development, by Steve McMichael
• A Practical Guide to the Business Object Framework, by Jason Duke
• Documentum Foundation Classes (DFC) 5.3 Development Guide: Chapter 3 – Using the Business Object Framework (BOF)
• Apache Ant