New in D6 Platform: Property Bag

This post continues the elaboration of D6 platform topics, which was started with the post on aspects. Property bag was barely mentioned in that post but is explored in detail here.

Before we try to put our arms around the property bag or try to get inside it, we may want to look at a few other concepts. These will help us understand the why as well as how of this new feature.

Concepts

What’s in a bag, anyway? Readers with exposure to computer science theory and data structures would recognize the bag as a standard data structure. Those who know what a set is can understand that a bag is similar to a set except for the fact that a bag can contain duplicate values while a set cannot. Does property bag have anything to do with the bag data structure? The reference documentation doesn’t make any comment in this regard – the clients of Content Server will not need to worry about it since the Content Server will take care of encoding and decoding information as it gets into or out of the property bag, respectively.

As we will see shortly, a property bag holds values for certain properties that don’t have columns of their own in the underlying tables. So it may be best to think of the property bag as just a place to hold values for multiple properties.

D6 introduces the concept of non-qualifiable properties. So the properties as present in prior versions should be seen as qualifiable. In an object type definition, a property is qualifiable by default (if you don’t specify it to be qualifiable or otherwise). A non-qualifiable property cannot be referenced in an expression or in a qualification (WHERE clause) in a query unless the query is an FTDQL query. Thus, making a property non-qualifiable limits what you can do with it later. You can store and retrieve values for this property but cannot search on it unless it is full-text indexed. Why would you want to put such a limitation on the metadata? The only explanation appears to be related to performance optimization.

How does Content Server distinguish between qualifiable and non-qualifiable properties? It does so in two ways:

1. The attr_restriction property in the dm_type object identifies each property as either qualifiable or non-qualifiable.
2. A qualifiable property is represented by a column in the appropriate underlying database table for the type that contains the property. A non-qualifiable property is stored in the i_property_bag property of the object.

Thus, non-qualifiable properties live in the property bag. The property bag stores the names and values of several properties in a serialized format. Since non-qualifiable properties don’t have their own columns they can’t participate in constraint checking such as primary key or foreign key.

Despite the above-mentioned differences the non-qualifiable properties have many similarities to the qualifiable properties:

• They can be full-text indexed
• They can be referenced in the selected values list of a query statement
• Selected non-qualifiable properties are returned by a query as a column in a query result object
• They can be repeating or single-valued

The Property Bag

We now have an idea about the property bag – it is a place for holding non-qualifiable properties in an object. Let’s dive deeper to see how it is implemented.

The property bag is a special property used to store:

• Non-qualifiable properties and their values
• Aspect properties and their values if the properties are defined with fetch optimization

Under the Hood

The property bag is implemented as a pair of properties:

• i_property_bag, which is a single-valued string property of 2000 characters
• r_property_bag, which is a repeating string property of 2000 characters

i_property_bag stores the names and values of non-qualifiable properties. It is also used to store the names and values of aspect properties if the properties are added to the aspect with the OPTIMIZEFETCH option.

r_property_bag stores overflow from the i_property_bag property – that is if i_property_bag doesn’t have enough space to hold all the property name-values that it needs to hold then r_property provides the additional space. This property is automatically added to an object type definition if the definition includes the i_property_bag property. Thus, this property makes the property bag expandable.

Enabling Property Bag

dm_sysobject and its subtypes automatically have a property bag. Other custom types can be altered to contain a property bag using the ALTER TYPE statement. This adds the above-mentioned properties to the type.

If you include a non-qualifiable property in the definition of a NULL type (has no supertype) or whose supertype is not in the dm_sysobject hierarchy, then the i_property_bag property is added automatically to the type.

The property bag cannot be removed once it has been added to an object type.

Property Identifiers

Internally, every property has an identifier, which is unique within its type hierarchy. This identifier is used to identify the property when it is stored in the property bag. The identifier is an integer value stored in the attr_identifier property of each type’s dm_type object. A property’s identifier cannot be changed.

Limitations

If a non-qualifiable property is of string datatype, its length must be less than the value in the max_nqa_string key in the server.ini file if that key is set. If this key is not set, it defaults to 2000.

New in D6 Platform: Aspects

The long wait for D6 (Documentum Version 6.0) is over and the D6 product and documentation downloads are now becoming available. Note, however, that it is not a synchronized release of the complete suite of Documentum products. While version 6.0 of the core products (Content Server, Webtop, DA, etc.) is available now, it may take months for the complete suite to become available in version 6.0. So plan according to their release schedule and you will be well-advised to wait for service pack 2 before going to production.

D6 continues to have the rich detail in its documentation that it has had in the past. After a cursory look at the documentation, it seems that the changes in D6 relative to 5.3 have been called out decently. However, full details of these changes still remain embedded in multiple documents. Here, I will share some of these topics in detail with an attempt to present the new/changed concepts in simple terms. In one of my other posts I had discussed aspects in Alfresco. The general concept of aspects remains the same while we take a look at the specifics in D6 here.

Overview

In D6, aspects add a new framework for extending object behavior and attributes. Recall that behavior and attributes (properties) for objects are managed through object types. However, an object type applies to all objects of that type (and of its subtypes, via inheritance). In contrast, aspects allow you to attach additional behavior and attributes to an object, independently of its type. Using aspects can speed up development and improve code reuse, because they offer the possibility of extending attributes and behavior without altering the underlying type definitions. However heed the caution in the other post regarding the abuse of aspects.

Let’s explore the concept, constraints, and implementation specifics about aspects in D6 with some examples below. Note that the information presented below is based on the available documentation for D6. At the time of writing this post the Content Server API Reference was not available for D6.

Example

Let’s start with an example. Suppose that an organization manages documents for projects within Documentum. The documents that are related to projects must have an attribute called project, which identifies the project that a document is related to. Note that this relationship is cross-cutting as far as object types are concerned. In other words, any object (no matter what its type) can be associated with a project. On the other hand, there will be many objects that have nothing to do with projects. This is a perfect situation for utilizing aspects. (Note that there are other alternatives for capturing this information but we are illustrating aspects here)

In this scenario, consider the documents:

1. mydoc.txt of type dm_document
2. mydoc.doc of type dm_document
3. myemail of type dm_email_message

mydoc.doc is related to project A, myemail is related to project B, while mydoc.txt is not related to any project. We can create an aspect named project_oriented with one attribute – project (It could contain additional project-related attributes if needed). Now we can attach this aspect to mydoc.doc and to myemail. As a result, the attribute project will become available on these objects and can be set to the respective project names.

Constraints

Probably, one question would have already popped up in keen readers’ minds – Are aspects allowed only on certain types of objects? Even though aspects can be attached independently of the type of object, aspects cannot be attached on arbitrary types of objects. Aspects can be attached under the following conditions on types:

1. An aspect can be attached to any object of type dm_sysobject or one of its subtypes.
2. An aspect can be attached to any object of NULL type (type with no supertype), once the type has been altered to allow aspects.

Another question is – Can multiple aspects be attached to one object? The answer is yes. In the example above, assume that there is another aspect called web_viewable. Then, myemail can have two aspects attached – project_oriented and web_viewable. The corresponding aspect properties (and methods) will become available on myemail. Note, however, that one aspect cannot be attached multiple times to the same object.

Managing Aspects

An aspect can be created using Documentum Application Builder. I expect other ways of creating aspects but I am unable to confirm that based on the available documentation.

It is possible to drop or modify existing properties or add new ones to an aspect. Such changes to an aspect immediately affect objects to which the aspect is currently attached.

Aspects can be altered using DQL as well. For example, the following DQL statement adds the project property to the aspect.

ALTER ASPECT project_oriented
ADD (project string(32))
OPTIMIZEFETCH


The keyword OPTIMIZEFETCH is explained later.

By default, aspect properties are not fulltext-indexed. However, aspect properties can be selectively configured to be full-text indexed. The following DQL configures all the attributes of the aspect to be full-text indexed.

ALTER ASPECT project_oriented
ADD_FTINDEX ALL


Using Aspects

If the object under consideration is of a NULL type, the type must first be altered to allow aspects on its objects. For example:

ALTER TYPE my_null_type ALLOW ASPECTS

This DQL statement performs the following actions:

1. Adds the repeating property – r_aspect_name, to the type definition of my_null_type if it does not have the property already
2. Adds the i_property_bag property to the type definition of my_null_type if the type does not have the property already

The central operation with aspects is to attach an aspect to an object. Unfortunately, I have been unable to locate this information in the currently available documentation.

The Content Server Fundamentals document and the DQL Reference document disagree about the qualifiable (A non-qualifiable property can be referenced in selected value lists, but may not be referenced in expressions or qualifications in a query) nature of aspect properties, so I am not sure which one is correct. If you are wondering about the significance of this nature, just remember that you will not be able to use a non-qualifiable property in the WHERE clause of DQL queries.

According to Content Server Fundamentals (pg 112)

Aspect properties are qualifiable properties. You cannot define an aspect property as a non-qualifiable property.

According to DQL Reference (pg 49)

Properties for a particular aspect must be either all qualifiable or all non-qualifiable. An aspect cannot have some properties that are qualifiable and some that are non-qualifiable.

Even though aspects are not automatically tied to object types sometimes you may have a need to do just that (often for convenience). For example, we may have an existing type called web_doc and we may want all objects of web_doc (and of its subtypes) to have the aspect web_viewable attached. If an aspect is associated with an object type, the aspect is automatically attached to each new object of the specified object type. Such an aspect is called a default aspect for the specified type. Of course, this aspect can still be attached to objects of other types (subject to constraints mentioned earlier). Default aspects can be specified as follows:

ALTER TYPE web_doc
ADD DEFAULT ASPECTS project_oriented, web_viewable


Multiple default aspects can be associated with one object type. An object type inherits all the default aspects defined for its supertypes. When you add a default aspect to a type, the newly added aspect is only associated with new instances of the type or subtype created after the addition. Existing instances of the type or its subtypes are not affected. Similarly, removal of a default aspect does not affect the existing instances of the type.

In a SELECT statement, aspect properties must be qualified by aspect name. For example:

SELECT project_oriented.project FROM web_doc

If there are multiple type names in the FROM clause the type name must also qualify the aspect name. For example:

SELECT web_doc.project_oriented.project, hr_doc.object_name
FROM web_doc, hr_doc
WHERE ...

Under the Hood

An aspect is a BOF (Business Object Framework) module that customizes behavior or records metadata or both for an instance of an object type. It is represented by an object of type dmc_aspect_type.

Content Server creates and manages an internal object type for each aspect. The names and definitions of aspect properties are stored in this internal type. The internal type is named dmi_type_id where type_id is the object ID of the internal type.

Incorporating aspect properties with regular object type properties in normal interaction adds computational overhead. Therefore, an option for enhancing query performance for aspects is available. If the OPTIMIZEFETCH keyword is specified in the ALTER ASPECT statement, it directs Content Server to store all the aspect’s properties and their values in the property bag of any object to which the aspect is attached, if the object has a property bag. (NOTE: property bag is also a new concept in D6 and is discussed in more detail in my other post)

Default aspects for an object type are recorded in the default_aspects property in the corresponding dmi_type_info object.

Conclusion

D6 has made the power of aspects available for new Documentum deployments. However, the complete Content Server documentation needs to be available for gaining an effective grasp on the depth and breadth of the available functionality. It will also be interesting to find out how the core applications such as Documentum Administrator and Web Publisher support and make use of aspects.

A suitable JVM could not be found!

If you see this error, it has to do with the installer configuration. More likely, you may find this with an uninstaller, particularly when the installed product versions have been mixed up. For example, uninstalling Documentum Web Publisher server files, more likely with the older versions. I say that because there are several EMC support notes about this so we may hope that the newer versions have addressed this issue.

We encountered this with an uninstaller which claimed to be version 5.2.2. The uninstaller just balked with a message, “A suitable JVM could not be found. Please run installer again using the option -is:javahome <JAVA HOME DIR>“. That sounded really helpful, so we tried pointing to 9 JRE’s (one by one, among many more present) embedded at various paths within Documentum products. Nothing worked and we continued to get the same error.

Then we found a support note on the InstallShield support site about this message. Even though it helped us understand the potential cause, it didn’t help fix it.

Research on Powerlink brought forth similar information. The key to making progress was the discovery of the option -is:log. This option enabled us to capture logs of the uninstaller activity.

The first thing we discovered was that the uninstaller was looking for IBM JVM 1.3.1 (yes, that old!) for the specific OS and was unable to locate it.

So we passed the path using -is:javahome and captured the log again. Now we realized that it was appending jre to the path and was unable to locate it. So we massaged the path to match what the uninstaller was expecting and that did it!

Finally, the command looked like the following

./uninstaller.bin -is:log log.txt -is:javahome path_to_parent_of_jre_folder

Lifecycle lost, document disappears in WP 5.2.5 SP2!

After some changes to the content server configurations in a multi-content-server environment intermittent weird issues became consistent. An author would try to edit an Active document and the document would disappear from WP.

Investigation revealed various things over time, as summarized below:

• The document was successfully checked out and a new version was created
• The new version still showed the lifecycle and state to be the same as the Active version
• The version label WIP disappeared from the Active one and was not placed on the new version
• Messages showed “checkout successful” but “failed to clean up temporary object”
• The ACL wasn’t changed to the expected one

Research on user groups and support forums revealed that there was a major known bug in WP 5.2.5 SP2 which resulted in the behavior described above. Even though it was related to inheriting ACL from User or Folder, that configuration setting alone could not fix the problem. SP3 fixed the bug partially while SP4 fixed it completely.

Why wasn’t the problem detected in QA? One possibility – QA had 5.2.5 SP5!