The MaJorToM ( Merging Topic Maps Engine ) project was founded to develop a lightweight, merging and flexible Topic Maps engine satisfying different business use cases. The engine provides a couple of new features above to other engines based on the Topic Maps API version 2.0.

The project is hosted on http://code.google.com/p/majortom/. If you find bugs or have feature requests please file a ticket.

The JavaDocs of the MaJorToM API are located here.

1. Introduction

MaJorToM is more than just a Topic Maps engine, it’s a kind of philosophy of designing an independent Topic Maps engine. The MaJorToM API extends the Topic Maps API 2.0 to provide a simple way for accessing the advanced features of next generation engines. Because of this downward compatibility, the engine can be used as drop-in replacement for any other Topic Maps engine based on the TMAPI without any modifications. The architecture of MaJorToM is designed for satisfying the goals of simplicity and flexibility and splits the information and application domain from each other. The application only uses a set of object stubs to communiate with the underlying store. The objects do not contain any knowledge about the subjects they represent. The knowledge is encapsulated by the underlying store, which offers a set of abstract methods to access this information. The store may not be a real topic map store which means it doesn’t have to use the Topic Maps paradigm for data storage.

This document tries to offer an overview about the advanced features of the MaJorToM engine and their usage. Additionaly the document describes the core architecture of MaJorToM to offer the possibility for integrate your own special domain specific features to your customized MaJorToM instance.

2. Architecture

The MaJorToM engine is designed in a flexible and modular way. The core elements for data storage are independent from the accessing objects used by the upper application. This modular design enables the interchangeability of different Topic Maps stores without any adjustments of the real application.

2.1. The Information Objects

The TMDM information items represented by the API objects do not store any knowledge or information. An information item are only represent by an object stub with a unique identity and the knowledge about its parent property (e.g. a name item knows its parent topic ). Any other information are encapsulated by the underlying topic map store and can be accessed by the object stubs using a atomic operation with a specific signature to tell the store about the information wanted. If such an object stub use an invalid signature, the request will be rejected by the store with an exception.

2.2. The Topic Map Store

Each Topic Map Store instance is bound to a specific topic map instance. The store provides a communication API to access the information stored in the real data store ( database, in-memory etc.). The data access is transparent for the calling object stubs and enables the interchangeability of topic map stores for an application without adjusting any lines of code.

The separation of information items and the real data store enables the implemenation of other Topic Map Stores without any effects for the core implemention of MaJorToM. At the moment MaJorToM provides a set of different Topic Map stores based on different data stores like RDBMS, ORDBMS or just in memory.

3. How to use the MaJorToM Topic Maps Engine

Before you can use the MaJorToM Topic Maps engine, the corresponding jar has to be located in the classpath of the application. If you use the Maven Dependency Management, it would be necessary to add the following to the pom.xml.

 <repositories>
      <repository>
          <id>TMLab Public Repository</id>
          <url>http://maven.topicmapslab.de/public/</url>
      </repository>
  </repositories>

  <dependencies>
      <dependency>
          <groupId>de.topicmapslab.majortom</groupId>
          <artifactId>majortom-model</artifactId>
          <version>1.1.1</version>
      </dependency>
  </dependencies>

To add the MaJorToM API and the core implementation to your classpath.

<repositories>
      <repository>
          <id>TMLab Public Repository</id>
          <url>http://maven.topicmapslab.de/public/</url>
      </repository>
  </repositories>

  <dependencies>
      <dependency>
          <groupId>de.topicmapslab.majortom</groupId>
          <artifactId>majortom-inMemory</artifactId>
          <version>1.1.1</version>
      </dependency>
  </dependencies>

To add a specific topic map store to your classpath ( in this case the in-memory topic map store. )

The MaJorToM API is downward compatible with the Topic Maps API 2.0.2. Because of that creation of a new topic map is similiar to the usage of TMAPI except the used topic map store.

TopicMapSystemFactory factory = TopicMapSystemFactory.newInstance();
TopicMapSystem topicMapSystem = factory.newTopicMapSystem();
TopicMap topicMap = topicMapSystem.createTopicMap("http://psi.majortom.test");

3.1. Use a Specific Topic Map Store

If the classpath contains more than one Topic Map store implementation of MaJorToM, it is necessary to specify the implementation class of the store to use. There are different possibilities to specify that.

final String propertyName = TopicMapStoreProperty.TOPICMAPSTORE_CLASS;
final String className = "TheTopicMapStoreClassName";
TopicMapSystemFactory factory = TopicMapSystemFactory.newInstance();
factory.setProperty(propertyName, className);

Using the property to specify the topic map store to use.

TopicMapSystemFactory factory = TopicMapSystemFactory.newInstance();
ITopicMapSystem topicMapSystem = (ITopicMapSystem) factory.newTopicMapSystem();
InMemoryTopicMapStore store = new InMemoryTopicMapStore(topicMapSystem);
TopicMap topicMap = topicMapSystem.createTopicMap("http://psi.example.org/majortom", store);

Using the additional method of the topic map system interface of MaJorToM.

#content of engine.properties
de.topicmapslab.majortom.topicmapstore.class = de.topicmapslab.majortom.inMemory.store.InMemoryTopicMapStore

Using a property file with the name engine.properties to specify the topic map store to use.

Please note: If there are only one implementation located in the classpath, it is not necessary to specify it to use, because the store implementation will be registered automatically by the Java Services or OSGi Framework.

Please note: Some topic map stores comes with the restriction to specify an additional set of properties to enable their usage. Please read the corresponding section to learn more about that.

3.2. MaJorToM API Feature String

The set of supported features of each topic map store implementation can be differ. The following table contains the name of all feature strings known by every topic map store. Some of them can be changed before the creation of a topic map system, to enable or disable the feature, if the store supports that.

Feature String

Discription

Default (if supported)

Modifiable

http://tmapi.org/features/automerge

If an implementation supports this feature, then whenever the implementation detects that two Topics should be merged (by one or more of the merge features defined under http://tmapi.org/features/merge/ ), then the implementation MUST merge the properties of these two Topics automatically and transparently to the API client. This means that the client must be able to continue using a reference to either of the two original Topics but should be viewing/manipulating the same merged Topic. If an implementation does not support this feature, then whenever the implementation detects that two Topics should be merged, it MUST raise an org.tmapi.core.TopicsMustMergeException.

enabled

yes

http://tmapi.org/features/merge/byTopicName

If the feature is set, two topics are detect as equal if the has the same topic-name (same value, same type and same scope)

disabled

yes

http://tmapi.org/features/type-instance-associations

If the feature is set, the topic maps engine creates an type-instance-association if the application calls the addType() method of a topic. The getTypes() method also includes all types specified by the type-instance-association if the feature is set

enabled

yes

http://tmapi.org/features/supertype-subtype-associations

If the feature is set, the topic maps engine creates an supertype-subtype-association if the application calls the addSupertype() method of a topic. The getSupertypes() method also includes all types specified by the supertype-subtypes-association if the feature is set

enabled

yes

http://tmapi.org/features/readOnly

If the feature is enabled, the topic map store does not support any write access.

depends on topic map store

no

de.topicmapslab.majortom.topicmapstore.history

If the feature is set, the topic map store save each modification of the topic map into the internal history.

disabled

yes

de.topicmapslab.majortom.topicmapstore.transaction

If the feature is set, the topic map store, provides a in-memory transaction access to modify the topic map without effecting changes outside the transaction before commit.

depends on the topic map store

no

de.topicmapslab.majortom.constraints.deletion.reification

If the feature is set, the topic map store remove the reified construct if the reifier was removed by the application.

disabled

yes

The feature string has to be set before the topic map system was created by the topic map system. To declare the feature string, MaJorToM comes with a static class containing all features strings, which can be set.

TopicMapSystemFactory factory = TopicMapSystemFactory.newInstance();
// Enable automatic merging
factory.setFeature(FeatureStrings.AUTOMATIC_MERGING, true);
// Create topic map system and topic map
TopicMapSystem system = factory.newTopicMapSystem();
TopicMap topicMap = system.createTopicMap("http://psi.example.org/");

4. Features of MaJorToM and Their Usage

The following section describe some of the additional features of the MaJorToM topic maps engine. Please note, that some of the features not supported by all topic map stores.

4.1. Temporal and Spatial Domains

In the context of the modelled domain, an occurrence can represent a temporal or spatial characteristics of a topic, e.g. the geographical coordinate of a topic representing a city or the birthdate of a topic representing a person. The MaJorToM API comes with a special occurrence interface to support geographical and temporal meanings of occurrence values like the described one.

To set a temporal value, the application has to use the datatype xsd:date or xsd:dateTime. Or use the special modification method of an existing occurrence item.

Topic person = topicMap.createTopic();
Topic typeDateOfBirth = topicMap.createTopic();
Locator datatype = topicMap.createLocator(XmlSchemeDatatypes.XSD_DATE);
// create an occurrence with temporal value
IOccurrence dateOfBirth = (IOccurrence)person.createOccurrence(typeDateOfBirth, "1810-10-10", datatype);

// modify the value as temporal value
dateOfBirth.setValue(new GregorianCalendar(1810, 10, 10));

To set a spatial value, the application has to use the datatype xsd:geoCoordinate. Or use the special modification method of an existing occurrence item.

Topic city = topicMap.createTopic();
Topic typeLocation = topicMap.createTopic();
Locator datatype = topicMap.createLocator(XmlSchemeDatatypes.XSD_GEOCOORDINATE);
// create an occurrence with geographical value
IOccurrence location = (IOccurrence)city.createOccurrence(typeLocation, "[51.0N;10.0E]", datatype);

// modify the value as geographical value
Wgs84Degree longitude = new Wgs84Degree(51.0, Orientation.N);
Wgs84Degree latitude = new Wgs84Degree(10.0, Orientation.E);
location.setValue(new Wgs84Coordinate(longitude, latitude));

4.2. Paging

Since version 1.1.0 the MaJorToM engine supports a paging mechanism which can be used by applications instead of the "unpaged" methods of MaJorToM API. The paged methods can be used by a set of special index classes or a set of paged construct classes.

4.2.1. Using Paged Indexes

As an extension of the base indexes, the new MaJorToM version includes a set of special indexes supporting a paging mechanism. Each method exists in two versions expecting two or three arguments. The optional third argument has to be an instance of java.util.Comparator which should be used to sort the returned list. If the comparator is missing, the list will be sort by default rules.

IPagedLiteralIndex index = topicMap.getIndex(IPagedLiteralIndex.class);
/*
 * open the index if it is closed
 */
if ( !index.isOpen()){
        index.open();
}
/*
 * get the first 10 names
 */
List<IName> names = index.getNames(0, 10);

Each paged index can be accessed by the topic map instance using the class object as parameter of the getIndex() method. At first each application should check if the returned index is already open and should open it before first usage to avoid an error.

4.2.2. Using Paged Constructs

As an alternative for the paged indexes is the usage of paged constructs.

Topic topic = topicMap.createTopic();
IPagedTopic pagedTopic = (IPagedTopic) topic;
/*
 * get number of names
 */
int number = pagedTopic.getNumberOfNames();
/*
 * get the first 10 names
 */
List<Name> names = pagedTopic.getNames(0, 10);
/*
 * get the first 10 names sort by their value
 */
List<Name> sortedNames = pagedTopic.getNames(0, 10, NameByValueComparator.getInstance(true));

The MaJorToM core comes with a set of useful comparator instances which can be used as third argument of the paged index and paged constructs methods. The following table contains a set of provided comparators and a short description of their behaviour. The comparators are located in the package de.topicmapslab.majortom.comparator.

Comparator Name

Description

AssociationComparator

An association a1 is smaller than a2 if the type of a1 is smaller or if both have the same type the number of roles of a1 is smaller than of a2.

ConstructByItemIdentifierComparator

As construct c1 is smaller than c2 if the first of the sorted item-identifiers of c1 is lexicographically smaller than of c2 or if c1 has no item-identifier.

LocatorByReferenceComparator

A locator l1 is smaller than l2 if its reference is lexicographically smaller than the reference of l2.

NameByValueComparator

A name n1 is smaller than n2 if the value of n1 is lexicographically smaller than the value of n2.

OccurrenceByValueComparator

An occurrence o1 is smaller than o2 if the value of o1 is lexicographically smaller than the value of o2.

RoleComparator

A role r1 is smaller than r2 if the type of r1 is smaller than the type of r2. If both roles have the same type, the players are compared. If the players also equal, the parent associations are compared.

TopicByIdentityComparator

A topic t1 is smaller than t2 if the first of the compared subject-identifiers of t1 is smaller than of t2. If both have no subject-identifiers the subject-locators of both compared and after that the item-identifiers.

TopicByNameComparator

A topic t1 is smaller than t2 if the first of the sorted names of t1 is smaller than the sorted names of t2.

VariantByValueComparator

A variant v1 is smaller than v2 if the value of v1 is lexicographically smaller than the value of v2.

4.3. Event Tracking

The MaJorToM engine provides an event management mechanism which enables the possibility of listening to each event or even special events of the topic map.

Applications which wants to listen to an event of the topic map has to implement the ITopicMapListener interface of the MaJorToM API.

public class MyListener implements ITopicMapListener{

  @Override
  public void topicMapChanged(String id, TopicMapEventType event, Construct notifier,
                                                      Object newValue, Object oldValue) {
     ...
  }

}

Addionally each listener has to registered at the topic map, it wants to listen to.

MyListener listener = new MyListener();
topicMap.addTopicMapListener(listener );

If an application wants to stop listening, the listener can be unregistered, too.

topicMap.removeTopicMapListener(listener );

The MaJorToM event manager notifies each event to all registered listeners. Therefore the method topicMapChanged will be called with the following attributes.

Param

Description

id

the internal id of this event

event

an enumeration value representing the kind of modification

notifier

the context of modification (e.g. the topic which get a new name)

newValue

the new value if a construct was created or its value was modified (can be null)

oldValue

the old value if a construct was removed or its value was modified (can be null)

4.4. History

If the feature is enabled (see de.topicmapslab.majortom.topicmapstore.history) and supported by the topic map store, MaJorToM stores each modification of the underlying topic map in a revision management system. Each revision represents one atomic call of the MaJorToM API and contains at least one change. If the case of merging two topics, e.g. caused by adding a subject-identifier, the revision contains each atomic change caused by the merging process. The next example tries to clarify the meaning of that.

#szenario definition
ITopic topic = createTopic();
topic.createName("Name");
topic.createName("Other").setReifier(createTopic());
topic.addSubjectIdentifier("si");

ITopic other = createTopic();
other.createName("Other");
other.addSubjectLocator("sl");

other.addSubjectIdentifier("si");

#change set of the merge revision
1. TOPIC CREATED #a new topic as subject for the merging ones
2. SUBJECTIDENTIFIER REMOVED si #the subject identifier was removed from topic, to avoid cycle merging
3. SUBJECTIDENTIFIER ADDED si #the subject identifier will be added to the new topic
4. SUBJECTLOCATOR REMOVE sl #the subject locator was removed from other, to avoid cycle merging
5. SUBJECTLOCATOR ADDED sl #the subject locator was added to the new topic
6. NAME CREATED #create the name with value "Name"
7. NAME CREATED #create the name with value "Other"
8. SET REIFIER null #Remove reification of the second name of topic
9. SET REIFIER #set reification to the name of the new topic
10.REMOVE TOPIC #remove topic
11.REMOVE TOPIC #remove other

4.4.1. Accessing the history

The history can be accessed by a special index implementation called IRevisionIndex. The index instance is provided by the topic map instance using the getIndex() method.

IRevisionIndex index = topicMap.getIndex(IRevisionIndex.class);

Please note: If the topic map store does not supports the reivision management feature, this method will throw an execption by calling it.

There are different possibilities to get the revision the application wants. Because the history is designed as a linked list of revisions in temporal order, each revision provides a getPast() and getFuture() method to navigate over the chain of temporal changes. If there is no revision before or last the current one the method will return null. The first and last revision can be accessed by the index methods getFirstRevision() and getLastRevision().

IRevisionIndex index = topicMap.getIndex(IRevisionIndex.class);
// open index before first usage
if ( !index.isOpen()){
        index.open();
}

IRevision firstRevision = index.getFirstRevision();
IRevision lastRevision = index.getLastRevision();

Please note: If the topic map wasn’t changed before, the methods will return null.

The history supports a set of methods to filter the revisions by different criterias, as the following table shows.

Method

Arguments

Description

getLastModification

Returning the timestamp of the last modification of the whole topic map

getLastModification

Topic

Returning the timestamp of the last change of the given topic

getRevisions

Topic

Returning the revisions in temporal order containing at least one change of the given topic

getChangeset

Topic

Returning a virtual changeset contains only the changes of the given topic in temporal order

getAssociationRevisions

Topic

Returning the revisions in temporal order containing at least one change of an association of the given type

getAssociationChangeset

Topic

Returning a virtual changeset contains only the changes of an association of the given type in temporal order

getRevision

Calendar

Returns the last revision created before the given timestamp

getRevision

String

Returns the last revision created before the timestamp represenent by the given tag

getRevision

long

Returns the revision with the given id

4.4.2. Working With a Revision

The revision object represents a change of the topic map and can contain more than one change. The changes of a revision are represented as a changeset object containing all changes in temporal order. The changeset can be accessed by the getChangeSet method.

Each contained change store the information of an atomic change as a tuple containing the following information

Param

Description

revision

the parent revision

type

an enumeration value representing the kind of modification

context

the context of modification (e.g. the topic which get a new name)

newValue

the new value if a construct was created or its value was modified (can be null)

oldValue

the old value if a construct was removed or its value was modified (can be null)

A revision object also store the timestamp representing the time the modification takes place. This information can be accessed by the getTimestamp method.

4.5. Transaction

MaJorToM in-memory store provides an embedded transaction mechanism which can be used to modify any constructs of the topic map without direct effects to the underlying topic map. Creating a Transaction

To open a new transaction context the additonal ITopicMap interface of the MaJorToM API has to be used like the following once. Therefor the TopicMap? interface of TMAPI has to be cast to ITopicMap.

ITransaction transaction = topicMap.createTransaction();

The transaction context behaves like a topic map, which enables a normaly usage.

4.5.1. Transaction Context

Each construct which should be used in the transaction has to be in the transaction context, which means, that the internal "topic map" has to be the transaction itselfs. Please note, that construct outside the transaction context modify the topic map directly.

To move a construct created outside the transaction context inside, the method moveToTransactionContext can be used.

/*
 * outside the context
 */
ITopic type = topicMap.createTopic();
/*
 * inside the context
 */
Topic type_ = transaction.moveToTransactionContext(type);

Please note: Each construct created within the transaction context are already inside the context!

4.5.2. Commit and Roll-back

To commit all changes of the transaction the method commit has to be used.

transaction.commit();

To remove all changes of the transaction the method rollback can be used.

transaction.rollback();

Please note: Please note, that the transaction context is invalid after the successful execution of commit or rollback.

5. Topic Map Stores

The MaJorToM Topic Maps engine comes with a set of different topic map stores based on different data paradigms, like memory-based or databases. The following section explain the usage and benefits of the supported topic map stores.

5.1. In-Memory Topic Map Store

The In-Memory topic map store is a powerful and fast store to access information as fast as possible. All information are stored in the memory during the runtime and will be destroyed after shuting down the upper application. This means the In-Memory topic map store does not persist any information, this has to be realized by the upper application.

The current version of this topic map store supports all features of a MaJorToM engine, like transaction support and revision management.

5.1.1. Usage

To use the in-memory topic map store, the majortom-inMemory.jar should be located in the class path. If the Maven dependecy management is used, the following line should be added to the POM file.

<dependencies>
      <dependency>
          <groupId>de.topicmapslab.majortom</groupId>
          <artifactId>majortom-inMemory</artifactId>
          <version>1.2.0-SNAPSHOT</version>
      </dependency>
  </dependencies>

5.2. (O)RDBMS Topic Map Store

MaJorToM provides a generic JDBC topic map store to support different databases by one topic map store implementation, as shown in the following table.

Name

Supported

Optimized

Property

MySQL

yes (since version 1.3.0)

no

MYSQL

PostGreSQL

yes

yes

POSTGRESQL

Please note: To optimize the communication with the databases, MaJorToM implements a set of optimized SQL processors using stored procedures or trigger-based modifications. If the store procedures are provided by the database, the query processor will use them to speed up the performance.

As additional propteries the (O)RDBMS topic map store requires the connection properties to establish the connection to the used database. The following table contains the set of required properties of the (O)RDBMS topic map store. If at least one of the properties is missing an error occurred.

Property

Description

de.topicmapslab.majortom.jdbc.host

The host name or IP address of the database server

de.topicmapslab.majortom.jdbc.database

The database name which should be used

de.topicmapslab.majortom.jdbc.user

The user name to access the database

de.topicmapslab.majortom.jdbc.password

The password to access the database

de.topicmapslab.majortom.jdbc.dialect

The used database system type btw. the SQL dialect

5.2.1. Usage

To use the (O)RDBMS topic map store, the majortom-db.jar should be located in the class path. If the Maven dependecy management is used, the following line should be added to the POM file.

<dependencies>
      <dependency>
          <groupId>de.topicmapslab.majortom</groupId>
          <artifactId>majortom-db</artifactId>
          <version>1.2.0</version>
      </dependency>
  </dependencies>

After starting the application the (O)RDBMS topic map store checks if the database exists and the contained schema is valid for usage with MaJorToM. If the database is empty the schema will automatically created by MaJorToM.

5.3. Queued Topic Map Store

The Queued topic map store is a combination of the In-Memory and the RDBMS topic map store. All information are stored in the underlying database handled by the RDBMS topic map store. To speed up the write access, each modification is handled as process of an event queue. Until this process is finished, the information created or modified by this process are stored in a virtual in-memory layer. After finishing the process the information are removed from virtual layer and can be directly accessed by the RDBMS store. Additionaly the queued store also supports the commit functionality which stops the process called until all tasks of the event queue are finished.

Because of that the queued store is the best combination of the fast read and write access of memory backends and the persistence of RDBMS backends.

The current version of this topic map store supports all features of a MaJorToM engine, like transaction support and revision management and also caching.

5.3.1. Usage

To use the queued topic map store, the majortom-queued.jar should be located in the class path. If the Maven dependecy management is used, the following line should be added to the POM file.

<dependencies>
      <dependency>
          <groupId>de.topicmapslab.majortom</groupId>
          <artifactId>majortom-queued</artifactId>
          <version>1.2.0</version>
      </dependency>
  </dependencies>

Please note: Because of the fact that the queued store combines the memory and the RDBMS store, all properties of both are needed.

5.4. Redis Topic Map Store

Since version 1.2.0, MaJorToM supports a generic topic map store using a redis backend. The Redis Store is the best combination of performance and persistency. Currently the redis does not supports any features of the API, in example the history management and the transaction are not supported yet. The schema of the redis store is documented here.

Please note, that a topic map store only supports one topic map instance for each topic map system.

As additional properties, the Redis topic map store requieres a set of connection parameters. For easy useage, the store supports an interface IRedisTopicMapStoreProperty containing all this parameter keys as string constants.

Property

Description

de.topicmapslab.majortom.redis.host

The host name or IP address of the database server. This parameter is mandatory.

de.topicmapslab.majortom.redis.database

The database id which should be used. This parameter is mandatory.

de.topicmapslab.majortom.redis.password

The password to access the redis store. This parameter is optional.

de.topicmapslab.majortom.redis.port

The port of the redis store. If the server uses the default port, the parameter can be missed.

5.4.1. Usage

To use the redis topic map store, the majortom-redis.jar should be located in the class path. If the Maven dependecy management is used, the following line should be added to the POM file.

<dependencies>
      <dependency>
          <groupId>de.topicmapslab.majortom</groupId>
          <artifactId>majortom-redis</artifactId>
          <version>1.2.0</version>
      </dependency>
  </dependencies>
topicmapslab.de