Introduction

Following the growth and evolution of a system, sometimes we have to make some modification to the structures of the data objects stored in the system. This could be done by updating all the affected legacy data in the database. However the update process might take a long time if the amount of data is large. It will cause inconvenience to users as they cannot access the system during the data update process. Hence, we need a way to modify the data objects structure and at the same time do not affect the availability of the system services. This report provides the details on how to modify JDO entity classes in TEAMMATES without having to manually update the legacy data.

Java object fields & Datastore entity properties

Before discussing the way to modify entity classes, we need to know how the conversion between Java object fields and datastore entity properties are done during the storing and retrieval operation carried out by JDO API.

Basically, when a Java object is saved into the datastore by JDO, it becomes an entity in the datastore. Each persistent field of the object will become a property of the entity in the datastore. If we don’t make any changes to the class of the data object, then the corresponding entity stored in the datastore always have the same amount and the same type of properties with the fields in the original data object. But if some modifications have been done on the entity class (the class of the data object that corresponds to the datastore entity), then things might get a little bit complicated. Below are a few examples showing the different cases that might happen:

Entity has extra properties than the fields in entity class

If the datastore entity has an extra property which is not available in the entity class, then that property is not accessible from the object. For the above example, the property “Nationality” is invisible and not accessible after loading into the Java object. If the object is saved back to the datastore, then the extra property in the original entity will be deleted.

Entity has less properties than the fields in Entity class

If a datastore entity is loaded into an entity class and doesn’t have a corresponding property for one of the fields, the field will be given a null value, provided that the field’s type is a nullable value type. For the above example, the loaded object’s “Nationality” field is set to null. When the object is saved back to the datastore, the entity will have a new property called “Nationality” and its value will be set to null.

In the case where the extra field in the entity class is not of a nullable value type (e.g. it is of primitive type like int), then loading an entity without a corresponding property will cause an exception to be thrown.

Example: Modifying Evaluation entity class to support storing of longer instructions

Problem

For each evaluation session created by users, the length of instruction can only contain up to 500 characters. This is due to the String data type of properties stored in App Engine Datastore only allows a maximum of 500 Unicode characters1. To store a string with more than 500 characters, the new data type “Text” provided by Datastore must be used. Since the length of 500 characters might not be long enough for certain users, we should change to the “Text” data type to support storage of longer strings.

Solution

The more practical way is to add another attribute of Text data type called “longInstructions” to the class of Evaluation data object to store instruction without length limitation. All Evaluation data objects created will make use of attribute longInstructions to store the instructions provided by users. This solution does not require modification to legacy data. The reason is because App Engine Datastore is actually a schema-less object database, which doesn’t require all entities of a given type to have the same set of attributes. Hence, it is allowed to have some entities in the database contain the attribute longInstructions, while the old entities don’t contain it. By this solution, we can store instructions longer than 500 characters, and at the same time we also can retrieve instructions from the legacy data without any problems.

Implementation

A new attribute of Text data type is added to the class of Evaluation data object to allow the storage of instruction longer than 500 characters. The change is illustrated as below.

During retrieval of data, JDO will load an Evaluation entity from Datastore into an Evaluation data object. If the entity happens to be from legacy data and doesn’t contain the new attribute longInstructions, then the loaded Evaluation data object will have a null value for the attribute longInstructions. Hence, to correctly retrieve instruction data from Evaluation data object, we have to use the mechanism as shown in the activity diagram below.

Firstly we retrieve the value from longInstructions attribute. If the value is null, then we retrieve and return the value from the old instructions attribute instead. Else, we just directly return the value to the caller as it is.

Evaluation

By the solution above, we have successfully increased the capacity of evaluation instruction to allow more than 500 characters. After any updates done on an Evaluation data object and saved to the database, the corresponding Evaluation entity will also be updated to the latest format automatically. Hence after certain amount of time, most legacy data should have been updated to the latest format automatically.

Future work

Research for ways to change the primary key to other field

References

[1] Java - Google Developers. Entities, Properties, and Keys.

https://developers.google.com/appengine/docs/java/datastore/entities#Java_Properties_and_value_types

Resources

1. More information about JDO entity class: https://developers.google.com/appengine/docs/java/datastore/jdo/dataclasses

--- end of report ---