July 93 - NeoPersist–An Easy Way to Persistent Objects?
NeoPersist–An Easy Way to Persistent Objects?
Mary Elaine Califf
Last year, Bob Krause introduced the MADA community to NeoAccess™, NeoLogic's
cross-platform object-oriented database component. At MADACON '93 Bob discovered
that a number of developers were interested in the object persistence provided by
NeoAccess, but did not need the full set of database features and didn't want to pay for
them. The result is NeoPersist, a component which provides persistent object
management for applications, simplifying object storage and retrieval. NeoPersist
would be helpful to developers who want to store objects of multiple classes in a single
file or who want to avoid keeping all of a document's data in memory.
What is NeoPersist?
NeoPersist is a C++ class library that provides a way to create and manage up to five
classes of persistent objects with up to 64k of objects of each class. It works with
MacApp and the Think Class Library. NeoPersist provides a subset of the features of its
bigger sibling NeoAccess at a lower cost with no runtime fees. It uses the same
interface as NeoAccess so that developers who outgrow the functionality of NeoPersist
can move to NeoAccess with minimal effort.
NeoPersist provides classes which support storage and retrieval of objects, keeping
track of where the objects are in the file. Objects can be retrieved from the file in
groups or individually. NeoPersist also provides object-level locking. This provides
for concurrency control within an application, allowing an object to be marked "busy.
HOW IT WORKS
NeoPersist provides four classes that implement object persistence. The first is
CNeoFile, a subclass of TFile (under MacApp) or CDataFile (under TCL). This class
encapsulates all of the file management code necessary to support persistent objects
and is usually accessed via a group of routines which add and remove objects to and
from a file, add classes to a file, and "sync" the file with the objects in memory.
CNeoFile only updates the disk file when its sync() method is called, so "Revert
capability is easily supported. Individual objects in a file have IDs which allow them to
be identified. These IDs are used by NeoPersist to index objects within their classes
and thus can be used for fast retrieval of specific objects. The IDs are usually unique
within a class and file, but do not have to be. Unique IDs are necessary to
unambiguously identify an individual object, but multiple objects of the same class in
a file may share an ID.
A second class, CNeoPersist, is the ancestor of all persistent objects. The application
developer creates subclasses of CNeoPersist and overrides several of its methods,
including those that provide information about the class (such as getClassID(),
getFileLength() and getMetaClass()) and those that handle I/O. With MacApp, the
latter involve overriding static methods ReadFrom() and WriteTo() to handle reading
and writing any permanent attribute values from and to a TStream.
The search methods provided by CNeoPersist are FindByID(), which finds objects of a
given class with the ID specified, FindIDRange(), which finds objects of a given class
whose IDs fall within the range specified, and FindEvery(), which finds all objects of
the given class:
static CNeoPersist * FindByID(CNeoFile *aFile, const NeoID aClassID,
const NeoID aID, const Boolean aSubclass,
NeoTestFunc1 aFunc, void *aParam);
static CNeoPersist * FindIDRange(CNeoFile *aFile,
const NeoID aClassID, const NeoID aMinID,
const NeoID aMaxID, const Boolean aSubclass,
NeoTestFunc1 aFunc, void *aParam);
static CNeoPersist * FindEvery(CNeoFile *aFile, const NeoID aClassID,
const Boolean aSubclass, NeoTestFunc1 aFunc,
void *aParam);
In all of these methods, aClassID specifies the base class desired; aSubclass indicates
whether or not subclasses are also desired (as in a draw program where all of the
TShapes in a given area should be updated whether they're TCircles or TSquares);
aFunc if not nil indicates a function to be applied to each object found; and aParam can
be a parameter to aFunc, a pointer to an array in which to store multiple objects, or
nil. If aFunc and aParam are both nil, the methods return the first object found that
meets the class and ID criteria; otherwise, all objects meeting the criteria are
retrieved and either passed to the function or stored in the array.
Two other methods are provided to aid in a sequential traversal of a particular class in
a file. These are getNextSibling and getPreviousSibling which return the object of the
same class immediately preceding or following this object in the list. Both return nil
if the object requested does not exist.
CNeoBlob is a subclass of CNeoPersist which provides for storage of variable-length
data in a NeoPersist file. CNeoBlob objects have two parts on disk, an object part just
like the CNeoPersist ancestor and a blob part, which can contain free-form
variable-length data. The separation allows the object to be in memory while the
potentially large blob stays on disk. CNeoBlob provides methods to get and set the
blob's data and to mark the state of the blob (whether it has changed, whether it is in
use and in a potentially inconsistent state) separately from the object part of the
CNeoBlob. setBlob reads data from a handle, and getBlob returns a handle to the data.
Note that the data should be in a handle.
CNeoMetaClass is used by the NeoPersist classes to keep track of information about
each of the persistent classes belonging to the application. NeoPersist uses an array of
metaclass objects belonging to a file. The developer must add a CNeoMetaClass object to
the array for each of the application-specific persistent classes.
There are several differences in working with NeoPersist between the MacApp
environment and the TCL environment. These differences show up because of the
different structures of the two application frameworks. CNeoFile, CNeoPersist and
CNeoBlob differ somewhat in their methods and ancestry between the two. Under
MacApp, NeoPersist also provides subclasses of TApplication, TEditionDocument, and
TFileHandler for developers to base their classes on. These subclasses create and
manipulate CNeoFiles rather that TFiles and automatically handle a few things like
CNeoDocument::DoWrite() calls sync() on the file. Under TCL such subclasses are not
available, but the documentation spends several pages discussing the design of a
document object which uses a CNeoFile. In this review, I will focus on the MacApp
environment because the examples and documentation provided for TCL are slightly
better and because I don't have Think C.
Using NeoPersist
Documentation
If I have one gripe about NeoPersist, it is with the lack of a tutorial in the
documentation. The documentation that exists is easy to read, thorough, and fairly
clear, with a Quick Reference section explaining what metaclasses and blobs are, how
to do various operations, and how some things work in NeoPersist; a section on the
application, document, and file handler classes for use with MacApp; a section on
designing a document class using NeoPersist with TCL, and full descriptions of
CNeoFile, CNeoPersist, CNeoBlob, and CNeoMetaClass. There are also three sample
applications provided on disk, one written with MacApp and two written with TCL. The
MacApp example is a rewrite of the Calc sample using NeoPersist. I found these
invaluable in figuring out how to go about writing an application using NeoPersist.
However, the package would benefit from a document that walks the developer through
the definition of a persistent object class, specifying which methods have to be
overridden and why, and which ones can be overridden and under what circumstances
one would want to override them. A section on setting up MacApp document and
application classes would also be helpful, as would some advice on how application
design might be affected by the use of NeoPersist. For instance, most applications have
some permanent data associated with each document (e.g. Calc documents store the
dimensions of the spreadsheet, the CalcMode, the allocated cells, and the edit row and
column). When using NeoPersist, you'll want (need) to create a persistent object to
hold the document's permanent data on disk. Then in DoRead() the document can either
retrieve the object, copy its data into the document, and release the object, or retrieve
the object and retain a reference to it.
The information belonging in a tutorial document is in the current documentation and
examples, but it is not always immediately apparent to the novice user. This makes the
learning curve feel artificially steep.
Creating an application
That said, NeoPersist is fairly easy to use once you figure it out. For the MacApp
developer, the first step is to create subclasses of CNeoApplication and CNeoDocument.
These are fairly straightforward. The application must override MakeNeoFile() and
add the application specific persistent classes to the metaclass table. The document
must override DoMakeFile() and add each of the application specific classes to the file
using CNeoFile::addClass(). The document's DoRead method will need to read in any
objects which should be resident in memory or are needed to track down other data,
e.g. the spreadsheet object to hold Calc's document data. The document may or may not
need to override DoWrite(). CNeoDocument::DoWrite() calls the file's sync method,
but some applications will need to update objects in the DoWrite() method before
synchronizing the file.
The interesting part of using NeoPersist is in creating the persistent object classes
and manipulating the objects. Several methods must be written for each persistent
object class. In order for objects to be retrieved, you must write a class method which
creates and initializes an object of that type, usually called New.
CNeoPersist *CComic::New(void)
CComic * aComic;
aComic = new CComic;
aComic->IComic();
return aComic;
}
You specify that class method when calling INeoMetaClass in the application's
MakeNeoFile() method.
// Add CComic class to metaclass table
metaClass = new CNeoMetaClass;
metaClass->INeoMetaClass(kComicID, kNeoPersistID, "\pCComic",
CComic::New, nil);
You must also override the methods getClassID(), getLength() and getFileLength(),
which should return, respectively, a unique ID for the class, the length of the class in
memory, and the length of the class on disk (which may differ from the in memory
length). For each class you must also provide I/O methods. In TCL this involves
overriding readObject and writeObject. In MacApp, you override ReadFrom() and
WriteTo(), calling the inherited method and then reading or writing the permanent
attributes of the object.
pascal void CComic::ReadFrom(TStream* aStream)
inherited::ReadFrom(aStream);
aStream->ReadString(fSeries, sizeof(fSeries));
aStream->ReadBytes(&fNumber, sizeof(fNumber));
aStream->ReadString(fPublisher, sizeof(fPublisher));
aStream->ReadString(fCondition, sizeof(fCondition));
aStream->ReadBytes(&fCoverPrice, sizeof(fCoverPrice));
aStream->ReadBytes(&fPurchasePrice, sizeof(fPurchasePrice));
aStream->ReadBytes(&fValue, sizeof(fValue));
}
pascal void CComic::WriteTo(TStream* aStream)
inherited::WriteTo(aStream);
aStream->WriteString(fSeries);
aStream->WriteBytes(&fNumber, sizeof(fNumber));
aStream->WriteString(fPublisher);
aStream->WriteString(fCondition);
aStream->WriteBytes(&fCoverPrice, sizeof(fCoverPrice));
aStream->WriteBytes(&fPurchasePrice, sizeof(fPurchasePrice));
aStream->WriteBytes(&fValue, sizeof(fValue));
}
Manipulating the objects is fairly straightforward. Adding objects to the file and
removing them from the file are accomplished easily with the files addObject() and
removeObject() methods. Updating an object is as simple as calling
anObject->setDirty() and then calling the file's sync() method. To retrieve an object,
you can use any of search methods provided by CNeoPersist. NeoPersist keeps track of
the number of references to objects and releases an object's memory only if space is
low, there are no references to the object, and the object is not dirty. Retrieving an
object automatically creates a reference to the object. To create additional references
to an object, you call anObject->referTo. To drop a reference, you call
anObject->unrefer.
One important thing to remember in manipulating persistent objects is that adding,
removing, and Find...ing an object all require a reference to the file object. This means
that you could find yourself passing a reference to the file object around quite a bit.
The Bottom line
Is NeoPersist for you? That probably depends on the application. If you don't need to
keep track of data from multiple object classes for a single document and you can
easily keep all of your data in memory, then NeoPersist probably won't gain you
anything. If on the other hand, you need to store objects from a number of different
objects or you need multiple indexes, then you may want to look beyond NeoPersist to
NeoAccess.
However, applications that need to store objects of a few different classes or that want
to keep their objects on disk, reading them in as needed, will probably be much easier
to write with NeoPersist, especially once the initial learning curve is conquered. The
API is fairly simple and straightforward. And to bring up the ever-popular issue of
cross platform development, NeoPersist itself is currently available for the Macintosh
with both MacApp and TCL. Other platforms and/or frameworks require moving up to
NeoAccess, which is a cross-platform class library .