Wrapping up a .NET dataset in a Delphi class
This paper describes the inner workings of a Delphi class which wraps up a
.net XML dataset. It is used in the GekkoDotNetDataset
component to receive the XML from a webservice. The class has a method to extract the XML
dataset from a SOAP-response, publishes all data in a (number of) ClientDataSet(s) and has a method which returns
an XML diffgram reflecting any updates
made in the clientdataset's.
Introducing tDNdataset
To wrap up the XMLdataset and expose its functionality in a manner which
other Delphi components as well as the webservice understand, I will create the tDNdataset
class. If you are not familiar with XML datasets, I suggest you read my paper on
data in .NET which introduces them from a Delphi
perspective. For now it is important to note the following things :
- An XML dataset is an XML document, the tDNdataset class will inherit from
the VCL tXMLdocument class.
- An XML dataset can house several tables of data, in the tDNdataset these
tables are available as an array of tClienDataSets
- All incoming responses from a webservice are available as a stream, a
tDNdataset object can be instantiated from a stream
- All updates of the data are sent to the webservice as an XML
diffgram, which is another XMLdocument. The tDNdataset class has a
method which generates this document.
- In a diffgram will be info on all updates of the data. To track
these tDNdatset uses the tDNdatasetTable class which directly
inherits from tClientDataSet and does the necessary bookkeeping.
The array of tables is implemented as a property with a Variant as indexer.
This way the tables can be referenced by tablename and by index
property DataTable[TableIndex : variant] : tDNdataSetTable read
GetDataTable;
function tDNdataSet.GetDataTable(TableIndex : variant): tDNdataSetTable;
var i : integer;
begin
if tVarData(TableIndex).VType = VarInteger then
begin
result:= fDataStore[tVarData(TableIndex).Vinteger];
end
else
begin
i:= 0;
while fDataStore[i].Name <> TableIndex do
inc(i);
result:= fDataStor[i];
end;
end;
Data is represented as text in the XML. The code assumes that all
localization formatting uses standard settings. To do so the class has two
helper functions which set and reset the localization settings.
procedure tDNdataSet.SetFormatIntnl;
begin
CurrentDecimalSeparator:= DecimalSeparator;
CurrentDateFormat:= ShortDateFormat;
DecimalSeparator:= '.';
ShortDateFormat:= 'yyyy-mm-dd';
end;
procedure tDNdataSet.SetFormatLocal;
begin
DecimalSeparator:= CurrentDecimalSeparator;
ShortDateFormat:= CurrentDateFormat;
end;
The current settings are stored in private fields of the class.
Reading the XMLdataset
Analyzing the schema
After calling its constructor the CreateFromStream method has to be called to
instantiate the dataset. This method will start by filling itself from the stream
which holds an entire SOAP response.
// Construct the datadocument from the stream (self is (also)
a tXMLdocument)
self.LoadFromStream(aStream);
Now I can step through all the nodes of the dataset using tXMLdocument's
properties and methods. To get an idea of this let's take a look at this helper
method:
function tDNdataSet.GetSubNode(const startNode: IxmlNode; NodeName : string): IxmlNode;
begin
result:= startNode;
while (result <> nil) and (result.LocalName <> NodeName) do
result:= result.ChildNodes[0];
end;
This function starts on a node passed in the first parameter and will search
down the tree until it finds a node named as the second parameter. (Note that
the method assumes it will find the node, there is no check on the availability
of childnodes !)
First I will navigate to the actual response of the webmethod:
// Set context to result node of webmethod
Icontext:= self.DocumentElement.ChildNodes[0].ChildNodes[0].ChildNodes[0];
A diffgram returned by the webservice always has the same structure. It's
first subnode holds the schema of the dataset and is named schema, the
second subnode holds the data and is named diffgram. The code will check
for these nodes to test if the document is an XML dataset.
// Does the node have exactly 2 subnodes ? Schema and diffgram ?
if (Icontext.ChildNodes.Count = 2) and
(Icontext.ChildNodes[0].LocalName = SchemaAttribute) and
(Icontext.ChildNodes[1].LocalName = DiffGramAttribute) then
Now the ID and the namespace of the dataset
are read from the attributes. The actual schema is the first <element>
subnode. Both schema node and datanode are stored in properties of the class.
// It looks like a DotNet dataset .....
fNameSpace:= Icontext.ChildNodes[0].Attributes[AttrNSname];
fDataSetID:= Icontext.ChildNodes[0].Attributes[AttrIDname];
// The first <element> is the actual schema
SchemaNode:= GetSubNode(Icontext.ChildNodes[0], AttrNameElement);
// The second subnode of the context is the diffgram
DataNode:= Icontext.ChildNodes[1];
Before really extracting the schema there is one more test, the schemanode
should have an attribute named IsDataSet and this attribute should have true
as value
if SchemaNode.Attributes[IsDataSetAttribute] = TrueValue then
// It is a Dataset !
The tables described in the schema can now be looped through, the code will
process them one by one and create datasets
out of the definitions. The resulting dataset is added to the array
TableNode:= GetSubNode(SchemaNode.ChildNodes[0], AttrNameElement);
// Loop through all tabledefs in the schema's
while (TableNode <> nil) do
begin
SetLength(fDataArray, length(fDataArray) + 1);
fDataArray[length(fDataArray) -1]:= CreateDataSetTable(TableNode);
// Next table
TableNode:= TableNode.NextSibling;
end;
The datasets are ready, they can be filled from the datanodes in the
document. The dataset has the RecordFromNode method which will create and fill a record from a
node in the document
// Set to the first table
Icontext:= DataNode.ChildNodes[0].ChildNodes[0];
// Loop the tables
while Icontext <> nil do
begin
self.DataTable[Icontext.LocalName].RecordFromNode(Icontext);
Icontext:= Icontext.NextSibling;
end;
Finally the logging of updates in the table is initiated
// Activate the changes log
for i:= 0 to length(fDataArray) -1 do
self.DataTable[i].Activate;
Creating the DNdataset object.
To sum things up
procedure tDNdataSet.CreateFromStream(aStream: tStream);
var Icontext : IxmlNode;
TableNode : IxmlNode;
i : integer;
begin
SetFormatIntnl;
// Construct the datadocument from the stream (self is (also) a tXMLdocument)
self.LoadFromStream(aStream);
// Set context to result node of webmethod
Icontext:= self.DocumentElement.ChildNodes[0].ChildNodes[0].ChildNodes[0];
// Does the node have exactly 2 subnodes ? Schema and diffgram ?
if (Icontext.ChildNodes.Count = 2) and
(Icontext.ChildNodes[0].LocalName = SchemaAttribute) and
(Icontext.ChildNodes[1].LocalName = DiffGramAttribute) then
begin
// It looks like a DotNet dataset .....
fNameSpace:= Icontext.ChildNodes[0].Attributes[AttrNSname];
fDataSetID:= Icontext.ChildNodes[0].Attributes[AttrIDname];
// The first <element> is the actual schema
SchemaNode:= GetSubNode(Icontext.ChildNodes[0], AttrNameElement);
// The second subnode of the context is the diffgram
DataNode:= Icontext.ChildNodes[1];
if SchemaNode.Attributes[IsDataSetAttribute] = TrueValue then
// It is a Dataset !
begin
// Create tables get the first tabledef in schema
TableNode:= GetSubNode(SchemaNode.ChildNodes[0], AttrNameElement);
// Loop through all tabledefs in the schema's
while (TableNode <> nil) do
begin
SetLength(fDataArray, length(fDataArray) + 1);
fDataArray[length(fDataArray) -1]:= CreateDataSetTable(TableNode);
// Next table
TableNode:= TableNode.NextSibling;
end;
// Transfer the data in the xml to the clientdataset
// Set to the first table
Icontext:= DataNode.ChildNodes[0].ChildNodes[0];
// Loop the tables
while Icontext <> nil do
begin
self.DataTable[Icontext.LocalName].RecordFromNode(Icontext);
Icontext:= Icontext.NextSibling;
end;
// Activate the changes log
for i:= 0 to length(fDataArray) -1 do
self.DataTable[i].Activate;
end;
end;
SetFormatLocal;
end;
All table data will be wrapped up in a ClientDataSet. I will build tDNdataSetTable, a class which directly inherits from
tClientDataset. In there some code takes care of the update functionality.
Creating the tDataSet and filling it with data is done just like in the ancestor
tClientDataSet. Every row in a diffgram has a diffgr:id attribute to
identify the row. For this Id an extra field is created in the new table. A
second field will record the state of the row, when the row is inserted or
updated this field will hold this status.
result:= tDNdataSetTable.create(self);
result.Name:= TableNode.Attributes[NameAttribute];
result.FieldDefs.Add(idFieldName , ftString, 30);
result.FieldDefs.Add(statusFieldName , ftWord);
The field nodes are iterated, some fieldproperties can be directly read from
the node's attributes
FieldNode:= GetSubNode(TableNode.ChildNodes[0], AttrNameElement);
while FieldNode <> nil do
begin
dfTypeName:= FieldNode.Attributes[TypeAttribute];
dfRequired:= FieldNode.Attributes[RequiredAttribute] = FalseValue;
dfReadOnly:= FieldNode.Attributes[ReadOnlyAttribute] =
TrueValue;
.............
FieldNode:= FieldNode.NextSibling;
end;
This information is used to create a field definition
if dfTypeName = TypeIntegerAttribute then
begin
dfType:= ftInteger;
dfSize:= 0;
end
else if dfTypeName = TypeDecimalAttribute then
begin
dfType:= ftCurrency;
dfSize:= 0;
end
else if dfTypeName = TypeDateTimeAttribute then
begin
dfType:= ftDateTime;
dfSize:= 0;
end
else
begin
dfType:= ftString;
dfSize:= 30;
end;
Note that this list of fieldtypes is not complete at al, for the sake of demo
I have only included the most common fields. Another problem is the size of a
stringfield. The contents found in the diffgram could be any size and there is
no information in the schema on the sizes to expect.
Based on these values I can now create the field
result.FieldDefs.Add(FieldNode.Attributes[NameAttribute], dfType, dfSize, dfRequired);
if dfReadOnly then
result.FieldDefs[result.FieldDefs.Count -1].Attributes:= result.FieldDefs[result.FieldDefs.Count -1].Attributes +
[Db.faReadOnly];
When all nodes are processed the dataset is actually created by calling
tClientDataSet's CreateDataSet method.
result.CreateDataSet;
To sum things up :
function tDNdataSet.CreateDataSetTable(const TableNode: IxmlNode): tDNdataSetTable;
var dfTypeName : string;
dfType : tFieldType;
dfSize : integer;
dfRequired : boolean;
dfReadOnly : boolean;
FieldNode: IxmlNode;
begin
result:= tDNdataSetTable.create(self);
result.Name:= TableNode.Attributes[NameAttribute];
result.FieldDefs.Add(idFieldName , ftString, 30);
result.FieldDefs.Add(statusFieldName , ftWord);
FieldNode:= GetSubNode(TableNode.ChildNodes[0], AttrNameElement);
while FieldNode <> nil do
begin
dfTypeName:= FieldNode.Attributes[TypeAttribute];
dfRequired:= FieldNode.Attributes[RequiredAttribute] = FalseValue;
dfReadOnly:= FieldNode.Attributes[ReadOnlyAttribute] = TrueValue;
if dfTypeName = TypeIntegerAttribute then
begin
dfType:= ftInteger;
dfSize:= 0;
end
else if dfTypeName = TypeDecimalAttribute then
begin
dfType:= ftCurrency;
dfSize:= 0;
end
else if dfTypeName = TypeDateTimeAttribute then
begin
dfType:= ftDateTime;
dfSize:= 0;
end
else
begin
dfType:= ftString;
dfSize:= 30;
end;
result.FieldDefs.Add(FieldNode.Attributes[NameAttribute], dfType, dfSize, dfRequired);
if dfReadOnly then
result.FieldDefs[result.FieldDefs.Count -1].Attributes:= result.FieldDefs[result.FieldDefs.Count -1].Attributes + [Db.faReadOnly];
FieldNode:= FieldNode.NextSibling;
end;
result.CreateDataSet;
end;
The tDNdataSetTable class
The tDNdataSetTable class adds some extra functionality to tClientdataset :
- All records have an extra identifying diffgramId field.
- All records have an extra status field.
- The dataset can create a new record from an xmlNode.
- The dataset can produce the contents of a record as an xmlNode.
The extra diffgramId field will be used as the
active index. The table will maintain an own recordcounter to correctly
calculate new id's. Using the the RecNo property of the dataset would result in
problems when records are deleted before new records are added. A helper object
lists all changes, the loading flag will explicitly activate this object. All
these variables are initialized in the AfterConstruction method.
procedure tDNdataSetTable.AfterConstruction;
begin
inherited;
self.IndexFieldNames:= idFieldName;
RecCounter:= 0;
Loading:= True;
end;
Creating a new record from a node is handled by the RecordFromNode
method. This method will activate the dataset when necessary, increment
the recordcounter, fill the IdField from the ID
attribute of the node and set the status of the row to unchanged
procedure tDNdataSetTable.RecordFromNode(const RowNode : IxmlNode);
var FieldNodes : IxmlNodeList;
i : integer;
aField : tField;
begin
if not self.Active then
self.Active:= True;
inc(RecCounter);
self.Insert;
self.FieldByName(idFieldName).Value:=
RowNode.Attributes[DiffGrIdAttribute];
self.FieldByName(StatusFieldName).Value:= rsUnChanged;
The node passed in the RowNode parameter will look like this
<Customers diffgr:id="Customers3" msdata:rowOrder="2">
<Address1>Pleasant Grove</Address1>
<Address2>54D East Grindstead</Address2>
<idCustomer>3</idCustomer>
<Name>Peter's place</Name>
</Customers>
The name of the subnode is the name of the field, the value of the fields is
between the tags. By iterating the collection of subnodes the fields are filled.
The type of the field determines how the text between the tags has to be
translated to a tField value
FieldNodes:= RowNode.ChildNodes;
for i:= 0 to FieldNodes.Count -1 do
begin
aField:= self.FieldByname(FieldNodes[i].LocalName);
try
case aField.DataType of
ftDateTime : aField.Value:= StrToDate(copy(FieldNodes[i].Text, 0, 10));
ftCurrency : aField.Value:= StrToFloat(FieldNodes[i].Text);
else aField.Value:= FieldNodes[i].Text;
end;
except
end;
end;
self.Post;
Note that I make implicit assumptions about the date and numberformat. The
format has been set to fixed values. Not all field-types are supported (yet) in this implementation, to catch any
problems the assignment is placed in a try except block.
Now we have a Delphi class which can be instantiated from an XML dataset
whose data can be browsed in (a couple of) clientdataset(s).
Updating the XMLdataSet
Logging the updates
Updates in the dataset are posted back as a
diffgram. In this diffgram will
be the status of every row as well as the previous contents of modified
and deleted records. So for every row I have to log it's state. The possible
states are described by a type
type tRowState = (rsUnChanged, rsModified, rsInserted,
rsDeleted);
The actual state of a record is stored in the extra statusField.
A row can be updated several times during a session. A row already marked as
Inserted should not be marked as modified. When transferring data to the database
it is of no importance to know that the row has been inserted and modified
again, to the database it is only important to know that is a new row. The
actual update is performed when the record is posted, to hook in I
override the Post method of tDNdataSetTable. Before performing the actual
post in the inherited implementation the changes will be recorded in the status
field. This is only done when the dataset is not loading, the loading
flag indicates the dataset is filled from the original dataset.
procedure tDNdataSetTable.Post;
begin
if not Loading then
begin
case self.State of
dsInsert : begin
inc(RecCounter);
self.FieldByName(idFieldName).AsString:= Format('%s%d',[self.Name,
self.RecCounter]);
self.FieldByName(StatusFieldName).Value:= rsInserted;
end;
dsEdit :
if self.FieldByName(StatusFieldName).Value <> rsInserted then
self.FieldByName(StatusFieldName).Value:= rsModified;
end;
end;
inherited;
end;
For a new row I have to make up a new DiffgramId, this can be assembled
from the name of the dataset and my own recordcounter.
Assembling a diffgram with updates
The .NET webmethod needs of parameter containing an XML diffgram to update the
database. What I have to do is construct the diffgram based on the original
diffgram passed in, the data in the clientdatasets and the changes in the log. I
will use an tXmlDocument again to construct this diffgram.
The UpdateXML method will create a document whose root element will be named according
to a parameter passed in. This root will have two sub-nodes, one with the schema and
one with the data. The schema node is created, its attributes are set and then
the whole schema can be copied (cloned) in one go using the SchemaNode property of
the tDNdataSet class. The locale format is set to USintnl in the beginning and
reset when finished.
SetFormatIntnl;
xmlDoc:= tXMLdocument.Create(nil);
xmlDoc.Active:= true;
Iroot:= xmlDoc.AddChild(ParamName);
// Schema
Icontext:= Iroot.AddChild(SchemaxsAttribute);
Icontext.Attributes[IdAttribute]:= self.DataSetId;
Icontext.Attributes[NStargetAttribute]:= self.NameSpace;
Icontext.Attributes[NSmstnsAttribute]:= self.NameSpace;
Icontext.Attributes[NSxmlnsAttribute]:= self.NameSpace;
Icontext.Attributes[NSxmlnsXsAttribute]:= NSxs;
Icontext.Attributes[NSmsDataAttribute]:= NSmsData;
Icontext.Attributes[attributeFormDefaultAttribute]:= Qualified;
Icontext.Attributes[elementFormDefaultAttribute]:= Qualified;
Icontext.ChildNodes.Add(SchemaNode.CloneNode(true));
Next the data node is created and it's attributes are set
// Diffgram data
Idiffgram:= Iroot.AddChild(DiffgramDgAttribute);
Idiffgram.Attributes[NSmsDataAttribute]:= NSmsData;
Idiffgram.Attributes[NSdiffgrAttribute]:= NSdiffgr;
Icontext:= Idiffgram.AddChild(self.DataSetId, '');
Icontext.Attributes[NSxmlnsAttribute]:= self.NameSpace;
Next come the the datanodes. To fill these the tDNdataSetTable Class has a DiffGramNodes method.
Which will append all rows as childnodes to the node passed in the parameter.
procedure tDNdataSetTable.DiffGramNodes(const atNode: IxmlNode);
begin
self.First;
while not self.eof do
begin
self.NodeFromRecord(atNode);
self.Next;
end;
end;
All this method does is loop through all records and call the (private) NodeFromRecord
Method. The method will create a node for the row and set it's DiffgrIdAttribute
to the value found in the Id field. The RowOrderAttribute and the HasChangesAttribute
is set as well after which the field sub-nodes can be written. The diffgram
expects a string to describe the row status, which only has
to be set for changed rows. After the row's attributes are set the data of the
fields can be written. This only has to be done for fields with content, all
null values can be skipped.
function tDNdataSetTable.NodeFromRecord(const atNode : IxmlNode{; UseFormat: tFormatSettings}): IxmlNode;
var i : integer;
ItmpNode : IxmlNode;
RowState : tRowState;
RowStateVerbose : string;
aField : tField;
buffer : string;
begin
result:= atNode.AddChild(self.name, '');
result.Attributes[DiffgrIdAttribute]:=
self.FieldByName(idFieldName).AsString;
result.Attributes[RowOrderAttribute]:= Format('%d', [RecNo - 1]);
RowState:= self.FieldByName(StatusFieldName).Value;
if RowState <> rsUnChanged then
begin
case RowState of
rsModified : RowStateVerbose:= 'modified';
rsInserted : RowStateVerbose:= 'inserted';
rsDeleted : RowStateVerbose:= 'deleted';
end;
result.Attributes[HasChangesAttribute]:= RowStateVerbose;
end;
for i:= 0 to self.Fields.count - 1 do
begin
aField:= self.Fields[i];
if aField.FieldName <> idFieldName then
if not aField.IsNull then
begin
ItmpNode:= result.AddChild(aField.FieldName, '');
case aField.DataType of
ftCurrency : ItmpNode.Text:= FloatToStr(aField.Value);
ftDateTime : begin
DateTimeToString(buffer, 'yyyy-mm-dd', aField.Value);
ItmpNode.Text:= buffer;
end;
else ItmpNode.Text:= aField.Value;
end;
end;
end;
end;
Again I make implicit assumptions of the localization
format in which the fields will be written.
An XML DataSet can have multiple tables. When assembling the updateXML the
code will loop all tables and let them create the datanodes.
for i:= 0 to self.TableCount -1 do
self.DataTable[i].DiffGramNodes(Icontext);
If there are any changes the XML needs a diffgr:before node which will
hold all the original data from the modified rows as well as all the data from
the deleted rows. All original data is still available, it was originally loaded
from the stream and then copied into the clientdataset's. The latter have been
updated but the former data is still unmodified and available via the DataNode property.
The code loops through all original
datanodes, checks the status of the row and will copy the original datanode when
the row had been changed.
Icontext:= DataNode.ChildNodes[0].ChildNodes[0];
while Icontext <> nil do
begin
if DataTable[Icontext.LocalName].IsChanged(Icontext.Attributes[DiffgrIdAttribute]) then
begin
if Ibefore = nil then
// Create the diffgr:before node
Ibefore:= Idiffgram.AddChild(DiffgramBeforeAttribute);
Ibefore.ChildNodes.Add(Icontext.CloneNode(true));
end;
Icontext:= Icontext.NextSibling;
end;
To finish the locale format is reset
SetFormatLocal;
Now the diffgram is complete and I can return it's textual representation.
result:= xmlDoc.XML.Text;
Where are we ?
All complexities of an XML dataset are wrapped up in the tDNdataSetClass. Objects of the class are
instantiated by a stream which holds a complete SOAP response. The data will
be available in a number of tClientDataSet (derived) tables. To send any updates
in the data back to a webservice the class has an UpdateXML method which
returns one big XML string describing the updates. The tDNdatSetClass is the
core of the GekkoDotNetDataset
component.
What's next
|