Passing data in and out of web services using (typed) xml (-datasets)

This article is based on a demo VS.NET solution (which can be downloaded here). This solution contains three projects:

1. WebDataService. This assembly implements the two web services, TypedDataService and UnTypedDataService.
2. UntypedDataConsumer. An application for a pocketPC which consumes the untyped service.
3. TypedDataConsumer. A winform application which consumes the typed service.

The article assumes you are familiar with building applications with Visual Studio.NET and the .NET framework. It is the sketch of a part of my upcoming Apress book on Whidbey. You are invited to play with the ideas presented here and feel free to comment on them.

Introduction

I have always been in favor of strong typing. Variables should be declared as the most descriptive type possible. Working that way can lead to more reliable and faster applications. The compiler does the work, not the run time system. In .net you can type your database data just as strong as native vars. The framework generates class wrappers on xsd schema's. These classes are available as typed datasets and include typed members for all fields. Inside your C# (or vb.net , etc) code these classes are quite a blessing to work with. When it comes to building web services you can publish your data as strongly typed (using XSD) datasets. Whether this is always desired is a different matter. In this article I will take a look at several variations on a web service which returns and accepts xml-data.

A web service with typed datasets

In one of my contributions for the dotnetjunkies I have described in detail a webservice which works with strongly typed datasets. The service returns database data in XML datasets with an embedded schema and it accepts strongly typed datasets to update the database. A .net windows forms application can work with this web service and edit the data in a datagrid. Works like a charm, a rich client communicating with its database over plain HTTP. The schema describing the dataset is known at design time, all controls and code know in detail (down to specific field properties) what to expect. When editing the received dataset in a datagrid the application itself will report any invalidations in the data. The winform application Included in the demo project does this same trick.

At first sight this looked to me as the Nirvana of object orientation. All my data was available as a strongly typed dataset object and I had a distributed app. I still consider it great but heaven is only a small place. It works well in an application which uses the full .net framework, but it does not work at all in an environment which is not that rich. Like the .net compact framework. Referencing the web service alone is enough to break your smart (device) application. The compiler will trip over the imported typed dataset class. The compact framework does not support typed datasets at all. Thank goodness it does support the DataSet base class.

A web service with untyped datasets

I will create a new web service which works with a looser definition of the data, it will expose it as a DataSet object. I don't have to start a new project for every new web-service. In the project I already have a dataset schema which describes the data. (Two tables, a lookup with brands and a working table with instruments. The keys and the relation between the two tables are described in this part of the schema

<xs:unique name="DataSetInstrumentsKey1" msdata:PrimaryKey="true">
   <xs:selector xpath=".//mstns:Brands" />
   <xs:field xpath="mstns:idBrand" />
</xs:unique>

<xs:unique name="DataSetInstrumentsKey2" msdata:PrimaryKey="true">
   <xs:selector xpath=".//mstns:Instruments" />
   <xs:field xpath="mstns:idInstrument" />
</xs:unique>

<xs:keyref name="BrandsInstruments" refer="mstns:DataSetInstrumentsKey1">
   <xs:selector xpath=".//mstns:Instruments" />
   <xs:field xpath="mstns:idBrand" />
</xs:keyref>

The project has a component which handles the actual database access. The main methods of the component return and accept data.

public DataSetInstruments Data()
{
   DataSetInstruments ds = new DataSetInstruments();
   sqlDataAdapter2.Fill(ds.Brands);
   sqlDataAdapter1.Fill(ds.Instruments);
   return ds;
}

public string UpdateInstruments(DataSetInstruments ds)
{
   string result = "OK";
   try
   {
      sqlDataAdapter1.Update(ds.Instruments);
   }
   catch(Exception e)
   {
      result = e.Message;
   }
   return result;
}

Both web services in the project are built on these methods. The typed web services goes like this :

public class TypedDataService : System.Web.Services.WebService
{

   [WebMethod]
   public DataSetInstruments Instruments()
   {
      using (DataComponent dc = new DataComponent(this.Container))
         { return dc.Data();}
   }

   [WebMethod]
   public string UpdateInstruments(DataSetInstruments ds)
   {
      using (DataComponent dc = new DataComponent(this.Container))
      { return dc.UpdateInstruments(ds);}
   }

}

public class UntypedDataService : System.Web.Services.WebService
{
   [WebMethod]
   public DataSet InstrumentsDataSet()
   {
      using (DataComponent dc = new DataComponent(this.Container))
      { return dc.Data();}
   }

   [WebMethod]
   public string UpdateInstruments(DataSet ds)
   {
      using (DataComponent dc = new DataComponent(this.Container))
      { return dc.UpdateInstruments(ds); }
   }

}

This untyped webservice can be used in a .net compact framework application. The app really understands the dataset and its schema. If you edit the data this will be validated against the schema and the smart app will throw an exception when the validation fails.

A webservice with XML strings

A smart device app works well with the second web service. But there is a drawback. The DataSets coming in and out of the service are all very heavily decorated and all contain the schema of the dataset. Enlarging the amount of data on the wire. In a scenario with limited bandwidth, like most smart device apps, this is a problem. Another drawback with the web services is that not all web service consumers do understand the DataSet type. Every consumer understands strings and numbers but when it comes to a complex type like a DataSet many will give up. For instance Delphi. The good thing with a tool like that is that it has a large library of classes and full access to an XML parser. Using this I built a Delphi component which analyzes the schema, builds a collection of data-tables and fills these with the contents of the dataset. (Full story here). In other tools there is just no way to make them understand a DataSet.

A third version of the webservice just passes the basic XML, without any decoration or schema, as a plain string. Reading the data from the dataset in this format is a matter of using one of its other methods:

[WebMethod]
public string Instruments()
{
   using (DataComponent dc = new DataComponent(this.Container))
   {
      return dc.Data().GetXml();
   }
}

The GetXml method returns the bare minimum XML representation of the data.

The compact application of the demo has do a little work to get the data into a grid.

DataSet ds = new DataSet();

UntypedDataService dws = new UntypedDataService();
System.IO.StringReader sr = new System.IO.StringReader(dws.Instruments());
ds.ReadXml(new System.Xml.XmlTextReader(sr)) ;
ds.AcceptChanges();

dataGrid1.DataSource = ds.Tables["Instruments"];

You need a StringReader to read the xml string and a XmlTextReader to get it into the dataset. If you don't make a call to AcceptChanges the RowState of all rows will read Added.

After editing the dataset is sent back to the service to have that update the database

DocumentServices.UntypedDataService dws = new CassiKijken.DocumentServices.UntypedDataService();
dws.UpdateInstruments(ds.GetXml());

For the service to correctly write the data to the database the web service has to do some work to match incoming rows with existing rows

[WebMethod]
public string UpdateInstruments(string onXML)
{
   using (DataComponent dc = new DataComponent(this.Container))
   {
      System.IO.StringReader sr = new System.IO.StringReader(onXML);
      DataSetInstruments dsNew = new DataSetInstruments();
      dsNew.ReadXml(sr);

      DataSetInstruments dsOld = dc.Data();
      dsOld.Merge(dsNew);

      return dc.UpdateInstruments(dsOld);

   }
}

First the webmethod builds a new (typed!) dataset and loads it with the Xml passed in as a string. Next it creates a second dataset and loads it with the data in the database. The updated data are then Merged into this second dataset, the id field will bring together the different versions of the rows. Note that this way of working does see updates in a row as well as new rows but misses the deletion of rows.

Let the web service do the validation

The consumer of this simplest web service is not passed the schema of the data. Which implies that it cannot validate the data on its own. You can put anything in the dataset and will not notice any problems until the moment the service tries to write the updates to the database. A solution for this would be to let the web service do the validation. I will add a web method which accepts a string with Xml data to be validated and returns any problems found.

The actual handling of XML in the framework is done by the XMLreader and XMLwriter classes. Amongst them is XmlValidatingReader, instances of this class read a stream of XML and validate it against any schema assigned to it. When the xml does not comply to the schema an exception is thrown. The class has a ValidationEventHandler,  if you register an eventhandler here the event will fire instead of the exception being thrown. Let's start with a method which constructs the validating reader

private XmlValidatingReader validator(string onXML)
{
   System.IO.StringReader sr = new System.IO.StringReader(onXML);
   XmlTextReader xtr = new System.Xml.XmlTextReader(sr);
   XmlValidatingReader result = new XmlValidatingReader(xtr);
   result.ValidationType = System.Xml.ValidationType.Schema;

   DataSetInstruments ds = new DataSetInstruments();
   XmlSchema xs = XmlSchema.Read(new System.IO.StringReader(ds.GetXmlSchema()), null);
   xs.Compile(null);
   result.Schemas.Add(xs);

   return result;
}

In the first four lines the reader is constructed and its validation type is set to xsd schema. In the next four lines the schema is extracted from the (typed) dataset, compiled and added to the validator. The result is a reader which will validate the Xml passed in using the schema of the expected dataset. It will construct a list of any error messages, the event handler collects the messages in a stringbuilder:

private System.Text.StringBuilder vm;

private void validationEvent(object o, ValidationEventArgs e)
{
   vm.Append(e.Severity.ToString());
   vm.Append("   ");
   vm.Append(e.Message);
   vm.Append(System.Environment.NewLine);
}

In the validating web method all of this is tied together:

[WebMethod]
public string ValidateWithInfo(string xmlData)
{
   vm = new System.Text.StringBuilder("");
   XmlValidatingReader xvr = validator(xmlData);
   xvr.ValidationEventHandler +=new ValidationEventHandler(validationEvent);

   while (xvr.Read()) {};

   return vm.ToString();
}

The stringbuilder is initialized, a validating reader for the xml passed in is constructed and the event handler is assigned to it. Now the reading can start, any errors found will be collected by the event handler. The result is returned by the web method, if everything was OK this will be an empty string.

Validating the data from a consumer now boils down to invoking the web service

UntypedDataService dws = new UntypedDataService();
textBoxValidation.Text = dws.ValidateWithInfo(ds.GetXml());

Don't overestimate the validator

The Xml passing or failing the validation test is not a guarantee for a (un-)successful database update. Lets take two examples. If the xml does not contain any values for the id field the validation will fail as the id is required and should be unique. The database doesn't care, the id's are generated by the database. The validation failed but the update succeeded. It can be the other way round as well. If the foreign keys values in the data are found in the lookup table of the dataset the validation will pass. In the update the database will search the key-value in the database table, if the key is gone the update will fail.

A schema can help you with an idea what the data should look like. But the only way of finding out if all assumptions are really valid remains a question of trial and error, in C# speak that is try and catch. But validating against a schema does help to dramatically increase your rate of success.