Automation collections

Collections are a part of automation. Languages like VBA offer language constructs which work with automation collections. A lot of methods and API calls use an automation collection as a parameter. In this chapter I will wrap up a stringlist inside a Delphi component and describe step by step how to publish the strings contained as an automation collection.

A programmers way to look at collections

In the file-manager example the user can select and deselect multiple files. To program this ever changing bunch of selected files in code I could use an array, but that would not be the most handy thing. It would take a lot of resizing and would create quite some confusion as the file corresponding to array element #2 could be element #4 a few clicks later. 

A better way to work with the selected files is a collection. A collection is just a bunch of related items which you can walk through. You start at the beginning, step through the items (mostly) one at a time and that's just all. The world of Windows is full of collections. Some good examples are the windows in an application, the controls on a form, the computers logged on to the network, etc. etc.

IenumVariant describes an automation collection

The Ienumvariant interface is part of the automation specification. Clients of Automation collections clients can check a property for this interface and use the interface to get to all the items. It is often seen like this in VB(A) :

Sub ListFiles()
   Dim File As SelectedFile 
   For Each File In MyManager.Selected
      Selection.InsertAfter (File.Name)
      Selection.InsertParagraphAfter
   Next
End Sub

Behind the scenes the code  takes the selected property of my Filemanager, tries to extract it's Ienumvariant interface and will iterate through all the selected files using the methods of Ienumvariant.

Ienumvariant is described as follows in ActiveX.pas

{$EXTERNALSYM IEnumVariant}
IEnumVariant = interface(IUnknown)
   ['{00020404-0000-0000-C000-000000000046}']
   function Next(celt: LongWord; var rgvar : OleVariant;  out pceltFetched: LongWord): HResult; stdcall;
   function Skip(celt: LongWord): HResult; stdcall;
   function Reset: HResult; stdcall;
   function Clone(out Enum: IEnumVariant): HResult; stdcall;
end;

It is a very simple interface having only four methods. If I implement these four methods in my selected property the VBA snippet should work.

Preparing the project

To make the selectedfilesproperty an automation  collection I first have to do some fiddling with the project. In the former version I had implemented Selected as an indexed property of type Iselectedfile. Now I will change the type to a new automation class, a class which does implement IenumVariant and will be a collection.

I will create the class, named SelectedFileCollection, the same way as I built the SelectedFile class in the chapter on advanced properties. Beside the collection methods this class has two other properties.  An indexed property item to retrieve an individual file and an integer count property which stands for the number of selected files. The result-type of the selected property in the file-manager was IselectedFile. It is changed to IselectedFileCollection, a collection of IselectedFile objects.

Every time the selected collection property is used a new SelectedfileCollection object is created by the selected property getter. It will iterate the filelistbox component, add the selected files to a stringlist and pass this stringlist to the constructor of the SelectedFileCollection.

function TFileZapper.Get_Selected: ISelectedFileCollection;
   var i : integer;
   begin
   fSelectedFiles.Clear;
   for i:= 0 to Form1.FileListBox1.items.count - 1 do
      if Form1.FileListBox1.Selected[i] then
         fSelectedFiles.Add(Form1.FileListBox1.Items[i]);
   result:= tSelectedFileCollection.CreateEx(fSelectedFiles);
end;

The SelectedFileCollection class now has a list of all the selected files to work with. To retrieve an individual file, the getter of it's Item property uses this stringlist to find a filename. This is passed to the constructor of the SelectedFile object.

function TSelectedFileCollection.Get_Item(Index: Integer): ISelectedFile;
   begin
   if index < fFiles.Count then
      result:= tSelectedFile.CreateEx(fFiles[index]);
end;

Finally I will give the ISelectedfile interface a new read-only property name, so the file name can be read by a client. Now I have all the information ready to start implementing the Ienumvariant interface.

(Note: Actually I am committing a sin when I change an existing interface. An automation interface is supposed to be an immutable contract. Just imagine what would happen to  a client who was used to working with the former filemanager and is now confronted with this new interface. It has the same interface ID but a different signature for the selected property. The only reason for me rebuilding the existing project is for the sake of demo. But don't try this at home !)

IselectedFileCollection

Now the plan for the implementation of the collection is ready, it's time to take a look what has to be done in Delphi. The SelectedfileCollection is in the typelibrary, I have to do some more work there to define it as a class which implements a collection. To be one the CoSelectedfileCollection class has to implement the IenumVariant interface beside its default IselectedFileCollection interface. In the type library editor I add Ienumvariant in the implements tab :

The Delphi tSelectedFileCollection class implements three groups of methods, 

Count is a read-only property with a very straightforward implementation:

function TSelectedFileCollection.Get_Count: Integer;
   begin
   result:= fFiles.Count;
   end;

The number of selected files matches the count of the stringlist fFiles, which was passed in the constructor.

We have already seen  how to implement an indexed property. I add the index parameter to the item property.

Now item is an indexed property and I can get an individual selected item in the collection. (See the article on advanced properties for a full description). Item should be the default property of the SelectedFilesCollection object. Which means that the property name can be omitted in script or VBA code like this:

MyManager.Selected(1).Name

instead of

MyManager.Selected.Item(1).Name

To make the item property the default property it has to have a dispId of 0

and the item has to be flagged as the default collection item

Implementing IenumVariant 

After all this, the only thing left is implementing the methods of the Ienumvariant interface itself.

The method _NewEnum is not defined in IenumVariant but has to be implemented for VB(A) to work with a collection. The method is invoked by its fixed dispid of -4. It will return my collection as an Iunknown interface. A proper declaration in the typelibrary will lead to this text representation of the _NewEnum property :

[
propget, 
id(0xFFFFFFFC)
]
HRESULT _stdcall _NewEnum([out, retval] IUnknown ** Value );

The implementation of the method is straightforward.

function TSelectedFileCollection.Get__NewEnum: IUnknown;
   begin
   result:= self;
   fIndex:= 0;
   end;

The TSelectedFileCollection object itself can be passed as a result. As it inherits from tAutoIntfObject it does implement Iunknown.  Internaly the collection has an integer findex field which indicates the current element. This index of the collection is reset to the first item.

The reset method is even simpler. It resets the collection pointer to the first item

function TSelectedFileCollection.Reset: HResult;
   begin
   fIndex:= 0;
   result:= S_OK;
end;

I reset my internal pointer and report that that went well by returning S_OK.

By using the skip method the pointer in the collection has to be moved to another item. In the  parameter the number of items to skip is passed.

function TSelectedFileCollection.Skip(celt: LongWord): HResult;
   begin
   if (fIndex + celt) > (fFiles.count -1) then
      begin
      fIndex:= fFiles.Count -1;
      result:= S_FALSE;
      end
   else
      begin
      fIndex:= fIndex + celt;
      result:= S_OK;
      end;
end;

My implementation will check if it can skip as many items as desired. If the user of the collection tries to skip further than possible the pointer will be set to the last item and return S_FALSE. If my collection is large enough then the internal pointer is incremented and S_OK is returned.

Ienumvariant.Next, the workhorse of automation collections

All the real work is done in the next method. This method will have to return the actual items themselves. In the ActiveX.pas unit it is defined according to the official COM specs as:

   function Next(celt: LongWord; var rgvar : OleVariant;  out pceltFetched: LongWord): HResult; stdcall;

The first parameter is the number of items requested. The second one is an array of OLEvariants to hold the result and the third parameter indicates the number of items actually returned. There are some problems with this method declaration. First of all passing back a Delphi array of OLEvariants does not provide the user of the collection with enough information on the contents of the variants. And as a second problem many users of collections do not provide a pceltFetched parameter. When the parameter is typed as a longword it is almost impossible to check if anything was actually passed.

As an alternative the following declaration of the next methods circulates among Delphi automation collection builders:

  function Next(celt: LongWord; out elt; pceltFetched: PLongWord): HResult; stdcall;

Here the array for the items is completely untyped and the out parameters holding the number of items passed is declared as a pointer to a longword. Let's take a look at a working implementation of next:

function TSelectedFileCollection.Next(celt: LongWord; out elt; pceltFetched: pLongWord): HResult;

   var i : longword;
   Ifile : Idispatch;
   begin
   i:= 0;

   while (i < celt) and (fIndex < fFiles.Count) do
      begin
      // Prepare the item to return
      Ifile:= tSelectedFile.CreateEx(fFiles[fIndex]) as IselectedFile;
      Ifile._AddRef;
      // Return the type and the value of the returned collection element
      tVariantArgList(elt)[i].vt:= varDispatch;
      tVariantArgList(elt)[i].dispVal:= Pointer(Ifile);
      inc(i);
      inc(fIndex);
      if pceltFetched <> nil then
         inc(pceltFetched^);
      end;

   if (i >= celt) and  ((pceltFetched = nil) or (pceltFetched^ = celt)) then
      result:= S_OK
   else
      result:= S_FALSE;
   end;

The method will iterate through the fFiles stringlist starting on the current collection location given by fIndex. The object to be returned is created by a call to the constructor of tSelectedFile and stored in a local variable IFile, typed as Idispatch . This return value being an automation object introduces another problem to be tackled. After the IFile object is created and passed back there is a big chance that that nobody will have a reference to it. Which will mean that the refcount of the object (see objects and interfaces) can fall back to 0, resulting in a (self) destruction of the object. The official trick to do something about this is manually increase the refcount by calling an _AddRef. This is a technique you do not have to use very often in Delphi, but it is a good way to prevent a very serious crash in the user of your collection.

The elt parameter holds the result. The IenumVariant is a collection of variants, a typecast to a tVariantArglist (found in Activex.pas) is the thing to do. Each variant element (described in tVariantArg) is actually a record with two important fields. The vt field is the actual type of the value held and the second field, whose name differs for each type, holds (a pointer to) the data. This collection houses (Idispatch interfaces to) objects, so I will set the type as being a vardispatch. Its value is an Idispatch variable as a bare pointer. When I return the result in this format any user of my collection will recognize the collection.

The variants in the collection can hold different types. The implementation of the collection would differ only in the filling of the return values in this next method. If it were actually real numbers, you would code like this : 

      tVariantArgList(elt)[i].vt:= varDouble;
      tVariantArgList(elt)[i].dblVal:= MyDouble;

In the third parameter pceltFetched I have to pass back the number of items actually returned. The parameter is allocated by the user of the collection. I always have to test on the parameter being nil. When MS-Word uses my collection I will find that Word does not supply me the parameter. Word will always ask for just one item in a call to next and is apparently not interested in the number of items actually returned. The result of the method, being S_OK when the number requested equals the number returned, will do. 

Let' run a macro in Word

Dim MyManager As New FileManager.FileZapper

Sub UseManager()
   MyManager.Directory = "C:\"
   MyManager.Filemask = "*.*"
   MyManager.AllowDelete = False
End Sub
Sub ListFiles()
   Dim File As SelectedFile 
   For Each File In MyManager.Selected
      Selection.InsertAfter (File.Name)
      Selection.InsertParagraphAfter
   Next
End Sub

The macro will result in the names of the selected files being written in the current Word document.

Adding and removing items, preparing the class

An automation client can now browse through the selectedfiles using a standard automation interface. A next thing will be the possibility to add or remove items. The collection in my filemanager example represents the collection of items selected in the visual component, adding or deleting items would mean manipulating  the visual component itself.

Again I will have to do some work on SelectedFileCollection class. I used to pass a stringlist containing the names of the selected files to the constructor, now I will pass the filelistbox itself. The collection class keeps a private reference to the component in the fFileListbox variable. I will create three helper functions to work with the selected files in the component.

The private function count counts the actual number of selected files, it is used in the property getter of the interface's Count property.

function TSelectedFileCollection.Count: integer;
   var i : integer;
   begin
   result:= 0;
   for i:= 0 to fFileListBox.items.count - 1 do
      if fFileListBox.Selected[i] then
         inc(result);
   end;

The private function SelectedItemIndex translates the index in the selected subset to the index in the list of all the items.

function TSelectedFileCollection.SelectedItemIndex(Index: Integer): integer;
var i, j : integer;

   begin
   result:= -1;
   i:= 0;
   j:= -1;
   while (i < fFileListBox.items.count) and
      (j < index) do
      begin
      if fFileListBox.Selected[i] then inc(j);
      if j <> index then inc(i);
      end;
   if j = index then
      result:= i;
   end;

To get the filename of a selected item there is the function SelectedItemName

function TSelectedFileCollection.SelectedItemName(Index: Integer): string;
   var i : integer;

   begin
   i:= SelectedItemIndex(Index);
   if i >= 0 then
      result:= fFileListBox.Items[i];
   end;

I used the items in the fFiles stringlist on index. Switching to the SelectedItemName function the existing code will continue to work. The gain is that I now have the filelistbox component available in my collection class and can manipulate the selected setting of individual files.

(Note: In this re-arranging of code I did not change any interface definitions, so I am free of sins here).

Adding and removing items, implementing Add and Remove

The guidelines to automation collections make some very clear recommendations on adding and deleting items. The names and the results of the methods are clearly described.

Adding an item should be done using a method named add. The result of this method should be the item added itself. Deleting an item should be done using a method named remove, which takes the index as a parameter and does not return a result. I add the methods using the typelibrary editor.

 

The add method needs an extra parameter, the filename indicating which item is being added. Having the filelistbox component at hand the implementation is not to hard

function TSelectedFileCollection.Add(const FileName: WideString): ISelectedFile;
   var I : integer;
   begin
   for i:= 0 to fFileListBox.Items.Count - 1 do
      if CompareText(fFileListBox.Items[i], FileName) = 0 then
         begin
         fFileListBox.Selected[i]:= True;
         result:= tSelectedFile.CreateEx(FileName);
         end;
   end;

The add method can iterate all the items in the filelistbox. If it finds a match it sets the item's selected property to true and it creates a selectedfile object based on the given file name.

To test the method I will take another Word macro which uses the selected text in the document to find a file with that name. If the file can be found it's filesize is written to the Word document.

Sub AddItem()
   Dim File As SelectedFile
   Set File = MyManager.Selected.Add(Selection.Text)
   If Not (File Is Nothing) Then
      Selection.InsertAfter (" " & File.Size)
   End If
End Sub

After a successful call to add, I have a selectedfile object which I can ask for the corresponding file size. If add fails, because the current directory does not hold a file with the given name, the result of add will be a null object, known in VBA as nothing.

The remove method follows the same idea :

procedure TSelectedFileCollection.Remove(Index: Integer);
   var FileName : string;
   i : integer;
   begin
   FileName:= SelectedItemName(Index);
   if FileName <> '' then
      for i:= 0 to fFileListBox.Items.Count - 1 do
         if CompareText(fFileListBox.Items[i], FileName) = 0 then
            fFileListBox.Selected[i]:= False;
   end;

The helper function provides the filename of the item after which the filelistbox can be iterated. If the item is found it is de-selected.

Another Word macro will test the method

Sub RemoveItem()
   MyManager.Selected.Remove (0)
End Sub

Calling this macro a couple of times deselects the files in the component one by one.

A collection is not an array

The collection has an ever changing content due to the user clicking the Delphi component and pieces of code manipulating the component. If I would have worked with the selectedfiles as an array, it would have cost me a lot of difficult housekeeping. A collection is a far simpler approach than an array. The main difference in code is that there is no data structure which holds the collection, all code works on wrappers which only provide a different view on underlying data. The data itself is inside the Delphi component.

Where are we ?

By implementing IenumVariant automation clients will work with the selected files in my automation server in a standard way. The most obvious one is the for each language construct in VB(A), but when you work with automation, or COM in general, there are a lot of API functions and methods which will work with this automation collection. 

In the process we met some glitches where some hacking was needed but the result is to the full satisfaction of clients using the collection.

What's next ?

Update : There was a small bug in the code up to the 19th of august 2003. The pCellFetched pointer was not being dereferenced (the ^ was missing). This will have lead to an incorrect report of the number of items returned. Most consumers of automation collections do not use this parameter. They always request the items of the collection one by one. Thanks to Jeroen van Blitterswijk who reported this.