Automation arrays
Automation has a good support for arrays. Arrays can be used to pass large
amount of data to and from automation servers. This data can be anything from
very flexible structures holding diverse data to a very fast way to pass binary
data. Automation arrays are self descriptive, beside the data itself they house
information on the array size and dimensions as well on the element size.
Safearrays
Automation arrays are called safearrays. Safe because they themselves have
the knowledge of their dimensions, so they provide a protection against
"out of bounds" errors. They are also safe because they have a
knowledge of the size of the elements, getting or setting an item will address
the correct range of memory.
In the Delphi VCL class library safearrays will be wrapped up as Variant
arrays. These variantarrays are compatible with an OleVariant. The VCL unit variants.pas houses a couple of routines and
types to work with variantarrays.
A variant array is declared as record.
TVarArray = packed record
DimCount: Word;
Flags: Word;
ElementSize: Integer;
LockCount: Integer;
Data: Pointer;
Bounds: TVarArrayBoundArray;
end;
You can see how the different elements of the record describe the array:
a count of the number of dimensions, a flag to work with custom variants, the
size of an individual array element, the number of locks on the array (for
efficient use), a pointer to the array's data and an array describing the lower
and upper bounds of all the dimensions of the array. You can see here that the
description of the array and the actual data itself are separated. We will meet
most of these in the story to come.
The elements of a variant array can be almost any type, as I am covering automation I will
limit the types used to automation compatible types. OleVariant is among these
types. A variant array is compatible with an OleVariant so I can
construct another variant array and put that in the item. Repeating the process
could lead to ragged treelike structured. You might wander where the elementsize fits in in this scenario.
As the items are arrays themselves there seems no longer to be a fixed item
size. Let's take a sidestep and look at the variant type itself in some more
detail.
The Variant type
Delphi variants can be very complicated types, in Delphi 6 it is even
possible to construct custom variant types. The entire VCL unit variants.pas
is dedicated to them. The OleVariant in the narrower, automation sense is described in
System.pas. It is also a record
PVarData = ^TVarData;
{$EXTERNALSYM PVarData}
TVarData = packed record
VType: TVarType;
case Integer of
0: (Reserved1: Word;
case Integer of
0: (Reserved2, Reserved3: Word;
case Integer of
varSmallInt: (VSmallInt: SmallInt);
varInteger: (VInteger: Integer);
varSingle: (VSingle: Single);
varDouble: (VDouble: Double);
varCurrency: (VCurrency: Currency);
varDate: (VDate: TDateTime);
varOleStr: (VOleStr: PWideChar);
varDispatch: (VDispatch: Pointer);
varError: (VError: LongWord);
varBoolean: (VBoolean: WordBool);
varUnknown: (VUnknown: Pointer);
varShortInt: (VShortInt: ShortInt);
varByte: (VByte: Byte);
varWord: (VWord: Word);
varLongWord: (VLongWord: LongWord);
varInt64: (VInt64: Int64);
varString: (VString: Pointer);
varAny: (VAny: Pointer);
varArray: (VArray: PVarArray);
varByRef: (VPointer: Pointer);
);
1: (VLongs: array[0..2] of LongInt);
);
2: (VWords: array [0..6] of Word);
3: (VBytes: array [0..13] of Byte);
end;
{$EXTERNALSYM TVarData}
This declaration and the values of the constants tells us some things:
- The externalsym keyword indicates tVardata being a type known in
the Windows API.
- A variant is physical a record, 14 bytes in size.
- The data of variables without a fixed size, like a string or an array, is stored outside of the
variant. The variant holds a pointer to this data.
- When a variant is an array, the varArray flag will be set. This flag
is combined with an other flag to indicate the type of the array.
A variant can be typecasted to this tVarData type and can be used for
testing the actual content of the variant.
if tVarData(MyVariant).VType <> varString then
ShowMessage( 'This variant does not hold a string !' );
When querying variants for their type you will meet Vtypes that are unknown
to the TvarData type. A Delphi VariantArray has a vType of $200C. This is the varArray
bit $2000 and the vType $C. This does stand for varVariant, which is
omitted in tVarData. This is where the Delphi Variant gets separated from the
Windows variant.
Another
thing you can do with tVarData is casting the variant's value to
a certain type :
if tVarData(MyVariant).VType = varDate then
ShowMessage('DateTimeToStr( tVarData(MyVariant).VDate ) );
Now we have seen some of the internals of a variant I will return to the
variant array.
Creating a ragged variant array
In the filemanager example I will add a
new property GroupedByName to the manager. This property will hold a list of files
grouped by filename, excluding the extension. These grouped names will be
returned in a variant array, each item of the array holds a subarray. The
first item of the subarray will hold the filename, the next items the
extensions found for this filename.
In the typelibrary editor I will create a new read only property GroupedByName,
of type variant. In the property getter the array is created and filled.
I
will need a helper function to remove the extension from the filename :
function BlankFileName(InString : string): string;
var Ext : string;
begin
Ext:= ExtractFileExt(InString);
result:= Copy(InString, 1, Length(Instring) - Length(Ext));
end;
To build the array will take the following steps
- Set the functions result to an array sized to the number of filenames in the
filelistbox.
- Scan the array for every filename in the listbox.
- Not found : Create a new subarray and set the first item to that filename.
- Found : Get the subarray.
- Increase the size of the subarray to hold one more file extension and fill
that item.
- Copy the subarray into the result array.
- Finally resize the result to remove any empty items.
This can be done with the following code:
function TFileZapper.Get_GroupedByName: OleVariant; var i, j : integer;
TempArray : Variant;
begin
result:= VarArrayCreate( [0, Form1.FileListBox1.Items.Count], varVariant );
VarArraycreate creates as new safearray with one dimension, ranging
from 0 to the number of files in the FileListbox. The second
parameter indicates that all the items in
the array will be variants.
for i:= 0 to Form1.FileListBox1.Items.Count -1 do
begin
j:= -1;
Found:= False;
while ( j < VarArrayHighBound(result,1) ) and not Found do
I will scan first dimension of the result-array, its upperbound is returned
by the VarArrayHighbound function .
begin
inc(j);
TempArray:= result[ j ];
Found:= ( VarArrayDimCount( TempArray ) = 1 ) and
( CompareText( TempArray[0], BlankFileName( Form1.FileListBox1.Items[i]) ) = 0 );
end;
The array item to be tested and potentially altered is copied to the private variant TempArray. Using VarArrayDimCount
I can check the number of dimensions, which will be 0 in an uninitialized
element. When the element is initialized the filename will be in element 0.
if not Found then
begin
TempArray:= VarArrayCreate( [0, 1], VarOleStr );
TempArray[ 0 ]:= BlankFileName( Form1.FileListBox1.Items[i] );
j:= 0;
while VarArrayDimCount( result[ j ]) > 0 do
inc(j);
end;
When not found, the new filename will be added to the result. VarArrayCreate
will create the new subarray. This time I pass varOLEstr to the function, all items in the array
will be a (Wide)string. To find the index of the first uninitialized array item, I will
scan the result array with VarArrayDimCount. All initialized items
are arrays with (at least) one dimension and will be skipped. The new item will be
added after the last item containing data.
Now I will add the extension of the file to the subarray.
VarArrayRedim( TempArray, VarArrayHighBound(TempArray, 1) + 1 );
TempArray[ VarArrayHighBound(TempArray, 1) ]:= ExtractFileExt( Form1.FileListBox1.Items[i] );
I will increase the size of the subarray to the currenct VarArrayHighBound
plus one. VarArrayRedim will resize the subarray to this length. The
file-extension is copied into the new item and to conclude the TempArray
will be copied into the result array again.
result[ j ]:= TempArray;
end;
Rests cleaning up. There is a big chance that some of the items
in result are unused. The number of items was initialized as the number
of filenames. If a filename occurs more than once, with just a different
extension, then there will be some empty items left.
j:= VarArrayHighBound(result,1);
if j > 0 then
begin
while VarArrayDimCount( result[j] ) = 0 do
dec( j );
VarArrayRedim( result, j );
end;
By checking the element for
zero dimensions I can remove the items by resizing the result to a smaller size,
using VarArrayRedim again.
Using the ragged array
Word would be a good user of the variant array. In a macro I will use the
property to create a report on all files grouped by name. Before I can work with
a variant array I will first have to copy the array to a local variant as VBA
will interpret MyManager.GroupedByName(1) as the groupedByName being an indexed
property.
Sub ListRaggedArray()
Dim AllFiles As Variant
Dim FileGroup As Variant
Selection.EndKey Unit:= wdStory
AllFiles = MyManager.GroupedByName
For i = 0 To UBound( AllFiles )
FileGroup = AllFiles( i )
For j = 0 To UBound( FileGroup )
Selection.InsertAfter ( FileGroup( j ) & vbTab )
Next
Selection.InsertParagraphAfter
Next
Selection.EndKey Unit:= wdLine
Selection.TypeParagraph
End Sub
The Groupedbyname property is copied to the local variant AllFiles.
The VBA function Ubound is the VBA counterpart of VarArrayHighBound,
it will return the number of elements in the array. Again the subarray property
is first copied to the FileGroup local variant. And the items of FileGroup
can be
inserted into the document.
Evaluating the variantarray
Working with a variant array we have seen a couple of things
- VarArrayCreate needs an even number of bounds parameters. So it
can create only rectangular multi-dimensional array. To create ragged arrays you have to set individual items to an entire
new variant array.
- Arrays can be resized to grow and shrink in size. This can only be
done on the right-most dimension.
- The current dimensions and sizes of the array can be queried using API
functions.
- Arrays are always (deep) copied in full. When assigning the local temparray
array variable to an item in the result, all the data in the
local array was copied to the result array.
This approach to safearrays is very flexible but of course there is a lot of
overhead while resizing and copying. The coding sample would have been a lot
faster if the data was pre-processed first so that all arrays could be created
an filled in one go. I did it this way to show as much of the of the array manipulating
functions and their possibilities as possible.
Creating an integer safearray
Now I will dive deeper into the safearray api to create a fast and efficient
safe array. The VCL functions all work with VariantArrays and house in
the Delphi variants unit. In the ActiveX API there are more safearray handling
routines. Let's take a look.
In ActiveX.pas you will find this declaration of a safearray
PSafeArray = ^TSafeArray;
{$EXTERNALSYM tagSAFEARRAY}
tagSAFEARRAY = record
cDims: Word;
fFeatures: Word;
cbElements: Longint;
cLocks: Longint;
pvData: Pointer;
rgsabound: array[0..0] of TSafeArrayBound;
end;
TSafeArray = tagSAFEARRAY;
{$EXTERNALSYM SAFEARRAY}
SAFEARRAY = TSafeArray;Only the names differ from the variant safearray
we met before. A number of API functions work with these arrays. I will use
these to create a typed safearray.
In the typelibrary editor I will create a property FileSizes, as type
I will choose SAFEARRAY(long). Which describes an array of (long)
integers.
The property will hold an array with the sizes of all
files currently in the FileListbox. VBA (in Word) will recognize the FileSizes
property being an array of long (integers). So SAFEARRAY is an automatable
property type.
The filemanager implementation in Delphi will have to create the
safearray. The safearray is accessed via a PSafeArray, a pointer. This will be the
result of property getter. In the getter I will create the
safearray and allocate it's data. I will fill a record with the array bounds and
I can create the array. After accessing the data I will fill its items with
file-sizes.
function TFileZapper.Get_FileSizes: PSafeArray;
var ArrayBounds : TSafeArrayBound;
i : integer;
ArrayData : pointer;
F : tSearchRec;
fFileSizes : PSafeArray;
type
IntegerArray = Array of integer;
begin
ArrayBounds.lLbound:= 0;
ArrayBounds.cElements:= Form1.FileListBox1.Items.Count;
fFileSizes:= SafeArrayCreate( varInteger, 1, ArrayBounds );
if SafeArrayAccessData( fFileSizes, ArrayData ) = S_OK then
begin
for i:= 0 to Form1.FileListBox1.Items.Count - 1 do
if FindFirst( Form1.FileListBox1.Items[i], faAnyFile, F) = 0 then
begin
IntegerArray(ArrayData) [i]:= F.Size;
FindClose(F);
end;
SafeArrayUnAccessData(fFileSizes);
end;
result:= fFileSizes;
end;
The record holding the arrays dimensions, tSafeArrayBound is declared in ActiveX.Pas.
The lower bound is set to 0
and the upper to the number of files in the listbox. SafeArrayCreate will
create an array of integers. The data itself is accessed through SafeArrayAccessData.
Working with the variant array I did a lot on resizing and reshaping of the
array. The ability to resize an array makes life hard on other pieces of code
which are using the array. To speed up access safearrays can be locked. A locked
array cannot be resized or reshaped. SafeArrayAccessData places a
lock on the array and provides a pointer to the data. I can work with this data
as an array of integer if I typecast it to the proper type. The IntegerArray
type will do just that, now I can fill the array with file sizes. SafeArrayUnAccessData frees
the lock. A safearray keeps a count of the
number of locks placed, when the number of locks has sunken to 0 the array can
be reshaped or destroyed.
VBA did recognize the property and I can use it just like the variant array.
After copying the array to a local variant again it can be scanned and its items
inserted into the document.
Sub ListSizes()
Dim VarSizes As Variant
Selection.EndKey Unit:=wdStory
VarSizes = MyManager.FileSizes
For i = 0 To UBound(VarSizes)
Selection.InsertAfter (VarSizes(i))
Selection.InsertParagraphAfter
Next
Selection.EndKey Unit:=wdLine
Selection.TypeParagraph
End Sub
SafeArray versus VariantArray
With a VCL VariantArray you can do almost anything you can do with a
safearray. Locking and reshaping function are available to both. The big difference
between a variant- and a safe- array is that a variantarray is to COM of type
OleVariant and cannot be used to implement an
automation property of type SAFEARRAY. Properties of type SAFEARRAY can be recognized by an
automation client as being an array of a specific type. Variant arrays are
published as VARIANT in the typelibraray. As a variant can be almost anything
the client will have to inspect the property to find out if it is an array and
what type of items it holds.
SafeArrays have as a second advantage that they do not need the variants
unit. Some people find it be desirable to keep that unit out of their
application.
Passing binary data
The locked integer safearray works a lot faster than the variantarray.
Safearrays can be used to pass large amounts of data really fast. If you
want to pass any binary data from an automation clients to automation servers
you should work with a locked safearray of bytes. Any binary object, like an
image or a document can be serialized to an array of bytes. The size of a
safearray can be over 64K, so it is possible to pass a huge amount in one go.
Passed bytes do not need any translation. With multibyte values you have
to decide on the ordering of the upper and lower
byte.
Arrays in VBscript
VBscript does know how to work with arrays. Arrays can be declared and resized or
even be created
in one go with the Array() function. When working with array results VBA did have some trouble with the
distinction between an array and an indexed
property. You can also create automation objects, like the FileManager,
in a script. VBscript works fine
with the object but mistakes all array properties for indexed properties and
will report error $800A000D "Type mismatch". That is, the type is unknown to
VBScript. As a workaround indexed properties can be a full functional
replacement for arrays and are easy to implement. In many situations a collection
will be sufficient.
Where are we ?
The automation API provides with a number of structures and functions to work
with array structures. These have been designed and implemented in such a manner
that they are safe to use. Besides that they are selfdescribing, trying to
access an element outside the range will not lead to a serious crash.
Arrays can be very flexible, when it comes to more than one dimension they
are even more flexible than collections. Arrays can
also be a very fast and efficient manner to pass (large amounts of) data. On the
other hand VBA and VBscript do not always know how to handle an array the right
way.
The Delphi VCL provides a couple of wrapper functions to work with safearrays
of variant. For a better understanding of these some insight in the
internals of a variant can be very enlightening. Working directly with the automation safearray API requires a little more
knowledge.
What's next
|