Installing large numbers of files with WIX ~ dev

Many posts (e.g.: http://blog.deploymentengineering.com/2007/06/dealing-with-very-large-number-of-files.html, http://www.wintellect.com/cs/blogs/jrobbins/archive/2007/10/19/wix-the-pain-of-wix-part-2-of-3.aspx, http://blog.deploymentengineering.com/2007/06/burning-in-hell-with-heat.html) have been written about the problem of adding large number of files to a WIX installer. This problem is the most painful when you want to add content files that do not really have any special purpose, but just have to be there (e.g. code samples or source code packages).

I also struggled with this problem, and finally I found myself creating a small MsBuild tool (WixFolderInclude.targets) that you can include in your WIX project and use to generate file installers for entire folders on the disk. I call it a tool, as I don’t have a better name for it, but it is not (only) an MsBuild target, nor is it a Task. Actually it is a WIX MsBuild extension, but WIX already has a “WIX extension” term, which is something else. So let’s stick to “tool”.

The WixFolderInclude tool

Let’s see how you can use this tool; it was tested with the latest WIX framework (v3.0.4220), but it probably works with older v3x versions as well. I’m assuming that you are more or less familiar with the WIX and MsBuild concepts. If not, you can grab the necessary information quickly from Gábor Deák Jahn's WiX Tutorial and the MSDN docs.

WIX projects (.wixproj) are MsBuild projects, and you can extend them with other MsBuild property definitions or targets. One option is to modify the wixproj file in a text editor… This is fine, but I like the WIX project to open in Visual Studio, and in this case modifying the project file is not easy. Instead, I usually always start by creating a “Build.properties” file in the WIX project (it has type “Content” so it does not modify the WIX compilation), where I can write my MsBuild extensions. I have to modify the project file only once, when I include the Build.properties file. I usually include it directly before the wix.targets import:

</ItemGroup>
<Import Project="$(MSBuildProjectDirectory)\Build.properties" />
<Import Project="$(WixTargetsPath)" />

But you can directly write into the project file as well, if you don’t use the VS integration.

Let’s take a very simple example: I would like to include two code samples in the installer. They are located in some folder (C:\Temp\ConsoleApplication1 and C:\Temp\WebApplication1) and I would like to install them in a “Samples” folder inside the program files entry of my installed application. Of course both samples contain sub-folders that I also want to include.

To achieve that with my tool,

you have to define MsBuild project items that describe the aspects of the installation of these folders
you have to define some property to utilize my tool
the tool generates some temporary WIX fragment files during compilation (and includes them in the compilation), which contain the definition of the Directory/Component/File structure and a component group that gathers the components generated for the files in the directory structure.
you have to include references to the generated component groups in the installation features of your choice in the normal wxs files (e.g. Program.wxs).

So first, let’s create the folder descriptions for my sample. The tool searches for project items called “WixFolderInclude”, so we have to create such items for the folders we want to include:

<ItemGroup>
<ConsoleApplication1Files Include="C:\temp\ConsoleApplication1\**" />
<WixFolderInclude Include="ConsoleApplication1Folder">
    <SourceFiles>@(ConsoleApplication1Files)</SourceFiles>
    <RootPath>C:\temp\ConsoleApplication1</RootPath>
    <ParentDirectoryRef>Dir_Samples</ParentDirectoryRef>
</WixFolderInclude>
</ItemGroup>

<ItemGroup>
<WebApplication1Files Include="C:\temp\WebApplication1\**" Exclude="*.scc" />
<WixFolderInclude Include="WebApplication1Folder">
    <SourceFiles>@(WebApplication1Files)</SourceFiles>
    <RootPath>C:\temp\WebApplication1Files</RootPath>
    <ParentDirectoryRef>Dir_Samples</ParentDirectoryRef>
</WixFolderInclude>
</ItemGroup>

As you can see, you can define the set of files to be included with the standard possibilities of MsBuild, so you can include deep folder structures, exclude files, or even list the files one-by-one. In the example here I have excluded the source-control info files (*.scc) from the second sample.

In the WixFolderInclude items, you have to note the following things.

The main entry (ConsoleApplication1Folder and WebApplication1Folder) describes the name of the folder installation. The generated component group ID will be based on this name, so you can use any meaningful name here, not necessarily the folder name.
The “SourceFiles” metadata should contain the files to be included in this set (unfortunately, you cannot use wildcards here directly, so you have to create a separate item for them).
The “RootPath” metadata contains the folder root of the folder set to be included in the installer. This could also be derived from the source file set (by taking the common root folder), but I like to have it more explicit, like this.
The “ParentDirectoryRef” metadata specifies the ID of the <Directory>, where the folder should be included in the installer. Now I have created a directory (Dir_Samples) for the Samples folder in the program files, so I have specified that as parent.

As we are ready with the definition, the next step is to set up the tool. It is very simple; you just have to include the following lines in the Build.properties (or the project file):

<CustomAfterWixTargets>$(MSBuildProjectDirectory)\WixFolderInclude.targets</CustomAfterWixTargets>
</PropertyGroup>

The value of the CustomAfterWixTargets should point to the tool file. If you have it in the project folder, you can use the setting above directly. Also note that the tool uses the Microsofr.Sdc.Tasks library (http://www.codeplex.com/sdctasks). I have tested it with the latest version (2.1.3071.0), but it might work with older versions as well. You should import the Microsoft.Sdc.Common.tasks file only once, so if you have already imported it in your project, you can skip that line.

Now we are done with the entries in the Build.properties, so let’s include the folders in the installer itself. As I have mentioned, the tool generates fragments that contain a component group for each included folder. The component group is named as follows: CG_WixFolderInclude-name. In our case, these are CG_ConsoleApplication1Folder and CG_WebApplication1Folder. So let’s include them in the main feature now:

<Product ...>
...

<Directory Id="TARGETDIR" Name="SourceDir">
    <Directory Id="ProgramFilesFolder">
      <Directory Id="INSTALLLOCATION" Name="WixProject1">
        <Directory Id="Dir_Samples" Name="Samples">
        </Directory>
      </Directory>
    </Directory>
</Directory>

<Feature Id="ProductFeature" Title="WixProject1" Level="1">
<ComponentGroupRef Id="CG_ConsoleApplication1Folder"/>
<ComponentGroupRef Id="CG_WebApplication1Folder "/>
</Feature>
</Product>

And that’s it. We are ready to compile!

Fine tuning

The tool supports some additional configuration options, mainly for debugging purposes: you can specify the folder where the temporary files are stored (by default, it is the value of %TMP% environment variable) and whether it should keep the temp files (by default, it deletes them after compilation). These settings can be overridden by including the following lines in the Build.properties.

Possible problems

Of course, life is not that easy... so you might encounter problems with using this tool as well. One is that it kills MsBuild’s up-to-date detection, so it will recompile the project even if nothing has changed. I think this could be solved by specifying some smart output tags on the target, but it is not easy, and usually I want to be sure that the installer package is fully recompiled anyway.

The other – probably more painful – problem is that you cannot include additional files from WIX to a subfolder of an included directory. We had this problem when we wanted to create a shortcut to the solution files of the installed samples. The problem is that since the IDs that the Sdc Fragment task generates are GUIDs, you have no chance of guessing what the subfolder’s ID was.

I have extended the WixFolderInclude.targets to support generating deterministic names for some selected folders. The folders to be selected can be defined with the “DeterministicFolders” metadata tag of the WixFolderInclude item. The value should be a semicolon-separated list of folders relative to the RootPath. Please note that as these are folders, you cannot really use MsBuild’s wildcard support, but you have to type these folder names manually. Let’s suppose that we have a Documentation folder inside the ConsoleApplication1 sample, which we might be able to extend from WIX later. We have to define this as the following:

<ItemGroup>
<ConsoleApplication1Files Include="C:\temp\ConsoleApplication1\**" />
<WixFolderInclude Include="ConsoleApplication1Folder">
    <SourceFiles>@(ConsoleApplication1Files)</SourceFiles>
    <RootPath>C:\temp\ConsoleApplication1</RootPath>
    <ParentDirectoryRef>Dir_Samples</ParentDirectoryRef>
    <DeterministicFolders>Documentation</DeterministicFolders>
</WixFolderInclude>
</ItemGroup>

As a result, the ID for the Documentation’s <Directory> element will be: Dir_ConsoleApplication1Folder_Documentation, so we can extend it from our Product.wxs:

Attachment

In the attached ZIP file, you will find the WixFolderInclude.targets file, and also the sample that I have used here to demonstrate the features (without the silly ConsoleApplication1 and WebApplication1 folders). Feel free to use them!

dev

stolen from around the world

Thursday, November 12, 2009

Installing large numbers of files with WIX