XAP Optimization – Part III Versioning through program

In my previous blog I explained how to do manual versioning. At the end, I also mentioned I would write a ruby script to automate it so that when you have tons of third party controls, you can version them using a tool.

Initially I was going to write the script in Ruby and it turned out not many people does not know ruby and it could cause problem down the road so I stick to what everyone knows (at my place), C#. So what are we trying to do? Most of the third party controls comes with extmap files which is used for assembly caching.  By default the extmap file does not have versions. But in real life not all our applications are going to use one version in production all the time, we would end up using multiple versions in production as we release new features by using new controls while maintaining old features. Till the old features are migrated to use new controls you would need to keep both version in production. One another requirement for assembly caching is that the dll that you are going to cache need to be signed. I use component one controls and they are all signed, one less work for me.

Now lets see how are we going to do it, Lets first see the code

   1:  class Program
   2:      {
   3:          static void Main(string[] args)
   4:          {
   5:              if (args.Count() < 2)
   6:              {
   7:                  Console.WriteLine("Usage: XAPOptimizationVersioningdirectory versionnumber versionname");
   8:                  Console.WriteLine("     : Directory need to be full path.");
   9:                  Console.WriteLine("     : Version number can be anything you need, in my case it would be three digit number.");
  10:                  Console.WriteLine("     : versionname is optional. If it is available then it would consolidate all the dll into single version with this name.");
  11:                  Console.ReadKey();
  12:              }
  13:              else
  14:              {
  15:                  bool replaceFullAssemblyCacheFileName = false;
  16:                  string assemblyCacheZipFileName = string.Format(".{0}.zip", args[1]);
  17:                  if (args.Count() == 3)
  18:                  {
  19:                      assemblyCacheZipFileName = string.Format("{0}.{1}.zip", args[2], args[1]);
  20:                      replaceFullAssemblyCacheFileName = true;
  21:                  }
  22:   
  23:                  foreach (var extFileName in Directory.GetFiles(args[0]).Where(p=> p.Contains("extmap")))
  24:                  {
  25:                      Console.WriteLine("Processing File {0}", extFileName);
  26:                      XDocument doc = XDocument.Load(extFileName);
  27:                      IEnumerable<XElement> element = doc.Descendants("extension");
  28:                      string attr = element.ElementAt(0).Attribute("downloadUri").Value;
  29:                      if (replaceFullAssemblyCacheFileName)
  30:                          attr = assemblyCacheZipFileName;
  31:                      else
  32:                          attr = attr.Replace(".zip", "." + args[1] + ".zip");
  33:                      element.ElementAt(0).Attribute("downloadUri").Value = attr;
  34:                      doc.Save(extFileName);
  35:                  }
  36:                  Console.WriteLine("Versioning Complete.");
  37:                  Console.ReadKey();
  38:              }
  39:          }
  40:      }

It is as simple as 40 line code but as you can see most of the magic happen between line 23 and 35. First lets split the program in a way it make sense. First of all this is a console application where you need to pass 2 optionally 3 parameters. Lets look at the usage code;

   1:  if (args.Count() < 2)
   2:              {
   3:                  Console.WriteLine("Usage: XAPOptimizationVersioningdirectory versionnumber versionname");
   4:                  Console.WriteLine("     : Directory need to be full path.");
   5:                  Console.WriteLine("     : Version number can be anything you need, in my case it would be three digit number.");
   6:                  Console.WriteLine("     : versionname is optional. If it is available then it would consolidate all the dll into single version with this name.");
   7:                  Console.ReadKey();
   8:              }

When you run the program, you call the name of the program, full directory path to the location of dlls, version number and finally and optionally a version name. First two parameter is self explanatory. What is the third parameter? Well, by default when you do assembly caching, if you have 10 dlls, then it will generate 10 different zip files.but what if you want all of them to be in single file zip file but separated out of the solution XAP file? here is it, pass the name of the file you want all these dlls zipped into. So if you would run the program just the first two parameter

XAPOptimizationVersioning c:\temp\componentone192 192

with the above parameters, it will modify all the extmap files to add version number and thus creating versioned dlls. For example, if I have a file called C1.Silverlight.dll, its expmap would let you create C1.Silverlight.zip out of the box, but after running this tool, it will create C1.Silverlight.192.zip in the client bin.

On the other hand if you would call the program using

XAPOptimizationVersioning c:\temp\componentone192 192 ComponentOne

then anytime when the program uses any dll from c:\temp\componentone192, it will be bundle all dlls referenced in the solution and create one single dll called ComponentOne.192.zip. This will reduce number ZIP files at the server side but on flip side, if someone for their project doesn’t require one of the dll used in your project but pointing to same reference folder then it will create new ComponentOne.192.zip without your reference dll and replace existing one in the server. Since the ZIP is changed, when the user goes to your page next time, it will download new ZIP and error out since one of your required dll is missing. So use this approach with Caution. I will not recommend this solution as it is, but you can get to accomplish this with little workaround I mentioned at the bottom.

Now lets look at the meat of the code

   1:  foreach (var extFileName in Directory.GetFiles(args[0]).Where(p=> p.Contains("extmap")))
   2:                  {
   3:                      Console.WriteLine("Processing File {0}", extFileName);
   4:                      XDocument doc = XDocument.Load(extFileName);
   5:                      IEnumerable<XElement> element = doc.Descendants("extension");
   6:                      string attr = element.ElementAt(0).Attribute("downloadUri").Value;
   7:                      if (replaceFullAssemblyCacheFileName)
   8:                          attr = assemblyCacheZipFileName;
   9:                      else
  10:                          attr = attr.Replace(".zip", assemblyCacheZipFileName);
  11:                      element.ElementAt(0).Attribute("downloadUri").Value = attr;
  12:                      doc.Save(extFileName);
  13:                  }
 

Line 1: Identify all the extmap files available in the directory. This implementation does not recursively look for extmaps in sub folders. You can extend the code to do it.

Line 4: Load the extmap file as XDocument

Line 5: Identify the extension element which have the downloadUri attribute.

Line 7: Check to see if we are going to replace the full name with user supplied zip file name or append only version to the zip filename.

Line 8: Replace the default file to user supplied zip file name.

Line 10: Appends version number to zip file name.

Line 11: Replace the value with modified value.

That’s about it, but here are some thoughts on the whole ‘XAP Optimization’ worth considering.

  • By adding XAP optimization your very first page load will have same load time as the one you would do with out ‘XAP Optimization’.
  • When you modify the code and none of the ZIP changed, the file download time will be drastically reduced because it downloads only changed dll. So the benefit is in the future downloads.
  • Here is a very important catch in ‘XAP Optimization’, if you are using, say 15 or 20 reference dlls and they are zipped up into seperate ZIP files, for every page visit, regardless if you are coming to the page first time or next time, there is a round trip call the web server to see if a files is changed for all ZIP files. This time will add up so quick and users will see downloading screen every time, even though it is not downloading. So please keep this in mind. This could annoy powers users and could create a perception that, the application is downloading every time.
  • If you would like to reduce ZIP files in the server side, one option is to run the tool and create version specific complete zip file or group them in such a way you have few ZIP files. Create these ZIP files before hand manually. When you deploy, only deploy these prepackaged ZIP instead of the ones from client bin folder. This will reduce the zip files, reduce the server side look up and also no one accidently step on and remove any dlls.

Use it properly to fit your need. Hope this helps someone.

Advertisements

XAP Optimization – Part II Versioning

So we are all not just building one simple Silverlight project and publishing simple web pages. Most of us develop enterprise LOB application in Silverlight. When you develop that big application you will end up using same component again and again over multiple projects and also you will end up using multiple version of same controls over multiple places as you develop. When you develop major application, if you get a new version of a third party control or even your own control, you will not replace it everywhere because of the regression testing. You will end up versioning and upgrade the solution with version.

In my previous blog on XAP Optimization, I looked at the simple attempt to reduce the size of a small Silverlight application. We looked at two different approaches. From 880Kb, we were able to get it down to ~230KB. I looked at the XAP file to see what is taking that much space and it turned out, I am using Component One controls for UI and Jounce for MVVM Framework. My dll by itself, unzipped only 14k. I was ok with the size of the file since, some of my application would use one version of Component One control and some other XAP files would use different version of the Component One control so If I would strip them out and make it common zip file then I would run into version conflict. So I was happy with the result till last weekend 🙂

It turned out, even though we use different version of controls, as we make changes we bring the projects to latest controls, so 80% of them require version of the controls. So it makes sense to even strip the controls out of XAP to reduce the file size and also reduce repeated loading of common controls, but somehow put versioning in place.

So by default, System controls become ZIP file when we use XAP optimization and how would be able to do the same for other controls and our own libraries? The answer is very simple. If you would open up the XAP file, you would see

image

The first file is AppManifest.xaml,

<Deployment xmlns="http://schemas.microsoft.com/client/2007/deployment" xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" EntryPointAssembly="XAPOptimization1" EntryPointType="XAPOptimization1.App" RuntimeVersion="4.0.50826.0">
  <Deployment.Parts>
    <AssemblyPart x:Name="XAPOptimization1" Source="XAPOptimization1.dll" />
    <AssemblyPart x:Name="C1.Silverlight.FlexGridExcel" Source="C1.Silverlight.FlexGridExcel.dll" />
    <AssemblyPart x:Name="Jounce" Source="Jounce.dll" />
    <AssemblyPart x:Name="System.Windows.Interactivity" Source="System.Windows.Interactivity.dll" />
  </Deployment.Parts>
  <Deployment.ExternalParts>
    <ExtensionPart Source="c1.silverlight.FlexGrid.4.183.zip" />
    <ExtensionPart Source="c1.silverlight.FlexGridFilter.4.183.zip" />
    <ExtensionPart Source="System.ComponentModel.Composition.zip" />
    <ExtensionPart Source="System.Windows.Data.zip" />
    <ExtensionPart Source="C1.Silverlight.183.zip" />
    <ExtensionPart Source="System.Xml.Linq.zip" />
    <ExtensionPart Source="C1.Silverlight.Zip.183.zip" />
    <ExtensionPart Source="C1.Silverlight.Pdf.183.zip" />
    <ExtensionPart Source="System.ComponentModel.Composition.Initialization.zip" />
  </Deployment.ExternalParts>
</Deployment>

If you see there are two parts of Deployments. One is Deployment.Parts which specifies all the dlls that are in the XAP file and associated name space. The second part specifies all the zip files that might be required for application to run. In my opinion the only thing that should be in XAP file will be the application dll. So the two part question is how would you get all the common dlls separated into its own ZIP file and how did you update the Application Manifest to reflect it?

Here is a very good article explaining how to Configure an assembly for use with application library caching. Please read this link since it goes more detail about the Application Manifest file and also ext map XML file.

Any dll that is common and it need to be in a separate Zip file, should have associated ‘extmap’ file. What it means is, in the above project you see that I am using System.Xml.Linq.dll, if you would find the file location using the file property and navigate to the file location, you will see following files, The file we are interested in is the dll file name with ‘extmap.xml’

imageIf you would crack open the System.Xml.Linq.extmap.xml, you will see the following

<?xml version="1.0"?>
<manifest xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <assembly>
    <name>System.Xml.Linq</name>
    <version>2.0.5.0</version>
    <publickeytoken>31bf3856ad364e35</publickeytoken>
    <relpath>System.Xml.Linq.dll</relpath>
    <extension downloadUri="System.Xml.Linq.zip" />
  </assembly>
</manifest>

The first two line is name and version and the third line is the public key of the dll, that means anything you want to have as separate dll, have to be strong named. Next two specifies the relative path of the dll and where do you get it from. In our case, the dll is in a zip file in the same path of the XAP file.

Have a look at this blog, here the author explains how it is done and also provides a tool to create your own extmap files for your dlls.

Once we have extmap files created for the dlls that need to be separated and placed on the same path where the dll located, when compiled with XAP optimization enabled, all the ones with extmap files will be in separate ZIP files.

Now comes the question of versioning, some XAP files would use different version of same controls, so by creating extmap file we would end up overwriting the old one with new one and potentially crashing the  application. How do we solve it, again it is very simple, now most of all third party controls comes with extmap files, including Component One. Whenever we get latest version of a control or our own library, I run a small ruby script, which run through all extmap files and adds version number to the destination zip file. So instead of

<ExtensionPart Source="C1.Silverlight.zip" />

our implementation would look like the following

<ExtensionPart Source="C1.Silverlight.183.zip" />

That’s about it. So when I compile my project pointing to the new version dll, it uses the modified extmap and generate version numbered zip file thus making all the project work without any problem. I am working on a script which will then walk though all the XAP files and look at the Application Manifest and remove any unwanted zip files from the deliver location thus keeping the folder clean.

Once I have the second script done, I will post both the scripts in the blog here.

To test this, I created two Silverlight applications. Both are exactly same except the first one uses version 152 of Component One dlls and second one uses version 183. Now if I would run, each application should use appropriate ZIP and run without any problem. So I ran the fiddler and then ran the application to see the what is it downloading and is it caching.

image

First I ran XAPOptimization with version 152 and as you can see the first green box shows all the files that got downloaded to the client. Now when I tried the second version of the XAP Optimization with version 183, as you can see it did get the version 183 and also it skipped the common files that was already downloaded by the first version. For example. System.Windows.Data.zip was not downloaded since it was downloaded along with the first XAP file.

I see lot of potential in using XAP optimization when developing large application. Hope this helps someone.

One thing I forgot to mention, with versioning and keeping couple dlls in the XAP file itself, we are at ~100kb. That is a excellent saving. So next time when we make a change the the solution and none of te thrid party control changes, our download is only the XAP file size that is ~100kb not 880kb.