Thursday, December 4, 2008

Using ILMerge in real life

There are some notable figures blogging about how you can use ILMerge to merge assemblies in Visual Studio by tweaking the project file. Scott Hanselman wrote a good post in how to merge assemblies generated from multiple languages into a single assembly. Starting out with that post I was trying to get this to work in a real world situation. In the samples provided in various blog posts, including Hanselman's, the assembly being generated with ILMerge was the final result of the build. But what if you need Visual Studio to pick up the modified assembly and use it as a reference for another project in the solution? I ran into a couple of issues but managed to get it working in the end.

The project file
First, setup your project so it builds with all the references it needs. Then to get ILMerge working you need to tweak the project file. Basically we need an AfterBuild target that:

  • Enables the project output to be used as a dependency for other projects in the same solution.
  • Runs ILMerge only when the project output is updated.
  • Maintains the assemblies strong name.
  • Enables unit tests to run on the merged project output.

Here's the code for the MSBuild target:

<Target Name="AfterBuild" 
        Condition="'$(_AssemblyTimestampBeforeCompile)'!='$(_AssemblyTimestampAfterCompile)'">
<CreateItem Include="@(ReferencePath)" 
            Condition="'%(ReferencePath.IlMerge)'=='true'">
  <Output TaskParameter="Include" ItemName="IlmergeAssemblies" />
</CreateItem>
<Message Text="MERGING: @(IlmergeAssemblies->'%(Filename)')" Importance="High" />
  <Exec Command="..\..\..\Tools\ILMerge\ILMerge.exe /internalize /keyfile:"..\..\myapp.snk" /out:Migrate.dll "..\..\@(IntermediateAssembly)" @(IlmergeAssemblies->'"%(FullPath)"', ' ')"       WorkingDirectory="$(MSBuildProjectDirectory)\$(OutputPath)" />
  <Copy SourceFiles="@(MainAssembly)" 
      DestinationFolder="$(IntermediateOutputPath)" 
      SkipUnchangedFiles="true" OverwriteReadOnlyFiles="$(OverwriteReadOnlyFiles)">
  </Copy>
  <Copy SourceFiles="@(_DebugSymbolsOutputPath)" 
      DestinationFiles="@(_DebugSymbolsIntermediatePath)" SkipUnchangedFiles="true"       OverwriteReadOnlyFiles="$(OverwriteReadOnlyFiles)" 
      Condition="'$(_DebugSymbolsProduced)'=='true' and '$(SkipCopyingSymbolsToOutputDirectory)' != 'true'">
  </Copy>
</Target>

Copy paste the target into the project file.

ILMerge and paths
The code above uses ILMerge from a folder relative to the project. You may have it installed on your system somewhere. If that is the case, set the absolute path to the tool. By default ILMerge works in the current working directory and fortunatly the <Exec> task has a WorkingDirectory setting. Setting that to $(MSBuildProjectDirectory)\$(OutputPath) makes the command operate on the output folder (e.g. bin\Release). All other paths supplied to ILMerge are either relative to that folder or absolute paths.

Marking files to merge in
The code above uses custom metadata (<ilmerge>True</ilmerge>) on the reference to figure out what references to merge into the target assembly. A reference with the ILMerge metadata will look like this:

<Reference Include="Datalayer, Version=2.0.4.2, Culture=neutral, PublicKeyToken=76bf2dedaa68ccb5, processorArchitecture=MSIL">
  <SpecificVersion>False</SpecificVersion>
  <HintPath>..\..\..\..\lib\Datalayer.dll</HintPath>
  <IlMerge>True</IlMerge>
  <Private>False</Private>
</Reference>

The Private tag reflects the CopyLocal setting on the reference's properties in Visual Studio. The first part of the task in the first code snippet builds a list of the assemblies marked for inclusion using the ILMerge tag.

Working with Visual Studio
The next problem I encountered is that Visual Studio, or rather the MSBuild targets used underneath, do some unexpected things. First, when the assembly is referenced from another project, it's not picked up from the output folder (bin/release) but from the intermediate folder (obj/release). Merging the assemblies in the intermediate folder is not an option because the source and target assembly (and .pdb) cannot be the same file. The solution is to copy the merged assembly and .pdb back to the intermediate folder after merging using the <Copy> task. To prevent problems with builds that don't update the project output I also added a condition on the target. This causes ILMerge to run only when the assembly is updated. In order for this to work the RunPostBuildEvent needs to be set:

<RunPostBuildEvent>OnOutputUpdated</RunPostBuildEvent>
This setting corresponds to the following setting in Visual Studio 2008: Visual Studio 2008 Build Events Tab Make sure you don't change that setting in Visual Studio because it can result in build errors.

Friday, November 21, 2008

Low-tech AJAX requests with jQuery and ASP.NET 3.5

ASP.NET has all kinds of fancy features for out-of-band ajax communication. There's web services, script methods, page methods and what not. I was looking for an easier, low profile method that will also work without the Microsoft's AJAX framework and doesn't require proxy generation by ScriptHandlerFactory. The goal is to be able to do some AJAX stuff with a jQuery.

MVC leads the way
So, looking at the MVC Beta, I noticed there's a JsonResult class. The gut of the class does nothing more than this:

Response.ContentType = "application/json";
JavaScriptSerializer serializer = new JavaScriptSerializer();
Response.Write(serializer.Serialize(data));
Now, that's what I call easy. JavaScriptSerializer is in the System.Web.Extensions assembly, so that's commonly available in ASP.NET 3.5 and not part of the MVC framework. Great, that means that from a basic ASP.NET 3.5 application we can do something like this with jQuery: $.getJSON("Default.aspx", dataReady); This will trigger a post to the ASP.NET page. The dataReady function will be invoked when the request returns. But how do we know whether to return JSON data or normal web content? One option is to append a parameter to the url (for example ajax=true). A nicer solution is to check the headers. jQuery will set the Accept header to indicate it expects a JSON result: Accept: application/json, text/javascript, */* Implementation
Now, by overriding the Render method in the page it's relatively easy to handle both the web requests and the ajax requests:
protected override void Render( HtmlTextWriter writer )
{ 
  if ( Request.AcceptTypes.Contains( "application/json", StringComparer.OrdinalIgnoreCase ) )
  {
    Response.ContentType = quot;application/json";
    JavaScriptSerializer serializer = new JavaScriptSerializer();
    Response.Write( serializer.Serialize( data ) );
  }
  else
  {
    base.Render( writer );
  } 
}
JSON trouble
As it turns out, there is just one snag here. ASP.NET and jQuery apparently don't speak the exact same dialect of JSON. This basically means that DateTime values wont be turned into JavaScript Date objects on the client. We can fix that by moving this solution to the server:
JavaScriptSerializer serializer = new JavaScriptSerializer();
Response.Write( System.Text.RegularExpressions.Regex.Replace( serializer.Serialize( data ), @"\""\\\/(Date\([0-9-]+\))\\\/\""", "new $1" ) );
Additional considerations
Please note that this is just a proof of concept; in a real life application you should impose all kinds of security measures before responding to an AJAX request. You may also want to split the page and the handling of the AJAX callbacks into separate classes. Handling the AJAX request through an IHttpHandler a lot more efficient.

Friday, October 3, 2008

NAnt-Extensions - First Release

The first release for the new NAnt-Extensions project is a fact. Alexander GroƟ has worked hard to create a set of NAnt tasks that integrate nicely with the TeamCity continuous integration server. I was happy to donate the code I posted earlier for MbUnit support in TeamCity and help out in reviewing, testing and documenting the current release. This release includes: - MbUnit support for TeamCity - MSpec support for TeamCity - Logging of statistics and build status to TeamCity Please check out the online documentation for more info. Downloads and source code are available on google code.

Wednesday, September 17, 2008

TeamCity and Subversion with NTLM authentication

One of the subversion repositories I work with was just migrated to a brand new server. The server uses NTLM authentication. Everything works like a charm, except for TeamCity builds. It keeps throwing errors like: Cannot request dated revision from svn: svn: Authentication required for https://....Googling for the solution, it turns out the problem is with SVNKit. Apparently the workaround is to add a setting: svnkit.http.methods=Basic,Digest,NTLMGreat, so where do I put this setting? It needs to be set before the main JVM launches the server or agent. On the server that's simple enough. Adding the setting into the script that starts the TeamCity server process fixed that problem (hurray for unix scripts). On a Windows based build agent this is a lot less trivial, especially since the build agent runs as a service... I tried modifying all kinds of settings; build properties in TeamCity, agent properties, startup scrips (uhm, I mean batch files)... no dice. Some more digging leads to the wrapper that's used to launch the build agent service. Additional Java settings can be added to <BuildAgent Root>\launcher\conf\wrapper.conf : ... # Application parameters. Add parameters as needed starting from 1 wrapper.app.parameter.1=jetbrains.buildServer.agent.StandAloneLauncher wrapper.app.parameter.2=-ea wrapper.app.parameter.3=-Xmx384m wrapper.app.parameter.4=-Xrs wrapper.app.parameter.5=-Dlog4j.configuration=file:../conf/teamcity-agent-log4j.xml wrapper.app.parameter.6=-Dteamcity_logs=../logs/ wrapper.app.parameter.7=-Dsvnkit.http.methods=Basic,Digest,NTLM wrapper.app.parameter.8=jetbrains.buildServer.agent.AgentMain wrapper.app.parameter.9=-file wrapper.app.parameter.10=../conf/buildAgent.properties ...That works! Note that the extra svnkit setting needs to be inserted before the class name for this to work.

Thursday, September 4, 2008

Extending ASP.NET validation on the client side

One of the more annoying shortcomings of standard ASP.NET validation is it's lack of support for extending the the UI on the client side. For example, there is no way to set the css class of the input control when validation fails. I was working on some javascript to fix this when I came across an article that did just about the same thing. Combining my work with Scott's yields a bit more efficient javascript:
ValidatorUpdateIsValid  = function() {
 Page_IsValid =  AllValidatorsValid(Page_Validators);
 SetValidatorStyles();
}

SetValidatorStyles  = function() {
  var i;
  // clear all
  for (i = 0;  i < Page_Validators.length; i++) {
   var inputControl = document.getElementById(Page_Validators[i].controltovalidate);
   if( null != inputControl ) {
     WebForm_RemoveClassName(inputControl, 'error');
   }
 }
  // set invalid
  for (i = 0;  i < Page_Validators.length; i++) {
   inputControl =  document.getElementById(Page_Validators[i].controltovalidate);
   if( null != inputControl && !Page_Validators[i].isvalid ) {
        WebForm_AppendToClassName(inputControl, 'error');
   }
 }
}
ValidatorUpdateIsValid is part of the ASP.NET validation client script. It's invoked once whenever client side validation functions are triggered. For example after a field's value has been opdated (OnChange). This makes it the ideal place to hook into ASP.NEt client side validation. Hooking in to JavaScript is quite easy since in JavaScript any function can be redeclared. The last declaration is the one that is used. By inserting the script in the page content the function ValidatorUpdateIsValid is guaranteed to override the ASP.NET function that is loaded in the page header. I've packaged the script in a control that can be dropped into any (master) page to extend existing validation controls. I'll post the sources and a sample asap.

[11 okt 2010] Updated source code as suggested by Markus.

Tuesday, June 24, 2008

MbUnit with NAnt support for TeamCity 3.x

This week I've switched to TeamCity 3.1 after using CruiseControl.NET for a while. CC.NET is ok, but feels like something built on an off day to get automated builds working. TeamCity on the other hand feels much more mature. It's a lot easier to configure (no more XML fiddling for basic setup!). The UI is smooth and easy to work with. It's just a friendly tool.

No MbUnit support
TeamCity 3.1 does however have one big drawback for me; it only supports NUnit unit testing on .Net. There is no support for MbUnit, my current unit testing framework of choice. Fortunately, TeamCity can be extended quite easily. So, I was exprecting someone had already found a solution for this shortcoming. After some googling around I quickly found there is no easy solution available yet. I've tried some stuff like writing an XSL for the XML generated by MbUnit, but that did not yield any usable results.

The fix
TeamCity will monitor the build log and capture all sorts of information from it (described here), including test results. This is the easiest way to relay information from a build to TeamCity. MbUnit tests are executed by a custom NAnt task that comes with MbUnit (MbUnit.Tasks assembly). Since MbUnit is open source the source code is available and I can expand the task to do as I want. So, I've added support for an additional report type for TeamCity. Instead of logging to a file, it uses the build log to directly report results to TeamCity.

Not quite perfect
Unfortunatly, due to the way the MbUnit test runner works it's not possible to log test results in real-time as TeamCity expects. The results are only available after all tests in the task have completed. At that time the results are exported. This means timing information is not valid. The test results however are quite detailed, including console output, error output and a stack trace.

Using the task in a build
To use the new MbUnit task in a NAnt build script there are 3 options:

  1. If you copied the MbUnit.Tasks.dll into the NAnt bin folder, replace it with Alanta.MbUnit.Tasks.dll.
  2. Load the tasks from your build script: <loadtasks assembly="path-to-custom-tasks/Alanta.MbUnit.Tasks.dll">
  3. Load the tasks from the command line.
I use NAnt build scripts to build from the command line as well as for automated builds. So, I've defined the report type for MbUnit in a property: <property name="mbunit-report-type" value="Html" overwrite="false" /> ... <target name="test"> ... <mbunit types="${mbunit-report-type}" report-output-directory="TestResults" halt-on-failure="false"> <assemblies> <include name="MyProject.*.dll" /> </assemblies> </mbunit> </target>In TeamCity I can now set the mbunit-report-type property on the command line ( on the Runner tab ):
-D:mbunit-report-type=html;teamcity

This will generate both an html report and output the test results to the log file for TeamCity.

Update This solution is now part of the NAnt-Extensions project. You can download it here.

Friday, April 25, 2008

Custom security and SqlDependency don't mix

Custom security objects may seem like a good idea in a web application. The .NET security model is extensible making it easy to store more than just the basic information in the security principal (IPrincipal).

Having said that, there is also a potential problem with this when the security principal crosses appdomains. Crossing an appdomain boundary requires serialization on both sides and that's where things can go wrong.

If you run a basic web application odds are you don't register your assemblies in the GAC; that's what xcopy deployment is all about, right? Well that also means deserialization is impossible because the receiving app domain is likely not to be aware of your web application's /bin folder.

So, make sure this doesn't happen, you might think. Well, in some cases you don't have a choice. A prominent example is SqlDependency.Start. Internally the SqlDependency creates a new app domain to handle the SqlServer notifications and it will fail if you're using custom implementations of IPrincipal and/or IIdentity that are serializable. Note that if you don't make these classes serializable SqlDependency will work.

I came across this issue after switching to out-of-process session state on one of my projects. Out-of-process session state requires serialization. Since objects representing users were stored in session state, I implemented serialization support on them. All of the sudden, SqlDependency.Start starts throwing exceptions like "Assembly not found". Thankfully, Microsoft has blessed us with the sources to the .NET framework which made debugging this issue a lot less painful.