Building and testing Async support in Reflector

The journey towards asynchrony

When we first started working on the current version of Reflector, one of our main objectives was to provide full support for new Microsoft technologies. Since then our team has worked on making Reflector available for Dev11, and compliant with its new visual theming scheme. The async features in the new versions of C# and VB are a major step forward towards asynchrony and its implementation has been made much easier in practice.

With the introduction of the new Async/await keywords and new approach to Task-based programming, we realised that it wouldn’t be long before our users wanted to find out more about what’s happening under the bonnet with the new asynchronous programming model.

We started work on providing Async support back in April. There were quite a few challenges, mostly around understanding the intricacies of asynchronous programming from the source available out there, trying to understand the different machine states, and trying to reverse engineer the code. The Async/await keywords do not exist in the IL and the compiler rewrites your VB and C# code producing hidden classes or compiler generated classes, which you can see using Reflector.

These hidden classes have a defined state variable and rely on the different state transitions to provide asynchrony. A given method can be in any one of these Async states at any given time. The movenext() method within the hidden class generated by Reflector shows all the different state transitions possible. For more information on how to use Reflector to understand any Async implementation, have a look at Clive’s earlier Async blog post.

We also had to keep in mind that the Dev11 examples compile under the C#5 compiler, and the asynchronous implementations built into the .NET 4.5 framework have been compiled using a separate internal compiler. From a development perspective, it meant that we would have to introduce additional rules to handle the two separately.

Once we’d figured out how to solve those issues, we were faced with the new challenge of testing the sheer amount of asynchronous implementation contained within Framework 4.5. There are around 360+ asynchronous methods within mscorlib, System, System.Xml and System.Web libraries alone.

We created a spreadsheet of all these async CodeLinks. It’s available to download

Our objectives for testing were to ensure that the code translation from IL to C# was correct, that Reflector handled the state machine transitions in a proper manner without missing out any logic, and that the final code looked much like its corresponding synchronous implementation. Apart from the task of testing the .NET 4.5 framework libraries, we also wanted to test the executables and libraries generated using the C#5 compiler.

Testing is all about information and communicating the health of the feature being developed. It’s about coming up with new ideas and approaches to test a particular piece of software. The most traditional approach to testing is writing test cases and tracking the health via its pass and fail status for each build. Every project modifies this approach to suit its needs, of course.

In our case, we had only one test case – verifying that Reflector correctly decompiled Async methods. On the other hand, our test data is infinite!

We had to draw our scope of testing and prioritize based on the time and resources we had. Our strategy was quite straight-forward: automate as many tests to validate the decompilation, hand-pick a few of these methods to test the logic, and maintain a visual graph which would keep the team focused and our vision insight.

The moving graph

Once all the methods with asynchronous implementation had been identified, the CodeLinks were generated to make it easier for us to open these methods in Reflector for testing. You can download a copy of the exhaustive list of asynchronous methods in .NET 4.5 if you’re interested.

A codelink is like a bookmark, and is a combination of assembly version, the namespace, and the method it denotes. The assemblies should already be loaded in Reflector assembly browser before the methods can be opened.

Phase 1:

key:
Unimplemented methods
Implemented with correct translation
Implemented with logic yet to be tested
Error in translation

Phase 1 involved sizing the Async work and identifying the scope of testing. We were still trying to answer few questions around asynchrony -most asynchronous methods failed to decompile in .NET 4.5 and most of what you could see in the final code was the implementation of the movenext() method. Clive was still scratching his head around the machine states and trying to fit the pieces of the puzzle together. The async and the await keywords were finally added to the Reflector dictionary!

Phase 2:

In phase 2 Reflector could generate the code for most asynchronous methods with fair reliability. We also added support for the Dev11 examples.

There are quite a few asynchronous methods which have a corresponding synchronous implementation within the framework library which helped us counter check the logic. The logic could still be very different, and I wouldn’t suggest this as a primary mode of testing. The best approach to follow would still be to understand the different state transitions and check if the final code followed the same workflow.

An example

Let’s have a quick look at ParsePIAsync() method within System.Xml.Dtdparser and its corresponding synchronous implementation ParsePI():

Asynchronous:

private async Task ParsePIAsync()
{
	this.SaveParsingBuffer();
	if (this.SaveInternalSubsetValue)
	{
		await this.readerAdapter.ParsePIAsync(this.internalSubsetValueSb);
		this.internalSubsetValueSb.Append("?>");
	}
	else
	{
		await this.readerAdapter.ParsePIAsync(null);
	}
	this.LoadParsingBuffer():
}

Synchronous:

private void ParsePI()
{
	this.SaveParsingBuffer();
	if (this.SaveInternalSubsetValue)
	{
		this.readerAdapter.ParsePI(this.internalSubsetValueSb);
		this.internalSubsetValueSb.Append("?>");
	}
	else
	{
		this.readerAdapter.ParsePI(null);
	}
	this.LoadParsingBuffer();
}

On close inspection you’ll find that both the methods apply the same logic except that in ParsePIAsync() the call to ParsePIAsync(StringBuilder sb) is being done asynchronously. This is one of the many examples lying around in the framework 4.5 libraries.

Microsoft has introduced an asynchronous version for many of its time-consuming operations, and you can see all of these in our CodeLinks spreadsheet.

Phase 3:

In phase 3, we achieved a pass rate of 91%. The percentage is calculated using only the asynchronous implementations specified in the graph. The figure is not absolute and is only suggestive of the trend.
The red areas are mostly to deal with code clean-up and removal of redundant or unreachable code and unused variables from the final output.

Our Async work took a pause at Phase 3, mostly because we were now at a stage where we could claim that Reflector supports decompilation of asynchronous implementations. Also, we wanted to shift our development focus towards enhancing SharePoint support within Reflector. If you have SharePoint Foundation or SharePoint Server installed on your machine, Reflector should be able to automatically identify all the SharePoint-related assemblies for you and provide you with the option to create an exclusive SharePoint Assembly List.

What’s not covered in our tests?

We haven’t given much time to test the Async methods within the WinRT assemblies; however you should still be able to view their implementation using Reflector. Our emphasis for testing has mostly been C#.

Our plan is to clean up the final output and fix these issues before 7.6 is released.

What does this mean for our users?

You can now decompile any of the asynchronous methods within the 4.5 Framework and view the details of it. You should also be able to decompile other third party assemblies compiled using the C#5 compiler. With our debugging features within Visual Studio, you can decompile any asynchronous method, set breakpoints and examine the threads being used by your application.

As the saying goes – “testing can never be 100% complete” – we’d love to hear from you and receive your feedback on how Reflector decompiles your asynchronous code. It would help us greatly to improve the quality of decompiled code Reflector generates.

Leave a Reply

Your email address will not be published. Required fields are marked *