Houston, We have a memory leak

Recently I was investigating a memory leak with other colleagues. The windows service was giving out of memory exception. This windows services in short does lot of xml processing and serializes xml into business objects.

Naturally, first our needle of suspicion fell upon usual suspects i.e.. XMLDocument and XMLReader. Calling close and set them null multiple times didn’t help. While googling for a strong evidence against XMLDocument, I stumbled upon a clue that we were looking at wrong places and culprit may be XMLSerializer class. I googled for a connection between XMLSerializer and found tons of damning blogs and articles.

It is even documented by Microsoft in their MSDN documentation

To increase performance, the XML serialization infrastructure dynamically generates assemblies to serialize and deserialize specified types. The infrastructure finds and reuses those assemblies. This behavior occurs only when using the following constructors:

XmlSerializer..::.XmlSerializer(Type)

XmlSerializer..::.XmlSerializer(Type, String)

If you use any of the other constructors, multiple versions of the same assembly are generated and never unloaded, which results in a memory leak and poor performance. The easiest solution is to use one of the previously mentioned two constructors. Otherwise, you must cache the assemblies in a Hashtable, as shown in the following example.

Reference:

http://msdn.microsoft.com/en-us/library/system.xml.serialization.xmlserializer.aspx
http://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=98384


A very good article about debugging memory leaks

http://msdn.microsoft.com/en-us/magazine/cc163491.aspx

A nice website about memory leaks

http://blogs.msdn.com/tess/archive/2006/02/15/532804.aspx

There a knowledge base article about this problem
http://support.microsoft.com/kb/886385/en-us

Are we facing a a virtual bytes leak, a native leak or a .NET leak
Important Counters

1.Process\Private Bytes
2.Process\Virtual Bytes
3. .NET CLR Memory\# Bytes in all heaps
4. .NET CLR Memory\# Total committed bytes
5. .NET CLR Memory\# Total reserved bytes
6. .NET CLR Loading\Current Assemblies


If the curves for private bytes and bytes in all heaps diverge we either have a "native leak" which means that we have a native component that is leaking (in which case debug diag would be the next step), or we have an assembly leak.


http://blogs.msdn.com/tess/archive/2008/03/17/net-debugging-demos-lab-6-memory-leak.aspx