97 Things Every Software Architect Should Know

Some highlights: 1. Don't put resume ahead of requirements SOA == WebSerices came from above. 2. Chances are you biggest problem is not technical 3. Learn from Architect of Buildings. Survic is not going to like this one 4. My Additions: 1. User Interface rules. Ignore it at your peril. http://97-things.near-time.net/wiki/show/97-things-every-software-architect-should-know-the-book

Houston, We have a memory leak

Recently I was investigating a memory leak with other colleagues. The windows service was giving out of memory exception. This windows services in short does lot of xml processing and serializes xml into business objects.

Naturally, first our needle of suspicion fell upon usual suspects i.e.. XMLDocument and XMLReader. Calling close and set them null multiple times didn’t help. While googling for a strong evidence against XMLDocument, I stumbled upon a clue that we were looking at wrong places and culprit may be XMLSerializer class. I googled for a connection between XMLSerializer and found tons of damning blogs and articles.

It is even documented by Microsoft in their MSDN documentation

To increase performance, the XML serialization infrastructure dynamically generates assemblies to serialize and deserialize specified types. The infrastructure finds and reuses those assemblies. This behavior occurs only when using the following constructors:

XmlSerializer..::.XmlSerializer(Type)

XmlSerializer..::.XmlSerializer(Type, String)

If you use any of the other constructors, multiple versions of the same assembly are generated and never unloaded, which results in a memory leak and poor performance. The easiest solution is to use one of the previously mentioned two constructors. Otherwise, you must cache the assemblies in a Hashtable, as shown in the following example.

Reference:

http://msdn.microsoft.com/en-us/library/system.xml.serialization.xmlserializer.aspx
http://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=98384


A very good article about debugging memory leaks

http://msdn.microsoft.com/en-us/magazine/cc163491.aspx

A nice website about memory leaks

http://blogs.msdn.com/tess/archive/2006/02/15/532804.aspx

There a knowledge base article about this problem
http://support.microsoft.com/kb/886385/en-us

Are we facing a a virtual bytes leak, a native leak or a .NET leak
Important Counters

1.Process\Private Bytes
2.Process\Virtual Bytes
3. .NET CLR Memory\# Bytes in all heaps
4. .NET CLR Memory\# Total committed bytes
5. .NET CLR Memory\# Total reserved bytes
6. .NET CLR Loading\Current Assemblies


If the curves for private bytes and bytes in all heaps diverge we either have a "native leak" which means that we have a native component that is leaking (in which case debug diag would be the next step), or we have an assembly leak.


http://blogs.msdn.com/tess/archive/2008/03/17/net-debugging-demos-lab-6-memory-leak.aspx

New Ford Slogan-- Seen on TV

Quality is about finding solutions before the problems happen.

Model/View/Provider and Black Art of Mock Objects

References:
1. http://haacked.com/archive/2006/08/09/ASP.NETSupervisingControllerModelViewPresenterFromSchematicToUnitTestsToCode.aspx

2.

http://www.jpboodhoo.com/blog/AnswersToSomeGoodQuestions.aspx


3. http://blog.vuscode.com/malovicn/archive/2007/02/04/tdd-rhino-mocks-part-1-introduction.aspx


4. Rhino Mock Documentation

SOA and CRUD

SOA and CRUD

There is complete census that CRUD is Anti-SOA pattern


Simon thinks that one can think of CRUD as business events rather than service operations

http://www-128.ibm.com/developerworks/blogs/page/johnston?entry=crud_vs_business_operations_events


Maarten Mullender’s says to take the best of both worlds. Use CRUD interfaces for service when concurrent updates can be avoided because
1. Updates are seldom, or
2. Updates have only one source (person or system)

http://blogs.msdn.com/maarten_mullender/archive/2004/07/23/193524.aspx


Ramkumar Kothandaraman writes
If one of the services fails to handle the CRUD request, then the EA service should be able to handle this business exception. One of the mechanisms to handle a business exception involves executing a flow that compensates for prior activities.
A Business Analyst usually determines Compensation Logic. Compensation Logic can be either automated or manual. For example, a compensation action may involve alerting the monitoring facility when one of the services returns a business exception, leading to a manual resolution.

http://msdn2.microsoft.com/en-us/library/ms954596.aspx


My thoughts
There are going to be CRUD operation even for a SOA scoped Application. Best way of writing optimum CRUD operations is to visualizing a client talking to a service rather than a traditional RPC application.
Service APIs will be designed or dictated by client application and it is responsibilty of client to submit bulk CRUD operation with concise payload. It is responsibility of client to make sure that it has correct state.

Have no Architect in Team and this is what you get

The Winchester Mystery House

ObjectMother vs Mock/Stub

http://vikasnetdev.blogspot.com/2007/01/no-stubs-no-mocks-just-use.html
I was quoted on following forum
http://www.infoq.com/news/2007/08/object_mother

Very interesting article and also do read the comments.

I think that I am on the same page with vast majority of programmatic TDD practitioners.

How do you introduce NHibernate to VB6.0 programmers

Recently I tried to use nhibernate with a small project and have spent 60 hours to figure it out and still trying to figure out. I think that it is too many hours for a small project. In Midway, I thought of droping the idea of using NHibernate.

Best documentation for Nhibernate or best place to start is reading Java book for Hibernate or Hibernate documentation.

Ayende has some useful posts.

Let me quote from Java Experts first

Richard Conway. JDJ.SYS-CON.COM writes

Hibernate has come a long way since it was first released. It has a bewildering number of options for configuring your object persistence mappings and behavior – as well as great tools to make it if not painless then at least no so painful.

If you want to quickly persist your objects for a small project and you can manage uniqueness and referential integrity within your application code – look no further than DB4O. It just doesn’t get any easier.

The Cache/Jalapeno combination provides a compelling option for quickly persisting your java objects with a minimum of effort while providing excellent control over database-specific functionality.

Bruce A. Tate of Beyond Java book fame writes

Most of us try to learn an object relational mapper, like Hibernate. While it does relieve some of your persistence burdens, it also imposes a steep learning curve.

My Advice for fellow ex-VB6.0 programmers
Go for Code Generation route if you can go

Very soon I am going to write some posts showing O/R mapping examples using Nhibernate Custom Attributes.

Agile Methodology /Domain Driven Design wih Rich Client Application

Recently I tried Agile Methodology/Domain Driven Design for a small project. It simply rocks.
Some highlights


Step 1 Architecture



Step 2 Architecture



1. Non-disposable clean prototype.
It resulted in Non-disposable prototype. Nothing more connects with users like Screen Designs/Working prototype. Previously, I used to create disposable prototype, I was designing Screens backed by dummy database (by using DataControls). At the end, the prototype was so corrupted and had so bad references and bad code, We have to abandon the prototype.
Prototype fired by an Object Oriented Engine is real application (Tracer bullet)

2. Rich Middle Tier
Was able to nail process/Workflow classed early in the stage


3. Technical Feasibility
I was able to evaluate technical feasibility, efforts required and hence financial feasibility quite early in the game.

4. Agile
Having no database is really helping me at this time. Making changes to business objects and factory objects is very quick as compared to making changed to database and propagating it to middle tier. Carry less baggage in early really helps but I am reaching a point where I will need a database very soon.


5. 100 percent utilization of efforts

You may say that set up data for business objects is waste of time. Take a breath. I am using this code for my setup for tests. They will represent the functional tests and they are going to be independent of database and hence faster.

6. Free Set up for Functional Unit Tests


7. Window Forms Rock
Sometime back, Survic posted that he is going to use Windows Forms to develop prototype for Web Applications. It really shocked me. Now I see some weight in that argument. I remember that to get the docking, anchoring and other effects in older visual basic version, I have to write some very mathematics intensive routines. Now it is coming out of box.


8. Design Patterns Applied

1. Factory
2. Proxy
3. Clone

9. Application Validation Block Rocks

It is very easy to use (within one hour, I was able to set it and use it comfortably.
It is extensible enough to coexist with others custom validation manager like CLSA.
It also supports reading validation messages from Resource Files which is very essential for global applications. I highly recommended for small projects. Since it uses custom attributes and reflection, one may be careful about batch validation.


10. NHibernate

Very extensible and complex. It requires a hugh learning curve. I don't recommend it to fellow ex-VB6 developers to try in a time-critical project for first time.

11. Resharper Rocks
Excellent productiviy booster. It is must have tool for C# developers. I am going to buy a personal copy very soon. I used the evaluation copy.


12. Under Investingation
1. Windows Form Databinding
3. Licensing Software
3. Microsoft Map

No Stubs, No Mocks -- just use ObjectMother -- TDD

http://vikasnetdev.blogspot.com/2006/05/when-to-use-mock-objects.html
Couple of days back, I wrote that I am not using Mock/Stub objects but I have moved creation of my complex business objects in my testing framework to another helper classes. Today I came to know that there is pattern for this concept from Thoughtworks named as ObjectMother

My Default SOA Architecture using WCF

Continued from http://vikasnetdev.blogspot.com/2006/07/soa-friendly-architecture-version-of.htmlWith the advent of Net3.0, I am upgrading the implementation of my default SOA friendly architecture to use WCF as a medium to connect my business layer with database layer


Note: The author of this blog does not believe that you should use WebSevice/WCF in standalone application when none required. But we live in a world where we don't make rules. If you database is protected by another firewall, you may find this architecture handy. It is more important to have logically separation in architecture that will scale well if application is deployed in a distributed manner.


Here is SOA friendly Architecture


Here is deployment view


Major Components

1. UI Components/Views:
Provide User Interface

Controllers
Get the Business Objects from Gateway and provides to UI

Gateway/Factory
It is factory layer that provides the business object. It knows how to create or get the object across the network boundaries. First looks in repository if none, gets the object across the wire.Our example is going to exploit the capabilities of Web Service. It will convert the proxy object into real domain object using Mapper utility.

References:
http://www.martinfowler.com/eaaCatalog/gateway.html
http://www.martinfowler.com/eaaCatalog/repository.html
http://www.martinfowler.com/eaaCatalog/dataTransferObject.html


Business Object Manager
It populates the business object using data object. This is where one can aggregate the business objects before transferred across the network.


Business Object/Entity Object
Contains the Attributes and operations related to domain entity

Process Object/Workflow Object
Business objects collaborate together to perform some useful operations to users.


Data Object
Data Objects retrieve the data from Database. Will contain all the CRUD operations.

Here is Code.

Gateway


namespace Gateway
{
public class Contact
{
static Contact()
{
//
}
private static List contactListCache = null;
private static Object syncRoot = new Object();
public static void ClearCache()
{
lock (syncRoot)
{
contactListCache = null;
}
}
public List GetContact()
{
return ContactList;
}

public List ContactList
{
get
{
if (contactListCache == null)
{
lock (syncRoot)
{
contactListCache = new List();
ContractServiceWS.ContractServiceClient contractService = new ContractServiceWS.ContractServiceClient("WSHttpBinding_IContractService");
ContractServiceWS.Contact[] proxyContactList = contractService.GetContract();
for (int i = 0; i <= proxyContactList.Length - 1; i++) { BusinessObject.Contact contact = new BusinessObject.Contact(proxyContactList[i].UserName, proxyContactList[i].Name, proxyContactList[i].PhoneNumber, proxyContactList[i].Age); contactListCache.Add(contact); } contactListCache = (List)Converter.MapProperties_Fields((System.Collections.IList)proxyContactList, typeof(List), typeof(BusinessObject.Contact));
}


}
return (contactListCache);
}
}
}
}


namespace Gateway
{
public abstract class Mapper
{

public static object MapProperties_Fields(object SourceObj, Type DestinationType)
{
if (SourceObj == null)
{
return null;
}
Type sourceType = SourceObj.GetType();
object destinationObj = Activator.CreateInstance(DestinationType);

foreach (PropertyInfo sourceProperty in sourceType.GetProperties())
{

if (sourceProperty.Name.Equals("ExtensionData") == false)
{
MemberInfo destinationMember = DestinationType.GetMember(sourceProperty.Name)[0];

if (destinationMember.MemberType == MemberTypes.Property)
{
PropertyInfo destinationProperty = ((PropertyInfo)(destinationMember));
if ((destinationProperty.CanWrite == true))
{
if (destinationProperty.PropertyType.Equals(sourceProperty.PropertyType))
{
destinationProperty.SetValue(destinationObj, sourceProperty.GetValue(SourceObj, null), null);
}
}
}
}
}
return destinationObj;
}



public static IList MapProperties_Fields(IList SourceList, Type DestinationListType, Type DestinationElementType)
{
if (SourceList == null)
{
return null;
}
IList destinationList = ((IList)(Activator.CreateInstance(DestinationListType)));
foreach (object sourceObj in SourceList)
{
destinationList.Add(MapProperties_Fields(sourceObj, DestinationElementType));
}
return destinationList;
}
}
}





WCF Service

[ServiceContract()]
public interface IContractService
{
[OperationContract]
IList GetContract();
}

public class MyService : IContractService
{
public IList GetContract()
{
BusinessObjectManager.Contact contactManager = new BusinessObjectManager.Contact();
return contactManager.GetContacts();
}
}



Business Object Manager

namespace BusinessObjectManager
{
public class Contact
{
public List GetContacts()
{
DataTable dataTable;
List list = new List();
dataTable = DataObject.Contact.GetContact();
try
{
foreach (DataRow dataRow in dataTable.Rows)
{
BusinessObject.Contact contact = new BusinessObject.Contact(dataRow[0].ToString(), dataRow[1].ToString(), dataRow[2].ToString(), Int32.Parse(dataRow[3].ToString()));
list.Add(contact);
}
}
catch (Exception ex)
{
throw ex;
}
finally
{
dataTable = null;
}
return(list);
}
}
}



Business Object


namespace BusinessObject
{
[DataContract]
public class Contact
{
private String _userName;
private String _name;
private String _phoneNumber;
private int _age;
public Contact()
{
}
public Contact(string userName, string name,String phoneNumber, int age)
{
_userName = userName;
_name = name;
_phoneNumber = phoneNumber;
_age = age;
}
[DataMember]
public String UserName
{
get
{
return _userName;
}
set
{
_userName = value;
}
}
[DataMember]
public String Name
{
get
{
return _name;
}
set
{
_name = value;
}
}
[DataMember]
public String PhoneNumber
{
get
{
return _phoneNumber;
}
set
{
_phoneNumber = value;
}
}
[DataMember]
public int Age
{
get
{
return _age;
}
set
{
_age = value;
}
}
}
}



Data Object
namespace DataObject
{
public class Contact
{
private const String ContactTableName = "Contact";

private class Fields
{
public const string UserName = "ContactUser_Name";
public const string Name = "Contact_Name";
public const string PhoneNumber = "Contact_Phone_Number";
public const string Age = "Contact_Age";
}


public static DataTable GetContact()
{
System.Text.StringBuilder sql = new System.Text.StringBuilder();
System.Data.DataTable dataTable;

//Build the SQL

AppendBaseSelectColumns(sql);

sql.Append(" FROM");
sql.Append( " " + ContactTableName);

try
{
dataTable = GetDataTable(sql.ToString());
}
catch (Exception ex)
{
//Log the error
throw ex;
}


return(dataTable);


}

private static String AppendBaseSelectColumns(System.Text.StringBuilder sql)
{

sql.Append("SELECT");
sql.Append(" " + Fields.UserName + ",");
sql.Append(" " + Fields.Name + ",");
sql.Append(" " + Fields.PhoneNumber + ",");
sql.Append(" " + Fields.Age );
return sql.ToString();

}

private static DataTable GetDataTable(string sql)
{
//Real life one will connect to Database and return the
DataTable dataTable = new DataTable();
dataTable.Columns.Add(new DataColumn("Contact_User_Name", typeof(String)));
dataTable.Columns.Add(new DataColumn("Contact_Name", typeof(String)));
dataTable.Columns.Add(new DataColumn("Contact_Phone_Number", typeof(String)));
dataTable.Columns.Add(new DataColumn("Contact_Age", typeof(System.Int32)));
dataTable.Rows.Add(new Object[]{ "VK", "Vikas","222-222-2222",int.MaxValue});
dataTable.Rows.Add(new Object[] { "SV", "Survic", "222-222-9999", int.MaxValue });
return dataTable;

}

}
}


Presenter/Controller
namespace Presenter
{
public class ContactPresenter
{
public ContactPresenter(View.IContactView view)
{
_view = view;
_view.LookUp += new EventHandler(_view_LookUp);
_view.Persist += new EventHandler(_view_Persist);
}

private View.IContactView _view;
private BusinessObject.Contact _currentContact;


public View.IContactView View
{
get { return _view; }
}
public BusinessObject.Contact CurrentContact
{
get { return _currentContact; }
}


public void SaveContact()
{
if (_currentContact != null)
{
_currentContact.Name = _view.ContactName;
_currentContact.PhoneNumber = _view.ContactPhoneNumber;
_currentContact.Age = _view.ContactAge;
}
}

public void LookUpContactByUserName(string userName)
{
Gateway.Contact contactGateway = new Gateway.Contact();
List contactList = contactGateway.GetContact();

foreach (BusinessObject.Contact contact in contactList)
{
if (contact.UserName.ToUpper().Trim().Equals(userName.ToUpper().Trim()))
{
_currentContact = contact;
break;
}
}

_view.ContactUserName = _currentContact.UserName;
_view.ContactName = _currentContact.Name;
_view.ContactPhoneNumber = _currentContact.PhoneNumber;
_view.ContactAge = _currentContact.Age;
}

private void _view_LookUp(object sender, EventArgs e)
{
this.LookUpContactByUserName(_view.UserNameToLookUp);
}

private void _view_Persist(object sender, EventArgs e)
{
this.SaveContact();
}
}
}

Pitfalls of Development Driven Testing -- TDD

There comes a time during battles when elite commandos have to throw away their best gadgets which may include sophisticated gears, body armors, tools and weapons and resort to hand-to-hand combat. During one of those moments, I wrote the following blog post
Confessions of a failed extreme programmer

After the fog of war was over, I was told by my colleagues about the bugs in my components in certain scenarios. These bugs would have been never there, if only I had written my unit tests. So I started writing my tests using the framework which I prepared.
To me surprise, my test was succeeding while in real condition, it should have failed. After serious debugging, I found a serious bug in my framework and as a result, 40% of my automated tests were flawed.


Well, if you survived your hand-to-hand combat, you have to oil your gear.

Moral of the story: It does not hurt to use a knife during hand-to-hand combat. Not using it may prove to be fatal. Automated Unit tests are knives to Developers locked in a mortal combat with very aggressive deadlines.

Why are all great developers not the best Architects and vice-versa?

I have seen lot of exceptions though.

First the philosophical spin. Michael Plat says that Architect are analytic thinkers and programmers are logical thinkers and hence the mis-match. He gives a simple test to figure out whether you are a Architect or not. I failed the test but I still believe that I have excellent Architect’s prospective. :)


I think that real issue is that strong programmer fails to abstract or trivialize certain aspects of system. They are too involved in details and thus , are unable to see the system from 18000 feet.

As Survic said, "abstract a little -- not too high level, but a little bit higher than programming - "design pattern."

If you abstract too high, it is time to ask your management to promote you to astronaut or ivory-tower architect designation. Don’t mistake me, I don’t have any contempt for them. I do think that Corporate needs them also very badly.