Sevket Seyalioglu

This is really interesting for unit testing. A new approach called approval testing. Well not too interesting but nicely done approach.

Requires some thinking over this.
And wondering really how this will work with TFS automated builds (unit testing integrated mode)

Plus there is more in ASP.NET unit testing or we shall more say results testing.
I am posting this because it is interesting concept. I don’t really approve changing the base code to do testing.

Ohh yes, check this framework for more details here: http://approvaltests.sourceforge.net/

Well,

I am sure, many of you had frustrating experience from time to time.
How bad it is to be in situation where a developer causes the whole release to be delayed unnecessarily!
Or one mistake by a developer causes all developers/development teams to suffer?
Or even worse, how a developer cause a huge live bug!

None of those are fun to have and indeed very much frustrating.

What shall we do?
Someone shall take the responsibility because I hate to hear sorry after sorry!

We shall penalize the developer also?
Even better, the TEAM that developer belongs to be penalized so they will be more careful next time!

But what is the solution? How do we penalize a tech team?

Of course not with money!
And firing is also not a problem.
– LOL
I have a solution which works perfectly!

I simply force the whole team to do one of the following in order for them to be allowed to be checking in.

• Fix some code analysis rules (I change the warnings to “errors”). Build fails if we have erroneous rule failures (VS 2010 code analysis or fxcop 10.0)
• Enforce to write more unit tests to increase code coverage rate.

Thanks to TFS 2010 Gated Check-ins.

• Each check in is controlled by gated check-ins
• Check-in fails if
• Solution does not build
• There are errors in code analysis rules
• A Unit Test Fails
• Code coverage rate is below last coverage value (we’ve customized build template in order to achieve this)
• If they need to be penalized I enforce them to cover 1% more code with unit tests
• Here is an example result:
  

That is our solution and results are improved code quality.
I thank Microsoft for TFS 2010 and automated builds.
And thanks to Serdar for implementing  build template modification.
It will be a bit odd but I thank myself for the innovation here and getting deep into the capabilities which can be controlled.

Static code analysis is very important to save time and retain code standards.

We actively use Visual Studio 2010′s Code Analysis feature which is very close to FXCop 10.0. (It is better)
And the best thing is, we were able to add this to team build to enforce gated check-in to fail when rules fail

This is saving tons of time for us and prevent horrible code. Well it is not a guarantee  that code will be great, developers can still find innovative ways of messing up the code but at least we have certain standards. On top of MS provided tools, the following tools also worth checking

http://www.jetbrains.com/resharper/
http://www.ndepend.com

Recently I’ve been analyzing the GUI test automation tools and two of those tools stand out very well.

http://watir.com/

http://seleniumhq.org/ (Thanks Jalal)

We are currently evaluating those tools and soon I hope we will hook them up to TFS Team Builds for better build results.

So far http://watir.com/ seems to  be the champion on simplicity of the implementation and coverage among browsers.
I think soon those tools will shake up the global market in QA Automation testing.

Also it worth to check the OpenQA site for other variaties. http://openqa.org/

It has been constantly asked by the designers to not to wait for their non code related check-in attempts to fall into gated check-in process.

They really have a point here actually.

Microsoft failed to think about this part however they made it very smart to make build process customization via xaml (WF). Finally today we will start bypassing full build cycle when only design related items are attempted to check-in.

TFS (Team foundation sever) 2010 is the central management and extensible.
I don’t wish to think a tech (.NET) team without TFS in place.
You simply get so much out of TFS.

One of the most important features of TFS 2010 is Gated Check-in which is integrated into Team Build (Automated TFS build engine)

Gated Check-in, gives us so much control over the quality of the code that is attempted to check-in.

At the moment, we are enforcing the following for check-in to be allowed

1. Code must build of course
2. All unit tests must succeed (defined ones as per branch)
3. All enabled (error giving) code analysis rules must pass without any errors
1. All reliability rules are turned on
2. Some maintainability rules are turned on
3. Some performance rules are turned on