I am sure, many of you had frustrating experience from time to time.
How bad it is to be in situation where a developer causes the whole release to be delayed unnecessarily!
Or one mistake by a developer causes all developers/development teams to suffer?
Or even worse, how a developer cause a huge live bug!
None of those are fun to have and indeed very much frustrating.
What shall we do?
Someone shall take the responsibility because I hate to hear sorry after sorry!
We shall penalize the developer also?
Even better, the TEAM that developer belongs to be penalized so they will be more careful next time!
But what is the solution? How do we penalize a tech team?
Of course not with money!
And firing is also not a problem.
I have a solution which works perfectly!
I simply force the whole team to do one of the following in order for them to be allowed to be checking in.
- Fix some code analysis rules (I change the warnings to “errors”). Build fails if we have erroneous rule failures (VS 2010 code analysis or fxcop 10.0)
- Enforce to write more unit tests to increase code coverage rate.
And how actually I guarantee that it is done properly.
Thanks to TFS 2010 Gated Check-ins.
- Each check in is controlled by gated check-ins
- Check-in fails if
- Solution does not build
- There are errors in code analysis rules
- A Unit Test Fails
- Code coverage rate is below last coverage value (we’ve customized build template in order to achieve this)
- If they need to be penalized I enforce them to cover 1% more code with unit tests
- Here is an example result:
That is our solution and results are improved code quality.
I thank Microsoft for TFS 2010 and automated builds.
And thanks to Serdar for implementing build template modification.
It will be a bit odd but I thank myself for the innovation here and getting deep into the capabilities which can be controlled.
Static code analysis is very important to save time and retain code standards.
We actively use Visual Studio 2010′s Code Analysis feature which is very close to FXCop 10.0. (It is better)
And the best thing is, we were able to add this to team build to enforce gated check-in to fail when rules fail
This is saving tons of time for us and prevent horrible code. Well it is not a guarantee that code will be great, developers can still find innovative ways of messing up the code but at least we have certain standards. On top of MS provided tools, the following tools also worth checking
Recently I’ve been analyzing the GUI test automation tools and two of those tools stand out very well.
http://seleniumhq.org/ (Thanks Jalal)
We are currently evaluating those tools and soon I hope we will hook them up to TFS Team Builds for better build results.
So far http://watir.com/ seems to be the champion on simplicity of the implementation and coverage among browsers.
I think soon those tools will shake up the global market in QA Automation testing.
Also it worth to check the OpenQA site for other variaties. http://openqa.org/
It has been constantly asked by the designers to not to wait for their non code related check-in attempts to fall into gated check-in process.
They really have a point here actually.
Microsoft failed to think about this part however they made it very smart to make build process customization via xaml (WF). Finally today we will start bypassing full build cycle when only design related items are attempted to check-in.
TFS (Team foundation sever) 2010 is the central management and extensible.
I don’t wish to think a tech (.NET) team without TFS in place.
You simply get so much out of TFS.
One of the most important features of TFS 2010 is Gated Check-in which is integrated into Team Build (Automated TFS build engine)
Gated Check-in, gives us so much control over the quality of the code that is attempted to check-in.
At the moment, we are enforcing the following for check-in to be allowed
- Code must build of course
- All unit tests must succeed (defined ones as per branch)
- All enabled (error giving) code analysis rules must pass without any errors
- All reliability rules are turned on
- Some maintainability rules are turned on
- Some performance rules are turned on
Woow, even those are a huge step when you don’t have any control over the code check-ins.
Also planning the following enforcements in the near future
- Overall branch code coverage shall not go down
- If previous check-in reach to coverage level of 7.56% as an example, the next check-in attempt shall not bring it down to 7.5599999%
- Of course we will enforce it, only if total coverage is below 25%
- I have not decided when it will be over 25% (I guess compared to now, I will be very happy)
- All new methods/properties/classes implemented in the check-in attempt must have at least 25% code coverage otherwise check-in will be rejected.
- Well, developers are not writing unit tests because they still don’t understand it. I guess the only way to teach is to enforce. Sad but true. I don’t have any other choice..
- But I have a tool
- All nightly builds will trigger QTP automation against the auto deployed branch codes to make sure nothing is broken.
- Enable code metrics analysis and reject check-ins based on certain criteria
- Rules yet to be determined…
I guess it is well enough here to see why I love TFS….