Wednesday, March 24, 2010

Automation of Testcases is Must

The following are the general problems raised by many in the Software Industry.

  • The developers are not spending enough time in testing.
  • The code quality of the developers is too low, and there are many regressions.
  • We are not getting enough testers. Everyone is opting only for the development.
  • The test team is always overloaded.
  • Eventhough the developers are ready to change the architecture, the test team is not ready to take the change.
  • Test team is not ready to take even small bug fixes, eventhough the development team is ready to fix.
  • We got few issues in production, but the issues are not reproducible in debug build.

The single solution for all the above is, automation of all the testcases. Any other solution is not an optimum solution for this. Automating all the testcases takes extra time, but, it is definitely worth it.

Once the developer is done with the functionality, the developer must write all the automated testcases with respect to the functionality, including the scenarios that are impacted by this feature. The automated testcases can be reviewed by a tester or any other person to make sure whether the developer covered all the scenarios or not. The goal should be, zero manual testcases to execute. If the scenario is too expensive to automate and not very frequent, then probably manual testcase is fine. But, the number of automated testcases should be minimum 99%.

For any developer, for each small feature/bug fix that he/she does, apart from the testing of their feature/bug, if they have to spend more time in doing manual testing for the existing functionality, they will get bored. Eventually, they may not spend full effort in testing existing functionality. That may lead to regressions, and code quality may go down. The developer would be blamed for the regressions, and if this continues, then the project would be forced to have a separate test team which tests everything before releasing. But, if there are automated testcases, before checkin, the developer can run the entire suite of testcases and make sure that there are no regressions.

If the developer is working on a very big functionality, the entire development would be in different phases. Initially, the developer writes for main line scenarios, then tests those, and implements for the other scenarios, tests, and finally refactors the code, and again tests those. While doing these, in the last step, if the developer misses any testcase, and if there is a bug in that, he would be blamed for that. Instead, if the developer writes the automated testcases, then after that, there will not be any need in testing further.

If anyone has good programming skills, then they will not be interested in doing manual testing. But, they may prefer to automate the testcases. If a company does not get enough testers, they should understand why they are not getting testers, and fix the actual issue. If they force all the testers to do manual testing citing reasons that they don't have enough time, then they will not get enough testers to test those. If there are not enough testers, then obviously, testers would be overloaded.

If a developer fixes some bug or changes some architecture, and if it is easy to verify that the existing functionality is not broken, then that can be taken at any point in the release. If there are no automated testcases, most of the testers would not prefer to take those at the end of the release.

Some times, testers test with debug build, and do not execute the same testcases on the release build. Finally, in the production they get issue with the release build. Eventhough, most of the time, the code reviews (if at all the development team has) take care of this, in case of issues like timing etc., it may not be very obvious.

All the above problems would be easily solved by automation of all the testcases. All the project leads tell that it is very expensive to do that. Still it is worth it.

1 comment: