Monday, December 17, 2007

Is it a Story Card or is it a Bug?

I continue to participate in many discussions about whether a change is a bug to an existing card or if it should be tracked as a new card. Now that the third major release of our project there are some things that have come back to cause trouble.

In my opinion it gets to testing. If the change violates an acceptance test, then it's a bug. If the change is new functionality or a situation that had not been thought of, then a new acceptance test needs to be created. This can either be added to an existing story or a new story can be created for it.

The problem that we have faced is that new situations that aren't even hinted to in a story are discovered later, they happen to be not a big change, so they get put into the system as a bug. No acceptance test is created for them, and they are fixed. Weeks later, during testing, we get a question, "Is this supposed to work like this?" Yes, it is because of a bug that was entered. That's the problem. There is no tracked acceptance test anywhere that indicates the correct behavior of the system now. A new tester would be at a complete loss if he had to walk into this project, and the tester that has been with us since the beginning will forget things about the way the system works. Too many situations that require an acceptance test have not been documented, causing the same questions to be raised over and over again.

It seems almost as if this situation has created an us vs. them mentality between the developers and the tester. A bug by the tester means a failure to program the story correctly, while a new card is an admission of failure to correctly identify requirements by our tester. Both situations are true though, no matter the method used to capture the data. I push for cards not because I think the problem is not mine, but because it more accurately represents the testing status and reason for changes in the application. When a new tester walks into the project he should be able to look at the acceptance tests for all cards and perform a full regression test of the system. There should be no need to go through the hundreds of bugs and identify which of those actually caused a change to accepted behavior.

After a while, the application working the way it's supposed to is not going to be enough. If it is no longer properly documented why it's working that way, then maintenance of the application from both a development and testing aspect become a major problem.

No comments: