Agile Testing
(illustration by Rybson)
Agile, the hottest software development lifecycle (SDLC) nowadays, was initially used in small projects and considered not suitable for medium and large projects. But now large companies have picked up the trend and adopting it into their major projects.
By now we’ve agreed about the importance of testing in all stages of software development project. Yet, those stages are actually closer to the waterfall or other iterative and incremental SDLC with longer phase period. Agile iteration, typically called sprint, are a lot shorter and only last for 2-4 weeks.
In this blog post we’ll see the challenges, myths, and how the testing in agile can be performed.
The sources of discussion are the following videos:
- Agile Quality Assurance in Software Testing
-
Agile Testing: Google TechTalk
- Agile Testing: Some Myths Exposed
- Balancing Metrics, Lean, and Agile
Some Challenges
Agile Quality Assurance in Software Testing
Quickly one can sense challenges for tester here. First, the testing period is also shorter, and the developers that were working on the modules are already working on the next iteration if defects are discovered later.
So as we can see, as good and promising the agile SDLC is, it’s still not the holy grail that is without challenges.
Agile Testing
Google Tech Talks by Elisabeth Hendrickson from Quality Tree Software, Inc.
*The PDF mentioned in the video is no longer available, this Agile Testing Overview is the closest one I found.
Elisabeth was very enthusiastic and here are some of the interesting take ways from the presentation:
“I felt loved and appreciated, now that’s a weird feeling for a tester.”
Traditional testing – test/bug fix at the very end of the project, it’s tester responsibility to “assure the quality” at the end.
Shift focus from being the last line of defense to providing information, feedback, and visibility.
Comparing traditional testing and agile testing:
-
Change
Traditional testing: manage and control it
Agile testing: accept it - Planning
Traditional testing: comprehensive up front test design
Agile testing: plan as you go - Documentation
Traditional testing: verbose
Agile testing: only as much as absolutely necessary - Handoffs
Traditional testing: formal entrance and exit criteria with signoffs
Agile testing: it’s not a relay race. Collaborate - Automation
Traditional testing: system level, built by tool specialists, created after the code is “done”
Agile testing: all levels, built by anyone, an integral part of the project
“The more informal it is, the easier it is to change…”
Whiteboard, sticky note, index cards, wiki vs databases, gantt/pert charts, comprehensive, documents from standard template.
“Capture the essence, not the details.”
Co-locate testers and programmers. – but sitting side by side does not ensure communication
“I love test driven development, I think it is the coolest thing since forever..”
Tester collaborate with developers to make the tests, no untestable code.
“We are all testers, no matter what our title is.”
“Who has the biggest problem moving to agile development? Testers of developers? Yes.”
This video was made in 2005, 7 years from now (June 2012), and although most of the agile principles are still the same, a lot of things have happened since then.
Now the agile methodology has a lot more practitioners, tools, and adoption in the industry. However, the change is not easy. Agile methodology is not implemented by just calling the project “agile”, buying the agile project management software, or even doing the agile rituals. It requires a paradigm shift, or even brainwash if necessary, of all the development team members.
Thus, it’s not a common thing to see that people misunderstand how to do agile and myths about the methodology are circling around.
The Myths
In this video, George Wilson from Original Software talks about the myths of testing in the software projects adopting Agile. It is really an insightful presentation, and I strongly recommend anyone to watch the full video to clarify some of the common misconception about testing in Agile project.
Here are the myths:
1. You only need TDD
We use TDD, so we do not need to perform separate testing.
2. Re-use Unit tests
We don’t need to build separate regression tests, we will re-use the unit tests.
As good and complete the unit tests are, they are not enough for regression test. Again, at the code level the unit tests are needed to make sure that new updates don’t break the existing functions. However, they’re not enough at functional level.
3. No testers or tools
We don’t need testers or automation tools.
4. Automation is impossible
Moving goalposts and application evolution meaning automation is wasted effort.
5. Unit tests = specification
100 % of the specification in the unit test.
6. Start with easy tasks
Gets the process started with the simple requirements.
7. No documentation
Documentation not required, the delivered system is the focus.
8. Fix errors later
Focus on the end result and ignore errors.
It’s interesting how George personally believes that Test-Driven Development (TDD) is not ‘testing’, it’s merely making sure the programmers know the cases and what should happen in each case.
I’m not fully agree with his statement although there is some truth in it. The tests in TDD are not enough, there are certainly more testing to be performed on top of the unit test in TDD. But at the code level, the unit tests is indeed part of the overall testing in a software project.
George mentioned that testing is an important thing, and actually even more tests are needed in an agile project. For this reason, replacing the manual tests with automation tools is neeeded.
Indeed, hearing the word “Agile” we are attempted to think that it’s a less formal methodology, in which everything is magically faster, better, and successful.
This is far from the truth, agile methodology is a disciplined method where testing is built-in from the beginning instead at the end of the project.
Although in agile the document is not as bulky as in the waterfall approach, documentation is also still an important part of the delivery. The documentation is not replaced by the tests. The idea is to keep the document to have the necessary information, and not become the all-in-all of the project.
But then how we strike this balance? Can we be agile and yet have all the complete project documentation and metrics?
The Agile/Lean Project Documentation and Metrics
Rex Black from RBCS, a consulting service company post this question about how to stay “lean and agile” while maintaining visibility?
There are two great comments to his question by James Coplien and Steen Lerche-Jensen.
James reminded us that lean does not mean lightweight, it means reducing waste along the value stream, and agile leverage a lot on automating the process.
Another key point he brough is that the most metric that we need to measure is value. This is one of the important point of agile, the software produced must bring value to the stakeholders. People can become concern about the schedule and cost of the project, instead of the product value.
Steen listed the type of metrics that are necessary in the four Agile quadrant, from the metrics to cover the User Stories, exploratory test, UAT and usability level, and test on the finished product. He mentions that these metrics are useful to make use of the knowledge and become a learning organization.
The article can be read here, and here is his video blog on Youtube.