Use Case Thread Generation for
Acceptance Testing
From Theory to Practice
Doug
Rosenberg
ICONIX
www.iconixsw.com
Since I’ve been working with (and writing books about) use
cases since the early 1990s, it’s not surprising that I’ve been running into
the problem of generating acceptance tests from use cases for a couple of
decades. Back in the mid-90s, one of my
co-workers at ICONIX built something called a “thread generator” as an add-in
to one of the popular modeling tools of that era, while working onsite as a
consultant for one of our training clients.
Thanks to the development team at Sparx systems, this test case
generation capability is now available in version 8 of Enterprise Architect.
The basic concept of “thread generation” from use cases is
quite simple. I’ll illustrate it with an
example later in this article, but here’s a quick summary. Use cases generally have a “sunny day
scenario” (the typical sequence of user actions and system responses), and several
“rainy day scenarios” (everything else that can happen, like errors,
exceptions, and less typical usage paths).
When you’re modeling systems with use cases, you make an upfront
investment in “writing the user manual” one scenario at a time, with the
expectation of recovering your investment later on in the project. If you do it right,
you virtually always recover your investment multiple times over.
One of the many places you can leverage the investment
you’ve made in writing use cases is in acceptance testing. Simply put, if you consider a hypothetical
use case with one “sunny day” scenario and four “rainy day” scenarios, you can
expand that use case into at least five “threads”, each of which defines a potential
acceptance test for some part of your system.
There’s one “thread” for the sunny day scenario and others for some
portion of the sunny day combined with each of the rainy day scenarios, in
turn. When you’ve covered all of the
permutations, you’ve got a pretty good start on “black box” acceptance-testing
your use case from the user perspective.
So, while I’ve long been aware of this problem and the
desirability of solving it, it has always been a manual, labor-intensive
process for most companies. However,
over the last year, several factors have converged which have now resulted in a
remarkable implementation by Sparx Systems as part of the new “structured
scenario” capability in Version 8 of Enterprise Architect.
As it happens, I’ve been involved as an instigator,
specifier of requirements, and reviewer of the capability, although most of the
hard work was done by the team at Sparx.
In the remainder of this article, I’ll explain how the capability came
about, how it works, and why your company’s Quality Assurance team will be very
happy to have it.
Being the grain
of sand in the oyster
Everyone knows that it requires a grain of sand as an
“irritant” in order for an oyster to make a pearl. In some respects, I’ve played that role with
respect to getting this capability automated – agitating and occasionally
irritating the team at Sparx with the result being a gem of a new
capability.
Briefly, the story begins with a book I’m currently writing
with Matt Stephens called Design
Driven Testing. The DDT book is about driving testing from
use cases at multiple levels. Some of
those levels (for example unit testing with JUnit and NUnit) have already been
supported in the Agile/ICONIX
add-in
that we’ve worked with Sparx to develop over several years.
Figure 1: Design Driven
Testing covers both Developer Testing and QA/Acceptance Testing
Since we wanted to write a chapter on scenario-based
acceptance testing, we asked the folks at Sparx (who have a relentless drive to
improve their product, the likes of which we’ve seldom seen) if they’d be
interested in implementing a “thread expander”.
During this time, one of our training clients, a project
called the Large
Synoptic Survey Telescope, was in communication with us regarding the
thread expansion topic, as their Project Manager for Data Management (Jeff
Kantor)
happens to have worked for ICONIX in a past lifetime (in the mid 1990s) and was
in fact responsible for implementing the first incarnation of the “thread
expander” way back in the stone ages. To
make a long story short, I asked Jeff to produce a specification for the thread
generator, sent it over to Sparx, and the result is now available within the
structured scenario editor in EA 8.
The Sparx structured scenario capability includes a lot more
than the “thread expansion” capability discussed in this article,
but test generation is certainly one of it’s key aspects.
So, what’s a use
case thread expander, and why do I need one?
Let’s return to our hypothetical use case, with one “sunny
day” and four “rainy day” scenarios.
Except now let’s make it a Login use case, with the sunny day path
resulting in the user logged in successfully, and rainy day paths for the user
clicking cancel, invalid passwords, incorrectly entered account names, and the
system being unable to start a session.
For acceptance testing purposes, we’ll want to expand this
use case into five “threads”, as shown in Figure 2. Each of these threads defines a script that
can be followed by the Quality Assurance team during independent QA testing of
a system. Simply put, when we’ve tested all the threads for all
the use cases of a system, then we know that our system meets the “behavior
requirements” that have been agreed to by all parties during the
analysis/design stage of a software release.
As you can see, even a simple Login use case can have quite
a bit of behavior associated with it. In
many organizations these scenarios will be tested by a QA organization, but
there’s nothing to stop developers from scenario-testing their own code if they
want to. Regardless of who does the
testing, it’s important that all the threads get tested.
Figure 2: Acceptance tests need to verify all
“sunny-day/rainy-day threads” of a use case
Four steps from
use case to scenario tests
We’re writing a detailed description of the steps from use
cases to scenario tests in the Scenario Testing chapter of Design Driven
Testing, and in the Structured Scenarios eBook, but for this article we’ve
boiled it down to four main steps:
1)
Start with a
narrative use case
2)
Convert the
use case to a structured scenario, add detail for testing
3)
Check your
structured scenario carefully
4)
Generate
scenario tests, and show them on a test case diagram
We’ll discuss each of these in turn, and illustrate with our
Login use case, in the remainder of this article.
Start with a
narrative use case
If you’ve ever used software that’s cumbersome, difficult to
use, or doesn’t seem to work quite right (and all of us have), you’re almost
certainly using software that didn’t start out with somebody writing a good
narrative “user manual style” use case.
The easiest way to think about writing narrative style use
cases is simply “write the user manual
before you write the code”. Writing
the user manual (in use case form) forces developers to think through the user
experience in detail before the code gets written. Simply put, this is important because once
the code is written, it’s usually too late.
There’s plenty of detail on how to write good “ICONIX style”
narrative use cases in Use Case
Driven Object Modeling – Theory and Practice, but for now we’ll just assume
we have one, and present it here in Figure 3.
You’ll notice that we wrote our original narrative use case in the
“General” tab of the use case specification dialog.
Figure 3 – A narrative
“ICONIX style” use case for Login
Convert the Use
Case to a Structured Scenario, add detail for testing
We’ll skip over the ICONIX Process analysis, design, and
coding activities for our
Login use case for the purposes of this article, and get directly to the point
where we’d like to define scenario tests so that our QA department can
independently verify that the developers have implemented Login correctly. To do this we’ll convert our narrative use
case to a structured scenario. Without
going into all the details, the result should look like Figure 4 (note that
we’re now using the Scenarios tab of the use case spec).
Figure 4 – Structured
Scenario for Login
You’ll notice several differences between the structured
scenario and the narrative version of the use case. These include:
- Automatically
generated step numbers (that re-number automatically as needed)
- Steps
explicitly designated as User or System steps
- Alternate
Paths explicitly linked to specific numbered steps in the Basic Path
- Join
information specifying where each Alternate Path rejoins the Basic Path
- Contextual
linkages to domain objects and screens
This additional information, along with the use case’s
Preconditions and Postconditions, is useful and necessary when preparing to
generate scenario tests. However, if you
try to specify all of it immediately up-front without writing the narrative use
case first, you might get bogged down during analysis and design. So you can think of the process of converting
from a narrative use case to a structured scenario as “preparing the use case
for test generation”.
Check your
structured scenario carefully
It’s possible to make mistakes when converting from a
narrative use case to a structured scenario (and sometimes you discover errors
in the narrative use case).
Two common mistakes are:
- Alternate
paths with no step text
- Incorrect
“join” information
Enterprise Architect makes it easy for you to detect these
errors by automatically generating an Activity Diagram that shows the path
logic, with branching.
Figure 5 shows an activity diagram we’ve generated for
Login. Our experience has shown that
checking your structured scenario in this manner leads to better results in
test case generation, as the test scenarios won’t generate correctly if the
path logic is wrong.
Figure 5 – Generating an
Activity Diagram verifies the logic of your structured scenario
Generate Scenario
Tests and show them on a Test Case Diagram
Once you’ve checked your logic, it’s a simple matter to
generate “thread tests” for your structured scenario (although it’s not always
simple to find the tests you’ve generated).
We used the Generate External Tests option, and created the following
diagram showing a note with the reconstructed “whole use case view”, and the
test case in “rectangle notation”. The
resulting diagram is shown in Figure 6.
/p>
Figure 6 – A test case
diagram makes it easy to find your scenario tests.
Once you’ve created the test case and placed it on the test
case diagram, you can simply double-click on any of the scenarios within it to
bring up EA’s testing view, as shown in Figure 7. From here you can add additional detail
that’s useful for generating test plans for your QA department.
Figure 7 – Drilling into the
EA Testing view from your test case diagram
Summary
This article has provided a high level overview of
acceptance test case generation from use cases, using the “thread expansion”
capability that exists within the “structured scenario editor” in version 8 of
Enterprise Architect. We’ve skipped over
a lot of details in this article, partly because the software is still in
betatest at the time of writing, and partly because there are just too many
details to include in a short article.
Additional details are forthcoming in both the Design Driven Testing
book and the Structured Scenarios eBook that are now under development.
The author welcomes feedback and further inquiries via email
at umltraining@iconixsw.com.
