Many testers entering the field get nervous when they consider the need to test at different levels of an application, particularly if that involves working with developers. Many testers will enter into environments that claim they are practicing Test-Driven Development (TDD) and/or Behavior-Driven Development (BDD). What I want to do here is show that this stuff isn’t really all that scary and it’s really not that arcane. Further, if you want to be competitive as a tester and continue to add value, it’s in your best interests to learn what it means to test at different levels.
Can I Function Test My Units?
Many testers will tell you that they eventually had to work with their development group to draw the line between unit testing and functional testing. This, of course, implies that there is some sort of a difference between the two. And, in fact, if you go by the literature — professional or otherwise — of the testing industry, there supposedly is a difference.
I’ve just never liked that fact because “functional” is such a broad term. To my way of thinking, unit testing is a form of functional testing. I think testers who use the term “functional” as a separating category within tests are stuck in an “old school” mode of thinking.
An Extended Parameters Mechanism for Procedures in QTP
A problem with VBScript is that it doesn’t allow you to have optional parameters in your procedures. This also means you can’t do function overloading. There are various ways to deal with this but many of them can lead to a maintenance problem and require you to change how your new (“extended”) procedures are called. Ideally, you don’t want to have to change how the function is called each time you extend it. So let’s look at a few ideas here.
Continue reading An Extended Parameters Mechanism for Procedures in QTP
Is Quality Really Everyone’s Responsibility?
Who is “responsible” for quality? This is one of those interesting questions that on the surface of it seems really simple to answer depending upon your viewpoint … and yet the question itself hides a few thorny issues. In my experience, most people who claim they are part of a “Quality Assurance Team” are not aware of these issues and thus perpetuate a lot of false thinking regarding the question itself.
It’s very important to think about what this question actually implies. How this question is considered directly ties in with how we think about the notion of responsibility and about how we communicate that notion to others.
Continue reading Is Quality Really Everyone’s Responsibility?
What Makes Testing Complicated?
I think the question of what makes testing a complicated discipline is a good one. It’s one all testers should be able to answer about their profession and, further, answer in a way where the answer couldn’t be applied to just about any other profession. The problem I’ve found is the answers you get are usually simplistic. You’ll hear people say “miscommunication” or “unrealistic schedules.” Well, that’s great but miscommunication can make any profession difficult as can an unrealistic schedule or poor estimating. I bet day traders, doctors, police officers, lawyers, military strategists, nuclear power plant operators and so forth could make the same claim.
Surely we can do better than that, right?
Automated Testing with Watir and RSpec, Part 2
If you went through the first part of my experiential case study, you’ll know that I ended up creating a few tests that used Watir-WebDriver to execute logic against a web site. While I managed to get some output from those tests, it wasn’t the best looking output in terms of readability. Partly that was design, of course. After all, I could have made better output simply by considering how my tests should return success or failure information. Ideally, though, I would create some sort of modularized reporting functionality as part of my framework. This module would do nothing but handle the outputting of test results. That’s certainly a worthwhile thing to focus on, but you might also consider other solutions. As I mentioned at the close of the previous post, one such framework has already been written to handle this, which is RSpec.
Continue reading Automated Testing with Watir and RSpec, Part 2
Automated Testing with Watir and RSpec, Part 1
Sometimes it can be hard to get started with open source automated testing. The documentation is usually on the bad side of horrible. Further, many of the tools were written by developers with a development focus in mind, which is different from those tools developed by testers. Even when you do find technical people that are able to describe how to use something well, the context for using the tool may not be what you are looking for. I certainly found this when I started out wanting to use Watir, WebDriver, and RSpec. You can spend a lot of time looking through various Wiki pages or blog posts to figure out the bits of information you need to eventually have a fully working solution.
And that can be really annoying. And yet here I am apparently offering yet another blog post covering the stuff. What I’m doing here is presenting the information I wish I had when I started out. I provide a little historical detail, a lot of implementation detail, and some reference information behind the implementation.
Continue reading Automated Testing with Watir and RSpec, Part 1
The Value of Testing … Relative to Quality
The perceived value of testing is often related to how test teams believe — and promote — the idea that they “test for quality.” The reason I say that is because I maintain quality is value to someone whose opinion matters. You might want to check out my post on testing for quality where I covered a lot of these points. What I didn’t cover in that post is the value of testing itself. The value of testing can be trickier than you may think. Let’s start with this: people will place different value to the quality they perceive. Do we agree on that? If so, I argue that the value of testing that people perceive will be in direct proportion to how much they believe your testing tells them what they need to know. Different people will need to know different things given that their valuations regarding quality will be different in some cases.
What you end up with is dynamic assessment of value and it’s here that testers can start to think a bit more deeply about how what they do provides value.
“Testing for Quality” and Making Informed Decisions
Sometimes you’ll hear testers talk about how they “test for quality.” My problem with that phrase is that it’s a potentially dangerous one. It’s an even more dangerous expectation unless some ideas of what the phrase actually means are sorted out. Once you have an idea of what “test for quality” actually means, maybe you can craft intelligent processes around the phrase.
If you’re going to do that, make sure you know why you test and make sure you have some idea of what quality means. Without some thinking around those two areas, “testing for quality” is nothing more than something you say to make managers feel good.
Continue reading “Testing for Quality” and Making Informed Decisions
Testers Are Not Either-Or
F. Scott Fitzgerald, in his essay called The Crack-up from 1936, defined a “first rate intelligence” as the ability to hold “two opposed ideas in the mind at the same time and still retain the ability to function.” David Lapin, one time CEO of Strategic Business Ethics, and now CEO of Lapin International, referred to the “space of paradox” wherein trust and innovation had to co-exist. Donald Schön talks about the “generative metaphor” in his 1963 The Displacement of Concepts, which was the idea of bringing domains that appear to be different together in a single concept or design. Dorothy Leonard-Barton talked about “creative abrasion” in her 1995 The Well-Springs of Knowledge which was focused on bringing people with differing views together.
James Collins and Jerry Porras encapsulated a lot of these ideas in a style of thinking called “the genius of the and” in their 1997 Built to Last: Successful Habits of Visionary Companies. It’s that book I’ll focus on here because I think the concept of the “genius of the and” is one that people can readily rally around. Built to Last is not a book that talks about quality assurance or testing at all, but it does talk about a very common thinking pattern in both of those fields.
Deviations Magnify – And That Can Be a Good Thing!
As I write this, NASA’s UARS (Upper Atmosphere Research Satellite) is about to plop back to Earth. The orbit of the satellite puts it somewhere between 57 degrees north latitude and 57 degrees south. That spans the width of the world between northern Canada and the tip of South America. Where exactly it will come down is hard to say because the deviations that are possible, due to a variety of factors, make such predictions potentially wildly inaccurate.
I think the degree to which deviations can magnify like this is interesting and may even have some correlation in what we deal with in software projects.
Continue reading Deviations Magnify – And That Can Be a Good Thing!
How Big Is Your Testing Umbrella?
I’ve worked with lots of testers who often take on more work than they should because they have the idea that the work they are taking on is what testers should be doing. It’s often not a question of whether they should do this work given their other constraints, but rather just a focus on how they can regardless of those constraints. Granted, many companies seem to foster this by job descriptions that suggest someone needs to be a developer, DBA, business analyst, and tester all in one. A main aspect of my job description as a quality assurance and test practitioner comes down to defining a role, usually separate from the relatively anodyne job descriptions we’re all used to.
Testing By Any Other Name…
In my post Are We Defocusing Our Development and Test Practices? I was talking about the various “driven-development” terms. I had brought up how Behavior Driven Development (BDD) seems to have largely come about because some people felt that the term “test” was a bit too confusing for some people or, at the very least, off putting for some reasons that are not explained very well. There was thus apparently a shift to “examples” that talked about “behavior” and this was somehow different than a test. And this somehow made everything a bit better. But did it? What was it about the terms “test” and “testing” that were just not working? I’d rather people deal with that problem before thinking we need to come up with new acronyms. I say that because, ultimately, the concepts of a “test” and the activity of “testing” is still very much there.
Are We Defocusing Our Development and Test Practices?
As a tester, you’ll hear about behavior-driven development (BDD). Sometimes this goes by acceptance test-driven development (ATDD) or maybe even acceptance test-driven planning (ATDP). You may hear that these are related to test-driven development (TDD). After a while you might start to wonder if these are just different ways of saying the exact same thing.
Continue reading Are We Defocusing Our Development and Test Practices?
Show Some Class When Using QTP
A lot of testers use QTP as a record-and-playback tool first and then add some logic to start building a framework around whatever their test scripts are. That’s an approach that can work but those same testers often don’t utilize one of the key strengths of QTP, which is it’s object-oriented structure.
Rather than treat it as obvious that using classes is a good idea and without giving a tutorial to object-oriented programming here, I plan to showcase some specifics to keep in mind when using a class-based approach in QTP as well as give a specific example of how this approach can be useful by showing how you can encapsulate your logic for better maintainability.
Winning Battles, Losing Wars
One thing I’ve found that’s helpful to me in my career as a tester is the ability to abstract information from one venue to another. Testers and especially those who do Quality Assurance activities must be reasonably adept at abstracting knowledge from one domain and applying it in another, all the while accounting for those parts that don’t apply in the new domain. There are many ways to look at this but here I want to consider problem solving as a technique. I’m going to look at the technique of problem solving in relation to the domain of combat. My goal is to show how you can translate aspects of the combat domain into your more traditional day-to-day scenarios.
What Can Time Travel Teach Us About Testing?
I have long maintained that testing is ultimately about thinking. A lot of your work is going to be predicated upon spotting incongruous elements (whether in documents or application code), resolving potential ambiguities in what you read or see, and dealing with potential inconsistencies or outright contradictions. Sometimes people will frame problems in such a way that your ability to “solve” or deal with the problem effectively is compromised from the start.
A good way to practice your thinking skills is to apply them in an entirely different arena than software testing.
Continue reading What Can Time Travel Teach Us About Testing?
Acceptance Tests Are Just Tests
Some testers have problems conceptualizing acceptance tests as I described them in my post on writing tests from stories. The problem seems to be one of seeing how the concept of an acceptance test applies or translates to concepts they already know, such as test conditions and/or test cases.
Be Acceptable: Write Tests from Stories
In my post on Stories, Features, and Acceptance Tests I talked about a lot but I didn’t really cover the acceptance tests themselves. The tests are quite important. Keep in mind that the goal is to have specifications that are clear, sufficiently testable and falsifiable, and in line with client expectations prior to significant development or testing. In essence, the idea is to work out how a system would be tested as a way to check whether the requirements give you enough information to build the system in the first place. You shouldn’t dive straight away into how to implement something, but rather think about how the finished system will be used — i.e., acceptable use — and then double-check the requirements armed with that knowledge.
Acceptance tests should reflect the customers’ perception of when the application meets their requirements. This does not mean that such tests must be defined by a customer or business analyst but it does often mean that such tests are discussed with them.
But how do you write them? What do they look like? Well, let’s talk about that.
Stories, Features, Acceptance Tests: Which Am I Writing?
If you’re a tester that works in an environment where your teams practice test-driven development (TDD), behavior-driven development (BDD), or some other variation that sounds really similar, you’ll quickly be learning that people toss around terms like “story”, “feature”, “acceptance criteria”, and “scenario.” What they often won’t toss around is the context for those terms. That’s context is important to have because it depends entirely on how the terms are used in your environment. While there are not necessarily standards, there are lots of good opinions that have led to good practices in context. Liz Keogh’s Acceptance Criteria vs Scenarios and Matt Wynne’s Features != User Stories are two good examples.
As a tester, what do you have to know? How involved are you in these things? Is all this just for developers? If you’re writing stories, are you also writing tests? Is acceptance criteria the same thing as an acceptance test? Here’s my take on a few of these questions.
Continue reading Stories, Features, Acceptance Tests: Which Am I Writing?