Published inQualityTesting

“Testing for Quality” and Making Informed Decisions

written by

Sometimes you’ll hear testers talk about how they “test for quality.” My problem with that phrase is that it’s a potentially dangerous one. It’s an even more dangerous expectation unless some ideas of what the phrase actually means are sorted out. Once you have an idea of what “test for quality” actually means, maybe you can craft intelligent processes around the phrase.

If you’re going to do that, make sure you know why you test and make sure you have some idea of what quality means. Without some thinking around those two areas, “testing for quality” is nothing more than something you say to make managers feel good.

Continue reading “Testing for Quality” and Making Informed Decisions →

Published inThinking

Testers Are Not Either-Or

written by

F. Scott Fitzgerald, in his essay called The Crack-up from 1936, defined a “first rate intelligence” as the ability to hold “two opposed ideas in the mind at the same time and still retain the ability to function.” David Lapin, one time CEO of Strategic Business Ethics, and now CEO of Lapin International, referred to the “space of paradox” wherein trust and innovation had to co-exist. Donald Schön talks about the “generative metaphor” in his 1963 The Displacement of Concepts, which was the idea of bringing domains that appear to be different together in a single concept or design. Dorothy Leonard-Barton talked about “creative abrasion” in her 1995 The Well-Springs of Knowledge which was focused on bringing people with differing views together.

James Collins and Jerry Porras encapsulated a lot of these ideas in a style of thinking called “the genius of the and” in their 1997 Built to Last: Successful Habits of Visionary Companies. It’s that book I’ll focus on here because I think the concept of the “genius of the and” is one that people can readily rally around. Built to Last is not a book that talks about quality assurance or testing at all, but it does talk about a very common thinking pattern in both of those fields.

Continue reading Testers Are Not Either-Or →

Published inThinking

Deviations Magnify – And That Can Be a Good Thing!

written by

As I write this, NASA’s UARS (Upper Atmosphere Research Satellite) is about to plop back to Earth. The orbit of the satellite puts it somewhere between 57 degrees north latitude and 57 degrees south. That spans the width of the world between northern Canada and the tip of South America. Where exactly it will come down is hard to say because the deviations that are possible, due to a variety of factors, make such predictions potentially wildly inaccurate.

I think the degree to which deviations can magnify like this is interesting and may even have some correlation in what we deal with in software projects.

Continue reading Deviations Magnify – And That Can Be a Good Thing! →

Published inQualityTesting

How Big Is Your Testing Umbrella?

written by

I’ve worked with lots of testers who often take on more work than they should because they have the idea that the work they are taking on is what testers should be doing. It’s often not a question of whether they should do this work given their other constraints, but rather just a focus on how they can regardless of those constraints. Granted, many companies seem to foster this by job descriptions that suggest someone needs to be a developer, DBA, business analyst, and tester all in one. A main aspect of my job description as a quality assurance and test practitioner comes down to defining a role, usually separate from the relatively anodyne job descriptions we’re all used to.

Continue reading How Big Is Your Testing Umbrella? →

Published inBDDTesting

Testing By Any Other Name…

written by

In my post Are We Defocusing Our Development and Test Practices? I was talking about the various “driven-development” terms. I had brought up how Behavior Driven Development (BDD) seems to have largely come about because some people felt that the term “test” was a bit too confusing for some people or, at the very least, off putting for some reasons that are not explained very well. There was thus apparently a shift to “examples” that talked about “behavior” and this was somehow different than a test. And this somehow made everything a bit better. But did it? What was it about the terms “test” and “testing” that were just not working? I’d rather people deal with that problem before thinking we need to come up with new acronyms. I say that because, ultimately, the concepts of a “test” and the activity of “testing” is still very much there.

Continue reading Testing By Any Other Name… →

Published inBDDTesting

Are We Defocusing Our Development and Test Practices?

written by

As a tester, you’ll hear about behavior-driven development (BDD). Sometimes this goes by acceptance test-driven development (ATDD) or maybe even acceptance test-driven planning (ATDP). You may hear that these are related to test-driven development (TDD). After a while you might start to wonder if these are just different ways of saying the exact same thing.

Continue reading Are We Defocusing Our Development and Test Practices? →

Published inAutomationQTP

Show Some Class When Using QTP

written by

A lot of testers use QTP as a record-and-playback tool first and then add some logic to start building a framework around whatever their test scripts are. That’s an approach that can work but those same testers often don’t utilize one of the key strengths of QTP, which is it’s object-oriented structure.

Rather than treat it as obvious that using classes is a good idea and without giving a tutorial to object-oriented programming here, I plan to showcase some specifics to keep in mind when using a class-based approach in QTP as well as give a specific example of how this approach can be useful by showing how you can encapsulate your logic for better maintainability.

Continue reading Show Some Class When Using QTP →

Published inThinking

Winning Battles, Losing Wars

written by

One thing I’ve found that’s helpful to me in my career as a tester is the ability to abstract information from one venue to another. Testers and especially those who do Quality Assurance activities must be reasonably adept at abstracting knowledge from one domain and applying it in another, all the while accounting for those parts that don’t apply in the new domain. There are many ways to look at this but here I want to consider problem solving as a technique. I’m going to look at the technique of problem solving in relation to the domain of combat. My goal is to show how you can translate aspects of the combat domain into your more traditional day-to-day scenarios.

Continue reading Winning Battles, Losing Wars →

Published inTestingThinking

What Can Time Travel Teach Us About Testing?

written by

I have long maintained that testing is ultimately about thinking. A lot of your work is going to be predicated upon spotting incongruous elements (whether in documents or application code), resolving potential ambiguities in what you read or see, and dealing with potential inconsistencies or outright contradictions. Sometimes people will frame problems in such a way that your ability to “solve” or deal with the problem effectively is compromised from the start.

A good way to practice your thinking skills is to apply them in an entirely different arena than software testing.

Continue reading What Can Time Travel Teach Us About Testing? →

Published inTesting

Acceptance Tests Are Just Tests

written by

Some testers have problems conceptualizing acceptance tests as I described them in my post on writing tests from stories. The problem seems to be one of seeing how the concept of an acceptance test applies or translates to concepts they already know, such as test conditions and/or test cases.

Continue reading Acceptance Tests Are Just Tests →

Published inQualityTesting

Be Acceptable: Write Tests from Stories

written by

In my post on Stories, Features, and Acceptance Tests I talked about a lot but I didn’t really cover the acceptance tests themselves. The tests are quite important. Keep in mind that the goal is to have specifications that are clear, sufficiently testable and falsifiable, and in line with client expectations prior to significant development or testing. In essence, the idea is to work out how a system would be tested as a way to check whether the requirements give you enough information to build the system in the first place. You shouldn’t dive straight away into how to implement something, but rather think about how the finished system will be used — i.e., acceptable use — and then double-check the requirements armed with that knowledge.

Acceptance tests should reflect the customers’ perception of when the application meets their requirements. This does not mean that such tests must be defined by a customer or business analyst but it does often mean that such tests are discussed with them.

But how do you write them? What do they look like? Well, let’s talk about that.

Continue reading Be Acceptable: Write Tests from Stories →

Published inQualityTesting

Stories, Features, Acceptance Tests: Which Am I Writing?

written by

If you’re a tester that works in an environment where your teams practice test-driven development (TDD), behavior-driven development (BDD), or some other variation that sounds really similar, you’ll quickly be learning that people toss around terms like “story”, “feature”, “acceptance criteria”, and “scenario.” What they often won’t toss around is the context for those terms. That’s context is important to have because it depends entirely on how the terms are used in your environment. While there are not necessarily standards, there are lots of good opinions that have led to good practices in context. Liz Keogh’s Acceptance Criteria vs Scenarios and Matt Wynne’s Features != User Stories are two good examples.

As a tester, what do you have to know? How involved are you in these things? Is all this just for developers? If you’re writing stories, are you also writing tests? Is acceptance criteria the same thing as an acceptance test? Here’s my take on a few of these questions.

Continue reading Stories, Features, Acceptance Tests: Which Am I Writing? →

Published inAutomationQTP

Modularizing Descriptive Programming in QTP

written by

In a previous post, I talked about how I prefer the choice of descriptive programming in terms of my QTP logic for recognizing objects. Descriptive Programming (DP) is a specific approach that QTP makes possible for constructing your recognition strings programmatically. Anyone who has had to modify SilkTest frame files or create XPath-based locators for Selenium will quickly realize that “descriptive programming” is really just a name that QTP has given to what many tools already make possible. In fact, it’s the Object Repository (OR) — the alternative approach in QTP — that tends to be a value-add of QTP beyond other tools, at least for some people.

In any event, my previous post was a little skimpy on specific details so here I’ll look at a specific example of modularizing some test logic using the descriptive approach in QTP.

Continue reading Modularizing Descriptive Programming in QTP →

Published inAutomationQTP

Should I Ditch the Repository and Be Descriptive in QTP?

written by

QTP allows you to use what the tool calls Descriptive Programming (DP) as a way to write your automated test logic. This is really nothing more than a way of referencing objects, similar to the recognition method path (object strings) in Rational Robot, the GUI map strings of WinRunner, the tags and locators in SilkTest, or the selectors in Selenium. This style of object recognition is often put in opposition to another form that QTP allows you to do, which is store strings in the Object Repository (OR).

So which one should you use: DP or OR? Does it really make a difference? Are there advantages to one over the other? Can you use both? Should you? I’ll talk about all this a little here.

Continue reading Should I Ditch the Repository and Be Descriptive in QTP? →

Published inAutomationCapybaraCucumberSelenium

Testing with Cucumber, Capybara and Selenium

written by

In a prior post, I talked about using Capybara and Selenium as just a few among many tools. In a related post on using RSpec and Capybara, I brought up the possibility that “the natural language parts are, in fact, the executable code.” This was in reference to the idea of code logic expressed as natural language in RSpec but that still required actual code that sounded almost like the natural language statements. Wouldn’t it be nice if you didn’t have to write it twice? You just write it one way and then it executes? The argument then might be to just write the code, and assume the intent of the test can be explained via comments or something else. But many people really wanted to keep the focus on natural language tests.

Continue reading Testing with Cucumber, Capybara and Selenium →

Published inAutomationCapybaraRSpec

Using RSpec and Capybara Without Rails

written by

Building off of my post regarding spiking automated testing solutions, one thing I didn’t really do was show you any sort of effective coding practice there at all. In that post I was showing how to get a series of technologies working together for that old “sense of accomplishment” thing. What I want to do here is focus on using two of those technologies together — specifically, Capybara and RSpec — while also showing you how you can start to separate out bits of logic as you start thinking about how to construct automated test frameworks for testing browser-based applications.

Continue reading Using RSpec and Capybara Without Rails →

Published inAutomationCapybaraRSpecSeleniumWatir

Spiking With Open Source Testing Tools

written by

If you are a tester that’s charged with automating the execution of a web application, there’s a fairly good chance that you won’t be using some of the cost solutions out there like QTP or SilkTest. There’s a better than even chance that you’ll be looking at open source testing solutions. If this is your first experience with such technologies, you might find it a bit daunting. Oftentimes the one thing you won’t find is a nice concise script that will at least give you a start. Complicating this is many blogs seem to indicate various tools all interconnect in some ways, or can only be used in certain contexts, leaving you to figure out a lot of the details yourself.

For me, when I look at these tools I need to know, as quickly as possible, if (1) they work at all and (2) if I can wrap my head around how to get started in the first place. Sometimes you just need a gentle nudge in the right direction to see how the technology works. When I practice with technologies, I create little “spike” files that do just enough to show me what I need to know. Here I’ll be showing you my spikes.

Continue reading Spiking With Open Source Testing Tools →

Published inTesting

Why Do We Test? To Communicate, Of Course

written by

“Perfection of means and confusion of ends seem to characterize our age.” So said Albert Einstein. I think that sentiment can be applied to tests and their role as a communication mechanism. When you get involved in projects that are focusing on “agile” approaches like behavior-driven development (BDD), I’ve found that you’ll want to make sure you keep focusing on how testing will have a focus on intent (what) as opposed to implementation (how). It’s easy for testers to fall into a group think mentality with developers, who are often pushing an agile focus with an emphasis on what amounts to quality of implementation rather than quality of intent. This means testers have to have a good answer to the question, “Why am I testing?”

Continue reading Why Do We Test? To Communicate, Of Course →