Author: Jeff Nyman

Anything I put here is an approximation of the truth. You're getting a particular view of myself ... and it's the view I'm choosing to present to you. If you've never met me before in person, please realize I'm not the same in person as I am in writing. That's because I can only put part of myself down into words. If you have met me before in person then I'd ask you to consider that the view you've formed that way and the view you come to by reading what I say here may, in fact, both be true. I'd advise that you not automatically discard either viewpoint when they conflict or accept either as truth when they agree.
Published inTesting

Does Planning For Testing Require a Test Plan?

written by

I previously talked about test strategy and I made it clear that in my opinion the test strategy and the test plan are two entirely different things. I’ve had a love-hate affair with test plans over the course of time, mostly because I’ve seen them become an end unto themselves. I’ve seen them become monolithic documents that people who need a sense of control use when they feel they can’t get that control in more effective ways or can’t deal with projects that are more fluid in nature.

But I wonder if I’m too reactionary in nature on this topic or if I’m just reacting to seeing a lot of badly written, rarely read test plans.

Continue reading Does Planning For Testing Require a Test Plan? →

Published inTesting

Your Test Strategy Is A Framework

written by

I see a lot of testers — and their managers — misuse the phrase “test strategy.” I say they misuse it not because they don’t agree with some definition I have but rather because I rarely see a good distinction made between test logistics and test strategy. I often see a test strategy equated with a test plan. I hear some testers say that having a test plan is part of your test strategy. I hear others say that it’s the job of the test plan to indicate the test strategy.

So let’s talk about test strategy a bit.

Continue reading Your Test Strategy Is A Framework →

Published inTesting

Can I Stress The Load On My Volume?

written by

In my post “Can I Function Test My Units?”, I talked about how testers often promoted a misuse of the phrase “functional testing.” I’ve found the same thing happens with the phrase “performance testing.” What I’ve found is that many testers don’t realize that performance testing, just like functional testing, is a test approach. This means there are various testing techniques that you can apply to the approach. If testers don’t have this kind of focus, I believe that the purpose of performance testing can be lost.

Continue reading Can I Stress The Load On My Volume? →

Published inQuality Assurance

Quality Assurance – Obviously

written by

So it’s obvious we need quality assurance, right? Okay, it’s true that quality is a dynamic concept that is contextual and situational. And, in a way, you can never really know the “actual” quality of an application in some universal sense. This is partly because quality is a value judgment based on the eye of the beholder. And it’s probably true that quality is not open to some absolute definition. (Similar to how other abstract words like justice, beauty, democracy, and so on are not absolute concepts.) But still. We obviously need some way to assure this beast we call quality. We obviously need a team called Quality Assurance.

I mean, it’s obvious. Err … right?

Continue reading Quality Assurance – Obviously →

Published inTesting

The Art of Testing at Different Levels

written by

Many testers entering the field get nervous when they consider the need to test at different levels of an application, particularly if that involves working with developers. Many testers will enter into environments that claim they are practicing Test-Driven Development (TDD) and/or Behavior-Driven Development (BDD). What I want to do here is show that this stuff isn’t really all that scary and it’s really not that arcane. Further, if you want to be competitive as a tester and continue to add value, it’s in your best interests to learn what it means to test at different levels.

Continue reading The Art of Testing at Different Levels →

Published inTesting

Can I Function Test My Units?

written by

Many testers will tell you that they eventually had to work with their development group to draw the line between unit testing and functional testing. This, of course, implies that there is some sort of a difference between the two. And, in fact, if you go by the literature — professional or otherwise — of the testing industry, there supposedly is a difference.

I’ve just never liked that fact because “functional” is such a broad term. To my way of thinking, unit testing is a form of functional testing. I think testers who use the term “functional” as a separating category within tests are stuck in an “old school” mode of thinking.

Continue reading Can I Function Test My Units? →

Published inAutomationQTP

An Extended Parameters Mechanism for Procedures in QTP

written by

A problem with VBScript is that it doesn’t allow you to have optional parameters in your procedures. This also means you can’t do function overloading. There are various ways to deal with this but many of them can lead to a maintenance problem and require you to change how your new (“extended”) procedures are called. Ideally, you don’t want to have to change how the function is called each time you extend it. So let’s look at a few ideas here.

Continue reading An Extended Parameters Mechanism for Procedures in QTP →

Published inQuality AssuranceTesting

Is Quality Really Everyone’s Responsibility?

written by

Who is “responsible” for quality? This is one of those interesting questions that on the surface of it seems really simple to answer depending upon your viewpoint … and yet the question itself hides a few thorny issues. In my experience, most people who claim they are part of a “Quality Assurance Team” are not aware of these issues and thus perpetuate a lot of false thinking regarding the question itself.

It’s very important to think about what this question actually implies. How this question is considered directly ties in with how we think about the notion of responsibility and about how we communicate that notion to others.

Continue reading Is Quality Really Everyone’s Responsibility? →

Published inTesting

What Makes Testing Complicated?

written by

I think the question of what makes testing a complicated discipline is a good one. It’s one all testers should be able to answer about their profession and, further, answer in a way where the answer couldn’t be applied to just about any other profession. The problem I’ve found is the answers you get are usually simplistic. You’ll hear people say “miscommunication” or “unrealistic schedules.” Well, that’s great but miscommunication can make any profession difficult as can an unrealistic schedule or poor estimating. I bet day traders, doctors, police officers, lawyers, military strategists, nuclear power plant operators and so forth could make the same claim.

Surely we can do better than that, right?

Continue reading What Makes Testing Complicated? →

Published inAutomationRSpecWatir

Automated Testing with Watir and RSpec, Part 2

written by

If you went through the first part of my experiential case study, you’ll know that I ended up creating a few tests that used Watir-WebDriver to execute logic against a web site. While I managed to get some output from those tests, it wasn’t the best looking output in terms of readability. Partly that was design, of course. After all, I could have made better output simply by considering how my tests should return success or failure information. Ideally, though, I would create some sort of modularized reporting functionality as part of my framework. This module would do nothing but handle the outputting of test results. That’s certainly a worthwhile thing to focus on, but you might also consider other solutions. As I mentioned at the close of the previous post, one such framework has already been written to handle this, which is RSpec.

Continue reading Automated Testing with Watir and RSpec, Part 2 →

Published inAutomationRSpecWatir

Automated Testing with Watir and RSpec, Part 1

written by

Sometimes it can be hard to get started with open source automated testing. The documentation is usually on the bad side of horrible. Further, many of the tools were written by developers with a development focus in mind, which is different from those tools developed by testers. Even when you do find technical people that are able to describe how to use something well, the context for using the tool may not be what you are looking for. I certainly found this when I started out wanting to use Watir, WebDriver, and RSpec. You can spend a lot of time looking through various Wiki pages or blog posts to figure out the bits of information you need to eventually have a fully working solution.

And that can be really annoying. And yet here I am apparently offering yet another blog post covering the stuff. What I’m doing here is presenting the information I wish I had when I started out. I provide a little historical detail, a lot of implementation detail, and some reference information behind the implementation.

Continue reading Automated Testing with Watir and RSpec, Part 1 →

Published inQualityTesting

The Value of Testing … Relative to Quality

written by

The perceived value of testing is often related to how test teams believe — and promote — the idea that they “test for quality.” The reason I say that is because I maintain quality is value to someone whose opinion matters. You might want to check out my post on testing for quality where I covered a lot of these points. What I didn’t cover in that post is the value of testing itself. The value of testing can be trickier than you may think. Let’s start with this: people will place different value to the quality they perceive. Do we agree on that? If so, I argue that the value of testing that people perceive will be in direct proportion to how much they believe your testing tells them what they need to know. Different people will need to know different things given that their valuations regarding quality will be different in some cases.

What you end up with is dynamic assessment of value and it’s here that testers can start to think a bit more deeply about how what they do provides value.

Continue reading The Value of Testing … Relative to Quality →

Published inQualityTesting

“Testing for Quality” and Making Informed Decisions

written by

Sometimes you’ll hear testers talk about how they “test for quality.” My problem with that phrase is that it’s a potentially dangerous one. It’s an even more dangerous expectation unless some ideas of what the phrase actually means are sorted out. Once you have an idea of what “test for quality” actually means, maybe you can craft intelligent processes around the phrase.

If you’re going to do that, make sure you know why you test and make sure you have some idea of what quality means. Without some thinking around those two areas, “testing for quality” is nothing more than something you say to make managers feel good.

Continue reading “Testing for Quality” and Making Informed Decisions →

Published inThinking

Testers Are Not Either-Or

written by

F. Scott Fitzgerald, in his essay called The Crack-up from 1936, defined a “first rate intelligence” as the ability to hold “two opposed ideas in the mind at the same time and still retain the ability to function.” David Lapin, one time CEO of Strategic Business Ethics, and now CEO of Lapin International, referred to the “space of paradox” wherein trust and innovation had to co-exist. Donald Schön talks about the “generative metaphor” in his 1963 The Displacement of Concepts, which was the idea of bringing domains that appear to be different together in a single concept or design. Dorothy Leonard-Barton talked about “creative abrasion” in her 1995 The Well-Springs of Knowledge which was focused on bringing people with differing views together.

James Collins and Jerry Porras encapsulated a lot of these ideas in a style of thinking called “the genius of the and” in their 1997 Built to Last: Successful Habits of Visionary Companies. It’s that book I’ll focus on here because I think the concept of the “genius of the and” is one that people can readily rally around. Built to Last is not a book that talks about quality assurance or testing at all, but it does talk about a very common thinking pattern in both of those fields.

Continue reading Testers Are Not Either-Or →

Published inThinking

Deviations Magnify – And That Can Be a Good Thing!

written by

As I write this, NASA’s UARS (Upper Atmosphere Research Satellite) is about to plop back to Earth. The orbit of the satellite puts it somewhere between 57 degrees north latitude and 57 degrees south. That spans the width of the world between northern Canada and the tip of South America. Where exactly it will come down is hard to say because the deviations that are possible, due to a variety of factors, make such predictions potentially wildly inaccurate.

I think the degree to which deviations can magnify like this is interesting and may even have some correlation in what we deal with in software projects.

Continue reading Deviations Magnify – And That Can Be a Good Thing! →

Published inQualityTesting

How Big Is Your Testing Umbrella?

written by

I’ve worked with lots of testers who often take on more work than they should because they have the idea that the work they are taking on is what testers should be doing. It’s often not a question of whether they should do this work given their other constraints, but rather just a focus on how they can regardless of those constraints. Granted, many companies seem to foster this by job descriptions that suggest someone needs to be a developer, DBA, business analyst, and tester all in one. A main aspect of my job description as a quality assurance and test practitioner comes down to defining a role, usually separate from the relatively anodyne job descriptions we’re all used to.

Continue reading How Big Is Your Testing Umbrella? →

Published inBDDTesting

Testing By Any Other Name…

written by

In my post Are We Defocusing Our Development and Test Practices? I was talking about the various “driven-development” terms. I had brought up how Behavior Driven Development (BDD) seems to have largely come about because some people felt that the term “test” was a bit too confusing for some people or, at the very least, off putting for some reasons that are not explained very well. There was thus apparently a shift to “examples” that talked about “behavior” and this was somehow different than a test. And this somehow made everything a bit better. But did it? What was it about the terms “test” and “testing” that were just not working? I’d rather people deal with that problem before thinking we need to come up with new acronyms. I say that because, ultimately, the concepts of a “test” and the activity of “testing” is still very much there.

Continue reading Testing By Any Other Name… →

Published inBDDTesting

Are We Defocusing Our Development and Test Practices?

written by

As a tester, you’ll hear about behavior-driven development (BDD). Sometimes this goes by acceptance test-driven development (ATDD) or maybe even acceptance test-driven planning (ATDP). You may hear that these are related to test-driven development (TDD). After a while you might start to wonder if these are just different ways of saying the exact same thing.

Continue reading Are We Defocusing Our Development and Test Practices? →

Published inAutomationQTP

Show Some Class When Using QTP

written by

A lot of testers use QTP as a record-and-playback tool first and then add some logic to start building a framework around whatever their test scripts are. That’s an approach that can work but those same testers often don’t utilize one of the key strengths of QTP, which is it’s object-oriented structure.

Rather than treat it as obvious that using classes is a good idea and without giving a tutorial to object-oriented programming here, I plan to showcase some specifics to keep in mind when using a class-based approach in QTP as well as give a specific example of how this approach can be useful by showing how you can encapsulate your logic for better maintainability.

Continue reading Show Some Class When Using QTP →