Author: Jeff Nyman

Anything I put here is an approximation of the truth. You're getting a particular view of myself ... and it's the view I'm choosing to present to you. If you've never met me before in person, please realize I'm not the same in person as I am in writing. That's because I can only put part of myself down into words. If you have met me before in person then I'd ask you to consider that the view you've formed that way and the view you come to by reading what I say here may, in fact, both be true. I'd advise that you not automatically discard either viewpoint when they conflict or accept either as truth when they agree.
Published inThinking

Test Teams Need Inventors: Seeing and Thinking Differently

written by

In various posts I’ve tried to show how I believe testers can invent solutions to problems they are encountering. These solutions do not always have to be tool-based in nature. Sometimes you are presenting a new way of thinking about processes, sometimes you are reframing problems, sometimes you a providing a hopeful vision, sometimes you are in fact coming up with ad-hoc tools, and sometimes you are coming up with new techniques. At the core of this, however, is thinking about problems and thinking about solutions. It’s about being an inventor.

Continue reading Test Teams Need Inventors: Seeing and Thinking Differently →

Published inCommunication

Cars, Shapes, and Tests

written by

I think the future of testing relies less on a sole focus on test execution and more on the ability to write tests in the language of the business domain, effectively tying together the role of business analyst and tester. I think that future also includes the ability of testers to instrument certain artifacts, like requirements documents, so that the requirements, acting as tests, can be converted to different formats, most notably automated tests. There’s a lot of contention out there around these ideas from “purist” testers: those who feel that these attempts provide a single-source test format is getting away from testing and more into development.

While I agree in part that this is putting some focus on development activities, I disagree that this gets away from testing. Effective testing, to me, is about putting testing where it will do the most good in the most responsible fashion. Sometimes that testing is finding new bugs, sometimes it’s providing confidence that bugs haven’t come back. Either way, testing is all about communication.

Continue reading Cars, Shapes, and Tests →

Published inThinking

Epistemological Angst (or When Does “Before” Lose Meaning?)

written by

I find myself in a philosophical mood today and it’s based on some experiences with testers that simply don’t ask questions. As testers we have to ask questions. Lots of questions. We also have to recognize when we are getting answers. Sometimes, however, we have to realize that sometimes answers aren’t that easy to come by. And here I’m not just talking about the idea that no one has the answers. What about when answers just aren’t possible at all? Can that happen? Of course it can. It depends on the types of questions being asked and the context in which answers are expected. (See my time travel post for one example of this.)

So — just bear with me here — I want every tester out there to consider a question: what existed before the big bang?

Continue reading Epistemological Angst (or When Does “Before” Lose Meaning?) →

Published inBDDTesting

Testing’s Brave New World

written by

As a tester you will often encounter environments that desire to practice some form of Behavior-Driven Development (BDD) and testing practices must fit within that. The term “BDD” may be used more or less vaguely. One thing that’s often somewhat common is that people will start to promote so-called “BDD tools.” Examples here would be tools like Cucumber, Spinach, Turnip, SpecFlow, Behat, Lettuce, and so forth. These tools all ask you to adhere to some sort of organization based on terms like “feature”, “story”, “scenario” and so on. What this often ends up doing is totally missing the point.

But what is the point?

Continue reading Testing’s Brave New World →

Published inTesting

Finding Those Hidden Bugs

written by

One area of testing that often gets neglected are the various ways that bugs can be ferreted out by considering how they “hide.” Testing an application is sometimes like walking a mine field, where you have to trigger the mines in order to see where they are hidden. Another analogy might be a virus hunter, who has to find signs of an outbreak by testing the population, some of whom are hiding to cover up the fact that they have a virus. Here I want to present some information about why testers (and developers!) should consider a technique known as hidden-fault analysis and, more importantly, show how the notion of hidden faults can skew test results.

Continue reading Finding Those Hidden Bugs →

Published inTesting

Welcome Mutants to Your Testing Strategy

written by

Have you ever come across the term “mutation testing”? My experience is that some people might have heard about the concept of mutation testing — perhaps in passing — but rarely have they heard too much substantive about it. Even when I talk with developers and testers, I was initially surprised at how few of them readily accommodated this sort of technique. On the other hand, if there isn’t a lot out there about it, my surprise is unwarranted. That’s even more so the case if the material out there does not talk about how mutation testing can actually be practical.

Continue reading Welcome Mutants to Your Testing Strategy →

Published inThinking

Cautionary Tales on Metrics

written by

Some testers — and most managers — like to talk about metrics. One thing that often doesn’t get discussed is what I call metric dissociation. Here’s my thought: metrics should be easily or directly interpretable in terms of product or process features. If they’re not, I think they are dissociated from the actual work that’s being done and thus there’s great danger of the metrics being misleading, at best, or outright inaccurate, at worst.

Continue reading Cautionary Tales on Metrics →

Published inCapybaraCucumber

Thinking and Testing with Executable Specs – Part 5

written by

This is the fifth post in my series of using a tool that supports writing executable specifications. If this is the first post in the series you’re reading, then you are also reading the last post in the series. Which means this post will be almost totally meaningless to you. To catch up you can read parts one, two, three, and four. (Whether those make this post less meaningless is then up to you!)

Continue reading Thinking and Testing with Executable Specs – Part 5 →

Published inCapybaraCucumber

Thinking and Testing with Executable Specs – Part 4

written by

This post is the fourth in my series regarding approaching testing when you are using an executable specification type tool. The first, second and third posts are pretty much necessary reading for this one to make any sense to you at all.

Continue reading Thinking and Testing with Executable Specs – Part 4 →

Published inCapybaraCucumber

Thinking and Testing with Executable Specs – Part 3

written by

This post is the third in my series regarding approaching testing when you are using an executable specification type tool. This one will following on directly from the activities done in the first two posts. (I recommend reading part one and part two if you want to follow along, since these really are interconnected threads.)

Continue reading Thinking and Testing with Executable Specs – Part 3 →

Published inCapybaraCucumber

Thinking and Testing with Executable Specs – Part 2

written by

This post is the second in my series regarding approaching testing when you are using an executable specification type tool. Assuming you want to follow along, the first post is fairly critical to understanding how the project is currently set up.

Continue reading Thinking and Testing with Executable Specs – Part 2 →

Published inCapybaraCucumber

Thinking and Testing with Executable Specs – Part 1

written by

I’ve been doing some presentations and training sessions on using tools like Cucumber. What’s interesting about these is that in many cases people have tried to play around with the tool but have conceptual difficulties with figuring out how to get started in terms of making decisions continuing to using the tool.

As a tester, when introducing yourself to this area of practice, you might read that You’re Cuking It Wrong or that You’re Cuking It Right. It still may not be clear which you are doing. You might read about being imperative versus declarative and wonder which is for you. (Can’t I do both?) You might even come to the conclusion that Cucumber sucks.

Continue reading Thinking and Testing with Executable Specs – Part 1 →

Published inThinking

Good Testers Think Operationally

written by

When a professional discipline has a vocabulary for expressing the fundamental concepts on which it’s based, then you have a means for how practitioners in the discipline can make sure they’re talking about the same things. The problem with establishing such a vocabulary is that many people have come to use a variety of different terms or, even in those cases where the terms are the same, the definitions used are different. This is why I like to encourage people to think operationally.

So why are so many testers I meet so bad at this? Is this just not something they’re encouraged to do? Or am I just being too picky about how people use terms?

Continue reading Good Testers Think Operationally →

Published inAutomationCapybara

Starting to Tame Capybara

written by

Capybara is one of those open source testing tools that, when you get to use it, is really nice. However, it’s new enough (or, at least, changing enough) that good documentation on it is a bit sparse when your goal is simply to get started and you don’t have a lot of supporting frameworks in place. Case in point: what if I want to test against a browser that’s displaying my web application? I’m not using Rails. I have no other automated test frameworks in place. I just need to get started as quick as possible to determine if this tool will do what I need.

I’ll show you how I started out here.

Continue reading Starting to Tame Capybara →

Published inTesting

Testers Write Tools — Sometimes With Gems!

written by

I recently talked about testers writing tools. That particular tool, however, was a simple script. You can definitely improve your career options by being able to write more substantive tools.

As a tester I’ve recently had to start writing some Ruby programs that would serve as effective test frameworks. As it turns out, it was best for me to package up my logic in the form of a Ruby gem so that I could allow others to install it easily along with any dependencies.

Continue reading Testers Write Tools — Sometimes With Gems! →

Published inThinking

The Ubiquity of Ubiquitousness

written by

I really enjoyed the book Ubiquity: Why Catastrophes Happen by Mark Buchanan. This book doesn’t talk about quality assurance or testing at all, but it does talk about how things can change very quickly and how a certain context can dictate how things change. I definitely recommend the book for anyone working in a software development context because I found I could abstract ideas out of the context that the book talks about and apply them to the idea of change initiatives, faulty software, and the social context of software engineering environments I have worked within.

I’ll distill a little bit of what I got out of the book here, in the hopes of encouraging you to read it.

Continue reading The Ubiquity of Ubiquitousness →

Published inAutomationQTP

Action Sets in QTP

written by

Assume you have some data that tells your scripts how to function when they run a particular set of actions. Instead of this logic being stored in a test script, common action sets are stored off in a data source. In this case, I’ll use Excel but you can use any data source at all. One thing I’ve found fairly effective in terms of an approach here is to create a class that contains the methods for building a dictionary that will essentially bring in the “action sets.” Here an action set refers to the data plus the test actions to take with that data against the application.

Continue reading Action Sets in QTP →

Published inTesting

Testers Write Tools — Even Simple Ones!

written by

I was in an environment where it was clear some of the testers were struggling with the idea if comparing Excel workbooks: either as actual Excel files or CSV files that were imported into Excel. Expensive automation tools were being used to try to handle some of this. What I immediately thought of is simply: this is why testers do need to have some technical skills in the programmatic arena.

Continue reading Testers Write Tools — Even Simple Ones! →

Published inTesting

Put Some Heuristics Between Your Strategy and Your Tactics

written by

I like to consider a strategy and tactics distinction when you have a test team that’s striving to operate within the sphere of quality assurance. When you have that situation, then operationally considering strategy and tactics (or logistics, if you prefer) becomes important. The emphasis needs to be on a gradual implementation of good testing practices that then filter up to a full quality assurance process. Since any such quality function, in my opinion, starts with good testing practices of some sort, I’ll mainly make my distinctions here in the context of the idea of testing.

Continue reading Put Some Heuristics Between Your Strategy and Your Tactics →