This post follows on immediately from the last one, so let’s get started with the exploration!
In that last post, I had said:
Keep in mind your goal for this post: it’s just to reason about things; to spot things; to draw conclusions based upon limited evidence. One of the most important aspects of exploratory testing is attention to detail, particularly in the face of uncertainty.
I don’t want to put too fine a point on it, but the whole focus of these posts is to have people actually try out the exploration ideas. Obviously there’s no way to really enforce that but not doing so is akin to buying a programming book but then not bothering to do any of the examples that it provides. So what I’m hoping is that if you are reading these — and particularly if you are a tester — you are not only trying these ideas out but also determining if this kind of concept might be helpful to introduce testing or to help refine testing skills.
Okay, so we were working with release 3 of the game. You had a bunch of new source text that I called out in the previous post to guide your thinking. So … what did you reason about it? If you didn’t get a chance, now’s the time to go back and try it out.
Go ahead. I’ll wait.
Results of Exploration
The best I can tell you here is what I hope you at least noticed in broad strokes. Different people are going to explore differently and thus notice different things. So let’s just consider a few ideas.
You saw that something referred to as a ‘backdrop’ was added for some trees. The description specifically mentions “brisk east wind.” Which should have clicked when you saw one of the kinds:
The prevailing wind is a direction that varies. The prevailing wind is east.
That would seem to suggest the description for those trees should need to be conditionalized at the very least.
You saw the player has three forms of currency: a credit card, a twenty-dollar bill, and a seven-sided coin. Along with that, You saw some properties added and two should have stuck out a bit:
A person can be compensated. A person can be providing a paid for product.
This ties in with the with the “Bird Woman” who you can read is “selling bags of crumbs”.
You should also have noted that the bird woman seems to have conditions placed upon how she is described. While you may not understand the details, the very nature of how the text is situated certainly should have triggered your exploratory ideas.
Speaking of conditions, you see some context that talks about every turn and provides a condition:
Every turn when the player is in Broad Walk: ...
That should tell you that it’s possible for something to recur but only under a specific condition. In the above case, there were two specific conditions. And one of those should give the tester in you pause: anything that provides functionality that says “a random chance of” should certainly have caught your attention.
Did you notice anything about the bird feed section in particular? Two things should have stood out:
In the bag of bird crumbs are some crumbs. The crumbs are edible.
So it sounds like we have something that is acting as a container, which should suggest some things to try. Opening it, perhaps, and closing it. But what about the idea of putting things in it. The crumbs would fit in a bag of bird crumbs. What about other objects?
Also, the crumbs are specifically marked as edible which should imply a few tests for you.
There seem to be some scenes and, at least from one of them, it seems they start and end under defined conditions:
Eastward Wind is a scene. Eastward Wind begins when play begins. Eastward Wind ends when the prevailing wind is west.
That scene, in fact, seems to suggest a change in the wind, which should harken back to the fact that you saw a backdrop about trees exhibiting wind effects and that prevailing wind direction you saw as the kind.
You see a scene for “Getting the Bag” and “Getting the Ruby”, which certainly should have linked in your mind to the “bird feed” section, which talks about “bags of crumbs” (plural) and a “bag of crumbs” (singular). You also saw the section about red buy.
Exploration Can Tell You a Lot!
As the above should show you, exploring even when you have only a vague understanding of all the elements can actually tell you quite a bit.
There are various ways someone can interpret the results of exploration, of course, but I think it’s fair to say that one possible viewpoint we’re getting with the above observations is that the player is going to have to get a bag from the woman — she’s selling bags, and the player has money, after all — and (maybe?) get a ruby. That ruby, by reading the source text, is in the bag.
Okay, so we have someone selling the bags and since currency seems to play a part, it’s likely that we’re going to need a buying action of some sort. If you had already pieced that together, then the title of this post would not have been surprising to you. If you had not, however, the title might have seemed a bit odd.
My overall point here is that all of this was possible to reason about without knowing the requirements of the business scenario that was hinted at in the prior post.
Did You Explore By Playing?
There are many possible commands you could have tried, should you have tried to play the provided release 3 version of the game. These are commands you could have thought of even if you did not have a working game to play around with. Here are some examples:
Those get you to the location of the Broad Walk and check the room description to see if it varies based on what the source text provides.
- look at the trees
- look at the pigeons
- look at the woman
This is just checking that the textual descriptions appear.
- look at the bags
- look at the bag
- look at the crumbs
- look at the small coin
- look at the change
- look at the ruby
These are interesting because they have you looking at things that are held by another character. It’s easy to imagine the game might handle those types of items differently. (Keep that in mind! I will return to it in a bit.)
- look at the credit card
- look at the twenty-dollar bill
- look at the seven-sided coin
- look at the fifty pence
This is making sure that things the player holds can be described as well. (Keep this in mind, too!)
- take the bags
- take the bag
Trying to take something held by another character. Exploring whether that’s even allowed makes sense. The above choices in particular are also testing whether the game seems to understand the distinction between “bags” and “bag”. This may also, in a roundabout way, lead you to consider ideas like
rob the woman or
steal the bag.
- buy the bags
- buy the bag
Given the apparent focus in “selling” and “currency”, these seem likely commands to at least try. Again, trying the singular and the plural.
Exploration Frames Testing — and Checking
So, keeping attention to detail in mind, I talked with you previously about the test statements in Inform and that might have led you to think about statements like these:
Test currency-describe with "examine credit card / examine twenty-dollar bill / examine seven-sided coin / examine fifty pence". Test room-context with "examine trees / examine pigeons / examine woman" in Broad Walk. ...
Those become “checks” in the current parlance that many testers like to adopt. But the design for them came out of all that exploration, which was very much about testing and test design.
Regardless of how you structured and performed your tests, there are definitely some interesting things to note in terms of how the game presents itself.
Explore for Knowledge-Based Interaction
Keep in mind that you only know about “the bag” (singular) from reading the source text. The text of the game never separately mentions “the bags” versus “the bag”. In fact, this distinction of two types (“bags”, “bag”) is being done for implementation reasons. It’s because eventually the player can buy a bag and that bag needs to be referred to individually at that time so the player can take it.
The same applies to the small coin that the woman is carrying. In fact, you might wonder why the woman is carrying currency at all but you also may be wondering if the player should be allowed to refer to it at all. If you tried the “look” command on it, you would see that the player could refer to it. But perhaps they should not be able to since the player character would not be aware of it.
This point is important because it brings up knowledge. Consider a player that plays the game for the first time. They won’t have the source text so they are not going to know the small coin is there. Now let’s say that as the game goes on, it becomes apparent to the player that the coin is there — perhaps the woman gives it to the player. So now the player restarts the game and plays again. Now the player knows about the coin — even before the woman introduces it — but the player character (the one being controlled by the player) would not. This suggests the coin can only be referred to at certain times.
The wider issue here is an important one: considering the state of knowledge of users; what they may know about, what they may not know about, and what they may know about only because they have certain knowledge gleaned from experience. In a different context, some users of web sites or applications are aware of how to add parameters to URLs to see what happens, even though the site or application itself is not particularly drawing attention to this. The user, in this case, is relying on knowledge from elsewhere.
Explore for Qualities
Clearly these games are predicated upon the display of text and, even more particularly, of that text flowing well. This is an example of a quality that might be hard to state as a requirement.
At the simplest, and most objective, level, one thing to note is that with “take the bags” versus “take the bag”, it’s good that the game seems to recognize plural (“Those seem to”) versus singular (“That seems to”).
At a more subjective level, you might think that the description of Broad Walk ends up reading a bit clumsy, with the “You can also see some pigeons here.” kind of tacked on.
Yet, in contrast, notice that only on the first ‘look’ in the Broad Walk room do you see “A cloud of pigeons fills the air! They circle overhead and congregate around a nearby bench, where an aged woman is selling bags of crumbs.” All second ‘look’ actions just show “An aged woman is selling crumbs nearby.” You might have intuited that a bit from the source text but you certainly would have seen it if you went hands-on. But the point is that this is why the “You can also see some pigeons here” text is added: to remind the player they are still there.
However, if you looked at the source text, you would find no text in there that says “You can also see some pigeons here.” This is something to figure out: how should items in rooms be displayed to the player? Is one mention enough? Or should the game text provide cues to objects that are in the room? This is only a hint of what turns out to be a much larger problem in textual interactive fiction.
Explore for Interaction Problems
Of particular import, notice the following:
> look at the crumbs Which do you mean, the bags of crumbs, the bag of bird crumbs or the crumbs?
Interesting, right? Here the game is attempting disambiguation: it’s trying to figure out what you meant given that you used a term that could mean different things. In this case, the game couldn’t decide on one item to pick over any other one, so it asks which item you did in fact mean.
But notice how I can only type in the things I’ve said before (“bags” or “bag”) or I can just type in “the crumbs” — which is what I just did type in. And if I type that again, I get the same question from the game. This suggests the game needs to be provided a better way to handle this. Specifically, it’s showing that “crumbs” — by itself — is not being disambiguated sufficiently enough.
You might also note also that “look at fifty-pence” — again, assuming you tried it — doesn’t work, even though the “seven-sided coin” is in fact fifty-pence. However, the “look at the small coin” and “look at the change” did work! And if you investigated the source, you might have seen why:
Understand "change" as the small coin.
So clearly providing ways for the player to be able to refer to things as they are referred to in the game is important.
How about that ruby? Obviously this doesn’t seem to make a lot of sense to show up automatically in the description. Did you notice that it did?
> look at the bag In the bag of bird crumbs are some crumbs and a red ruby.
Now, based on what you know right now, perhaps that’s all perfectly fine and that’s exactly how the game should tell the player about the ruby. Yet … we saw this:
Ruby Exposed is a scene.
That would imply that the ruby is going to be “exposed” at some point and thus should probably not just automatically be described.
Earlier, when showing you the commands you might have thought about or tried, I mentioned a few things to keep in mind: referring to or seeing items held by another character. Okay, so here’s a little bit of side credit that hopefully occurred to you at some level: the source text mentions that the player carries all that currency. But how does the player know that?
Well, how do you tell what the player is carrying. I never said anything about that. Although, if you explored the material available to you, like the IF Postcard, you would have read:
“Once you’ve acquired a few, you can also TAKE INVENTORY (or just type I for short) and get a list of things you’re carrying.”
Or even the smaller postcard, which says:
I: take INVENTORY of what you possess.
Had you tried that with the current game, you’d find that it’s results are less than stellar.
> inventory You are carrying: a vacation outfit (being worn) > look at vacation outfit You see nothing special about the vacation outfit.
Huh. That’s not ideal. There is no mention at all of the items that are in the pocket of the outfit. In fact, the pocket itself isn’t even mentioned. But the player can look at and even take each of the bits of currency they have (try it!). That’s a big bug waiting to happen and note that it’s one that would be easy to let slip as a tester because, after all, you know that stuff is being carried. But your eventual players do not.
All of this is exploring!
And lest all of this seem unfair (after all, I am writing the game), I would argue none of this is more or less difficult than what I presented when I was applying attention to detail in testing Star Wars: The Old Republic or exploring how to make the game world more realistic when testing Grand Theft Auto V. This is the kind of stuff that testers, particularly specialist testers, should be very good at.
Now let’s talk about buying for a second. When we went through the “photographing” example, we found that Inform didn’t even provide that verb. Inform, does, however, provide for the concept of buying as an action.
The Standard Rules that are built in to Inform define a “buying” action in only a minimal way, essentially blocking it entirely with a check rule. In other words, by default while buying is allowed as an action, it does absolutely nothing. Behind the scenes what Inform recognizes is a command of this type: “buy/purchase [something]”. (Oh, hey! Did you explore other ways of trying to indicate ‘buying’, such as ‘purchase’?)
You already saw some ideas about implementing the photographing action and how this was done with check and report rules. Do you want to take a stab at figuring out how you might test a newly created buying action? Think of this as working with the developers and the business to put pressure on design. This is a form of driving design by the consideration of tests.
What should stand out right away is that any action like “buy bag” has to make a lot of assumptions. The game would have to presume you meant the woman (and only if you were in the same location as her). Further, it would have to try to assume what you might to buy the bag with. Notice how you can think about this without seeing any implementation at all, based just an exploration of the ideas.
So … is that it?
Not quite. Let’s close by talking about that business scenario, which I promised you in the previous post.
Business / Game Scenario
Here’s the scenario, provided by the business:
We want to have an aged woman selling bird crumbs in the Broad Walk. There are going to be pigeons that the player can feed these crumbs to. However, the player has to buy a bag of crumbs from the woman. It’s important that the game make it clear that the player can buy these crumbs. Maybe the woman can say something? But she should say it somewhat regularly, just to keep reminding the player.
Okay, not a bad start. As a tester, had you been provided with this at the start, how much easier would the above exploring have been? In either case, you will have some sources of truth. In the case I took you through, you had the start of an implementation. Had you been provided with the scenario first, that’s what you would have had to go on. Perhaps you would have both.
The key point is to explore the sources of truth you have and see what you think about them. Are you framing some questions to ask? Are you thinking about some things you would want to check with this kind of implementation?
The scenario by business continues:
The player is going to start the game carrying some currency on them but not all of that currency would be applicable for this particular purchase. In fact, we want the woman to only accept British currency. After the player buys the bag of crumbs, the woman will offer the bag and some change to the player. The player needs to grab both of those items because they will be important later and the woman should indicate this. However, at some point, she’ll get fed up asking the player to take their purcahse and she’ll just keep it. So that’s the key scene: the woman offers the bag of crumbs if they’ve been paid for and the scene ends when she stops offering it.
Certainly a refinement of the scenario, right? And none of that was present in the source text. So keep exploring; keep thinking.
The business continues:
The main point of this is that we want another scene where the player uncovers a key object. It’s going to be a red ruby that’s hidden in the bag of crumbs they bought. Finding this causes the wind to start blowing a different direction in the game, which is going to be necessary for another puzzle later on. Finding this ruby triggers the end of the scene.
Okay, let’s stop there. A lot of detail. No doubt if you had the business person in the room with you, you would be asking questions as you explored this idea. But take some time — I can’t stress this enough! — to break down the scenario above into what you think you would test and what you think you need more information on.
The next post will close out this discussion by showing an implementation of this business scenario.