There’s a lot of talk out there about using large language models to help testers write tests, such as coming up with scenarios. There’s also talk out there about AI based tools actually doing the testing. Writing tests and executing tests are both a form of performing testing. So let’s talk about what this means in a human and an AI context. Continue reading AI-Powered Testing: Exploring and Exploiting with Reinforcement
Category: AI
Text Trek: Navigating Classifications, Part 6
In this final post of this series, we’ll look at training our learning model on our Emotions dataset. This post is the culmination of everything we’ve learned in the first three posts in this series and then implemented in the previous two posts in this series. So let’s dig in for the final stretch! Continue reading Text Trek: Navigating Classifications, Part 6
Text Trek: Navigating Classifications, Part 5
This post, and the following, will bring together everything we’ve learned in the previous four posts in this text classification series. Here we’re going to use the Emotions dataset we looked at in the last post and feed it to a model. Continue reading Text Trek: Navigating Classifications, Part 5
Text Trek: Navigating Classifications, Part 4
In this post, we’re going to look at the Emotions dataset that we briefly investigated in the previous post. Here we’re going to consider the basis of that dataset. Then we’ll load it up and see if what we have to do in order to feed the data to a training model. Continue reading Text Trek: Navigating Classifications, Part 4
Text Trek: Navigating Classifications, Part 3
In this post, we’ll explore some particular datasets. The focus here is just to get a feel for what can be presented to you and what’s available for you to use. We’ll do a little bit of code in this post to get you used to how to load a dataset. Continue reading Text Trek: Navigating Classifications, Part 3
Text Trek: Navigating Classifications, Part 2
Here we’ll continue on directly from the first post where we were learning the fundamentals of dealing with text that we plan to send to a learning model. Our focus was on the tokenization and encoding of the text. These are fundamentals that I’ll reinforce further in this post. Continue reading Text Trek: Navigating Classifications, Part 2
Text Trek: Navigating Classifications, Part 1
Let’s start this “Thinking About AI” series by thinking about the idea of classifying text. Classifying, broadly speaking, relates to testing quite well. This is because, at its core, the idea of classification focuses on categorization of data and decision making based on data. More broadly, as humans, we tend to classify everything by some categories. Continue reading Text Trek: Navigating Classifications, Part 1
Thinking About AI
Many are debating the efficacy of artificial intelligence as it relates to the practice and craft of testing. Perhaps not surprisingly, the loudest voices tend to be the ones who have the least experience with the technology beyond just playing around with ChatGPT here and there and making broad pronouncements, both for and against. We need to truly start thinking about AI critically and not just reacting to it if we want those with a quality and test specialty to have relevance in this context. Continue reading Thinking About AI
AI Testing – Generating and Transforming, Part 3
We come to the third post of this particular series (see the first and second) where I’ll focus on an extended example that brings together much of what I’ve been talking about but also shows the difficulty of “getting it right” when it comes to AI systems and why testing is so crucial. Continue reading AI Testing – Generating and Transforming, Part 3
AI Testing – Generating and Transforming, Part 2
This post continues on from the first one. Here I’m going to break down the question-answering model that we looked at a bit so that we can understand what it’s actually doing. What I show is, while decidedly simplified, exactly what tools like ChatGPT are essentially doing. This will set us up for a larger example. So let’s dig in! Continue reading AI Testing – Generating and Transforming, Part 2
AI Testing – Generating and Transforming, Part 1
The idea of “Generative AI” is very much in the air as I write this post. What’s often lacking is some of the ground-level understanding to see how all of this works. This is particularly important because the whole idea of “generative” concepts is really focused more on the idea of transformations. So let’s dig in! Continue reading AI Testing – Generating and Transforming, Part 1
AI Testing – Measures and Scores, Part 2
AI Testing – Measures and Scores, Part 1
There are various evaluation measures and scores used to assess the performance of AI systems. As someone adopting a testing mindset in this context, those measures and scores are very important. Beyond simply understanding them as a concept, it’s important to see how they play out with working examples. That’s what I’ll attempt in this post. Continue reading AI Testing – Measures and Scores, Part 1
Human and AI Learning, Part 2
In part 1 of this post we talked about a human learning to play a game like Elden Ring to overcome its challenges. We looked at some AI concepts in that particular context. One thing we didn’t do though is talk about assessing any quality risks with testing based on that learning. So let’s do that here. Continue reading Human and AI Learning, Part 2
Human and AI Learning, Part 1
Humans and machines both learn. But the way they do so is very different. Those differences provide interesting insights into quality and thus the idea of testing for risks to quality. I found one way to help conceptualize this is around the context of games. Even if you’re not a gamer, I think this context has a lot to teach. So let’s dig in! Continue reading Human and AI Learning, Part 1
The Spectrum of AI Testing: Case Study
The Spectrum of AI Testing: Testability
It’s definitely time to talk seriously about testing artificial intelligence, particularly in contexts where people might be working in organizations that want to have an AI-enabled product. We need more balanced statements of how to do this rather than the alarmist statements I’m seeing more of. So let’s dig in! Continue reading The Spectrum of AI Testing: Testability
AI Test Challenge Follow Up
So, not surprisingly, the AI test tooling community didn’t want to engage on my AI test challenge. They saw it as being inherently unfair. And, to a certain extent, it could be. But what this really showcased was people are talking about AI and test tooling with way too much hyperbole in an attempt to gain traction. So was my test challenge unfair? Is there too much hyperbole? Let’s dig in a bit.
AI Test Challenge
This is not a challenge for testers to test an AI. Although that is a worthy challenge, one I tackled a bit. For right now, I want to propose a challenge for those promoting tools that claim to perform testing, particularly when the claim is that such tooling stands a chance of replacing human testers.
Will AI Perform Testing?
The title of this article is actually a little too simplistic. It’s more about asking: “Will AI Truly Perform Testing?” Or perhaps; “Will AI Perform Actual Testing?”