In the previous post, we got familiar with LangChain templates and dipped our toes into messages. In this post, I’m going to focus a bit more on those messages since these are the key to communicating with AI.
Category: AI
AI and Testing: LangChain Templates
In this post I’m going to follow the thread from the previous post and dig more into the LangChain ecosystem and start looking at the idea of templates for prompts.
AI and Testing: Local LLMs and LangChain
The previous post covered the concept of Ollama, to get a local LLM on your machine. Here, I’ll focus on using that LLM and introduce two key properties of testability in this context. Doing so will introduce LangChain.
AI and Testing: Ollama and Models
In this post I want to take the initial steps to get some basic tooling available and operating. This is step one if you’re going to work in a technologist context with AI applications.
AI and Testing: Evaluating the Future
As our technocracy continues to grow and as (at least some) technologists continue to push us toward a potentially dehumanized and dehumanizing future, I want to focus on how we can work from within this technocracy to make sure that human experimentation is front and center.
Navigating the AI Shift: A Tester’s Mandate
It’s very clear that artificial intelligence has become more democratized than at any other time in history. It’s also fairly clear that this democratization will not only continue but likely accelerate. What is the mandate for quality and test specialists in this context?
Continue reading Navigating the AI Shift: A Tester’s Mandate
AI-Powered Testing: Exploring and Exploiting with Reinforcement
There’s a lot of talk out there about using large language models to help testers write tests, such as coming up with scenarios. There’s also talk out there about AI based tools actually doing the testing. Writing tests and executing tests are both a form of performing testing. So let’s talk about what this means in a human and an AI context.
Continue reading AI-Powered Testing: Exploring and Exploiting with Reinforcement
Text Trek: Navigating Classifications, Part 6
In this final post of this series, we’ll look at training our learning model on our Emotions dataset. This post is the culmination of everything we’ve learned in the first three posts in this series and then implemented in the previous two posts in this series. So let’s dig in for the final stretch!
Continue reading Text Trek: Navigating Classifications, Part 6
Text Trek: Navigating Classifications, Part 5
This post, and the following, will bring together everything we’ve learned in the previous four posts in this text classification series. Here we’re going to use the Emotions dataset we looked at in the last post and feed it to a model.
Continue reading Text Trek: Navigating Classifications, Part 5
Text Trek: Navigating Classifications, Part 4
In this post, we’re going to look at the Emotions dataset that we briefly investigated in the previous post. Here we’re going to consider the basis of that dataset. Then we’ll load it up and see if what we have to do in order to feed the data to a training model.
Continue reading Text Trek: Navigating Classifications, Part 4
Text Trek: Navigating Classifications, Part 3
In this post, we’ll explore some particular datasets. The focus here is just to get a feel for what can be presented to you and what’s available for you to use. We’ll do a little bit of code in this post to get you used to how to load a dataset.
Continue reading Text Trek: Navigating Classifications, Part 3
Text Trek: Navigating Classifications, Part 2
Here we’ll continue on directly from the first post where we were learning the fundamentals of dealing with text that we plan to send to a learning model. Our focus was on the tokenization and encoding of the text. These are fundamentals that I’ll reinforce further in this post.
Continue reading Text Trek: Navigating Classifications, Part 2
Text Trek: Navigating Classifications, Part 1
Let’s start this “Thinking About AI” series by thinking about the idea of classifying text. Classifying, broadly speaking, relates to testing quite well. This is because, at its core, the idea of classification focuses on categorization of data and decision making based on data. More broadly, as humans, we tend to classify everything by some categories.
Continue reading Text Trek: Navigating Classifications, Part 1
Thinking About AI
Many are debating the efficacy of artificial intelligence as it relates to the practice and craft of testing. Perhaps not surprisingly, the loudest voices tend to be the ones who have the least experience with the technology beyond just playing around with ChatGPT here and there and making broad pronouncements, both for and against. We need to truly start thinking about AI critically and not just reacting to it if we want those with a quality and test specialty to have relevance in this context.
AI Testing – Generating and Transforming, Part 3
We come to the third post of this particular series (see the first and second) where I’ll focus on an extended example that brings together much of what I’ve been talking about but also shows the difficulty of “getting it right” when it comes to AI systems and why testing is so crucial.
Continue reading AI Testing – Generating and Transforming, Part 3
AI Testing – Generating and Transforming, Part 2
This post continues on from the first one. Here I’m going to break down the question-answering model that we looked at a bit so that we can understand what it’s actually doing. What I show is, while decidedly simplified, exactly what tools like ChatGPT are essentially doing. This will set us up for a larger example. So let’s dig in!
Continue reading AI Testing – Generating and Transforming, Part 2
AI Testing – Generating and Transforming, Part 1
The idea of “Generative AI” is very much in the air as I write this post. What’s often lacking is some of the ground-level understanding to see how all of this works. This is particularly important because the whole idea of “generative” concepts is really focused more on the idea of transformations. So let’s dig in!
Continue reading AI Testing – Generating and Transforming, Part 1
AI Testing – Measures and Scores, Part 2
In the first part of this post, I used a simple binary classification task to show some ideas around measures and scores and then provided some running commentary on how the tester mindset and skillset can situate in that context. That post was about depth; this post will be more about breadth.
AI Testing – Measures and Scores, Part 1
There are various evaluation measures and scores used to assess the performance of AI systems. As someone adopting a testing mindset in this context, those measures and scores are very important. Beyond simply understanding them as a concept, it’s important to see how they play out with working examples. That’s what I’ll attempt in this post.
Human and AI Learning, Part 2
In part 1 of this post we talked about a human learning to play a game like Elden Ring to overcome its challenges. We looked at some AI concepts in that particular context. One thing we didn’t do though is talk about assessing any quality risks with testing based on that learning. So let’s do that here.