Here I’m going to continue the thread from the previous post, where we started to look at the concept of Runnables, which is really what puts the “Chain” in “LangChain.”
Continue reading AI and Testing: LangChain and Orchestration
Author: Jeff Nyman
Anything I put here is an approximation of the truth. You're getting a particular view of myself ... and it's the view I'm choosing to present to you. If you've never met me before in person, please realize I'm not the same in person as I am in writing. That's because I can only put part of myself down into words.
If you have met me before in person then I'd ask you to consider that the view you've formed that way and the view you come to by reading what I say here may, in fact, both be true. I'd advise that you not automatically discard either viewpoint when they conflict or accept either as truth when they agree.
AI and Testing: LangChain Messages
In the previous post, we got familiar with LangChain templates and dipped our toes into messages. In this post, I’m going to focus a bit more on those messages since these are the key to communicating with AI.
AI and Testing: LangChain Templates
In this post I’m going to follow the thread from the previous post and dig more into the LangChain ecosystem and start looking at the idea of templates for prompts.
AI and Testing: Local LLMs and LangChain
The previous post covered the concept of Ollama, to get a local LLM on your machine. Here, I’ll focus on using that LLM and introduce two key properties of testability in this context. Doing so will introduce LangChain.
AI and Testing: Ollama and Models
In this post I want to take the initial steps to get some basic tooling available and operating. This is step one if you’re going to work in a technologist context with AI applications.
AI and Testing: Evaluating the Future
As our technocracy continues to grow and as (at least some) technologists continue to push us toward a potentially dehumanized and dehumanizing future, I want to focus on how we can work from within this technocracy to make sure that human experimentation is front and center.
Testing for Quality, Betting on Value
There’s an irony worth noting with my previous posts on Hollywood quality and gaming quality: testing exists, in part, to mitigate risk but only by helping people understand the risks that exist. Yet, quality itself often requires reasoned and reasonable risk-taking! Let’s dig in to this.
The Sunk Cost of Quality: Lessons from Gaming’s Biggest Failures
In the first part of this series, I examined how Hollywood’s financial model (commit hundreds of millions upfront, spend it all before release, then discover whether audiences agree with your projections) creates a high-stakes gamble on predicting quality. Studios bet enormous sums on forecasting how diverse audiences will perceive value years in the future, often with not-so-great results. The gaming industry faces a strikingly similar challenge, but with crucial differences that make the quality prediction problem even more complex.
Continue reading The Sunk Cost of Quality: Lessons from Gaming’s Biggest Failures
The Sunk Cost of Quality: Lessons from Hollywood’s Biggest Failures
When we talk about quality assurance in software, we often treat quality as something measurable, testable, and, if we’re honest, somewhat objective. Does the code work? Does it meet requirements? Does it perform under load? However, quality isn’t entirely objective. It’s a shifting perception of value over time, influenced by customer expectations, cultural context, and changing needs. To understand why this matters, let’s step outside software for a moment and look at an industry that bets hundreds of millions of dollars on predicting quality: Hollywood.
Continue reading The Sunk Cost of Quality: Lessons from Hollywood’s Biggest Failures
Guarding Quality From Drift to Discipline
Quality doesn’t collapse overnight. It drifts. It drifts in the seams between teams, in the silence between a feature being built and a feature being tested, in the gap between what we meant to cover and what we actually did. That drift is often invisible. Until it isn’t. That’s why testing can’t live in a corner of the organization. It has to be democratized, distributed, and deliberately practiced. And if we’re serious about doing that, we need ways to see the drift before it becomes damage. Let’s dig in!
Using Narratives to Sharpen Testing Skills
As testers, we spend much of our time reviewing requirements, specifications, and user stories. We’re looking for ambiguities, inconsistencies, and contradictions. However, these analytical muscles can be exercised anywhere, including in narrative fiction. Let’s dig in!
Testing Has Something To Do With Mass Extinction
Okay, I’ll admit my title is a bit of click-bait. The better title would be “Testing Has Something To Do With Paleontology” but even that would not be correct since what I really would have to say is “Testing Has Something To Do With Paleontological Debates About Mass Extinctions in the Fossil Record.” Ugh. Even worse. You know what, let’s just dig in. (Pun slightly intended.)
Continue reading Testing Has Something To Do With Mass Extinction
The Vibe Around Vibium
There’s a lot of buzz around Jason Huggins’ “Vibium” tool. It’s generating a lot of chatter, but there’s a fair bit of skepticism about whether it’s a tangible product or something more speculative, maybe even a tongue-in-cheek nod to the testing community. Let’s unpack this.
Quality Assurance for Society
As someone who spends their days thinking about quality assurance and testing, I’m trained to look beyond whether something works to ask whether it works well, and for whom. Quality isn’t just about technical functionality; it’s about how humans interact with technology, what happens when systems fail, and whether the design serves user needs or merely designer intentions. These questions become critical when we’re not just testing software, but evaluating proposals to restructure society itself around technological systems.
The Quality Constant: Think and Act Experimentally
I was asked what one piece advice I have that I would give to testers starting out in the industry. It had to just be one piece of advice. It’s an interesting challenge.
Continue reading The Quality Constant: Think and Act Experimentally
Nothing to Do with Testing?
You will hear a certain segment of people say things like “TDD has nothing to do with testing” or “automation has nothing to do with testing.” This is often an ill-framed argument. Let’s talk about why this matters.
Testers, Code and Automation, Part 3
Working in our hypothetical developer-tester context over the last two posts, we’ve done some good work. We have a working implementation and we have some tests. Here we’ll finish up the work and close with some thoughts on the journey.
Testers, Code and Automation, Part 2
This post will continue from the first post. Here I’ll continue the exploration of testers interacting and intersecting with code, while making sure to consider good test and code practices.
Testers, Code and Automation, Part 1
There is much talk out there about whether testers should learn code. There is even more talk out there about automation. What there isn’t, at least so far as I can see, is much that shows actual examples that break down some concepts, particularly for testers entering the field. Here’s one of my attempts.
Test Defender!
With the advent of 2025, I realized it was the forty year anniversary of one my earliest exposures to testing. This exposure involved helping an arcade operator test cabinet-based coin-op arcade games. I decided to try and recreate this context and allow others to do the same thing.