Session two closed with a question I couldn’t answer.1 When a product scores 78% on a given metric, what tells you whether that’s good enough to ship? I flagged it as something session three would probably address, and it did, though not in the way I expected. The question can’t be meaningfully answered until you’ve […]
The anatomy of a metric Read More »
Session one left me with two things I hadn’t resolved.1 The first was a line the instructor said almost in passing: “the hard part is scalability, not automation.” I wrote it down because it piqued something, but I couldn’t quite work out what problem it was pointing at. The second was a question I kept
The AI evals field chose a flawed tool and stuck with it Read More »
If you’ve been following me on Linkedin, or indeed have been reading my blog for a while, you’ll know that I’m interested in AI applications. So it’s probably not coming as a total surprise to you that my main learning goals this year evolve around AI evals, “which are systematic tests and measurements to that
What I don’t understand about AI evals (yet) Read More »
You know the feature inside out. You have tested it, broken it, rebuilt the test suite around it twice. So you ask the AI to generate a few test cases. Save yourself twenty minutes. What comes back looks reasonable. The structure is right. The naming conventions are close enough. Then you start reading. The first
You were promised speed. You got a new job instead. Read More »
I’ve been seeing a lot of posts on LinkedIn recently that make the same claim: speed of writing code was never the bottleneck in software engineering. The real bottleneck was always thinking, choosing the right thing to build, understanding the problem. AI has just made that painfully obvious. I partly agree, but the framing is
Why the bottleneck was never just about thinking Read More »
Every time the topic of using AI for code reviews comes up, someone will eventually say that it is like letting the AI mark its own homework. It sounds clever. It also sounds responsible. Underneath it sits a familiar discomfort about oversight, trust, and professional judgement. As an analogy, it points attention away from the
Review as analysis, not authority, when working with AI Read More »
Sometimes I despair about the QA profession. Not because the work is hard, or because quality is complex. Both of those are true, but they always have been. What gets to me is where so much of our collective energy still goes. We have much bigger fish to fry. We have the tools to do
AI did not break testing Read More »
Every tester has had that moment. You look for guidance on a current problem and end up with advice that feels twenty years old. You find articles on writing test cases in Excel, or long arguments about whether automated testing can ever replace manual testing. It is a little surreal. It is easy to assume
The reason so much testing content feels stuck in the past Read More »
The experimenter’s mindset beats the automation mindset A few months ago, someone at a conference asked me whether I thought testers would still have jobs in five years. It wasn’t a joke. You could hear the anxiety in the room, because the arrival of generative AI has reignited a very old fear in our industry:
Why the Experimenter’s Mindset Outlasts Automation in Software Testing Read More »
Document less, share more: A modern take on test evidence Do we still need test case management tools? In my experience, not really. At least, not in the way most teams are still using them. Test case management tools are everywhere, but they’re not helping us make better decisions. In modern software teams, they’re often
Document less, share more: A modern take on test evidence Read More »
