Earlier this week I left a comment on a LinkedIn post about testing in an AI-assisted workflow. The post itself was fine, better than most. It argued that when the same AI writes both the function and the test, the two share the same blind spots, so you should write the test first and decide […]
Should the tester role even exist? Read More »
Here is what I do when I write code these days. I describe the change I want to one agent and let it write the code. I hand the result to a second agent, configured as a reviewer, using Bryan Finster’s Agentic Dev Team plugin for Claude. The reviewer reads what was written and gives
The Wrong Human-in-the-Loop Read More »
I read a post on LinkedIn this week that went, roughly, like this. The level of quality in software has never been as low as it is right now. We as quality professionals have been fighting for years to raise it, and now everyone is chasing speed, more lines of code faster, everything is broken, and nobody seems to care. Someone replied underneath, in the way only LinkedIn replies do: nobody cared about quality before AI either, so why did you expect that to change now?
The back-and-forth that followed went in the usual direction. Someone mentioned professional software craftsmen who did care before and still do. Someone else brought up the analogy that AI is a second industrial revolution. The frame kept coming back to craftsmanship on one side and slop on the other, with the industrial revolution doing real work in the middle, as if the lesson of that period were obvious and on the side of the craftsmen.
Quality professionals are fighting the wrong battle Read More »
This is part of a series where I write about what I’m learning in an AI evals and analytics course. Earlier posts cover the basics of evals, sniff tests, and quantitative evaluation with LLM-as-a-judge. In traditional testing, the oracle is trustworthy. You know what the right answer is, or at least you know who decides.
The oracle has opinions Read More »
Additional thoughts on session three of the AI Evals and Analytics Playbook1. The first part is here2. Every major shift in how we build and ship software has been followed by a wave of tooling that automates the tractable part and leaves the actual problem to the practitioner. Agile gave us story pointing ceremonies and
The Hard Part of AI Evals Isn’t the Tooling Read More »
Session two closed with a question I couldn’t answer.1 When a product scores 78% on a given metric, what tells you whether that’s good enough to ship? I flagged it as something session three would probably address, and it did, though not in the way I expected. The question can’t be meaningfully answered until you’ve
The anatomy of a metric Read More »
Every tester has had that moment. You look for guidance on a current problem and end up with advice that feels twenty years old. You find articles on writing test cases in Excel, or long arguments about whether automated testing can ever replace manual testing. It is a little surreal. It is easy to assume
The reason so much testing content feels stuck in the past Read More »
The struggle every quality professional knows too well If you’re responsible for quality in your organisation, this scenario probably sounds painfully familiar: You’ve identified critical quality issues that need addressing. You’ve meticulously documented the bugs, you’ve raised concerns about technical debt, and you’ve even suggested process improvements that could prevent these problems. But when you
From frustrated tester to respected quality advisor: The Q.E.D. method Read More »
with Risk-Based Testing and the Eisenhower Matrix In software development, testing is often a balancing act. On one hand, we want to ensure comprehensive coverage to catch every possible bug. On the other hand, resources are limited, and time is always against us. This can lead to a challenge familiar to many testers: how do
Maximising Efficiency in Software Testing Read More »
Recently somebody mentioned to me that they don’t really like the test pyramid, and prefer the swiss cheese model. So I looked it up (again), and thought “yeah, that’s nice”. And you know how it sometimes goes, it kept niggling away at the back of my brain – now there’s a mental image you didn’t
The Swiss Cheese Model vs. Test Pyramid: A Complementary Approach Read More »
