TestGuild Automation Podcast

Running a proof of concept on test automation tools can feel like a horror movie. Vendors promise everything, your boss wants answers fast, and you don't know what you don't know. In this Halloween special, Paul Grossman, the Dark Arts Wizard, walks through his free personal automation tool POC checklist and shares the AI and Playwright tricks he's picked up this year. Key takeaways: Use a 177 feature checklist to compare tools side by side, saved locally so your data stays yours, with room to add any tool you find. See how AI driven and codeless tools stack up on build time and run time using Paul's 10 step Candy Mapper challenge. Practice on sandbox sites like Candy Mapper, Ministry of Testing's Shady Acres, and the SelectorsHub shadow DOM page. Save tokens and time with Copilot CLI, and keep your AI on your rules with a copilot instructions file in .github. Catch disappearing toast messages with Playwright listeners and use slowMo to stop false failures on fast loading pages. Why AI won't replace…

Performance testing has never been easier to start, or more important to get right. As teams ship more AI generated code, performance problems are slipping through, and they tend to surface during your highest traffic moments. In this episode, Joe Colantonio talks with Lars Holmberg, lead maintainer of the open source Locust project, and Abhinav Premsekhar of Microsoft's Azure App Testing team. They explain how AI is changing load testing, from script creation to results analysis. You'll see live demos of Azure Load Testing, including: Quick URL based tests for beginners Running existing JMeter and Locust scripts at scale A browser extension that records user journeys and generates load test scripts automatically Creating a Locust test from a Postman collection with the Azure Load Testing extension for VS Code Abhinav also walks through a real failed test run, showing how client side metrics, server side metrics, and AI insights work together to locate a backend bottleneck and recommend a fix. The…

Cloud device farms make it easy to get started with real device testing. But what happens when you need to scale? Costs climb fast, you wait in a queue for the devices you need, and you lose visibility into what's actually happening when a test fails. In this episode, Nick Metcalfe, Head of Technical Sales Engineering at Cambrionix, joins Joe to explain why more testing teams are bringing their device labs in house. Nick shares how one team was quoted roughly a million dollars a month to outsource their real device testing, then cut that cost to around $300K by building their own lab. And cost is only part of the story. You'll discover why an in house device lab can outperform a cloud lab: Lower cost at scale. Cloud pricing works for small test runs, but at enterprise scale an in house lab can save hundreds of thousands of dollars every month. No waiting in line. Your devices are yours, available whenever you need them, with no queue and faster turnaround. Full control and visibility. See exactly what's…

I asked thirteen people inside AI testing one question: can AI test your software for you? Founders, CTOs, project leads, and quality leaders, many of them selling AI testing tools right now. Not one said yes. This episode is my session from the Testear.la 2026 conference, built entirely from twelve months of interviews on this show with the people actually building this technology, including Applitools, Maestro, Bright Security, QA.tech, Cucumber, Ito, and the lead of the Selenium project. In it you'll learn why one AI testing vendor deliberately removed AI from the part where tests run, the real productivity number teams are measuring versus the number in the case studies, the four incompatible architectural bets vendors are making and how to tell which one your tool has chosen for you, why AI generated code is producing vulnerabilities far faster than it produces code, and the one architecture rule that stops an agent from editing the test that was supposed to catch it. You'll also get the list. Every…

Is AI going to replace software testers, or make them more valuable than they have ever been? Jonathon Wright says both, and in this episode he does not back down from either position. Jonathon Wright has spent thirty years in test automation, going all the way back to WinRunner. He now runs thirty AI agents across twelve screens, he sat with Gartner for their research on agentic platforms, and he says the last six months changed the job more than the previous three decades combined. We get into what he calls confidence engineering, why he has not opened a browser in five years, and what happened when he generated thirty seven thousand test scenarios from space agency specifications and put them in front of NASA. He also tells the story of a testing leader who taught his agents everything he knew about accessibility work and then got let go. A few minutes later he tells you your career has another fifteen good years in it. I pushed him on the contradiction. Listen for how he answers it. What you will learn…

Everything you knew about performance testing changes when the system you're testing is non deterministic. In this episode of the TestGuild Automation Podcast, Joe Colantonio sits down with Kandasamy Selvaraj, Principal Architect and author of the free book Rethinking Performance Engineering for Agentic AI, to unpack what it really takes to move an AI agent from a working demo to an enterprise system handling millions of conversations per hour. Checkout his free book: https://leanpub.com/agentic-ai-performance Kandasamy shares the practical playbook he's built running agentic AI in production, including why the same request can take three seconds one run and eight seconds the next, how to use harnesses to bound tool calls, reasoning loops, and token budgets, and why your SRE dashboard can look perfectly healthy while your token costs quietly balloon to five times baseline. You'll learn how his team: Shifts performance gates left into every commit with JMeter Shifts right with synthetic monitors on blue green…

Most testers are stuck arguing about whether AI is coming for their jobs. Swati Seela thinks that's a distraction from a much more useful question: when we say "AI testing," what do we actually mean? What do you think? Take our TestGuild State of Test Automation Survey now: https://testgld.link/27data In this episode, Swati walks through the AI Testing Landscape, a practical framework she presented at CAST 2026 that breaks AI testing into five overlapping areas: AI assisted testing, AI evaluation and trust, AI development quality, testing products that contain AI functionality, and testing the AI models themselves. Each one has a different test intent, a different technique, and a different toolset, which is exactly why lumping them together leaves new testers with no idea where to start. Along the way Swati gets specific about the failure modes she keeps hitting in real work. Why AI opens almost every response by agreeing with you, and why that false sense of correctness is the modern version of a green…

Manual testing never died. IoT is the proof. In this episode, Joe Colantonio talks with Oleksii Cherkashyn, a QA team lead, automation engineer, and IoT testing specialist who built a complete test architecture from scratch across hardware, firmware, mobile, web, and API. IoT breaks the usual automation playbook. You are not testing a shopping cart. You are connecting a real microcontroller, flashing a sketch, validating that a physical command produced the right change in a dashboard widget, and then doing the same thing again through the REST API and the mobile app. No marketplace framework covers that, so Oleksii built his own. You'll learn: Why manual QA remains essential in IoT and which cases can never be automated How Oleksii built a custom Node.js library to simulate up to 50,000 device connections for performance testing without buying the hardware Why he chose WebDriverIO over Playwright and the mobile automation reason behind it How he uses WebDriverIO MCP and cloud coding agents every day to…

In this episode 600 of the TestGuild Automation Podcast, Joe Colantonio talks with Jason Arbon, founder of Testers.ai, Jank.AI and IcebergQA and author of the new book Testing AI: Engineering Confidence in Non-Deterministic Systems. Take Our 2027 Survey Now: https://testgld.link/27data Jason makes a case most testers have not heard yet. Coding is being absorbed by AI. Specification work is thinning out. Product, development, and test roles are converging into one. And when the music stops, the only seat left belongs to the person who can look at what the machine produced and make an evidence backed call on whether it ships. He calls that confidence engineering, and he argues it is not a rebrand of QA. It is what QA was always supposed to be. Along the way, Joe and Jason get into the containment problem and why alignment, not lockdown, is now the real safety goal. They dig into why testing cost scales quadratically, meaning ten times more generated code creates roughly a hundred times more testing demand.…

Your developers just got supercharged by AI coding agents. Your test coverage did not. So how do you keep quality high when product code is shipping faster than any test team can follow? In this episode Andrew Knight, the Automation Panda and Senior Director of Product and Engineering at Cycle Labs, shares how his six person team is running the biggest release quarter in company history using AI coding agents, spec driven development, and Playwright. You will discover: How Playwright turned itself into an AI automation platform with the MCP server, the planner, generator, and healer agents, and the new CLI and skills approach that cuts your token usage way down. Why Andy uses Spec Kit to codify his testing strategy once, in markdown, so quality standards get baked into every single pull request instead of being caught in review. How to decide which AI generated tests are actually worth running when compute time and budget are finite. What AI slop looks like from a manager's seat, and how to build a team…

Keith Klain has spent 25+ years leading enterprise quality programs in financial services and is one of the testing industry's most respected voices. In this episode, he joins Joe to discuss his BrowserStack talk, "The Death of Test Engineering," why testing isn't dying—but outdated testing roles are—and how AI is reshaping the future of software quality. In this episode: Why "The Death of Test Engineering" isn't what you think The difference between checking and testing Will AI replace software testers? Why confidence engineering matters How to communicate testing in business terms Where automation adds real value Why agentic AI testing is overhyped Notable Quotes "If you look at AI as a competitor, you're screwed. If you look at it as just another tool... it's not a threat." "Uncertainty in testing is a feature, not a bug." "If an organization doesn't value testing, you're doing it wrong." "You can be incredibly confident about something and incredibly wrong." One Thing to Try This Week Ask yourself after…

Nandini Srinivasan has spent 25 years in the quality industry and leads a global QA organization of over 150 engineers across the US, Canada, India, and Pakistan. In this episode, she breaks down exactly how she built a QA AI acceleration charter, ran a train-the-trainer model, and used a phased proof-of-concept approach to separate real AI from what she calls "powerful automation dressed up as AI. We get into: How she frames quality metrics for executives Using language around revenue protection Risk mitigation Feature velocity instead of test coverage percentages. She talks about the four pillars she uses to present her team's value: quality, scalability, performance, and availability. Nandini also shares her take on the future of QA hiring, why the judgment layer will always require a human, and what skills testers need to stay relevant as AI agents take over the more mechanical parts of automation. She is also writing a five-part LinkedIn series called "The Voice of QA in the AI Era" if you want to…

Matt Wynne, co-creator of Cucumber and BDD practitioner, joins Joe for the first time in over a decade to talk about what two years inside a Silicon Valley AI startup taught him about the future of software testing. Matt spent time at Mechanical Orchard working alongside experienced XP practitioners to modernize legacy COBOL mainframes using LLMs, and then spent a week with the team that coined the term "software factory," where the rule was simple: humans never write the code, never read the code. In this episode, Matt breaks down what harness engineering actually means, why shared understanding is still the real bottleneck even in an agentic world, and how testers can use multiple LLMs to review AI-generated pull requests without reading every line. He also gets honest about the grief that comes with realizing you can encode years of hard-won expertise into a Markdown file, and why that does not mean your skills are worthless. If you are working in a brownfield codebase, wondering how to handle the flood…

Amit Rawat is an agentic engineer who spent two decades in QA before shifting fully into building AI agents. He's the creator of PromptWright, a desktop tool that turns natural language prompts into automated Playwright browser tests, complete with screen recording, Gherkin scenario generation, and self-healing locators. In this episode, Amit and Joe get into what it actually takes to work with AI agents at a high level, starting with why the planning phase matters more than the prompt itself. Amit breaks down his own workflow, brainstorming with AI, building a detailed plan in HTML before ever executing, and why curiosity and technical depth still matter even as AI gets more capable. They also cover why Amit believes QA professionals, more than developers or DevOps engineers, are best positioned to thrive in the agentic era, how he tracks the ROI on his $200-a-month Claude subscription, the "Chief of Staff," "Chief Health Officer," and "Chief Financial Officer" AI agents he's built to help manage different…

Most API testing stops at the happy path. The problem is that the bugs that actually hurt you in production are sitting in everything many testers skip, like the boundary values, the oversized payloads, the missing tokens, the security headers, the inputs that make no sense at all. In this episode, Joe sits down with Liudas Jankauskas, who has spent almost twenty years breaking software and testing APIs since 2008. Liudas demonstrates Rentgen, his free and open-source API testing tool, live on screen. You'll watch him take a single request from a real app, map it in seconds, and generate dozens of tests covering security, boundaries, performance, and load—all from one click. You'll learn: How to discover APIs hiding under the hood of any application, even when there is zero documentation Why happy path testing leaves you exposed How to run a fast hygiene check before your real automation ever starts Liudas also explains why Rentgen runs completely locally with no server and no data leaving your machine,…

Your AI code review tools read the diff. They stare at your code. But they never actually run it. So the bugs that only show up at runtime, the broken user flows, the bad query plan, the duplicate submission, sail right past review and land in front of your customers. In this episode, Joe Colantonio sits down with Evan Marshall, founder of Ito and a fifteen year engineer who spent five years in applied cryptography securing hundreds of millions of dollars for millions of people. Evan is taking that ship fast without breaking things discipline and pointing it straight at testing. Ito is an agentic QA platform that builds and runs your actual app on every pull request, navigates it like a real user, exercises the frontend and backend as one system, and brings back real runtime evidence: video replays, logs, the exact lines responsible, and steps to reproduce, posted right in your PR. You will learn: Why static code review misses the bugs that cause real production incidents How Ito spins up ephemeral…

What happens to QA when AI is writing ten times more code than your team can test? That is the exact problem Ivan Barajas Vargas set out to solve with Amikoo, a purpose-built AI QA agent designed to help testers, SDETs, and even developers move faster without sacrificing coverage or quality. Ivan is no stranger to AI in testing. Before generative AI became mainstream, he co-founded MuukTest, a test automation platform built on symbolic reasoning and expert systems. After six years and thousands of customer conversations, he went back to first principles to build Amikoo from scratch, this time with a harness of 12 specialized agents and 43 tools trained specifically for testing workflows. In this episode, Ivan and Joe dig into the real-world gap between AI code generation and AI-powered testing, why the QA role is being elevated rather than replaced, how Amikoo uses Playwright and page object model patterns under the hood, and where human judgment still has to stay in the loop. Ivan also shares practical…

Everyone is talking about AI replacing testers, writing tests, and transforming software quality. But what if we're asking the wrong question? In this solo episode, Joe Colantonio shares a growing concern he's seen while traveling across the country for TestGuild IRL events: a decline in testing fundamentals at the exact moment AI hype is reaching a fever pitch. Drawing insights from Carissa Véliz's book Prophecy: Prediction, Power, and the Fight for the Future, Wayne Roseberry's work on AI and meaning, and Tariq King's concept of Human Experience Testing, Joe explores why AI systems may be far less intelligent than many believe, and why human testers remain more important than ever. You'll discover: ✅ Why large language models generate plausible answers without understanding truth ✅ The difference between prediction, correlation, and genuine understanding ✅ Why AI can test software but cannot experience software ✅ What "Everything is tested, but nothing is experienced" really means ✅ How AI hype may be…

AI coding tools promised to make development faster — and they delivered. But here's the problem nobody talks about enough: when you speed up coding, you don't eliminate the bottleneck in the SDLC. You just move it. And for most teams, it lands squarely in QA. In this episode, Joe sits down with Vilhelm von Ehrenheim, Co-founder and Chief AI Officer of QA.tech, to dig into how agentic AI is reshaping software testing from the ground up. Vilhelm brings serious ML credibility, he helped build Motherbrain, one of the earliest production LLM systems in venture capital, and he's now applying that experience to one of the hardest problems in software delivery: testing at AI development velocity. You'll learn how QA.tech's behavioral knowledge graph gives AI agents the context they need to actually understand your application, why validating user intent beats checking element identifiers every time, how autonomous agents can review PRs, reproduce bugs from Slack messages, and generate targeted tests without a…

What happens when AI agents can not only write mobile app code, but also validate their own work automatically? In this episode, I sit down with Maestro Co-founder and CEO Leland Takamine to explore one of the biggest shifts happening in software testing right now: agentic mobile testing. Leland shares how his team went from solving mobile performance testing challenges to building one of the fastest-growing mobile automation frameworks used by companies like Microsoft, Meta, Amazon, and DoorDash. We dive deep into: How AI coding agents are changing mobile testing workflows What "closing the agentic feedback loop" actually means Why deterministic testing still matters in the age of AI How Maestro MCP lets AI agents validate mobile app changes automatically Why mobile test maintenance may finally become manageable The future role of testers as AI-generated code explodes Leland also gives a live demo showing an AI agent building, validating, debugging, and generating a reusable mobile test completely…

AI-powered testing tools are exploding across software engineering teams… but so are the hidden costs. In this episode, Joe sits down with Arthur Hicken to unpack the growing problem of runaway AI token usage, unexpected LLM billing, and the operational risks of deploying AI agents into testing and DevOps pipelines. Inspired by Arthur's article on the emerging "Token Tax," this conversation explores why many teams are underestimating the true cost of AI automation. You'll learn: Why AI-generated testing can create unexpected scaling costs How runaway AI agents and infinite loops happen Real-world examples of massive AI billing surprises Why deterministic problems shouldn't always use LLMs The hidden risks of "vibe testing" and autonomous AI remediation How QA teams can monitor, test, and control token usage Why performance testing and service virtualization matter more than ever in AI systems Practical strategies to avoid expensive AI deployment mistakes Whether you're a software tester, automation engineer,…

AI-powered testing tools promise faster automation and less maintenance, but most require teams to abandon their existing frameworks. In this episode, we explore Alumnium, an opensource AI-native end-to-end testing solution created by Alex Rodionov, an engineer at Airbnb and a tech lead on the Selenium project. Instead of replacing tools like Playwright or Selenium, Alumnium adds an AI layer on top, helping teams: Reduce test maintenance by removing brittle locators Build more resilient, self-healing tests Write less code while increasing coverage Run tests across web and mobile using intent-based steps We also go beyond the hype and break down what actually matters for real teams: Why AI-driven tests can still become flaky The performance and cost tradeoffs of LLM-based execution What "context rot" is—and how it impacts long test runs How to safely introduce AI into existing test suites without rewriting everything Check it out now: https://testguild.me/alumAI

AI is changing how we build and test software, but most teams are still struggling to turn AI-generated tests into real production value. Use code TESTGUILD3 try for yourself free now for 3 months: https://links.testguild.com/Kobiton In this episode, we break down what actually works when it comes to AI-powered mobile test automation, especially when running tests on real devices not simulators from Claude. You'll learn: How teams are generating and running Appium tests using natural language in minutes Why AI-generated tests often fail—and how to avoid costly false positives The real impact of AI on test automation roles and responsibilities How real device testing exposes issues AI alone can't catch Practical ways to reduce test maintenance while increasing coverage We also explore a major shift happening right now: AI is making it easier to create tests—but dramatically increasing the volume of code and risk that needs to be validated. That means one thing: Testing isn't going away—it's becoming more…

AI coding tools are helping teams move faster than ever, but there's a hidden cost. In this episode, we break down new insights from a DevOps industry report revealing a growing "velocity paradox": teams are shipping more code, but experiencing more failures, rollbacks, and burnout. You'll discover why AI adoption is heavily skewed toward coding, but not testing, pipelines, or observability, and how that imbalance is creating fragile systems that break under pressure. More importantly, you'll learn what high-performing teams are doing differently to maintain quality while scaling speed. What You'll Discover: ✔️ Why AI is increasing deployment failures (and how to stop it) ✔️ The "velocity vs quality" trap hurting modern DevOps teams ✔️ How to reduce flaky tests and pipeline instability ✔️ Why observability and feature flags are now critical, not optional ✔️ Practical ways to improve your CI/CD pipeline for AI-driven development ✔️ The role of QA engineers in the age of AI (and why it's growing, not…

AI is changing how we build and test software, but most teams are struggling to turn that promise into real results. In this episode, we break down what it actually takes to scale quality engineering across global teams without creating bottlenecks, burnout, or broken processes. You'll learn: why most test automation and transformation initiatives fail how to separate AI hype from reality what high-performing teams are doing differently to ship faster with confidence Today's expert, Sunita McCoy, a Global Engineering Leader and Transformation Specialist, shares practical insights from leading large-scale engineering transformations, including: how to build a culture that supports AI adoption why "quality as a phase" is dead how to shift toward treating quality as a product If you're a QA leader, automation engineer, or DevOps professional trying to improve reliability, reduce risk, and future-proof your skills in the age of AI, this episode gives you a clear path forward.

Mobile test automation is still one of the biggest bottlenecks in modern software delivery. In this interview, QApilot's Co-founder Aditya Challa explains why most AI testing approaches fail and how to fix them. Learn more about QApilot: https://links.testguild.com/flutterqa If your mobile tests are flaky, slow, or hard to trust, you're not alone. Most teams are trying to apply LLM-based AI to problems that actually require deterministic reliability—and that's where things break down. In this video, you'll learn: Why mobile test automation breaks at scale The real issue with "99% accurate" AI in testing LLMs vs deterministic AI (and why it matters for mobile apps) How flaky tests destroy confidence in your pipeline How QApilot approaches mobile testing differently What reliable, scalable mobile automation should look like What this means for you: Fewer false positives, faster releases, and mobile tests you can actually trust. 00:00 Why Mobile Test Automation Is Still Broken 01:10 QApilot Overview 01:51 Why…

Are you the only tester on your team—and expected to ensure quality across everything? In this episode, we break down the growing challenge of solo QA testing in the age of AI-driven development—where code is generated faster than ever, but confidence hasn't caught up. Christine Pinto shares real-world insights from her experience as a solo tester and now as a founder building tools designed to help testers reduce risk, collaborate better, and make smarter release decisions. You'll learn: Why "all tests passing" doesn't mean your product is safe The hidden risks of AI-generated code and test automation How to shift from test coverage to risk-based testing Practical ways solo testers can avoid burnout and isolation How to bring collaboration back into QA—even if you're the only tester Why better requirements still matter more than better AI

What if your production logs could automatically generate new test cases? In this episode, Joe Colantonio sits down with Tanvi Mittal to break down how AI-powered log mining is changing the way teams approach software testing, quality engineering, and DevOps. Most teams ignore production logs or use them only for debugging. But those logs contain real user behavior, real failures, and real edge cases—the exact scenarios your test suite is probably missing. 👉 Learn how to: Convert production logs into automated regression tests Use AI to detect real-world failure patterns Apply shift-right testing to catch bugs earlier (and smarter) Handle the challenge of testing non-deterministic AI systems Reduce flaky tests and automation debt with real data If you're working with Playwright, Selenium, Cypress, or AI-driven testing tools, this episode will give you a completely new way to think about test coverage.

How do you ensure software quality when the system you're testing doesn't give the same output twice? Go to https://links.testguild.com/inflectra and start your free 30-day trial, no credit card, no contract required. That's the core challenge facing every QA team building or testing AI-powered applications today and it's breaking all the rules we've relied on for decades. In this episode of the TestGuild Automation Podcast, I sit down with Adam Sandman, co-founder of Inflectra, to get into what non-deterministic AI testing actually means in practice, why traditional pass/fail testing no longer cuts it, and what quality professionals need to do differently right now. We cover: Why AI-generated code is raising the stakes for QA teams while budgets stay flat The fundamental difference between deterministic and non-deterministic systems — and why it changes everything about how you test How to set acceptable risk thresholds for AI systems (hint: it depends on whether you're building an e-commerce chatbot or an…

What does it really take to build a test automation tool that millions of testers rely on, without venture capital, paid ads, or a massive team? In this episode, we explore how SelectorsHub grew into one of the most widely used productivity tools in software testing, reaching over 1.6 million testers worldwide. You'll discover: How to build test automation tools that solve real QA pain Why community-driven development beats chasing funding How to prioritize features when you have thousands of users Whether AI testing tools will replace selector-based automation How to choose between Playwright vs Selenium using automation analysis What founders and QA leaders can learn from scaling without VC If you're an automation engineer, QA lead, DevOps professional, or tool builder looking to scale smarter, this episode delivers real-world insight without hype. Whether you're building frameworks internally or launching your own automation product, you'll walk away with a clearer strategy for solving problems testers…

AI test automation is evolving fast — but most tools still generate brittle code that breaks with every UI change. See it for yourself now: https://links.testguild.com/Thunders In this episode of the TestGuild Podcast, Joe Colantonio sits down with Karim Jouini, founder of Thunders, to explore a radically different approach to AI testing: executing test automation in plain English without generating Selenium or Playwright code. Instead of "auto-healing selectors," Thunders interprets natural language directly — allowing teams to: Ship twice as fast Achieve 10x test coverage with the same resources Reduce regression cycles from weeks to days Eliminate massive automation maintenance overhead Karim shares real-world case studies, including: A European bank that reduced a 3-year core banking upgrade testing effort to 4 months A SaaS company that transitioned from a traditional QA team to AI-assisted product-led testing We also discuss: Whether AI test agents replace QA roles How QA managers must shift from…

Is traditional performance testing becoming obsolete? In this episode, performance engineering expert Akash Thakur shares why AI is fundamentally transforming load testing, scripting, observability, and shift-left strategies. With 17 years of real-world enterprise experience, Akash explains how AI-augmented tools are already reducing scripting time by 30%, improving analysis speed, and helping teams move from reactive performance testing to predictive intelligence. You'll learn: How AI is accelerating performance scripting and analysis Why shift-left performance testing is finally becoming realistic The role of structured data in predictive QA models How to test AI applications (LLMs, GPUs, inference throughput) differently than traditional web apps What the future role of performance engineers looks like — architect, not script writer If you're a performance tester, SRE, QA leader, or DevOps engineer wondering how AI will impact your role — this episode gives you practical, actionable insights you can apply…

Most teams find defects after the damage is done — during regression, late-stage testing, or production incidents. That's expensive, stressful, and completely avoidable. Try Spec2Test AI now: https://testguild.me/spec2testdemo In this episode, Joe Colantonio sits down with Missy Trumpler, CEO of AgileAILabs, to explore how Spec2TestAI helps teams prevent defects before code ships by applying AI directly to requirements. You'll learn: Why traditional test automation still misses critical risk How predictive, requirements-based AI testing works in practice What "shift-left" actually looks like beyond the buzzword How to reduce escaped defects without writing more tests Why secure, explainable AI matters for QA and enterprise teams This conversation is especially valuable for software testers, automation engineers, and QA leaders who want earlier visibility into risk, faster feedback, and higher confidence releases. Don't miss Automation Guild 2026 - Register Now: https://testguild.me/podag26

Performance testing often fails for one simple reason: teams can't see where the slowdown actually happens. In this episode, we explore Locust load testing and why Python-based performance testing is becoming the go-to choice for modern DevOps, QA, and SRE teams. You'll learn how Locust enables highly realistic user behavior, massive concurrency, and distributed load testing — without the overhead of traditional enterprise tools. We also dive into: Why Python works so well for AI-assisted load testing How Locust fits naturally into CI/CD and GitHub Actions The real difference between load testing vs performance testing How observability and end-to-end tracing eliminate guesswork Common performance testing mistakes even experienced teams make Whether you're a software tester, automation engineer, or QA leader looking to shift-left performance testing, this conversation will help you design smarter tests and catch scalability issues before your users do.

AI testing is everywhere — but clarity isn't. In this episode, Joe Colantonio breaks down the real test automation trends for 2026, based on data from 40,000+ testers, 510 live Q&A questions, and 50+ interviews with industry leaders. This isn't vendor hype or futuristic speculation. It's what working testers are actually worried about — and what they're doing next. You'll learn: Why 72.8% of testers prioritize AI, yet don't trust it alone The real reason AI testing feels harder instead of easier How integration chaos is blocking automation success Why "AI auditor" and "quality strategist" are emerging career paths What agentic AI, MCPs, and vibe testing really mean in practice How compliance, accessibility, and security will redefine QA in 2026 If you're a tester, automation engineer, or QA leader trying to stay relevant — this episode gives you the signal through the noise, and a clear path forward. If you're a software tester, automation engineer, or QA leader looking ahead to 2026, this episode lays out…

This is a special end-of-year episode of the Automation Testing Podcast. With family in town and a busy holiday season, Joe didn't want to skip a week without checking in and saying thank you to the TestGuild community. In this short episode, Joe shares: A huge milestone as the podcast approaches its 13-year anniversary Why the Automation Testing Podcast is moving from Sundays to Tuesdays starting in 2026 How loyal listeners can still get $100 off a full 5-day Automation Guild 2026 pass A sneak peek at TestGuild IRL — live, in-person events coming next year Gratitude for the listeners, YouTube community, and sponsors who make TestGuild possible If you're a software tester, automation engineer, or QA leader looking ahead to 2026, this episode lays out what's coming — and how to stay connected. Discount code: 100GUILDCOIN (https://testguild.me/podag26) Questions or ideas? Email Joe directly at joe@testguild.com As always — test everything, and keep the good.

AI is transforming how software is built, but testing AI systems requires an entirely new mindset. Don't miss AutomationGuild 2026 - Register Now: https://testguild.me/podag26 Use code TestGuildPod20 to get 20% off your ticket. In this episode, Joe Colantonio sits down with Imran Ali to break down what AI testing really looks like when you're dealing with LLMs, RAG pipelines, and autonomous QA workflows. You'll learn: Why traditional pass/fail testing breaks down with LLMs How to test non-deterministic AI outputs for consistency and accuracy Practical techniques for detecting hallucinations, grounding issues, and prompt injection risks How RAG systems change the way testers validate AI-powered applications Where AI delivers quick wins today—and where human validation still matters This conversation goes beyond hype and gets into real-world AI testing strategies QA teams are using right now to keep up with AI-generated code, faster release cycles, and DevOps velocity. If you're a tester, automation engineer,…

What if understanding your codebase was no longer a blocker for great testing? Most testers were trained to work around the code — clicking through UIs, guessing selectors, and relying on outdated docs or developer explanations. In this episode, Playwright expert Ben Fellows flip that model on its head. Using AI tools like Cursor, testers can now explore the codebase directly — asking questions, uncovering APIs, understanding data relationships, and spotting risk before a single test is written. This isn't about becoming a developer. It's about using AI to finally see how the system really works — and using that insight to test smarter, earlier, and with far more confidence. If you've ever joined a new team, inherited a legacy app, or struggled to understand what really changed in a release, this episode is for you. Registration for Automation Guild 2026 Now: https://testguild.me/podag26

Performance testing has traditionally been one of the hardest parts of QA,slow onboarding, complex scripting, difficult debugging, and too many late-stage surprises. Try Gatling Studio for yourself now: https://links.testguild.com/gatling In this episode, Joe sits down with Stéphane Landelle, creator of Gatling, and Shaun Brown to explore how Gatling is reinventing the load-testing experience. You'll hear how Gatling evolved from a developer-first framework into a far more accessible platform that supports Java, Kotlin, JavaScript/TypeScript, and AI-assisted creation. We break down the thinking behind Gatling Studio, a new companion tool designed to make recording, filtering, correlating, and debugging performance tests dramatically easier. Whether you're a developer, SDET, or automation engineer, you'll learn: How to onboard quickly into performance testing—even without deep expertise Why Gatling Studio offers a smoother way to record traffic and craft tests Where AI is already improving load test authoring…

Manual regression testing isn't going away—yet most teams still struggle with deciding what actually needs to be retested in fast release cycles. See how AI can help your manual testing now: https://testguild.me/parasoftai In this episode, we explore how Parasoft's Test Impact Analysis helps QA teams run fewer tests while improving confidence, coverage, and release velocity. Wilhelm Haaker (Director of Solution Engineering) and Daniel Garay (Director of QA) join Joe to unpack how code-level insights and real coverage data eliminate guesswork during regression cycles. They walk through how Parasoft CTP identifies exactly which manual or automated tests are impacted by code changes—and how teams use this to reduce risk, shrink regression time, and avoid redundant testing. What You'll Learn: Why manual regression remains a huge bottleneck in modern DevOps How Test Impact Analysis reveals the exact tests affected by code changes How code coverage + impact analysis reduce risk without expanding the test suite…

Automation Guild turns 10 this year, and the 2026 survey revealed some of the strongest trends and signals the testing community has ever shared. Register now: https://testgld.link/ag26reg In this episode, Joe breaks down the most important insights shaping Automation Guild 2026 and what they mean for testers, automation engineers, and QA leaders. You'll hear why AI-powered testing is dominating every category, why Playwright has officially become the tool testers want most, the challenges that continue to follow teams year after year, and how testers are navigating shrinking teams, faster releases, and rising expectations. This episode gives you a clear, data-driven snapshot of why Automation Guild 2026 matters — and how this year's event is designed to help you stay relevant, sharpen your skills, and tackle the problems that keep slowing down teams. Perfect for anyone considering joining the Guild, planning their 2026 automation strategy, or just trying to make sense of the rapid changes happening in…

AI is accelerating software delivery, but it's also introducing new security risks that most developers and automation engineers never see coming. In this episode, we explore how AI-generated code can embed vulnerabilities by default, how "vibe coding" is reshaping developer workflows, and what teams must do to secure their pipelines before bad code reaches production. You'll learn how to prompt more securely, how guardrails can stop vulnerabilities at generation time, how to prioritize real risks instead of false positives, and how AI can be used to protect your applications just as effectively as attackers use it to exploit them. Whether you're using Cursor, Copilot, Playwright MCP, or any AI tool in your automation workflow, this conversation gives you a clear roadmap for staying ahead of AI-driven vulnerabilities — without slowing down delivery. Featuring Sarit Tager, VP of Product for Application Security at Palo Alto Networks, who reveals real-world insights on securing AI-generated code, understanding…

In this solo episode, Joe Colantonio shares four powerful free TestGuild tools designed to help testers, automation engineers, and QA leaders work smarter. Discover how to instantly find the right testing tool for your team, assess automation risk, check your site's accessibility, and benchmark your automation maturity — all in one session. Whether you're looking to improve test coverage, adopt better practices, or simply save time, these tools were built with you in mind. What You'll Learn: – How to choose the right test automation tool fast – How to identify and reduce testing risk – How to check your site's accessibility compliance – How to assess your team's automation maturity level Try the tools free: Tool Matcher: https://testgld.link/toolmatcher Accessibility Scanner: https://testgld.link/scanner Risk Calc: https://testgld.link/riskcalc Automation Readiness Quiz: https://testgld.link/scorequiz ️ Join us for the 10th Annual Automation Guild Conference: https://testgld.link/IrHaNIVX

As AI tools like Copilot, Claude, and Cursor start writing more of our code, the biggest challenge isn't generating software — it's trusting it. In this episode, JP (Jayaprabhakar) Kadarkarai, founder of FizzBee, joins Joe Colantonio to explore how autonomous, model-based testing can validate AI-generated software automatically and help teams ship with confidence. FizzBee uses a unique approach that connects design, code, and behavior into one continuous feedback loop — automatically testing for concurrency issues and validating that your implementation matches your intent. You'll discover: Why AI-generated code can't be trusted without validation How model-based testing works and why it's crucial for AI-driven development The difference between example-based and property-based testing How FizzBee detects concurrency bugs without intrusive tracing Why autonomous testing is becoming mandatory for the AI era Whether you're a software tester, DevOps engineer, or automation architect, this conversation will…

In this Halloween special, Joe Colantonio and Paul Grossman discuss the evolution of automation testing, focusing on the integration of AI tools, project management strategies, and the importance of custom logging. Paul shares insights from his recent job experience, detailing how he inherited a project and the challenges he faced. Paul also goes over his Optimus Prime framework and uses it to explore various automation tools, the significance of dynamic waiting, and how to handle test case collisions. The discussion also highlights the role of AI in enhancing automation frameworks and the importance of version control in software development.

Flaky Playwright tests got you down? Discover Vibe Testing, a new AI-driven approach that lets Playwright tests understand design intent, adapt to UI changes, and self-heal intelligently. In this episode, Joe Colantonio talks with Vasusen Patil, Co-Founder and CEO of Donobu, about how their platform extends Playwright with AI-powered "Vibe Testing." You'll discover how this approach blends visual assertions with contextual understanding to build resilient, low-flake tests that keep shipping smooth. You'll take away: What "Vibe Testing" really means and why it's a game-changer How AI-authored Playwright tests can self-heal without false positives The key to balancing autonomy with tester control Why Donobu's local-first model keeps your data safe while cutting test flakiness under 2 % How to try Donobu's free Playwright AI toolkit If you want to see where test automation is heading next — and how to future-proof your QA career — don't miss this one.

Did you know that Playwright offers an elegant, unified framework that seamlessly integrates both UI and API testing within a single language and test runner? Don't miss the early bird Automation Guild discount: https://testguild.me/ag26early This episode explores how Playwright empowers teams to simplify test maintenance, eliminate silos between dev and QA, and gain true full-stack confidence. You'll discover: How to make your tests 10x faster and more reliable by using API requests for setup instead of brittle UI flows. How to write hybrid tests that validate both UI actions and backend APIs in a single flow. A modern, unified testing strategy that reduces operational friction and helps teams deliver high-quality applications with confidence. Our guest, Naeem Malik, brings 15 years of QA and automation expertise. As the creator of Test Automation TV and bestselling Udemy courses, Naeem specializes in making complex test automation concepts simple, practical, and impactful for engineering teams. Whether…

Are your tests falling behind your requirements? Check out our new AI Requirement course here: https://testguild.me/aireqcourse In this special highlight episode, Joe Colantonio shares lessons from a new course created with Keysight: Turning Requirements into Reliable Tests with AI-Augmented Design. You'll hear from AI strategist Noelle Russell, Keysight's Chief AI Officer Jonathon Wright, and product expert Ian Sharp as they explore: Why responsible AI is like raising a baby tiger — exciting but dangerous if ignored. The hidden costs of manual test design and why happy-path testing isn't enough. How AI-powered tools like Keysight Generator transform natural language requirements into reliable, traceable test cases in minutes. If you're a tester, automation engineer, or QA leader looking to cut weeks of manual scripting, improve coverage, and finally align tests with real requirements — this episode is for you. Check out the full Keysight course here: https://testguild.me/aireqcourse Become one of our…

Scaling CI/CD for mobile apps is hard. Faster test runs often lead to more tests, more infrastructure, and more complexity. So how do you keep your pipelines healthy and reliable while still shipping at speed? In this episode, we sit down with Anton Malinski of Marathon Labs to explore the real-world lessons he's learned building and optimizing mobile CI/CD pipelines. You'll discover: How to scale mobile test automation without introducing friction What "healthy CI growth" looks like in practice Why real devices still matter, even with a massive emulator fleet How backend mocking and dedicated mobile API gateways transform shift-left testing Practical advice for teams evolving from weekly releases to on-every-commit confidence Whether you're a QA leader, automation engineer, or DevOps practitioner, this conversation gives you the insights and metrics you need to take your mobile testing pipelines to the next level.

In this episode of the TestGuild Automation Podcast, Joe Colantonio sits down with Jason Huggins—the original creator of early versions Selenium, Appium, and Sauce Labs—to discuss his latest project: Vibium. Not sure what tool to use? Try my free Test Tool Matcher now: https://testguild.me/toolmatch Jason describes Vibium as "Selenium for AI," designed to fix flaky WebDriver tests and leverage AI for smarter, model-based testing. We explore why AI-driven automation is the next wave, how WebDriver BiDi changes the game, and what QA leaders need to know about the rise of "vibe coding." If you've been wondering what comes after Selenium and Playwright, this is a must-listen. Get insider insights from the creator who helped shape modern testing—and is now building its AI-powered future.