AI is changing how software is built and how software behaves. Quality Engineering needs to evolve with it.
AI-assisted development is increasing the speed at which software can be designed, built and released. At the same time, AI is becoming part of the products themselves — introducing new challenges around model behaviour, data, outputs, reliability and security.
Anchor Quality helps engineering teams adapt their Quality Engineering practices to this changing environment — combining proven testing and automation principles with a pragmatic approach to AI-assisted and AI-enabled software.
How do you maintain quality when software is being developed faster with AI?
AI is no longer confined to experimental projects. It is increasingly influencing how software is designed, developed, tested and operated — while AI itself is becoming part of the software that organisations deliver to customers.
Anchor Quality believes both require a modern Quality Engineering approach.
AI is changing the quality equation
That creates two related Quality Engineering challenges.
How do you test software when some of its behaviour is being determined by AI models rather than traditional deterministic code?
Two ways AI is changing Quality Engineering
Quality Engineering for AI-assisted development
AI can dramatically increase development velocity. Developers can generate code, tests, documentation and technical solutions faster than before. But faster code generation does not automatically produce higher-quality software.
AI-assisted development
AI coding assistants and generative AI tools can dramatically increase the speed at which engineers produce software. That creates greater demand for:
- meaningful automated test coverage
- reliable regression testing
- testability
- quality gates
- continuous feedback
- maintainable automation
- engineering practices that keep quality aligned with development velocity
The objective isn't to slow AI-assisted development down. It's to build enough confidence to move at its new speed.
AI-enabled products
When AI becomes part of the product itself, traditional software testing alone may not be sufficient. Teams may need to consider:
- model behaviour
- hallucinations
- inconsistent or non-deterministic outputs
- prompt and input variation
- output quality
- model and prompt regression
- data quality
- data contamination or poisoning
- adversarial inputs
- prompt injection
- inappropriate or unsafe outputs
- monitoring and evaluation
AI changes not only how software is built, but what it means to test software effectively.
More code, faster
AI can increase the amount of code produced by engineering teams. Automated coverage and meaningful quality gates become increasingly important as change accelerates.
AI-generated code still needs engineering judgement
Generated code can contain defects, edge cases, security issues or maintainability problems. Quality Engineering remains responsible for establishing confidence in what reaches production.
AI-generated tests need evaluation
AI can produce tests quickly, but generated tests still need to be assessed for relevance, coverage, reliability and maintainability.
Testability matters more
As development velocity increases, poorly testable architecture can become an even greater constraint. Quality needs to influence design and development rather than simply validate completed features.
Quality signals need to keep pace
Faster release cycles require fast, reliable feedback. Teams need meaningful signals rather than increasingly large manual regression cycles.
QA roles continue to evolve
Quality Engineering increasingly involves influencing architecture, testability, automation, delivery pipelines and engineering practices — not simply executing tests after development is complete.
Testing software that uses AI
Moving from deterministic code to probabilistic AI models changes how we establish confidence. Traditional testing relies on 'given X, expect Y'. AI testing requires a move toward 'given X, expect a distribution of Y within acceptable boundaries'. That doesn't make testing impossible. It means Quality Engineering needs to use different techniques to establish confidence.
Model behaviour & hallucinations
AI models will, by definition, produce incorrect or nonsensical outputs. Quality Engineering focuses not on zero-hallucination (which is mathematically improbable), but on identifying, measuring and controlling unacceptable behaviour through consistent evaluation.
AI output evaluation
Automated evaluation of AI responses involves: Semantic similarity checks, LLM-as-a-judge patterns, and strict validation of structured output formats (like JSON) against expected schemas and business rules.
Prompt & input testing
Prompts are the new source code. Confident releases require: Prompt sensitivity analysis, variation testing across different model versions, and testing for boundary cases in natural language inputs.
Prompt injection & AI security
Testing for adversarial inputs, prompt injection, and data leakage. Quality Engineering works alongside security engineering to establish safety boundaries and protect user data from malicious manipulation.
Data quality & data poisoning
The quality of the model is determined by the quality and integrity of the data. We address data contamination risks and bias detection as part of an overall quality strategy, not standalone data science services.
Model, prompt & data regression
Changes to the underlying model, the system prompt, or the RAG data source can all cause regressions. Establishing high-speed regression suites across all three variables is critical for delivery reliability.
Code change
Prompt change
Model change
Data change
Behaviour change
All can affect the quality of the product.
AI won't eliminate the need for Quality Engineering. It changes what Quality Engineering needs to do.
Quality Engineering isn't just about finding bugs. It is about building and maintaining the systems that allow engineering teams to release software with confidence. In an AI-augmented world, the goal remains the same. The methods, however, must evolve to address new speeds of development and new categories of product risk.
It's to build confidence at the speed your engineering team now operates.
AI changes the environment. The fundamentals of good Quality Engineering still matter.
While AI introduces new variables, the foundations of a reliable product remain the same. Our existing capabilities in QA Strategy, Quality Engineering, Test Automation, Testability, Regression Strategy, Delivery Reliability, CI/CD quality gates, Automation frameworks, and Risk-based testing only become more critical when building with AI.
Our approach to AI & Quality Engineering
Quality Engineering doesn’t change its objective because of AI. It changes its techniques. Anchor Quality works across four pillars to help teams release AI-powered software with confidence.
Quality Strategy
Aligning your testing, tools, and engineering culture to the speed of AI-assisted development.
Test & Evaluation Strategy
Using LLM-as-a-judge, model benchmarks, and robust evaluation suites to measure AI performance.
Intelligent Automation
Building resilient CI/CD quality gates that handle both deterministic code and probabilistic AI outputs.
Quality at Speed
Ensuring that the increase in development velocity doesn’t lead to a decrease in release stability.
Where Anchor Quality can help
Assess
Benchmarking your team’s current use of AI tools and the testability of your AI-enabled features.
Strategise
Developing a pragmatic roadmap for Quality Engineering that balances AI speed with product reliability.
Build
Hands-on implementation of evaluation suites, regression gates and intelligent automation frameworks.
Advise
Senior QA leadership to guide complex decisions around model testing, prompt management and risk.
