In safety-critical systems, assurance matters more than productivity
Artificial intelligence is changing how quickly software can be developed. Tasks that once took weeks of engineering effort can now be completed in hours, sometimes minutes. In aviation, much of the conversation has focused on productivity, delivery schedules and cost.
For air traffic management (ATM), however, those may not be the most important questions. If AI can generate code, documentation and test artefacts at speed, what becomes the constraint on delivery? More importantly, how do we provide the assurance needed to introduce change safely into live operations?
The bottleneck moves
For air navigation service providers (ANSPs) and ATM suppliers, software development is only one part of a much larger process. A new capability does not create value simply because the code is complete. It must be integrated, verified, validated, safety-assessed, accepted and deployed into an operational environment.
AI can reduce the effort involved in creating software. It does not automatically reduce the effort needed to demonstrate that the software is safe, fit for purpose and ready for use.
In many programmes, verification, validation, safety assessment and regulatory acceptance already account for a substantial part of the delivery timeline. As development accelerates, these activities are likely to determine how quickly innovation can reach operations.
The issue is not whether assurance is slowing progress. It is whether assurance is equipped to deal with a greater volume of change, produced at a much faster pace.
Building it right, and building the right thing
The debate around AI often focuses on whether it can write software correctly. That matters, but it is only part of the question.
Verification asks whether a system has been built correctly against its specified requirements. Validation asks whether it is the right system for the operational need in the first place.
As AI makes it easier to generate and test solutions, the emphasis may shift further upstream: understanding the operational problem, defining clear requirements, making sound architectural choices and considering the consequences of those choices in the wider ATM environment.
This is where assurance delivers its greatest value. Before confirming that a system has been built correctly, organisations need confidence that they are solving the right problem for the right operational purpose.
Established approaches such as the V-cycle remain relevant. They provide a disciplined connection between operational need, requirements, design, implementation, verification and validation. That traceability becomes even more important when parts of the process can be generated at machine speed.
Automation can help, but it must not assure itself
The temptation is to automate assurance in the same way AI automates development. To some extent, this is already happening: AI can support test generation, requirements traceability, code reviews and static analysis. These tools can improve productivity and help teams manage growing volumes of change. But assurance is not just evidence generation. It relies on independent challenge.
If the same AI model generates requirements, code, tests and traceability evidence, apparent consistency may be misleading. Four artefacts agreeing with each other is reassuring only if they were produced independently. Otherwise, it is AI marking its own homework, repeating the same mistake with confidence.
Automated verification also checks only what it has been instructed to check. Safety-critical failures often arise from behaviours nobody anticipated when requirements or tests were defined. This is why integration testing, operational trials and long-duration soak testing remain essential: they expose the unknown unknowns.
And while automation accelerates productivity, it also accelerates the propagation of errors. Traditional checks and balances operate at human speed. AI operates at machine speed. Independent oversight therefore becomes even more important.
How roles will need to change
Software engineers already spend less of their time writing code line by line than they did a decade ago. Their value increasingly lies in understanding operational needs, designing architectures, integrating systems and making sound engineering decisions.
Verification and validation (V&V) specialists are heading in the same direction. The V&V engineer of the future will spend less time manually executing checks and more time designing assurance strategies, configuring automated pipelines, evaluating evidence and investigating anomalies that automated tools cannot explain.
Reviewing AI-generated content is becoming a specialist skill. AI can present incorrect information with confidence and sophistication, making errors harder to detect. Teams will need to understand the operational and regulatory context in which a system will be used. They will also need enough understanding of AI tools to question their outputs properly. AI-generated material can appear plausible and confident even where its reasoning is incomplete, unsupported or wrong. Identifying those weaknesses will be a core assurance skill.
Accountability still sits with people
This leads to a vital question: who is accountable when both the software and a significant proportion of its supporting assurance evidence have been generated by AI?
For the foreseeable future, approval will rest with organisations, processes and named decision-makers, not with an AI model. A person will remain accountable for safety-significant change.
That accountability must remain meaningful. There is a risk that people defer too readily to machine-generated recommendations, particularly when delivery pressure makes human review appear to be the slowest part of the process. Culture, process and oversight must evolve to ensure humans remain confident and empowered to challenge AI-generated artefacts, even when those artefacts are produced at scale.
The organisations best placed to benefit from AI will be those that develop robust assurance processes alongside it. They will be clear about where automation adds value, where independent evidence is required, and where human authority must be retained.
Industry activity: building on existing foundations
The good news is that we are not starting from scratch. Industry bodies including EUROCONTROL, CANSO, EUROCAE and EASA are already exploring how AI can be safely adopted within ATM. Through initiatives such as EUROCONTROL’s FLY AI programme, ongoing work on standards and guidance, and discussions on trustworthiness, certification and human-machine collaboration, the focus is shifting from what AI can do to how it can be used responsibly in a safety-critical environment.
We are actively contributing to these conversations through industry forums, working groups and collaborative projects with customers and regulators.
The assurance strategy
AI is already demonstrating that it can accelerate development. For ATM, the challenge is how assurance evolves to keep pace.
The future is unlikely to be a choice between human judgement and automation. More likely, it will be a combination of both: AI accelerating development and evidence generation, while skilled professionals design, govern, challenge and approve the process.
Developing a clear strategy for integrating AI into both engineering and assurance activities will be just as important as the technology itself. We are actively exploring these questions with customers, industry partners and regulators, helping organisations understand how to use AI while maintaining the levels of safety, assurance and accountability that ATM demands. Are you ready?
