From AI-Assisted Testing to Autonomous Quality Engineering: Agent-Based Investigation and Reproduction of CI Failures

Abstract

Incorporating quality issues early is achieved by modern software teams that use continuous integration. However, even today the most frequent CI failures cannot be automatically investigated due to the noisy nature of logs and the fact that failure causes are intermixed with other ones. The paper includes an agent-based framework that shifts quality engineering to than failure detection, to an autonomous investigation and reproduction. The framework assembles several AI agents in a regulated workflow, which includes the failure detection, data gathering, hypothesis formulation, environment examination, execution context reconstruction, reproduction, and resolution advice. The framework incorporates protection of constrained autonomy, evidence-based thinking, human escalation and reproducibility of agent actions. Experimental analysis of 420 cases of CI failure indicates an average diagnostic F1 result of 0.756 and a reproduction success rate of 75 percent as well as a 71 per cent reduction in time to conduct a manual investigation versus baseline level in manual baselines. The findings indicate agent-based systems can serve as engineering partners, which find reasons of CI failures but retain full traceability and human control.