A quality framework for infrastructure as code in the modern DevSecOps era

Abstract

Infrastructure as Code (IaC) is transforming how organisations deploy and manage their infrastructure through version-controlled scripts. While this creates opportunities for automation, it also introduces quality and security concerns. Many IaC analysis tools focus on one or two platforms and rely on pattern-based detection, but do not take advantage of the information contained within repository metadata and the relationships between repositories. A survey of current IaC analysis tools reveals shortcomings in their ability to conduct comprehensive quality assessments, allowing security smells to propagate through dependency chains and code reuse patterns without being identified.

This thesis presents a systematic process for assessing the quality of IaC scripts. It introduces a nine-category framework that extends the principles of traditional software quality to infrastructure code. The categories include aspects such as metadata and platform-specific characteristics that were not addressed in previous works. This work also utilises policy-based weighted mechanisms to allow organisations to place their own emphasis on the various categories when conducting assessments. Building on this framework, the research assesses security smells by utilising metadata-assisted methods for detecting vulnerabilities in the supply chain of dependencies. Our findings show that vulnerable dependency chains affect multiple downstream repositories. For platforms that lack fork metadata, similarity-based methods are employed to identify the paths through which security smells diffuse. Clone detection proved effective at tracing these diffusion paths across such platforms. This work also demonstrates the applicability of the framework to the evaluation of AI-generated infrastructure code. The results showed that 15 out of the 16 tested AI models produced code that scored below human-written repositories. However, structured prompts with quality and compliance integration enabled the top-performing AI models to produce code that exceeded human-written implementations in meeting industry benchmarks for security. Together, these findings support the usefulness of the proposed framework for the integration of quality assurance into modern DevSecOps workflow practices.