Improving generated programs using large language models

Abstract

Generative AI and large language models have shown a great capacity to generate and modify programs according to the user's intent, expressed in natural language. In fact, most modern development environments include coding agents or tools based on these models. However, to what extent can programs generated or modified by such systems be considered reliable? In this paper, we present a framework to optimize programs using large language models and validate the transformed code by differential testing. We study the previous question across one specification language and several modern imperative programming languages, and integrate the approach to improve imperative code generated from Maude specifications in a recent work.