by Corneliu Cofaru , Adrian Iurov at 0x0α Research.

In this article we explore 1 how LLM-generated code can be reliably

used in data science experiments and workflows. The aim is to

illustrate the potential of combining language models with code and

data quality tools. We devise a simple regression experiment in which

a LLM generates code in R, starting from anonymized tabular data based

on the ’Ames housing’ dataset. The generated code is then executed and

interactively analyzed with a linter capable of running checks on both

data results and code simultaneously. Results from the linter are then

fed back into the LLM and new code is generated which in its turn

produces an improved model with a lower error than the original one.

The theory

LLMs are starting to be used in generating analysis code in data science

workflows with seemingly good results [1, 2].…