Why String Concatenation Can Kill Performance in Large Text Generation

Generating large configuration files efficiently is critical for infrastructure-as-code tools like Terraform. When Terraform's query -generate-config-out command produces huge HCL outputs, slowdowns and high memory usage can frustrate users and complicate automation pipelines. This performance bottleneck stemmed from a seemingly innocent pattern: repeated string concatenation.

The Problem with Repeated String Concatenation

In Go, strings are immutable. Each time you concatenate strings with the + operator or += , a new string is allocated and the contents copied over. This is fine for small or fixed-size strings but becomes problematic when building very large outputs incrementally.

Terraform’s internal/cloud/backend_query.go was accumulating generated HCL text for each resource address by repeatedly concatenating strings.…