Menu

What 27 merged PRs taught me about testing a Rust rewrite as a black box
📰

What 27 merged PRs taught me about testing a Rust rewrite as a black box

DEV Community: rust·Emir Mrkaljević·7 days ago
#p651fxC9
#devtorust#devto#dev#uutils#merged
Reading 0:00
15s threshold

I have been recording how GNU coreutils 9.12 behaves, then replaying the same cases against uutils , the Rust rewrite. The recorder is behave : a Go CLI (and MCP server) that probes flags from --help , odd stdin, missing files, and other corners, then diffs stdout, stderr and exit codes.

So far that work has led to about 30 bug reports and 37 fix PRs upstream. 27 of those PRs are merged. Here is what those merges taught me about black-box testing a rewrite, with concrete examples.

1. Match the reference release, not "whatever is on PATH"

CI on Ubuntu often ships an older GNU than the release the rewrite targets. I record from GNU 9.12 because that is what uutils aims at. A case that is correct on 9.12 can fail on 9.4 (for example how wc or expand counts a multi-byte character). Specs that depend on a version gap stay out of CI, or they teach people to ignore failures.

Lesson: pin the reference binary. Diffing against a random system coreutils produces noise that looks like product bugs.

2.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.