Menu

#Node_ModuleS

4 posts

Feed·
4 of 4 posts
0Read More

Best way to clean GitHub data (remove node_modules, lockfiles, etc) for LLM fine-tuning?

Reddit r/datasets·u/Ok_Rub3312·7 months ago
#eEmnyF9f
#github#node_modules#lockfiles#fine#article

Anyone else wasting hours cleaning GitHub data for LLM fine-tuning? I tried building my own dataset (instead of relying on Hugging Face), but scraping repos is messy node_modules, lockfiles, minified code, binaries… tons of junk.…

15s