GitHub: https://github.com/tingwei628/my-gpt
Introduction
I've been exploring how GPT models work internally, from basic mathematical operations to Transformer architecture and GPU acceleration.
Inspired by Andrej Karpathy's minGPT , I started my-gpt , an open-source educational project that implements a small, character-level GPT using C# and ILGPU .
One of the most interesting aspects of this project is how it was developed:
Most of the implementation was carried out by OpenAI Codex.
Rather than writing every component manually, I provided the requirements, hardware constraints, and feedback, while GPT through Codex handled much of the coding, educational documentation, training operations, and verification.
The goal was not to build a production-ready LLM, but to explore two questions:
Can we implement and train a small GPT using C# and GPU acceleration?
How far can an AI coding agent like OpenAI Codex take a technical project involving neural networks and GPU programming?
Why C# and…