Menu

Building a Tiny GPT with C# and ILGPU — Powered by OpenAI Codex
📰

Building a Tiny GPT with C# and ILGPU — Powered by OpenAI Codex

DEV Community: csharp·Tingwei·about 4 hours ago
#ubx4AE5E
#devtocsharp#devto#dev#model#training
Long article (1557 words) — no AI summary yet.
Reading 0:00
15s threshold

GitHub: https://github.com/tingwei628/my-gpt

Introduction

I've been exploring how GPT models work internally, from basic mathematical operations to Transformer architecture and GPU acceleration.

Inspired by Andrej Karpathy's minGPT , I started my-gpt , an open-source educational project that implements a small, character-level GPT using C# and ILGPU .

One of the most interesting aspects of this project is how it was developed:

Most of the implementation was carried out by OpenAI Codex.

Rather than writing every component manually, I provided the requirements, hardware constraints, and feedback, while GPT through Codex handled much of the coding, educational documentation, training operations, and verification.

The goal was not to build a production-ready LLM, but to explore two questions:

Can we implement and train a small GPT using C# and GPU acceleration?

How far can an AI coding agent like OpenAI Codex take a technical project involving neural networks and GPU programming?

Why C# and…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.