About our project

Our project is an assistant that tells people which bin an item belongs in, and why. Some items are easy, like a plastic bottle. Others are tricky, like a greasy pizza box or a broken light bulb. We want the assistant to answer in a clear format, for example “Category: Recyclable. Reason: …”.

In my NLP class I built a next-word predictor using GPT, but I only used the basic version and never reached GPT-2. Researching this blog helped me see that next-word prediction is exactly what every modern LLM does, just at a much bigger scale. An LLM gives a probability to every possible next token and picks one, over and over, until it has a full answer.

Our team has no budget and uses the free Google Colab T4 GPU, which has 16 GB of memory. So I searched for open models of about 4 billion parameters or fewer.

The four models I looked at

Qwen3-4B-Instruct-2507 (Alibaba, Qwen team). It has 4 billion parameters and a very large context window of 262,144 tokens. The licence is Apache-2.0.…