Menu

#Vocabulary

32 posts

Feed·
20 of 32 posts
Tokenization: BPE from Scratch, Vocab Size, and What Tokenizers Hide
📰
0Read More

Tokenization: BPE from Scratch, Vocab Size, and What Tokenizers Hide

DEV Community·Muhammad Saud·6 days ago
#UbpkJzSK
#dev#tokens#plus#model#token

We built BPE from scratch on 83,159 words and measured it. Urdu costs 3.75x the tokens of English, and vocab size stops paying off after 5,000 merges.

15s
0Read More

I created a free guide with 300 essential Russian words for beginners

Hi everyone! I've put together a Russian vocabulary guide for beginners who want to build a solid foundation in everyday Russian. It includes 300 essential words divided into 15 practical topics, such as family, food, home, daily routines, transport, and…

15s
Practice Japanese On An Inkplate 4Tempera EInk Device
🖼️
0Read More

Practice Japanese On An Inkplate 4Tempera EInk Device

#hackaday#cards#learning#language#vocabulary

Learning a new natural language can be a daunting proposition at the best of times, with repetition and immersion generally considered to be the best ways to build up not only vocabulary but also t…

15s
676Read More

The load-bearing vocabulary of Claude

louisabraham.github.io·3 months ago
#uVAjLpCF
#stack#claude#analysis#vocabulary#scrolling

const DATA = "analysis.js"; let A = null, T = null, V = null; const HALFROW = 14; // half a row: how far the line sits above the mark let LINE = 0.33; // and the mark, as a fraction of the box: `--line` const THUMB = 8

15s