Menu

pg_ilib: Compact Typed Binary Serialization for PostgreSQL
πŸ“°

pg_ilib: Compact Typed Binary Serialization for PostgreSQL

DEV CommunityΒ·EugeneΒ·6 months ago
#b8yQLJ0y
#postgres#database#opensource#select#type
Reading 0:00
15s threshold

The problem

PostgreSQL's bytea type is powerful for storing raw binary data, but it carries no type information. Once you store a value as bytes, you need out-of-band metadata to know whether those bytes represent a number, a UUID, a timestamp, or a JSON object.

This gets painful when you're building dynamic schemas β€” EAV tables, schemaless document stores, or audit logs β€” where a single column holds values of different types. You end up carrying a separate type column everywhere, writing CASE expressions to decode it, and hoping they stay in sync.

pg_ilib solves this with a simple idea: prefix every serialized value with a 2-byte typed header.

The format

Byte 0: [ op_id (4 bits) | params_hi (4 bits) ] Byte 1: [ params_lo (8 bits) ] Bytes 2…N: payload

Enter fullscreen mode

Exit fullscreen mode

The op_id identifies the type.…

Continue reading β€” create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.