Menu

Idempotent Bulk Ingestion into NebulaGraph from .NET: Solving the Duplicate Problem at the VID Level
📰

Idempotent Bulk Ingestion into NebulaGraph from .NET: Solving the Duplicate Problem at the VID Level

DEV Community: dotnet·Mehdi Mohseni·16 days ago
#dw2R8vxk
#devtodotnet#devto#dev#edge#string
Reading 0:00
15s threshold

Why 'Duplicates' Mean Something Different in a Graph DB

If you come from a relational background, or from the ArangoDB migration covered earlier, your first instinct for duplicate prevention is a unique index or an ON CONFLICT DO NOTHING clause. NebulaGraph has no unique index to lean on, and that difference is behind a lot of ingestion bugs.

In NebulaGraph, vertices are keyed by VID (vertex identifier) and edges are keyed by (src_vid, dst_vid, edge_type, rank) . There are no uniqueness constraints apart from these keys. A plain INSERT VERTEX behaves like a key-value PUT : if you insert a vertex with an existing VID and tag, the later write overwrites the earlier one. INSERT VERTEX IF NOT EXISTS and INSERT EDGE IF NOT EXISTS do exist, but they compare only the key (VID and tag, or the edge key), not the properties, and the documentation warns that the extra read has a significant cost.

So idempotency is built into the storage model, but only if your VIDs are deterministic.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.