Menu

A future revision should not change yesterday's data selection
📰

A future revision should not change yesterday's data selection

DEV Community: database·Muhammad Abdullah·2 days ago
#dX3sKIXM
#devtodatabase#devto#dev#value#available
Reading 0:00
15s threshold

An as-of join can run successfully and still answer the wrong historical question. The important timestamp is when a particular version became available, not just the period the value describes.

Suppose a synthetic September fact is published on October 3 with value 100, then revised on October 7 to 80. For an October 4 decision, the eligible value is 100. A pipeline that joins today's latest revision to every earlier date silently returns 80 instead.

There is a useful regression property here: adding a future revision must not change a past selection.

from datetime import datetime

def available ( value ): result = datetime . fromisoformat ( value ) if result .…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.