Menu

Five Months of Silent Data Loss: Monitor Data Age, Not Containers
📰

Five Months of Silent Data Loss: Monitor Data Age, Not Containers

DEV Community: docker·Salah·about 12 hours ago
#yRSXjQYn
#devtodocker#devto#dev#version#running
Reading 0:00
15s threshold

I design cloud solutions for connected products. Data collection is the part I think about most at work. So it was a bad surprise to find that my own telemetry stack had collected nothing for five months.

Nothing looked broken. The containers were up. The web interface answered. Grafana drew its graphs. The database had not received a new row since 12 June.

In this article I'll explain how a stack stays healthy and idle at the same time, why none of my checks caught it, and what I check now.

The setup

I self-host TeslaMate , an open source logger for Tesla cars. It polls the car's API and stores drives, charges and positions. It runs in Docker on a small VPS: the app, a PostgreSQL database, an MQTT broker, and Grafana for the dashboards. It had been collecting since March without a problem.

How I found out

Not from an alert.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.