You read that correctly, Pandas should go extinct. Not the cute fluffy things used for international diplomacy , but the Python DataFrame library. Why? Because Pandas’ inefficiencies force you to adopt distributed querying systems before your workloads justify the added complexity. I posit that most workloads will never justify those systems, they are just well marketed “silver bullets”. To understand what I’m talking about we first must understand the typical adoption pathway for Pandas. Why do we use Pandas? The diagram below shows a rough guide of when you typically would consider adopting a given DataFrame library based on the data size you are working with. Following it from left to right, you also see the typical adoption pathway for data analysis tools, and the cliff that Pandas’ users experience beyond a certain data size. People typically start with Excel and graduate to Pandas somewhere in the GB range.…