DuckDB Smashes SQLite Speed Records in Data Performance Test

// The Benchmark Queries We tested each engine on the same four everyday analytical tasks: - Total transaction value: summing a numeric column - Group by domain: aggregating transaction counts per category - Filter by location: filtering rows by a condition before aggregation - Group by domain & location: multi-field aggregation with averages # Benchmark Results // Query 1: Total Transaction Value Here we measure how Pandas, DuckDB, and SQLite perform when summing the Value column across the dataset. // Pandas Performance We calculate the total transaction value using .sum() on the Value column. pandas_results = [] def pandas_q1(): return df['Value'].sum() mem_before = memory_usage(-1)[0] start = time.time() pandas_q1() end = time.time() mem_after = memory_usage(-1)[0] pandas_results.append({ "engine": "Pandas", "query": "Total transaction value", "time": round(end - start, 4), "memory": round(mem_after - mem_before, 4) }) pandas_results Here is the output.

// DuckDB Performance We calculate the total transaction value using a full-column aggregation. duckdb_results = [] def duckdb_q1(): return duckdb.query("SELECT SUM(value) FROM bank_data").to_df() mem_before = memory_usage(-1)[0] start = time.time() duckdb_q1() end = time.time() mem_after = memory_usage(-1)[0] duckdb_results.append({ "engine": "DuckDB", "query": "Total transaction value", "time": round(end - start, 4), "memory": round(mem_after - mem_before, 4) }) duckdb_results Here is the output.