kunishou/tpch_tables_scale_1

polars-tpch ===========

This repo contains the code used for performance evaluation of polars. The benchmarks are TPC-standardised queries and data designed to test the performance of "real" workflows.

From the TPC website:

TPC-H is a decision support benchmark. It consists of a suite of business-oriented ad hoc queries and concurrent data modifications. The queries and the data populating the database have been chosen to have broad industry-wide relevance. This benchmark illustrates decision support systems that examine large volumes of data, execute queries with a high degree of complexity, and give answers to critical business questions.

Generating TPC-H Data

Project setup

# clone this repository
git clone https://github.com/pola-rs/tpch.git
cd tpch/tpch-dbgen

# build tpch-dbgen
make

Notes:

• —For MacOS, the above make command will result in an error while compiling like below,

   bm_utils.c:71:10: fatal error: 'malloc.h' file not found
   #include <malloc.h>
            ^~~~~~~~~~
   1 error generated.
   make: *** [bm_utils.o] Error 1

To fix this, change the import statement #include <malloc.h> to #include <sys/malloc.h> in the files where error is reported (bm_utils.c and varsub.c) and then re-run the command make.

Execute

# change directory to the root of the repository
cd ../
./run.sh

This will do the following,

• —Create a new virtual environment with all required dependencies.
• —Generate data for benchmarks.
• —Run the benchmark suite.