swadhinbiswas added:cookebook for this project · 8/1/2026
ce5d03e
18 lines · 480 Bytes
Raw Download

« Back to Index

ADR: Why Polars?

Context: We need a library to perform data transformations (ETL).

Options Considered: Pandas, PySpark, Polars.

Decision: Polars.

Rationale:

  • PySpark is overkill (too much overhead) for single-node processing.
  • Pandas is slow and memory-inefficient for large datasets.
  • Polars offers blazing fast multi-threaded execution and a clean, expressive API.

« Back to Index