Backend Boilerplates python Intermediate New

CSV to PostgreSQL Bulk Loader

Stream a large CSV directly into PostgreSQL using COPY for maximum throughput.

python
import psycopg2, csv, io

def csv_to_postgres(filepath: str, table: str, conn_str: str):
    with open(filepath, newline="") as f:
        reader = csv.reader(f)
        headers = next(reader)
        buf = io.StringIO()
        writer = csv.writer(buf)
        for row in reader:
            writer.writerow(row)
        buf.seek(0)

    with psycopg2.connect(conn_str) as conn:
        with conn.cursor() as cur:
            cur.copy_expert(
                f"COPY {table} ({",".join(headers)}) FROM STDIN WITH CSV",
                buf,
            )
        conn.commit()
        print(f"Loaded into {table}")

Prerequisites

  • psycopg2-binary >= 2.9
  • PostgreSQL

Pro Tips

  • → COPY is 10-100x faster than row-by-row INSERT for bulk loads.

Was this module useful?

Share your experience — it helps the team improve.

Leave Feedback

Commercial Licensing

Acquire a production access token or template license for enterprise deployment.

Module Info

Language
python
Level
Intermediate
Read time
3 min
Published
Aug 2026
License
Commercial License

Newsletter

New modules every week, directly to your inbox.