Concepts / Creational patterns / Object pools

Object pools

Reusing expensive objects instead of making new ones.

Low-level design conceptCreational patterns2 Nov 2026

01The intuition

The problem it solves

A school portal makes a database connection for every page load. Opening a connection takes about 50 milliseconds and a handshake with the server; the query itself takes 2 milliseconds.

def students_page():
    conn = connect("db.school.local")     # 50 ms, every single time
    rows = conn.query("select * from students")
    conn.close()
    return render(rows)

Ninety-six percent of the time is spent building and tearing down something that could be reused. If 200 pages load at once, 200 connections open at once, and the database, which allows 50, refuses the rest. The program needs a small number of expensive objects, shared carefully.

An object pool keeps a set of ready-made objects. A caller borrows one, uses it, and returns it. The pool limits how many exist, waits when all are busy, and never hands the same object to two callers at once.

Build it one piece at a time

Step 1: keep made objects in a list and hand them out. If the list is empty, make a new one. This already removes most of the 50-millisecond cost.

class ConnectionPool:
    def __init__(self, make):
        self._make = make
        self._idle: list = []
    def acquire(self):
        return self._idle.pop() if self._idle else self._make()
    def release(self, conn) -> None:
        self._idle.append(conn)

Step 2: cap the total and block when all are busy. A counter tracks how many exist; a condition variable lets acquire wait until someone releases. This is where the pool becomes safe for many threads.

import threading

class ConnectionPool:
    def __init__(self, make, max_size: int):
        self._make, self._max = make, max_size
        self._idle: list = []
        self._total = 0
        self._cv = threading.Condition()
    def acquire(self):
        with self._cv:
            while not self._idle and self._total >= self._max:
                self._cv.wait()
            if self._idle:
                return self._idle.pop()
            self._total += 1
            return self._make()
    def release(self, conn) -> None:
        with self._cv:
            self._idle.append(conn)
            self._cv.notify()

Step 3: track who holds what, so a double release is a no-op. A set of borrowed objects makes the pool robust to a caller that releases twice or releases something it never borrowed.

class ConnectionPool:
    # fields as in step 2, plus self._busy: set[int] = set()
    def _new(self):
        self._total += 1
        return self._make()
    def acquire(self):
        with self._cv:
            while not self._idle and self._total >= self._max:
                self._cv.wait()
            conn = self._idle.pop() if self._idle else self._new()
            self._busy.add(id(conn))
            return conn
    def release(self, conn) -> None:
        with self._cv:
            if id(conn) not in self._busy:
                return                       # not ours, or already released
            self._busy.remove(id(conn))
            self._idle.append(conn)
            self._cv.notify()

Step 4: make borrowing impossible to forget. A context manager returns the object even when the caller's code raises. Add a health check on release so a broken connection is discarded, and a timeout on wait so a leak shows up as an error instead of a hang.

from contextlib import contextmanager

@contextmanager
def borrowed(pool):
    conn = pool.acquire()
    try:
        yield conn
    finally:
        pool.release(conn)

with borrowed(pool) as conn:
    rows = conn.query("select * from students")
StepConnections for 200 concurrent pagesCost per pageDouble releaseForgotten release
Start200, database refuses 15052 msn/an/a
1unbounded2 ms after warm-upduplicates in the poolleak
2at most 50, rest wait2 ms plus waitingduplicates in the poolleak, others wait forever
3 and 4at most 502 ms plus waitingignoredreturned by the context manager

Why it works

The pool turns a creation cost into a one-time cost and turns an unbounded resource into a bounded one. Its invariant is "every object is either idle in the pool or borrowed by exactly one caller, and idle plus borrowed never exceeds the cap". The condition variable is what lets waiting callers sleep instead of spinning, and the while around wait handles spurious wake-ups and the case where another thread grabbed the released object first.

Where it breaks: pooled objects that carry state between borrowers, such as a connection left inside a transaction, so reset on release; a make that fails inside the lock, so decrement _total on failure; and a cap that is too small for the workload, which shows up as every request waiting on acquire.

The trade-off you must say out loud

A pool is a small piece of concurrent code that is easy to get subtly wrong, and for cheap objects it is slower than just creating them. Say the numbers that justify it: creation cost versus use cost, and the external limit you must respect. The follow-up is usually about leaks, and the answer is leases with a timeout and a health check, plus a metric for "waiting callers" so the cap can be tuned.

Where it shows up in the 100 Google problems

02Used in these problems