Johnny Metz

Django Custom Managers Are Silently Leaking Data

Sun, 22 Mar 2026 00:12:00 -0700

Django custom managers are a common way to exclude rows by default, such as inactive or soft-deleted rows. However, they don’t run everywhere you’d expect, which leads to unintended data exposure. This post covers where that happens and how to fix it.

The Setup

Let’s model stores and products with a soft-deletable relationship:

class Store(models.Model):
    name = models.CharField(max_length=255)

class Product(models.Model):
    name = models.CharField(max_length=255)

class StoreProductManager(models.Manager):
    def get_queryset(self):
        return super().get_queryset().filter(active=True)

class StoreProduct(models.Model):
    store = models.ForeignKey(Store, on_delete=models.CASCADE)
    product = models.ForeignKey(Product, on_delete=models.CASCADE)
    active = models.BooleanField(default=True)
    created_at = models.DateTimeField(auto_now_add=True)

    objects = StoreProductManager()  # default: active only
    all_objects = models.Manager()   # escape hatch: everything

The custom manager is declared first, making it the default manager. That feels like it should protect us everywhere, but it doesn’t.

Django Time-Based Lookups: A Performance Trap

Mon, 19 Jan 2026 00:00:00 -0700

Django’s field lookups are one of the ORM’s best features, but time-based lookups can quietly bypass database indexes, turning fast queries into expensive full table scans.

A Slow Production Query

I ran into this while debugging a 30 second query on a large table (~25 million rows):

class Event(models.Model):
    timestamp = models.DateTimeField(db_index=True)

Event.objects.filter(timestamp__date=datetime.date(2026, 1, 5)).count()

It generates SQL like this:

SELECT COUNT(*) FROM event
WHERE timestamp::date='2026-01-05';

At first glance, it looked totally reasonable — a simple filter on an indexed field. But after checking the query plan, I discovered the issue: the query can’t use the index and falls back to a full table scan because it casts the field to a date.

Avoiding Duplicate Objects in Django Querysets

Sat, 03 Jan 2026 00:00:00 -0700

When filtering Django querysets across relationships, you can easily end up with duplicate objects in your results. This is a common gotcha that happens with both one-to-many (1:N) and many-to-many (N:N) relationships. Let’s explore why this happens and the best way to avoid it.

The Problem

When you filter a queryset by traversing a relationship, Django performs a SQL JOIN. If a parent object has multiple related objects that match your filter, the parent object appears multiple times in the result set.

7 Heroku Features Every Developer Should Be Using

Sat, 13 Sep 2025 12:00:00 -0600

Heroku makes hosting simple, but most developers overlook its advanced capabilities. After years of running apps on the platform, I’ve found several underrated features that speed up your workflow and make your apps more reliable. Here are seven you shouldn’t miss.

1. Preboot

Preboot spins up new dynos and starts routing traffic to them before the old ones are terminated. The result is seamless, zero-downtime deployments.

The tricky part is knowing when the cutover is complete, since the Heroku dashboard doesn’t show it. Use this command to confirm when the new dynos are live:

The Django db_default Gotcha That Breaks Your Code

Sun, 03 Aug 2025 13:46:58 -0700

Starting in Django 5.0, you can use db_default to define database-level default values for model fields. It’s a great feature but comes with a subtle and dangerous gotcha that can silently break your code before the object is ever saved.

The Gotcha in Action

Let’s say you’re building a simple task manager, and you want to track whether a task is completed:

from django.db import models

class Task(models.Model):
    is_completed = models.BooleanField(db_default=False)

The db_default option ensures the database applies the default in raw INSERT queries and prevents errors during a rolling deployment.

Stop Using Django's squashmigrations: There's a Better Way

Sat, 26 Jul 2025 17:00:00 -0700

Squashing merges multiple database migrations into a single consolidated file to speed up database setup and tidy up history. Django’s squashmigrations command promises to handle this, but it’s error-prone and unnecessarily complex. A clean reset is faster and simpler.

⚠️ Why `squashmigrations` Falls Short

Broken in Practice

In medium-to-large projects, I’ve consistently seen squashmigrations generate poor results, such as failing to optimize basic no-ops (like adding and deleting the same field) and even breaking migrations. The official docs acknowledge this:

Speed Up Django Queries with values() over only()

Sun, 29 Jun 2025 13:11:26 -0700

If your Django queries feel slow, the problem might not be your database — it might be your ORM. Recently, I was working with a query that took 25 seconds to run through the Django ORM, but the underlying SQL completed in just 2 seconds. With a single change, I got the ORM query down to 2 seconds as well — and reduced the memory footprint by 70%.

Let’s walk through what happened, and why using .values() instead of .only() can dramatically improve performance.

Avoiding PostgreSQL Pitfalls: The Hidden Cost of Failing Inserts

Sun, 15 Jun 2025 18:00:00 -0700

A simple insert query turned into a silent performance killer.

Our frontend pings our server every few minutes to track device activity. Each ping attempts to insert a row into a DevicePingDaily table, which has a unique constraint on (device_id, date) to ensure only one record per device per day.

In Django, the logic looked like this:

try:
    DevicePingDaily.objects.create(device=device, date=today)
except IntegrityError:
    pass

It seemed harmless. But as traffic grew, latency spiked and API timeouts increased. Observability tools quickly pointed to the culprit:

Disable Redundant Gunicorn Access Logs on Heroku

Mon, 25 Nov 2024 12:00:00 -0800

When hosting an application on Heroku, managing logs efficiently is crucial for maintaining system health and keeping costs down. Heroku provides built-in logging for all incoming requests, but by default, Gunicorn, the Python HTTP server often used in Heroku deployments, also logs incoming requests. This duplication can clutter your logs, making them harder to parse and more expensive to store. Let’s explore why this redundancy exists and how to fix it.

Heroku Router Logs

Heroku’s Router automatically logs all incoming HTTP requests, providing a wealth of data for monitoring and debugging your application. These logs are always enabled and include detailed information, such as the HTTP method and URL path of the request, the response status code, the client’s IP address, and the request processing time (see Heroku Router Log Format).

5 Ways to Get the Latest Book Per Author in Django

Thu, 14 Nov 2024 12:00:00 -0700

In a Django application, fetching the latest record for each group is a common yet challenging task, especially when working with large datasets. Whether you’re building an analytics dashboard or managing grouped data, finding an efficient solution is key. In this blog post, we’ll explore five different approaches to tackle this problem, ranked from least to most effective based on performance and readability, using the following Book model:

class Book(models.Model):
    title = models.CharField(max_length=255)
    author = models.ForeignKey(
        Author, on_delete=models.CASCADE, related_name="books"
    )
    genre = models.CharField(max_length=255)
    published_at = models.DateTimeField()
    is_featured = models.BooleanField(default=False)

Solution 1: Python Max with Prefetch

latest_books = [
    max(author.books.all(), key=lambda x: x.published_at, default=None)
    for author in Author.objects.prefetch_related("books")
]

Performs heavy computation in Python rather than leveraging the database, which is bad for performance. Also makes two database queries (better solutions do it in one).

Zero Downtime Django Deployments with Multistep Database Changes

Sun, 27 Oct 2024 12:00:00 -0700

Preventing downtime during deployments is crucial for maintaining service availability and ensuring a positive user experience. Blue-green deployments have emerged as a popular strategy to achieve this goal. However, they introduce challenges, especially when dealing with database changes. This article delves into what blue-green deployments are, why database changes can be tricky in this context, and how to navigate common change scenarios effectively in Django.

Blue-Green Deployments

A blue-green deployment is a release management strategy that utilizes two separate production environments called “blue” and “green”. At any given time, only one environment is live, serving all production traffic. Changes are deployed to green, and after thorough testing, traffic is switched over from the blue to green. This approach minimizes downtime and provides a quick rollback option by reverting traffic to blue if issues occur.

Mastering Code Search with JetBrains Scope

Sat, 30 Sep 2023 10:55:29 -0400

Background

As software engineers, one of the most crucial skills we develop is the ability to search through code efficiently. Whether it’s finding a specific function, understanding how a certain feature is implemented, or tracing a bug, being able to quickly navigate a codebase is essential for productivity. However, many codebases can be complex and sprawling, leading to noisy search results that hinder rather than aid our progress. JetBrains provides a few tools to help you refine your code search and focus on what’s important.

Supercharge Your Django App: 7 Sneaky Tricks to Crush Slow Database Queries

Sun, 13 Aug 2023 12:15:13 -0400

Optimizing Django query performance is critical for building performant web applications. Django provides many tools and methods for optimizing database queries in its Database access optimization documentation. In this blog post, we will explore a collection of additional and essential tips I’ve compiled over the years to help you pinpoint and resolve your inefficient Django queries.

Kill Long-Running Queries with a Statement Timeout

PostgreSQL supports a statement_timeout parameter that allows you to set a maximum time limit per query. This is useful for preventing long-running queries from tying up precious resources and slowing down your application. My team at PixieBrix experienced an incident where a few long-running queries resulted in a full database outage. Setting a statement timeout in your Django settings can help prevent this from happening.

Optimize Django Query Performance by combining Select Related and Prefetch Related

Sat, 13 May 2023 15:48:39 -0400

Introduction

When building a Django application, one of the key challenges developers face is optimizing database query performance. Django provides two tools,select_related and prefetch_related, that reduce the number of database queries, and increase the performance of your application.

This blog post explores the power of these two methods and how to combine them to maximize your application’s query performance.

My team at PixieBrix implemented these techniques and have seen a significant improvement in query performance and customer satisfaction.

Free up disk space with Docker prune

Sat, 21 Jan 2023 14:11:25 -0500

Docker is a platform for developing, shipping and running applications in isolated, lightweight and portable containers. It is a critical part of a developer’s toolbelt and one I use just about everyday.

Docker can consume a large amount of disk space. Per the Docker documentation:

Docker takes a conservative approach to cleaning up unused objects (often referred to as “garbage collection”), such as images, containers, volumes, and networks: these objects are generally not removed unless you explicitly ask Docker to do so. This can cause Docker to use extra disk space.

[Cross-Post] Django performance: Sort by a Custom Order

Mon, 02 Jan 2023 17:00:00 -0800

See the post here: Sort a Django Queryset by a Custom Order

Disable a direct push to GitHub main branch

Thu, 29 Dec 2022 10:36:08 -0800

On a development team, you never want to push directly to the main branch. Instead, you want to require changes to be made through pull requests so they can be properly reviewed by other developers.

Some developers, including myself, occasionally forget to push to a new branch so I like to have an automated check to prevent this mistake. Here are two methods to block direct pushes to the GitHub main branch.

Using Hadolint, a Dockerfile linter, To Enforce Best Practices

Thu, 17 Mar 2022 19:32:01 -0700

Misconfigured Docker containers can be slow, heavy and difficult to maintain. This leads to sluggish releases, high storage costs and frustrated developers. Moreover, inadequate Docker containers can be a major security liability. A recent report by Aqua Security found that 50% of new Docker instances are attacked within 56 minutes of being deployed. Docker problems can be devastating for any business and software teams need to put in the time and energy to alleviate these risks.

Dockerize a Next.js app using multi-stage builds

Sat, 26 Feb 2022 01:14:31 -0800

See the source code for a working example.

Next.js is a popular React Framework that comes with a lot of great features such as hybrid static & server rendering, smart bundling, file system routing, and much more. I prefer it over vanilla React.

When we dockerize a Next.js application, we want the final production and development images to be small, fast to build and easy to maintain. Let’s see how we can use multi-stage builds and other image optimization techniques to create the ideal images.

Dockerize your Cypress tests

Mon, 31 Jan 2022 21:08:44 -0800

See the source code for a working example.

UI testing is a critical part of any modern web application. My favorite testing framework is Cypress which enables you to write clean, fast and reliable tests. It consists of two main commands:

cypress run: Runs Cypress tests from the CLI without a GUI. Used mostly in CI/CD.
cypress open: Opens Cypress in an interactive GUI. Used for local development.

I like to dockerize my entire application so it can be run anywhere (my machine, coworker’s machine, CI/CD, etc.) and Cypress is no exception. We’re going to dockerize the cypress run and cypress open commands for a simple Todo application written in Django and Next.js (check out the source code).

Free up Django CPU usage in Docker with Watchman

Tue, 04 Jan 2022 23:27:56 -0800

You can use watchman in your Django project to make auto-reloads more CPU efficient. Adam Johnson’s Efficient Reloading in Django’s Runserver With Watchman blog post does an excellent job describing how to set this up. I highly recommend giving it a read. The tutorial explains how to install watchman on macOS with brew install watchman but does not explain how to install it in a Python docker container. Considering all of my Django projects are dockerized, I decided to figure it out.

Set Django Model Field Choices in your Frontend the Right Way

Wed, 01 Dec 2021 10:31:13 -0800

In Django, you can define a set of choices for any field. If you’re using a SPA frontend, such as React or Vue, then you’ll need to access these choices in a form. Let’s look at two ways to do this.

As an example, we’ll use the following Device model:

class Device(models.Model):
    class Size(models.TextChoices):
        SMALL = "S"
        MEDIUM = "M"
        LARGE = "L"

    name = models.CharField(max_length=255)
    size = models.CharField(max_length=255, choices=Size.choices)

Hardcode choices in frontend

The fastest approach is to harcode these choices in the frontend.

5 ways to get all Django objects with a related object

Fri, 01 Oct 2021 13:08:48 -0700

In Django, a related object is a model instance used in the one-to-many or many-to-many context.

For example, let’s look at the built-in User model, which has a many-to-many relationship with the Group model.

class User(models.Model):
    groups = models.ManyToManyField(Group, related_name="groups")

For any given User object, all linked Group objects are called “related objects”. Here are 5 ways to fetch all User objects with at least one related Group object.

Iterate over each object in Python

users = []
for user in User.objects.prefetch_related("groups"):
    if user.groups.exists():
        users.append(user)

Probably the most popular approach but there are two problems:

Why you need to use Subqueries in Django

Wed, 21 Jul 2021 11:56:09 -0700

The Django ORM is a powerful tool but certain aspects of it are counterintuitive, such as the SQL order of execution.

Let’s look at an example of this trap and how we can fix it using subqueries:

class Book(models.Model):
    class Meta:
        constraints = [
            models.UniqueConstraint(
                fields=["name", "edition"],
                name="%(app_label)s_%(class)s_unique_name_edition",
            )
        ]

    name = models.CharField(max_length=255)
    edition = models.CharField(max_length=255)
    release_year = models.PositiveIntegerField(null=True)

I want to write a query that reads:

Check your Django Migrations on every commit

Sat, 08 May 2021 12:10:47 -0700

Keeping your models in sync with your migrations is an important part of any Django app. My team and I frequently make changes to our models and we occassionally forget to create new migrations for those changes. This results in errors and data loss. Let’s look at an easy way to ensure your models and migrations are always in sync:

We’ll use a simple Product model.

class Product(models.Model):
    name = models.CharField(max_length=255)

Let’s say we want to add a quantity field.

Find all N+1 violations in your Django app

Tue, 13 Apr 2021 23:10:50 -0800

The N+1 problem is a common database performance issue. It plagues ORM’s, such as Django and SQLAlchemy, because it leads to your application making more database queries than necessary.

Let’s look at a basic example in Django.

class Artist(models.Model):
    name = models.CharField(max_length=255)

class Song(models.Model):
    artist = models.ForeignKey(Artist, on_delete=models.CASCADE)
    name = models.CharField(max_length=255)

def print_songs():
    for song in Song.objects.all():
        print(f"{song.artist.name} - {song.name}")

Now let’s create a unit test to ensure print_songs runs successfully. We’re using pytest and pytest-django so we can create the test data using a pytest fixture. When we run the unit test, you’ll notice it passes and our songs are printed to the terminal. Everything seems fine on the surface.

Mon, 01 Jan 0001 00:00:00 +0000

About Me

Full Stack Software Engineer located in San Francisco, CA. I specialize in web development with Python, Django, React, Docker, Kubernetes and AWS.

Side Projects

Notifire