Python

If your Django queries feel slow, the problem might not be your database — it might be your ORM. Recently, I was working with a query that took 25 seconds to run through the Django ORM, but the underlying SQL completed in just 2 seconds. With a single change, I got the ORM query down to 2 seconds as well — and reduced the memory footprint by 70%. Let’s walk through what happened, and why using .values() instead of .only() can dramatically improve performance. ...

A simple insert query turned into a silent performance killer. Our frontend pings our server every few minutes to track device activity. Each ping attempts to insert a row into a DevicePingDaily table, which has a unique constraint on (device_id, date) to ensure only one record per device per day. In Django, the logic looked like this: try: DevicePingDaily.objects.create(device=device, date=today) except IntegrityError: pass It seemed harmless. But as traffic grew, latency spiked and API timeouts increased. Observability tools quickly pointed to the culprit: ...

When hosting an application on Heroku, managing logs efficiently is crucial for maintaining system health and keeping costs down. Heroku provides built-in logging for all incoming requests, but by default, Gunicorn, the Python HTTP server often used in Heroku deployments, also logs incoming requests. This duplication can clutter your logs, making them harder to parse and more expensive to store. Let’s explore why this redundancy exists and how to fix it. Heroku Router Logs Heroku’s Router automatically logs all incoming HTTP requests, providing a wealth of data for monitoring and debugging your application. These logs are always enabled and include detailed information, such as the HTTP method and URL path of the request, the response status code, the client’s IP address, and the request processing time (see Heroku Router Log Format). ...

In a Django application, fetching the latest record for each group is a common yet challenging task, especially when working with large datasets. Whether you’re building an analytics dashboard or managing grouped data, finding an efficient solution is key. In this blog post, we’ll explore five different approaches to tackle this problem, ranked from least to most effective based on performance and readability, using the following Book model: class Book(models.Model): title = models.CharField(max_length=255) author = models.ForeignKey( Author, on_delete=models.CASCADE, related_name="books" ) genre = models.CharField(max_length=255) published_at = models.DateTimeField() is_featured = models.BooleanField(default=False) Solution 1: Python Max with Prefetch latest_books = [ max(author.books.all(), key=lambda x: x.published_at, default=None) for author in Author.objects.prefetch_related("books") ] Performs heavy computation in Python rather than leveraging the database, which is bad for performance. Also makes two database queries (better solutions do it in one). ...

Optimizing Django query performance is critical for building performant web applications. Django provides many tools and methods for optimizing database queries in its Database access optimization documentation. In this blog post, we will explore a collection of additional and essential tips I’ve compiled over the years to help you pinpoint and resolve your inefficient Django queries. Kill Long-Running Queries with a Statement Timeout PostgreSQL supports a statement_timeout parameter that allows you to set a maximum time limit per query. This is useful for preventing long-running queries from tying up precious resources and slowing down your application. My team at PixieBrix experienced an incident where a few long-running queries resulted in a full database outage. Setting a statement timeout in your Django settings can help prevent this from happening. ...