Django Performance & Optimization
Downloaded from PrepIQ (https://prepiq.online)
Q1. What is the N+1 query problem in Django and how do select_related and prefetch_related solve it?
The N+1 problem occurs when iterating over N objects and accessing a related object on each one triggers a separate query per iteration (because Django's ORM is lazy by default), resulting in N+1 total queries instead of one efficient join — a common and easy-to-miss performance bug in list views and templates.
select_related() solves this for forward foreign-key and one-to-one relationships by performing a SQL JOIN and fetching the related object in the same query. prefetch_related() solves it for reverse foreign-key and many-to-many relationships, where a JOIN would multiply rows, by running a second separate query and joining the results in Python instead.
orders = Order.objects.all()
for order in orders:
print(order.customer.name) # extra query every iteration
orders = Order.objects.select_related('customer')
customers = Customer.objects.prefetch_related('orders')
Q2. How does Django's QuerySet laziness work, and when does a queryset actually hit the database?
A Django QuerySet is lazy: creating or chaining it (filter(), exclude(), order_by()) only builds up a SQL query description without executing anything against the database. This lets querysets be composed and passed around cheaply, and lets Django combine multiple chained filters into a single efficient query rather than running one per method call.
The query only actually executes when the queryset is evaluated — iterating over it, calling len(), slicing with a step, calling list(), printing/repr in a shell, or calling a method that must return a concrete value like .count(), .exists(), .first(), or .get(). Understanding this distinction is essential for spotting accidental double-evaluation.
qs = Product.objects.filter(price__gt=100) # no query yet
qs = qs.order_by('name') # still no query
list(qs) # query executes now
Q3. How do you add database indexes in Django models, and when should you use them?
A database index is a separate data structure that lets the database look up rows matching a condition without scanning the entire table, dramatically speeding up queries that filter, order, or join on that column — at the cost of extra storage and slightly slower writes, since indexes must be updated on every insert/update.
In Django, an index is added by setting db_index=True on a field or, preferably for multi-column and more control, adding entries to a model's Meta.indexes list using models.Index. Indexes are most valuable on columns frequently used in filter(), order_by(), or as foreign keys (which Django indexes automatically); indexing every column indiscriminately hurts write performance without meaningful read benefit.
class Order(models.Model):
status = models.CharField(max_length=20, db_index=True)
customer = models.ForeignKey(Customer, on_delete=models.CASCADE)
created_at = models.DateTimeField()
class Meta:
indexes = [
models.Index(fields=['status', 'created_at']),
]
Q4. How do you use only(), defer(), and values()/values_list() to optimize queries?
By default, a queryset fetches every field of a model, which wastes bandwidth and memory when only a few fields are actually needed — especially with large text or binary fields. only('field1', 'field2') restricts the SELECT to just the named fields (plus the primary key), while defer('field1') does the opposite, fetching everything except the named fields, deferred until accessed.
values() and values_list() go further by skipping model instantiation entirely, returning dictionaries or tuples instead of full model objects — much cheaper when you only need raw data for a report, export, or API response and don't need model methods or related-object traversal. Accessing a deferred field triggers an additional query per instance, so defer()/only() should be used carefully alongside select_related() when related fields are also needed.
Product.objects.only('name', 'price')
Product.objects.defer('description')
Product.objects.values('name', 'price')
Product.objects.values_list('name', flat=True)
Q5. How does Django's per-view and per-site caching work, and when should you use low-level cache API instead?
Django offers caching at several granularities. Per-site caching (UpdateCacheMiddleware + FetchFromCacheMiddleware) caches every GET/HEAD response for the whole site, simplest to enable but too coarse for pages with any personalized or frequently-changing content. Per-view caching with @cache_page(timeout) caches the rendered output of one specific view, better suited to pages that are the same for every visitor, like a public blog post.
The low-level cache API (cache.get(), cache.set()) gives full manual control to cache anything — a single expensive queryset result, a computed aggregate, an external API response — independent of the request/response cycle, which is necessary whenever only part of a page needs caching or the cached value needs custom invalidation logic tied to specific data changes.
from django.core.cache import cache
from django.views.decorators.cache import cache_page
@cache_page(60 * 15)
def blog_post(request, slug): ...
def get_top_products():
result = cache.get('top_products')
if result is None:
result = list(Product.objects.order_by('-sales')[:10])
cache.set('top_products', result, timeout=3600)
return result
Q6. How do you use Django's database query optimization tools: explain(), Prefetch objects, and annotate()?
queryset.explain() returns the database's query execution plan (showing whether an index is actually used, join strategy, and estimated row counts), the primary diagnostic tool for understanding why a query is slow instead of guessing. annotate() lets aggregate calculations (counts, sums, averages) be computed in the database rather than pulled into Python and looped over, which is both faster and uses far less memory for large datasets.
A Prefetch object gives fine-grained control over prefetch_related() beyond a plain field name — it can filter, order, or limit the related queryset being prefetched, and store the result under a custom attribute name, letting you prefetch only the relevant subset of related objects instead of all of them.
from django.db.models import Prefetch, Count
Order.objects.filter(status='pending').explain()
Category.objects.annotate(product_count=Count('products'))
Customer.objects.prefetch_related(
Prefetch('orders', queryset=Order.objects.filter(created_at__gte=last_month))
)
Q7. What are Django async views and when do they actually improve performance?
Since Django 3.1, views can be defined as async def functions, run under ASGI instead of WSGI, allowing a single worker to handle many concurrent requests without blocking on I/O — useful when a view spends most of its time waiting on network calls (external APIs, websockets) rather than doing CPU-bound work.
Async views provide no benefit, and can even hurt performance, for views dominated by synchronous ORM calls, since Django's ORM is still fundamentally synchronous (async ORM support is limited/newer) — calling the sync ORM from an async view requires wrapping it with sync_to_async, which reintroduces blocking. Async is most valuable for views that primarily fan out to multiple external APIs concurrently using asyncio.gather, not as a blanket performance upgrade for typical CRUD views.
import asyncio, httpx
async def aggregate_view(request):
async with httpx.AsyncClient() as client:
weather, news = await asyncio.gather(
client.get("https://api.weather.example/today"),
client.get("https://api.news.example/latest"),
)
return JsonResponse({"weather": weather.json(), "news": news.json()})
Q8. How do you use bulk_create, bulk_update, and iterator() to handle large datasets efficiently?
Calling .save() in a loop to create or update many rows issues one database round-trip per object, which is slow at scale. bulk_create() inserts a list of unsaved model instances in a single (or few, batched) SQL statement, and bulk_update() similarly updates many existing instances in one query, both dramatically reducing round-trip overhead — though bulk operations skip save(), so custom save() logic and signals like pre_save/post_save don't fire.
For reading large querysets, plain iteration loads the entire result set into memory at once (via internal caching), which can exhaust memory for very large tables. iterator(chunk_size=2000) streams results from the database in batches instead of loading everything upfront, trading the queryset's result caching for bounded memory usage — the right choice for one-off exports or migrations over millions of rows.
Product.objects.bulk_create([Product(name=f"P{i}") for i in range(10000)])
products = Product.objects.filter(category='clearance')
for p in products:
p.price *= 0.8
Product.objects.bulk_update(products, ['price'])
for product in Product.objects.all().iterator(chunk_size=2000):
process(product)
Q9. How does Django's connection pooling and CONN_MAX_AGE setting affect performance?
By default, Django opens a new database connection for each request and closes it at the end (CONN_MAX_AGE = 0), which is safe but adds the overhead of a full connection handshake — including authentication and TLS negotiation for remote databases — to every single request.
Setting CONN_MAX_AGE to a positive number of seconds (or None for unlimited) enables persistent connections: a connection is reused across multiple requests as long as it's still valid and hasn't exceeded the max age, avoiding repeated handshake overhead. For high-concurrency deployments, this is often combined with an external connection pooler like PgBouncer in front of PostgreSQL, since Django's built-in persistent connections are per-worker-process, not a true shared pool across all workers.
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'CONN_MAX_AGE': 60,
'CONN_HEALTH_CHECKS': True,
}
}
Q10. How do you profile a slow Django application in production (Silk, APM tools, logging slow queries)?
Django Debug Toolbar only works in development against a single request at a time; production profiling needs tools that run safely under real traffic with minimal overhead. Django Silk is a lightweight profiler that can run in production (with sampling) to record request timings and SQL queries into a database, browsable later through a UI, without requiring a live debugging session.
Application Performance Monitoring (APM) tools like Sentry, New Relic, or Datadog APM instrument the whole application automatically, capturing slow request traces, database query timing, external call latency, and error rates across all instances, with alerting when performance degrades. For database-specific slowness, most database engines also support logging queries that exceed a duration threshold (e.g. PostgreSQL's log_min_duration_statement), which surfaces slow queries directly from the source of truth regardless of which Django code path triggered them.
LOGGING = {
'version': 1,
'handlers': {'console': {'class': 'logging.StreamHandler'}},
'loggers': {
'django.db.backends': {
'handlers': ['console'],
'level': 'DEBUG',
},
},
}
Django Performance & Optimization
What is the N+1 query problem in Django and how do select_related and prefetch_related solve it?
The N+1 problem occurs when iterating over N objects and accessing a related object on each one triggers a sep...
How does Django's QuerySet laziness work, and when does a queryset actually hit the database?
A Django QuerySet is lazy: creating or chaining it (filter(), exclude(), order_by()) only builds up a SQL quer...
How do you add database indexes in Django models, and when should you use them?
A database index is a separate data structure that lets the database look up rows matching a condition without...
How do you use only(), defer(), and values()/values_list() to optimize queries?
By default, a queryset fetches every field of a model, which wastes bandwidth and memory when only a few field...
How does Django's per-view and per-site caching work, and when should you use low-level cache API instead?
Django offers caching at several granularities. Per-site caching (UpdateCacheMiddleware + FetchFromCacheMiddle...
How do you use Django's database query optimization tools: explain(), Prefetch objects, and annotate()?
queryset.explain() returns the database's query execution plan (showing whether an index is actually used, joi...
What are Django async views and when do they actually improve performance?
Since Django 3.1, views can be defined as async def functions, run under ASGI instead of WSGI, allowing a sing...
How do you use bulk_create, bulk_update, and iterator() to handle large datasets efficiently?
Calling .save() in a loop to create or update many rows issues one database round-trip per object, which is sl...
How does Django's connection pooling and CONN_MAX_AGE setting affect performance?
By default, Django opens a new database connection for each request and closes it at the end (CONN_MAX_AGE = 0...
How do you profile a slow Django application in production (Silk, APM tools, logging slow queries)?
Django Debug Toolbar only works in development against a single request at a time; production profiling needs...