Running Background Tasks in Django Using Celery, Redis and Celery Beat

Introduction

In most Django apps, tasks are handled synchronously: a user makes a request, Django processes it, and the response is returned. But what about tasks that take a long time—like sending emails, generating reports, or cleaning up old data?

If these tasks run synchronously, users wait unnecessarily and your server may get blocked.

This is where Celery comes in. Celery lets you run background tasks asynchronously, and with Redis as a message broker plus Celery Beat for scheduling, you can also run tasks periodically.

In this blog, we’ll build a Django app that runs background and scheduled tasks step by step.

Step 1: Set Up Your Django Project

django-admin startproject project
cd project
python manage.py startapp api
pip install django celery redis

Step 2: Configure Celery

Create project/celery.py:

from __future__ import absolute_import, unicode_literals
import os
from celery import Celery

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'myproject.settings')

app = Celery('project')
app.config_from_object('django.conf:settings', namespace='CELERY')
app.autodiscover_tasks()

Update project/__init__.py:

from .celery import app as celery_app
__all__ = ('celery_app',)

Step 3: Configure Redis

In settings.py:

CELERY_BROKER_URL = 'redis://localhost:6379/0'
CELERY_RESULT_BACKEND = 'redis://localhost:6379/0'

Start Redis:

redis-server

Redis acts as the message broker, passing tasks from Django to Celery workers.

Step 4: Create Background Tasks

api/tasks.py:

from celery import shared_task
import time
from datetime import datetime

@shared_task
def send_welcome_email(user_email):
    time.sleep(5)  # Simulate sending email
    print(f"Sent welcome email to {user_email}")
    return f"Email sent to {user_email}"

@shared_task
def print_current_time():
    print(f"Current time: {datetime.now()}")

• send_welcome_email → manual background task

• print_current_time → will be scheduled periodically

Step 5: Call the Task Asynchronously

from django.http import HttpResponse
from .tasks import send_welcome_email

def register_user(request):
    user_email = 'newuser@example.com'
    send_welcome_email.delay(user_email)
    return HttpResponse("User registered! Email is being sent in the background.")

.delay() sends the task to Celery without blocking the request.

Step 6: Schedule Periodic Tasks with Celery Beat

In settings.py:

from celery.schedules import crontab

CELERY_BEAT_SCHEDULE = {
    'print-time-every-minute': {
        'task': 'api.tasks.print_current_time',
        'schedule': crontab(minute='*/1'),  # every 1 minute
    },
}

Step 7: Run Celery Worker and Beat

Open two terminals:

1. Run the worker:

    celery -A project worker --loglevel=info
   

2. Run Celery Beat:

    celery -A project beat --loglevel=info
   

   Now, you’ll see the periodic task logging the current time every minute in your terminal.

Conclusion

We’ve taken a Django app from just running locally to handling background tasks with Celery, using Redis to queue jobs, and even scheduling periodic tasks with Celery Beat.

Now, your app can send emails, generate reports, or do any heavy lifting without making your users wait. Plus, it’s way more scalable and ready for real-world use.

Once you get the hang of it, adding more tasks, monitoring them, or even deploying this setup to the cloud becomes a breeze.

So go ahead, try it out, and watch your Django app come alive with background magic! ✨