Skip to content

Commit 05dd0f0

Browse files
n2ygkn2ygk
committed
Add tutorial for Celery task setup.
1 parent 56c8c66 commit 05dd0f0

File tree

5 files changed

+173
-0
lines changed

5 files changed

+173
-0
lines changed

‎CHANGELOG.md

+2
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
2121
### Added
2222
* #1106 OIDC: Add "scopes_supported" to the [ConnectDiscoveryInfoView](https://django-oauth-toolkit.readthedocs.io/en/latest/oidc.html#connectdiscoveryinfoview).
2323
This completes the view to provide all the REQUIRED and RECOMMENDED [OpenID Provider Metadata](https://openid.net/specs/openid-connect-discovery-1_0.html#ProviderMetadata).
24+
* #1128 Documentation: [Tutorial](https://django-oauth-toolkit.readthedocs.io/en/latest/tutorial/tutorial_05.html)
25+
on using Celery to automate clearing expired tokens.
2426

2527
### Changed
2628
* #1093 (**Breaking**) Changed to implement [hashed](https://docs.djangoproject.com/en/stable/topics/auth/passwords/)

‎docs/tutorial/admin+celery.png

65.5 KB
Loading

‎docs/tutorial/celery+add.png

71.2 KB
Loading

‎docs/tutorial/tutorial.rst

+2
Original file line numberDiff line numberDiff line change
@@ -8,3 +8,5 @@ Tutorials
88
tutorial_02
99
tutorial_03
1010
tutorial_04
11+
tutorial_05
12+

‎docs/tutorial/tutorial_05.rst

+169
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,169 @@
1+
Part 5 - Using Celery to Automate Maintenance Chores
2+
====================================================
3+
4+
Scenario
5+
--------
6+
In :doc:`Part 1 <tutorial_01>` you created your own :term:`Authorization Server` and it's running along just fine.
7+
However, the database is getting cluttered with expired tokens. You can periodically run
8+
the :doc:`cleartokens management command <../management_commands>`, but why not automate this with
9+
`Celery <https://docs.celeryq.dev/>`_?
10+
11+
Set up RabbitMQ
12+
---------------
13+
Celery components communicate via a message queue. We'll use `RabbitMQ <https://www.rabbitmq.com/>`_.
14+
15+
Install RabbitMQ on MacOS
16+
~~~~~~~~~~~~~~~~~~~~~~~~~~
17+
If you are using MacOS it's likely you are already using `Homebrew <https://brew.sh/>`_. If not, now's
18+
the time to install this fantastic package manager.
19+
20+
::
21+
22+
brew install rabbitmq
23+
brew service start rabbitmq
24+
25+
Install RabbitMQ with Docker
26+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
27+
This will start up a docker image that just works:
28+
::
29+
30+
docker run -it --rm --name rabbitmq -p 5672:5672 -p 15672:15672 rabbitmq:3.9-management
31+
32+
33+
34+
Install RabbitMQ on Windows
35+
~~~~~~~~~~~~~~~~~~~~~~~~~~~
36+
See the `RabbitMQ Installing on Windows <https://www.rabbitmq.com/install-windows.html>`_ instructions.
37+
38+
39+
Add Celery
40+
----------
41+
Make sure you virtualenv is active and install `celery` and
42+
`django-celery-beat <https://django-celery-beat.readthedocs.io/>`_.
43+
44+
::
45+
46+
pip install celery django-celery-beat
47+
48+
Update your list of installed apps to include both your :term:`Authorization Server` app -- we'll call it ``tutorial``,
49+
and ``django_celery_beat`` which extends your Django project to store your periodic task schedule
50+
in the database and adds a Django Admin interface for configuring them.
51+
52+
.. code-block:: python
53+
54+
INSTALLED_APPS = {
55+
# ...
56+
"tutorial",
57+
"django_celery_beat",
58+
}
59+
60+
61+
Now add a new file to your app to add Celery: ``tutorial/celery.py``:
62+
63+
.. code-block:: python
64+
65+
import os
66+
67+
from celery import Celery
68+
69+
# Set the default Django settings module for the 'celery' program.
70+
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'tutorial.settings')
71+
app = Celery('tutorial', broker="pyamqp://guest@localhost//")
72+
app.config_from_object('django.conf:settings', namespace='CELERY')
73+
74+
# Load task modules from all registered Django apps.
75+
app.autodiscover_tasks()
76+
77+
This will autodiscover any ``tasks.py`` files in the list of installed apps.
78+
We'll add ours now in ``tutorial/tasks.py``:
79+
80+
.. code-block:: python
81+
82+
from celery import shared_task
83+
84+
@shared_task
85+
def clear_tokens():
86+
from oauth2_provider.models import clear_expired
87+
88+
clear_expired()
89+
90+
Finally, update ``tutorial/__init__.py`` to make sure Celery gets loaded when the app starts up:
91+
92+
.. code-block:: python
93+
94+
from .celery import app as celery_app
95+
96+
__all__ = ('celery_app',)
97+
98+
99+
Run Celery Beat and the Worker
100+
------------------------------
101+
102+
RabbitMQ should already be running; it's the "glue" between Beat and the Worker.
103+
104+
It's best to run each of these in its own terminal window so you can see the log messages.
105+
106+
Start Celery Beat
107+
~~~~~~~~~~~~~~~~~
108+
109+
::
110+
111+
celery -A tutorial beat -l INFO --scheduler django_celery_beat.schedulers:DatabaseScheduler
112+
113+
Start Celery Worker
114+
~~~~~~~~~~~~~~~~~~~
115+
116+
::
117+
118+
celery -A tutorial worker -l INFO
119+
120+
Configure the ``clear_tokens`` task
121+
-----------------------------------
122+
123+
Go into `Django Admin <http://127.0.0.1:8000/admin/>`_ and you'll see a new section for periodic tasks:
124+
125+
.. image:: admin+celery.png
126+
:width: 500
127+
:alt: Django Admin interface screenshot
128+
129+
Now let's define a fairly short (10 second) interval. Go to: http://127.0.0.1:8000/admin/django_celery_beat/intervalschedule/
130+
and select Add Interval, set number of intervals to 10 and interval period to seconds and Save.
131+
132+
Then go to http://127.0.0.1:8000/admin/django_celery_beat/periodictask/ to add a new periodic task by
133+
selecting `Add Periodic Task <http://127.0.0.1:8000/admin/django_celery_beat/periodictask/add/>`_ and
134+
select ``tutorial.tasks.clear_tokens``, choose the ``every 10 seconds`` interval schedule, and "Save."
135+
136+
.. image:: celery+add.png
137+
:width: 500
138+
:alt: Django Admin interface screenshot
139+
140+
141+
Now your Celery Beat and Celery Workers should start running the task every 10 seconds.
142+
143+
The Beat console will look like this:
144+
145+
::
146+
147+
[2022-03-19 22:06:35,605: INFO/MainProcess] Scheduler: Sending due task clear stale tokens (tutorial.tasks.clear_tokens)
148+
149+
And the Workers console like this:
150+
151+
::
152+
153+
[2022-03-19 22:06:35,614: INFO/MainProcess] Task tutorial.tasks.clear_tokens[5ec25fb8-5ce3-4d15-b9ad-750b80fc07e0] received
154+
[2022-03-19 22:06:35,616: INFO/ForkPoolWorker-8] refresh_expire_at is None. No refresh tokens deleted.
155+
[2022-03-19 22:06:35,629: INFO/ForkPoolWorker-8] 0 Expired access tokens deleted
156+
[2022-03-19 22:06:35,631: INFO/ForkPoolWorker-8] 0 Expired grant tokens deleted
157+
[2022-03-19 22:06:35,632: INFO/ForkPoolWorker-8] Task tutorial.tasks.clear_tokens[5ec25fb8-5ce3-4d15-b9ad-750b80fc07e0] succeeded in 0.016124433999999965s: None
158+
159+
160+
References
161+
----------
162+
163+
The preceding is based on these references:
164+
165+
https://docs.celeryq.dev/en/stable/django/first-steps-with-django.html
166+
167+
https://docs.celeryq.dev/en/stable/userguide/periodic-tasks.html#beat-custom-schedulers
168+
169+
https://django-celery-beat.readthedocs.io/en/latest/index.html

0 commit comments

Comments
 (0)