Celery

v 9.0 v 10.0 v 11.0 v 12.0 Third Party 996

Availability	Odoo Online Odoo.sh On Premise
Odoo Apps Dependencies	Discuss (mail)
Lines of code	1687
Technical Name	`celery`
License	LGPL-3
Website	https://www.novacode.nl
Versions	10.0 11.0 13.0 12.0 14.0 9.0

You bought this module and need support? Click here!

Availability	Odoo Online Odoo.sh On Premise
Odoo Apps Dependencies	Discuss (mail)
Lines of code	1687
Technical Name	`celery`
License	LGPL-3
Website	https://www.novacode.nl
Versions	10.0 11.0 13.0 12.0 14.0 9.0

Celery: Distributed Task Queue

Odoo integration of the Python #1 asynchronous task queue

Features:

Put model-methods on the Celery Task Queue.
Monitor and manage the Task Queue in Odoo.
All Exceptions are catched and available as State=Failure with Exception message/trace shown.
Requeue of Failed and Pending (stale) tasks.
No complex installation and setup requirements are needed, except the Celery setup.

Check out the links below, for more info about Celery.

Celery Celery Installation

Put a model-method on the Celery Task Queue

Just Python code to call from an Odoo model

Example

Model and method:

model: "celery.example"
method: "schedule_import"

Celery Options:

Shall be provided to kwargs of the call_task method

celery = {
    'queue': 'high.priority',
    'countdown': 5,
    'retry': True,
    'retry_countdown_setting': 'MUL_RETRIES_SECS',
    'retry_countdown_multiply_retries_seconds': 60,
    'retry_policy': {
        'interval_start': 30
    }
}

Calling the task:

self.env["celery.task"].call_task("celery.example", "schedule_import", celery=celery)

Celery Options

All Celery options are optional (not required).

Option	Description	Celery Documentation
`queue`	Name of the Celery/worker queue, the task shall be routed to.	Routing Tasks
`countdown`	The countdown is a shortcut to set ETA by seconds into the future.	ETA and Countdown
`retry`	Set to `True` to enable the retry of sending task messages.	Message Sending Retry
`retry_countdown_setting`	Specify whether and how to increase the Retry Countdown by: ADD_SECS: `countdown = countdown + retry_countdown_add_seconds` MUL_RETRIES: `countdown = countdown * request.retries` MUL_RETRIES_SECS: `countdown = request.retries * retry_countdown_multiply_retries_seconds`	This is a custom option which affects the Celery `countdown` option.
`retry_countdown_add_seconds`	Specify the seconds (integer field) to add to the Retry Countdown.	This is a custom option which affects the Celery `countdown` option.
`retry_countdown_multiply_retries_seconds`	Specify the seconds (integer field) to multiply with request retries, which becomes the Retry Countdown.	This is a custom option which affects the Celery `countdown` option.
`retry_policy`	Options when retrying publishing a task message in the case of connection loss or other connection errors.	Message Sending Retry

Extra kwargs

Extra kwargs are optional (not required).

Kwarg	Description
`transaction_strategy`	Specifies when the task shall apply (ORM create and send to Celery MQ): `after_commit`: Apply task after commit of the main/caller transaction (default setting). `immediate`: Apply task immediately from the main/caller transaction, even if it ain't committed yet.

Monitor and control the Celery Task Queue

List of queued tasks

Task Failure info

Tasks waiting for Retry

Task waiting for Retry with Failure info

Requeue a Failed Task

Requeue multiple Failed Tasks

Installation and configuration

Celery and Message broker

Celery Installation

Odoo configuration

All you need is to determine the Odoo Celery user and setup the credentials for XML-RPC authentication.

This enables the Celery process to authenticate in Odoo, by the XML-RPC server.

To support different kind of deployment and build tools, the credentials can either be setup as:

(OS) Environment variables of the user running the Odoo server process: ODOO_CELERY_USER and ODOO_CELERY_PASSWORD

Put in the odoo.conf file, e.g:

celery_user = Odoo-User
celery_password = Odoo-User-Password

See: example.odoo.conf

Put in the odoo.conf file, under section [celery] e.g:
```
[celery]
user = Odoo-User
password = Odoo-User-Password
```
See: example_section_celery.odoo.conf

Start the Celery (Odoo) worker server

Check the Celery Documentation for more ways to start and manage the server/proces. E.g. by Supervisor

The Celery (Odoo) worker => Python file odoo.py, which is located directly under the "celery" module directory.

Start the worker (default/celery queue) on command line, whereas "odoo" references to the Python file:

# celery -A odoo worker --loglevel=info

Start a worker as a named queue:

# celery -A odoo worker --loglevel=info -Q high.priority -n high.priority

"Celery Example" module

Demo of 2 implemented example tasks, queued by 2 buttons

Check out the "Celery Example" module

After installation, go to the menu "Celery / Example Task".
Click button(s) "Queue create Line" shown in screensshot, which puts a task on the queue.
Check the queue (menu: Celery / Tasks). Check the form of the Example record - new Lines had been created.

Celery Example module

Changelog

0.18

Configurable (database) transaction strategy, when the task shall apply (ORM create and send to Celery MQ).
From now on - by default a task shall apply after commit of the main/caller transaction, instead of immediately in the main/caller transaction.

0.17

Task scheduling added - being able to schedule tasks in a specified time interval or certain day(s) of the week.
A new task state - Scheduled, is handled by an Odoo cron job - "Celery: Handle Scheduled Tasks".

0.16

Configurable celery queues added to task settings.

0.15

Scheduled (cron) cleanup of tasks - with optionally specifying/overriding: (1) timedelta (days=90, hours, minutes, seconds), (2) states to keep and (3) batch_size=100.
Create database index for the State Time (state_date) field.

0.14

Create database index for the Reference field (ref).

0.13

Get XML-RPC URL from ir.config.parameter (celery.celery_base_url) by Settings.

0.12

Also support to get the Celery (Odoo) user and password from the odoo.conf file, under section [options] too.

0.11

Put task into a new state Retrying, right before the Retry starts.

0.10

Renamed task retry settings: 'MUL_RETR' to 'MUL_RETRIES', 'MUL_RETR_SECS' to 'MUL_RETRIES_SECS'.

0.9

Hide (Odoo) password in the retry payload used by the MQ broker for XML-RPC authentication. Mentoined in GitHub issue: https://github.com/novacode-nl/odoo-celery/issues/17

0.8

Fix task retry countdown/interval ignored. For more info see GitHub issue
Add task retry countdown settings, to increase countdown during retries ('ADD_SECS', 'MUL_RETRY', 'MUL_RETRY_SECS').
Search view of jammed tasks: (1) Add/show field reference (2) search-view with filters and grouping.
Track task changes of fields model, method, handle_jammed
Disable create/copy in the task form-view.

0.7

Task Reference, which serves:
- Easier searching for tasks by Reference.
- Check (ORM query) before call_task() call, to prevent redundant creation of recurring Pending tasks.

0.6

Introduction of "Jammed" tasks. A jammed task could be caused, for example by: a stopped/restarted Celery worker, message broker, or server.

Settings: to specify when a task (model, method) is Jammed after seconds from Started or Retry (states).
Jammed Tasks Report: which shows Jammed (not completed) tasks.
Cron or manually put tasks in "Jammed" state.
Ability to cancel (Pending or Jammed) tasks, which never completed.
Track and messaging about state change. Chatter on form view.

0.5

Routing tasks to specific (named) queues.

0.4

FIX: Store task state (Started, Retry) before execution.

0.3

Hide (Odoo) password in payload used by the broker for XML-RPC authentication. Mentoined in GitHub issue: https://github.com/novacode-nl/odoo-celery/issues/4

0.2

Task state information.

0.1

Initial version.

Please log in to comment on this module

The author can leave a single reply to each comment.
This section is meant to ask simple questions or leave a rating. Every report of a problem experienced while using the module should be addressed to the author directly (refer to the following point).
If you want to start a discussion with the author, please use the developer contact information. They can usually be found in the description.