Celery

by
Odoo 1
v 9.0 v 10.0 v 11.0 v 12.0 Third Party 246
Download for 12.0 series Deploy on Odoo.sh
You bought this module and need support ? Click here !
Technical name celery
LicenseLGPL-3
Websitehttps://www.novacode.nl
Read description for v 11.0 v 10.0 v 9.0
Technical name celery
LicenseLGPL-3
Websitehttps://www.novacode.nl
Read description for v 11.0 v 10.0 v 9.0

Celery: Distributed Task Queue

Odoo integration of the Python #1 asynchronous task queue

Features:
  • Put model-methods on the Celery Task Queue.
  • Monitor and manage the Task Queue in Odoo.
  • All Exceptions are catched and available as State=Failure with Exception message/trace shown.
  • Requeue of Failed and Pending (stale) tasks.
  • No complex installation and setup requirements are needed, except the Celery setup.

Check out the links below, for more info about Celery.

Celery Celery Installation

Put a model-method on the Celery Task Queue

Just Python code to call from an Odoo model

Example

Model and method:

  • model: "celery.example"
  • method: "schedule_import"

Celery Options:

Shall be provided to kwargs of the call_task method

celery = {
    'queue': 'high.priority',
    'countdown': 5,
    'retry': True,
    'retry_countdown_setting': 'MUL_RETRIES_SECS',
    'retry_countdown_multiply_retries_seconds': 60,
    'retry_policy': {
        'interval_start': 30
    }
}

Calling the task:

self.env["celery.task"].call_task("celery.example", "schedule_import", celery=celery)

Celery Options

All Celery options are optional (not required).

Option Description Celery Documentation
queue Name of the Celery/worker queue, the task shall be routed to. Routing Tasks
countdown The countdown is a shortcut to set ETA by seconds into the future. ETA and Countdown
retry Set to True to enable the retry of sending task messages. Message Sending Retry
retry_countdown_setting Specify whether and how to increase the Retry Countdown by:
  • ADD_SECS:
    countdown = countdown + retry_countdown_add_seconds
  • MUL_RETRIES:
    countdown = countdown * request.retries
  • MUL_RETRIES_SECS:
    countdown = request.retries * retry_countdown_multiply_retries_seconds
This is a custom option which affects the Celery countdown option.
retry_countdown_add_seconds Specify the seconds (integer field) to add to the Retry Countdown. This is a custom option which affects the Celery countdown option.
retry_countdown_multiply_retries_seconds Specify the seconds (integer field) to multiply with request retries, which becomes the Retry Countdown. This is a custom option which affects the Celery countdown option.
retry_policy Options when retrying publishing a task message in the case of connection loss or other connection errors. Message Sending Retry

Monitor and control the Celery Task Queue

List of queued tasks

Task Failure info

Tasks waiting for Retry

Task waiting for Retry with Failure info

Requeue a Failed Task

Requeue multiple Failed Tasks

Installation and configuration

Celery and Message broker

Odoo configuration

All you need is to determine the Odoo Celery user and setup the credentials for XML-RPC authentication.

This enables the Celery process to authenticate in Odoo, by the XML-RPC server.

To support different kind of deployment and build tools, the credentials can either be setup as:

  • (OS) Environment variables of the user running the Odoo server process: ODOO_CELERY_USER and ODOO_CELERY_PASSWORD

  • Put in the odoo.conf file, e.g:

    celery_user = Odoo-User
    celery_password = Odoo-User-Password

    See: example.odoo.conf

  • Put in the odoo.conf file, under section [celery] e.g:

    [celery]
    user = Odoo-User
    password = Odoo-User-Password

    See: example_section_celery.odoo.conf

Start the Celery (Odoo) worker server

Check the Celery Documentation for more ways to start and manage the server/proces. E.g. by Supervisor

The Celery (Odoo) worker => Python file odoo.py, which is located directly under the "celery" module directory.

Start the worker (default/celery queue) on command line, whereas "odoo" references to the Python file:

# celery -A odoo worker --loglevel=info

Start a worker as a named queue:

# celery -A odoo worker --loglevel=info -Q high.priority -n high.priority

"Celery Example" module

Demo of 2 implemented example tasks, queued by 2 buttons

Check out the "Celery Example" module

  • After installation, go to the menu "Celery / Example Task".
  • Click button(s) "Queue create Line" shown in screensshot, which puts a task on the queue.
  • Check the queue (menu: Celery / Tasks). Check the form of the Example record - new Lines had been created.
Celery Example module

Changelog

0.14

  • Create database index for the Reference field (ref).

0.13

  • Get XML-RPC URL from ir.config.parameter (celery.celery_base_url) by Settings.

0.12

  • Also support to get the Celery (Odoo) user and password from the odoo.conf file, under section [options] too.

0.11

  • Put task into a new state Retrying, right before the Retry starts.

0.10

  • Renamed task retry settings: 'MUL_RETR' to 'MUL_RETRIES', 'MUL_RETR_SECS' to 'MUL_RETRIES_SECS'.

0.9

0.8

  • Fix task retry countdown/interval ignored. For more info see GitHub issue
  • Add task retry countdown settings, to increase countdown during retries ('ADD_SECS', 'MUL_RETRY', 'MUL_RETRY_SECS').
  • Search view of jammed tasks: (1) Add/show field reference (2) search-view with filters and grouping.
  • Track task changes of fields model, method, handle_jammed
  • Disable create/copy in the task form-view.

0.7

  • Task Reference, which serves:
    • Easier searching for tasks by Reference.
    • Check (ORM query) before call_task() call, to prevent redundant creation of recurring Pending tasks.

0.6

Introduction of "Jammed" tasks. A jammed task could be caused, for example by: a stopped/restarted Celery worker, message broker, or server.
  • Settings: to specify when a task (model, method) is Jammed after seconds from Started or Retry (states).
  • Jammed Tasks Report: which shows Jammed (not completed) tasks.
  • Cron or manually put tasks in "Jammed" state.
  • Ability to cancel (Pending or Jammed) tasks, which never completed.
  • Track and messaging about state change. Chatter on form view.

0.5

  • Routing tasks to specific (named) queues.

0.4

  • FIX: Store task state (Started, Retry) before execution.

0.3

0.2

  • Task state information.

0.1

  • Initial version.

Please log in to comment on this module

  • The author can leave a single reply to each comment.
  • This section is meant to ask simple questions or leave a rating. Every report of a problem experienced while using the module should be addressed to the author directly (refer to the following point).
  • If you want to start a discussion with the author, please use the developer contact information. They can usually be found in the description.
Please choose a rating from 1 to 5 for this module.
Odoo
Thank you for a useful module!
by on

Nice work.