Persistent Real-time Chat (Longpolling) & ir_cron Lock Errors

Subject: Odoo 18 Real-time Chat Not Working (Requires Refresh) & Recurring ir_cron Lock Errors in Multi-DB Docker Setup

Odoo Version: 18.0 (Community Edition) Deployment: Docker Compose on Ubuntu Server 20.04.6 LTS Reverse Proxy: Caddy Database: PostgreSQL 15 (Docker container) Issue Type: Real-time communication (longpolling/websockets) and database contention (ir_cron locks)

Problem Description:

We are experiencing two persistent and critical issues with our Odoo 18 setup, which runs multiple Odoo databases (one per domain) on a single live18-odoo Docker container connected to a shared live18-db PostgreSQL container:

1. OdooBot/Discuss Real-time Chat Not Working:
   • When chatting with OdooBot or in Discuss channels, replies and new messages do not appear in real-time. The page must be manually refreshed to see new content.
   • Direct access via http://144.126.135.5:18069 (bypassing Caddy/SSL) still shows "Real-time connection lost...", confirming the issue is internal to Odoo.
2. Recurring ir_cron Database Lock Errors (Root Cause Identified):
   • We frequently observe ERROR: could not obtain lock on row in relation "ir_cron" in our PostgreSQL logs, specifically for id IN (6) ("Mail: Fetchmail Service").
   • CRITICAL NEW FINDING: Odoo logs also show RuntimeError: OrderedDict mutated during iteration. This error occurs when Odoo's cron system tries to loop through its list of active databases, but that list unexpectedly changes or becomes unstable. This is the root cause of the ir_cron locking errors and indicates fundamental instability in Odoo's multi-database management for cron jobs.
   • This database registry instability leads to cron job failures, memory spikes, and general Odoo unresponsiveness.

Server and Odoo Configuration Details:

• Server IP: 144.126.135.5
• Odoo Docker Compose (~/docker-apps/odoo18/live18/docker-compose.yml): Standard setup with Odoo 18, PostgreSQL 15, exposing ports 18069 (main) and 18072 (longpolling). Uses ODOO_PROXY_MODE=True and ODOO_DB_FILTER=^%h$.
• Odoo Configuration (~/docker-apps/odoo18/live18/config/odoo.conf): log_level = debug, longpolling_port = 8072, workers = 4, max_cron_threads = 2, proxy_mode = True, dbfilter = ^%h$. web.base.url is not frozen.
• Caddy Configuration: Standard setup for Odoo 18, proxying to 18069 and 18072 with necessary headers for websockets/longpolling.
• Firewall (UFW): Ports 80, 443, 18069, 18072 are ALLOW IN. Mail ports (993, 995, 465, 587) are ALLOW OUT.
• Cloudflare: A/CNAME records for web traffic are Proxied (orange cloud). Mail-related records are DNS Only (grey cloud). SSL/TLS is "Full". No custom WAF rules.

[options]
admin_passwd = ##################
; # Database connection details are set via environment variables in docker-compose.yml
; # Addons paths - Explicitly list subdirs like old server + Odoo standard path
addons_path = /mnt/extra-addons, /mnt/extra-addons/odooapps/, /mnt/extra-addons/industry/, /mnt/extra-addons/CybroAddons/, /mnt/extra-addons/social,/usr/lib/python3/dist-packages/odoo/addons
http_timeout = 1800
logfile = /var/log/odoo/odoo-server.log
logrotate = True
log_level = debug
longpolling_port = 8072
session_store = db
workers = 4
max_cron_threads = 2
proxy_mode = True
dbfilter = ^%h$
; # Database Connection Pool
;db_maxconn = 95
;web.base.url = https://coqui.cloud  ; <--- ADD/ENSURE THIS LINE (use your primary domain)
;web.base.url.freeze = True

Troubleshooting Steps Attempted and Ruled Out:

• Caddy Configuration: Extensive testing and multiple Caddyfile iterations (including adding timeouts, explicit websocket directives) did not resolve the longpolling issue and sometimes caused Caddy to crash. Ruled out as primary cause.
• Browser Caching: Hard refresh and clearing browser cache/cookies. Ruled out.
• SSL/TLS Issues (from Caddy or Cloudflare): Direct HTTP access to Odoo's port 18069 still shows the longpolling issue. Ruled out.
• Nginx/Nginx Proxy Manager: Attempts to switch were unsuccessful due to setup issues. Ruled out for now.
• Odoo 15 Instances: Odoo 15 instances with similar Caddy configs work perfectly for real-time chat. Suggests issue is specific to Odoo 18.

Remaining Unresolved Issues & Hypotheses:

1. RuntimeError: OrderedDict mutated during iteration (Root Cause of Cron Instability):
   • This is the most critical server-side error. It's a fundamental issue in Odoo's multi-database registry management.
   • Hypothesis: Likely a bug in Odoo 18's core or a specific module interaction causing database registry instability when cron threads iterate.
   • Specific Symptom: "Multiple password values not supported" from mail server logs for Fetchmail authentication, even after manual password re-typing and with empty mailboxes. This indicates Fetchmail is failing in an unusual way.
2. Odoo 18 Real-time Chat Issue (Longpolling):
   • Hypothesis: Problem is definitively internal to Odoo 18's core longpolling/bus mechanism. Could be a subtle bug in Odoo 18's real-time stack, a conflict with a specific Odoo 18 module, or an interaction with its worker management when dbfilter is active. Logs at debug level show no errors, making diagnosis difficult.

Questions for the Community:

1. For the RuntimeError: OrderedDict mutated during iteration: What are common causes and fixes for this specific Odoo error in multi-database setups (especially with dbfilter=^%h$)? Are there known Odoo 18 bugs or specific module interactions that cause this?
2. For the Odoo 18 Real-time Chat issue: Given it's internal to Odoo and not Caddy/network, what are common causes for "Real-time connection lost..." when longpolling_port is listening and debug logs show no errors? Are there specific Odoo 18 modules known to interfere with bus or longpolling?
3. For "Multiple password values not supported" (from mail server): What could cause Odoo 18 to send multiple password values during IMAP/POP3 authentication, even after manual re-typing and with simple passwords?

Any help or insights would be greatly appreciated. Thank you!