Context:
Fresh Install of Odoo 9 using Official Docker Hub image and a standard Postgres Docker Image. We have this sitting behind an Nginx docker container, but otherwise as per the standard odoo docker image guidelines.
Problem:
We keep having rendered files disappear from the filestore that are crucial to the website functioning (odoo common assets scripts). The logs show this:
2015-10-30 15:28:28,266 1 INFO db-test werkzeug: 172.17.0.25 - - [30/Oct/2015 15:28:28] "GET /web/content/407-17599c5/website.assets_frontend.js HTTP/1.0" 200 -
2015-10-30 15:28:28,281 1 INFO db-test openerp.addons.base.ir.ir_attachment: _read_file reading /var/lib/odoo/filestore/db-test/e6/e69e06808b908fc0d85ebfea58fbc7df3788e72e
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/openerp/addons/base/ir/ir_attachment.py", line 151, in _file_read
r = open(full_path,'rb').read().encode('base64')
IOError: [Errno 2] No such file or directory: u'/var/lib/odoo/filestore/db-test/e6/e69e06808b908fc0d85ebfea58fbc7df3788e72e'
This happens with various filenames (generated hash values), but all of them are for the same actual file content which is related to script assets for the website. The content changes slightly as Apps are installed, which forces a change of the filename.
This happens frequently, e.g. within an hour of installation.
I've found a number of log entries for ir.attachments records being deleted, I assume as part of a housekeeping task that occurs at the end of a user action.
2015-10-31 13:34:07,916 1 INFO db-test openerp.models.unlink: User #1 deleted ir.attachment records with IDs: [284]
2015-10-31 13:34:09,034 1 INFO db-test openerp.models.unlink: User #1 deleted ir.attachment records with IDs: [285]
This is not initiated by a user (i.e. me) specifically, but happens regularly even when no missing file issue occurs. However it does happen near the time of the missing file (coincidental or not), so thought it worth mentioning.
The result of this is that the website and entire app becomes unusable - on the front end, the page will show from a specific url, but no menus will activate, on the backend, obviously nothing works, as the menu and content are brought back via XHR. This obviously renders Odoo completely unusable unless this file is restored.
So Far...
We tried this with a fresh install (no configuration), just apps installed.
We've eliminated a DB issue - this is a fresh install from the official postgres docker image, set-up as per odoo instructions, and has worked fine for v8.
We've eliminated any backup, restore and migrate jobs we had set-up, as these have not been used on this fresh install
We doubt it is to do with the EBS backed AWS EC2 Instance or CentOS or Docker on LVM, simply because it is the same file (although with different file names) each time, even when re-installed on a completely different server and disk and docker storage configuration.
Checked the automated jobs, but none seem to affect this type of thing, and running some of them manually didn't effect the file in this way
It only happens when the website builder is installed.
It only happens when the website is publicly accessible - when firewalled off just to me, this DOES NOT happen.
This strongly suggests that a request of some sort is triggering something in Odoo that triggers the deletion of the file, or the update/refresh of the file, which doesn't work / complete properly.
HELP NEEDED: Not being intimately familiar with Odoo architecture or codebase, or being a python developer, I would appreciate pointers to how the resource request pipeline in odoo actually works (so I can trace the code path) and what related functionality could possibly do this.
Any Ideas would be greatly appreciated - we are almost at the point of giving up on Odoo, as it is just not stable enough to run a business on given this issue.
So far I've not been able to replicate the problem, when the website is not public facing, or is on a domain that has never been used before (and therefore is never accessed by anything else). Could it be that something like the google bot is requesting an old page (I've seen a number of 404s and 304s in the logs), that this triggers some kind of housekeeping that inadvertently results in deleting this file?
If anyone has any ideas on this it would be greatly appreciated.