Community mailing list archives

community@mail.odoo.com

Avatar

Re: wkhtmltopdf severe memory leak

by
e-acta, Brahim EL ALLIOUI
- 05/03/2017 10:26:43
Hi,

Is there a solution under windouws since the workers do not work under windows

Best,


Brahim EL ALLIOUI
General Manager
Skype : belallioui
Tel : +212535749410
Mobile : +212661940077


2016-07-30 8:07 GMT+00:00 Sébastien Alix <sebastien.alix@osiell.com>:
Hi,

We developed a module based on the OCA/connector framework called 
'mass_reporting' to generate and assemble reports in different formats, 
e.g: on 'account.invoice', assemble a letter + the invoice in one PDF, 
using 'ir.filters' to fetch the records. Each PDF can then be merged 
together in one PDF, or in a ZIP file, or whatever (you can develop your 
own "generator" to do what you want).

We are using it to produce ~4000 invoices in less than 2h for a customer 
project. It is not totally finished but we wanted to publish it to 
OCA/reporting-engine.
Without it, the memory of wkhtmltopdf exploded for such large volume of 
reports.

Regards,


Le 29/07/2016 à 01:27, Graeme Gellatly a écrit :
> We already do that, right down to specifying destination printers /
> instances based on model and company rules,  reprinting etc. It's pretty
> rough but planning to migrate to OCA connector in next upgrade cycle.
> Just run the cron every 2 minutes and aside from the occasional cups
> crash has worked perfectly for 4 years. Not in a publishable state but
> only 150 loc. Designed more for integrating printing into workflow.
>
>
> On Fri, 29 Jul 2016 10:57 AM Naran M <nmoturi@ru3ix.com.au
> <mailto:nmoturi@ru3ix.com.au>> wrote:
>
>     +1, having the option to report generation queue would be great.
>
>     On Fri, Jul 29, 2016 at 8:22 AM, Daniel Reis <dreis.pt@hotmail.com
>     <mailto:dreis.pt@hotmail.com>> wrote:
>
>         As a future improvement idea, delegating the pdf generation to
>         an async  cron job could mitigate this issue.
>
>         Report generation queues is something you can find in ERPs like
>         SAP or Oracle Financials.
>
>         --
>         Daniel
>
>         No dia 28/07/2016, às 22:52, David Arnold <dar@devco.co
>         <mailto:dar@devco.co>> escreveu:
>
>>         pasted1
>>
>>         This is what has been caused by wkhtmltopdf on one of our
>>         smaller instances (4 GB on Google Cloud).
>>
>>         El jue., 28 jul. 2016 a las 16:37, David Arnold (<dar@devco.co
>>         <mailto:dar@devco.co>>) escribió:
>
>>             Hi,
>>
>>             thanks for this valuable input! On the linked issue, it
>>             has been reported that 16GB doesn't cause problems as it
>>             is presumably sufficient to buffer this extraordinary
>>             consumption. Also what you describe about
>>             the *limit_memory_hard* parameter to 2GB and previous
>>             errors could be interpreted as coinciding with the
>>             reported increase in memory demand.
>>
>>             I suppose it would be wise to state,
>>             that outsourcing wkhtmltopdf to another service is
>>             probably a good idea.
>>
>>             If all conclusions are right, then your setup could
>>             possibly explode when all 10 odoo workers
>>             get increased load from this wkhtmltopdf problem
>>             at around the same time, assuming that the memory demand
>>             of this malicious wkhtmltopdf process would be more than
>>             16GB This is extremely unlikely, indeed.
>>
>>             Assuming that you are on 0.12.1, this would then indicate
>>             that 0.12.1 has the same problem. This also coincides with
>>             the issue the wkhtmltopdf maintainer linked in my reported
>>             issue, which was going on for 2 years.
>>
>>             Any further reports on this are helpful. Thanks!
>>
>>             as to wkhtmltopdf, iirc: a patched version is required
>>             which is satisfied by either 0.12.1 or 0.12.3, yet NOT by
>>             0.12.2
>>             https://github.com/wkhtmltopdf/wkhtmltopdf/issues/2083#issuecomment-114045190
>>             Therefore I think, this is a different beast.
>>
>>             Best, David
>>
>>             El jue., 28 jul. 2016 a las 15:53, Ray Carnes
>>             (<ray.carnes@bistasolutions.com
>>             <mailto:ray.carnes@bistasolutions.com>>) escribió:
>
>>                 In our most recent deployment:
>>
>>
>>
>>                 8 CPU’s
>>
>>                 2 Threads per core
>>
>>                 4 Cores per socket
>>
>>                 1 Socket             1
>>
>>                 2,900 Mhz CPU
>>
>>                 16GB RAM
>>
>>
>>
>>                 10 Odoo workers
>>
>>
>>
>>                 Initially we ran into occasional error -11 with PDF
>>                 generation.
>>
>>
>>
>>                 After increasing the worker * limit_memory_hard*
>>                 parameter to 2GB we have had no further reports of any
>>                 problems.
>>
>>
>>
>>                 For those who don’t know, Odoo recommends 0.12.1 of
>>                 wkhtmltopdf as documented at
>>                 https://www.odoo.com/documentation/9.0/setup/install.html
>>
>>
>>
>>                 Ray.
>>
>>
>>
>>                 *From:*David Arnold [mailto:dar@devco.co
>>                 <mailto:dar@devco.co>]
>>                 *Sent:* Thursday, July 28, 2016 1:27 PM
>>                 *To:* Community <community@mail.odoo.com
>>                 <mailto:community@mail.odoo.com>>
>>                 *Subject:* wkhtmltopdf severe memory leak
>>
>>
>>
>>                 Hi,
>>
>>
>>
>>                 we have experienced the following issue:
>>                 https://github.com/wkhtmltopdf/wkhtmltopdf/issues/2950
>> >> In short wkhtml explodes to 1GB+ memory consumption in >> less then our metrics resolution (2s). >> >> That should worry anyone who runs wkhtmltopdf on the >> same instance as odoo runs and would kill every >> averagely scoped server instance under otherwise >> normal load. >> >> Be warned! >> Share your experience with this please! >> >> If I remeber correctly (I'm under way on cellphone): >> we are on wkhtmltopdf 0.12.3 >> the maintainer suggests to try 0.13 alpha >> >> I'll keep you updates once further findings drop in. >> >> Best, Regards >> David >> >> _______________________________________________ >> >> >> Mailing-List: https://www.odoo.com/groups/community-59 >> Post to: mailto:community@mail.odoo.com >> Unsubscribe: https://www.odoo.com/groups?unsubscribe >> >> _______________________________________________ >> Mailing-List: https://www.odoo.com/groups/community-59 >> Post to: mailto:community@mail.odoo.com >> <mailto:community@mail.odoo.com> >> Unsubscribe: https://www.odoo.com/groups?unsubscribe >> >> _______________________________________________ >> Mailing-List: https://www.odoo.com/groups/community-59 >> Post to: mailto:community@mail.odoo.com >> Unsubscribe: https://www.odoo.com/groups?unsubscribe >> >>
> > _______________________________________________ > Mailing-List: https://www.odoo.com/groups/community-59 > Post to: mailto:community@mail.odoo.com > <mailto:community@mail.odoo.com> > Unsubscribe: https://www.odoo.com/groups?unsubscribe > > _______________________________________________ > Mailing-List: https://www.odoo.com/groups/community-59
> Post to: mailto:community@mail.odoo.com <mailto:community@mail.odoo.com> > Unsubscribe: https://www.odoo.com/groups?unsubscribe > > _______________________________________________ > Mailing-List: https://www.odoo.com/groups/community-59 > Post to: mailto:community@mail.odoo.com > Unsubscribe: https://www.odoo.com/groups?unsubscribe >
-- Sébastien ALIX ABF Osiell

_______________________________________________
Mailing-List: https://www.odoo.com/groups/community-59
Post to: mailto:community@mail.odoo.com
Unsubscribe: https://www.odoo.com/groups?unsubscribe


Reference