Community archivos de la lista de correo

community@mail.odoo.com

Avatar

Re: wkhtmltopdf severe memory leak

por
Axel Mendoza
- 03/05/2017 15:47:02
Interesting thread here, I also have noticed some others performance issues related to odoo and wkhtmltopdf that was reported to Odoo here:

Mostly related to some blocking issues when there is a huge concurrency (>1300) against the print service using wkhtmltopdf, there is some bad legacy code that block transactions when that kind of concurrency level is reached at least at the scale of that specific deploy that has been configured with a large amount of resources due to the concurrency level that reach the lock in was just around the 30% of the expected. Could be happening too at a lower scale with less resources. You could check the if you have this kind of issues if on your database there are some long transactions blocked or delayed that are related to some delayed wkhtmltopdf process running without ending, perhaps due to some GIL lock since the report service use a thread(useless) to wait until the report generation is finished

Maybe None of this is related to your issue but it's worth to tell.

Best Regards


On Wed, May 3, 2017 at 9:30 AM, Brahim EL ALLIOUI <belallioui@gmail.com> wrote:
Hi,

Is there a solution under windouws since the workers do not work under windows

Best,


Brahim EL ALLIOUI
General Manager
Skype : belallioui
Mobile : +212661940077


2016-07-30 8:07 GMT+00:00 Sébastien Alix <sebastien.alix@osiell.com>:
Hi, We developed a module based on the OCA/connector framework called 'mass_reporting' to generate and assemble reports in different formats, e.g: on 'account.invoice', assemble a letter + the invoice in one PDF, using 'ir.filters' to fetch the records. Each PDF can then be merged together in one PDF, or in a ZIP file, or whatever (you can develop your own "generator" to do what you want). We are using it to produce ~4000 invoices in less than 2h for a customer project. It is not totally finished but we wanted to publish it to OCA/reporting-engine. Without it, the memory of wkhtmltopdf exploded for such large volume of reports. Regards, Le 29/07/2016 à 01:27, Graeme Gellatly a écrit :
> We already do that, right down to specifying destination printers / > instances based on model and company rules, reprinting etc. It's pretty > rough but planning to migrate to OCA connector in next upgrade cycle. > Just run the cron every 2 minutes and aside from the occasional cups > crash has worked perfectly for 4 years. Not in a publishable state but > only 150 loc. Designed more for integrating printing into workflow. > > > On Fri, 29 Jul 2016 10:57 AM Naran M <nmoturi@ru3ix.com.au > <mailto:nmoturi@ru3ix.com.au>> wrote: > > +1, having the option to report generation queue would be great. > > On Fri, Jul 29, 2016 at 8:22 AM, Daniel Reis <dreis.pt@hotmail.com > <mailto:dreis.pt@hotmail.com>> wrote: > > As a future improvement idea, delegating the pdf generation to > an async cron job could mitigate this issue. > > Report generation queues is something you can find in ERPs like > SAP or Oracle Financials. > > -- > Daniel > > No dia 28/07/2016, às 22:52, David Arnold <dar@devco.co > <mailto:dar@devco.co>> escreveu: > >> pasted1 >> >> This is what has been caused by wkhtmltopdf on one of our >> smaller instances (4 GB on Google Cloud). >> >> El jue., 28 jul. 2016 a las 16:37, David Arnold (<dar@devco.co >> <mailto:dar@devco.co>>) escribió: > >> Hi, >> >> thanks for this valuable input! On the linked issue, it >> has been reported that 16GB doesn't cause problems as it >> is presumably sufficient to buffer this extraordinary >> consumption. Also what you describe about >> the *limit_memory_hard* parameter to 2GB and previous >> errors could be interpreted as coinciding with the >> reported increase in memory demand. >> >> I suppose it would be wise to state, >> that outsourcing wkhtmltopdf to another service is >> probably a good idea. >> >> If all conclusions are right, then your setup could >> possibly explode when all 10 odoo workers >> get increased load from this wkhtmltopdf problem >> at around the same time, assuming that the memory demand >> of this malicious wkhtmltopdf process would be more than >> 16GB This is extremely unlikely, indeed. >> >> Assuming that you are on 0.12.1, this would then indicate >> that 0.12.1 has the same problem. This also coincides with >> the issue the wkhtmltopdf maintainer linked in my reported >> issue, which was going on for 2 years. >> >> Any further reports on this are helpful. Thanks! >> >> as to wkhtmltopdf, iirc: a patched version is required >> which is satisfied by either 0.12.1 or 0.12.3, yet NOT by >> 0.12.2 >> https://github.com/wkhtmltopdf/wkhtmltopdf/issues/2083#issuecomment-114045190 >> Therefore I think, this is a different beast. >> >> Best, David >> >> El jue., 28 jul. 2016 a las 15:53, Ray Carnes >> (<ray.carnes@bistasolutions.com >> <mailto:ray.carnes@bistasolutions.com>>) escribió: > >> In our most recent deployment: >> >> >> >> 8 CPU’s >> >> 2 Threads per core >> >> 4 Cores per socket >> >> 1 Socket 1 >> >> 2,900 Mhz CPU >> >> 16GB RAM >> >> >> >> 10 Odoo workers >> >> >> >> Initially we ran into occasional error -11 with PDF >> generation. >> >> >> >> After increasing the worker * limit_memory_hard* >> parameter to 2GB we have had no further reports of any >> problems. >> >> >> >> For those who don’t know, Odoo recommends 0.12.1 of >> wkhtmltopdf as documented at >> https://www.odoo.com/documentation/9.0/setup/install.html >> >> >> >> Ray. >> >> >> >> *From:*David Arnold [mailto:dar@devco.co >> <mailto:dar@devco.co>] >> *Sent:* Thursday, July 28, 2016 1:27 PM >> *To:* Community <community@mail.odoo.com >> <mailto:community@mail.odoo.com
>> >> *Subject:* wkhtmltopdf severe memory leak >> >> >> >> Hi, >> >> >> >> we have experienced the following issue: >> https://github.com/wkhtmltopdf/wkhtmltopdf/issues/2950
>> >> In short wkhtml explodes to 1GB+ memory consumption in >> less then our metrics resolution (2s). >> >> That should worry anyone who runs wkhtmltopdf on the >> same instance as odoo runs and would kill every >> averagely scoped server instance under otherwise >> normal load. >> >> Be warned! >> Share your experience with this please! >> >> If I remeber correctly (I'm under way on cellphone): >> we are on wkhtmltopdf 0.12.3 >> the maintainer suggests to try 0.13 alpha >> >> I'll keep you updates once further findings drop in. >> >> Best, Regards >> David >> >> _______________________________________________ >> >> >> Mailing-List: https://www.odoo.com/groups/community-59 >> Post to: mailto:community@mail.odoo.com >> Unsubscribe: https://www.odoo.com/groups?unsubscribe >> >> _______________________________________________ >> Mailing-List: https://www.odoo.com/groups/community-59 >> Post to: mailto:community@mail.odoo.com >> <mailto:community@mail.odoo.com> >> Unsubscribe: https://www.odoo.com/groups?unsubscribe >> >> _______________________________________________ >> Mailing-List: https://www.odoo.com/groups/community-59 >> Post to: mailto:community@mail.odoo.com >> Unsubscribe: https://www.odoo.com/groups?unsubscribe >> >>
> > _______________________________________________ > Mailing-List: https://www.odoo.com/groups/community-59 > Post to: mailto:community@mail.odoo.com > <mailto:community@mail.odoo.com> > Unsubscribe: https://www.odoo.com/groups?unsubscribe > > _______________________________________________ > Mailing-List: https://www.odoo.com/groups/community-59
> Post to: mailto:community@mail.odoo.com <mailto:community@mail.odoo.com> > Unsubscribe: https://www.odoo.com/groups?unsubscribe > > _______________________________________________ > Mailing-List: https://www.odoo.com/groups/community-59 > Post to: mailto:community@mail.odoo.com > Unsubscribe: https://www.odoo.com/groups?unsubscribe >
-- Sébastien ALIX ABF Osiell

_______________________________________________
Mailing-List: https://www.odoo.com/groups/community-59
Post to: mailto:community@mail.odoo.com
Unsubscribe: https://www.odoo.com/groups?unsubscribe


_______________________________________________
Mailing-List: https://www.odoo.com/groups/community-59
Post to: mailto:community@mail.odoo.com
Unsubscribe: https://www.odoo.com/groups?unsubscribe


Referencia