We have observed one problem in Postgresql as it doesn't uses multi core of CPU for single query. For example, I have 8 cores in cpu. We are having 40 Million entries in stock.move table. When we apply massive query in single database connection to generate reporting & observe at backend side, we see only one core is 100% used, where as all other 7 are free. Due to that query execution time takes so longer and our odoo system being slow. Whereas problem is inside postgresql core. If by anyhow we can share a query between two or more cores than we can get performance boost in postgresql query execution.
I am sure by solving parallel query execution, we can make Odoo performance even faster. Anyone has any kind of suggestions regarding this ?
------------------- * Editing this question to show you answer from Postgresql Core committee *------------------------------------
Here I am posting the answer which I got from one of top contributor of Postgresql database. ( I hope this information will be useful)
It is expected behave. PostgreSQL doesn't support parallel CPU for single query. This topic is under high development, and probably, this feature will be in planned release 9.6 ~ September 2016.
But table with 40M rows isn't too big, so probably more CPU should not too help to you (there is some overhead with start and processing multi CPU query). You have to use some usual tricks like materialized view, preagregations, ... the main idea of these tricks - don't try to repeat often same calculation. Check health of PostgreSQL - indexes, vacuum processing, statistics,.. Check hw - speed of IO. Check PostgreSQL configuration - shared_buffers, work_mem. Some queries can be slow due bad estimations - check a explain of slow queries.
There are some tools that can breaks some query to more queries and start parallel execution, but I didn't use it.
In my experience if you create and optimize indexes for your queries and manually test to ensure that PostgreSQL is indeed using your indexes and also giving you a quick response, you would see those queries still slow in Odoo. That's because you also need to take "order by" into account, I face this issue in some large tables and a quick solution was disable ordering for those results and the performance issues was fixed.
It would be helpfull if You post the typical resultset recordcount of thoose problematic queries. If there are thousand of records then the next big obstacle is the Odoo ORM object materialisation layer. The ORM is several magnitude slower than raw query execution, so you can optimise the performance with hand tuned queries issued through the cursor and with hand tuned object materialisation.
I can foresee that a simple recordset.browse(condition) can be huge bottleneck with several thousand records in resultset.
"How does PostgreSQL use CPU resources?
The PostgreSQL server is process-based (not threaded), and uses one operating system process per database session. A single database session (connection) cannot utilize more than one CPU. Of course, multiple sessions are automatically spread across all available CPUs by your operating system. Client applications can easily use threads and create multiple database connections from each thread.
A single complex and CPU-intensive query is unable to use more than one CPU to do the processing for the query. The OS may still be able to use others for disk I/O etc, but you won't see much benefit from more than one spare core."
You need multiple sessions connection to utilize multi-core CPU
Please try to give a substantial answer. If you wanted to comment on the question or answer, just use the commenting tool. Please remember that you can always revise your answers - no need to answer the same question twice. Also, please don't forget to vote - it really helps to select the best questions and answers!
About This Community
|Asked: 7/1/15, 7:24 AM|
|Seen: 3975 times|
|Last updated: 9/24/15, 2:24 AM|