Odoo Help

Welcome!

This community is for beginners and experts willing to share their Odoo knowledge. It's not a forum to discuss ideas, but a knowledge base of questions and their answers.

0

Import products in OpenERP

By
Ayyappan
on 7/20/14, 12:53 PM 807 views

I am trying to import 30,000 products in openerp as well as I have to import category, products related supplier also. For this process I have made an addon. But it takes more than 7hours to import. I have used ORM objects only. 

Note : I have a lot of validation while importing like as file value checking, special character removing and etc... That's why I have created the addon.

Is any otherway to reduce the timing and increase the performance? I have to do more validation on my script otherwise I will use openerp import functionality itself or some other tools. Please give me instruction to increase the performance.

0
Michael St-Georges
On 7/20/14, 9:18 PM

I would go for your custom module. The standard import functions might take even longer than your custom module as they process many different cases. The only things that MIGHT help is multi-threading your custom module or optimizing code/server configuration.

Thanks for your advice Michael. In my code i have checked every single line of the csv or xls file and make it as list of dictionaries and check the conditions and import single product after importing the single product that i am importing the product related suppliers and then insert another product. If we insert the product one by one, is it taking time to import? Can i use the queries rather than ORM functionality?

Ayyappan
on 7/21/14, 1:43 AM

While it is certainly possible to use raw querries, using ORM objects is preferable as there might be some unforseen impacts when manipulating the DB directly (Unknown impacts, different modules, etc...). There might also be future changes that might break your import method. However, if you intend to go forward with this kind of scenario, here is how I would suggest you proceed: - Backup database - Import a few objects using ORM method (try to pick a set that goes trought all edge cases you could think of) - Backup this version - Rollback to original backup - Import the same objects using database method - Backup this version - Compare the results from both import methods Doing so, ensures you process the objects closely enough to the original ORM model and that you haven't forgotten anything in the process. The pgsql backups being text based SQL dumps, they sould be easy enough to compare.

Michael St-Georges
on 7/21/14, 5:57 PM
0

Ludo - Neobis

--Ludo - Neobis--
2387
| 2 2 5
Hoogvliet, Netherlands
--Ludo - Neobis--

Developer

Ludo - Neobis
On 7/21/14, 3:26 AM

Maybe you could try to sanitize the data before inserting it into OpenERP? If you can run a script that validates the records and then inserts them using simple "create" method calls on XML-RPC it should take you no more than an hour or so.

Your Answer

Please try to give a substantial answer. If you wanted to comment on the question or answer, just use the commenting tool. Please remember that you can always revise your answers - no need to answer the same question twice. Also, please don't forget to vote - it really helps to select the best questions and answers!

About This Community

This community is for professionals and enthusiasts of our products and services. Read Guidelines

Question tools

1 follower(s)

Stats

Asked: 7/20/14, 12:53 PM
Seen: 807 times
Last updated: 3/16/15, 8:10 AM