Skip to Content
Menu
This question has been flagged
2 Replies
7040 Views

I actived the import option changed the file to csv it let me press import but it tells me to get a cup of coffee.. it has been saying this since 12pm it is now 3pm... what is wrong?

Avatar
Discard

I've seen about 20,000 records an hour get imported. Do you have more than 60,000 ?

Best Answer

Imports are slow. All of the server code that is necessary to create a new record in the standard interface is run for an import, you're just not loading up pages. I'm guessing offhand you're dealing with a CSV with tens of thousands of records? To make those imports not so painfully slow, you could use software like Talend to help write directly into the postgres database.

What I've been doing is writing up Python scripts to parse a CSV, then reformat the data/headers in a way the database can understand and just use insert statements with psycopg2 (Postgres Python library, used by openerp). Can get data into the database in seconds rather than hours, which means redoing the import to fix an error isn't so painful.

Avatar
Discard
Best Answer

Your import is slow if it handles 10 record per second; worst case scenario it shouldn't take longer than 1 record per second. It it does, something's wrong:

  • try with a small sample of record, just to confirm that it's not slowness.
  • have a look at the server log: there might be an error that not being passed on to the user interface, but is blocking the import.
Avatar
Discard

When I tried importing about 1100 products, it took a very consistent 7 minutes and 50 seconds, +/- 2 seconds, and that was with no errors. About 0.43 seconds to import a single record. Once you scale that up, it takes an hour to import less than 10,000 records.