Help

4

What is the best method to upload a large quantity of records?

Avatar
paul

What is the best method to upload a large quantity of records in version 7 on windows?

Overview:

  • I have ~500,000 new product records to upload
  • I have attempted the upload using the csv import process
  • But the application fails to complete the upload after providing the following status messages:
    • Loading
    • Still loading...be patient
    • Don't leave yet, it's still loading
    • You may not believe it, but the application is actually loading...
    • Take a minute to get a coffee, because it's loading
    • Maybe you should consider reloading the application by pressing F5...

Notes:

  • uploading smaller chunks of records is obviously doable but not very efficient
  • I let the process run over 12 hours
1 Comment
Avatar
Discard
Avatar
patrick
-

I don't know the speed of your machine, but I made a python script to read a CSV-file and use the openERP framework to import the data. This way, I know it is correctly imported and the triggers are processed. Maybe you can do the same thing on your machine. I do not know anything about the speed. Advantage of script is you can do additional checking (like line has been imported), so you can keep running the script if it fails during execution (for whatever reason).

3 Answers
0
Avatar
tlauber
Best Answer

We used Talend Open Studio to build a job which directly writes the database.

Avatar
Discard
0
Avatar
Gustavo
Best Answer

Another option is to upload the records with a Python script and web-services

Avatar
Discard
0
Avatar
Kevin McMenamin
Best Answer

we have used Pentaho as the ETL tool - loaded over 1M account_move_line records successfully.

Avatar
Discard