Skip to Content
This question has been flagged
importlimitopenerp
2 Replies
7990 Views
Avatar
Lars Erlandsen

Hi,

I have installed a copy of OpenERP 7 for evaluation (in ubuntu 12.04 LTS). By and large it looks like a definite improvement on OpenERP 6 :-) .

I tried to import a large number of leads from a csv file (18,000-odd). This failed miserably: A red bar with no text where I expect the "All records seem fine" green bar. By trimming it down to around 200 records or less per file, I have a reasonable chance of success with the upload. Except...

I have managed to upload around 3,000 records this way. However, I now get to the stage where I get all the right signs of success (green bar with "All records seem valid", followed by a successful import ending with a list of all the records imported so far. At this point, however, the record count doesn't increase. It is stuck at 3,162 records (confirmed with queries in the crm_leads table in the database itself). Also, whereas it would grind for a looong while with the validation and upload, it now comes back within a second or two. The log shows nothing abnormal (web requests stripped out below for clarity):

2013-05-20 17:24:44,678 25266 INFO testdb openerp.addons.base_import.models: done

2013-05-20 17:24:49,901 25266 INFO testdb openerp.addons.base_import.models: importing 199 rows...

2013-05-20 17:24:53,240 25266 INFO testdb openerp.addons.base_import.models: done

2013-05-20 17:27:17,227 25421 INFO testdb openerp.addons.base_import.models: importing 199 rows...

2013-05-20 17:27:20,555 25421 INFO testdb openerp.addons.base_import.models: done

2013-05-20 17:27:22,999 25421 INFO testdb openerp.addons.base_import.models: importing 199 rows...

2013-05-20 17:27:26,326 25421 INFO testdb openerp.addons.base_import.models: done

2013-05-20 17:28:39,683 25421 INFO testdb openerp.addons.base_import.models: importing 197 rows...

2013-05-20 17:28:43,192 25421 INFO testdb openerp.addons.base_import.models: done

2013-05-20 17:28:46,209 25421 INFO testdb openerp.addons.base_import.models: importing 197 rows...

2013-05-20 17:28:49,497 25421 INFO testdb openerp.addons.base_import.models: done

I have restarted postgres, restarted openerp server, tried with a different browser (chrome => firefox), but no difference has been observed.

Has anybody seen something similar..? Any built-in limits? Clues?

Thanks, Lars Erlandsen.

Avatar
Discard
Avatar
Lars Erlandsen
Author Best Answer

Sussed it in the end... :

Managed to get the import working fine from within a vmware host, but it failed on the live host (an identical Amazon AWS machine with the OpenERP service sitting behind an nginx proxy to have an https:// website with no native OpenERP://8069 port exposed). The difference? The proxy!! It looks like the proxy caches some of the earlier completion pages, and starts re-using them under certain circumstances. The request 'completes' immediately and the socket is closed before the OpenERP server has time to do the work, and the OpenERP server aborts the work and rolls back the work when the client aborts early.

I have turned off all caching on the nginx proxy, and so far so good...

Useful if anybody else gets into a similar situation.

Avatar
Discard
Mike Lindsay

I ran into something similar, but had to do with my nginx not allowing big enough files through. Sounds like you got it though!

Jeff Beidler

The guy that set up our machine also tweaked some Python config variables, I think. I don't know what or where those might be - I'm a PHP guy - but the equivalent would be 'upload_max_filesize' and 'PHP_max_execution_time'. (Just from memory... the actual names may be slightly different.)

Avatar
tim diamond
Best Answer

You just saved me hours of work! Thank you so much!

Avatar
Discard
Related Posts Replies Views Activity
How to import excel file in field?
import excel openerp
Avatar
Avatar
Avatar
2
Mar 15
13931
Big import reached the Odoo server limitation Solved
performance import limit cronjob
Avatar
Avatar
Avatar
Avatar
Avatar
4
Jun 25
4694
Data Import Doubt: How can we import datas into existing records in database?
import openerp7 openerp doubt
Avatar
Avatar
Avatar
2
Aug 20
6187
issues with 'Import error: No module named XXX'
installation module import openerp
Avatar
Avatar
1
Mar 15
10844
Import Contact Notes (Help! Which is the correct Odoo Field?) Solved
import
Avatar
Avatar
Avatar
2
Dec 25
212