Odoo Help

Welcome!

This community is for beginners and experts willing to share their Odoo knowledge. It's not a forum to discuss ideas, but a knowledge base of questions and their answers.

1

Import csv leads data stops accepting further records...

By
Lars Erlandsen
on 5/20/13, 4:01 PM 2,350 views

Hi,

I have installed a copy of OpenERP 7 for evaluation (in ubuntu 12.04 LTS). By and large it looks like a definite improvement on OpenERP 6 :-) .

I tried to import a large number of leads from a csv file (18,000-odd). This failed miserably: A red bar with no text where I expect the "All records seem fine" green bar. By trimming it down to around 200 records or less per file, I have a reasonable chance of success with the upload. Except...

I have managed to upload around 3,000 records this way. However, I now get to the stage where I get all the right signs of success (green bar with "All records seem valid", followed by a successful import ending with a list of all the records imported so far. At this point, however, the record count doesn't increase. It is stuck at 3,162 records (confirmed with queries in the crm_leads table in the database itself). Also, whereas it would grind for a looong while with the validation and upload, it now comes back within a second or two. The log shows nothing abnormal (web requests stripped out below for clarity):

2013-05-20 17:24:44,678 25266 INFO testdb openerp.addons.base_import.models: done

2013-05-20 17:24:49,901 25266 INFO testdb openerp.addons.base_import.models: importing 199 rows...

2013-05-20 17:24:53,240 25266 INFO testdb openerp.addons.base_import.models: done

2013-05-20 17:27:17,227 25421 INFO testdb openerp.addons.base_import.models: importing 199 rows...

2013-05-20 17:27:20,555 25421 INFO testdb openerp.addons.base_import.models: done

2013-05-20 17:27:22,999 25421 INFO testdb openerp.addons.base_import.models: importing 199 rows...

2013-05-20 17:27:26,326 25421 INFO testdb openerp.addons.base_import.models: done

2013-05-20 17:28:39,683 25421 INFO testdb openerp.addons.base_import.models: importing 197 rows...

2013-05-20 17:28:43,192 25421 INFO testdb openerp.addons.base_import.models: done

2013-05-20 17:28:46,209 25421 INFO testdb openerp.addons.base_import.models: importing 197 rows...

2013-05-20 17:28:49,497 25421 INFO testdb openerp.addons.base_import.models: done

I have restarted postgres, restarted openerp server, tried with a different browser (chrome => firefox), but no difference has been observed.

Has anybody seen something similar..? Any built-in limits? Clues?

Thanks, Lars Erlandsen.

1
Lars Erlandsen
On 5/22/13, 5:15 PM

Sussed it in the end... :

Managed to get the import working fine from within a vmware host, but it failed on the live host (an identical Amazon AWS machine with the OpenERP service sitting behind an nginx proxy to have an https:// website with no native OpenERP://8069 port exposed). The difference? The proxy!! It looks like the proxy caches some of the earlier completion pages, and starts re-using them under certain circumstances. The request 'completes' immediately and the socket is closed before the OpenERP server has time to do the work, and the OpenERP server aborts the work and rolls back the work when the client aborts early.

I have turned off all caching on the nginx proxy, and so far so good...

Useful if anybody else gets into a similar situation.

I ran into something similar, but had to do with my nginx not allowing big enough files through. Sounds like you got it though!

Mike Lindsay
on 5/22/13, 6:56 PM

The guy that set up our machine also tweaked some Python config variables, I think. I don't know what or where those might be - I'm a PHP guy - but the equivalent would be 'upload_max_filesize' and 'PHP_max_execution_time'. (Just from memory... the actual names may be slightly different.)

Jeff Beidler
on 10/23/14, 2:57 PM
0
tim diamond
On 7/3/14, 3:15 PM

You just saved me hours of work! Thank you so much!

Your Answer

Please try to give a substantial answer. If you wanted to comment on the question or answer, just use the commenting tool. Please remember that you can always revise your answers - no need to answer the same question twice. Also, please don't forget to vote - it really helps to select the best questions and answers!

About This Community

This community is for professionals and enthusiasts of our products and services. Read Guidelines

Question tools

1 follower(s)

Stats

Asked: 5/20/13, 4:01 PM
Seen: 2350 times
Last updated: 3/16/15, 8:10 AM