This question has been flagged
2 Replies
14834 Views

[UPDATE 2015-03-01]

This issue has been fixed in the following PR:

https://github.com/odoo/odoo/commit/68f14c68709bbb50cb7fb66d288955e1d769c5ff


I've been trying to figure out a way to import product images effectively into our database and have been running into the same issues as most in this forum (ie. red status notification with no text inside). I therefore slowly started trying to validate imports using larger product field sets and pictures every time (started with 5 and moved on to 20 at a time with larger images).

I then came across this error:

Import preview failed due to: field larger than field limit (131072). The issue is usually an incorrect file encoding.

I have googled the error and come across this article regarding the error in python:

http://lethain.com/handling-very-large-csv-and-xml-files-in-python/

and a possible solution in this stackoverflow post:

http://stackoverflow.com/questions/15063936/csv-error-field-larger-than-field-limit-131072

I've then backed up the following file:

/opt/odoo/odoo-server/addons/base_import/models.py

and added the following to the top of the file:

import sys
import csv
maxInt = sys.maxsize
decrement = True

while decrement:
    # decrease the maxInt value by factor 10 
    # as long as the OverflowError occurs.

    decrement = False
    try:
        csv.field_size_limit(maxInt)
    except OverflowError:
        maxInt = int(maxInt/10)
        decrement = True

and then I have restarted the server. I understand that modifiying core code is not a smart idea, so if it works I will post back here as an answer to confirm, however does anyone know how this might be implemented as a module to override the default setting rather than modifying the core files?

Avatar
Discard
Author Best Answer

The fix above definitely works. I have just finished an import with thousands of large images inside a CSV.

  1. make a backup of /opt/odoo/odoo-server/addons/base_import/models.py (or whereever /addons/base_import/ is located in your instance).
  2. add the following lines at the very top of the file:
    1. import sys
      import csv
      maxInt = sys.maxsize
      decrement = True

      while decrement:
          # decrease the maxInt value by factor 10 
          # as long as the OverflowError occurs.

          decrement = False
          try:
              csv.field_size_limit(maxInt)
          except OverflowError:
              maxInt = int(maxInt/10)
              decrement = True

  3. Restart the odoo server.

The issue was resolved due to this stackoverflow post:

http://stackoverflow.com/questions/15063936/csv-error-field-larger-than-field-limit-131072

This needs to be packaged into a module though as this method relies on editing core files to achieve a solution.

Avatar
Discard
Best Answer

above solution does not work on windows server, any suggestion

Avatar
Discard
Author

@Snehal, I think this solution should be platform independent. Have you tried restarting your odoo-server, or running an odoo-server --update=all when starting (make sure you're fully backed up first)?