Community mailing list archives

community@mail.odoo.com

Re: database cleanup

by
Graeme Gellatly
- 12/02/2015 23:24:07

Pretty much my process for testing and migrations goes (pseudocode)

CREATE DATABASE new;
install modules, do config etc.
CREATE DATABASE test FROM TEMPLATE new; (REASSIGN OWNED if needed)
do some testing, maybe migrations,  enter some data etc.
update new database with extra modules and config as needed from test.
Rinse repeat.  Once happy
CREATE DATABASE production FROM TEMPLATE new; or just use it. (REASSIGN OWNED if needed)

I hate these claims of performance problems with postgres.  This is a pure red herring assertion.  If you have postgres performance problems, invariably the problem is an admin managing a database too large for his skills or on a machine too underpowered for the job.  It wouldn't matter if it was postgres or any other database, the problem is the admin.  Postgres handles far more than a little ERP basically text database with a few hundred tables, each with a few dozen columns and a few with a few million rows, especially for read performance.    (By comparison scientists use postgres with tables with billions of rows and I bet they aren't saying to IT to please delete their research to make it go faster, if it sucked they'd use something else). 

FYI
Limit   Value
Maximum Database Size       Unlimited
Maximum Table Size          32 TB
Maximum Row Size            1.6 TB
Maximum Field Size          1 GB
Maximum Rows per Table      Unlimited
Maximum Columns per Table   250 - 1600 depending on column types
Maximum Indexes per Table   Unlimited

And you can shard tables if needed although not sure Odoo supports that.


Partial and/or composite indexes that hold readily needed information based on common predicates work wonders, but even then are actually really only needed on a few really really big tables.  I tested a few a while back (basically either partial on state, or active) and it made stuff all difference on a 10m row table when compared to using a sensible composite.







On Thu, Dec 3, 2015 at 3:27 PM, Andreas Becker <andi@lisandi.com> wrote:
Well Claudal that's a nice and good approach I think. right! Thanks for the idea


With kind regards,
Mit freundlichen Grüßen,
Con un cordial saludo,
Cordialement,
с сердечным приветом,
เรื่องที่เกี่ยวกับชนิด,
與親切的問候,

 ANDI BECKER

CEO/General Manager LisAndi Co., Ltd.

--------------------------------------------------

LisAndi Co. Ltd., Phuket, Thailand (lisandi.com)
15/21 M.2 Viset Road, Rawai, Muang, Phuket, Thailand 83130

VoIP:   +49 (0)711 50 88788 50
Fax:     +49 (0)711 50 88788 50
Skype:          lisandi
Facebook:     andibecker
Google Talk/Facetime/eMail:  andi@lisandi.com

--------------------------------------------------

This email may contain confidential and/or privileged information. If you are not the intended recipient (or have received this email by mistake), please notify the sender immediately and destroy this email. Any unauthorized copying, disclosure or distribution of the material in this email is strictly prohibited. Email transmission security and error-free status cannot be guaranteed as information could be intercepted, corrupted, destroyed, delayed, incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the contents of this message which may arise as a result of email transmission

On Thu, Dec 3, 2015 at 7:37 AM, Caudal Eric <caudaleric@gmail.com> wrote:

I would rather have an archive/inactive field at invoice/stock moves/SO/PO level that could be setup once a year via wizard.
Then you just have to adapt the views (quite a lot though but far much cleaner in general)


On Thu, Dec 3, 2015, 00:46 Andreas Becker <andi@lisandi.com> wrote:
Well Martin

Depending in which country you are living data has to be kept for a certain time - in Thailand it is 5 years - and afterwards you should actually try to get rid of it as also here they call it privacy etc (even no one seems to check it as long as your neighbour does not tell them!) 

A growing database also results in speed and performance losses. Especially if you have lots of other stuff stored there to. i.e. from the website module and you want to get rid of old articles or stuff which is not at all relevant for seo anymore.

Removing transactions within a year can be dangerous, you are absolutely right! I would not advise it either. But anyway an easy way to cut of all data no more needed would be fine.

Otherwise you are ending u like a Facebook Backup which is 2-3 or even more GB to be downloaded, even you would need only the last months or only a certain part of it, what has been stored in the DB.




With kind regards,
Mit freundlichen Grüßen,
Con un cordial saludo,
Cordialement,
с сердечным приветом,
เรื่องที่เกี่ยวกับชนิด,
與親切的問候,

 ANDI BECKER

CEO/General Manager LisAndi Co., Ltd.

--------------------------------------------------

LisAndi Co. Ltd., Phuket, Thailand (lisandi.com)
15/21 M.2 Viset Road, Rawai, Muang, Phuket, Thailand 83130

VoIP:   +49 (0)711 50 88788 50
Fax:     +49 (0)711 50 88788 50
Skype:          lisandi
Facebook:     andibecker
Google Talk/Facetime/eMail:  andi@lisandi.com

--------------------------------------------------

This email may contain confidential and/or privileged information. If you are not the intended recipient (or have received this email by mistake), please notify the sender immediately and destroy this email. Any unauthorized copying, disclosure or distribution of the material in this email is strictly prohibited. Email transmission security and error-free status cannot be guaranteed as information could be intercepted, corrupted, destroyed, delayed, incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the contents of this message which may arise as a result of email transmission

On Wed, Dec 2, 2015 at 10:32 PM, Martin Trigaux <mat@odoo.com> wrote:
On 02/12/15 16:12, Andreas Becker wrote:
> This is a very good question - Keeping your database clean should be
> easy as it would be good to do this every year at least.

I don't really think that's a good idea. Removing some records such as 
accounting transactions or other moves is a risk to lose information, 
break flows or be stuck at migrations.
I wouldn't advise it if you don't know exactly what you are doing or 
have very good reason to do so (other than "keep my db tidy").

@Franco you ask something very specific (remove this but keep that) so 
this should probably be a manual operation.
But again I wouldn't advise it.

And, as said Pedro, make copy of your database.

-- 
Martin Trigaux
Odoo (Formerly OpenERP)
https://odoo.com
https://github.com/mart-e

_______________________________________________

_______________________________________________
Mailing-List: https://www.odoo.com/groups/community-59
Post to: mailto:community@mail.odoo.com
Unsubscribe: https://www.odoo.com/groups?unsubscribe

--

Eric  Caudal (from my mobile)

_______________________________________________
Mailing-List: https://www.odoo.com/groups/community-59
Post to: mailto:community@mail.odoo.com
Unsubscribe: https://www.odoo.com/groups?unsubscribe


_______________________________________________
Mailing-List: https://www.odoo.com/groups/community-59
Post to: mailto:community@mail.odoo.com
Unsubscribe: https://www.odoo.com/groups?unsubscribe