Community mailing list archives

community@mail.odoo.com

Re: The Speed of UnitTests

by
DevCO Colombia , David Arnold
- 08/17/2016 11:08:53
In Memory PSQL: 3 mins down of ca 60 min, If possible, I'd like to know further details on this.
SSD: 2 mins down

Interesting, if the above thing about in memory transaction really means "PSQL odes never touch any disk in this mode", then I'm surprised by this and my hypothesis that it's disk that is slowing down test execution would not be valid.

If this holds true, then I guess it's neither disk, nor memory, but CPU which limits speed. But looking close, you can see that really a lot of time (weather be it disk, memory or CPU) is spend on initializing modules and, yes, module data specifically. I haven't measured (as it's not as easy as sum up the test execution seconds), but it feels like the responsible when watching test logs as they come in.

Still I don't entirely trust the scope of the in memory transaction in PSQL. See also here, which states, that it is not possible: http://stackoverflow.com/questions/7872693/running-postgresql-in-memory-only
Were you referring to the buffer ram? This greatly reduces disk IO on READS and is really powerful, still if you get 3 mins this means a 5% improvement on the DB READ part of the equation only.

Best, David

El mié., 17 ago. 2016 a las 4:18, Jairo Llopis (<jairo.llopis@tecnativa.com>) escribió:

2016-08-17 10:42 GMT+02:00 Nhomar Hernandez <nhomar@vauxoo.com>:
well that's a quick hipothesis (which is that, 'just' an hypothesis) BTW, if you measure the inner points in your environment you can measure every part by yourself.

Indeed, it's just an hypothesis.
 
We did it, set an environment all those 13 minutes is Postgresql + set the base database + other magic, in our test environments on CI it is like 8/10 minutes (we manage the R&D by ourself) and it is due to the test framework itself (as I said it is integration tests not only unittests).

But BTW that do not MUST be by anybody,that "should be" done by the people that need the change (in our case when we find a performance problem we attack that with our enterprise and work with odoo side/by/side we expend A LOT of time in R&D case) as you mentioned this time must be paid and covered and that's why the enterprise comes alive and/or the community work, nobody has any dutie until an exchange of responsability with a commercial agreement is set (either you-your customer, you-you, you-odoo-customer....).

As you read in the licence (and in any other opensuource project) it comes without any responsibility and it is distributed as it is.

But BTW if you dedicate a while to an specific topic and put such specific conclusion on github this will help more than claim right and duties on others, don't you think?

Relax, please. I was just saying that it means a lot of R&D. Of course, we also invest a lot of time in R&D, but in other things that you probably will use some day too. That's the good point of communities! 😊

You are asking me to invest that time just to prove my hypothesis, but you are missing the point that I'm just posting an email, a little opinion on where the real problem probably is, not a 20h R&D project. I guess if you post to a mailing list, you expect some response, and I'm only trying to help here.
 
I do not understand this, odoo is data driven and you have only one backend.....

Weren't you talking about SQLite in memory?
 
   If you could, however, use in-memory Postgres transactions, that would get what you want: skip hard disk.

We did (this put down 3 minutes the time in a 1 hour test environement we put back to disk but no remember why)

Or simply buy a fast SSD.

We did also 2 minutes down in 1  hour environment.

Then indeed the solution must be be in other place.

Thanks for your time & effort on improving on this matter!

Regards.


--
Jairo Llopis

_______________________________________________
Mailing-List: https://www.odoo.com/groups/community-59
Post to: mailto:community@mail.odoo.com
Unsubscribe: https://www.odoo.com/groups?unsubscribe