Community mailing list archives

The Speed of UnitTests

XOE Corp. SAS, David Arnold
- 08/14/2016 17:20:58

It has been said (don't make me search for the reference, please):
"If your unit tests take more than 10ms to complete, you are definitely doing something wrong."

The rationale is obvious: Faster unittests allow for shorter testing cycle, optimally, you want to to TDD (test driven development) and trigger a testing cycle on every save.

Now Odoo takes about 15 min to complete tests, even if you are just test one module, because of an overloaded base database (data of all dependent modules getting initialized) and the mere fact of connecting to an external persistence system (disk!!) just makes testing not really a pleasure. Add around 5 seconds of startup time to the server so running tests on every save becomes quickly a matter of hours waiting.

NOT cool!

What can be done?

That's basically my question. Of course I'm aware that the majority of tests in the odoo test suite are rather integration tests than real unit tests, and it also holds true, that most integration test will need a database to check if the application logic is executed correctly. This being said, proposals have been made to mock the database with a sqlight3
You should have a look at the code!

I've not seen in the code a place where module data loading would be prevented, this in the case of large data tables slows down tremendously the execution (countries, cities, COAs, etc). However, I'm not aware if some tests depend on loaded data, if they do, this is definitely a design flaw: They shouldn'! So in conclusion, there should be something to prevent loading of data when executing test or as the bare minimum, there should be a flag which disables data loading and tests depending on it (I repeat, no test should, but rather should construct its own testing data on a temporary cursor!)

Only then one could think of integrating test runs as an onsave trigger.

Any thoughts? Any smart tips?

Best, David