So here is how I debugged the problem, what was the problem and the way I solved it.
When Odoo nags about access permission it is very cold! There is just a dialog saying that you cannot access a model and that you have to contact the Administrator and if you're the administrator, you gonna' have hard times once your colleagues come to you just to find you clueless about source of the problem.
This is how I debugged it:
I modified the file ./models.py in root folder of Odoo to add traceback and variable dump functionality to it. As I was using official docker image of Odoo I had to modify my docker file to make odoo installation folder writable:

RUN chmod a+rwx -R /usr/lib/python2.7/dist-packages/odoo \

    && apt-get update && apt-get -y install vim

And to add the vim. Later I modified:
     /usr/lib/python2.7/dist-packages/odoo/models.py
And here is the diff:
29d28

< import traceback

3172,3175d3170

< _logger.info('HERE!')

< _logger.info(vars(self))

< for line in traceback.format_stack():

< _logger.info(line.strip())
Looking into traceback and vars output told me that the issue is happening due to trying to access a location with some ID.
What was the probem?
Looking into database I found the root cause. Some incompetent personnel of us couldn't do something easy in Odoo so desparately went to warehouse configuration and tried to change anything he could find by random, including changing parent location of locations, etc. so messed with our whole inventory system. In our case an old and archived location was set as parent location of 'Physical Location' therefore rendering all physical locations useless!
How I solved the issue?
Pretty much reverted his changes through database tables.

Conclusion

The access permission error message is pretty useless and just sends employees to administrators. The log output of server doesn't help either. Adding a traceback and vars output to log output helps a lot. I grep'ed the whole source tree for the error message to find that the error is happening in two places within models.py and added traceback and vars output to one of them so I was able to solve the issue.
Thanks.