Maybe make the button returns the action of the A form view (if A record still in context, else, just return A tree view)

Update:

you can try this then https://www.odoo.com/fr_FR/forum/help-1/question/solved-close-window-on-button-press-4348

Just use:

<button data-dismiss="modal" string="Close my modal"/> in your view. 

I waste some time, while dig this, because this dont work if attribute "type" 
of the button is set to anything )

Hi py, I got the same issue, I try your solution https://www.odoo.com/fr_FR/forum/help-1/question/solved-close-window-on-button-press-4348, but It doesn't work, any idea?

Thanks.

I already tried that. The result is that the action takes effect in the background window. The popup stays open.