Create a domain on the action that opens that customer list view, that will supercede your search filter domains. You can quickly see this by going into debug mode, clicking on Sales -> Address Book -> Customers, then going to the debug dropdown and selecting "Edit Action". For the domain field, enter [('customer','=',1)] and save it. Now no matter what filters you choose, only customers show up.

To make that change last, you'd need to write some custom code to put that domain into the action XML. Something as simple as this would do it (because you're referring to the already existing action and only writing one change - the domain - and not re-writing the same info that's already there):

<record id="base.action_partner_form" model="ir.actions.act_window">
    <field name="domain">[('customer','=',1)]</field>
</record>

This makes your filter buttons pointless though, so you might want to get rid of those if you really want to go down this route.