What I mean is the link of the file uploaded will be disabled to prevent downloading it by certain user groups.

The suggestion for hiding attachment global menu.

You need to override fields view get method of your model where you want to hide the attachment menu.

You need to apply some delete operation on that menu item when your model view loaded.

or

Go to Settings -> Technical -> Database Structure ->Models. Then you can search for "ir.attachment" and view it. In the Access Rights tab, edit the record and deselect read access for group or Your custom group .

Please another example with code in comment.

I hope This will help you.

Thanks & Regards,
Email: odoo@aktivsoftware.com
Skype: kalpeshmaheshwari