change your action varsas :-

optionally you can add views as a tuple like

tree = self.env.ref('yourmodel.tree_view_id')

form = self.env.ref('yourmodel.form_view_id')

return {

            'model': 'ir.actions.act_window',

            'type': 'ir.actions.act_window',

            'view_type': 'form',

            'view_mode': 'tree,form',

            'views': [(tree, 'tree'), (form, 'form')],

            'res_model': 'upload.file',

            'res_id': self.id,

            'target': 'new',

            'context': {'default_invoice_id': self.id}

        }

Hi,Problem :

You're passing res_id=self.id, but self refers to the account.invoice record. However, in the action res_model='uplod.file', the res_id is from account.invoice, which doesn't match the uplod.file model. There might be a mismatch as the res_id doesn't exist in the 'upload.file' model.Solution :
You need to first create record of upload.file after you can use it in res_id. we can do like this.

class AccountInvoice(models.Model):
  _inherit = "account.invoice"  def attach_documents(self):
    upload_file = self.env['upload.file'].create({
     # your values
    })
    return {
      'type': 'ir.actions.act_window',
      'view_type': 'form',
      'view_mode': 'form',
      'res_model': 'upload.file',
      'res_id': upload_file.id,
      'target': 'new',
      'context': {'default_invoice_id': self.id}
    }