For example, In order to define a specific team you can give the privacy of the project as on invitation only, and the invite/add those employees whom you want to be in the project as followers in the chatter box. Only those members can access the project and associated tasks.

If you are doing a customer specific project and you want the customer to view the details of the project, you can give the privacy as visible by following customers and specify the customer in the customer field. When you save the document the customer would be automatically added as the follower of the document.