Normally you should define what is the field that define a different color(using the color attribute of calendar tag) based on the values it may have. So in your case if that field is the state field you could do something like this:

<calendar string="Appointments" color="state" date_start="appointment_date" date_delay="duration">
    <field name="patient_name"/>
</calendar> 

Edited:

For your requirement to manually map your states with a color you will need to extends the widget instance.web_calendar.CalendarView the init function to initialize the object "this.color_map" with your map of values to color