Created two groups
User Own Contact and User All Contacts

<record id="group_user_own_contact" model="res.groups">
<field name="name">User Own Contact</field>
<field name="category_id" ref="base.module_category_hidden"/>
</record>

<record id="group_user_all_contact" model="res.groups">
<field name="name">User All Contacts</field>
<field name="category_id" ref="base.module_category_hidden"/>
</record>

Then added record rules:
<record id="contact_rule_user_own" model="ir.rule">
<field name="name">Contact</field>
<field ref="base.model_res_partner" name="model_id"/>
<field name="domain_force">[('user_id','=',user.id)]</field>
<field name="groups" eval="[(4, ref('group_user_own_contact'))]"/>
</record>

<record id="contact_rule_show_all" model="ir.rule">
<field name="name">All Contacts</field>
<field ref="base.model_res_partner" name="model_id"/>
<field name="domain_force">[(1,'=',1)]</field>
<field name="groups" eval="[(4, ref('group_user_all_contact'))]"/>
</record>

Hello Asha,

I have created same group and record rule as you mentioned, your code isn't wrong. But it is not working because of conflicting of other record rules. I have attached the screenshot of them. Try by archiving them then your code will work. you have two options. You can either modify these conflict record rules as per your condition or you can archive them.
Please check link for attchment :-
https://drive.google.com/file/d/1xWXdevda91I55PNEJu2dQTiK2nNzp1jG/view?usp=sharing

Hope it will help.

Thanks