If you want to keep the contact but you don’t want him/her to receive emails you can do that by changing the status of the contact to Unsubscribe.
You can choose to Unsubscribe this contact from a specific list or from all lists.
Contact List Page
All you need to do is to click on Edit next to the contact you want:
Then change the status to Unsubscribe:
Contact Activity Page
You are also able to do it inside of the Contact Activity Page. On the top-right dropdown select the Unsubscribe From All Lists option.
