If you want to keep the contact but you don’t want him/her to receive emails you can do that by changing the status of the contact to Unsubscribe.
You can choose to Unsubscribe this contact from a specific list or from all lists.

Contact List Page

All you need to do is to click on Edit next to the contact you want:

Then change the status to Unsubscribe:

Contact Activity Page

You are also able to do it inside of the Contact Activity Page. On the top-right dropdown select the Unsubscribe From All Lists option.

Get Started