Yes — but your account must always have at least one verified way to log in.
What this means:
You cannot remove all emails — at least one verified email is required
You can also keep your account secure by linking Google or Apple login as an alternative to email/SMS.