Don't touch the front of a mask when you take it off. This way, you avoid touching any infectious droplets it might have blocked. Instead, take the mask off by the ear loops.

For the types of cloth or surgical masks that the general public wears, the fibers they're made of aren't dense enough to block the exchange of gases like oxygen & carbon dioxide. Also, these masks aren't so tight that air can't get in around the sides.