This paper sets out to unpack the pragmatic structure of refusal – its illocutionary nature, success conditions, and normative effects. I argue that our ordinary concept of refusal captures a whole family of illocutions, comprising acts such as rejecting, declining, and the like, which share the property of being ‘negative second-turn illocutions’. Only proper refusals (i.e. negative replies to permission requests), I submit, require speaker authority. I construe the ‘refusal family’ as a subclass of the directives-commissives intersection. After defending my view against a number of potential objections, I highlight how a theoretically grounded analysis of refusal is not only of intrinsic value, but may also have significant moral and legal implications.