In order to serve hundreds of millions of users, contemporary content providers employ tens of thousands of servers to scale their systems. The system software in these environments, however, is struggling to keep up with the increase in demand: contemporary network and storage stacks, as well as related APIs (e.g., BSD socket API) follow a `one-size-fits-all' design, heavily emphasising generality and feature richness at the cost of performance, leaving crucial hardware resources unexploited. Despite considerable prior research in improving I/O performance for conventional stacks, substantial hardware potential still remains unexploited because most of these proposals are fundamentally limited in their scope and effectiveness, as they still have to fit in a general-purpose design.

In this dissertation, I argue that specialisation and microarchitectural awareness are necessary in system software design to effectively exploit hardware capabilities, and scale I/O performance. In particular, I argue that trading off generality and compatibility, allows us to radically re-architect the stack emphasising application-specific optimisations and efficient data movement throughout the hardware to improve performance.

I first demonstrate that conventional general-purpose stacks fail to effectively utilise contemporary hardware while serving critical Internet workloads, and show why modern microarchitectural properties play a critical role in scaling I/O performance. I then identify core decisions in Operating Systems design that, although they were originally introduced to optimise performance, are now proven redundant or even detrimental. I propose clean-slate, specialised architectures for network and storage stacks designed to exploit modern hardware properties, and application domain-specific knowledge in order to sidestep historical bottlenecks in systems I/O performance, and achieve great scalability. With thorough evaluation of my systems, I illustrate how specialisation and greater microarchitectural awareness could lead to dramatic performance improvements, which could ultimately translate to improved scalability and reduced capital expenditure simultaneously.