Immune responses demonstrate a high level of intra-species variation, compensating for the specialization capacity of pathogens. The recent advent of in-depth immune phenotyping projects in large-scale cohorts has allowed a first look into the factors that shape the inter-individual diversity of the human immune system. Genetic approaches have identified genetic diversity as drivers of 20-40% of the variation between the immune systems of individuals. The remaining 60-80% is shaped by intrinsic factors, with age being the predominant factor, as well as by environmental influences, where cohabitation and chronic viral infections were identified as key mediators. We review and integrate the recent in-depth large-scale studies on human immune diversity and its potential impact on health.