Immune checkpoint blockade (ICB) benefits some patients with triple-negative breast cancer, but what distinguishes responders from non-responders is unclear1. Because ICB targets cell-cell interactions2, we investigated the impact of multicellular spatial organization on response, and explored how ICB remodels the tumour microenvironment. We show that cell phenotype, activation state and spatial location are intimately linked, influence ICB effect and differ in sensitive versus resistant tumours early on-treatment. We used imaging mass cytometry3 to profile the in situ expression of 43 proteins in tumours from patients in a randomized trial of neoadjuvant ICB, sampled at three timepoints (baseline, n = 243; early on-treatment, n = 207; post-treatment, n = 210). Multivariate modelling showed that the fractions of proliferating CD8+TCF1+T cells and MHCII+ cancer cells were dominant predictors of response, followed by cancer-immune interactions with B cells and granzyme B+ T cells. On-treatment, responsive tumours contained abundant granzyme B+ T cells, whereas resistant tumours were characterized by CD15+ cancer cells. Response was best predicted by combining tissue features before and on-treatment, pointing to a role for early biopsies in guiding adaptive therapy. Our findings show that multicellular spatial organization is a major determinant of ICB effect and suggest that its systematic enumeration in situ could help realize precision immuno-oncology.