Streamwise-coherent structures were observed in schlieren images of a Mach 2.5 flow in an empty supersonic wind tunnel with rectangular cross-section. These features are studied using RANS computations in combination with wind tunnel experiments. The structures are identified as regions of streamwise vorticity embedded in the sidewall boundary layers. These vortices locally perturb the sidewall boundary layers, and can increase their thickness by as much as 37%. The vortices are caused by a region of separation upstream of the nozzle where there is a sharp geometry change, typical in supersonic wind tunnels with interchangeable nozzle blocks. Despite originating in the corners, the vortices are transported by secondary flows in the sidewall boundary layers so they end up near the tunnel centre-height, well away from any corners. The successful elimination of these sidewall vortices from the flow is achieved by replacing the sharp corner with a more rounded geometry, so that the flow here remains attached.