Instance segmentation of pavement crack presents notable challenges but is pivotal for practical applications such as crack visual tracking and automated crack repair. However, slender cracks are plagued by class imbalance, low tolerance for interference in unstructured road scenes, and the presence of topological features. In this article, a novel approach called S2TNet is proposed, which enhances the contrast between the crack image foreground and background using local contrast enhancement. Specifically, the proposed anchor ratio IoU-Balanced Sampling (ARIS) and Balanced Fine-Grained Features (BFGF) effectively improve the performance of the detector in predicting bounding box and segmenting instance binary mask. Moreover, we establish a framework using an unmanned wheeled robot with a four-wheel independent differential drive system to deploy the S2TNet approach. Extensive experiments conducted on self-created (S2T-Crack) datasets present the effectiveness of the proposed method in achieving real-time segment-to-track at a speed of 0.05 m/s, with a segmentation precision of 80.21% for pavement cracks.