We report an algorithm based on renormalization to compute the probability that a particular state, or set thereof, is visited along the first passage or transition paths between two endpoint states of a finite Markov chain. The procedure is numerically stable and does not require dense storage of the transition matrix.