Phonon lifetime in materials is an important observable that conveys basic information about structure, dynamics, and anharmonicity. Recent vibrational transient-grating measurements, using picosecond infrared pulses from free-electron lasers, have demonstrated that the vibrational-population decay rates of localized high-frequency stretching modes (HSMs) in hydrogenated and deuterated amorphous silicon (a-Si:H/D) increase with temperature and the vibrational energy redistributes among the bending modes of Si in a-Si:H/D. Motivated by this observation, we address the problem from first-principles density-functional calculations and study the time evolution of the vibrational-population decay in a-Si:H/D, the average decay times, and the possible decay channels for the redistribution of vibrational energy. The average lifetimes of the localized HSMs in a-Si:H and a-Si:D are found to be approximately 51-92 ps and 50-78 ps, respectively, in the temperature range of 25-200 K, which are consistent with experimental data. A weak temperature dependence of the vibrational-population decay rates has been observed via a slight increase of the decay rates with temperature, which can be attributed to stimulated emission and increased anharmonic coupling between the normal modes at high temperature.