Photoperiod- and thermo-sensitive genic male sterility (PGMS and TGMS) are the core components for hybrid breeding in crops. Hybrid rice based on the two-line system using PGMS and TGMS lines has been successfully developed and applied widely in agriculture. However, the molecular mechanism underlying the control of PGMS and TGMS remains obscure. In this study, we mapped and cloned a major locus, p/tms12-1 (photo- or thermo-sensitive genic male sterility locus on chromosome 12), which confers PGMS in the japonica rice line Nongken 58S (NK58S) and TGMS in the indica rice line Peiai 64S (PA64S, derived from NK58S). A 2.4-kb DNA fragment containing the wild-type allele P/TMS12-1 was able to restore the pollen fertility of NK58S and PA64S plants in genetic complementation. P/TMS12-1 encodes a unique noncoding RNA, which produces a 21-nucleotide small RNA that we named osa-smR5864w. A substitution of C-to-G in p/tms12-1, the only polymorphism relative to P/TMS12-1, is present in the mutant small RNA, namely osa-smR5864m. Furthermore, overexpression of a 375-bp sequence of P/TMS12-1 in transgenic NK58S and PA64S plants also produced osa-smR5864w and restored pollen fertility. The small RNA was expressed preferentially in young panicles, but its expression was not markedly affected by different day lengths or temperatures. Our results reveal that the point mutation in p/tms12-1, which probably leads to a loss-of-function for osa-smR5864m, constitutes a common cause for PGMS and TGMS in the japonica and indica lines, respectively. Our findings thus suggest that this noncoding small RNA gene is an important regulator of male development controlled by cross-talk between the genetic networks and environmental conditions.