Background: African pouched rats sniffing sputum samples provided by local clinics have significantly increased tuberculosis case findings in Tanzania and Mozambique. The objective of this study was to determine the reproducibility of rat results. Methods: Over an 18-month period 11,869 samples were examined by the rats. Intra-rater reliability was assessed through Yule’s Q. Inter-rater reliability was assessed with Krippendorff’s alpha. Results: Intra-rater reliability was high, with a mean Yule’s Q of 0.9. Inter-rater agreement was fair, with Krippendorf’s alpha ranging from 0.15 to 0.45. Both Intra- and Inter-rater reliability was independent of the sex of the animals, but they were positively correlated with age. Both intra- and inter-rater agreement was lowest for samples designated as smear-negative by the clinics. Conclusion: Overall, the reproducibility of tuberculosis detection rat results was fair and diagnostic results were therefore independent of the rats used.