Background Mitomycin (MMC) has been frequently used as the compound for intravesical treatment. The relatively new pyrimidine analog gemcitabine (GEM) has exhibited anticancer effect on various solid cancers, such as the advanced bladder cancer. In this study, the GEM and MMC in treating non-muscle invasive bladder cancer (NMIBC) cases was compared through systemic review. Methods In accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement, the electronic databases, including Embase, PubMed, Chinese biomedicine literature database, the Cochrane Library, the National Institute for Health and Clinical Excellence, NHS Evidence, Chinese technological periodical full-text database, and Chinese periodical full-text database, were systemically reviewed from inception to October 2018. Then, the RevMan 5.0 software was applied for data analysis. Five randomized controlled trials (RCTs) involving a total of 335 patients were included. Results For MMC group, the recurrence rate in the mitomycin arm increased compared with that in GEM group (OR = 0.44 95% CI [0.24, 0.78]), and the difference was statistically significant between the two groups. GEM was associated with reduced incidence of chemical cystitis compared with that of MMC (OR = 0.23 95% CI [0.12, 0.44]). Differences in hematuria (OR = 0.46 95% CI [0.16, 1.31]), skin reaction (OR = 0.49 95% CI [0.14, 1.70]) and liver and kidney function damage (OR = 0.51 95% CI [0.09, 2.85]) displayed no statistical significance between the two groups. Conclusion Findings in our study demonstrate the superior efficacy of GEM over MMC in reducing the relapse rate among NMIBC patients following transurethral resection (TUR). In addition, GEM is associated with reduced local toxic effects on the bladder compared with those of MMC. However, more future studies are needed to examine GEM safety when used as the monotherapy or polytherapy for bladder patients. More RCTs with high quality are also required to validate our findings due to the limitations of the current meta-analysis.