In 2017, 400 million people in Sub-Saharan Africa (SSA) were still using unimproved drinking water sources, 80 million of whom relied on untreated surface water. Urban areas are vastly expanding all over the continent and many larger cities in SSA struggle to provide safely managed drinking water. Phytoremediation implemented in constructed wetlands (CWs) is a low-cost and sustainable alternative to highly costly and energy consuming wastewater treatment plants. In addition, CWs offer the potential to be integrated into farming and aqua-culture systems and can therefore improve food quality and production. The most prominent pollutants in urban SSA surface waters and the pollutant removal efficiencies for microbial and chemical contaminations of different plant species were identified from the literature and the accumulation rates for Pb, Cr, and Cd were compared to each other. A strong focus was given to studies conducted in SSA or other (sub)tropical regions. This review identified a range of potential phytoremediators to treat contaminated surface water and highlights the need for further in-situ studies in SSA. Plant species such as Lemna minor, Ipomoea aquatica, Spirodela polyrhiza and Brachiaria mutica show a high potential to phytoremediate the heavy metals Pb, Cr and Cd from surface water.