Evidence is mixed as to whether less- or more-advantaged fathers suffer penalties for taking paid family leave and the reasons for this. Perhaps selection into taking leave differs among fathers, or taking leave increases some fathers’ commitment to family over paid work, or taking it sends a negative signal to employers about future work-family priorities. We contribute to the literature by distinguishing between the initial paternity leave taken by the majority of fathers, and subsequent solo paternal leave taken by fewer fathers that indicates and signals greater family commitment. We also develop competing hypotheses about why low- or high-wage fathers may be penalized more for taking family leave. These are tested analyzing 2001 to 2014 waves of Finnish administrative panel data using unconditional quantile regression with various fixed-effects models. Net of selection, no fathers incur a sustained wage penalty for taking paternity leave, although distributed fixed-effects models reveal the highest-wage fathers receive a temporary penalty that we attribute to signaling. All fathers who also take solo paternal leave have decreasing post-leave wage trajectories. Only lower-wage fathers accrue significant penalties, however, suggesting that taking the leave shifts their priorities more toward family. We conclude the repercussions of taking shorter or longer family leaves and their sources differ across fathers’ wage distribution.