LRRK2 serine/threonine kinase is associated with inherited Parkinson's disease. LRRK2 phosphorylates a subset of Rab GTPases within their switch 2 motif to control their interactions with effectors. Recent work has shown that the metal-dependent protein phosphatase PPM1H counteracts LRRK2 by dephosphorylating Rabs. PPM1H is highly selective for LRRK2 phosphorylated Rabs, and closely related PPM1J exhibits no activity towards substrates such as Rab8a phosphorylated at Thr72 (pThr72). Here, we have identified the molecular determinant of PPM1H specificity for Rabs. The crystal structure of PPM1H reveals a structurally conserved phosphatase fold that strikingly has evolved a 110-residue flap domain adjacent to the active site. The flap domain distantly resembles tudor domains that interact with histones in the context of epigenetics. Cellular assays, crosslinking and 3-D modelling suggest that the flap domain encodes the docking motif for phosphorylated Rabs. Consistent with this hypothesis, a PPM1J chimaera with the PPM1H flap domain dephosphorylates pThr72 of Rab8a both in vitro and in cellular assays. Therefore, PPM1H has acquired a Rab-specific interaction domain within a conserved phosphatase fold.