Tauopathies encompass a group of neurodegenerative disorders characterised by diverse tau amyloid fibril structures. The persistence of polymorphism across tauopathies suggests that distinct pathological conditions dictate the adopted polymorph for each disease. However, the extent to which intrinsic structural tendencies of tau amyloid cores contribute to fibril polymorphism remains uncertain. Using a combination of experimental approaches, we here identify a new amyloidogenic motif, PAM4 (Polymorphic Amyloid Motif of Repeat 4), as a significant contributor to tau polymorphism. Calculation of per-residue contributions to the stability of the fibril cores of different pathologic tau structures suggests that PAM4 plays a central role in preserving structural integrity across amyloid polymorphs. Consistent with this, cryo-EM structural analysis of fibrils formed from a synthetic PAM4 peptide shows that the sequence adopts alternative structures that closely correspond to distinct disease-associated tau strains. Furthermore, in-cell experiments revealed that PAM4 deletion hampers the cellular seeding efficiency of tau aggregates extracted from Alzheimer's disease, corticobasal degeneration, and progressive supranuclear palsy patients, underscoring PAM4's pivotal role in these tauopathies. Together, our results highlight the importance of the intrinsic structural propensity of amyloid core segments to determine the structure of tau in cells, and in propagating amyloid structures in disease.