Polycomb group proteins have a critical role in silencing transcription during development. It is commonly proposed that Polycomb-dependent changes in genome folding, which compact chromatin, contribute directly to repression by blocking the binding of activating complexes. Recently, it has also been argued that liquid-liquid demixing of Polycomb proteins facilitates this compaction and repression by phase-separating target genes into a membraneless compartment. To test these models, we used Optical Reconstruction of Chromatin Architecture to trace the Hoxa gene cluster, a canonical Polycomb target, in thousands of single cells. Across multiple cell types, we find that Polycomb-bound chromatin frequently explores decompact states and partial mixing with neighboring chromatin, while remaining uniformly repressed, challenging the repression-by-compaction or phase-separation models. Using polymer simulations, we show that these observed flexible ensembles can be explained by 'spatial feedback'-transient contacts that contribute to the propagation of the epigenetic state (epigenetic memory), without inducing a globular organization.