The ongoing evolution of SARS-CoV-2 requires monitoring the capability of immune responses to cross-recognize Variants of Concern (VOC). In this cross-sectional study, we examined serological and cell-mediated immune memory to SARS-CoV-2 variants, including Omicron, among a cohort of 18-21-year-old Marines with a history of either asymptomatic or mild SARS-CoV-2 infection 6 to 14 months earlier. Among the 210 participants in the study, 169 were unvaccinated while 41 received 2 doses of mRNA-based COVID-19 vaccines. Vaccination of previously infected participants strongly boosted neutralizing and binding activity and memory B and T cell responses including the recognition of Omicron, compared to infected but unvaccinated participants. Additionally, no measurable differences were observed in immune memory in healthy young adults with previous symptomatic or asymptomatic infections, for ancestral or variant strains. These results provide mechanistic immunological insights into population-based differences observed in immunity against Omicron and other variants among individuals with different clinical histories.