Understanding immune memory to severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) is critical for improving diagnostics and vaccines and for assessing the likely future course of the COVID-19 pandemic. We analyzed multiple compartments of circulating immune memory to SARS-CoV-2 in 254 samples from 188 COVID-19 cases, including 43 samples at ≥6 months after infection. Immunoglobulin G (IgG) to the spike protein was relatively stable over 6+ months. Spike-specific memory B cells were more abundant at 6 months than at 1 month after symptom onset. SARS-CoV-2-specific CD4 T cells and CD8 T cells declined with a half-life of 3 to 5 months. By studying antibody, memory B cell, CD4 T cell, and CD8 T cell memory to SARS-CoV-2 in an integrated manner, we observed that each component of SARS-CoV-2 immune memory exhibited distinct kinetics.