The HIV-1 envelope glycoproteins (Env) undergo conformational changes upon interaction of the gp120 exterior glycoprotein with the CD4 receptor. The gp120 inner domain topological layers facilitate the transition of Env to the CD4-bound conformation. CD4 engages gp120 by introducing its phenylalanine 43 (Phe43) in a cavity ("the Phe43 cavity") located at the interface between the inner and outer gp120 domains. Small CD4-mimetic compounds (CD4mc) can bind within the Phe43 cavity and trigger conformational changes similar to those induced by CD4. Crystal structures of CD4mc in complex with a modified CRF01_AE gp120 core revealed the importance of these gp120 inner domain layers in stabilizing the Phe43 cavity and shaping the CD4 binding site. Our studies reveal a complex interplay between the gp120 inner domain and the Phe43 cavity and generate useful information for the development of more-potent CD4mc. The Phe43 cavity of HIV-1 envelope glycoproteins (Env) is an attractive druggable target. New promising compounds, including small CD4 mimetics (CD4mc), were shown to insert deeply into this cavity. Here, we identify a new network of residues that helps to shape this highly conserved CD4 binding pocket and characterize the structural determinants responsible for Env sensitivity to small CD4 mimetics.