Some estrogen receptor-α (ERα)-targeted breast cancer therapies such as tamoxifen have tissue-selective or cell-specific activities, while others have similar activities in different cell types. To identify biophysical determinants of cell-specific signaling and breast cancer cell proliferation, we synthesized 241 ERα ligands based on 19 chemical scaffolds, and compared ligand response using quantitative bioassays for canonical ERα activities and X-ray crystallography. Ligands that regulate the dynamics and stability of the coactivator-binding site in the C-terminal ligand-binding domain, called activation function-2 (AF-2), showed similar activity profiles in different cell types. Such ligands induced breast cancer cell proliferation in a manner that was predicted by the canonical recruitment of the coactivators NCOA1/2/3 and induction of the GREB1 proliferative gene. For some ligand series, a single inter-atomic distance in the ligand-binding domain predicted their proliferative effects. In contrast, the N-terminal coactivator-binding site, activation function-1 (AF-1), determined cell-specific signaling induced by ligands that used alternate mechanisms to control cell proliferation. Thus, incorporating systems structural analyses with quantitative chemical biology reveals how ligands can achieve distinct allosteric signaling outcomes through ERα.