Plant acclimatory responses to phosphate (Pi) starvation stress include the accumulation of carbohydrates, namely sugar and starch. However, whether altered endogenous carbohydrate profile could in turn affect plant Pi starvation responses remains widely unexplored. Here, two genes encoding the large and small subunits of an ADP-glucose pyrophosphorylase (AGP) in rice (Oryza sativa), AGP Large Subunit 1 (AGPL1) and AGP Small Subunit 1 (AGPS1), were functionally characterized with regard to maintenance of phosphorus (P) homeostasis and regulation of Pi starvation signaling. AGPL1 and AGPS1 were both positively responsive to nitrogen (N) or Pi deprivation, and expressed in almost all the tissues except in the meristem and mature zones of root. AGPL1 and AGPS1 physically interacted in chloroplast, and catalyzed the rate-limiting step of starch biosynthesis. Low-N- (LN) and low-Pi (LP)-triggered starch accumulation in leaves was impaired in agpl1, agps1 and apgl1 agps1 mutants compared with the wild-type plants. By contrast, mutation of AGPL1 and/or AGPS1 led to an increase in the content of the major sugar, sucrose, in leaf sheath and root under control and LN conditions. Moreover, the Pi accumulation was enhanced in the mutants under control and LN conditions, but not LP conditions. Notably, the LN-induced suppression of Pi accumulation was compromised attributed to the mutation of AGPL1 and/or AGPS1. Furthermore, the increased Pi accumulation was accompanied by the specific suppression of OsSPX2 and activation of several Pi transporter genes. These results indicate that a balanced level of carbohydrates is vital for maintaining plant P homeostasis.