Antibody-drug conjugates (ADCs) have recently received enormous attention as an attractive approach for cancer therapy. Although ADC design has been believed to be important for the relative efficacy of ADCs, it remains unexplored how the structural characteristics of ADCs would impact the internalization process and intracellular trafficking of the molecules. Herein, we report our efforts in investigating the cellular endocytosis implications of the conjugation and linker chemistry in designing antibody-based agents. A series of anti-MUC1 single-chain variable fragment (scFv-SM3) conjugates were designed with unique structural characteristics ranging from conjugation methods, sites of attachment and linker chemistry. In vitro confocal imaging showed that both random lysine-conjugation and site-specific conjugation, including C-terminus modification or internal site conjugation, could afford antibody conjugates with similar binding affinity and cellular uptake to target-expressing cells. Time-course internalization studies demonstrated that SM3-conjugates with short polyethylene glycol linkers outcompeted those that lack any hydrophilic linkers for higher cellular uptake and faster internalization rate. The SM3-conjugates with the highest affinity and internalization rate were also tested in mouse xenograft models using MUC1-overexpressing tumor cells. Our results indicate that the linker and conjugation chemistry play an important role in the internalization process of antibody conjugates, and this in turn could impact the therapeutic effects of ADCs.