There is an increasing demand for small-diameter blood vessels. Currently, there is no clinically available small-diameter artificial vessel. Bacterial nanocellulose (BNC) has vast potential for applications in artificial blood vessels due to its good biocompatibility. At the same time, medical polyurethane (PU) is a highly elastic polymer material widely used in artificial blood vessels. This study reports a composite small-diameter BNC/PU conduit using a non-solvent-induced phase separation method with the highly hydrophilic BNC tube as the skeleton and the hydrophobic polycarbonate PU as the filling material. The results revealed that the compliance and mechanical matching of BNC/PU tubes were higher than BNC tubes; the axial/radial mechanical strength, burst pressure, and suture strength were significantly improved; the blood compatibility and cell compatibility were also excellent. The molecular and subcutaneous embedding tests showed that the composite tubes had lighter inflammatory reactions. The results of the animal substitution experiments showed that the BNC/PU tubes kept blood flow unobstructed without tissue proliferation after implantation in rats for 9 months. Thus, the BNC/PU small-diameter vascular prosthesis had the potential for long-term patency and acted as an ideal material for small-diameter vessels.