Charging piles in the bus depot provide charging services to multiple electric bus (EB) routes operating in the area. As charging needs may overlap between independently operated routes, EB fleets often have to wait in line for charging. However, affected by the ambient temperature, the length of the waiting time will cause the battery temperature to change at the beginning of each charging, thereby influencing the charging performance and charging time of the battery. To this end, this paper considers the influence of ambient temperature on battery charging performance, and collaboratively optimizes the number of charging piles in the bus depot and the scheduling problem of EB charging. Aiming at minimizing the cost of laying charging piles in bus stations and the charging costs of bus fleets, as well as minimizing the empty time of electric bus fleets and waiting time for charging in queues, a mixed-integer nonlinear programming model is established, and the immune algorithm is used to solve it. At last, an actual bus depot and four EB routes are taken as examples for verification. The results show that by optimizing the charging waiting time of the electric bus at the bus station, the rapid decline in charging performance caused by the sharp drop in battery temperature is avoided. Without increasing the charging cost of the electric bus fleet, the established method reduces the charging pile installation cost, improves the bus depot’s service efficiency, and ensures the punctuality and integrity of the regional bus route operation.