Online inference is demanding on mobile devices such as vehicles, drones, smartphones, cameras, and even smart glasses. The computation, storage, and communication resources, however, are quite restricted on such devices. As a result, collaboration among mobile devices, edge servers, or/and cloud servers is the key to online inference on mobile devices. A balanced tradeoff between the computation and communication costs and the inference accuracy becomes the main design goal.