If Ali had not given the oath of allegiance to Abu Bakr, then the enemies of Ali, (among them the Umayyads, the Nasibis, and the Kharijis) would have used it as a reason to discredit 'Ali over the centuries. Since they have never done that, we can conclude that Ali DID give his oath of allegiance to Abu Bakr, as well as to Umar and Uthman, may Allah be pleased with them all.

This is confirmed by the sound narrations to that effect, but the reasoning above is more than sufficient for whoever wishes to see.