Ghosts seem to only have one guardian- if they didn't, then the greats could easily just be revived by multiple ghosts. I'm more interested in what the actual connection is between the Traveler/Ghosts and the Speaker. Egypt's rulers claimed to be able to speak with the gods and act according to their will, but no real link existed (which is because it is considered mythology and no longer religion). Maybe the Speaker has no link and that is why Osiris parted ways with him.