I think that maybe the reason we are hated amongst the city is kind of something Admiral Sabree said, we are to prtect the Light and the City, and it is just coincidental that we are protecting the City. We, as Guardians, have to protect the City considering that it is the first line of defense against the traveler, and that the City was strategically placed UNDER the Traveler, in order to kind of 'force' the Guardians to protect them. Also, a nic,e little Grimoire card I found helps me back up another theory: The Guardians are death. As said in the Grimoire card when the Guardian is talking to a Ghost "You are a dead thing made by a dead power." What if they moreover FEAR the Guardians because they are said 'dead power', I mean, we got references that the Traveler is just 'dormant', but we got far more that he is actually dead. That me made Ghosts to protect his embodiment and finish what he started with his last breath. There is another theory that Guardians are actual Darkness. The theory bases its foundation on that Grimoire card, in the sense that Darkness is related to Death in some way shape or form. Another supporting part of the game is a video clip of a Dreg saying "Darkness" over and over as the player assaulted the poor Dreg. This caused an outcry from Year 1, as people started piecing the puzzle together. Guardians are DEAD, and maybe Darkness is a Universal word? I mean, even if it isn't, Variks, the Loyal, can speak English so why can't another Fallen? The point is, Guardians are what they fight-Darkness. If you want more to THIS specific theory, check out Matpat's channel, Gametheorists. He has a whole video dedicated to this theory to those who seek knowledge.