A friend showed me a Reddit post that fixed this issue. I had to call my ISP, Comcast, had them put my router/modem into bridge mode. Than each xbox was assigned it's own IP address. Restarted everything including a hard reset on the Xboxes and now we can join each other's fire team