Firstly, if you reread the OP, nothing in it implies he feels guilty about his actions. The only thing that seems to be bothering him is that his wife and son are giving hia girlfriend the cold shoulder. Secondly, although guilt is a terrible feeling, I've learned from experience that pleasure outweighs guilt. People will constantly feel guilty and terrible after their mistakes, but will commit the same offense if it pleasures them in some way.