Why has this made me mad!

My SO had a couple of drinks last night and said I'd love to have ss with us all the time, atm it's 50/50 but we have a weekend free... It used to be every weekend that we had him and then the schedule changed but still 50/50this actually hit a nerve with me and I feel so bad that he can't express that to me although I just said yes it's tough but inside like no oooooo, I don't even know how he could say that our lives have actually improved since the change of custody and I actually think I would leave the relationship if we had him full time, also I love my partner but he seems pretty disengaged with his son when he's here anyway and his son is just dislikeable in general so I feel like he's not really telling the truth or just feels guilty.... I know I might get hate for what I just said cause it's petty and ridiculous but this is my safe place.