My husband won’t let me do anything for myself

Just for reference I am a 23F with two kids 1F and 3M. I work 40 hours and week and ever since I had my youngest daughter my husband makes passive comments about me doing well, about anything for myself. I wanted to volunteer (just an hour a week) at a barn for kids with disabilities so I can have something else to do with my time besides just being home, doing chores and working. I work from 6am - 230 and am going to volunteer just for an hour after work on Thursdays. I just told my husband and he is giving me slack about it. “And the kids? They are going to be at the babysitter late then?” I get he wants to know logistics but it’s not late, it’s until 4pm. When I wanted to go to the gym he asked why. I said I wanted to better myself. I honestly feel like he’s making backhanded comments and asking questions to shame me into not doing ANYTHING for myself. Mind you all, I pay half of bills and do 100% of the housework. I have no idea why he doesn’t want me to anything for myself, it makes me feel like cr*p, I work so hard and just want an hour to do something nice. What do you all think I should do or say to him? Should I tell him how I feel?