How do I make better connections with better people

I (22) have had social anxiety most of my life but in the past two years or so have really tried to improve my people skills. It's been pretty hard, and I've seem to consistently, and unintentionally, become associated with kinda shitty people.

It's the same cycle every time. People first meet me and think I'm pretty cool and interesting and want to give me lots of attention, in a weird way almost idolizing me. Then they pull back and start to either intentionally or unintentionally exclude me as they find a new "favorite person." I notice that they can go on and on talking about something to me but the second I start talking they kinda lose interest and don't pay as much attention and keep their responses really short. Eventually it feels like they get passive aggressive with me too. It's sad cause I meet most of these people through shared interests

I have a hard time keeping conversations going so my default is to just be a really good listener. I've had people say I'm very calm and collected and quiet. I end up being way too agreeable with people too cause I'm scared of offending others and don't want to be outcasted. I know they can pick up on this and probably think I'm meek.

But no matter what kinda people I'm around i always end up feeling excluded and they lose interest. I want to be more personable and be able to carry interesting conversations and make connections, but at the same time I don't want to force myself to be an extrovert.

Does anyone have any advice?