Downgrading to HCOL Area

Any experiences with moving to HCOL area and downgrading living situation?

I am considering moving back to the northeast from a southern state in MCOL area. Driving factors are primarily family and community for my kids. My current housing situation is pretty ideal however - beautiful waterfront property, boat in the backyard, pool, good access to beach, tons of waterfront restaurants nearby… etc. It’s a lifestyle property for us, and people go on vacation to places like this. BUT… we are 2 full time working parents, kids have no family nearby, school system is disjointed with large catchment areas so setting up play dates is like pulling teeth, and neighborhood is older with very few kids.

The move would put us in or near a northeast beach town. The town has a lot of amenities, good dining, ton of community events, all the kids go to the local school and hang out with each other in the neighborhood. I also have lots of family (including cousins same age as my kids) nearby which is major plus for kids and me/wife. Objectively, these are great towns and great places to raise kids. My kids absolutely love visiting, and tell me they want to live there.

Housing search however has been lackluster. Trying to limit price around 2M (maybe 2.5M) to not completely blow my FIRE plans, but also to limit the amount of of property tax I am willing to pay (very high taxes, so cost isn’t simply a about the purchase price). Homes are older, outdated, small lots, less sheltered from traffic.. etc. I could get something walkable to the beach but small property with no yard, or drivable to the beach/downtown with smallish yard and maybe room for a pool (would probably chose second option). I could go 15 minutes inland and get a nicer house/property, but the McMansion developments just don’t interest me at all, and would put us too far from the action and preclude thing like kids biking to the beach to surf after school which is kind of the point. Waterfront is out of the question unless I go 30 minutes away to a much less desirable area. Boat will be 5 minutes down the road at a marina which isn’t bad, but not the same as having it just out the back door.

I know this is a first world problem, but what are your experiences downgrading your house in exchange for gains in the family and lifestyle categories?

Early 40s, HHI 800K, NW 7M (4.5M invested, 1M vacation/rental, 2M primary with 1.5M equity in primary).