All my jeans break in this area what do?

One is raw denim from Nudie Jeans and the other one is Levis 511. The nudie is 34w and the Levis is 32w. I dont really wash them. Pretty sure 36 is too large for me. What do? I feel like all my jeans last at most 2 years because of this. It is getting quite costly...