Decently high-end PC way slower than expected

Post has gotten removed 3 times now, hopefully it works this time.

Anyways, about a year ago I upgraded my entire PC except my GPU, and with only a 1060 6GB quite a bit of lag was to be expected.

However, last month I upgraded to a Gigabyte RTX 4070 12GB... But my framerate barely went up, maybe 80 FPS instead of 60 in some games.

Turns out that I had been running my RAM at 2500MHz instead of 3600MHz this whole time (BIOS said it was at 3600 but it wasn't applied), and after fixing that my performance went up noticeably... Though it still seems way too low considering my specs.

Some examples: In BeamNG, I often drop to 30 FPS on high settings. In Apex Legends I get pretty bad lag spikes on medium-high settings, and in some parts of the map I don't go above 120 FPS at all.

The biggest problems are in VR though (on a Quest 2 if that matters). I remember trying Euro Truck Simulator 2 in VR on a friends PC (he only has a 3050) and we were able to go to almost 175% resolution before lagging... I can barely reach 135% before getting 15fps.

Clearly something is wrong, but what could it be? This is driving me insane.

Specs (no overclock on anything):

CPU: i7 12700k
RAM: G.Skill DDR4 32GB 3600MHz (don't remember the exact model number but can find it if it helps)
GPU: Gigabyte RTX 4070 12GB Triple Fan
MB: ASUS PRIME B660-PLUS D4
PSU: Corsair RM750x 750W
Main SSD: Kingston SNV2S1000G
Most of my games are on this HDD: Seagate ST4000DM004-2CV104
Windows 10 Home
XMP is on

Let me know if you need any more info, thanks!

Edit: Forgot to mention, I'm on 1080p 240hz (although that shouldn't be a problem in VR)