Why did Richard say he watched them all die?

I’m rewatching Lost for the first time in a long time so maybe I just don’t remember. But in season 5 episode 15, Sun asks Richard Alpert if he was around in 1977, and if he remembers Jack, Kate, Hugo and her husband. And Richard says he remembers them because he watched them all die.

Why did he say that? Some of them don’t die, and Richard isn’t around for some of them that do.