Why are Presidents counted twice for serving non-consecutive terms, yet House Speakers that served non-consecutive term are counted once?

If we’re going to count Presidents twice for serving non-consecutive terms, then why don’t we count each count each term as separate presidencies (e.g. Washington being the 1st & 2nd POTUS)?