I hear this type of grammatical error too frequently.

Problem:
As I have written about “today” and “tomorrow” (both adverbs), “yesterday” is an adverb, so preceding it with the preposition “on” is incorrect.

Explanation:
“Yesterday” already tells you when.

For example, the answer to “When did they start to paint the house?” could be “They started to paint the house yesterday.”

In contrast, “Tuesday” is a proper noun, not an adverb, so the answer to “When did they start to paint the house?” could be “They started to paint the house on Tuesday.”

In other words, the days of the week, which are proper nouns, require the “on” preposition to tell you when, whereas “yesterday” already tells you when.

Solution:
“I saw him yesterday.”