A rugby fan became "annoyed" and angrily pushed a woman during a bad-tempered confrontation that was sparked after they had both been to watch the same match and were about to drive home.