The problem with this is, he doesn't know whether or not you noticed he was at your rear and whether or not you're gonna give way. If you were to notice him earlier and give way, then no problem. But if he were to just overtake to the left, and suddenly you noticed and move on to the left as well, there's gonna be a collision. He didn't know what were you going to do and since you didn't budge for quite sometime, he assumed you're not gonna move and decided to overtake from the left.
Those who are oblivious to their surroundings represent the biggest threat to other drivers. You need to be focused all around, that's what the side and rear mirrors are for! Just focusing at what's in front makes you a bad driver and to me, you are not fit to drive. Oblivious aunties, I'm looking at you.