Endermen only become hostile if you look at them for a long enough time

Endermen will only become hostile when looked at for over a second or so. This makes this feature practically useless, as if you accidentally look at one, you have time to move the cursor off.