The actual query is a lot longer, I just removed everything that didn't influence the speed. So in actuality only the relevant data is fetched.
I experimented a bit more with what you said, the "explain" command. And there is a difference when I use "!=" or "NOT LIKE". When using !=, the key is...