Because everyone here isn't a wizard? You have to have magic to be able to use it, and if you haven't got it, you can't learn it.

There's one little untrained baby wizard here, at least, but just one--and that's got nothing to do with where we are and everything to do with him, and his brilliance, handsomeness, et cetera.