Well, back when he was alive, all the people lived in cities on Lionturtles. Most of them didn't even know that there were other humans. The Lionturtles were what gave people bending, and... with the help of Raava, Wan was able to collect all four because they fused at Harmonic Convergence. That was when they sealed off Vaatu.

[Leaving out that whole bit about how the threat of Vaatu was technically Wan's fault in the first place.]