The term "silent majority" isn’t a reference to the popular vote. The United States is not a democracy. It's a democratic republic. The Supreme Court retains the right to veto unconstitutional democratic legislation. That’s a good thing because the electoral college was set up so that the majority couldn’t impose its will on the minority.