The Federal Constitution (amended over time) has a list of reasons and qualifications which cannot be used to deny the vote. It is otherwise silent on the matter. Until the 14th and 15th amendments (1868, 1870), that list only included "religious test" (as in no religious test may be used to deny someone the franchise).
The several states each had (and have!) their own laws and regulations. Immediately following the ratification of the Constitution, states largely restricted voting rights to landholding white men. But by the 1840s most states had extended the right to vote to almost all white men, having done away with the requirement to own land, or pay taxes.
States continue to have different laws regarding voting rights, perhaps most notably surrounding the the rights of convicted felons.
Depends on the state, but federal constitutional protection of voting rights for adult men didn't exist until the 14th Amendment in 1868. Until then, voting qualifications were a state matter, and even after that, the "penalty" was simply a proportional reduction in representation for the state.
Wow, I definitely didn't know this. From when were non-landholders (i.e., all adult white male citizens) allowed to vote?