So now you know:

Common entertainment included playing cards. However, there was a tax levied when purchasing playing cards but only applicable to the 'Ace of Spades.' To avoid paying the tax, people would purchase 51 cards instead. Yet, since most games require 52 cards, these people were thought to be stupid or dumb because they weren't 'playing with a full deck.'