To us, the phrase refers to another matter entirely than the subject at hand – so you could say “I always thought you hated history but then I found out you read History Revealed. That’s a horse of a different colour!”
It is speculated that the phrase comes from horse-trading. When horses are sold, their registration of birth also changes hands. As some horses change colour from youth to adulthood, however, the registration may not match the actual animal, causing confusion and accusations of foul play.
The expression, like so many in the English language, was popularised by William Shakespeare. In Twelfth Night, the scheming Maria utters “horse of that colour”, meaning ‘the same thing’.
Over the centuries, and definitely by the mid-1800s according to records, the use of the phrase reversed.