Why do we think that killing animals per se is not morally wrong? Why do we think that death is not a harm for non-human animals?
Before the 19th century, animals were mostly regarded as things. Neither our use nor our treatment of them mattered morally or legally. We could have obligations that concerned animals, such as an obligation not to damage our neighbour’s cow, but that obligation was owed to our neighbour as the owner of the cow, not to the cow.
To say that we thought of animals as things didn’t mean that we denied that they were sentient, or subjectively aware, and had interests in not experiencing pain, suffering or distress. But we believed that we could ignore those interests because animals were our inferiors. We could reason; they couldn’t. We could use symbolic communication; they couldn’t.