Humans have emotions that often cloud judgement, they get angry, they hold grudges, they bare malice. The worst judgement they make is transposing these emotions to animals, particularly horses. If a horse gets angry it immediately deals with the cause (another horse) by shooing it away - then it´s over. No grudges, no malice like humans. But as horses mostly have to work with humans they have to learn our body language and tone of voice, because they can´t understand our verbal language. They have perfected this ability to the point where they can detect our emotions without us showing any outward clues.
When they sense your nerves, they to are nervous. As a flight animal they will be constantly looking around for the cause of your nerves, preparing to flee if necessary - hence you often see ´dancing´ horses at the beginning of a competition for example, as the rider is nervous of the course ahead.
When the riders nerves escalate to anger, because their horse wont stand still (!), the horse now senses fear, on top of the nerves, and goes into flight mode. That is their survival instinct. It is not an emotion or a characteristic. Yes, you can breed for characteristics (calmer minds). Yes, some breeds are flightier than others. But flight for survival is the basic instinct of all horses and the sooner humans stop trying to impose human emotions onto horses as a way to explain a certain ´behaviour´, and the sooner they look to the cause of the behaviour, be it the nervous rider, or the horse with back pain to give a very small example of causes, the sooner there will be a lot less nervous horses and happier riders.