Once upon a time, a horse owner said to an alternative therapist: “Thanks very much for treating Billy last week. He was much more relaxed than usual in the stable that night, and he went really well when I rode him the next day – more forward going, more supple and more willing than usual. I’d like you to come and do him again”.
Very satisfactory for all concerned. The horse was going well, the owner was happy and the therapist had a new client. Except for one thing – the therapist hadn’t treated Billy last week. She’d gone to the yard as requested and met not the owner but the groom, who through a misunderstanding had asked her to treat another horse. The owner, not knowing this, had ridden Billy the next day and had attributed her good ride to the treatment she thought the horse had had the day before.
Now this alternative therapist had an enquiring and scientific mind and decided to conduct an experiment. She asked a friend of hers whether she could give the friend’s horse a free session of her therapy. She didn’t want the friend to watch what she did, but she did want the friend to give her feedback afterwards. And she didn’t do anything to the horse at all. While the friend thought she was doing the treatment, she was actually sitting in the manger reading a book and the horse was eating his hay. When the therapist later asked the friend what had happened, she was not altogether surprised to hear a tale about a very relaxed horse who “went so much better than usual when I rode him the next day”.
Unlike most stories that begin ‘once upon a time’, this one is true. It illustrates very nicely the danger of attributing a change in our horse’s behaviour or performance to something we have just done. Or, as in this case, that we think we have done.
You might suspect that I enjoyed telling this story because I am highly sceptical of alternative therapies, and you would be right. However, that isn’t really the point here. The therapy in question, unlike many, actually had some biological plausibility: it was a manipulative technique that many people find relaxing and invigorating, and it is not beyond the bounds of possibility to suppose that horses might also get at least temporary benefits. The point is that the intervention (or non-intervention) could have been anything: a veterinary treatment; a new feed or feed supplement; a new saddle or bridle; a visit from the farrier or equine dentist – you name it. The only necessity was for the owner to believe that any improvement in the horse’s behaviour following the intervention must have been a result of that intervention. As we have seen, the intervention didn’t even have to happen. All that was required was for the owner to want to make that connection.
The fact is that a horse will vary from day to day in how lively, enthusiastic, supple or willing he feels and there can be many reasons for this: working hard the day before, a slip in the field, more time than usual in the stable, the weather, the time of day, his social relationships and the amount of sleep he’s had, to name only a few of the possibilities. It is just unfortunate that we, with our pattern-recognising, all-too-human brains, are likely to come to the wrong conclusions about the reasons why, because of what we want to believe. If we have given the horse something we believe will improve his performance – and especially if we’ve paid a lot of money for it – we are likely to think that any subsequent improvement in performance is a result of what we did. As you can see, the fact that a change in behaviour followed an intervention does not prove that the intervention caused the change, and this is where science comes in.
Science is just a way of testing things. It’s not esoteric or mysterious. All you do in essence is ask a question, test it and then come to a conclusion. Imagine you own a nervous horse who is reluctant to eat from a bucket in the stable. Your bucket is black. You wonder whether the colour of the bucket makes any difference. So you go round the yard and borrow as many different-coloured buckets as you can. Each day you give your horse his feed in one of them (including the original black one), and record something like the time he takes to start or finish the feed, or the number of times he knocks the bucket over: something you can put a number to. Then once you have tested them all several times over several weeks, you compare the results from the coloured buckets with those from the original black bucket. You’ve used the scientific method to test whether bucket colour makes a difference to the time it takes your horse to eat a feed. You have results that you can use. If the horse eats best from a blue bucket you’ve solved your original problem. If you find no difference, you’ve still got a result. Now it’s time for a new idea: what else might make my horse reluctant to eat? This is how we make progress in science.
Unscientific ideas tend to start with the conclusion. You have the same horse with the same problem, but rather than doing the experiment, you just go out and buy him a yellow bucket because you believe that according to ancient Chinese philosophy, yellow is the colour associated with comfort, security and eating. The first problem with this approach is all the untested assumptions it makes: that the Chinese philosophy makes sense; that horses see the same colours in the same way that we do; that they respond psychologically to colours in the same way that we do and so on. The second problem is the likelihood that having bought the bucket you will then go on to notice all its positive effects while ignoring any negative ones, thereby reinforcing your original decision. And whether the results are good or bad, you have no idea whether the bucket really made any difference, because you haven’t done anything to find out. You haven’t compared buckets of different colours before making a decision: you’ve started with the decision.
What if you then wanted to start a business selling buckets for nervous horses? The person who found out that her horse preferred the blue bucket would have to do a more careful experiment to see whether her results were likely to hold true for all nervous horses, not just her own. She would need to try harder to avoid having her results affected by human errors such as wishful thinking, existing beliefs and her expectations, or by variation due to age, sex or breed among the horses, and would need to show that the positive results for say blue buckets were more than she would expect purely by chance. She would need to know whether the results made sense in relation to what is already known about horse biology, such as whether horses can actually distinguish blue from other colours. I won’t go into details, but the scientific method includes techniques to deal with all of these, helping to make a study as objective as it can possibly be. This person could then use her evidence to advertise her buckets: “Blue buckets decrease eating problems for 96% of nervous horses”, if that was what she found. The person who began with the assumption that yellow buckets were the ones to use, however, would have to convince other people to accept her beliefs without any evidence, and so would probably rely almost entirely on testimonials from satisfied customers. You might have thought that was fair enough, before you read the story above and discovered how easy it is to be led astray.
So, if you wanted a bucket for your own picky anxious horses, which bucket-seller would you choose? The person who sold blue buckets because she had done the experiment on lots of horses and discovered that most of them ate best out of blue ones, or the person who sold auspicious Chinese yellow buckets because she believed them to be effective? Sure, this is a trivial example, but horse owners make almost daily choices on behalf of their animals, and some of these choices could have large effects on their horses’ health, soundness and sanity, not to mention their own bank balances. Much of what we are offered has never been properly tested to see whether it is safe and effective. Wouldn’t it be better to buy from someone who had done the work and testing before putting the product on sale? Some things have been tested, and have been shown to be ineffective, but they are still for sale. Wouldn’t you like to be able to tell which these are, before you spend your money? That’s why we need science.
Note to readers: I’m well aware that it is not always easy to tell the difference between what is supported by science and what is not, so I intend to write more about this in my next article.
By Alison Averis
(Enjoyed this article? Then please donate a little to The Equine Independent to keep us writing. You can donate via paypal to email@example.com – Many thanks (Emma Lethbridge -editior))