TrustPilot Lower Conversion Rate – Test Result 18% Decline
Direct Express Auto Transport implemented for five days TrustPilot’s widget TrustBox Carousel to display on the quote page of our web site. The idea was that at the moment of decision TrustPilot would lend still more assurance to the consumer and increase our conversion rate in the manner of some of their case studies that reported as much as 20% higher. That is a very impressive figure and even if we were able to realize a fraction of that, it would pay for the hefty account membership fee. However, we did not have the same positive experience, instead realizing a nearly 18% decline in conversion rate, and offer an opinion here as to possibly why.
- The TrustBox Carousel shows your company‚Äôs star rating, number of reviews and TrustScore, including a carousel showing your latest 4 and 5 star reviews.
Our five day test compared online car shipping quotes with TrustPilot’s Carousel implemented on the quote page from August 14th to 18th 2014, in comparison to August 7th to 11th when the site did not display TrustPilot’s widget Carousel. There was nothing else changed or different on the car transport quote page during those two five day periods. We chose the August 7th to 11th comparison dates because they lined up exactly with the same days of the week, Thursday thru Monday, as the TrustPilot August 14th to 18th dates. Each five day period had over 1,000 auto shipping quotes or visits to that page, what we consider to be a high enough statistical pool. Conversions declined nearly 18% during those five days.
TrustScore Based On The Bayesian Principle
Our “Average Rating” during the five days was 4.7 and 4.8, which rounded up should equal 5, or so one would think. However, TrustPilot displayed only 4 Stars, as apparently they convert their controversial TrustScore of 8.7 to a star score that equates closer to 4 than 5. The TrustScore is not the same as the Average Rating, which is what most other review sites use, and they apparently ignore it. The TrustScore algorithm is based on the¬ Bayesian Principle, which was “named for Thomas Bayes, an English clergyman and mathematician. Bayesian logic is a branch of logic applied to decision making and inferential statistics that deals with probability¬†inference: using the knowledge of prior events to predict future events. Bayes first proposed his theorem in his 1763 work …¬†. Bayes’ theorem provided, for the first time, a mathematical method that could be used to calculate, given occurrences in prior trials, the likelihood of a target occurrence in future trials. According to Bayesian logic, the only way to quantify a situation with an uncertain outcome is through determining its probability.”
In essence, TrustPilot’s so called TrustScore is not a reflection of the reviewers Average Rating as it is a prediction of the future. It applies greater weight to the most recent or freshest reviews, which in our opinion is a double edged sword. On the one hand, negative reviews in the distant past are mathematically diminished as the fresher more positive reviews are given more weight, which could result in a higher TrustScore than the Average Rating. But on the other hand the reverse is true, a recent negative review will be afforded more mathematical weight, giving the complainer still more pop, and possibly reducing the TrustScore to less than the Average Rating. And it might take considerable time for the effect to taper off. A major assumption is employed with Bayesian logic that is not necessarily true. To wit, that recent events are a better barometer than older events to predict the future. Some might argue the exact opposite, that to be fair the longer history of any company puts into relative perspective any fresh event.
We believe the so called TrustScore of 8.7 that we were assigned, the 4 stars consequently displayed, and choosing not to show our actual TrustPilot Average Rating of 4.8 played a factor in lowering our conversion rate nearly 18% online for those five days. Seems to us folly to advertise anything less than 5 Stars or a 9.0 TrustScore. We submit here that TrustPilot’s use of Bayesian logic superimposes itself on the consumer. Instead of the potential customer viewing the average ratings of a company and basing their own predictions of the future upon what they see and read, TrustPilot’s Bayesian logic attempts to do that thinking for them. We are not at all convinced that 250 years ago Thomas Bayes figured out an accurate mathematical formula for scoring online reviews.
Other Mitigating Factors Possible
Other factors could have played a part, which we readily acknowledge. Perhaps market auto transport pricing changed and we were momentarily not as competitive. We doubt it, but possible. Perhaps it was a statistical anomaly and had we tested for a longer period that the car shipping orders conversion rates might have improved. Indeed, Google Adwords suggests nearly double the period of time for their own customers. However, when you feel in your gut you are losing car transport business for a particular reason, it sure is hard to keep going. Perhaps we chose the wrong display widget or placed it improperly. Could be a design thing, so to speak. But all agreed that the TrustBox Carousel looked handsome on our auto shipping site, so we doubt that too.
We took the TrustPilot Carousel down as we are unconvinced that it helps, and may even hurt sales. When in doubt – punt.
Making Sense Of Thomas Bayes and TrustScore
We are amateurs when it comes to statistics. So take this with a grain of salt. No … a shaker. We don’t see where the recency of positive reviews pushes the TrustScore higher. We only see the mere existence of one negative review pushing it down.
In our opinion, time is not a major factor. In the report of our company reviews below, the oldest review is 36 days old and the newest 1 day old. We say that 36 days makes them all fresh, and that as of today, August 21, 2014, our 4.83 Average Rating should be rounded to a 5 Star Rating. But as of today we have a TrustPilot TrustScore of 8.8, which is barely changed as five fresher reviews of 5 stars have been entered. The fresh 5 star reviews apparently don’t have the same weight or effect on scoring as negative reviews. So is the 4 stars emanating from a TrustScore of 8.8 a reliable indicator of reality, much less probability? We don’t think so.
Let’s break this down by recency since TrustPilot purports to weigh greater the freshest. Remember that at present TrustPilot is reporting our overall ¬ TrustScore as 8.8 (equals 4.4) and awards only 4 Stars:
Group 1: Most recent six auto transport reviews (about 20% of total) equaling an Average Rating of 4.83
Five @ 5 Stars
One @ 4 Stars
Group 2: Second most recent six car shipping reviews (about 20% of total) equaling an Average Rating of 4.33
Four @ 5 Stars
One @ 4 Stars
One @ 2 Stars
Group 3: Third most recent six car transport reviews (about 20% of total) equaling an Average Rating of 4.83
Five @ 5 Stars
One @ 4 Stars
Group 4:¬†Fourth most recent six auto shipping reviews (about 20% of total) equaling an Average Rating of 4.50
Four @ 5 Stars
One @ 4 Stars
One @ 3 Stars
Group 5:¬†Fifth most recent six vehicle transport reviews (about 20% of total) equaling an Average Rating of 5.00
Six @ 5 Stars
Group 6:¬†The oldest automobile shipping review, only 36 days old (about 3% of total) equaling an Average Rating of 5.00
One @ 5 Stars
As you can plainly see, of our 31 vehicle shipping reviews: 25 @ 5 Stars, 4 @ 4 Stars, 1 @ 3 Stars, and 1 @ 2 Stars, the overall average rating is 4.83. Currently as of this writing, TrustPilot reports on their website our average rating is 4.9. Yet our TrustScore today is 8.8, equating to 4.4. They show us publicly as a 4 Star company. What gives? Obviously the lone review of 2 stars and the other of 3 stars does not just weigh heavier in the Bayes inspired TrustScore, we say it is somehow exponential and drags the overwhelming majority of positive 5 Star reviews down. We paid thousands of dollars in account membership fees for the pleasure.
Therefore as an automobile transport business owner sensing that the negative review is likely exponential, it is a risk to rely upon the TrustScore system to attract customers.
We like the TrustPilot company organization and their website interface. We think they could be a very good service. However, they need to retire Thomas Bayes and his probability statistical formula in favor of just reporting the facts. As you can hopefully tell, there is nothing malicious in this review of TrustPilot. It is meant to be constructive criticism.¬†We think the TrustPilot sales (Lucia) and customer service (Rachel) staff are top notch, highly professional people.¬†They are talented and hard working. Can’t say enough good things about them. If you are considering TrustPilot for your web site, we hope you find our experience informative as we saw nothing else quite like it upon doing our due diligence.
Update On Conversion Rate Six Months Later
After a few frustrating months of stellar 5 Star reviews and a tortoise like climb to a TrustScore of over 9.0, we gave it another go in January 2015 because TrustPilot began accurately reporting Direct Express Auto Transport as a 5 Star company. They were very late to the party, especially for a paying customer, but they did show up. So we displayed the TrustPilot 5 Star rating on our site and conversions went up nearly 18%. That number 18 keeps popping up. Finally TrustPilot was worth it!
The Lesson Learned
Display on your site¬†a TrustScore of less than 9.0 resulting in 4 Stars or less and watch conversions go down. However, display a TrustScore of over 9.0 resulting in a rounded 5 Stars and watch¬†conversions go up. So for TrustPilot and probably all other review boards … it’s 5 Stars or bust.