SRA pilot working with law firms and comparison sites
The Solicitors Regulation Authority (SRA) has been working with law firms and comparison websites to increase the information about the quality of legal service providers that is readily accessible to the public.
It is worth noting a clause from the pilot’s voluntary code of practice for comparison sites, namely, “information should be displayed in a way which is fair, clear and not misleading”.
Issues for law firms to consider
Firms should consider the following when assessing the use of comparison websites:
- Those using a free service could be at a disadvantage over those paying for a premium service
- A firm may be listed on a comparison site that they have not given approval for
- A firm may be ranked lower than a firm with worse scores
- Competitor firms may be listed and recommended on a firm’s comparison site page if it only has a few reviews
- Some firms are shown with the SRA’s logo under ‘Accreditations’ giving the impression they are accredited by the regulator
Case Study 1 - Relevant reviews
The firm in this case was given a 5* rating based on 51 reviews, which according to the site were no more than 12 months old.
Observations:
- Only 25 of the 51 reviews were within the 12 month period
- A barrister who had been previously instructed by the firm, had left a review about how great it was (looking for more instructions?)
- One review was duplicated
Question – if the barrister and duplicate reviews were removed, and only the ‘current’ reviews used, would the firm still have obtained a 5* ranking?
Case Study 2 - Inaccurate ranking
This case study is based on data from the same website.
Firm A Firm B
Ranked #1 #2
Reviews 915 1,899
Scores (average) 94.3% 97.3%
Question – if firm B had far more reviews and higher scores that firm A, why was it ranked lower?
Case Study 3 - Alphabetical ranking
This case study shows that some firms are ranked in alphabetical order rather than based on their performance, etc.
- Firm A (Ac) rated 2.5* based on 8 reviews
- Firm B (Al) rated 4.5* based on 8 reviews
Case Study 4 - Quantity over quality
This case study shows that more weight is given to the number of reviews rather than being based on performance, etc.
- Firm A rated 1.5* with 8 reviews (ranked higher than B)
- Firm B rated 5* with 5 reviews
Ranking calculations of comparison sites
It is clear that comparison websites use various algorithms to work out rankings, but are they fair, clear, and not misleading; here are some things to consider:
- Time span – more weight is given to newer reviews
- Frequency – higher scores if reviews received frequently
- Bayesian average - used for businesses with fewer reviews start off with a balanced score (value of 7 reviews worth 3.5* each included at the start)
Question – if comparisons are supposed to be based on service, performance, cost, etc., should any weighting be applied for ranking purposes?
Dealing with unfavourable reviews
Unfavourable reviews are a fact of life as sometimes things do go wrong, and these can lead to unhappiness, it is how you deal with these that will determine whether you can achieve a positive or negative outcome for all.
One law firm took a client to court after they left a negative review and won considerable damages, however, its reputation was severely damaged as a consequence, so assess the risks of taking such a step; it may be best to just move on!
Who may be interested in reviews?
Although the SRA has said it will not be using reviews for enforcement purposes, there is nothing stopping PII insurers and accreditation bodies from doing so, for example, could an insurer see a firm rated 2* as a poor firm and therefore not good enough to be covered?
Law firms and comparison sites: A conclusion
It is clear that the use of comparison websites needs serious consideration by firms looking to use them, and those already using them, as getting it wrong can impact on reputations and future workloads; surveys in the past have shown that consumers will only instruct firms with 4* or 5*, so keeping clear of lower scores is critical!
Our specialist risk and compliance team can review your files and help you to maintain high standards to improve your chances of getting more 5-star reviews.