Why Business Ratings Should Be Open

When the federal government hires a contractor, it judges the company by rules anyone can read. They are written in the Federal Acquisition Regulation. Among other things, a contractor needs a satisfactory performance record, a record of integrity, and adequate resources.

When you choose a plumber from an online directory, you usually can't see the rules. You get a list, some stars, and maybe a badge. The directory rarely explains why one company sits above another.

The order of that list decides who gets the call and who gets into your home. We think the rules behind it should be public too.

How ratings are made today

Most platforms describe what goes into their rankings. Few say how much each part counts.

Google says local results are mainly based on relevance, distance, and popularity. It also says no one can pay for a better local ranking. Google says it keeps the algorithm details confidential to make ranking fairer.

Yelp's default sort considers search terms, distance, ratings, user engagement, and reviews. It also gives extra weight to transactions made through Yelp, such as quote requests and reservations. Star ratings count only the reviews that Yelp's software recommends. In 2025, that was about 70% of new reviews, according to Yelp's Trust & Safety Report.

Thumbtack ranks pros by job match, rating, response speed, and how engaging their profile is.

Angi explains in its FAQ that Angi Approved pros appear above other providers. According to the same FAQ, Approved pros pay to advertise on Angi.

Unlike these four, the Better Business Bureau publishes the elements of its letter grade and the points for each. For example, unanswered complaints can cost a business up to 40 points out of 100. Customer reviews are not part of the grade.

Why closed ratings hurt consumers

A closed rating isn't necessarily dishonest. But from the outside, an honest list and a dishonest one look the same.

Most directories make money from the businesses they list. A company at the top may have earned that spot with good work, an ad budget, or both. When the rules are private, readers can't tell whether good work or money put a company on top.

Regulators have seen this happen. In 2020, the Federal Trade Commission (FTC) settled with LendEDU, a loan comparison site. The FTC said LendEDU presented its rankings as objective while giving better spots to companies that paid. In 2025, Vermont's attorney general settled with Angi over the term "Angi Certified Pro." Vermont has no certification for contractors, only registration. Angi agreed to drop the term, pay the state $100,000, and point users to state tools for checking credentials.

FTC staff guidance for review platforms says companies should favor transparency. That includes explaining how they determine overall ratings, to the extent needed to avoid misleading consumers.

Open rules make a rating transparent, so anyone can check whether it is honest.

Why closed ratings hurt honest businesses

Closed ratings cost businesses too.

A plumber who falls in the rankings rarely learns why. It could be a bad review, a slow reply, or a competitor's new ad campaign. Without the rules, there is nothing to fix and nothing to dispute.

When the rules are unknown, a business sees two levers: ads and reviews. Both favor size. A bigger budget buys more ads, and more customers leave more reviews. Neither is a direct measure of the work itself.

Sometimes payment was built into the rating itself. Until 2010, the Better Business Bureau gave extra points to businesses that paid for accreditation. Only accredited businesses could earn an A+. A company with a clean record but no membership started behind a paying competitor. After an ABC News investigation, the BBB said it would stop giving points for accreditation.

Three years later, the BBB expelled its largest chapter, based in Los Angeles. The chapter had been accused of trading good ratings for membership fees.

Open rules give an honest business a clear list of what to work on.

Open weights are not enough

Several newer rating sites now publish their weights. On most of the ones we checked, reviews carry the largest share of the score. Publishing the weight of reviews tells you how much the stars count. It doesn't tell you whether the stars are real.

Weights also don't explain why two lists disagree. We compared two well-known "best of" sites in the same metro areas. Of 45 companies one site named among the best, 8 appeared on the other's lists under the same name. Counting similar names, the number is 12. Without the weights and the sources behind each pick, readers can't tell whose reasons hold up.

To us, an open rating has five parts:

• the factors and the weight of each;
• a source and a date for every fact;
• a breakdown of each company's score;
• a free way to correct mistakes;
• a version number and a log of changes.

Three systems that already work in the open

None of these systems is fully open. Each publishes enough for outsiders to understand and question it.

Credit scores

FICO doesn't publish its formula. It does publish the five categories behind its scores and how much each one matters. Payment history counts for 35% and amounts owed for 30%. Length of credit history adds 15%, and new credit and credit mix add 10% each.

The law adds two more rights. If a lender turns you down, it must give you specific reasons, at least on request. Saying you didn't reach the lender's score isn't enough. And if your credit report has an error, you can dispute it. The credit bureau generally has 30 days to investigate.

Government contracts

When a federal agency reviews a contractor's past work, it rates factors such as quality, schedule, cost control, and management. Each gets one of five levels, from exceptional to unsatisfactory, with a written explanation. The contractor has up to 14 days to respond, and disagreements go to a higher-level reviewer.

The rules also protect newcomers. A company with no relevant track record can't be judged on that alone.

Science

A research paper describes its methods so others can repeat the work. Many researchers now register their hypotheses and tests before they collect data. Peer reviewers check the work before it is published. When errors turn up later, journals publish corrections.

In credit and federal contracting, the law requires reasons and a chance to respond. No similar rule requires a directory to explain its rankings. There, openness has to be a choice.

"But won't businesses game an open rating?"

It's a fair question. Google's answer, as we saw above, is to keep the details confidential.

Secrecy makes sense when a rating runs on signals that are cheap to fake. Reviews can be bought, and listings can be faked. Public records are different. Faking a state license or a building permit means falsifying a government record.

When a rating rests on records like these, gaming it starts to look like improving the business. To move up, a company has to get licensed, carry insurance, answer complaints, or publish its prices.

Some details should stay private. We'll publish the factors and their weights, but not the exact thresholds we use to spot fake data. A bank publishes its interest rates, but not its fraud rules. We'll also count what a company says about itself for less than what we can check in public records.

What "open" means at US Directory

US Directory is still being built, and we'll publish our full methodology separately. Its principles are already public. Here is how US Directory will cover the five parts above.

• Factors and weights. A methodology page will list every factor and its weight, along with the current version.
• Sources and dates. Every fact on a listing will show where it came from and when we checked it. Facts we verified in public records will be marked apart from what a company says about itself.
• Score breakdown. Each listing will show how its score adds up.
• Corrections. Any business can ask us to fix a factual error, free of charge. We'll recalculate its score by the same rules, but we won't change a ranking just because someone asks.
• Change log. When the formula changes, we'll record what changed and why.

Our rating also has limits. We don't inspect jobs. And no score can promise how a repair will go.

An open invitation

We don't want to be the only ones checking our rating. We're looking for people who can tell us where it's wrong:

• professors who teach business, statistics, or consumer behavior;
• chambers of commerce and trade associations;
• consumer advocates;
• researchers who study ratings and reviews.

Here is what we'd ask you to do:

• review our factors and weights before we publish them;
• challenge the assumptions behind them;
• suggest public data sources we've missed;
• help us compare our rating with the judgment of outside experts.

In return, you'll see our drafts before they go public. We'll credit each contribution by name in our change log. Readers will see that the rating wasn't built behind closed doors, and you'll get public credit for your work.

To get involved, write to us at [email protected].