How to read a country risk rating without fooling yourself

Sooner or later, most organisations that trade, source or expand across borders end up looking at a country risk index. Something has put a country on the table, and somebody wants a number. ISO 31000 puts this squarely in scope: understanding the external context means examining "the social, cultural, political, legal, regulatory, financial, technological, economic and environmental factors, whether international, national, regional or local" that bear on your objectives. A country rating is an attempt to compress most of that sentence into one figure.

Compression is not the sin. Nobody can weigh forty indicators in their head, and a defensible summary is genuinely useful for deciding where to look harder. The sin is compression with the workings hidden, because a rating you cannot interrogate is not analysis, it is a vibe with a colour scheme. Five questions separate the two, and they apply to every index on the market, including the free one we publish.

1. Is the method public?

Not the marketing page: the method. Which indicators, from which sources, weighted how, normalised how, refreshed when. If the answer is proprietary, you cannot know what the rating is sensitive to, which means you cannot know when it is wrong. A rating built from published weights over named public sources can be argued with, and being arguable-with is the entire value of writing a method down.

The licence trail matters for a second reason. An index built on data its publisher is not licensed to redistribute has a compliance problem baked into the product, and one day it becomes your compliance problem. The test is simple: does every indicator name its source and the terms it is used under?

2. Are the bands absolute or relative?

Most indices band by rank: a country is "High" because it sits in a particular slice of the scored cohort, not because it crossed an absolute line. There is nothing wrong with that, quartiles are a perfectly honest way to band, but it has a consequence you must know about: a country's band can move without anything in that country changing, because the cohort moved around it. If the index does not tell you which kind of band you are reading, you cannot know whether a downgrade is news about the country or news about everyone else.

The follow-on discipline: compare like with like. An overall band ranked against every scored economy and a category band ranked against the smaller set with data for that category are different statements wearing the same four words.

3. What happens when the data is missing?

This is the question that most reliably separates careful indices from confident ones. Data coverage is wildly uneven across economies: the same indicator set that covers Germany densely covers a Pacific micro-state barely at all. Something has to fill the gap, and there are only two honest answers: disclose it, or refuse to rate.

The unhonest answer is silent imputation, filling the hole with a regional average and rating as if the data existed. A rating built that way tells you about a country's neighbours, in the tone of a statement about the country. What you want to see is a coverage figure on every rating, floors below which the index declines to publish a number at all, and a visible "insufficient data" state that is allowed to be the answer. An index with no such state is asserting that it can rate everywhere, which is another way of saying it is estimating somewhere and not telling you where.

4. Does statistics capture legal reality?

A country can score moderately on its indicators while being legally difficult or outright prohibited to do business with. Sanctions regimes, severed correspondent banking and government travel advisories are facts about your ability to operate that no weighted indicator average will surface in time, because the data that feeds indices lags the gazette by months or years.

A usable index needs a deliberate, documented gate for this: a rule that comprehensive sanctions force the worst band regardless of score, and that serious legal constraints floor the band, with the override visibly disclosed so you can see what the data alone would have said. If the index treats legal reality as just another indicator, the one fact that should dominate the rating is being averaged away.

5. Does the rating hold still when you quote it?

You will put the rating in a paper, the paper will be read in a month, and the decision will be revisited in a year. If the index recomputes continuously, the number in your paper is unverifiable the day after you wrote it: nobody can check what you saw, and you cannot check what changed. Ratings should come from dated, frozen snapshots, so that "High, as at June" is a statement anyone can reproduce, and so that a change between snapshots is itself information.

The as-at date is not a technicality. It is what makes the rating quotable in a governance process at all.

The same questions, pointed at ours

We publish GeoRisk, a free country risk dataset built for Australian businesses, and it was built by asking these five questions in the mirror. The method is published down to each indicator's source, weight and licence. Bands are quartiles of the scored cohort, cut at the 25th, 50th and 75th percentiles, and the pages say so rather than implying an absolute scale. Nothing is imputed: every rating carries its coverage, categories below their floors are not scored, and in the current snapshot 34 of 218 economies show "insufficient data" because that is the true answer. A documented sanctions gate sits after the scoring, with any override disclosed next to what the data alone computed. And everything is generated from a frozen, dated snapshot that is named on every page and stamped into every export.

None of that makes the ratings correct. It makes them checkable, which is the property the five questions are really testing for. A country rating, ours included, is a place to start looking, a way to structure the questions for people who know the market, and a means of keeping the conversation anchored to evidence with a date on it. Any index that offers you more certainty than that is offering you its confidence, not its method.