I am Article Layout

Now Reading: ESG ratings under the microscope

Select your investor profile:

This content is only for the selected type of investor.

Analysing corporate ESG ratings

October 2023
Marketing Material

ESG ratings under the microscope

ESG corporate ratings are redrawing the financial landscape, yet investors often struggle to interpret them.

01

Overview

Britain’s system of imperial measurements has long bamboozled and frustrated foreign visitors. The British themselves aren’t averse to complaining about it either. In a lecture to American students in 1884, the eminent Scottish mathematician Lord Kelvin described the UK’s ounces, yards and gallons as a “wickedly brain-destroying system of bondage under which we suffer.”1

Similar criticisms have been levelled at corporate ESG ratings. Not unlike the imperial system, the scoring frameworks that assess how far companies take environmental, social and governance considerations into account often sow confusion. Quantification doesn’t always provide clarification.

There are more serious grievances, too. Advocacy groups claim ESG scores reveal little about a company’s broader impact on society or the environment. The ratings are, they say, invariably specious and inconsequential.

The charge sheet, then, is a substantial one. But not all of it is reasonable.

This study takes a closer look at both the benefits and limitations of ESG corporate ratings and analyses the differences in philosophy and methodology across ratings providers. It then offers guidance on how ESG ratings are best used.

02

Misunderstood objectives

Many of the misgivings about ESG ratings simply reflect a misunderstanding of raters’ objectives.

There are two main approaches that ESG rating agencies take, each serving a  different purpose.

The first, which is used by the two dominant providers MSCI and Morningstar Sustainalytics, trains the spotlight on the financial materiality of ESG. It looks at how, say, changes in social attitudes, weather patterns or regulations might affect a company’s revenue growth and future profitability.

Crucially, the framework doesn’t claim to issue pronouncements on whether a firm is a good corporate citizen or whether its products and services are positive for society or the environment.

The second approach, embraced by a number of smaller ESG ratings agencies, does attempt to assess the broader impact companies have on the wider world.

The framework takes into account matters such as a firm’s compliance with global labour or human rights norms, and the promotion of access to basic sanitary services. For this reason, it's an approach that can be useful for investors pursuing environmental or social objectives.

We refer to these two approaches as ‘issuer-ESG risk’ and ‘stakeholder-impact’-based ratings, respectively.


Fig. 1 - Interconnected - the two approaches in ESG ratings systems
fig1- Stakeholder vs issuer_v2
Source: Pictet Asset Management

As Fig. 1 illustrates, the two approaches are not entirely separate from one another: impact assessments do sometimes feature in issuer risk ratings just as the financial materiality of ESG can occasionally be exposed by stakeholder impact analyses.

Issuer-risk rating agencies such as MSCI consider a range of positive and negative ESG impacts, insofar as they represent a future economic cost or benefit to the company. Take the example of a logging firm whose activities are known to damage flora and fauna in protected areas. Under an issuer risk ESG scoring system, the risk of the company losing its license to operate in that region would be captured in the analysis precisely because it’s an outcome that has severe economic consequences.

By the same token, impact ratings can serve as indicators of future ESG-related financial risks. These aren’t near-term threats but slower-evolving ones that, left unattended, could reduce profitability or revenue over the long term. One example of such a risk is biodiversity loss. Few companies currently attach much importance to the impact they have on natural habitats. But with the protection of biodiversity becoming a priority for both policymakers and insurers worldwide, businesses may soon need to change tack or risk, say, a future regulatory crackdown that could have financial repercussions.

03

Methodological miscellany

Much of the criticism aimed at ESG scoring stems from the methodological inconsistencies that exist across ratings providers. Even when two agencies pursue the same objective – whether that’s measuring financial materiality or stakeholder impact - they do so in different ways, which means that the scoring systems can give contradictory signals.

A company judged to be insulated from ESG risks by one rating agency can be rated as dangerously exposed by another.2 Complicating matters further, agencies have not been providing enough publicly available detail on how they reach their conclusions.

Investors often find they have to reconcile conflicting ratings even when the providers have the same stated objective.

Academic research testifies to the investor’s plight.

Numerous studies have found that agencies often give divergent assessments of the same company; the correlation between any two ratings on the same firm varies from 0.71 to 0.38.3 By comparison, the correlation in the credit ratings of Moody’s and S&P is estimated to be in the 0.96 to 0.98 range.4

In an influential study, Berg et al. (2022)5 identified the root causes of ratings divergence across six ratings providers.

These fell into three categories:

  1. ESG ratings agencies differ in terms of the scope of ESG categories they analyse (see appendix 2 for an illustration of scope choices within consumer industries).
  2. Different ESG ratings providers rely on different measurement approaches within ESG categories. They often use different indicators (e.g. same indicators across industries or industry-specific indicators) and/or process the same indicators differently (see box in this section).
  3. Different ESG ratings providers apply different weights to ESG categories.

The authors found that differences in measurement, scope and weightings accounted for 56 per cent, 38 per cent and 6 per cent, respectively of the ratings divergence.

The discrepancies in measurement were concentrated in a small number of ESG categories, including climate risk management, product safety, corporate governance, corruption and environmental management systems.

This analysis suggests that there may be room for some limited convergence of ratings over time particularly if ESG data published by companies becomes more standardised.

But even if agencies use the same data, they will likely process that information in different ways. That is largely because ratings providers will continue to differ in their philosophical approach to ESG.

Building on this analysis, we conducted our own study, focusing on how industry sectors and geographical regions were scored by the two main ratings providers Morningstar Sustainalytics and MSCI. Our frame of reference was the MSCI All Countries World index, for which the common coverage between the two providers is approximately 2,900 companies.

To conduct the study, we first made a number of adjustments to render the data comparable. The recalibrations were necessary because of methodological differences between the two agencies. Sustainalytics assigns each company an absolute score, which means a corporate ESG rating can be compared to that of any other business in any other industry.

By contrast, MSCI computes relative ESG scores. These are deliberately designed to allow for intra-industry comparisons, identifying the laggards, leaders and average performers by industry.

Figure 2 illustrates this, using the average standardised scores behind the ratings (z-scores) of both providers for two industries: oil, gas and consumable fuels (OGCF) and media.

Fig. 2 - Contrasting outcomes - absolute vs relative ESG ratings of selected industries
Average z-score behind ESG ratings 
fig 2
Source: MSCI, Morningstar Sustainalytics, Pictet Asset Management, as of 28.02.2023

For MSCI, the average scores are close to zero, reflecting the normalisation of the scores within those industries. In contrast, Sustainalytics’ scores for both industries are skewed, but in opposite directions. Companies in high risk industries such as OGCF are assigned worse ESG scores than those operating in lower risk industries like media.

To compare MSCI and Morningstar Sustainalytics ratings, we use both the scores behind MSCI’s headline ratings (relative scoring) and – to obtain an apples-to-apples comparison – we also consider the absolute version of MSCI ESG scores. Separately, we switch the sign of the Sustainalytics scores (absolute scoring), since Sustainalytics provides a risk score (lower is better), while for MSCI, higher values are better.

The correlation between Sustainalytics and MSCI’s headline ESG score is found to be 0.49. Figure 3 provides results for Sustainalytics against the absolute version of MSCI ratings, where the average correlation across the global sample of companies is 0.55. There are two important observations from this analysis.

First, the difference between the two values results from the difference in the measurement approach (absolute vs relative) of the two rating agencies. The results do indicate, however, that even after accounting for this difference in approach, the correlations between the ratings given by the two providers remain relatively low; this suggests the divergence arises from other aspects of measurement, as well as differences in both the scope of activities being assessed and the weightings regimes used.

The second conclusion to draw is that there is considerable variation in the correlations of ESG scores between regions. Scores for the Middle East exhibited the highest correlation, while correlations across the Americas were the lowest. And when companies are grouped by market capitalisation terciles, it would appear that correlations are broadly similar for large, mid and small-cap firms at around approximately 0.5.

Fig. 3 - Agree or disagree?
Correlation of Sustainalytics-MSCI scores by region, market cap 
fig 3 correlations MSCI vs Sustainalytics
Source: MSCI, Morningstar Sustainalytics, Pictet Asset Management; data as of 28.02.2023

Differences in ESG measurement

Divergences in corporate ESG ratings stem from a number of factors:

1 - Data source: One distinction is the extent to which the data used in the ESG score is disclosed by the company (inside-out data) or provided by external sources (outside-in data). Inside-out data is provided in companies’ financial or sustainability reports, or in rare cases in surveys. Outside-in data comes from alternative sources such as non-governmental organisations or the media. If ratings agencies make greater use of non-public outside-in data, the divergence in ESG scores could widen.

2 - Data normalisation: Individual raw data points (e.g. greenhouse gas emissions) need to be transformed before they enter into an ESG rating. This typically means dividing by a measure of company size (e.g. assets or revenues). It also require transforming the data into a scale that makes it comparable with other indicators (e.g. a z-score or percentile ranking, truncation of outliers). Rating providers can reach different decisions on any of these points.

3 - Absolute or peer-group relative indicators: Even if two ratings providers were to make the same choices for data sources and normalization, the final presentation of their rating may still differ. Quite prominently, some rating providers provide absolute ratings that are comparable across industries, while others provide best-in-class ratings that are normalized within industries to ensure the presence of high-rated and low-rated companies within each industry.

Fund ESG ratings create further divergence

ESG fund ratings also merit close scrutiny. 

Typically, the ESG scores assigned to funds are based on the ratings of the individual companies held in the portfolio.  In other words, they represent a snapshot of the average ESG risk of the fund’s constituents.

Consequently, ESG fund ratings do not gauge the extent to which ESG considerations are integral to the fund’s investment process or the philosophy of the investment management team. Nor can investors assume ratings will remain stable over time.

Rendering these scores less reliable are the minimum coverage requirements agencies use in assigning fund ratings.

Funds can qualify for an ESG score even if as many as one in three of the portfolio’s holdings are unrated.

In cases where coverage is close to this minimum threshold, the ESG fund rating can paint a misleading picture of its holdings. So it is important for investors to take the coverage percentage into account when assessing ESG fund ratings.

Making matters more complicated are methodological divergences in how company-level ratings are aggregated to produce a fund-level rating.

One additional layer of complexity is the existence of absolute and relative fund ratings.

Some fund ratings, such as those provided by MSCI, are calculated in absolute terms while others, those given by Morningstar, for example, are expressed relative to a peer group of funds.

In sum, investors assessing fund ratings need to be aware of

  • different methodologies in the underlying issuer ESG ratings
  • different aggregation methodologies of the underlying ESG ratings, including the use additional ESG data (e.g. momentum)
  • different fund peer groups.

Note that until recently MSCI also incorporated issuer ESG momentum – the rate at which companies were making progress in addressing ESG issues – into its ratings. Under this system, funds that held companies with improving levels of disclosure were awarded higher ratings, which skewed the fund ratings distribution.

The methodology has recently been revised and a new system was released in April 2023 that abandons ESG ratings momentum. This change has reduced the skew in the distribution toward higher fund ratings. 

04

ESG ratings biases

It is often claimed that ESG ratings are subject to biases. Research shows larger cap companies tend to attract higher ESG ratings than smaller firms, and that US-based businesses score less well than their European counterparts.6

Our own analysis paints a similar picture. Using the same dataset of the previous analysis, we compute average z-scores for Sustainalytics and MSCI headline ratings by geographical region and by company size (broken down into market capitalisation terciles).

The analysis – summarised in Fig. 4 – shows the following:

  • On average, European companies score higher than firms in the Americas, which in turn fare better than those based in Asia Pacific.
  • Scores for developed markets tend to be higher than those for emerging markets
  • Large caps tend to score higher than mid caps, which in turn tend to score higher than small caps.7
  • These patterns hold for both MSCI and Sustainalytics ratings
Fig. 4 - Average Z-scores of corporate ESG ratings by region and market cap segment
fig 4 Z scores
Source: MSCI, Morningstar Sustainalytics, Pictet Asset Management. Data as at 28.02.2023

One striking difference between the two ratings agencies is how they score companies based in the Middle East - MSCI gives them much higher ratings than Sustainalytics.

This results from the differences in methodology, and in particular how oil and gas industries are analysed. MSCI headline ratings, which are normalised within industries, do not systematically penalise the region for its reliance on oil and gas companies, which as we saw before, are poorly assessed by Sustainalytics.

These rating biases can create problems for investors: they favour large-cap developed market equity portfolios and disadvantage more diversified all-cap strategies that cover all regions, including those invested in emerging market and thematic equities. 

05

What ESG ratings don't do

One important aspect of responsible investing that most rating agencies neglect is the extent to which a company’s products and services contribute to building a sustainable economy.

For a growing number of investors, that is a significant oversight. Companies that develop and sell, say, recycling products or energy-saving devices are central to the green transition.

An important characteristic of these specialist firms is that their products contribute to a systemic positive change. Take the example of a firm that develops and sells technologies that cut waste in manufacturing. Should its products prove commercially successful, it can potentially reduce the environmental footprint of entire industries – from consumer durables to fashion. ESG ratings fail to capture this.

All of which means the information ESG ratings contain is of limited use for investors wishing to finance companies that bring about a positive change. For such objectives, other tools are needed (see appendix 1 for an explanation of thematic purity).

Other shortcomings common to ESG ratings systems are that they don’t provide clear signals for investors seeking to apply portfolio exclusion lists based on companies’ product mix. Nor do they assess the severity of corporate controversies such as compliance failings. Such matters are instead treated as just one input among many others. Both use cases would require different datasets which are generally available from the same ESG rating firms.

06

ESG ratings as a source of alpha

In our opinion, investors should consider using ESG ratings in the same way they deploy the buy, hold and sell recommendations issued by investment research firms. In other words, they should treat the scores as investment opinions.

Doing so would help negate some of the criticism levelled at the industry.

First, it would reduce the need for ratings to be consistent across providers. As long as ratings agencies were sufficiently clear on what their scores show, their pronouncements could be judged in their own right.

Second, such a set-up would create the conditions under which select ESG ratings could distinguish themselves as a potential source of alpha. Imagine a situation in which ESG scoring systems converged and sufficiently large numbers of investors used them. Under this scenario, financial markets would very likely embed this information into securities prices. ESG ratings would offer no predictive insights and any ESG premium would largely disappear. The alpha seekers would end up dissatisfied.

In the alternative set-up, where ESG scores are investment opinions, raters with superior systems would have the opportunity to better demonstrate their alpha-generating potential.

In practice, while the correlations of the scores issued by different ratings providers are fairly low, evidence that ESG ratings can contribute to investment performance is mixed.

Studies conducted so far across different rating providers, markets or time periods suggest there is no clear relationship between ESG ratings and excess returns.8

There is, however, some evidence that higher ESG ratings are associated with lower future risk,9 and that a continuous improvement in a company’s ESG score can lead to an outperformance of its share price.10

Because ESG scoring systems vary from provider to provider, it is not surprising that researchers have found it difficult to determine whether ESG ratings are predictive of investment returns. Studies that focus on portfolios of companies mostly test one rating system at a time, yielding conclusions that can’t be generalised. Those that compare the performance of ESG and non-ESG funds, meanwhile, often aggregate different types of ratings systems, which undermines the credibility of their findings.

Another problem in trying to identify a performance signal within ESG ratings is that scoring methodologies are in flux. Ratings systems are regularly refined and many agencies retrospectively change their historical scores in order to offer longer time series.

While this is done to help investors compare how companies have managed their ESG risks over time with a consistent methodology, it has the potential to create an upward (positive) bias in the estimated link between ratings and future investment performance.

This is because the restated historical ratings may reflect information that was not available at the time. In other words, in some cases, the adjustments may overstate the ability of ESG ratings to predict future investment performance.11

07

ESG use cases

We suggest there are a number of practical applications for ESG corporate ratings: 

  • ESG ratings are commonly used in the construction of passive ESG indices, which are generally built in one of two ways: either through a process of screening, which excludes companies with lower ESG scores, or through tilting, whereby the company weights of a reference index are adjusted to favour higher-rated constituents.12 These indices are then used to create low-cost passive ESG investment vehicles.
  • Passive ESG indices are in turn used as benchmarks for active ESG investment strategies. The choice of an ESG index is, somewhat ironically, an active decision that require a thorough assessment by investors. Asset owners need to take great care in ensuring the benchmark’s construction is compatible with the portfolio manager’s philosophy and investment process. For example, for a sustainability-themed portfolio, using an ESG benchmark created by screening a universe with standard ESG ratings is not meaningful.
  • Security selection and portfolio construction. ESG ratings can be used in a variety of different ways to guide portfolio construction. They can be used to screen out companies that perform poorly on ESG or to tilt portfolios towards investments that carry high ESG scores. ESG ratings can also be used in valuation models to adjust the cost of capital. Still, by using ESG scores in these ways, investors should only choose ratings whose methodologies are compatible with their own investment objectives and philosophy (e.g. issuer-risk versus stakeholder-impact perspective, or absolute versus industry-relative perspective). Alternatively, data underlying ESG ratings can be used but processed differently to align with an investor’s understanding of optimal scope, measurement and weighting.
  • Capturing portfolio-level ESG credentials. This applies to reporting by asset managers with investors / clients as the target audience. Here there is often a tension between relying on ratings that are accepted and recognised by clients versus using ratings that reflect the actual process of ESG integration in the portfolio. It also applies to portfolio comparison done by fund selectors. Here the crux is relying on a rating system that captures the fund selector’s preferred ESG integration philosophy and can be applied at scale.

ESG analysis beyond ratings

Ultimately, ESG ratings can be a useful short cut in ESG analysis. It is undoubtedly convenient for investors to be able to aggregate a large number of ESG indicators into one comprehensive rating. However, this approach has its limits. In particular the aggregation of individual ESG signals carries the risk of hiding relevant information.

Our approach is to identify the most relevant ESG indicators and present them in a dashboard that portfolio managers can use to identify problem areas that require further analysis. One advantage of this approach is that it identifies outliers. Further, deeper analysis might be required, analysis but ultimately the dashboard enables investment managers to better understand the companies they own.

Fig. 5 - ESG scorecard example, Pictet Asset Management
ESG-ratings-paper_797px
Source: Pictet Asset Management

Where we believe ESG ratings aren’t particularly useful is in corporate engagement.

Effective engagement requires companies and shareholders to agree on a very narrow set of strategic priorities. Overall ESG scores – which aggregate information across multiple dimensions – don’t provide the level of detail that can inform engagement programmes. Investors would need access to disaggregated sub-indicators of ESG-related factors to uncover relevant weaknesses and strengths.

08

The drumbeat of ESG regulation

Alarmed by the ESG ratings industry’s deficiencies,  regulators have begun to take action. In 2022, Japanese authorities introduced a code of conduct for ESG ratings providers,13 while the EU has recently conducted its own consultation ahead of an expected regulatory overhaul.14 Many of those taking part in the consultation have apparently called for greater transparency on agencies’ methodologies. The European Commission is expected to issue a legislative proposal by the end of 2023.15 The UK government is due to begin its own consultation later this year.16

In public statements, regulators have aired a number of concerns and observations:

  • ESG-linked assets now account for a large proportion of investible assets, with ESG ratings playing a critical role in the development and growth of responsible investment mandates;
  • Consolidation is gathering momentum in the ESG ratings industry, which risks creating an oligopoly;
  • There are potential conflicts of interest when ratings providers also offer ESG consulting services to the companies they rate;
  • ESG providers need to be more transparent over the methodologies and processes they deploy.
09

Summary and conclusion

  • ESG ratings provided by the two dominant agencies do not assess the environmental or social impact of a company’s products and services; their focus is on the financial risk to the issuer arising from ESG factors. While some ESG rating agencies do place more emphasis on the positive or negative externalities of corporate activities, they do not provide a detailed assessment of the positive impact a company’s products or services have in the building of a sustainable economy.
  • Ratings agencies use different methodologies, with diverging results that often sow confusion among investors. A company scored poorly by one agency can be awarded a high score by another. Raters do not provide enough public information on how they calculate ESG scores. Another problem is that every rater has its own systematic bias – regional and company size biases are among the most common. The underlying methodologies used by ratings agencies can also be incompatible with an asset owner’s own ESG philosophy.
  • ESG fund ratings aggregate the ratings of underlying portfolio holdings according to a formula. This adds a layer of methodological sources of divergence.
  • For portfolio construction: ESG ratings can be used to create passive ESG strategies and ESG benchmarks. Their use is subject to all the caveats discussed in this report. Managers of active portfolios can also consider disaggregating ESG ratings for greater insights and complement them with a broader sets of ESG data.
  • ESG ratings can serve as a starting point for ESG analysis. They provide incentives for companies to provide clearer information on the ESG risks they face and to adopt policies they may not have considered otherwise.
  • Because ESG ratings are an average of scores taken across multiple dimensions of company risk, they are an insufficient basis upon which to build a corporate engagement programme.
  • Looking ahead, we expect the ESG rating landscape to evolve rapidly, as a result of regulatory scrutiny and the emergence of new competitors relying on alternative data methods.

Appendix 1 - Thematic purity, green taxonomy

For investors looking to gain exposure to companies that make a positive contribution to the environment or society, traditional ESG ratings are of limited use.

In our view, such investors would be better served by focusing on other business metrics, such as the percentage of a company’s products and services that yields a positive contribution.

This is central to thematic investing, in which the percentage of revenues that is aligned with a given environmental or societal theme is a key portfolio construction building block.

The approach has arguably become more relevant in recent years as the emergence of broad public policy initiatives such as the EU’s Green Taxonomy and UN’s Sustainable Development Goals (SDGs) has led to the development of new data tools that can be used in portfolio construction.

These include databases that assess companies’ green revenue or SDG-aligned revenue percentages. While such tools do not negate the utility of ESG ratings, they are clearly intended to capture a very different dimension of sustainability.

Appendix 2 - Scope choices - an illustration using the Premium Brands investment strategy

Knowing which ESG dimensions to include in any ESG rating is an essential element of sustainable investing. Its importance can be illustrated in a case study.

The Pictet Asset Management Premium Brands strategy invests primarily in consumer companies. Its portfolio managers have developed a proprietary ESG scoring system, the key feature of which is the determination of the most material ESG dimensions for every segment of the portfolio. Even in such a relatively homogenous universe, materiality can vary quite significantly from industry to industry. For instance, consider the Travel and the Luxury segments. Based on the team’s assessment of materiality, both segments were found to have three material ESG dimensions in common and three distinct additional ESG dimensions each.

Different analysts, portfolio managers or ESG ratings providers may arrive at different conclusions about which ESG dimensions are the most material. This explains the variation in ratings outcomes arising from differences in scope.

Fig. 6 - Material ESG factors for travel and luxury goods sectors*

process step 4