ESG Data Quality & Rating Divergence

The reliability and comparability of ESG ratings depend fundamentally on the quality of underlying sustainability data and the consistency of rating methodologies across providers. However, the ESG ratings industry faces significant challenges on both fronts. ESG data quality concerns include incomplete disclosure, inconsistent reporting frameworks, lack of verification, and measurement difficulties for complex sustainability concepts.¹ Meanwhile, substantial divergence in ratings assigned to the same companies by different providers reflects fundamental methodological differences regarding which ESG factors to measure, how to measure them, and how to aggregate individual assessments into overall ratings.² These challenges complicate the use of ESG ratings for investment decisions, corporate benchmarking, and risk management, while spurring ongoing efforts to improve data quality and standardize disclosure requirements.

Understanding ESG data quality issues and rating divergence is essential for informed use of ESG ratings. Users must recognize that ratings represent informed opinions based on imperfect data and subjective methodological choices, rather than objective facts. Awareness of these limitations enables more sophisticated interpretation of ratings, appropriate skepticism about rating precision, and recognition of circumstances where ratings may be particularly unreliable. At the same time, ongoing improvements in disclosure standards, data verification, and methodological transparency offer prospects for enhanced rating quality and comparability over time.

ESG Data Quality Challenges

ESG data quality represents the foundational challenge facing the ratings industry, as rating reliability cannot exceed the quality of underlying data. Multiple factors contribute to ESG data quality concerns, including voluntary disclosure, inconsistent frameworks, lack of assurance, measurement difficulties, and reporting lags.

Incomplete Disclosure remains pervasive, as many companies do not report comprehensive ESG data despite growing stakeholder pressure and regulatory requirements. A Bloomberg survey of European financial market participants found that 63% cited issues with company-reported ESG data as their biggest concern regarding ESG investing.³ Disclosure rates vary dramatically across ESG topics, with climate-related data more widely reported than social or governance metrics. Company size, industry, and geography significantly affect disclosure, with large companies in developed markets and high-impact industries generally providing more comprehensive data than small companies in emerging markets and low-impact sectors. Incomplete disclosure forces rating providers to estimate missing data, introduce penalties for non-disclosure, or exclude companies from coverage—all approaches that introduce uncertainty and potential bias.

Inconsistent Reporting Frameworks complicate data comparability, as companies use various voluntary frameworks including GRI (Global Reporting Initiative), SASB (Sustainability Accounting Standards Board), TCFD (Task Force on Climate-related Financial Disclosures), CDP (Carbon Disclosure Project), and others. While these frameworks share some common elements, they differ in scope, metrics, and reporting formats. A company reporting under GRI may provide different information than one reporting under SASB, even for the same ESG topic. Some companies report under multiple frameworks, creating voluminous but potentially inconsistent disclosures. Others create bespoke reporting approaches that do not align with any standard framework. This inconsistency forces rating providers to map diverse disclosures onto their assessment frameworks, introducing interpretation and potential error.

Lack of Assurance distinguishes ESG data from financial data, as most sustainability disclosures are not audited or independently verified. While financial statements of public companies undergo mandatory independent audits, ESG reports typically receive no external assurance or only limited assurance on selected metrics. The absence of assurance raises concerns about data accuracy and reliability. Companies may inadvertently report incorrect data due to measurement errors, system limitations, or misunderstanding of definitions. More concerning, companies may selectively report favorable information while omitting unfavorable data, or even deliberately misrepresent performance (greenwashing). Without independent verification, users of ESG data must rely on company representations, introducing uncertainty about data trustworthiness.

Measurement Difficulties affect many ESG topics that involve complex, multidimensional concepts not easily reduced to quantitative metrics. Environmental impacts including biodiversity loss, ecosystem degradation, and pollution effects involve scientific complexity and measurement challenges. Social factors including labor rights, community impacts, and human rights in supply chains require qualitative assessment and stakeholder input that resist standardization. Governance quality involves judgment about board effectiveness, corporate culture, and management integrity that cannot be fully captured through observable metrics. Rating providers must develop proxy measures and scoring approaches for these complex concepts, introducing subjectivity and measurement error.

Reporting Lags mean that ESG data often reflects historical performance from months or years earlier, potentially missing recent developments. Companies typically publish annual sustainability reports months after year-end, similar to financial reporting but with potentially longer delays. Some ESG metrics are reported even less frequently or with longer lags. Rating providers working with lagged data may not capture recent improvements or deterioration, reducing rating relevance for current decision-making. Real-time ESG data remains limited, though some providers including Bloomberg are incorporating more timely alternative data sources.

Boundary and Scope Issues complicate comparability when companies define reporting boundaries differently. Some companies report only direct operations (Scope 1 and 2 emissions), while others include supply chain impacts (Scope 3). Some report only majority-owned subsidiaries, while others include joint ventures and associates. Geographic scope may vary, with some companies reporting globally and others excluding certain regions. These boundary differences mean that reported metrics may not be comparable even when using the same frameworks and definitions.

Data Restatements occur when companies revise previously reported ESG data, similar to financial restatements but potentially more common given the evolving nature of ESG measurement. Companies may restate emissions data as calculation methodologies improve, revise social metrics as definitions evolve, or correct errors in prior disclosures. Restatements create challenges for rating providers and users who relied on original data, potentially requiring rating revisions and complicating trend analysis.

Rating Divergence: Evidence and Causes

Substantial divergence in ESG ratings assigned to the same companies by different providers has been extensively documented in academic research and industry analysis, raising questions about rating reliability and comparability.⁴

Correlation Evidence from academic studies reveals that ESG ratings from different providers show far lower agreement than credit ratings. Berg, Koelbel, and Rigobon (2022) found that correlations between ESG ratings from six major providers (KLD, Sustainalytics, Moody's ESG, S&P Global, Refinitiv, and MSCI) averaged only 0.54, with a range from 0.38 to 0.71.⁵ For comparison, credit ratings from different agencies show correlations of approximately 0.99. This substantial divergence means that a company rated highly by one ESG rating provider may receive a mediocre or poor rating from another provider, complicating investment decisions and corporate benchmarking.

Pillar-Level Divergence varies across environmental, social, and governance dimensions. Research indicates that governance ratings show somewhat higher agreement across providers than environmental or social ratings, though still far below credit rating concordance. MSCI's ratings exhibit particularly low correlations with other providers on environmental and social dimensions, while showing stronger alignment on governance.⁶ This pattern suggests that governance factors, which have been assessed by investors for decades, may be more standardized and better understood than environmental and social factors, which have received intensive focus more recently.

Company-Specific Variation in rating divergence reveals that some companies receive consistent ratings across providers while others face dramatic disagreement. Companies with clear ESG profiles—either consistently strong or consistently weak across all dimensions—tend to receive more concordant ratings. Companies with mixed profiles, strong on some ESG dimensions but weak on others, face greater rating divergence as different providers' weighting schemes emphasize different factors. Companies in industries where ESG materiality is contested or evolving may face particular divergence.

Causes of Rating Divergence have been systematically analyzed by Berg, Koelbel, and Rigobon (2022), who decompose divergence into three sources: scope differences (disagreement about which ESG categories to include), measurement differences (different indicators for the same ESG concepts), and weight differences (different importance assigned to various factors).⁷ Their analysis finds that scope differences account for the largest share of rating divergence (56%), followed by measurement differences (38%) and weight differences (6%). This decomposition reveals that rating providers disagree fundamentally about which ESG issues matter, not merely how to measure agreed-upon issues.

Scope Differences reflect divergent views about which ESG topics should be included in ratings and which are material. One provider may emphasize climate change and carbon emissions, while another prioritizes water management and biodiversity. One may focus heavily on labor practices and human rights, while another emphasizes community relations and product responsibility. These scope differences stem from different materiality frameworks, different stakeholder priorities, and different judgments about which ESG factors present financial risks or opportunities. The dominance of scope differences in explaining rating divergence suggests that methodological alignment on measurement and weighting would not eliminate divergence without also achieving consensus on scope.

Measurement Differences occur when providers agree that an ESG issue is material but use different indicators to assess it. For climate change, one provider may emphasize absolute emissions, another emissions intensity, and a third the quality of climate governance and strategy. For labor practices, one may focus on injury rates, another on labor relations and unionization, and a third on diversity and inclusion metrics. These measurement differences reflect different theories about which specific metrics best capture ESG performance on agreed-upon topics. The substantial contribution of measurement differences to rating divergence indicates that even when providers agree on which issues matter, they disagree about how to measure them.

Weight Differences reflect different judgments about the relative importance of various ESG factors. One provider may weight environmental factors most heavily, another may emphasize governance, and a third may attempt balanced weighting across E, S, and G pillars. Within pillars, providers apply different weights to specific issues based on materiality assessments, industry characteristics, or stakeholder priorities. Interestingly, weight differences contribute least to rating divergence, suggesting that providers' weighting schemes are more similar than their scope and measurement choices. However, weighting remains subjective and consequential, as companies strong on heavily-weighted factors will score better than those strong on lightly-weighted factors.

Methodological Choices beyond scope, measurement, and weighting also contribute to divergence. The choice between absolute and relative scoring creates fundamental differences, as discussed in the ESG Rating Methodologies page. Providers using relative scoring (MSCI) rate companies against industry peers, while those using absolute scoring (Sustainalytics) assess against universal standards. Treatment of controversies varies, with some providers integrating incidents directly into ratings and others maintaining separate controversies scores. Aggregation methods differ, with some using compensatory approaches where strengths offset weaknesses and others using non-compensatory approaches requiring minimum performance across all factors. These methodological choices reflect different philosophies about the purpose and proper construction of ESG ratings.

Implications for Rating Users

Rating divergence creates significant challenges for investors, companies, and other stakeholders using ESG ratings to inform decisions.

Portfolio Construction complications arise when different rating providers would lead to dramatically different portfolio compositions. An investor constructing an ESG portfolio by selecting top-rated companies according to one provider might exclude many companies rated highly by other providers while including companies those providers rate poorly. This divergence means that ESG portfolio performance and characteristics depend heavily on which rating provider is used, complicating evaluation of ESG integration effectiveness. Some investors respond by using multiple rating providers and focusing on areas of agreement, selecting only companies rated highly by most or all providers. This approach reduces divergence concerns but may significantly narrow the investable universe.

Performance Evaluation challenges emerge when assessing whether ESG integration improves investment returns. If ESG ratings diverge substantially, studies examining relationships between ESG ratings and financial performance may reach different conclusions depending on which rating provider is used. Meta-analyses of ESG-performance research reveal significant variation in findings, with some studies showing positive relationships, others neutral, and some negative.⁸ This variation may partly reflect the use of different ESG rating providers with divergent assessments. Investors seeking to evaluate ESG integration strategies face uncertainty about which rating provider provides the most financially-relevant assessments.

Corporate Benchmarking difficulties arise when companies monitor their ESG ratings and those of competitors. A company may receive strong ratings from one provider but weak ratings from another, complicating assessment of actual ESG performance and competitive positioning. Companies may face conflicting signals about which ESG issues to prioritize, as different rating providers emphasize different factors. Some companies respond by focusing on the rating provider(s) most influential with their investor base, though this approach may lead to optimization for specific rating methodologies rather than genuine ESG improvement.

Regulatory and Reporting complications may emerge as regulators increasingly reference ESG ratings in disclosure requirements, fund labeling rules, and other regulations. If ratings diverge substantially, regulatory frameworks that rely on ratings may produce inconsistent outcomes depending on which provider is referenced. The European Union's Sustainable Finance Disclosure Regulation (SFDR) and fund labeling framework have grappled with these issues, ultimately requiring disclosure of ESG ratings received but not mandating use of specific providers.

Standardization Efforts and Regulatory Developments

Recognizing ESG data quality and rating comparability challenges, regulators, standard-setters, and industry participants have undertaken various initiatives to improve disclosure consistency and data reliability.

ISSB Standards represent the most significant recent development in ESG disclosure standardization. The International Sustainability Standards Board (ISSB), established in 2021 under the IFRS Foundation, has developed global baseline standards for sustainability disclosure. IFRS S1 (General Requirements for Disclosure of Sustainability-related Financial Information) and IFRS S2 (Climate-related Disclosures) were issued in June 2023 and are being adopted by jurisdictions worldwide.⁹ The ISSB standards aim to create a global baseline of investor-focused sustainability disclosures, improving consistency and comparability of company-reported ESG data. By standardizing disclosure requirements, definitions, and metrics, ISSB standards may reduce measurement differences across rating providers and improve data quality.

SEC Climate Disclosure Rules proposed by the U.S. Securities and Exchange Commission in March 2022 would mandate climate-related disclosures for public companies, including greenhouse gas emissions, climate risks, and climate governance.¹⁰ While the rules face legal challenges and political opposition, their potential implementation would significantly improve climate data availability and consistency for U.S. companies. The proposed rules draw heavily on TCFD recommendations and would require third-party assurance of emissions data, addressing data quality concerns.

EU Corporate Sustainability Reporting Directive (CSRD), adopted in 2022 and being phased in from 2024-2028, mandates comprehensive ESG disclosure for large EU companies and EU subsidiaries of non-EU companies.¹¹ CSRD requires reporting under European Sustainability Reporting Standards (ESRS) developed by the European Financial Reporting Advisory Group (EFRAG), covering environmental, social, and governance topics. CSRD mandates third-party assurance of sustainability disclosures, addressing data quality concerns. The directive's double materiality approach requires reporting on both financial materiality (impact on company) and impact materiality (company's impact on society and environment), providing more comprehensive information than investor-focused frameworks.

Assurance Standards development by the International Auditing and Assurance Standards Board (IAASB) aims to improve ESG data reliability through independent verification. IAASB's International Standard on Sustainability Assurance (ISSA) 5000, finalized in 2024, provides a framework for assurance of sustainability disclosures.¹² As assurance becomes more common, whether mandated by regulations like CSRD or adopted voluntarily, ESG data reliability should improve. However, assurance of ESG data faces challenges including limited auditor expertise in sustainability topics, measurement difficulties for complex ESG concepts, and scope limitations when companies do not control all relevant data (e.g., supply chain emissions).

Rating Provider Transparency initiatives aim to improve understanding of rating methodologies and reduce user confusion about divergence. Major rating providers have published increasingly detailed methodology documentation, though significant proprietary elements remain. Some providers offer tools enabling companies and investors to understand rating drivers and simulate impacts of performance changes. Regulatory proposals in the EU and elsewhere would require rating providers to register, disclose methodologies, and meet governance and conflict-of-interest standards, similar to credit rating agency regulation.

Industry Consolidation may reduce rating divergence as major financial data providers acquire ESG rating firms. Morningstar's acquisition of Sustainalytics, S&P Global's acquisition of Trucost and RobecoSAM, and LSEG's acquisition of Refinitiv (including its ESG data business) have concentrated the industry. Consolidation may drive methodological convergence as acquirers integrate ESG ratings into broader financial data platforms. However, consolidation also raises concerns about reduced competition and diversity of perspectives.

Future Outlook

ESG data quality and rating comparability will likely improve gradually as disclosure standards become mandatory, assurance becomes more common, and methodologies mature. However, complete convergence of ESG ratings is neither likely nor necessarily desirable. Some degree of methodological diversity reflects legitimate differences in investment philosophies, stakeholder priorities, and judgments about ESG materiality. Investors with different objectives—financial return maximization, risk management, impact achievement, values alignment—may appropriately prefer different rating approaches.

The path forward likely involves improved data quality through mandatory disclosure and assurance, greater methodological transparency enabling informed provider selection, and user sophistication in recognizing that ratings represent informed opinions rather than objective facts. Investors and other rating users should understand the methodology of their chosen provider(s), recognize rating limitations, and consider using multiple providers or supplementing ratings with direct ESG analysis for critical decisions. Companies should focus on genuine ESG improvement rather than optimizing for specific rating methodologies, recognizing that strong actual performance will generally be recognized across providers despite methodological differences.

Further Reading

Academic research on ESG data quality and rating divergence is published in Review of Finance, Journal of Financial Economics, and Journal of Sustainable Finance & Investment. The ISSB provides sustainability disclosure standards at ifrs.org/issb. The SEC's proposed climate disclosure rules are available at sec.gov. The EU's CSRD and ESRS are available at ec.europa.eu. IAASB assurance standards are available at iaasb.org.

References