Data quality and availability are the most commonly cited reasons for investors not incorporating Scope 3 into portfolio target setting. In this article, we seek to quantify the size of the problem and explore the difference in quality between upstream and downstream Scope 3 data. This is a challenging task as reporting on carbon emissions, especially for Scope 3, is still relatively nascent. Reported numbers cannot be compared with a ground truth, so we use proxies, such as the level of agreement among data providers and the dispersion of data to characterize its quality.
The difficulty of measuring Scope 3 emissions
Scope 3 emissions consist of 15 categories emanating from upstream and downstream sources. These are inherently harder for a company to measure as they stem from processes which take place across its value chain and are not directly within the company’s control. Take, for example, end-of-life treatment of products, which is one of the seven downstream emission categories. This requires the selling company to track how its products are disposed of, which can be challenging if they are long-life products.
As a general rule, however, upstream categories such as business travel, purchased goods and services, and employee commuting can be estimated based on known quantities, such as the number of flights taken or the volume of raw materials purchased. Downstream emissions, on the other hand, consist of categories such as the use and processing of sold products which the selling company does not necessarily have visibility over.
In practice, these emissions can be calculated using either primary or secondary data: 1
Primary data: supplier and client-specific information is used to calculate upstream and downstream emissions
Secondary data: average emission factors (tCO2/unit of activity or revenue) are applied to a company’s activities, costs or revenues; these do not reflect the specific supplier and client attributes.
Primary data is, of course, more precise. However, a recent survey by the Science-Based Targets Initiative (SBTi) revealed that supplier-specific factor emissions only represent 6% of the calculations, with secondary factors being used in most of the cases.2
For downstream emissions, specifically regarding the use of products sold, a lot of assumptions are also needed. For example, an assumption needs to be made about the number of times a product will be used over its lifetime, the energy mix of the grid including the amount of power derived from renewable sources, and the location of use.
Disclosure is increasing but still lagging
Using the latest CDP survey published in December, the number of companies reporting Scope 3 remains lower than Scopes 1 and 2, as seen in Figure 1. 3
Nevertheless, the total number of companies reporting at least one category of Scope 3 upstream emissions is actually comparable to Scopes 1 and 2 (approximately 10% lower).4 Scope 3 downstream reporting has also seen a strong upward trend, but has still lower absolute values than scope 3 downstream.
To illustrate the process of estimating Scope 3 for a company, we look at the example of Unilever, a global producer of consumer goods.
Case study: Measuring Scope 3 emissions in the consumer goods sectors
The practicality of Scope 3 is that you can’t measure it, you can only model it. Take the example of Unilever, which has 400 brands, each producing multiple products sold worldwide. The company’s Scope 3 upstream emissions stem in large part from the emissions generated in the growing and sourcing of the ingredients and raw materials. The Scope 3 downstream emissions stem mostly from the end use of the products (e.g. boiling the kettle to make a cup of tea), from their downstream transportation, and finally from the waste generated at the end of the products’ life.
To estimate upstream and downstream emissions, Unilever needs a clear and transparent way of modelling the data. It does this by taking a representative sample of 3,000 products from across 14 countries and 12 segments which accounts for 60-70% of sales volume. For each product, emissions are estimated for each of the nine phases of the life cycle (e.g. primary packaging, ingredients). This is done using internal and external data sources. Examples of these are consumer end-use studies and the energy consumed by the manufacturer. The emissions sample is then scaled to the country and then ultimately to the overall company level. These estimation methodologies are continuously reviewed and may be adjusted over time. For example in 2020, the baseline for Scope 3 was restated to reflect new estimations which could improve their accuracy.
Figure 1 | Reporting for Scope 3 is increasing but still lagging behind Scopes 1 and 2
Source: CDP Survey 2022.
We need to remember that Scope 3 is made up of 15 categories (eight upstream, seven downstream) which are not all equally reported by companies. In fact, the most significant downstream category for many sectors – the use of sold products, which is estimated to represent the largest part of all carbon emissions – is still only reported on by 1,290 companies. In Figure 2, we show the number of companies reporting each of the 15 Scope 3 categories. We can see that nearly all upstream categories have higher level of reporting than downstream categories.
Figure 2 | Breakdown of reporting across Scope 3 categories in 2022
Source: CDP Survey 2022.
Less reporting means more estimation
We showed in the first article the importance of taking Scope 3 into consideration. But if companies are not reporting the data, there is then a significant reliance on estimation by data providers.
Most mainstream data providers employ similar estimation methodologies, using the company’s sub-industry and average emission factors to proxy emissions. Despite this, depending on methodological choices such as the sector classification used, the employment of revenue breakdown, or a bottom-up versus top-down approach, the outcome can show substantial differences in the estimates, as seen in Figure 3.
Figure 3 | Data providers agree most on Scopes 1, 2 and 3 upstream, whilst estimates of Scope 3 downstream diverge
Source: MSCI, S&P Trucost, CDP. Degree of agreement is the average R2 of the regression between each of the data providers’ data series.
It is noteworthy, however, that the level of agreement between data providers for Scope 3 upstream is similar to, if not higher than, that of Scopes 1 and 2. Scope 3 downstream, on the other hand, shows diverging views across providers. This makes the choice of provider more important and also highlights one of the dilemmas we face as investors in incorporating Scope 3 downstream into portfolio construction and investment decision making.
The issue of outliers
One of the main challenges regarding Scope 3 is the limited consistency in emission values between companies with similar characteristics. We would expect companies of similar size and in the same sector to have comparable emission values. In reality, we often see high variance among such companies, with the presence of extreme outliers. Such a difference can be attributed to real differentiation between companies in terms of sustainability, but also to the poor data quality of the reporting.
To illustrate this, in Figure 2, we look at the dispersion coefficient for each emissions scope. This is a robust statistical metric used to analyze the dispersion of values in a distribution, where a higher number means greater dispersion. As emissions vary substantially between different industries, we calculate the dispersion coefficient within industries and the reported figure is then the average over the industries.
Figure 4 | Dispersion amongst companies reporting in the same sector is highest for Scope 3 downstream
Source: CDP 2022 annual survey. Median of the dispersion coefficients (Q3-Q1)/(Q3+Q1) of the reported values (carbon intensity) calculated for each sector. We define the sector as the most granular GICS sectoral classification available with 10 or more observations.
We observe higher dispersion for Scope 3 upstream versus Scopes 1 and 2, although Scope 3 downstream is once again the real outlier. While dispersion does not provide conclusive evidence that the quality of the data is lower, it does add to ourconcerns about the consistency of reporting for Scope 3 downstream. 5
This dispersion can also be seen anecdotally. When we look through Scope 3 downstream data, we often see significant outliers. Indeed, companies can easily have Scope 3 downstream emissions that are 1,000 times higher than any other company in the same sector, whereas this would be rare for Scopes 1 or 2, or Scope 3 upstream. These outliers are not necessarily due to data errors, but rather are caused by the variety of assumptions that are made to estimate Scope 3 downstream, as well as by the divergence in reporting quality between companies. For example, a company which reports all downstream categories will have much higher emissions than one that does not. While data providers attempt to correct for this divergence in reporting, the complexity in computing these emissions makes this exercise challenging.
More work needed on Scope 3 downstream, but upstream is workable
Our analysis shows that there are still significant challenges with Scope 3 downstream that need resolving before we feel comfortable integrating downstream emissions into the portfolio construction of our entire product range. On a positive note, however, we have validated that the quality of Scope 3 upstream is akin to Scope 1 and 2, and therefore we feel comfortable combining it with Scope 1 and 2 emissions to measure and target reductions in the production-phase emissions of a portfolio.
Footnotes
1
Scope3_Calculation_Guidance_0.pdf (ghgprotocol.org)
2 Scope 3: Stepping up science-based action - Science Based Targets
3 A company is counted as reporting Scope 3 upstream or downstream if it reports on just one underlying category.
4 The number of companies reporting Scope 3 upstream alone does not tell the full story, as not all companies report all underlying categories. The most material category, ’purchased goods and services’, is reported by a total of 2,831 companies.
5 In some cases, it makes fundamental sense that there would be significant differences in Scope 3 downstream between companies in the same sector. For example, an internal combustion engine car manufacturer and a maker of electric vehicles are both in the same automotive sector, and would have similar Scope 1 and 2 emissions but very different emissions from the use of their sold products.