Making sense of financial market indices

Please note: This article is more technical in nature than our typical articles, and may require some background knowledge and experience in investing to understand the themes that we explore below.

Key points

• From the S&P 500 to the consumer price index, indices are sprinkled all over the financial landscape. But how can you get the best out of them, and so potentially make better investment decisions?
• When analysing indices with a view to deciding where to invest, beware not to use too much data and to use additional information sets as a check
• As the parts of an index change on a regular basis, using past index data may be less useful in drawing conclusions for the present. The impact of the internet on business since the 1990s helps to illustrate that
• Tech companies now account for over a quarter of the S&P 500 by value, altering the reaction of the index to economic storms compared to the days when energy stocks dominated. Indeed, as service providers might be less susceptible to interest rate rises than manufacturers, this may be one reason why the S&P 500 has followed US rates higher in recent months

Imagine viewing a financial market report without reference to an index? Whether it is consumer price indices (CPI) measuring inflation, stock market indices such as the S&P 500, or the Bloomberg Commodity Index (BCOM) – indices abound in financial literature. 

Index time series provide a convenient way to look at relationships between developments in the economy and in markets. This is because index providers generally try to represent a market in a consistent manner.

Some providers have been doing this for a long time: for most developed financial markets, 20 years of data is available, and for some early-starters, at least 40 years of data on indices are available. This wealth of information provides fertile ground for both human- and computer-led analysis of perceived relationships between various markets. 

Complacency: the cost of convenience

The convenience for analysts and market participants alike is that they do not have to worry about all the maths involved with weighting, as well as inclusions and exclusions of companies. If markets close on one day and some stocks stop trading while new ones replace them in the index, indices will not have sudden sharp movements due to reshuffling of their members and they should seamlessly continue as if nothing has happened. 

As with most conveniences in life, this comes at a price – especially when using indices for longer-term financial analysis. Do indices really consistently reflect markets over time? This article aims to unveil some common challenges faced when working with indices, so that investors can better use them when making portfolio investments. 

Equity indices and the evolution of economies

Much like shifts in economies (see the rise in services as a driver of US growth, measured using gross domestic product (GDP), in the following chart) the same can be seen with index constituents. In fact, sectoral shifts in indices can be much more pronounced, due to their weighting by market capitalisation.  

While an index still represents the pool of investable securities even at extreme weightings, such as information-technology businesses forming more than a quarter of the S&P 500 by value – it can disconnect from the labour market and gross domestic product (GDP).

Such extreme changes, and extreme weightings, can be problematic for index users if companies in sectors that are gobbling up greater weightings in an index (as seen in the service sector in recent decades) react differently to economic shocks. In this case, past index data may be less useful in drawing conclusions for the present, given that the latest constituent companies may react differently to similar economic shocks than the earlier set of constituents.

Sectoral weights matter

A recent study¹ explored the effects of interest rate hikes on small and large firms in services and manufacturing. Indeed, the study found rate hikes had a much worse effect on employment in large manufacturers than in similar-sized services providers. The finding was attributed to services prices being stickier than those in manufacturing.  

Evolution of economy and market indices over time

Share of services in US gross domestic product and 5-year snapshots of sectoral weights in the S&P 500 index (classification estimated) 

Price indices and the inflation basket

Staying with the effect of monetary policy, the most-discussed index these days is probably CPI. The index is at the heart of leading central banks’ policy, as they target a set rate of inflation and price stability. The tool used by the policymakers to achieve this is interest rates, as has been seen in spades in the last two years, as they strive to stimulate or cool the flow of credit and eventually consumer demand in an economy in order to hit their target. 

Much like the economy, the shopping products included in the basket chosen to reflect price changes have chopped and changed over the years. In the US, there has been a steady increase in the share of shelter costs in the basket since the end of the second oil shock in the early 1980s, while food comprises a lower weighting (see chart). These two categories can pose a headache for central banks, because people will want a roof over their heads and to eat, no matter what. 

Consumer spending habits

Changes in the weight of the shelter and food and beverages components in US consumer price basket (urban)

When the going gets tough, there is always supercore

Food inflation has usually been accelerated by interest rate hikes, as retailers pass them on in higher prices and consumers substitute other goods for food, keeping prices for food high. This is a substantial problem in developing countries, where the share of food is much more than the 13.5% weight seem in the US CPI². 

The share of shelter in the basket, as well as the amount of debt used to finance this shelter, can magnify the effect of monetary policy. As rates surge, the cost of servicing debt climbs, leaving less money available to buy other goods and cooling consumer spending³.

Most central banks target the headline CPI rate as it is judged to reflect the cost of living best. But during bursts in inflation, central banks often turn to core inflation (which excludes the more volatile food and energy components from the basket). 

In the latest inflation shock, central banks have focused on the “supercore” rate (which omits housing costs on top of food and energy because its lagged response to rate moves) with the idea to “look through” the noise to focus on the medium-term level of inflation⁴. Like analysts facing a shifting economy, a changing CPI basket means that central bankers risk charges of complacency when “looking through” too many items in deciding how quickly to change rates in the battle against inflation.

Bond indices and a question of duration

The final example for investors to be aware of relates to bond market indices. The returns on the US Bloomberg Bond Indices are commonly used when market participants judge how well bond funds have performed. 

As interest rate levels trended lower over the last four decades or so, coupons typically fell. This meant that more was repaid at the end of a bond’s life, rather than being made earlier via coupon payments. 

Duration, commonly used by investors, describes the point in time at which a bond holder can expect to have recouped their original investment. As a result of the decrease in coupon size, duration has increased from below four years to over six years for the Bloomberg US Government Bond Index (see chart).  

This has implications for the interest-rate sensitivity as a later repayment means that there is more exposure to the new interest-rate regime. When predicting the returns of this index, based on a change in rates, with data from 1980 onward, duration (or sensitivity to changes in rates) would likely be understated. 

US bond indices have become more rate sensitive

Modified duration analysis of the Bloomberg US Treasuries and US corporates indices since 1980 

Getting into good habits

Undoubtedly, indices make financial market analysis a cheaper task than it would otherwise be. But when using them, the key is to understand what securities underpin them and be aware of changes in their nature before drawing conclusions. The following principles should be helpful in trying to limit the effects of complacency:

• Use as much data as is needed, not as much as is available
• Perform ‘robustness’ checks by running the same model on different parts of the sample 
• Divide the sample into different groupings (like high interest rates versus low interest rates) and study the performance of each of them
• If possible, leave some data untouched as a test sample, to verify your model
• If possible, confirm findings using a different dataset

Despite the surging capabilities of Artificial Intelligence (AI) and its application for financial market analysis, humans still have the edge over machines in this field. But again, this is no reason for complacency: instead, searching to see if some of the techniques used in developing such models can also be applied to improve our financial analysis makes sense. Luckily, new AI tools can help investors to analyse data more easily, and to run robustness checks.