Visualising house prices per square metre by region

My second highlight from the GLA’s 2018 Housing in London report is the below chart showing the distribution of local authority-level average house prices per square metre, by region in 2004 and 2016. The data is from ONS.

Distribution of local authority average house prices per square metre by region, 2004 and 2016

This type of chart is great for comparing distributions, and is made very straightforward using the ggridges package for R. I’ve set out the full code for creating this plot on RPubs here.


Comparing housing and population growth in cities around the world

Yesterday the Greater London Authority published the 2018 edition of our annual Housing in London report, which acts as the main evidence base for the Mayor’s housing policies. Over the next few blog posts I’m going to go into more detail on some selected visualisations from the report that I hope might be of particular interest.

Some of the new visualisations in the report were done in R, but the bulk of the charts were made in Excel, which I continue to be a big fan of as an accessible and flexible dataviz tool. I think the chart that compares annual rates of population and housing stock growth in various international cities is a good example:

Annualised growth of population and housing stock in most recent five years for selected international cities - chart from GLA, Housing in London 2018

The annual rate of population growth for each city over the most recent five years available (generally 2011-16, see below for more details) is on the X axis, while the annual rate of growth in its housing stock is on the Y axis. The size of the bubble represents the population of each city, while the colour represents its region of the world. The dotted diagonal line represents equal rates of population and housing growth: cities above it have seen faster growth in the number of homes than in the number of people over the last five years, while in cities below it population has outgrown the housing stock. Unsurprisingly, none of these cities have seen a fall in their housing stock over this period (Dublin is closest with an annual growth rate of just 0.12%).

The data for this chart was assembled over quite a long period of time by trawling national or municipal statistics sites. Cities were included if they seemed like good comparators for London and if I could find suitable data on their population and housing stock. Most of them are fairly large cities – the two smallest are Vienna and Dublin, included because Vienna is often seen as quite advanced in terms of housing policy and because Dublin is a close neighbour of London’s, an extreme case of unresponsive housing supply, and my home town.

I would have liked to include some Chinese, Latin American and African cities but couldn’t find the right data. For cities that have a significant amount of informal housing I imagine official housing stock measures become less meaningful, anyway.

More broadly, if anyone can point me towards good data sources for any other cities I’d be happy to try and include them. At some point I also intend to put together and share a single dataset of population and housing stock observations for as many cities and as many years as possible.

Edit: You could legitimately criticise the kind of comparison made in this chart on the grounds that population growth is not exogenous to housing growth – fewer people will come to the city (or more will leave) if there isn’t enough housing to go around. This might be a particular issue in cities with lots of rent control, which limit the flexibility of the market to respond through over-occupation. I sympathise with that line of argument, but in the absence of any more widely adopted measure of demand or of comparable data on rents, I think population growth is probably the best thing to use for now when making international comparisons.

City boundaries

As I said above the data gathering was largely opportunistic, which in practice means that many of the figures, particularly for housing stock, relate to administrative city boundaries rather than any more theoretically sound or comparable boundary such as ‘functional urban area‘. The upshot is that in most cases the metropolitan area spreads beyond the boundaries of the city as defined here, with Barcelona the main exception as it is measured as the province, which has a population 5.5 million, rather than the functional urban area which the OECD says has a population of 3.8 million in 2014. Tokyo is at the other extreme, with these figures relating to its prefectural population of 13 million rather than the Greater Tokyo population of 36 million. For some of these cities it should in theory be possible to produce housing stock figures to the functional urban area boundaries using Census data for small geographies, which would enable some comparison of distributions within those boundaries.

City data sources

The sources used are below, including the definition of the area used. If anyone knows of any issues with these sources or any better options please let me know, either with a comment here or with an email to

How Tokyo built its way to abundant housing

Tokyo has rightly been getting some plaudits for housebuilding of late, and this post brings together some stats that illustrate just how impressive its record is.

First, some definitions and context. The statistics in this post are all for the Tokyo Metropolis area, also known as Tokyo Prefecture, with a population of around 13.5 million as of 2015. This is just part of the Greater Tokyo urban area, which holds around 37.8 million people, but for the purpose of this post when I say ‘Tokyo’ I mean the Metropolis/Prefecture.

The data in this post come from three online collections: the Tokyo Statistical Yearbook, the Japan Housing and Land Survey tables and Historical Statistics of Japan. As data sources these are all the more useful for being extremely old-fashioned: data is presented in tables using the same variable names and layouts going back decades, arranged on sites that are blessedly free of intrusive bells and whistles. Long may that continue, because when combined with an awesome commitment to publishing statistics in English, the end result is an amazingly accessible trove of historic data, probably more than is available for any other city I’ve looked at, including London.

It’s well known that Japan’s population is falling, but less so that Tokyo continues to grow, adding around 940,000 people (an extra 7.5%) between 2005 and 2015. That means Tokyo, like many big cities around the world, has the challenge of how to ensure there’s enough housing to go around. But unlike most big cities around the world, Tokyo is actually meeting that challenge.

Annual housebuilding statistics in Tokyo are expressed in terms of dwelling starts rather than completions. The chart below (compiled from various years of the Tokyo Statistical Yearbook) shows the trend in new dwelling starts in Tokyo between 1995 and 2015. Starts hit a peak of 192,000 in 2003 and a trough of 108,000 in 2009 but averaged 155,000 new homes over the two decades.

dwelling starts in Tokyo

But that’s new supply in gross terms and everyone knows the Japanese demolish housing at a much higher rate than most countries, so what’s the net growth like? The chart below (from table 5 here) shows how the number of homes in Tokyo changed between 1963 and 2013 (based on the five-yearly Housing and Land Survey, the latest one of which was carried out in 2013). In just a fifty year period Tokyo’s housing stock nearly tripled in size, from 2.51 million homes in 1963 to 7.36 million in 2013.

Tokyo dwelling stock 1963-2013

Tokyo does demolish a lot of housing – between 2002 and 2011 there were 1.58 million starts but between 2003 and 2013 the stock grew by 1.17 million, so if we assume that it takes an average of two years from start to completion then it looks like 0.41 million homes were demolished in a decade, or about 7% of the 2003 stock. Put another way, roughly one home is demolished for every four new ones built. But the scale of construction still means that Tokyo’s housing stock is growing very fast – roughly 2% a year, about twice as fast as that of Paris, London or New York as the chart below (from the GLA’s Housing in London 2017 report) shows.


In 1963 there were 2.69 million households in Tokyo, and by 2013 this had grown to 6.51 million. So while the number of households grew quickly, the number of homes grew faster, and Tokyo went from having a crude ‘housing deficit’ in 1963 to a ‘surplus’ in 1973, and an even bigger surplus every year after that. By 2013 there were 849,000 more homes than households.

Tokyo dwelling surplus, 1963-2013
What strikes me about this chart is that at every point after 1968 Tokyo had (a) more than enough housing to go around (by this measure, anyway) and (b) more than it had ever had before. It is often said that by those criteria the UK should stop focusing on new trying to increase new supply and should instead focus on a better distribution of its existing housing. But Tokyo illustrates another strategy: keep building more and more housing, far past the point of mere sufficiency and into the realm of abundance.

The next chart shows the change in the type of dwellings in Tokyo between 1978 (the furthest back the data goes) and 2013. The fastest growth has been in apartments, particularly the tallest ones (six storeys or more). In 1978 there were 823,000 homes in Tokyo in apartment buildings of 3 or more storeys. 25 years later there were 3.6 million. Over the same period the number of houses grew slightly while the number of low-rise apartments fell. All of this is consistent with a pattern of housing growth achieved primarily through densification rather than sprawl.

Tokyo housing type change 1978-2013

That’s backed up by data on land use, which indicates that the acreage devoted to housing in Tokyo grew by around 1.5% between 2006 and 2011, whereas the number of homes grew by 9.2%. On average there are around 110 dwellings per hectare of residential land in Tokyo, compared to roughly 60 in London as of 2005 (the last time similar land use statistics were collected).

Really fast densification in an already built-up area like Tokyo can only be achieved through demolition and redevelopment, and we’ve already seen that around one existing home is demolished for every four new ones. The next chart shows how that trend has affected the age of profile of Tokyo’s housing stock over time. For obvious reasons there aren’t many dwellings in Tokyo dating from earlier than 1950, but even the stock of homes built in the 1950s and 1960s is dwindling fast: in 1998 there were 856,000 homes originally built between 1951 and 1970, but only 15 years later in 2013 this had fallen to 451,000. Replacing swathes of old housing with taller, denser new housing is what enables Tokyo to grow its housing stock so fast.

Tokyo dwelling age 1998-2013

What does all this new construction mean for the amount of housing space available? Tokyo has a reputation for tiny apartments, and at 64m2 its average dwelling size is indeed smaller than what most Westerners would be used to, although it has risen over time.

Tokyo average dwelling size 1963-2013

The average size of new homes in Tokyo is very similar, at 65m2 (down from around 80m2 in the late 1990s).

Tokyo size of new homes 1995-2015

But what these figures on average dwelling size don’t tell you is that the average Tokyo resident has far more space today than they did fifty years ago. The reason is that the average number of people in a Tokyo household has plummeted over this period, from 3.6 in 1963 to 2.0 in 2013. Of course, this could only happen because there was so much housing available for people to move into. When there isn’t enough housing then you get the opposite effect – of multiple unrelated people living together, and rising household sizes. London’s a good example of this.

Tokyo average household size 1963-2013

When you combine the average floorspace per home and the average number of people per household you get the below trend in average floorspace per person (this shows the space in occupied homes only – if you took vacant homes into account it would be much higher). The amount of space per person saw a remarkable increase from 15m2 in 1963 to 32m2 in 2013. Londoners have a similar amount of space per person on average today (assuming similar methodologies for measuring floorspace – there’s not enough detail in the Tokyo stats to tell), but in stark contrast to Tokyo there has been little or no increase in London in around 20 years (according to this article, which in classic UK-housing-discourse style seems to suggest that it’s evidence against a housing shortage).

Tokyo average floorspace per person 1963-2013

So how come Tokyo is so good at building housing? That’s a long story in itself, but this Robin Harding article in the FT is a good place to start, and if you want to dig into the academic literature try here, here and here. In short, Japan has a relatively simple and unambiguous zoning code, one which the national government has repeatedly adjusted in order to allow for more housing growth in Tokyo. That has been done in the face of opposition at neighbourhood and even city level, opposition that in countries which have devolved land use decisions to a local level would be enough to stop densification or at least divert it to poorer areas. What you might call the ‘supersidiarity’ of Japan’s approach to housing policy is therefore quite progressive, as well as being extremely effective in getting housing built.

Yes, there is a housing shortage

Jonathan Ely of the FT tweeted today:

Reading the article, “there is no shortage of homes” is qualified with “nationally at least”, which is a rather important caveat. Because housing is generally both immobile and very long-lasting, it is perfectly possible to have both a growing national surplus of housing and growing local shortages in places where people most want to live.

To illustrate this, imagine we suddenly discovered magic money trees in the centres of some of our major cities. Lots of people would suddenly try to move to be near the magic money trees, causing overcrowding in city centers and de-population elsewhere. If there was no change in the housing stock the national surplus/deficit would be completely unchanged, but supply would still be failing to meet demand.

The point is: all housing shortages are local or regional, and whether there is a ‘national surplus’ or not is mostly irrelevant. It’s like saying there is no problem of air quality in city centers because when averaged across the whole country, including the vast swathes of rural areas, the air is pretty clean.

Ely echoes Ian Mulheirn in saying that we have a housing surplus because the number of dwellings has grown faster than the number of households, which has been much lower than official projections indicated:

In recent years, net new supply has been below the 210,000 dwellings that the government estimates will be needed each year from 2014 to 2039 in England. But long-range forecasts on household formation require big assumptions about longevity, fertility, household size and migration, and are subject to large margins of error. In 2008, the government estimated that 280,000 homes would be needed in the UK each year until 2016. In 2012, after the global financial crisis, that had dropped to 231,000 a year. Not only are the figures a moving target — the statisticians now have to factor in the impact of Brexit on migration, for instance — but they are also some way off the reality. The 2012 forecasts predicted 27.7m households by 2016. Figures from the Office for National Statistics say there are now 27.2m households — around 500,000 fewer than predicted.

This apparently widening gap between the number of homes and the number of households applies to London and the South East as well as nationally – so isn’t this evidence of a real surplus?

No, because household formation is constrained by housing affordability: if people (particularly young people) can’t afford to buy or rent a home of their own they can’t form their own household and have to remain living within another one, which usually means living with their parents or sharing a rented home with a bunch of other people who are in the same position.

Ely considers this possibility, but rejects it:

One possible explanation for this contradiction is that high prices have suppressed household growth. Households cannot form because 30-year-olds are still living in their parents’ spare bedrooms. The data do not really bear out the anecdotal evidence of the “boomerang generation”. Around a quarter of people aged 20-34 still live in the parental home. In 1996, when house prices were much lower relative to earnings, the proportion was still a fifth.

But sharp-eyed readers will have spotted that a quarter is higher than a fifth, and indeed when you look at the data you find that the number of 20-34 year olds in the UK still living with their parents has risen from 2.7 million in 1996 to 3.4 million in 2017. In fact it dipped for several years after 1996 so it would be just as valid to say there’s been an increase of a million or 40% since 2002. I feel like only in the UK would people see this data as evidence against a housing shortage. The fact that house prices have risen faster doesn’t mean that there isn’t a housing shortage, just that other factors like income growth and credit availability also affect house prices, which I don’t think is a controversial point.

We also have more direct evidence of constrained household formation. The source of the data on both the number of households and the number of young people living with their parents quoted above is the Labour Force Survey, which also gives us the number of ‘family units’ in the population. A family unit is defined as

either a single person, or a married/cohabiting couple, or a married/cohabiting couple and their never-married children who have no children of their own living with them, or a lone parent with such children

Given this definition, if there is enough housing to go around then it seems reasonable to expect that each family unit should have a home of its own. But not only is that not the case in England, but the number of family units without a home of their own (and who therefore have to share accommodation with one or more other family units) has been going up. These are called ‘concealed households’, and in 1996 there were 1.64 million of them in England. But by 2016 this had risen 50% to 2.45 million, far outpacing the rate of household growth.

This increase reverses the trend of most of the previous century, which had seen huge falls in the number of concealed households as housing supply outpaced population growth (see Holmans, ‘Historical Statistics of Housing in Britain’, which uses a different measure). And if there was truly ‘no housing shortage’ it would be falling rather than increasing.

The problem is also much worse in London, where the number of concealed households has increased 81% from 400,000 in 1996 to 717,000 in 2016. Obviously, that’s entirely consistent with London having a much more severe problem of housing affordability, and with separate data from the Resolution Foundation that shows a sharp increase in the number of families who are sharing privately rented accommodation with others, particularly in Inner London.

The supposed ‘growing housing surplus’ turns into a growing deficit when you compare the number of homes to the number of family units, rather than the number of households. The chart below shows that for England as a whole the dwellings/households ‘surplus’ grew but so too did the dwellings/families ‘deficit’.

When we make the same comparison for London (using annual rather than five-yearly data from the LFS, because I had more time to collect it) the picture is even starker:

By this admittedly crude measure, the deficit of homes compared to family units in London rose from 270,000 in 1996 to 490,000 in 2016.

So yes, there is a housing shortage and yes, it’s worse in London.

Historical housing and land values in the UK

Before getting on to explanations of rising land costs, I wanted to follow up the last post with some more detail, this time focusing on the UK only.

The research I discussed last week uses a standardised methodology to disaggregate house prices into structural and land prices in 14 different countries, but in the UK (and at least some of the other countries) we also have some official statistics that shed some light on this (but which don’t give exactly the same results due to differences in data sources and methodology).

The Office for National Statistics publishes data on the total estimated value of the UK’s housing (£5.5 trillion in 2015) and of the value of the dwelling structures only (£1.8 trillion). Implicitly, the remaining £3.7 trillion is accounted for by the land under the homes (and gardens, garages and anything else within residential plot boundaries).

By cobbling together various ONS datasets (and with the kind help of Brian Green) I’ve managed to trace these figures back to 1957. Here’s the estimated average value of a UK home, disaggregated into structure and land values and adjusted for inflation.

Collated from: ONS data on total value of homes (series ALLA and CGLK), net capital stock of dwellings excluding land (CIWZ and MJF8), GDP at current prices (YBHA) and composite price index (CDKO); and DCLG UK dwelling stock trend (table 101)

According to these figures the average value of a UK home was £192,040 in 2015, of which the structure accounted for £62,610, leaving a residual that we assume to be ‘land’ of £129,430. In 1957 these component figures were £15,890 and £8,350 respectively. By the way, if the average value of a UK home of just under £200,000 seems low to you, remember that this figure covers the entire housing stock, and the homes that are actually sold in any particular period are generally of significantly above-average quality and therefore value.

Obviously incomes have also risen a lot over this period, and one way to adjust for that is by calculating the total value (across the whole dwelling stock) of housing structures and housing land as a percentage of national GDP, as below.


This shows that the structural value of the UK housing stock is almost exactly equal to one year of national income (95%, up from 52% in 1957), while the land value is worth twice that (197% of GDP, up from 27% in 1957). Among other things, it also shows that there was very strong growth in land values in the early part of this period (roughly doubling as a share of national income between 1957 and 1968), while there was a notable increase in structural values under the Labour government (rising from 63% of national income in 1996 to 99% in 2009).

Looking only at the period since 1957 gives the impression of inexorable growth in residential land values, but as Knoll et al’s Figure 7 in the previous post shows the 1950s were a low point for land values across developed countries as a whole, being roughly three-quarters of the level seen at the end of the 19th century.


That picture is backed up by a separate series of estimates of urban land rents as a share of national income in England and Wales, compiled by Hans Singer (before his days as an international development economist), reported by Colin Clark in his book ‘The conditions of economic progress’ and charted below.


According to Singer’s estimates, the income from urban land rose from 1% of total national income in the 1840s to a peak of 4% at the end of the 19th century, which makes sense when you consider the extraordinary pace of urbanisation in England and Wales at the time. But while urbanisation carried on over the next few decades, the share of urban land rents in national income actually fell. In the next post in this series, I’m going to look at why that happened.

Accounting for the house price hockey stick

Every country and every city experiencing rising house prices has its own explanation for what’s going on, often rooted in what seems to be some local or national factor. In England we like to blame the Green Belt, the decline in council house building, poor foreigners (for moving here) and rich foreigners (for investing here).

These locally specific stories often have at least some truth to them, but what they miss is that rapid increases in house prices are a surprisingly pervasive phenomenon. The best analysis of cross-country house price trends is ‘No Price Like Home‘ (free version) by Katherina Knoll, Moritz Schularick and Thomas Steger, the result of a gargantuan research effort to uncover price, construction cost and land value data for 14 affluent countries going as far back as 1870. Their Figure 1 below shows the long-term trend in average prices in each of these countries. In almost every one, price growth took off in the latter half of the 20th century and was particularly rapid in the run-up to the 2007-08 crash. capturecapture

The authors average these trends to create a somewhat cheekily titled ‘Global House Price Index’, shown below (after adjusting for inflation).


The stability (or even slight decline) of prices in the two decades prior to WWI and the sharp increase between the mid 1990s and 2007-08 are particularly striking when you plot them against GDP, as they do in the next chart. Rising incomes were associated with stagnant real prices in the pre-WWI period, but very rapid price growth in the run-up to the recent crash.


The growth in prices in the latter part of the last century doesn’t seem to be the result of improvements in housing quality, because quality-adjusted price indices show a similar trend.


To show this more clearly, Knoll et al disaggregate the change in house prices into (1) changes in the value of the structure and (2) changes in the value of the land. They do so by comparing changes in construction costs to changes in prices, with land values estimated as the residual. In the two charts below, the orange line always represents the real house price trend – the one on the left compares it to the trend in construction costs, and the one on the right to the estimated trend in land prices.


What the chart on the left tells you is that on average, the cost of constructing housing roughly doubled in real terms between the end of World War I and 2010. That’s pretty significant, but it’s dwarfed by the increase in residential land prices, which roughly quadrupled in real terms over the same period.

So the answer to “why did housing get so expensive?” seems to be “because land did”. Having said that, it’s really important to bear in mind how these figures are calculated. In this kind of analysis, any change in house prices not directly related to a change in construction costs must by definition be attributed to a change in the land price. Even macroeconomic factors such as changes in interest rate, which you might think have a similar impact everywhere, are counted as changes in ‘land’ rather than ‘structure’ prices. The value of housing, just like any other asset, will tend to go up when interest rates go down, but by convention we tend to ascribe all that change to the land rather than the structure. That probably isn’t exactly right, but probably is mostly right.

With that important caveat in mind, the question is: what made land get so expensive? I’ll try to answer it in a future post.


The myth of the shrinking British home

Writing in today’s Guardian about the government’s hint that it might weaken England’s already not particularly strong housing space standards, Rupert Jones says

In 2014, researchers from Cambridge University found that, at an average of 76 sq m, the UK’s newly built homes were the smallest by floor area in Europe. At the other end of the spectrum was Denmark at 137 sq m (having all that space probably helps explain why it is allegedly the world’s happiest country).

The 76 m2 new build home is probably Britain’s favourite housing statistic, appearing regularly for years now in popular media and academic research. I’ve become very familiar with it over time, but I know that not everyone is aware of its history. So I thought I might set it out for those keen to learn more about this distinguished datum.

The Guardian helpfully provides the following table comparing new build home sizes in the UK to the rest of Europe.

Table: average size of a newly built home across Europe.

The source is this 2014 paper by Malcolm Morgan and Heather Cruickshank, who agree that ‘The UK has the smallest homes by floor area in Europe’ but who don’t come up with the figures themselves, instead citing a 2010 paper on ‘Internal housing space standards in Italy and England’ by Nick Gallent and others. Here’s Figure 5 from that paper:


Confusingly, the notes to the table say these are dwelling sizes for the EU in 2005 even though they are taken from ‘Housing Statistics in the EU, 2002’. So it seems that as of 2017 we are still quoting statistics from 2002 to describe the size of ‘new’ homes in England (or the UK, or Britain – the terms seem to be used interchangeably).

It’s actually worse than that, though. Go to the original EU statistics (you can find a Word version via Google) and in table 2.1 this is what you see:


The ‘UK’ (actually England) figure of 76 m2 dates from 1996. So the figures being used to describe new build homes in 2017 are around 20 years out of date.

But wait! Where would figures on the average size of new build homes have come from in 1996? As far as I know the only reliable source was the 1996 English House Condition Survey, a survey of around 13,000 dwellings across the country (the data is available to users of the UK Data Service here).

A sample of that size can’t give you reliable figures on homes built in the last couple of years, so I suspected that the 76 m2 ‘new homes’ figure was based on a longer period. Looking at the 1996 EHCS data, it divides homes up into several age categories, with the most recent one being anything built since 1980. And indeed, when you calculate the mean floor area of homes built in England since 1980 the result you get is … 76 m2.

So there you are. In 2017, our most commonly used statistic for the size of new homes in Britain is actually based on homes built between 20 and 35 years ago.

What’s doubly frustrating about this is that we have perfectly good, much more recent statistics that we should be using instead. The English Housing Survey is the successor to the English House Condition Survey, and the 2014/15 housing stock report says that the average size of homes built since 2005 is 87 m2, compared to 94 m2 across the stock as a whole. Note, both of those figures have increased since 1996: English homes are getting bigger, not smaller.

In fact, the EHS report has a whole chapter on space standards. This concludes (paragraph 3.19) that “there is no clear evidence to conclude that each cohort of English homes is, on average, smaller than the cohort before it”, and includes this chart:


What this indicates is that homes built in the 1980s are on average the smallest, with homes built since 1990 significantly larger.

For a much fresher estimate we can refer to statistics from Energy Performance Certificates, the latest release showing that new homes built in Q4 2016 had an average floor area of 92 m2. So please, use 87 m2 (homes built since 2005) or 92 m2 (homes built in late 2016) but just stop using 76 m2.

By the way, if for some reason you’re not convinced by any of this, go read Neal Hudson.

This post is already too long so I’m not going to get into any detail why people keep using this out of date factoid (hint: it makes new homes sound nasty), or how average new build sizes in England really compare to the rest of Europe (below average but not the smallest), what affects average home size (housing type, land prices) or whether we should demand that new homes are more spacious (this is relevant). If we could just stop perpetuating the myth of the shrinking British home that would be more than enough for one day.