Assessing success in Civic Tech: Measures of deprivation and WriteToThem

This blog post is part of a series investigating different demographics and uses of mySociety services. You can read more about this series here

WriteToThem is a service that assists people in writing to their representatives. Given a postcode, it lists the associated elected representatives at every layer of government and provides a form to write an email to them.

This can also be seen as a bundle of services. The main use of this website is to write to MPs, but this is just under half of messages ever sent (48%), with most messages sent to representatives in devolved or local government. Different services have different profiles of use and so need to have their effect judged separately.

In 2015, the British Election Study asked whether people had contacted a “politician, government or local government official” in the prior 12 months and found that 17% had. Based on this, over 11 million adults wrote to a representative or official that year — and WriteToThem’s 187,000 emails accounted for 1.6% of this. These results also showed that 20% of men had made contact compared to 15% of women, meaning that 57% of those doing the contacting were men. Extending this into a logistic regression shows that older respondents and those with higher levels of education were more likely to contact, with no significant difference for income and ethnicity once age and education were controlled for.

Demographic profile of WriteToThem users

Looking at the profile of people writing to MPs using WriteToThem, there is an uneven use by different demographics. Over all time,  60% of messages sent have been from men and  60% of people writing had written before. Using the index of multiple deprivation, more messages are sent by better off areas, with 55% of messages being sent by the less deprived half of the country, and 7% of messages coming from the most deprived decile (you would expect 10% if this were evenly divided).

There is a clear linear pattern of greater employment and income in an area being associated with a greater amount of messages sent.  Most of these gradients are slight, but in aggregate the effect is that WriteToThem reflects existing divisions in participation (although there are no good sources for the demographics of people who write to MPs specifically) .

But is this actually a problem? Should a service be judged for the proportion of existing represented groups making use of it, or what it does for the under-represented groups who do use it? WriteToThem has delivered 73,000 messages to MPs from people in the most deprived IMD decile alone, if this has led to dialogues that resolved issues that would not otherwise have happened, this is a positive regardless of whether the same is also true for more people in the least deprived areas. If WriteToThem lowers the cost of contact by making it easier, then it is unsurprising that many of the people making use of it would have made contact anyway — but also included in that are people who were previously unable to engage in the process.

When we look at the result of the survey asking whether a user of WriteToThem was writing for the first time, we can see that people from the bottom three IMD deciles were statistically more likely to be writing for the first time (this is also true when just looking at people writing to MPs, and when just looking at 2018). While generally the number of people using the site for the first time has decreased over time, this decline is demographically uneven and mostly occurs in less deprived areas.

For the complete time-span of the service, 47% percent of survey respondents in IMD 1 (most deprived) were writing for the first time compared to 38% of IMD 10 (least deprived). Looking at just 2018, this was 48% compared to 35%. While the service as a whole is used more by people in less deprived areas, of those using it in less deprived areas it is successfully facilitating a higher proportion of first time contacts.

The local picture

To return to the idea of bundles, WriteToThem is also quietly solving a much harder problem than contacting MPs. While people generally recognise their MP when prompted with a name, local councillors remain far more anonymous. From 2007 to 2018 WriteToThem has helped constituents send 450,000 emails to their local councillors (42,000 in 2018). This service has an effectively even gender ratio (with a female majority in 2018), with more reports coming from more deprived areas (54% by more deprived half).

If we imagine one of these bundled services being a site named “WriteToYourCouncillor”, it is in many respects a model service, with a user base displaying an even gender ratio, and more likely to be used in deprived areas. That in reality it is one function of a more well-used service in terms of numbers somewhat obscures this.

But while it is good to recognise where services are successfully reaching people we want to reach, it is also important to think about volume and overall impact.  One issue with a service used more by men or in better off areas might be if it shapes how resources are deployed or provides a false shape of the views of constituents (and emails received are certainly used by MPs to build a picture). Even a service that adequately represents under-represented groups may be ineffective if it exists in a wider ecosystem that does not.

At the moment, the systematic effect of any bias in WriteToThem outputs is marginal as WriteToThem accounts for a small fraction of parliamentary mail.  While the amount of physical mail entering the Houses of Parliament each year has decreased steadily, in 2018 it was still 24 times larger than the number of emails sent to MPs via WriteToThem. The average MP received 94 emails via WriteToThem in 2018; most MPs would receive more than this through other means in a week.

Returning to the British Election Study finding that 57% of contacting in 2015 was done by men, the equivalent figure for WriteToThem as a whole in 2018 was 55%. Being generous and bearing in mind the previous finding that the method used to assign gender from name undercounts women, this could be seen as a marginal improvement on the real world. However, it would be a marginal improvement in a pool that only represents 1.6% of the total amount of number of messages.

Defining success

Based on the above, we can think about three different kinds of ‘success’  of a civic tech service in serving under-represented groups:

Relative – The service improves under-representation relative to the current standard. e.g. a service where 60% of usage was by men is an improvement over an offline status quo of 70%.

Absolute – The service adequately (or over-) services under-represented communities to what would be expected based on their numbers in the general population.

Systematic – The service successfully services under-represented communities and is successful enough that this redresses issues of representation in competitor services/methods.

Working with these, we could say WriteToThem is a success on a relative level, servicing people in more deprived areas more than they would have been otherwise (larger proportion of first time writers), but not to the proportion of the population these groups represent.

The “WriteToYourCouncillor” part of the bundle is  a success on an absolute level, providing a relatively even amount of representation, with a slight weight towards groups who typically make contact less often.

But neither really makes a dent systematically. They may be redressing inequalities of access for individual users (which is good), but cannot significantly adjust inequalities in volume of messages and the corresponding perceptions of problems.

Making a dent in this problem is outside the scope of WriteToThem — and probably should be. While you can imagine a future where WriteToThem continues to lower the barrier to contacting representatives,  this is likely to create new users from currently-represented groups for each under-represented person successfully reached. Targeted interventions and partnerships with other organisations can avert this problem in terms of helping individuals make contact about their issues but turning the problem around, this is a platform that is unlikely to provide a balanced view of opinions and priorities of constituents.

If it is a problem that representatives have systematically skewed visions of the problems and views of their constituents, is an email platform that requires citizens rather than representatives to do work the best way to address that? A civic tech solution to this problem might look more like Consul (or similar general participation platform) than WriteToThem – but even explicitly designed online platforms still risk being skewed towards the online and present members of the community. Exploring better forms of local participation is something currently being explored through our Public Square project.