Freedom of information is a significant legal instrument in the UK - but currently very little is currently understood about how councils cope with this sometimes significant workload. While central government Freedom of Information (FOI) statistics are well covered by the Cabinet Office series, there is no equivalent picture for local government. The goal of this research was to construct a more detailed picture of FOI at the local level, to understand how many requests were being sent, how much internal visibility exists over the status of requests, and calculate the contribution of requests made through WhatDoTheyKnow to the total number of requests made overall.
This research is based on two sets of FOI Requests made to local authorities and an anonymous survey of FOI officers. The report aims to provide a clear illustration of the current scale and operational approach to FOI in local government. The key conclusions are:
- Approximately 467,000 FOI requests were sent to local government in 2017. This almost doubles a previous estimate of the number of FOI Requests sent to local government.
- Councils have relatively universal records on the number of requests received, and time taken to reply - but have fewer records on the volume of information disclosed, or on the status of appeals.
- Approximately 35%-50% of internal reviews in local government result in a change to the original outcome.
- Staff responsible for the administration of FOI in local government tend to hold FOI as one responsibility among several.
- FOI teams tend to be embedded in larger teams with few staff solely working on FOI. As such, FOI administration rarely appears as a specific budget item. Staffing levels devoted to FOI appear to increase in correlation with an increase in the volume of FOI Requests received.
- Most councils (66%) use some form of case management system, however there is no standard or universally adopted software for FOI case management, and the quality and operability of these systems varies significantly between local authorities.
- Most councils (64%) do not publish a disclosure log. Councils that receive higher numbers of FOI Requests requests are more likely to publish a log, but accounting for other drivers of reports, there is no positive or negative effect of publishing a log.
- Replicating this exercise every year would be prohibitively difficult, however a centralised repository of the statistics disclosures required by the new code of practice would make tracking change over time (over a sector or individual authority) more straightforward and unlock more value from those disclosures.
Foreword by Dr. Ben Worthy (Birkbeck College):
Freedom of information is local. Although it is often national scandals that grab the headlines, most FOI use is at the local level. Our research at the Constitution Unit back in 2010 in the UK found that almost 80% of requests go to local government. This makes sense, given that most political participation and activity is at the level of town halls rather than Westminster. All this local use means we have to re-think and remember what FOI is for. It may be less about MPs' expenses and more about street lights, fixing roads and the micro-politics of our changing the nearby area. The real value of FOI, as one Scottish Information Commissioner put it, is to be found in the pages of local newspapers.
The landscape of local government and FOI has also been a changing one. Our surveys of local government and FOI between 2005 and 2010 found that the requests numbers were very large and growing. In the space of five years, we found requests to local government jumped from 60,000 in 2005 to 197,000 in 2010. We also felt, based on our interviews and other anecdotal evidence, that this may have been a considerable underestimate. Despite this huge jump in numbers, we also found that local government learned to deal more effectively with requests, more than halving the estimated time spent on them.
Since 2010, when our surveys ended, we have known little about what's happening. Councils are dealing with years of huge and deep cuts that have meant the loss of both resources and experienced staff. The government's recent revised Code around FOI has promised local government data on FOI requests, and a few have begun to publish them aleady. But aside from these changes, that may take time, local government and FOI is hidden from us.
This is why this report is so welcome. It gives us a first comprehensive picture of what's happening with FOI at the local level. It shows that requests numbers have indeed continued to grow and that local authority staff face a series of obstacles in dealing with requests. Above all, it underlines the importance of FOI as a tool of local democracy.
This study focuses on the 418 local authorities in the UK (it does not include the large number of parish councils that are also subject to FOI), which have slightly differing powers and scope dependent upon the location and type of the council. For this report, mySociety sent a series of questions to local authorities in the UK grouped into two separate FOI requests. One group of questions related to FOI statistics held by the authority (specifically related to FOI Requests received in 2017, and the success and appeal rates of those requests). The second group of questions related to how FOI is administered internally, specifically concerning staffing, budget and software.
For each set a pilot was conducted, with sample FOI requests sent to 10 councils to test the validity/integrity of the questions, and to identify any amendments that needed to be made to the phrasing of the questions in order to achieve standardised results. This resulted in a clarification of one question, and the discovery that Scottish local authorities all publish their FOI statistics in a centralised portal (and so could be excluded from the request for FOI statistics). FOI Requests were then sent to the full cohort of councils and the results recorded.
Of the remaining 388 local authorities, 89% provided at least a partial response to the request for FOI statistics. Information for Scottish local authorities was retrieved from the Scottish Information Commissioner data portal.1 In several cases for Scottish local authorities these statistics included FOI Requests to the licensing board (which are treated by WhatDoTheyKnow as separate public bodies).
WhatDoTheyKnow was used to find how many FOI Requests were sent in 2017 via that platform. For Scottish authorities, the count from WhatDoTheyKnow was combined with a count for the relevant licensing board.
There were a small number of authorities that required clarification on the request made (either on the wording of a question, or in reverse, where the website listed as hosting the reply was unavailable). Some authorities refused the request but indicated that a reduced request would be viable. In some cases some statistics were detectable even in the event of a refusal (where for instance, the cost of querying X requests to determine information gave a value for the number of requests in the period).
A minority (6%) of non-Scottish councils referred to already available online releases of statistics. These were a mixture of spreadsheets reflecting date received and status of FOI requests, or pdf council reports detailing the overall statistics. While typically stored (with varying levels of visibility) on the council website, Leeds City Council released their FOI data in the shared open data repository Data Mill North. More councils releasing their statistics on the same portal (for instance https://data.gov.uk would make the information more transparent and easy to access.
The new FOI Code of Practice requires online disclosure of these statistics (and several councils made reference to plans to change their tracking systems to reflect this upcoming requirement), and this will represent a necessary change in practice for almost all councils. Individually, this will reduce the burden on each authority to spend time answering similar FOI Requests in the future, but without a centralised or standardized system of storing these statistics, this will make the job of externally gathering the information more time consuming.
Using the official FOI email address for the council, a survey (to be completed anonymously) was also distributed to FOI officers querying the structure of teams, and their perceptions of WhatDoTheyKnow. This received 129 responses.
Interpreting FOI statistics
There are several hurdles in working with statistics derived from FOI requests. These reflect internal tracking systems and so in turn reflect decisions made internally concerning the categorization and logging of requests within that authority, rather than representing a consistent schema for comparison.
What counted as a request being ‘refused' varied from council to council - in some cases it included those where the information wasn't held as well as those refused via an exemption. While asking these as separate questions would often have received two pieces of information (many councils did volunteer separate counts), other councils did not track to this level of detail. Many councils included in their response, warnings about the level of accuracy of the statistics provided (where for instance results were compiled from two different tracking systems) that, while suitable for aggregate analysis, make positive or negative judgements on particular authorities difficult to authoritatively construct. In some cases, two authorities (e.g. Christchurch and East Dorset District Councils) handled processes internally as one structure. Some councils included in their total a number of requests that were eventually referred onwards to other authorities, whereas others did not. Some councils recorded requests made under the Environmental Information Regulations separately to those made under the Freedom of Information Act and others combined the two. For consistency, the numbers in this analysis cover FOI and EIR requests.
Councils also differed in the time periods they used to record statistics. 61.5% recorded FOI Requests in calendar year, while 37.7% recorded in financial year (within this were councils who recorded by month or individually, and so could report in either). Two councils provided figures in other time periods as a result of changes in tracking systems during that period. Where it was unclear from the information provided, calendar year was assumed. This created issues in creating a figure for local government in ‘2017' as not all requests were over the same time period. Additionally, requests were not evenly distributed year on year, and so there is a difference in the average for those reported in calendar and financial year. As explored Appendix A, this effect remains (although is smaller) when accounting for other factors.
This is also reflected in requests made to WhatDoTheyKnow. For requests made through WDTK in the equivalent 2017 calendar and financial year, 38% more were made in the calendar year. As both counts are available for WDTK, this demonstrates the effect is in addition to other differences between the councils.
|Type||Count||%||Average FOI Requests|
Differences in the ‘success' rates of requests may also reflect differences in requests instead of differences in processing. One council might receive a large number of requests on an area with an exemption, while another may not. Similar requests may be joined in one area but submitted separately in another, resulting in one council returning ‘1 All Info Returned, 1 Refused On Exemption' while the other returns ‘1 Partial Info Returned'. Statistics on successes and failure of FOI requests (while illustrative when there are large differences) can only be said to be meaningful using a longitudinal study, in which the same public authority is examined periodically over a longer period of time.
This difficulty applies not only to statistics sourced through FOI, but also applies to the self-disclosed data in the Scottish Information Commission repository, which has a similar disclaimer to FOI gathered statistics, in that the numbers may represent different things for different authorities, and so have issues for comparative use.
How many FOI Requests were sent to local government?
There are two issues in assembling a complete picture of how many FOI Requests were sent to local government in 2017: 1) Different time ranges used in disclosures (Calendar or Financial year) and, 2) Missing data points from FOI Requests that weren't returned or that didn't record this information.
These two issues were addressed by creating a model that adjusted for any systematic differences between different time ranges. This model was then used to fill in approximate values for the remaining councils. The methodology for this approach can be seen in Appendix A.
The result of this is an estimate of 468,780 FOI requests received in the calendar year 2017. There is a 95% confidence this value falls between 467,587 and 469,975 (range of 2,387)2.
Change over time
Until 2010, UCL's Constitution Unit conducted an annual survey of a sample of councils in England and created estimates for the overall number of requests made based on council type3.
|Year||Number of Requests|
However, the methodology for this survey was based on a small number of respondents and the report says that "the number of responding councils did not reach the level of a representative sample, this is not a scientifically reliable method of calculating the total number" (p. 51). This means that there is not data to conduct a reliable comparison across time. Interviews with councils conducted by the original researchers led to the impression that the 2010 figure might be an undercount.4
Based on the data collected for this report, the estimate for English councils only, is 388,736 FOI Requests in 2017. This represents a 97% increase on the 2010 figure of 197,000 estimated by the Constitution Unit. In the same period FOI Requests sent to audited central government only increased by 6%. Whether this represents a historical under-count or a growth over time (10% a year), the current volume of FOI is nearly double the previously available estimate.
In evidence to the Independent Commission of Freedom of Information, Liverpool City Council said they had experienced an increase of 76% (1,217 to 2,139) between 2010 and 2014.5 This would represent an 15% annual increase. In their response to this study's FOI request, Liverpool had the third highest response - indicating the size of this increase is probably not typical, but that an average growth over time of 10% a year may not be unreasonable.
Differences in requests by council
There was a large amount of variation in the number of requests received by councils. This graph illustrates the distribution of requests by the number of councils. Where a council's request volume has been predicted, they are distributed as partial values over their probability curve.
How many FOI Requests came from WhatDoTheyKnow
In the 2017 Calendar year, 28,282 FOI Requests were made to local councils from WhatDoTheyKnow. Using the adjusted and predicted calendar year totals from above, WhatDoTheyKnow accounted for 6.02%[6.01,6.04] of local government FOI requests. Therefore 6% of FOI Requests made to local government can be identified as originating from WhatDoTheyKnow.
95% of councils vary between having 2% and 13% of their requests sourced from WhatDoTheyKnow. More councils fall below 5% than above, with a minority of councils raising the average. The median council has 4.89% [4.80,5.00] of requests result from WhatDoTheyKnow.
Success and timeliness of requests
Based on the partial returns described above, the following figures and charts demonstrate the different rates of success and completeness at different authorities. Percentages relate to the total number of councils with information for that area, rather than the total number of councils.
Completed inside statutory deadline
In local government, compliance rates with the statutory 20 days deadline for reply are high, with isolated exceptions.
- 8 (2%) councils reported 100% of replies fell inside the statutory deadline
- 260 (80%) reported upwards of 80% of replies fell inside the statutory deadline
- 7 (2%) reported compliance rates below 50%.
While the Institute for Government has reported a growing trend of refusals in Central Government (approaching 40% withheld in 20176), the picture in Local Government is generally more positive.
- 5 (2%) councils reported 100% of requests were granted in full.
- 106 (43%) reported upwards of 80% of requests were granted in full.
- 19 (8%) reported that fewer than half of requests were granted in full.
There is a specific exception in the respective FOI acts that allows authorities not to comply with "vexatious" requests. This is interpreted by the ICO as allowing authorities to refuse "any requests which have the potential to cause a disproportionate or unjustified level of disruption, irritation or distress".7
The proportion of vexatious requests are low, with a minority of councils regarding a request as vexious in 2017 (however, the volume of work involved in processing a vexatious request may be disproportionately higher compared to other requests).
- 101 (33%) councils had at least one request considered vexious, 61 (20%) had more than two.
- 27 (9%) of councils had at least five vexatious requests.
- Of councils that provided information concerning vexatious requests, 0.16% of requests were vexatious (with the true figure likely being lower).
Requesters have the option of asking an authority for an internal review if they are unhappy with the decision reached, where the decision would initially be reviewed internally by the council, or following this, subsequently appealing to the Information Commissioner's Office (ICO). Complaints are generally uncommon, but most councils dealt with at least one in this time period.
- 305 (94%) councils had at least one internal review.
- 217 (67%) councils had more than five.
- 208 (81%) councils had cases where the original decision was overturned.
- 183 (64%) councils had at least one complaint to the ICO.
- 44 (15%) councils had more than five
- On average 1.4% requests were appealed to Internal Review.
- Of all tracked internal reviews, in 51-64%8 of cases the original decision was upheld in full.
- Based on data from the ICO website, of 486 decision notices issued by the ICO related to local government in 2017, 350 (72%) contained complaints that were not upheld, 225 contained complaints that were upheld (46%), and 25 (5%) contained complaints that were partially upheld.9
Just as different local authorities have different roles and remits, the position and responsibilities of an FOI officer within those authorities varies. Based on the survey of FOI officers, most respondents (93.4%) reported that they had freedom of information as a primary responsibility, with data protection also being a significant responsibility for most (67.6%).
A minority (27.2%) held a professional qualification in Freedom of Information.
A majority (62.5%) had ‘FOI champions' (or similar, ‘hub and spoke') - intermediaries between the FOI administration team and the knowledge holder - in their organisation.
This report used FOI requests to query the staffing and budget allocated by each authority for the provision of FOI functions. The responses provide a picture of FOI as a responsibility typically shared among many people in a larger team, occasionally with a small 1-2 person team administering FOI full time. The median is of 1 person employed as responsible for FOI (average 1.9 - but this is inflated by councils who listed a large number of people with some responsibility for FOI)10.
FOI is typically embedded in larger departments, but the exact nature of that department varies. Some examples are:
- Information Governance
- Information Management
- Democratic Services
- Complaints and FOI
- Compliance and Customer Relations Team
- Performance Governance and Information Security,
- In Touch team.
Some councils made the case that FOI is effectively a responsibility of all staff. The costs of administering FOI are therefore not only in the cost of administration, but in the cost of reply (with several councils making a note that the budget did not include the costs of formulating replies). Of the 38 councils that were able to provide an estimate for the budget for FOI administration, this mostly accounted for salary costs, with an average of £57,862 and a median of £35,699.
This may inflate the average cost of FOI as administered if the spend is lower in the majority of councils that do not have a specific FOI budget. However, as several councils did provide estimates of the full time equivalent work spent on FOI when it was not itemed directly (and these were included into the above), not being able to enumerate the cost of FOI administration does not necessarily mean a lower spend.
Of the two councils who listed the cost of their case management system, these listed costs of £4,590 and £4,690 for iCasework, with a note that the cost of the contract declines over time.
Running regressions of requests received against approximate FOI staffing11 and budget demonstrated that these figures were responsive to the number of requests being received by the council. Based on a sample of 35 budgets, every request led to a budget increase of £57 [95% Confidence between: 19.91, 93.63], and, based on a sample of 266 councils, every thousand additional FOI requests led to a 0.75 [0.35, 1.15]increase in administrative personnel. These two figures are related as the increase in budget mostly reflects staffing budgets - as there was far more information returned on staffing counts (sample of 266) than budgets (sample of 35) the latter should be seen as more valid statistic.
The figure for budget above may also be higher than the 'true' figure - as it is possible the budget for FOI is more likely to be separately itemised when higher. Two councils (High Peak Borough Council and Staffordshire Moorlands District Council, which share some backend services including FOI) respond to FOI requests with the cost of processing of that particular FOI request. Examining over 250 responses from these councils on WhatDoTheyKnow shows the difficulty in trying to find a cost per FOI requests. Itemised values ranges from £12 to £450 - with only nine costs above £100. The median cost for a request was £25, with a slight difference in mean average (£40 vs £33). This reflects that differences in the cost of FOI can reflect the kind of request made as well as the overall volume of requests.
On average, 38% of requests required information to be sourced from multiple departments or teams, however the survey included organisations where significantly higher proportions of requests required information from multiple teams (Chart 6).
On average, 23% of requests required ‘double handling' (additional sign off by senior or specialist staff, or two persons conducting redaction duties), with the majority of councils reporting this in less than 10% of cases, and with a small number reporting this in a high number of cases (Chart B).
There was no consensus across authorities on the most effective or efficient case management systems in use, with a sizeable group (34%) claiming to use no Case Management/Client Relationship Management (CRM) system. However, this group might contain councils that are using Outlook or Excel to manage FOI replies, but do not consider this CRM software as such. As an example, one reply noted that "We don't have case management software, we use Excel". A next largest grouping built their own in-house systems. This might mean bespoke developed software, of use of an Excel sheet or Access database. This was sometimes an extension of existing in-house tools and project management software like SharePoint or Salesforce. For specific software, iCasework has the largest user-base at 7%. Other CRM includes 26 additional named pieces of software and several unknown software packages.
The use of case management software is responsive to the number of FOI requests received, with greater use of management software associated with a higher average number of requests. A ANOVA test confirms that this difference is statistically significant (p < 0.01). While on average those with full CRM packages had a lower success rate at replying within 20 days, a linear regression taking into account the effect of the case management package and the number of FOI Requests received demonstrated that there was no statistically significant effect of CRMs on the response rate, with the relationship more likely a result of councils with more incoming requests performing worse (on average) in terms of response rate.
|Case Management||Average Requests||Avg % Response in statutory window|
House on the Hill
Capacities of local government
In many responses to the FOI request about statistics, some questions went unanswered, as the respective FOI tracking systems did not allow quick recall of specific statistics, and would require the review of hundreds of individual requests (and hence trigger a time limit exemption). These exemptions help to reveal the capacity of local authorities to understand their own FOI throughput. This applies only to authorities that returned replies, which may possibly inflate the figures below, but which nonetheless provide a useful comparator.
How many FOI requests have you received?
How many requests were completed inside the statutory deadline?
How many requests were appealed to internal review?
How many FOI decisions have been appealed to the ICO?
How many decisions were upheld by the ICO?
How many decisions were upheld at internal review?
How many vexatious requests were received?
The number of requests where the information was granted?
The number of requests where the information was entirely withheld (no information provided)?
The number of requests where the information was partially withheld (some, but not all information requested, provided)?
The highest availability of knowledge were figures on numbers of FOI Requests received. The two areas where almost all authorities had records was the number of FOI requests received (98%) and how many were completed inside the statutory deadline (92%).
The next grouping was awareness of when requests had been referred to review. Records of internal review were held in 87% of cases and records of appeals to ICO were in 86% of cases. Records of outcomes of ICO decisions (84%) was more likely to be held than records off the results of internal review (80 %).
While records of the amount of vexatious requests was relatively high (81%), generally the questions with the most missing information related to how much of a request had been delivered. 73% had records of the number that were completely granted, 70% had records of the number that were entirely withheld, and 65% had records of partially withheld/disclosed requests.
Most respondents (57.4%) said that their organisation does not currently publish FOI responses on their website. A little less than a quarter (22.8%) said that their organisation did, with the remainder saying either that some requests were available online, or that they were in the process of pushing for / setting up a public disclosure log. A majority (57.8%) felt their organisation could publish responses better. Asking what they want to do better, a few expressed desires to publish more or make the website more searchable, with others saying that plans were already underway:
- We need to start publishing them quicker and also bilingually
- [L]imited by current software but planned to be fix with new systems
- A better search engine to identify metadata/keywords. This is in development.
- Our responses are published on our website as soon as they are sent out to the requester and I don't believe this needs improvement. However, I do think that our published FOI Requests can be difficult to locate.
The FOI results found that the majority did not publish a disclosure log. Of those that didn't, some referred to pages that published statistics or commonly answered questions that serve a similar purpose in reducing need to submit an FOI. There was a set of authorities (5%) that were planning to publish a disclosure log, with another 2% being further along with a time-scale specified to set one up. There were also three councils who had previously published, but had ceased pro-active publication due to the cost of resources required to maintain the service.
Councils that had a disclosure log on average received 172.703 [40.22, 305.18] more requests than those that didn't. This likely reflects that councils are more likely to set up a disclosure log if they receive comparatively more requests (and not that a disclosure log causes more reports).
When the binary variable representing if a council had a disclosure log is added to the model described in the FOI Statistics section (which accounts for factors such as type and service spend of council), there is no effect (positive or negative) of having a disclosure report on the number of reports received once other factors are accounted for. This validates the perception that disclosure logs do not reduce the number of requests.
While a majority of respondents to the survey were in favour of publishing more, there was a group sceptical of the benefits of having a searchable public record, with the feeling that this did not actually get used, and so did not reduce requests.
- We don't publish, but we should. Trying to develop a solution without creating a cottage industry. Not sure of any evidence that publishing responses adds any value whatsoever.
- When submitting FOI enquiries, the general public does not search for the information prior to submitting a request. It would involve a vast amount of administration to maintain a disclosure log simply for it to be unused - it would not result in any less requests and as all requests must be answered, it would not reduce the workload in any way.
- We don't publish them so no suggestion on improvement other than to start publishing. We did used to publish them c.5 years ago but analytics showed that nobody was looking at them so we decided to save the resource needed to publish.
- We tried publishing our responses and it made no difference to the number of requests we received. Nor did the public check the responses prior to making a request for the same information.
- We don't publish responses and I am personally very sceptical publication would lead to any reduction in future requests, which would be the primary driver for an organisation.
- We do not publish FOI responses as it is extremely time-consuming. Especially given that as soon as they are published they are already starting to become out of date. We publish some open data resources on a regular basis.
- I would recommend publishing FOI responses, if I thought this would result in a decline in the number of requests received. However, I have seen no evidence that this would occur. Many authorities and outlets such as "what do they know" publish FOI responses but the number of FOI requests received continues to rise. In addition I have seen comment from some requestors (media) who do not wish FOI responses published as this would hinder the "exclusiveness" of any story they wanted to write on the back of the response.
- I do not feel there would be any benefit to the organisation publishing all FOI responses on its website. I believe the vast majority of the requests we receive are for information that is of no interest to anyone other than the requester. Where there is a wider public interest in a response, it usually ends up being published anyway (either in the press, on WDTK, or elsewhere) and so the benefit of publishing all responses would be very little when compared to the amount of extra work this would generate.
In some cases even where there is the belief that more publication is a good goal, there are fundamental resource issues:
- Your system publishes each response. I would like my organisation to do the same, but there are cost and technical issues to be resolved.
- I believe that publishing an up to date disclosure log is as much as we can reasonably do. Publishing every FOI response would require extra resources we do not currently have.
- The FOI system that we use has a publication feature for responses, but we do not have the budget to take up this feature and we do not have time to publish all responses online by other means. Just keeping up with the volume of requests is difficult enough.
This report has found that the overall number of FOI Requests sent to local government is almost double the previously available estimate, and stands at approximately 467,000 in 2017.
There is substantial variation between different types of council - both in numbers of FOI Requests received and in how they are administered. These two elements are linked, in that increases in staffing, budget and use of CRM systems are associated with increased numbers of reports.
There is no clear standardisation of case management systems, with the largest groups of councils using no CRM or a custom/Microsoft Office based version. Having a more full-featured CRM makes no difference (positively or negatively) to the success rate in this current dataset, suggesting that obstacles to timely response exist elsewhere in the process.
Variations in compliance are an inadequate evaluation measure, as councils will necessarily be operating in different circumstances (as they are receiving different requests) as well as using different processes. A longitudinal study would be able to more usefully identify councils that are consistent outliers, as opposed to just those having ‘outlier' years.
Appendix B explores if this exercise could be repeated usefully to build a longer dataset, however finds that the minimum useful sample requires approximately 50% of councils to respond which, while a large reduction in work from this survey, remains too large to repeat on an annual basis. The most viable path to better knowledge in this area is for the Information Commissioner's Office to mirror the Scottish Information Commissioner and build a portal to centralise the publications of statistics for all authorities covered by the requirement in the new Code of Practice.
The authors would like to thank hundreds of FOI Officers working in local authorities for their time in replying to the FOI requests and the survey, as well as those inside and outside mySociety who have commented on drafts of this report.
A: Modelling FOI workload
This section describes the modelling approach used to predict missing values for FOI volume in local government. This uses the majority of councils that did return figures to predict the remaining councils and arrive at an estimate for overall FOI volume.
The assumption this method makes is that councils that replied to their FOI request are representative of those that did not in their FOI throughput. However, assuming these are not the results of technical issues, this might actually be highly suggestive that these councils are not typical, with either a greater FOI workload leading to problems in process, or equally a lesser FOI workload resulting in worse processes. Appendix B discusses alternative sampling methods that could address this issue, as reducing the number of councils can allow more follow-up of non-returns.
The type of council (Metropolitan District Council, London Borough, Unitary Authority, etc) was converted to a series of binary variables. The time period covered was converted to a dummy variable reflecting if the time period was the calendar year (with the few councils using neither financial or calendar discarded).
A dataset of spending on services from datasets for England, Scotland and Wales (measured in £ millions) is included as a predictive variable. For Northern Ireland, each authority was assigned the average spend. The 2017 mid-year population estimate was considered as a variable, however, this is highly correlated with spending on services and so was discarded (as the type of authority also conveys information about the population). The number of FOI Requests sent through WhatDoTheyKnow was considered as a variable, but this added little explanatory power as it is highly predicted by the other variables in the model.12
The final model of counts of authority reported FOI Requests had an adjusted r-squared of 0.55 - explaining just over half of the variance in total FOI Requests sent to local authorities.
Recording in calendar year led to 108.88 [30.75,187.00] more requests being recorded. Accounting for all other factors, for every £1 million increase in service spending, 0.84 [0.63, 1.06] more FOI Requests would be filed. It is possible that breaking this down into individual service spending or including other financial information might increase the power of the model. London Boroughs had a greater number of reports than the intercept, while all other council types had fewer reports than the intercept (metropolitan districts and unitary councils were not significant).
Using this model, the predicted value (and the upper and lower bounds) were calculated for all councils who had not replied. All councils who reported in financial year were adjusted upwards by 108.88 FOI requests (with a upper and lower bound increase of 187.00 and 30.75).
The result of this is an estimate of 468,780 FOI requests received in the calendar year 2017. There is a 95% confidence this value falls between 467,587 and 469,975 (range of 2,387)13.
|Predictor Variables||Estimate||Std. Error||t value||p value|
Service Spending(£ Millions)
Recorded as Calendar
LA - Northern Ireland
LA - Wales
B: Accuracy with smaller samples
This project involved sending FOI requests to every local authority outside Scotland (as the Scottish Information Commissioner makes these statistics available through a central portal). This was quite a costly way of determining the number of FOI Requests sent to local government, as sending an FOI request creates costs for authorities in answering the request, and costs for the researcher in processing the replies. These costs should be considered in thinking about how this exercise might be repeated to establish the change in FOI Requests over time, or when examining FOI volume in other areas of the public sector.
The new Code of Practice will require public bodies above a certain size to publish this information without needing an FOI (and several councils in their replies indicated that they were working towards publishing information in line with this). While this is in principle a welcome development, this might make it more difficult to create an aggregate understanding of FOI use. The most difficult replies in this project to process were those where the reply referred to information that was already published. These might refer to PDFs of council minutes, or Excel spreadsheets formatted in different ways that required querying to arrive at the requested numbers. On the other hand, those that did not proactively publish the information had a legal obligation to provide the information where they had it, and so produced it in an easier to process way than those that were doing the "correct" thing in publishing the information. Publication reduces costs for the public bodies and increases those for the researcher.
This is arguably a reasonable trade-off. FOI deals with free access to public information, not free access to the labour of council staff. However, there is a transparency paradox, where local authorities releasing information proactively increases the difficulty of assembling public information when it is not "held" in one place (but must be assembled from information from many different authorities).
While a system similar to the Scottish Information Commissioner's portal would make it trivial to build a picture of FOI use in England and Wales, it is worth investigating in the absence of that how accurate an impression a sample of authorities can give.
Given there already exists a sample of 89% of councils, this analysis re-runs the process above (using a model trained on a sample to predict remaining values) for smaller sample sizes.
For percentage levels of 10%, 20%, 30%, 40%, 50%, 60%, 70%, and 80% - five random samples (stratified by council type and service spend) were taken from the existing dataset to evaluate how these limited sample compared to result produced from the fuller (89%) sample. The below table and chart shows the distance between the lowest 95% lower-bound estimate produced and the highest 95% upper-bound estimate produced for each percentage level and the mid-points for each of the five samples. It shows the average R2 of the five predictor models for each sample size, and the standard deviation between those R2s.
This shows that the largest gains in reducing uncertainty started with the 50% sample size. This is also associated with a decrease in the standard deviation of the R2 of the 5 models tested for the sample size. Increasing the sample size beyond this point further decreases the uncertainty.
Increasing the sample size from 40% to 50% reduces the range of possible values 2.7x. Increasing from 50 to 80% only reduces the range 1.4x. Similarly the standard deviation in the value of R2 across the different predictor models has a significant drop moving from 40% to 50%. However, it is worth noting that the larger sample size produces tighter uncertainty but this is not entirely contained within the less precise 50% range - with a larger sample size suggesting an increased probability for values that fall outside the original 95% ranges.
This suggests that if this analysis had been conducted on a 50% survey of local authorities (210) it would have produced a number with a 95% range of 63,113. As this number is produced from knowing the outcome of multiple possible samples, the result actually produced from just one sample would suggest a misleading, tighter range (around 5,000). Using this range, the aggregate change between years would have be greater than 47,000 to be statistically significant. Assuming the change since the Constitution Unit survey is accurate - this would mean that the change in FOI Requests would only be statistically significant every other year.
These figures could be improved either by improvements to the predictive model, or by a better sample selection methodology. However, the improvements would have to be substantial to detect what are likely to be relatively minor year on year changes.
While sampling substantially reduces the amount of work required to build a picture of aggregate requests, the work required remains substantial. WhatDoTheyKnow covers over 23,000 public authorities, and while there is a long tail effect (a large number of these are small organisations such as schools and dentists), a representative sample of the different kinds of organisation present would require a large effort. While small samples would be useful for a general understanding of relative numbers of FOI Requests received, this is unlikely to result in an accurate aggregate picture of FOI use, and hence accurate understanding of the relative role WhatDoTheyKnow plays.
|Size of Sample||Lower Bound||Upper Bound||Range||Average R2||Std Dev R2|
2:While the predictions all have individual confidence intervals, in aggregate the uncertainty becomes substantially smaller.
4:Correspondence with Ben Worthy
5:https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/504139/Independent_Freedom_of_Information_Commission_Report.pdf, p. 44
8:This is expressed as a range because of a possible error in data collection - where some councils may have responded to a question asking how many ‘decisions' were upheld with how many ‘complaints' were upheld. This possible ambiguity was detected by a few who described the answer as the number of complaints - and it is possible that others did the same without explicitly stating so. To try and account for this, the answer was flipped for any council with a less than 30% upheld rate and where the difference was more than 3 (as a council with a original upheld rate in the 30s was explicit that they were recording correctly). The higher number assumes all of these councils incorrectly reported and that there are no councils with a rate below 30% - the lower that all councils reported the correct figure (with the true figure likely somewhere in between).
9:It is possible for complaints to be escalated to the ICO and resolved without the issuing a decision notice. The count of ICO decision notices does not then mark every time something has been escalated to the ICO, and it is more complicated to create a ratio of complaints raised to decisions upheld/overturned. The same decision notice may include upheld and not-upheld complaints.
10:The exact figures are complicated by differences in how councils responded to this question, with some giving rough calculations for the equivalent full time position, while others said they could not answer the question.
11:As best as possible given disclaimers around the makeup of numbers in individual replies - direct comparisons between two numbers are not useful, but in aggregate should have some explanatory power.
12:In fact the same model run against requests made to the council via WhatDoTheyKnow has an adjusted r-squared of 0.81 - these variables are far more predictive of requests made via WhatDoTheyKnow than requests in general.
13:While the predictions all have individual confidence intervals, in aggregate the uncertainty becomes substantially smaller.