by Louise Crow, Gareth Rees (mySociety)
In the course of this discovery and alpha project, we have prototyped several different solutions around the key user needs identified in user research around the case management of FOI requests and SARs. In this document we present options for continuing this work and the associated benefits cases.
We started this discovery and alpha project with the intention of prototyping transactions in an end-to-end open source case management system. As the project developed, we made two significant findings. Firstly, that one of the key unmet needs across local authorities was the – the ‘10,000 ft problem' requirement for a broader focus on assessing a service and identifying strategies for improvement, which might include better case management systems, but also includes other approaches. The second finding came thanks to an approach by a development team at the Ministry of Justice who have been working on an open source correspondence tool. This answers some of the case management needs we identified as not well met by commercial options, and could serve as a base for further development.
Consequently we outline here four related options for continuing this work:
Option 1 - A focused collaborative alpha project between two local authorities to further develop the Ministry of Justice correspondence tool into a full featured open source case management tool for use in local authorities.
Option 2 - A small beta project to make available a public demo of the Ministry of Justice correspondence tool.
Option 3 - A beta project around a self-assessment playbook for FOI/SAR service managers to assess their focus and get guidance on approaches to improvement.
Option 4 - A small alpha around data standards for performance metrics.
We think that there's a good benefits case around options 1 (collaborative further development of a case management system built on the Ministry of Justice tool) and 3 (the self-assessment playbook), but option 1 is a larger and more complex project. It would benefit from some staged development with targeted user testing around the development of more advanced features and option 2 (making available a public demo site) could act as a lower-risk enabling project.
This document outlines initial benefits cases around digital tools to improve the case management of Freedom of Information (FOI) requests and Subject Access Requests (SARs). It was prepared by mySociety in partnership with Hackney Council, Cornwall Council, Suffolk County Council, Stevenage Borough Council and East Herts District Council in early 2019, as part of the ‘Better case management of FOI and SAR requests' project, funded by the Local Digital Fund.
These benefits cases were developed based on interviews and discussion with the with the five partner councils, desk research, and on a research study using FOI requests to local authorities. mySociety sent a series of questions to local authorities in the UK grouped into two separate FOI requests. One group of questions related to FOI statistics held by the authority (specifically related to FOI requests received in 2017, and the success and appeal rates of those requests). The second group of questions related to how FOI is administered internally, specifically concerning staffing, budget and software.
mySociety has previously worked in a commercial relationship with Hackney council to develop the ‘smart suggestions' front end system described in this document, and has a commercial relationship with iCasework, one of the commercial case management systems mentioned here, for the supply of data feeds from WhatDoTheyKnow at a cost of £1,650 pa total. mySociety is also a supplier of a street issue reporting service to Stevenage and East Herts councils and a petitions service to Stevenage council.
The Freedom of Information Act 2000 and Environmental Information Regulations 2004 provide public access to information held by public authorities. There are now 408 unitary, upper tier and secondary local authorities in the UK who provide public access to information in accordance with the provisions of the Act. Subject access gives individuals the right, under data protection law, to obtain a copy of their personal data as well as other supplementary information from any organisation that holds it.
The effective handling of SARs and FOI requests (and EIRs) is a widely held statutory requirement on local authorities, but like many council services, it can also be thought of using the framework of social, financial and user value. Social value meaning the impact it has on the health, strength and sustainability of communities and society, user value in terms of how well it meets user needs, and financial value as quantifiable value for money for the community and the authority providing the service.
In trying to understand the potential benefits that changes to request management could produce, it's useful to think in terms of the value request handling produces, the cost of the service, the cost of the changes, and the changes in the different kinds of value they could produce.
There is relatively little quantitative measurement around the value provided by local authority request handling: few measures of user value are widely collected, the main one being the percentage of requests which are answered within the statutory deadline. 80% of local authorities have 80% of requests answered inside the deadline.
A 2011 Constitution Unit study of FOI in local government suggested that at the time, FOI wasn't seen by authorities as having a directeffect (positive or negative) on internal decision making or service delivery, but that it had value to requesters, who felt that it made authorities more transparent and accountable. This is an important form of value at a time when trust in public institutions is low. FOI also has indirect benefit through facilitating scrutiny of public funding in conjunction with other mechanisms, such as the media, the aggregate effect of which can have a positive financial effect. For instance, US-based research found that local newspaper closure was associated with higher borrowing costs by the local authority.
We know from mySociety's previous research that it is rare for principal authorities to have a specific budget for FOI, which is a responsibility typically shared among many people in a larger team, occasionally with a small one- or two-person team administering FOI full time. We also know that the responsibility for FOI and data protection is commonly shared by the same people, and that those responsibilities typically sit in a bigger department such as Information Governance, Information Management, Complaints or Compliance and Customer Relations. These factors make the task of estimating the cost to local authorities of FOI and SAR handling challenging.
It is clear, though, that the majority of the direct financial cost is in staff salaries, with some additional cost in the form of software licensing for authorities that use a case management system. A majority (around 66%) of principal authorities do use some form of case management system. Some of those systems are generic business software, such as Microsoft Excel and Access, but some are focused on correspondence, or more specifically on FOI and/or SAR, such as iCasework, Infreemation or CycFreedom. For the more request focused systems, it makes sense to consider their procurement costs as part of the overall cost of running a service as they would be procured chiefly or in large part to support the service. A further staff cost is in training — although a minority of the FOI officers in our study (around 27%) held a professional qualification in Freedom of Information.
In order to understand how the costs and benefits of FOI trade off, several attempts have been made to produce cost per request figures since the implementation of Freedom of Information legislation in the UK. These attempts have had different motivations, used different methods and gathered data from different authorities, and have produced different end figures. One distinction is whether you include in any estimate not only the cost of request administration, but also the cost of formulating a reply. As a rule of thumb, there is more variation in terms of resource spent at the point where an information holder is actually assembling a response to a request, as this is the point where different information systems and records may be needed, depending on the nature of the request, and the volume of material to be prepared for release may vary enormously, particularly when responding to Subject Access Requests. We've also outlined in our user research report that principal authorities vary significantly in terms of the number and complexity of requests that they handle, and in the size, organisational complexity and responsibilities of the authorities.
It's useful to have an average cost per request figure in order to make predictions around the impact of likely changes in the volume of requests submitted to local authorities. However, even if cost per request was measured in a standard way, we would expect the figures to vary across local authorities depending on the efficiency of the request handling process and the nature of requests received.
In our research, nearly all local authorities were able to return information on the number of FOI requests they received, and 38 local authorities were able to return information on their budget for FOI handling. With this limited data set, running regressions of the volume of requests received by a local authority against their budget showed that in general every request led to a budget increase of £57, with a 95% confidence of this figure being between £20 and £94. We believe that this wide confidence interval reflects genuine differences across authorities in the underlying cost value. We've used these figures to build an alpha benefits model around the costs of delivering FOI which suggests that the cost to the sector of handling FOI requests is somewhere between £9,000,000 and £43,000,000 — but we are keen to emphasise that this is an early estimate with available information and not a comprehensive figure.
We don't have a per request cost estimate for SARs but our user research does show that largely the majority of the time spent, and therefore cost, is in assembling the response, not in the administration of the requests.
Local authorities are taking a wide variety of approaches in order to meet their staff's needs around case management, including process documentation, ‘how to' guides, internally developed and commercial case management systems, internal deadline systems, and formal and informal training programmes. In our user research, we identified four sets of user needs which are not completely met across our partners or across the sector, and have, we think, the biggest impact on the efficiency and quality of FOI services, and where digital solutions might have an impact. Explored in the research report, there is the 10,000ft problem, the allocation problem, the chunking problem and the insight problem.
Some of the areas identified in our user research where staff needs are not consistently well met around FOI management have also been highlighted in studies as disproportionate areas of cost in the Constitution Unit's review of the cost of Freedom of Information in different countries — for example, internal reviews to central government in the UK have been assessed to cost five times the amount of an initial request. There's a time cost associated with the proportion of requests where an internal review is requested. There is also some evidence around the relatively high cost of complex requests and those that require multiple signoff — a common finding in each report was the financial impact of administering a small number of disproportionately expensive requests.
Non-compliance with statutory response times may incur an additional cost in the future, as the Information Commission, the UK's regulatory body for Freedom of Information and data protection law, has stated in its draft access to information strategy that it will ‘increase the impact of FOIA enforcement activity through targeting of systemic non-compliance'.
Our partners do also experience significant challenges in areas where the potential impact of a digital solution seems less clear. As the advocates for transparency and citizen rights within the organisation, FOI officers often end up in the difficult position of managing across and up into other service areas in order to try to spread a culture of transparency. One of the things that varies across organisations that are subject to FOI is the relative seniority of FOI officers, and the extent to which that culture is embedded across the organisation. While digital tools around things like reporting and insight can help support officers in this aspect of their role, we think that the role of technology here may be limited.
One significant factor in our benefits model is that we predict a rise in the number of FOI requests that local authorities will receive over the coming years. Our research indicates that in 2017, around 467,000 FOI requests were sent to principal authorities, with the number sent to English authorities almost double the Constitution Unit's 2010 estimate of 197,000. This would suggest that there may have been as much as a 10% year on year rise in the number of FOI requests to local government. Amongst our partner councils, Hackney have seen an average 6% rise in FOI requests per year for the last three years, and Cornwall have seen a 3% rise. Recent research by the Campaign for FOI show that across all London councils there was an average increase of 4.2% in the volume of requests between 2016/17 and 2017/18. We have chosen a relatively conservative predicted rise of 4.5% per year in our model.
We know less about the volume of SARs received across local government and the cost of responding to them. The General Data Protection Regulation came into force in May 2018, bringing with it an increased awareness of access rights. Hackney, a London borough, have seen a 15% year on year increase in SARs over the last three years, with a high level of month to month volatility, and an increase of 31% in requests from 2017/18 to 2018/19. In contrast, Cornwall and Stevenage have not seen a significant increase. It's unclear whether a consistent upward trend will emerge across local government. As with FOI requests, the nature and complexity of Subject Access Requests that an authority receives will vary with its responsibilities. For example, social care responsibilities can produce complex SARs when people want to find out their care history in childhood. The same team are often dealing with SARs and FOI requests, both in a central team and in service areas, and a significant change in the volume of one kind of request may have an impact on the other. We haven't included any projections about change in demand for SARs as we have little data to base one on.
There's significant evidence that there is potential for increased benefits around information request management. A key contrast that was highlighted by our user research with the council partners on this project was that while there is a general consistency of process, there is at the same time a variation in the emphasis placed on the different elements of the service — people, process and technology — and in the internal and external context. The consistency of process suggests that some of those benefits could be realised widely across local authorities in the form of increased automation from improved case management systems. However, it's clear from the diversity of emphasis and nature of the challenges faced, that some of the benefits that are possible would not be delivered directly by an improved case management system. There's expertise in how to deliver an efficient service across our partners, and across the sector, that's not currently available to other service managers because it is siloed and not shared publicly.
In general, improved efficiency in information request management would have a number of benefits. These are non-cashable benefits in the form of saved staff time and improved citizen outcomes, plus wider benefits to local authorities and the public. There is user value in faster service to citizens, and we should note that the Campaign for FOI found that in 2017/18, 25 out of the 34 London councils were falling short of the ICO's expectation that 90% of FOI requests should be answered within the Act's time limits.
There's also potential to improve the quality of service for citizens — although there's overall a low level of internal review, there is a high level of decision change in internal review, with 40-50% of reviews resulting in a change to the original outcome, suggesting there's potential for improved consistency in the way requests are handled between service areas and central teams. As we've noted, the number of internal reviews in progress is significant from the point of view of workload and cost too, given that these are a resource-intensive process: there's a potential for a vicious circle as the number of requests in internal review increases.
When, for service level staff, responding to requests is competing with frontline services, particularly in times of crisis, any improved efficiency of the case handling process is likely to be reflected in either better compliance or more time spent on frontline services. FOI officers tend to be sharing responsibilities (or even have FOI as a minority responsibility), so improving efficiency will result in increased time for them to spend on their other areas of responsibility. For authorities currently struggling to meet statutory compliance, improved efficiency in case management could release time to focus on longer-term improvements to records management that would positively impact information requests in the future, such as replacement of systems in a service area that's experiencing particular difficulties, or implementing appropriate retention policies around particular kinds of data.
The potential for benefits to the wider public sector is high. The statutory duty to respond to FOI requests and SARs is widely held — in addition to principal authorities, 10,000 local councils in the UK hold responsibility for responding to FOI, and outside local government, it is also held by Fire and Rescue authorities, Police and Crime Commissioners, NHS Trusts, Universities, and Central Government departments, amongst other organisations (WhatDoTheyKnow.com currently lists around 24,000 organisations, the majority of which have the legal responsibility to respond to FOI requests). Principal authorities may be situated at the higher end of complexity and need in that wider group, being organisations with a diverse set of responsibilities and a high impact on members of the public across a number of areas. An even wider set of organisations have responsibility for responding to SARs, as data protection law applies to any organisation that holds personal data, although many of these organisations will only be handling a small number of relatively straightforward requests.
This option would be to take the next steps in developing a full featured open source case management system, using the MOJ correspondence tool as a base. This tool already has features that meet some needs identified in our user research well (some examples are given in our prototyping and testing report). As an open source tool that is free to use and whose code is available for further development and improvement, the tool and any improvements made by one local authority could be shared across any other councils that wanted it. In order to develop a case management system that would meet user needs widely across the sector and produce the maximum benefits, features would need to be piloted around:
The full quantitative benefits model for case management is here.
Our research does not show that use of existing case management systems is correlated with better compliance with statutory FOI deadlines, although it was correlated with having a higher volume of requests, so it may be that use of a case management system is necessary for councils experiencing a high volume of requests in order to meet statutory deadlines. It's hard to tease out the relationship in the absence of better data on the staff time required to achieve that compliance.
However, there is some concrete evidence that benefits could be obtained from widespread use of a full featured open source case management system.
These benefits are non-cashable - although they're represented here as financial gains, at the authority level they generally represent less than the cost of one staff member, and should be thought of as improvements to efficiency (‘more for the same'), effectiveness (‘better for the same') and compliance (‘doing what's required'). The staff time saved here could be used in better compliance with statutory deadlines, and the delivery of frontline services. The chief differential for benefits that are variable in our model is demand in the form of the number of requests an authority receives (rather than digital uptake by external users).
One potential area for benefits is in enabling the more widespread use of disclosure logs and using information from them to prevent unnecessary requests.
In councils who responded to our research, those that publish a disclosure log were in a minority, with many expressing a perception that disclosure logs take effort to maintain and scepticism about their effectiveness in reducing the number of requests that need to be made; they're seen as requiring extra effort with no obvious gain. We assume that generally local authorities without a case management system that provides the option to automatically publish requests to a disclosure log will not do so manually.
However, there is some evidence that disclosure logs and publication in general can be used effectively to reduce the number of requests made - Barnet Council attributed a drop of 17.5% in the volume of requests made between 2016/17 and 2017/18 to the volume of open data published and the publication of a disclosure log. There's also evidence from previous work done at Hackney that introducing a front end to the request process that shows requesters information relevant to their request from the disclosure log can have a significant effect in reducing the number of requests. In the first seven months of use, in 50 cases where suggestions were shown to the requester, they did not subsequently make a request — around 7% of the total number of requests made through the front end during that time. Around 60% of the total requests received by Hackney during that period were from the front end, as opposed to email. We have conservatively assumed that the effectiveness of this service at demand reduction will increase by a few percentage points over time as it is populated with more data, and the matching algorithm is tweaked, and that there is a gradual channel shift from email to the front end as people become aware that there is a chance when using the front end that they won't need to go through the process of making a request and waiting for the response.
Although the front end service built is available as open source code, integration into, or compatibility with, an end-to-end case management system probably represents the most efficient route to wider adoption of this approach to reducing demand by serving citizen needs before the request is made. The variety of case management systems in use at the moment, the need to host the front end system in addition to the case management system, the number of integrations needed and additional effort required for widespread adoption of the front end service without a case management system makes the adoption process difficult as a standalone service.
Some of the commercial case management systems available are capable of automatically creating a record in the system for each email received through the authority's FOI email address. If a local authority doesn't use such a case management system, they're having to transcribe the contents of each request into their case management system on receipt. In the absence of specific figures, we have assumed that perhaps half the local authorities with a case management system have this capability.
Although it's possible to monitor key performance metrics outside a case management system, a case management system that's capable of quickly generating relevant detailed and summary metrics, particularly across service areas, seems like the fastest route to getting widespread access to these metrics across local authorities. This is a valuable tool in the dialogue between central teams and service areas.
The level of metrics resolution support in terms of issues like time to allocate was generally poor given the importance of the service, with only tracking of lag indicators (rather than lead indicators) so it's difficult to understand what processes are having positive and negative effects on the overall performance.
We have assumed that the process insight provided by an open source case management system would particularly benefit local authorities that currently aren't using a case management system — with a 5% efficiency improvement initially, and that for local authorities that are already using a case management system of some kind, the initial improvement might be around 3%. For both kinds of local authority, we assume a gradual increase in that improvement over time, based on tweaks and improvements to the system.
In this project, we prototyped approaches to reducing the time to allocate the request to the correct information holder which could be built into a case management system. There is also some evidence from individual authorities that other process efficiencies like the chunking of complex requests into service area specific ‘assignments' and use of internal deadlines for dealing with each stage of a request can significantly improve performance overall in the face of increases in demand. Ealing Council used internal deadlines and systematic chasing of staff to comply with them to improve its compliance with response times between 2010 (83%) and 2016 (92%), despite a 72% increase in the number of requests during the period. It's certainly possible to implement internal deadlines without the functionality being provided within a case management system, but the inclusion of an automated ability to set and remind staff of deadlines would remove the requirement for extra time spent on the part of central team staff. In addition, we've seen in the correspondence tool the embodiment of process improvements around allocation, like the requirement for a service area to accept the assignment of a request. Whilst process support from a case management system has to be complemented with institutional support in order to be effective, we think it could represent a significant source of efficiency gain.
Again, we've estimated that local authorities who aren't currently using a case management system would get the most benefit out of these improvements — starting with a 5% improvement in efficiency for them and a 3% improvement for councils that are already using a case management system. Again, for both we assume a gradual increase in this figure over time, rising to 7% and 5% respectively by the end of the 12 year period.
Given that there are existing case management systems in use, we're anticipating that the features bringing increased process insight and efficiency would be driven by a continued focus on user needs in local authorities, which is not currently being provided by commercial systems. Arguably it may be being better met by in-house developed and modified systems, although to our knowledge these don't become available for adoption outside one local authority, so the benefit of these systems is always just to the authority in which they were developed.
* These are all non-cashable benefits, see Variable net benefits introduction for more detail
In terms of the quantifiable fixed net benefits that such a system could be expected to produce, the main areas to consider are reduced software development costs for authorities whose case management system is developed in-house; or reduced software licensing fees and changes in the cost of procurement for those authorities that are using a commercial solution.
We know the annual licensing fees that are paid by our lead partner for FOI/SAR case management software. Only a few councils listed case management software fees as part of their FOI service costs in our research, so our estimate for the average costs of software licensing are based on an average of the handful of data points that we have.
We know of several councils that have commissioned case management systems to include the handling of SARs and FOI requests. The government contract finder shows recent closed contracts for similar services from Bristol Council, which recently advertised a contract to the value of £500,000 to develop a correspondence handling system, and Kent County Council, for £286,000 for provision of a similar system to include complaints, compliments and comments as well as FOI and SAR. We haven't included these costs in our quantitative benefits models as we don't have much information on how widespread these relatively large contracted development costs are.
The process of procuring, customising and delivering a commercial case management system has been a challenging process for our partners and other councils. The limited ability to ‘try before you buy' makes it hard to make an informed assessment of commercial systems — with no demo being available before purchase or it being a difficult and slow process to get a demo; and demos and training that are very generic and don't demonstrate well whether or not systems meet the specific needs of the council. Demos or trials that use real data from the councils (eg the actual service areas and responsibilities) are not facilitated by suppliers.
The software context in which councils are operating, with staff involved in the request process using many different bits of software each day, means there's a need to have all correspondence, including FOI/SAR and complaints, managed by one system. When looking for partners for this project, the need to have a general correspondence management system rather than a system that was specific to FOI/SAR was one of the reasons several partners did not commit. This need is in tension with the need for FOI and SAR specific features. The need for a case management system to integrate with other systems in use, such as collaborative document editing tools, is also a significant constraint on options for procurement.
Changing case management systems is painful, and takes a lot of staff time across service areas, potentially causing a loss of trust internally if there are teething problems. There is also a tension between suppliers wanting to offer a ‘fully featured' system and the risk of overwhelming users. Information holders in particular are often only occasionally or peripherally involved with the request process. We've seen examples of them receiving too many inappropriate notifications, and exhaustion from context switching into yet another complex system.
We have included in our benefits model procurement costs based on those of the lead council, who have recently been through a procurement process. To generate an estimate for an average council, we've assumed that as our lead council is at the higher end of request volume and organisational complexity. A conservative estimate would be to halve the lead council's costs. We've assumed that a procurement process might happen every three to four years for a complex council, again based on the experience of our lead council but that for an average council it might be less often — every seven years. We're assuming that the option to collaborate with other local authorities on the development and customisation of a full-featured open source case management system, or the existence of such a system, would effectively replace these procurement costs with development costs.
The main costs around a collaborative project to create a full featured open source case management system, tuned to the needs of local authorities, are associated with software development time. In our benefits model, we have estimated how this development time could break down, and how a collaborative development project, initially with two councils working together, could progress.
The open source system developed at the Ministry of Justice has some features that are required to meet user needs in local authorities but which are not present in the case management systems they're currently using. At the same time, it is missing features that authorities already using a commercial case management system will require. Thus we model an approach in which a council with a high volume of requests and a complex structure, similar to our lead partner, collaborates with a smaller council with a relatively high volume of requests and not currently using a case management system.
The smaller council drives a focused alpha and beta project to customise the software for use in a local authority, creating initial public documentation and training materials, whilst the larger, more complex authority would undertake to drive the development of missing features that would make this a strongly competitive system to commercial offerings for councils experiencing a higher volume of requests and more complex environment, such as ingestion of requests by email, and complaints handling.
The benefit to each partner in this model would be clear — the larger, more complex council gets to benefit from the initial work done in a smaller council to customise and document the service, without paying the development costs. The smaller council ultimately gets a fully featured case management system, again without paying the full development costs. Both councils get more opportunity to test and refine services before bringing them into production, hopefully mitigating the issues associated with procurement. Keeping the initial collaboration to two partners, we assume that the overhead on development costs added by collaboration is around 20%, and that further councils can subsequently benefit from the system developed at around 10% of the initial development cost (including collaboration overhead) of initial development. This figure is a conservative estimate based on an assumption that each council may need some customisation and configuration - the expectation would be that this would tail off as more councils adopt the software and common needs are covered by further feature development. We assume in general a shared codebase but separate hosting of the service.
We assume here four days of maintenance a year for the codebase for general technical maintenance, and security updates, paid for by the lead council.
We assume that renewal of the service happens approximately as frequently as reprocurement would in a procurement model, and that each council using the service requires around 10 days of development time to refresh the service.
Our model suggests that the adoption of an open source case management system would have a positive net benefit both for the councils that collaborated to develop it and for other councils that adopted it, with councils that currently have a commercial case management system gaining somewhat more benefits.
The model estimates that if 75 councils adopt the case management system over the next five years, starting with five councils in the first year, and rising steadily for the next four years, the net present value of such an approach considered over the next 12 years would be around £9,924,000.
We assume that this adoption would be largely from councils with case management systems as there's a clearer benefits case, and they're likely to be feeling more demand. We recognise that this is a relatively high adoption rate (about half the number of local authorities that we estimate are paying a license fee solely for information request management), and discuss below a further small project which could facilitate this uptake at a relatively low cost. Although we've assumed a shared codebase but separate hosting throughout, uptake would also be facilitated by the availability of the software under a SaaS model, comparable to that offered by commercial suppliers.
There are also wider opportunities for value in this option — the Ministry of Justice, the original developer of the codebase, would see an increased potential return on their investment as new features are added at a low cost, which they have the option of incorporating into their own use. Other organisations subject to FOI and SAR, including central government departments, such as MHCLG, which is currently procuring around discovery for correspondence management, would also have the opportunity to benefit from an improved open source option.
As we discussed above, there are significant costs and challenges around procurement for FOI/SAR case management, particularly around local authorities getting the opportunity to understand how well a particular system is going to work for them in procurement. There is an opportunity to allow authorities to demo the Ministry of Justice correspondence tool and any further development of it by creating a public demo site with accompanying tutorials for common workflows that councils can populate with their own sample data.
The benefit of this project would be to build awareness of the existence of an open source option for FOI/SAR case management and to demonstrate its strengths and weaknesses compared to other approaches.
The costs for this option arise from the time needed to create the tutorial, hosting and maintenance costs of the site, and the cost of outreach to local authorities to make them aware of its existence. We estimate, depending on whether training materials already exist, this could make a total project cost of up to £20,000.
This option is really an enabler to wider use of either the Ministry of Justice correspondence tool as it is, or any further projects to build on it and the benefits that might be expected to result from them.
This option focuses on the continued development of the self-assessment playbook prototype into a beta service. As described in more depth in our prototyping and user testing report, the self-assessment playbook helps FOI managers to assess how effectively the FOI Service as a whole is running by guiding them through a survey, presenting a visualisation of their focus in terms of people, process and technology and offering them tailored guidance in the form of appropriate guides.
The full quantitative benefits model for the self-assessment tool is here.
We envisage that the main quantifiable benefits arising from delivering this service would be indirectly delivered as FOI service managers use this tool to get insights into how to improve their own service, and the sector as a whole builds up an evidence base around improvement and good practice. This kind of benefit is quite difficult to quantify, so we have made an initial estimate that of the councils that use the self-assessment tool, 2% make changes that lead them to a 10% improvement in efficiency that they wouldn't otherwise have identified. We also assume that if this tool was developed by four councils in collaboration, each of the partners might make a 5% process improvement; we anchor these estimates in improvements in compliance with deadlines achieved by councils through changes in their processes as referenced by the Campaign for FOI, and on the specific experiences of our partner councils — Cornwall in particular have placed an emphasis on BCS level training for the majority of Service Champions (predominately in areas dealing with a large number of requests) — noticing that the more training was carried out, the quicker the request handling process became, and that quality of responses also improved, resulting in less follow-up being required from the central team, and fewer internal reviews and external reviews by the ICO. The relative rarity of formal FOI qualifications amongst officers suggests that more intensive training is not a commonly applied solution, and might be one that could be more widely used. We propose that the beta of this service includes a guide to procurement for request handling and also assume a similar rate of action and improvement around that guide: that 2% of authorities will use the information in the guide to save 10% of their procurement costs.
The main costs in the creation of this service would be development time to of a tool to calculate the scores around the self-assessment and to match the user with the appropriate guides, and user research and content creation time in order to develop the guide content. We assume that several councils could collaborate on the development of the service relatively easily, splitting the costs between them, and that as service users, other authorities would only incur costs in the staff time they spent working through the self-assessment questions and guides.
We estimate an annual hosting cost of under £1500 a year and a renewal cost comprised of around five days of development, plus user research or content creation time every three to four years.
Our model suggests that the lead and partner councils could make a small net benefit over 12 years, with other councils benefitting at an extremely low cost.
Our initial estimates show that the net value to the sector of a self-assessment tool around FOI services could be around £1,900,000 over the next 12 years, if it was used by 200 councils each year. One of the advantages of this option is that councils using the tool can do so at very little cost and risk, with the potential that at least some of the councils using the tool would make changes in their approach that would result in significant benefits. Guides could be developed to cover the breadth of context and approaches, and the service could be revisited multiple times as the context for a local authority changes.
As a single simple service, made publicly available, it doesn't have very high costs to the partners involved, who we estimate would be likely to make improvements to their own processes as a result of their engagement with the project. Overall, it's a relatively low investment in digital tooling in order to gather and spread best practices in service improvement, which are currently siloed in individual councils and communicated in one to one conversations between services in authorities and only publicly available in council management reports, not as a simple collected resource. The simplicity of the project makes it valuable to a wider range of authorities who don't have time or budget to spend on a more resource-intensive solution, and can adapt the ideas to their own context. If the service was successful in this form, there are also opportunities for other authorities to contribute to guides, and to match local authorities through the tool with similar authorities based on their context and approach via a double opt-in messaging system, which may facilitate further collaboration and knowledge sharing.
One of the questions we had at the start of this alpha project was ‘What performance metrics are used or would be useful across councils to assess the performance of a digital request handling service against its objectives?'
We found that the potential benefit of any intervention is hard to quantify with a high level of confidence at a sector-wide level at the moment, given that, in terms of understanding the resource costs of a service, there is not often a dedicated budget for FOI in a local authority. As we discussed in our user research report, in this context there's value in being able to track the amount of time spent on each request. This is not tracked widely, although some authorities do this as a matter of course, and some do as a time-limited exercise. The number of requests in internal review, result of internal reviews, and time to respond to internal reviews are also valuable metrics. In our prototyping, other potentially useful metrics emerged, such as the time taken to correct allocation of a request and the compliance with internal deadlines.
In addition to understanding what performance metrics would be used to assess the success of further case management projects, it's worth considering whether defining a standard set of metrics would have benefits as a project in its own right.
In an environment in which the number of requests received is unpredictable, the ability to use performance metrics to understand what's happening at any given point in time is key, and a majority of local authorities don't have that capability. If detailed performance metrics are not available, it makes it difficult to break down a dip in performance and trace it back to the cause quickly. In a situation where a service manager needs to make decisions about where to invest in the service, the lack of clear, easily accessible information about performance makes it hard to assess the value of any intervention made.
The production of reporting metrics can be an effective tool to improve the performance of service areas, as in the case of Brent, who have significantly improved their response time compliance rates by 46% over four years partly by circulating a list of requests due imminently, sending strategic directors reports indicating requests that were due and overdue, and providing monthly performance information to the Corporate Management Team. Our user research interviews also indicated that the use of statistics in weekly service area performance meetings have been an effective tool in multiple authorities.
Wider easy availability of standard performance metrics would be a key enabler of further benefits, allowing for faster feedback on a wide range of interventions that could be made in order to improve the efficiency of a service. One area in particular where they could have impact is in the procurement process for case management. Having performance metrics for request handling in terms of outcomes frames the problem in the correct way for software suppliers, and would allow authorities to identify the specific areas in the request process that are problematic in their context to ensure that suppliers have solutions for them and that the solutions are effective. They could also be helpful in making accurate assessments of whether a request is likely to hit the cost limit.
The cost to define data standards around performance metrics would be relatively low: an initial set could be piloted as part of the self-assessment tool. However, the cost to each authority to consistently track them is potentially high, in the absence of good automated reporting to support that. There's some clear interdependence here between getting better automated reporting in case management systems and developing the metrics which would help evaluate and procure them.
There is already a movement on the part of the regulatory bodies for FOI and SAR to require more reporting from authorities. The Code of Practice applying to Scottish public authorities has a section requiring the disclosure of statistics (section 2.1). The Scottish Information Commissioner in 2013 started asking authorities to enter these statistics into an online portal, where they can be downloaded and viewed in bulk. The new FOI Code of Practice will require online disclosure of some statistics, including number of requests in internal review. However, without clear standards that are commonly used, the numbers may represent different things for different authorities.
There's value to the sector in the wider use of performance metrics that are more focused on process, like the time spent to respond to requests, or internal deadlines, and the resource being spent to achieve that value, but there are significant barriers to adoption once the standards are defined. The benefits from the insight gained from these metrics have to outweigh the effort to produce them. The best paths to wider adoption may be through by authorities finding value in more direct comparison and collaboration with other authorities (an idea which might be piloted with the self assessment tool), in the more widespread use of these metrics through case management tools that make them easily available, or through the increasing recommendation and requirement of them from regulatory bodies.
In this document we have outlined the organisational need for better case management for FOI requests and SARs. The handling of these requests is a widely held statutory requirement which contributes financial, user and social value, but whose costs and benefits are currently not well quantified. We have outlined our methodology for coming to a cost per request value with a wide confidence interval using our previous research and noted that areas of user need identified in our user research map onto areas identified in other research as being costly. We have quantified year on year rises in the number of FOI requests amongst our partners in the last three years as around 4.5% and noted that other research also supports this trend. SAR volume appears to be less consistent, with some councils experiencing high volatility and others a relatively stable volume.
We started this discovery and alpha project with the intention of prototyping transactions in an end-to-end open source case management system. As the project developed, we made two significant findings.
Firstly, that one of the key unmet needs across local authorities was the ‘10,000 ft problem' – the requirement for a broader focus on assessing a service and identifying strategies for improvement, which might include better case management systems, but also includes other approaches. Given the consistency of process but diversity of emphasis on people, process and technology at the councils we worked with, we believe the benefits that we have modelled as being obtainable through better information request case management could be produced partly through a better case management system but also through sharing expertise in how to deliver a more efficient service across the sector. These benefits would be experienced in the form of better statutory compliance and quality of service for citizens and more time spent on frontline services. There's a great potential to share these benefits to the wider set of local councils and other authorities subject to FOI and data protection law.
The second finding came thanks to an approach by a development team at the Ministry of Justice who have been working on an open source correspondence tool. This answers some of the case management needs we identified as not well met by commercial options, and could serve as a base for further development.
We have consequently outlined four benefits cases around approaches to continuing the work done in this project:
Option 1 - A focused collaborative alpha project between two local authorities to further develop the Ministry of Justice correspondence tool into a full featured open source case management tool for use in local authorities. We think such a system would have significant benefits around demand reduction, through the use of suggestions on request submission, process efficiencies around double entry of data, chunking complex requests and setting internal deadlines, and process insight around issues such as the time to allocate a request to the correct information holder. These areas are not currently well addressed across the sector. We estimate a net benefit across the sector of £9,924,000 over 12 years.
Option 2 - A small beta project to make available a public demo of the Ministry of Justice correspondence tool. One of the significant issues encountered by our partners is the difficulty when procuring of getting a demo of commercial systems with real data. This project would be focused on resolving that issue for the correspondence tool and increasing the chance of wider uptake.
Option 3 - A beta project around a self-assessment playbook for FOI/SAR service managers to assess their focus and get guidance on approaches to improvement. As a single service, made widely available, this project represents a low investment in digital tooling to gather and spread best practices in service delivery around information request handling, with a potential to deliver benefits across the sector without the adoption costs and frictions of option 1. We estimate a net benefit of £1,899,000 over 12 years.
Option 4 - A small alpha around data standards for performance metrics. We do not recommend this as an option as there are indications that the relevant regulatory bodies are developing further reporting standards, and metrics specifically focused on performance will face significant barriers to adoption unless they are embodied in tools that make them easy to collect and report on.
We think that there's a good benefits case around Option 1 (focused collaborative further development of a case management system built on the Ministry of Justice tool) and Option 3 (the self-assessment playbook), but Option 1 is a larger and more complex project. It would benefit from some staged development with targeted user testing around the development of more advanced features and Option 2 (making available a public demo site) could act as a lower-risk enabling project.