Title: Protocol for a literature review in investigating the potential of DevOps for public sector Information Technology delivery

Background The United Kingdom (UK) public sector has spent more than £3.2bn on digital, data and technology services since 2012. The UK public sector borrowing year ending March 2018 was at £41bn. The debt at the end of February 2019 was at £1785bn (Office of National Statistics, 2019) and because of this spiralling debt, the UK government had to implement austerity. Therefore, there was a need to address solutions to help the government implement austerity. For example, the NHS Digital (NHSD) annual report and accounts 2017 to 2018 reported an increase in expenditure for permanent staff, software development, IT maintenance and IT support. It is evident there is an increase in expenditure compared to their 2016 to 2017 report. This provides scope for reduction in expenditure. Objective This literature review aims to investigate if DevOps has the potential to transform public sector IT delivery and provide value for money to the taxpayer by reducing expenditure in software development, IT maintenance and IT support. We will also be reviewing the maturity of DevOps in its current form and address the benefits and downsides from a public sector perspective. Method Our literature review will apply the Preferred Reporting Items for Systematic Reviews and Meta-Analysis Protocol (PRISMA-P). We will accomplish a two-part study and apply thematic organisation of our literature. We will use databases recommended by the University of Portsmouth Library. Grey Literature will be included, but we will exclude untrustworthy literature, social media, broadcasting, and press media. Only literature with confirmed validity will be included for further research.


Introduction
In 2018, the United Kingdom (UK) Government were working to find solutions to help balance the UK public sector financial books, so they implemented austerity. This austerity meant there were spending cuts which can have an impact on the way IT is delivered. We needed to take note of the Wanless Report (Wanless, 2002) which had mentioned the UK health service makes very poor use of ICT (Information, Communications and Technology). The report also stated there are examples of successful use of ICT at local level, but systems have typically been developed and installed in a piecemeal fashion. This prevents the effective integration and sharing of information across a wide range of health care providers. This is just one example of the problems in public sector IT delivery.
To help deliver IT in the public sector, thousands of private small and medium-sized enterprises (SMEs) are now signed up to provide their digital, data and technology services to the UK Government. This is known as a buying process which comes at a cost to the public sector (GOV.UK, 2018). For example, the NHS Digital (NHSD) Annual Report and Accounts 2017 to 2018 summarised the operating expenditure for permanent staff, IT maintenance and IT support yielded the highest percentage of the overall expenditure budget. The IT maintenance and IT support expenditure had increased by ten percent compared to the 2016 to 2017 report. The report also wrote, the work packages costs to provide short term specialist input, outsourced services and software development skills have increased to supplement internal teams. As a result, the capital expenditure had increased, particularly for internal and external software development, and this is also the same with the development expenditure (Health and Social Care Information Centre, 2018).
However, some teams in the NHSD organisation have successfully adopted DevOps and received the DevOps Industry Award 2017 as the best overall DevOps project in the public sector (NHS Digital, 2017). We can use this win as an example of successful DevOps implementation within the public sector.
This literature review aims to explore the potential of DevOps to resolve some of the issues within the public sector IT delivery. Also, the objective of this literature review is to investigate the existing benefits and downsides with DevOps, which will facilitate our research to see if it is able to transform the public sector IT delivery in a positive way.

Background
What is the Public Sector? The Financial Conduct Authority (Public Sector Entity, 2019) defines the public sector as non-commercial administrative bodies responsible to the central governments, local governments, or local authorities.
The Cabinet Office (Cabinet Office. Public Bodies Handbook Part 1, n.d., p.5) defines a public body as a formally established organisation that is (at least in part) publicly funded to deliver a public or government service, though not as a ministerial department. The term refers to a wide range of entities that are within the public sector.
Most public bodies that operate within the UK are established and operated by the Government. (House of Commons Library. Briefing Paper, 2018, p.4). Some, such as the Parliamentary and Health Service Ombudsman, are set up by and directly accountable to Parliament.
In the UK, the public sector has five sub sectors, Public non-financial corporations • Public financial corporations (or public sector banks) •

Bank of England
The public sector is not the same as the private sector. The private sector comprises of companies that are usually owned by individuals and driven by profit. The private sector is funded by private money, whereas the public sector is funded by government taxes. However, some public sector organisations do outsource to the private sector.
As of 31 March 2017, there were 305 known public bodies in the UK with £203 billion gross resource expenditure (Cabinet Office, 2017, p.6).

What is DevOps?
The term DevOps was coined around 2009, when Patrick Debois named a conference, DevOps Days held in Ghent, Belgium.
DevOps is two words combined, Development and Operations. The key differences between Development and Operations are, the development side mainly creates the product, and the operation side focusses on keeping the product alive. The development subcategories may be designing, programming, and testing. The operations subcategories may be network management, service management and systems administration.
Whilst there is no single concise definition for DevOps. According to the DevOps Handbook by John Willis, Patrick Debois, Jez Humble, Gene Kim (Willis, Debois, Kim, 2016), they define DevOps is a technique that mostly importantly enables teams to work together in humane work conditions without the need for firefighting or heroics. Without DevOps, there can be several obstacles and other problems such as manual effort and handoffs which can result in days or weeks, or even months of waiting, the DevOps Handbook divulgated complications can result in negative impact to customers and by solving these impacts with DevOps, we can improve performance, not just from a technology perspective but from an organisational perspective as well. They know that if left unmanaged conflict can exist between development and operations resulting in slower time to market, increased outages, and technical debts. Their comparison of tightly and loosely coupled applications has explained that tightly coupled applications are common in large complex organisations often require long approval processes and often rely on manual testing. If we have high autonomy in small teams and keep the applications loosely coupled and contained to prevent global disruptions, then DevOps is ideal.

Literature Review Questions
Primary Questions RQ1. How mature is the DevOps concept and what evidence exists of its practical benefits?
RQ2. What are the main reported issues in adopting a DevOps approach and how can they be managed?

Secondary Questions
RQ3. What are the potential benefits and downsides of adopting DevOps in public sector IT?
RQ4. What are the cultural and organisational barriers to adopting DevOps public sector IT and how can they be managed?
Our PICO Table   Population Intervention Comparison Outcome IT development and support teams in the public sector and private sector

DevOps
DevOps against traditional development/operations split Quality and productivity of the IT delivery

Study Rationale
The National Audit Office reported that commentators have focused on government's failure to deliver ICT projects to time, budget, and specification as well as the systemic reasons for these failures (National Audit Office, 2011, p.4). The NAO also conveyed the Government is considering new ways of working that could potentially introduce ICT that is more adaptive, can keep pace with changing business requirements and enable new systems to be brought into operation more quickly than historically has been the case.
The complications of the public sector IT projects seem to be exactly what DevOps intends to improve. The question is, can DevOps recover the UK's public sector IT delivery? Whilst DevOps is gaining popularity and recognition, we still need to grasp the fact if DevOps can strengthen IT delivery in the UK public sector, given the variety of challenges that arise within the UK's public sector and we also need to understand how the UK's public sector DevOps may differ from national and international private sector DevOps.
This literature review aims to apprehend how DevOps can or cannot improve IT delivery for the UK public sector and what the risks and benefits are and to measure DevOps maturity in its current form.

Study Objectives
The objective of the literature review is to identify, categorise, summarise, and synthesise literature around DevOps and know the presence and maturity for it. The benefits and risks of DevOps are not fully understood or entirely established, moreover the barriers and enablers of DevOps will take various forms, this literature review will provide informed knowledge around these entities. Our literature review will execute evidential claims study to clearly outline the DevOps benefits and risks and how these are managed.

Methods and Analysis
The EndNote Software (Clarivate Analytics, 2019) will be our data collection tool for the information management of this literature review.
All our inclusive and exclusive literature will be categorised using Endnote. Our bibliography will be built with EndNote. We also will record citations inside EndNote.
To ensure the success of our literature review, we will apply the PRISMA-P methodologies (PRISMA, 2019) and define, •

Search terms •
Appropriate searching tools for the web and databases • Identifying valid key publications • We will accomplish a two-part study, • Title and Abstract review • Full Text review Our literature review will be organised and categorised thematically. We will start with general keywords and narrow down our search in a funnel manner. We will identify perceptions and theories that are relevant to our research problem.
We will discuss the methodology the literature used and undertake comparisons to seek any arguments that are similar. We will also aim to identify differences in results and discuss ways that our research may help resolve these differences.
We will apply the evidential claim study.
• What evidence do we have from the literature that supports their claim? • What are the arguments given? • Can the evidence be warranted? • How reliable is the source of the evidence?
We will monitor our literature searches and identify core sources of information. We will also need to monitor new literature as they become available. Therefore, we will arrange our alerting tactics preferably email notifications with well-known databases as well as recursively revisiting the searches conserved in our database.
All planned searches will be recorded and tabulated, to allow these searches to be repeated.

Search Strategy
The search strategy will be designed to access both published and unpublished materials. These will be predominantly electronical to increase our accessibility to the literature, locally and internationally. Due to this we will need to efficiently use strategies to search, sort and manage the wider literature available to us. We will not be ruling out nonelectronic materials as if we are unable to obtain the literature electronically, we will obtain it in print if available.
The search will be refined in various ways to reduce the number of hits into manageable portions.
• Time Frame: Limited to 10 years' time span from current day • Disciplines: Software Engineering and Software Development • In English only We will define a list of keywords. To use those keywords in the topic search.

•
Where the keyword will appear in the title to produce fewer hits. First layer • Where the keyword will appear in the abstract to more hits. Second layer We will define our Boolean logic to widen the search to a larger number of sources when required.

Types of searching strategies
• Planning. Devising keywords and identifying appropriate tools • Browsing. Exploratory search to identify key areas • Methodical Searching. Once keywords and key areas are identified, we will conduct the methodical search and keep records of these searches • Citations. Linking to other area of relevance by citations. We will organise citation indexing and backward chain from these indexes • References. We will follow up references provided in the reference list of the source We will select several trusted worldwide university repositories to obtain the academic literature, that may have not been published as well as those that have been published. We will use databases recommended by the University of Portsmouth Library. We will include grey literature that may not be published by recognised publishers and have not been part of the published approval processes. Grey literature is defined in the Cochrane Handbook, as literature that is not formally published (Cochrane Handbook for Systematic Reviews of Interventions, 2011).

List of primary databases to search
We will search for businesses internal reports and documentations for their DevOps implementation processes and outlining their rationale and proposal in adopting DevOps. This will be considered as grey literature.
There are service manuals available from the UK government website: https://www.gov.uk/service-manual/technology , we will read these manuals and assess if they could support DevOps or not. Additionally, the UK government digital service provides guidance for the technology code of practice from their website: https://www.gov.uk/government/publications/technology-code-of-practice/technologycode-of-practice . We will read these as well and see if there is any relationship with DevOps.

PRISMA-P, Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols
We will follow the PRISMA-P checklist. Appendix A.
Full copies of the literature we have identified and those meeting the inclusion criteria of the PRISMA-P flow will be obtained for data synthesis. Other related literature to be identified through the reference list and bibliographic searches will be considered for data collection.
Our inclusion criteria: •

Data Synthesis
For our data synthesis will apply the Joanna Briggs Institute critical appraisal tools to assess bias (Critical Appraisal Tools, 2019). We will also examine the effects of heterogeneity as there will have been different ways DevOps will have been implemented and will have produce different outcomes and effects.
We will undertake a narrative synthesis (Ryan, 2013) on our literature and synthesise accordingly following the PRISMA-P flow diagram inclusion and exclusion criteria. Our narrative synthesis will summarize and describe our evidential study base.
To evaluate the maturity levels for DevOps in the literature, we will adopt principles from the Capability Maturity Model (CMM) (Paulk, Curtis, Chrissis, Weber, 1993). However, there is not a known formal DevOps Maturity Model at the time of writing this protocol. We propose DevOps maturity can be measured in four distinct areas: Culture, Automation, Measurement, and Sharing, represented as the acronym CAMS (Pais, 2014 p. 28

Limitations
As stated in our exclusion criteria, we will not be including these articles or papers in our literature review however we will allow academic blogs if we can prove authenticity of the author. We will not be including any literature that does not meet the criteria as defined in the PRISMA-P diagram.
We will narrow down search results and identify those that are most relevant to our research within the timescales. We will categorise these by differentiation, i.e. by group, by discipline and by source.
When we identify keywords, we will need to account for different phrases that similarly relate to the keywords identified as well as spelling differences.
We will consider synonyms using a Due to the growth of open access, there is a growing number of predatory publishers. These predatory publishers accept any article regardless of their credibility with the aim to make as much profit as they can. Unfortunately, these articles can be of poor quality and misleading. To support us avoid these predatory traps, we will only use trusted sources. We will also work with our research outputs team within the University of Portsmouth Library to validate the article if we suspect the journal to be predatory.
DevOps is an emerging concept and the popularity for it is rising exponentially fast. This rapid exponential rise could collapse any day. There is no guarantee that DevOps will continue to gain popularity in the near or far future. Given that DevOps is relatively new currently in organisations, it is certainly relatively new in the UK public sector. Therefore, we will need to reflect that we may not find much specific literature in this subject and may need to consider alternative strategies.
Some literature may not be available electronically, and we may have to use the University of Portsmouth inter-library loans, and this is currently averaging one week from request.
We might not be able to locate exactly the literature we want and could result us to recategorize our findings.