Adoption challenges of CI/CD methodology in software development teams

— Continuous Integration and Continuous Delivery (CI/CD) is one of the widely discussed topics in the recent information technology (IT) world within the software engineering and delivery sectors. CI/CD is an Agile based DevOps methodology in software engineering that has proven to help improved quality and efficiency in the software delivery process which results in cost effective and low risk business solutions. With the growing demand in the global IT industry specifically in the DevOps based software engineering, estimated to reach $8 billion by 2022, up from $3.9 billion in 2017 [1], we commonly see that more focus is now being given to process optimization to attain efficiencies. Sri Lanka has always been one of the topmost sought-after destinations for outsourced IT labor in the world and currently employs 115, 000 skilled IT professionals [2] and it is important to include the advancements to process optimization to gain better results and keep up that momentum going. CI/CD being classified in the international IT market an improvement as such, the adoption of CI/CD and its perceived impact among Sri Lankan IT teams were researched. The literature review carried out did not provide much detail based on statistical evidence to understand the current adoption of CI/CD as well as the degree to which it is adopted among the IT teams specific to the Sri Lankan context. If CI/CD is adopted within Sri Lanka already, although undocumented, it was also important to understand what factors impact the adoption rate. Agile development methodologies are often adaptable to suit the best working solutions for teams, companies and even industries, hence there can be major inapplicability when it comes to studying a global set of factors from different contexts impacting the Sri Lankan teams. Building from this, the three top factors stated in international literature, team factors, stakeholder buy-in and complexity were studied, specifically within the Sri Lankan context to evaluate their impact on CI/CD adoption.


I. INTRODUCTION
The research studied how Sri Lankan workforce in the Information Technology (IT) domain is adopting Continuous Integration and Continuous Delivery (CI/CD) as a software development methodology and the connection between the adoption and commonly seen challenges in the sourced literature within the Sri Lankan context. The main objective of the research was to build a relationship between the adoption of CI/CD and its challenges negatively impacting the adoption rate within Sri Lankan IT teams. The hypothesis of the negative impact further narrowed down the degree of its impact with the objective of finding insightful information helpful to the industry going forward.
A. What is CI/CD? Fig. 1. CI/CD is a DevOps methodology in software development and delivery [3] Continuous Integration and Continuous Delivery (CI/CD) is one of the most important and desired processes for modern code development and delivery for new and transforming software development environments. CI/CD provides shorter compilation or build times and faster release cycles with more automation and testing. Adopting CI/CD processes can drive innovation and faster time to market for commercial and enterprise applications [4]. The representation of the process is explained in Fig. 1 above.
CI/CD is a methodology among companies which has already adopted Agile and DevOps are picking up a lot of interest due to the myriad of benefits and efficiencies it offers. Due to faster integration of code, companies can deliver quality releases of software much faster thereby helping to gain a competitive edge in the market [5]. Through automation, continuous deployment ensures that software can be reliably released anytime as needed [6]. Continuous delivery provides the benefit of having all things automated, improving developer productivity, improving code quality, and delivering updates to the customers faster [7]. Combined, CI/CD gives the benefit of faster software releases which are reliable, of highest quality through test automation and continuous service provision. Due to the reasons above, CI/CD is currently being adopted rapidly in the software engineering industry. Davey [1] has predicted that worldwide DevOps software market to reach $8 billion by 2022, up from $3.9 billion in 2017. MediaOps Inc. [8] states that "When Mabl, a test automation company surveyed 500 software testers, 53% of the people said their team uses continuous integration, while 38% embrace continuous delivery and only 29% have adopted continuous deployment".

B. Where does Sri Lanka fit in in the gloable IT market?
Davey [1] stated that the global information technology (IT) and software development industry was estimated to be worth 5.2 trillion US Dollars in 2020 with ninety percent of the world's exports in software being from the United States and Europe. United States being the largest in the IT market represents about 32% of the total accounting to 1.1 trillion USD with Europe coming in to a close second with 0.34 trillion USD. However, despite the market size occupied by the two giants above, most of the technology spend comes from beyond their borders [9].
When talking about the IT industry in Sri Lanka, although on the above data representations, at a glance the contributions seem insignificant, the overall strength of the Sri Lankan workforce has reported to have grown from 82,854 in 2014 to 124,873 in 2018 which amounts to a growth of 50.7% representing a compound annual growth rate (CAGR) of 10.80% [2]. Within the country, IT companies have become the dominant employer with a share of 65.5% of the workforce, overtaking non-IT companies (22%) by a wide margin. Sri Lanka has captured the attention of global industry analysts too as a preferred destination for IT labor force. The ICTA [2], states the below rankings by several global industry watchers further proving this argument. In the Sri Lankan context of software engineering, Safwan et.al. [12], in their research has found out that 68% of the respondents to their survey are currently using and involved with Agile methodologies in their developing process. These shows that almost three quarter of the population are adopting Agile and even though there are no documented statistics to prove its growth, it can be fairly assumed it only increased from there till 2020. As the next step, it is safe to assume that just like any other country breaking through the economic barriers in the IT industry, most of the Agile following companies in Sri Lanka too are looking towards process optimization using CI/CD.

C. Research Question
This brings us to the problem statement which was the foundation of the research carried out. Although there is an understanding suggesting the adoption of CI/CD in Sri Lanka, there is very less documented evidence providing us with any details on the success or failure of adoption, adoption rates and the positive/negative impact of CI/CD in the Sri Lankan software development industry. Moreover, the challenges associated with CI/CD adoption have not been studied within the context of Sri Lankan teams which would be the main objective of the research. Due to the above significance of the IT industry to Sri Lanka and the international attention Sri Lanka has been getting for skilled labor in the domain, the importance of statistical evidence to how well Sri Lanka can adopt the next step of CI/CD towards process optimization and how the challenges contribute to its progress is vital. The presence of this information can help the IT industry of Sri Lanka immensely.
Related literature review commonly mentions team factors such as communication, skill, collaboration and openness towards change, stakeholder buy in and support for CI/CD adoption and lastly the complexity of the existing systems and framework as the three main common challenges causing the biggest impact to the CI/CD adoption among international software companies and it was assumed the same impacts Sri Lankan teams as well in the same magnitude. The main hypothesis and the research question of this research was that the above challenges have the same impacts to the Sri Lankan context as well. However, the highly adaptive nature of agile altering the mindset and the processes of the teams to a more comfortable and adapted everyday process for a long time may or may not have altered the impact or the perception towards the impact within the Sri Lankan context which was expected to be found out through the study carried out.

D. Significance of the topic
Researching the Sri Lankan context, a gap was identified between the knowledge available to make IT process optimization decisions in the Sri Lankan market and opportunities available to do so hence the study intended to bridge that gap at a high level. This preliminary research is then expected to be the foundation of further academic studies to be carried out when CI/CD continues to establish more strongly in the Sri Lankan IT community. Since Sri Lankan IT companies are eager to gain process optimization through CI/CD, the findings will provide a solid starting point to evaluate where they are, identify the pros and cons of the process, remove blockers, and mitigate risks in the adoption and continuation efforts of CI/CD. The data set would provide insights to make informed decisions about all future CI/CD adoption efforts within the country and within other IT markets in the Asian regions sharing common mindset and process cultures.

A. Theoretical Framework Formulation
The literature review gathered empirical evidence on the factors impacting CI/CD adoption in the global IT context which then used for factor analysis. The empirical evidence thus gathered which formulated the basis for the correlations studied are shown in Figure 2. Based on the secondary data studied above, the analysis below was carried out to select the three top issues CI/CD faces in general towards a successful implementation and adoption. The data are shown in Figure 3. It was evident from the literature review that Team Dynamics, Stakeholder buy-in and Complexity were the main factors impacting the adoption of CI/CD. Due to the agility of the development environments and heavy adaptation capabilities of the methodologies itself, these challenges may/may not have an impact on the same research problem in the Sri Lankan context and hence the question was formulated.
How does Team factors, Stake holder Buy-In and Complexity impacts the adoption of CI/CD in Sri Lankan Software Engineering team? The literature review showed less evidence on CI/CD specific models to discuss the adoption and impacting factors when formulating the theoretical framework. There is less research and secondary data on software engineering frameworks which were found to be relevant to the topic in discussion as well hence the research problem was looked at in the perspective of general technology adoption.
The dependent variable for the research being "CI/CD adoption in Sri Lankan software teams", the topic was generalized considering it to be another technology adoption use case and similar research carried out were studied. Several models and frameworks have been developed and used in research to explain user adoption of new technologies as per H. Taherdoost [12], such as the Technology Acceptance Due to the reasons above on its ease of use and high applicability, the TAM model was concluded to be the ideal model to discuss and analyze the adoption of CI/CD in Sri Lanka. Based on the literature review carried out and the variables thus narrowed down for the study, the below theoretical framework was developed. The research onion model developed by Saunders et. al. [12], was used as the basis for the evaluation of available options in determining the research methodology throughout.

B. Research Instrument
Considering the confidence level to be 95% and the margin of error 6.5%, for the population of size 25,000, a sample size of 225 was considered (+ or -10 respondents). The questionnaire was initially shared with 80 individual professionals via LinkedIn network and was requested to be shared among their company employees, colleagues, and friends to create a snowball sample.

C. Hypoethesis Formulation
As per the empirical evidence above, below hypothesis were formulated to be tested and proven through primary data collection.
• H1 -Team factors have a significant positive impact on the adoption of CI/CD in Sri Lankan Software Engineering teams • H2 -Having stakeholder Buy-In has a significant positive impact on the adoption of CI/CD in Sri Lankan Software Engineering teams • H3 -Complexity in existing systems and processes has a significant negative impact on the adoption of CI/CD in Sri Lankan Software Engineering teams

A. Demographic Factor Analysis
Few slices of the data collected by factors are below.   The negative skewness of -0.095 showed positive correlation, determining team factors to be of a higher influencing factor towards CI/CD adoption. The kurtosis of -0.456 being negative is due to the flat nature of the distribution of the responses, although the mode is 4, the rest of the data seem to have spread throughout the range between 2 to 5 causing a negative kurtosis. The mode, 4 shows that most of the respondents picked 'Agree' on the statements.The mean value is 3.8554 averaging above the High indicator with a standard deviation of 0.606 which shows a slight deviation from the mean value. However, overall, the team factors are in the high category showing a positivity in the responses received thereby showing an impact on the CI/CD adoption positively. The negative skewness of -0.072 showed positive correlation, determining team factors to be of a higher influencing factor towards CI/CD adoption. The kurtosis of 0.129 being slightly positive due to the peaked nature of the distribution of the responses between the range 3-4, with the mode of 4. The mode, 4 shows that most of the respondents picked 'Agree' on the statements depicting the importance of it for the independent variable. However, the mean value is 3.6747 averaging only slightly above the High indicator with a standard deviation of 0.641 which shows a slight deviation from the mean value. Compared to the team factors, the respondents seem to consider team factors is of higher importance than stakeholder buy-in. However, overall, the stakeholder buy-in factor is in the high category showing a positivity in the responses received thereby showing an impact on the CI/CD adoption positively. The negative skewness of -0.805 showed plosive correlation, determining complexity to be of a higher influencing negative factor towards CI/CD adoption. The kurtosis of 0.962, being comparatively positive due to the peaked nature of the distribution of the responses between the range 3-5, with the mode of 4. The mode value being 4 shows that most of the respondents picked 'Agree' on the statements depicting the importance of it for the independent variable. However, the mean value is 3.9378 averaging slightly above the High indicator with a standard deviation of 0.681 which shows a slight deviation from the mean value. Compared to the team factors and stakeholder buy-in, the respondents seem to consider complexity is of the highest importance in the order followed by team factors and then lastly, the stakeholder buy-in. However, overall, the complexity factor is in the high category showing a positivity in the responses received thereby showing a negative impact on the CI/CD adoption as anticipated.

C. Hypothesis Test Results
The relationship and prediction between the dependent and independent variables can be expressed in a linear regression equation as below. Where, Y = dependent variable which is CI/CD Adoption b0 = intercept value b1 = slope coefficient value The relationship and prediction between the dependent and all independent variables which shows positive correlations can be expressed in a linear regression equation as below. Where, Y = dependent variable which is CI/CD Adoption b0 = intercept value b1..k = slope coefficient value for each dependent variable x1..k = dependent variables which are team factors and stakeholder buy-in with significant positive correlations • Survey with close ended questions limited clarity • Limited use of independent variableswould have been also possible to find out the critical impacting factors had all variables been studied.

A. Analysis of the research question
After the data analysis, it was concluded that hypotheses H10 and H20 can be rejected successfully hence proving the H1 and H2 hypotheses. The Pearson's correlation coefficients observed for both team factors and stakeholder buy in variables showed a moderately positive linear relationship with complexity showing a weak positive linear relationship. The null hypothesis for Complexity, H30 hence could not be fully rejected. The main observation in this research during the hypothesis testing was that as per the literature reviewed in formulating the hypotheses, there was a significant difference between the contributing factors and the CI/CD adoption observed when the analysis was carried within the Sri Lankan context.

1) Team Factors
Among the most important team factors impacting the CI/CD adoption, knowledge and competency in the critical tasks, specialization in the specific tasks such as DevOps, Test automation, communication, collaboration capabilities, teamwork, improved motivation, improved perception on ease of use and usefulness of CI/CD, adoptive nature for change and agility are most important factors to be working on in teams. As leaders or software engineers who are attempting either to adopt or implement CI/CD in the teams, these are some of the areas they can invest on first. With regards to Team factors, 63% of the respondents stated it has a higher or the highest impact on CI/CD adoption. The software engineers can focus on building on the above skills individually and managers can motivate the same to upskill in these areas. Since the perceived usefulness and ease of use play a significant role, it is also important that awareness on the advantages and long term benefits are commonly discussed among the teams to uplift motivation and gain common interest on the subject.

2) Stakeholder Buy-In
The support of the upper management, effective risk and challenge management, customer support on the new process, more responsibility and confidence on CI/CD shown by the stakeholders and adequate investments are the most important criteria identified as driving factors through this research. As per data collected, 61.1% of the respondents stated that stakeholder buy-in have a higher or the highest impact on CI/CD adoption. The software engineers can present articulated proposals when gaining for stakeholder buy-in showing the benefits and impacts CI/CD have on the current and future goals of the company. As the top management of the company, it is important to build a sense of trust and confidence in CI/CD for the teams to reflect on. It is also important to get the buy-in for the change of processes by the customers focusing on the benefits it brings such as quick delivery, faster feedback, better tested software etc. so that the adoption process can be seamless.

3) Complexity
Even-though the data analysis results were not strong enough to statistically prove the significance of complexity negatively impacting CI/CD adoption, 52.4% of the respondents stated that complexity has a higher or the highest impact on CI/CD adoption. Factors such as the complexity of existing systems, dependencies and conflicts, technical challenges are seen as the main areas the teams need to work on in order the adopt CI/CD successfully. The management • Maintain a pipeline audit log to track the progress and figure out the causes of failure, identifying, and redirecting the team responsible for fixing any CI/CD challenges or issues.
In conclusion, the impact of team factors, stakeholder buyin and complexity in the current context of Sri Lankan software engineering teams are heavily reliant on country/team specific factors and a globally adapted processes cannot be directly implemented in environments farther from the source. The research did find out significant correlation between Team factors and Stakeholder buy-In with CI/CD and further research is needed to derive more conclusive correlations between Complexity and CI/CD. As observed, CI/CD is a significant buzzword in the global context of software engineering right now and it is evident from the results that it has already found a place of significance in the Sri Lankan context well. It is further delightful to see the degree to which it is currently exiting among the teams hence it is hoped that this research and its findings will further strengthen the CI/CD implementation efforts by teams.

6) Directions for future research
Below recommendations can be given for future research.
• For future research, to carry out studies with alternative or more variables to arrive at more definite conclusions on the contributing factors.
• Use a mixed method choice for further studies where the responses received can be interpreted much better based on the demographic aspects of the respondents.
• Narrow down the size of the population as much as possible to be specific to an intended sample but however, go for a bigger sample of population • Instead of the sampling method used which was nonprobabilistic snowball, use a probabilistic sampling method to get a more realistic and unbiased slice of the population. The random chance in a population can work against the research objectives hence its best to avoid the same.
• Use more variables as explained above for the conceptual framework to formulate a stronger model.
• Focus on future studies covering not only the impact of the contributing factors but also the degree of impact which can help companies with limited resources with prioritization to realize the most critical elements to address first.