Let’s CI/CD! How Agile x CI/CD Complement Each Other, Increasing Efficiency by Over 98%!
In the era of VUCA (volatility, uncertainty, complexity, and ambiguity), the field of software development is facing unprecedented challenges. Ambiguous requirements and technical complexity have become part of daily work. In order to overcome these highly challenging tasks, many software startups have embraced Agile, hoping to learn the formula for success through the process of rapid trial and error by delivering customer value incrementally over short cycles.
Traditional Dilemmas in Software Development
Taking the development of an e-commerce website as an example, under the traditional waterfall development model, an e-commerce website that includes searching for products, product introduction pages, adding to shopping cart, payment processing, order processing, and integration with logistics systems, assuming it takes one year to develop, at the initial stage, the team may spend two to three months on detailed planning and requirement gathering, and transform these requirements into clear specification documents for the development team to implement. The following five months will be mainly used for development work, then QA testing. During the testing process, hundreds or even thousands of issues may be found, requiring repeated corrections, which could consume more than a month. When the project reaches the tenth month, there may be a preliminary version for the customer to evaluate, at which point the customer may frown and say: “This website is different from what I initially thought.” or “Can I add some more features?”
In such a situation, if the development team is told to make changes or add features, they usually look troubled and feel overwhelmed, sometimes even needing to say: “This requires changing the underlying architecture, it might take another three months!” Of course, spending another three months would delay the project, so in the end, it’s the PM who communicates and coordinates with the customer, promising to gradually perfect these functions during the maintenance period of the next year, and reluctantly launching a version that the customer is not very satisfied with, and the development team has to spend a lot of time on additional patches, even if some progress is made, it comes at the cost of dual harm to both oneself and the customer.
Why Embrace Agile?
The concept of Agile development is very simple: cut the original one-year cycle into 24 segments, each segment being two weeks, and within two weeks complete planning, requirement gathering, analysis, design, development, testing, and then let the customer review and accept the deployment. Some people might think, if it can’t be done in a year, it would be best if it could be done in two weeks!
Dividing a large system into multiple small manageable modules or units can more effectively achieve rapid incremental delivery of customer value.
For example, if the goal for two weeks is only to do a good job on the product page, which includes the product image in the upper left corner, the product name and simple specifications in the upper right corner, and the product introduction and detailed specifications below, without doing the shopping cart, inventory system, or payment processing. In such a case, can it be completed in two weeks? The answer is of course yes! Subsequently, every two-week iteration will gradually add inventory systems, shopping carts, payment processing, etc., and each iteration will have customer review and feedback. This approach not only accelerates development progress but also provides a faster ability to respond to changes. Even if the customer wants revisions, since these features at most took two weeks to develop, adjustments can be made quickly. Even if the output does not meet expectations and needs to be cut, at most only two weeks of time and cost are wasted, still within an acceptable range.
Combining Agile with CI/CD
Agile development is good, but in practice, it still faces some challenges. For example, our team usually schedules a review meeting on the last day of every two-week iteration for the customer to accept and provide feedback on completed functions. However, functions that can enter the review meeting for acceptance must pass testing by QA and the Product Owner (PO) and be confirmed error-free before they can be presented at the review meeting. Therefore, it’s necessary to complete development and deploy to the testing environment at least two days before the meeting to allow QA and PO to conduct tests. However, there’s a problem here: the development team typically needs to spend about six hours on code building and deployment, which almost occupies a whole eight-hour workday.
That is to say, only one deployment can be done per day. If errors are found during testing, correction and redeployment have to wait until the next day, which delays the testing progress. This greatly limits our ability to correct errors at the end of the iteration. We only have one chance to fail, and if we fail a second time, we can say goodbye to the customer at Friday’s review meeting and try again next time.
To solve this problem, our development team introduced a CI/CD Pipeline. CI/CD stands for Continuous Integration and Continuous Deployment, a very important practice in modern software development, with common CI/CD tools like Jenkins, GitLab DevSecOps, and AWS CodePipeline, etc. CI is when RD submits code to repositories like GitHub, GitLab, or AWS CodeCommit, a series of tests are automatically run to ensure code changes do not break the product’s existing functionality. This practice helps discover problems early and improves code quality. CD follows CI, ensuring that once code passes tests and integrates, these changes can be quickly and safely deployed to testing or production environments.
The Value and Benefits of CI/CD
Whether it’s CI or CD, both require automation processes to reduce human errors and speed up delivery. Automation means that developers do not need to monitor and input commands tediously after submitting code, saving a lot of time. Apart from freeing up hands for more tasks, it also reduces the integration, building, and deployment time from 6 hours to just 6 minutes, improving efficiency by over 98%! This change means that the restriction of only being able to deploy once a day is now multiple deployments, greatly reducing the risk of not being able to showcase functions at the review meeting due to errors.
Furthermore, introducing CI/CD not only enhances development efficiency, reduces errors, and deployment difficulties but also helps shorten the time from conception to development. Under the backdrop of DevSecOps, the practice of CI/CD further enhances security, as security testing, reports, and review processes can be integrated into the entire development and deployment pipeline, further strengthening the overall security of the product.
In today’s VUCA era, the software development industry is undergoing a revolution. From traditional waterfall development to Agile development, and even the hybrid development that combines the two, we have witnessed how adaptability addresses the constantly changing market demands and technical challenges. Agile development not only mitigates the issues of demand and technical complexity but also makes the development process a close collaboration with customers, continuous feedback. Meanwhile, the introduction of CI/CD further optimizes this process, enabling development teams to respond to changes more quickly and efficiently, while improving product quality and security. This series of changes not only demonstrates the innovative capability of the software development field but also provides valuable experience for other industries in dealing with rapidly changing environments. With the continuous development of technology, I believe that the software development field will become even more efficient and secure.
