CI/CD for dummies
I realized some time ago that CI/CD (continuous integration/continuous delivery) is always explained for developers but never for those…
I realized some time ago that CI/CD (continuous integration/continuous delivery) is always explained for developers, but never for those people that are impacted by the way developers work (Product Owners, C’s levels, Customer Service, Clients).
Continuous Integration (CI)
Let’s use a metaphor to explain CI, imagine you have to work with some of your colleagues changing different parts of a big document, in that scenario you have several options to organize the work (there are much more but I’m going to focus on two):
Each person creates a local copy of the document. You have a meeting with them, split the work and later one person try to merge all the copies. (basically this is feature branches)
You create a Google Docs document and write things with others then you see the changes in real time, and you merge them immediately. (Continuous Integration)
You can imagine the kind of problems the group that has done copies and try to merge all of them at the end will have.
What happens if two of them change the same sentence?
What happens if they want to add the same section twice?
What happens if they want to change the initial plan because now has no sense again?
Their problems are all related to the time they are diverging their copies, more time without merging and having coordination (feedback) then more problems when they try to merge all the documents.
This is what Continuous Integration try to solve, the merge problem and code coherence.
The second option (Google Docs) has exactly the same problem trying to coordinate people, but this workflow does it while the document is written. Your coordination problems can be solved by chat because you see the problems, and you can solve conflicts with your colleagues during the document creation. In the long-lived copies (long-lived branches) the problem is that people don’t know they are diverging one from the other.
Now think in the psychological part of this, imagine this conversation:
Reviewer: “This is not what we have in our mind”
Writer: “No, but wait, I spend one week I have no more time”
Reviewer: “Sorry but ….”
At the end our writer will try to avoid doing any change because the only point to receive feedback is when he/she has finished.
In software changing the code to improve it (basically make it more readable or maintainable) is called refactor so in long-lived copies (long-lived branches) refactors are always the last part to do and usually everyone tries to avoid them.
The same problem in continuous integration is not so important because the writer has invested in that section one hour, it is not finished, but at that moment other writer has the opportunity to give him/her feedback about that section. The position of the writer will be a little different because it is not so much work to change, even you can ask for help to improve the section sooner.
This is another problem that tries to solve Continuous Integration, refactors are easily to be done and proposed. CI changes the environment from criticism and competition to learning and cooperation.
But you could say, wait if you don’t have a person in charge of merging the document (the reviewer) who will be in charge of verifying there are no errata or the document has sense?. We could consider an erratum like a bug in our code. Bugs or erratas can be found when reading the document (reviewing it). In continuous integration we could review little parts of the document when merged by the reviewer. Or writing the sections by pairs, so one writes and the other is reviewing (pair programming) this usually means no need to have a specific role of reviewer (that job is rotated inside the pair).
The style of our document can be improved by automatic tools that are executed frequently giving us feedback in near real time.
Other important thing is that writers cannot stop writing until things work. In Continuous Integration the document must be in a good state all the time, if the automating tools warns the writer that there is an error then it is very important to fix it immediately. It’s a bad idea to delay those fixes because in that way our team will have the broken window problem, feedback received or bugs found for the automatic tools must be fixed as soon as possible. This will reduce the cost of fixing bugs.
In feature branches the cost of fixing a bug is near the testing phase (when the reviewer is trying to fix everything) but in Continuous Integration this is inside the Development Phase so Continuous Integration is cheaper in that term.
As you can see everything is related to improve the way the coordination between writers is done using continuous feedback and automatizing as much as we can.
Continuous Delivery/Deployment (CD)
Let’s continue with our metaphor. CD is more related to the moment to show results to our boss (our clients, the users of our application). In the copies model we finished our document, and then we show it to our boss:
Our boss could say, this is great, it is exactly what I wanted to see. Answers all my questions, great.
Our boss could say, great, thanks, but perhaps you could change this or investigate more on this part, or I think this section is not working as i expected.
You will agree with me that it is more probable to receive feedback again by our boss or any other of our stakeholders.
At the end that feedback means extra work for the team. Think in the amount of time spent in the section that your boss wants to remove, in reality that means money lost. You have invested your time for something useless.
In Continuous Delivery you show your document to your boss when you decide but in the middle of the creation of the document. Any writer can show the document to his boss at any time because the document is always in a good state, it works. Perhaps the whole document is not finished (but a subset of the sections yes) then you have the opportunity to receive early feedback from your boss before finishing it, so you can change the direction of the document.
In Continuous Deployment your boss has access to the Google Docs all the time, but you have some mechanism to not show him some unfinished parts.
Continuous Deployment reduces at the minimum the lead time, usually lead time is a proxy of how difficult is to deploy your code to production. If you have a short lead time you can see results of your ideas earlier and take new directions if the results are not so good.
Again early feedback saves writers time so at the end means saves money for the company.
Techniques to hide sections in CI/CD
Branch by abstraction: This is a technique to have some sections of your document inside the document but not accessible by your readers (writers can see it and change it). Why this is interesting? Because writers have the ability to change a whole section of the document meanwhile readers are using the document but not affecting them. When the new section is ready and all the stakeholders are happy with it the old section can be substituted by the new one.
Feature toggles: This is basically the ability to show one new section (branch by abstraction) of the document to a subset of your readers to obtain feedback from them. This allows you multiple options like:
A/B testing, see if that section is well accepted for the readers to invest more time and money to improve it or not.
Bugs cost, the cost of a bug in production is also related to the time to fix it, when you have Feature toggles you can disable the new section if it is costing you money.
To illustrate this, let’s imagine that we can send a copy of the document to one of our stakeholders and other copy to another stakeholder. Each copy has a different version of one section, and we ask for feedback to our stakeholders about it. So we can evaluate using the feedback our stakeholders provide us if the new version is better than the old one. In that case we can put the new version and remove the old one, we create a hypothesis, and we try to validate it against our audience (hypothesis driven development).
Feature toggles or branch by abstraction follow the idea that creating something new is more secure than changing something old. Again here it is very important to remove the sections not shown when your tests has finished, and you know which path to follow. If you don’t remove that sections you will increase the complexity of your document though the time and can produce bugs at the end or extra work to maintain the document (technical debt).
The good point is that removing code under a feature toggle is usually an easy thing to do.
CI/CD saves money to your company.