Guide: Software Release Management For Small Teams
ormalizing The Release Management Process (If There’s Any)
In some team configurations, especially ones that are found in startups, there are no DevOps, nor infrastructure engineers, to provide support when releasing a new version of the product.
Moreover, unlike large bureaucratic companies with defined formal processes, the CTO or Head of Software Development team in a startup is often not aware of the complexities of the software release management process; a few developers in the company may be aware of the complex details of the process, but not everyone. If this knowledge is not documented thoroughly, I believe it could result in confusion.
In this article, I’ll try to provide some tips about how to formalize the release process, particularly from the developer’s point of view.
Enter The Software Release Checklist
You may be familiar with the idea of a checklist for some operations, as per the Checklist Manifesto, a book by Atul Gawande. I believe a formal release process (like many other tasks in the software development world) provides developers with an opportunity to implement this protocol. A release process checklist should be kept in a shared document, preferably in the form of a collaborative wiki, or a spreadsheet on Google Drive.
By sharing this vital document with the team, and by granting editing permissions, every member has access to the formally defined release process. This allows them to understand how the process works. Moreover, following discussions with other team members, it empowers the team to improve upon it every now and then. This should bring transparency and allow the entire team to have real-time access to what’s going on during the release, what steps have been completed, and by whom.
By looking at this spreadsheet, the stakeholders can decide on a ‘GO’ vs ‘NO GO,’ based on the outcome of the steps. For example, if a stress test goes wrong in a test environment, based on the event the project manager might decide to call off the production release.
Suggested Steps To Use As A Foundation
In this section, I’ll propose some steps you can use to build your own checklist for your release process. Some of these steps are by no means mandatory. Each app is different and each organization works in a different way, so feel free to adapt and make changes that will fit better into your workflow.
1. Create A Release Branch
It’s likely that you are familiar with the concept of Git Workflow, or with the idea of release branches, a topic that has been explained in a previous blog post.
Ideally, you should have at least three branches:
- master: this should reflect the current state of what is in the production environment. Every new commit on master should comprise a new release only.
- develop: this branch should contain the completed (and tested) upcoming features. It is common to create a separate branch for each feature and then merge it to develop when the feature is ready.
- release: release branches are a collection of commits that are ready to be sent to production, plus some additional minor bug fixes related to the release.
Notice the release branches should be removed once the release is completed, therefore these branches are created and destroyed all the time, unlike master or develop, which are always the same.
In order to create a new release branch, from the develop branch in your git terminal, type:
$ git checkout -b release/x.y.z
It is convenient to use a naming convention, such as the one defined above, replacing x.y.z by the major.minor.patch version number according to your needs (it is a policy that you should define within your team and stick to it).
It is important to say, as well, that if you code some bug fixes into the release branch, you should not forget to merge them back to develop. The main purpose of the release branch is to have a preview snapshot of how the app should behave after it goes into production.
2. Bump Version
The next step would be to bump (modify or increase) the version number on the release branch.
You should open the AndroidManifest.xml / package.json / pom.xml / or wherever the app’s version is stored in your project (YMMV), update the version number, and then commit the changes to the current releasebranch.
It is important to update the version number for two reasons.
First, you can track and map the features that were introduced in each version, and second, you will be aware of the version they are using should they need to do some troubleshooting or to contact you for support. If you are building a mobile app, the version number you are updating in this step is usually displayed on the user end, in the About section or in Google Play or Apple App Store.This step is also a good opportunity to update environment-dependent-configuration files (although I’d suggest keeping those in a separate repository), such as making the branch point to the production database, or any other tweaks needed on the build process.
Finally, it is recommended that you push the release branch to origin, so it is available for your other developers:
$ git push -u origin release/x.y.z
3a. Merge release branch to master and tag it
Only the master branch should be deployed for production, thus in this step, we need to merge the releasebranch into master.
$ git checkout master
$ git merge --no-ff release/x.y.z
$ git push
The --no-ff flag is optional, however, its use is recommended in order to force the creation of a new commit object, even though the merge can be completed using the fast-forward technique.
Next, it is time to tag the release on master branch:
$ git tag -a x.y.z -m 'description of new version, features or fixes included'
Tags are useful because you are persisting this specific point in history in the git repository, and you can come back later to recreate a separate branch off a particular tag.
3b. Use a pull request to merge the release branch
Another alternative used often, is to use a pull request to merge the release branch into master.
There are numerous advantages to this approach. A new space is created for collaboration, which the team can use to discuss various release-related issues. This point is a good time to add an extra gate for incorporating a code review process, while having more eyeballs to monitor the code that will be introduced, and to discuss potential modifications.
Some tools that allow you to implement pull requests into your workflows are GitHub, Bitbucket and GitLab(merge request as they call it on the latter). With these tools, you don’t type the git commands manually, instead, you use a web interface to set the source branch (release) and the destination branch (master), and then you add one or more reviewers, all of whom will be able to write inline comments on these new changes, suggest improvements, and so on.
After all the reviewers approve the pull request, you can automatically merge the changes into master by pressing a button on the UI.
4. Deploy Master To Production Environment
It is good practice, at this stage, to have a tester on your team do a smoke test (this could be defined on a separate checklist) before deploying the app. A good suggestion is to deploy the master branch into a separate testing environment as done by professional automation testing companies. The tester can then perform some basic actions to make sure nothing went wrong after the merge on the latest build. How to conduct a smoke test is beyond the scope of this article, but you can find a lot of material on the web about it. The result of the smoke test can be integrated into the release checklist/spreadsheet, documenting anything that went wrong.
At this point, you are ready to deploy the changes and make them live. Go ahead and deploy master branch. This step will depend on the infrastructure stack you are using. It might involve connecting to your Amazon EC2 instance to upload your app, or pushing to a Heroku remote, or connecting via ssh to your VPS to copy the new version, or any other process.
After the app is uploaded, make sure it was deployed successfully and that it works as expected.
5. Merge Back Into Develop And Delete Release Branch
Now the release is almost completed, you’ll want to merge the release branch into develop, to update the version number on the latter and to transfer all the bug fixes made to the main development branch:
$ git checkout develop
$ git merge release/x.y.z
Now it is time to remove the release branch:
$ git branch -d release/x.y.z
6. Changelog Generation
There should be a file at the root of your project named CHANGELOG.md (or an equivalent) where you should add a new entry whenever there is a new release in order to document everything that’s included in it, like bug fixes, new features, known issues, and any other relevant information in the form of release notes. This is really useful for users and contributors to see what changes have been made between each release (or version) of the project.
The changelog entry includes the date, version number and some notes about the release. The entries should be kept in reverse chronological order. Here’s a simple template I’ve been using that you can adapt to your project:
<app's name or component released> <version x.y.z> | <date>
<developer's name in charge of release> | <developer's email>
Features:
* <ticket/issue number>: <ticket/issue summary> (<link to issue tracker>)
* ...
Bugs fixed:
* <ticket/issue number>: <ticket/issue summary> (<link to issue tracker>)
* ...
Enhancements:
* <ticket/issue number>: <ticket/issue summary> (<link to issue tracker>)
* ...
Optional: known issues plus other release notes.
In addition, this step can be completely automated by either coding a basic script that traverses the git log and automatically generates the changelog entry, or by using a tool that does the same, such as:
- Github Changelog Generator, by skywinder,
- ReadmeGen by fojuth
- github-changes, by lalitkapoor
Keep in mind though, that the degree of automation you get is directly proportional to the strictness of your commit message format. I believe it is always good practice to agree on a specific format for commit messages with the team. By following guidelines on the style of the commit messages, they will be easier to parse and hence more likely that you will be able to automate the generation of the changelog.
7. Communicate With Stakeholders
This is where you would usually do some of the following:
- Let the team know via an internal messaging tool (e.g.: Slack) that a new release has been completed. I recommend creating a new channel (i.e.: #releases) for the sole purpose of communicating release related events. It is easy to add hooks in a Slack channel to post a message after an action has been taken.
- Alternatively, send an email to the stakeholders with a link to the changelog, or the changelog file attached.
- Write a blogpost (if you have a blog for your app or product) or a tweet.
More actions can be taken depending on the nature of your organization. The important thing is not forgetting to communicate that a new version of your product is available.
8. Grooming The Issue Tracker
After a release is executed, you will probably need to update the status of some of your tickets to keep track of the bug fixes, or features, currently in production. Generally, this involves changing some tags (for small projects I use a release-pending tag, which I remove after the release is completed).
If you use milestones for each new version you’ll probably need to update their status or mark them as completed. Some issue trackers even let you plan the release and align it with sprints, track whether a bug is blocking a release, and other useful information.
It all depends on how you use the tool. I simply want to point out that the task of updating the information in the issue tracker should be included in your release checklist.
About Automating The Release Process
The reader might have noticed that, apart from the changelog step outlined above, many of the aforementioned steps can be automated, too.
The ability to automate some parts of the release process is a huge win and saves a lot of time. I suggest creating scripts, or figuring out how to automate individual steps and then work towards a continuous deliverygoal. Continuous delivery can reduce risk, reduce costs, and reduce the time developers need to spend in managing the release. Consequently, you’ll be able to release more often and be more productive in terms of the hours allocated for development.
The holy grail of DevOps is to be able to launch a new version by pressing a button (or running a command) that would trigger the release process automatically, or even better, a system that would release a new version of your software at a designated time. Of course, this is difficult to achieve because you also need to automate a lot of the testing process, but it is not impossible.
Embracing Best Practices
In this section I’ll describe a couple of recommended practices that I’ve found convenient, either to make the release process smoother or for taking safety measures in case something goes wrong.
Find the most suitable day to carry out the release
I usually release apps I am working on on Thursdays, between noon and close of business.
If you work Monday to Friday, it is not a good idea to launch on a Friday. If something breaks down after the release, you won’t have time to fix it until Monday (unless you want to work during the weekend). That’s why it is more convenient to do releases on Thursday, because you have Friday to monitor the new version after being deployed, fix any problems, or do a rollback.
Another important thing to mention if you are managing a web app, is to know the time zone the majority of your users are located. You should time the release during a low traffic period to minimize the potential damage if something fails. Sometimes, this can be tricky when your user-base is spread across the whole world, but anyway, you should do some research and decide on the best time.
Backup Your Database Before A New Release
If you don’t have periodical backups of your DB scheduled, I strongly suggest you add a step into your release process to perform a backup before starting the release.
Staged Rollouts
Ever wonder why, when a publisher announces they’ve launched a new feature, it takes days, or even weeks, for that feature to be available on your phone? That’s because many companies use staged rollouts.
Facebook has been doing this for a long time. It tests a new feature on five or 10 percent of its users, gradually increasing it until they reach 100 percent of the user-base. During the staged rollout phase, you’ll need to look closely at user feedback and crash reports. With this information, you can then postpone the release, or fix errors, before they affect every user.
There’s a nice tool on the Android’s Developer Console that implements staged rollouts for your Android apps.
Continuous Integration
Continuous Integration is a practice worth embracing for many reasons. Firstly, it allows you to detect mistakes early, increasing the rate of successful releases. Secondly, it is the first logical step before implementing Continuous Delivery and full automation as previously described.
Martin Fowler is a big advocate of Continuous Integration. He describes a huge amount of benefits that can be added to your release process when using this practice. This is a large topic and there are many books and blog posts about it, and I am mentioning it here because I believe it will give you much more confidence in your operations. Amongst the many advantages of using CI, you will find: reduced risk, increased visibility to be aware what is working and what not, earlier detection of bugs, increased frequency of deployments, and many more.
The starting point to implementing CI is to set up a “continuous integration server;” some nice tools to try are: CruiseControl, Bamboo, Jenkins and Travis.
Exitlude: It Will All Work Out
Aggressively, we all defend the role we play Regrettably, times come to send you on your way We’ve seen it all, bonfires of trust, flash floods of pain It doesn’t really matter, don’t you worry, it’ll all work out
Exitlude, The Killers
To wrap up, I would say that it’s very important to have a well-defined release process for your app, regardless of its complexity, userbase, or how small your organization is.
If you don’t, I suggest you start thinking about some basic steps, using this guide and others like it, and brainstorming with your team to come up with a first draft. Try it out on your next release, then iterate. Eventually, you’ll end up building your own release process.
After that, start thinking about how to automate parts of the process. Think about areas that can be improved. Explore ways of reducing the release time by incorporating small optimizations. Automation should be your ultimate goal; however, don’t plan that from the start, or you will fail by attempting such a big leap. As with with every process, it is better to improve it gradually.
Do you have any other tricks or guidelines you use for launching new versions of your app? If so, let me know. I don’t come from the DevOps world, I am just a developer who happens to be quite organized and structured, but compared to many veterans, I am still a novice in this respect, so feel free to comment or contact me if you have something to add.
This post originally appeared in the Toptal Engineering blog
