Git workflows
@git @workflow
Other notes:
Introduction
A Git workflow is about how you use and integrate Git into your development work as a team. Ask yourself these questions when determining which flow is appropriate to you:
- Who do we trust?
- Where lies the responsibility? i.e. who has to account for faults?
- Do you want power and responsibility for yourself as a normal developer?
- Are we working on features together or alone?
- How big are features? Do we really need multiple branches per feature?
- Do we have a main-framework that needs to stay up to date, or is the project going to be highly modular?
- Do we release code to the public?
- Do we need continuous deployment? From main or a separate branch?
- How do we balance development speed and verification?
- Is there hierarchy within the team and does the workflow need to reflect this?
TODO
- Do we want to squash commit before merging into main?
- Con: not a fair view of what commits were actually made
- Pro: this ensures that the history of main tells a clear story (e.g. "add feature x" or "fix issue X") and is not polluted by 10 "fix typo" commits.
Git Best Practices
- Each commit should contain an isolated, complete and meaningful change.
- If you change makes two conceptually different edits, make two commits.
- This makes it easy to revert your changes to a checkpoint if you decide to take a different approach.
- Commit messages should reflect the intention of the code (what does it do?)
- If appropriate, mention linked issues
- Push regularly to named branches, this effectively back ups your local work remotely
- This way,
git fetchwill effectively serve as an up to date TODO list
- This way,
- Many flows use an upstream first policy: If you need to do bug fixes, do them in
mainand cherry-pick them into the release/production branch- This avoids mistakes where you forget to pull changes made in your release branch back into main, in which case you will encounter the same bug again for the next release
- Do not push large files into the repository
- This can cause time-outs on important operations like cloning
- Be aware of pushing data sets into a repository
- Use a branch naming convention
- Link issues to pull requests
- Only use rebase if you really know what you are doing
Types of Git workflows
The following flows are organized (more or less) from lean to heavy.
- Centralized: main only
- Trunk-based Development
- Feature Branches: main + branching for features and subfeatures
- GitHub Flow essentially implements feature branching
- Pretty much assumes continuous delivery nowadays
- GitHub Flow vs. Gitflow
- Atlassian Simple Git workflow: similar to Github Flow / feature branching, but rebase feature branches instead of merging them.
- Release Flow
- Gitlab Flow
- Gitflow Workflow: main + develop + feature branching + release + hotfix
- main: for releases
- develop: stable and complete features are merged in
- supporting branches for features, releases and hotfixes
- Forking workflow
Centralized: main only
- One central repository without branches
- Suitable for single developer and very simple projects, e.g. a small private website
- If one feature is done, but another feature is half done, you cannot release the code in the main branch to the public.
Trunk-based Development
- The whole development team collaborates on a single branch called the "trunk" (in Git: the
mainbranch) - This method strongly advocates against making other long-lived branches and committing as much as possible to the main branch (e.g. at least once per day per developer)
- So no release branch
- So no develop branch
- This can only be done by a very strong emphasis on continuous integration, involving testing, pre-integration checks, etc.
- Everything said here assumes these steps and best practices are in place.
- This development style is thus a key enabler of continuous integration: it's a must to get it right.
- All these additional practices differentiate this method from the simple "centralized" workflow.
- Very small teams can commit directly to the main trunk
- Requires local pre-integration testing
- Otherwise you can do a scaled trunk workflow with short-lived feature branches
- Short-lived means: one person doing a feature that is finished and merged in the trunk within a couple of days max
- PR-centric: uses pull-request style code-review & build automation to make merging flow smoothly
- This is all focused on avoid merge hell, which occurs more easily with long-lived feature branches
- But the trade-off is that this requires that PRs are handles very quickly, otherwise PRs for many small features become a deadlock
Notes:
- A core element of the scaled flow is that individual developers are active concurrently in short-lived branches, not groups of developers in longer lived feature branches with subbranches (e.g. as in Gitflow)
- PR-centric Trunk-based development is almost the same as GitHub Flow (see below), but in GitHub Flow you define releases from a feature branch (small chance of regression, see here) whereas in trunk-based dev you only release from the main trunk.
- NOTE I could not verify this claim by reading the GitHub Flow page.
- No commit pushed to trunk should jeopardize the ability to go live at any moment
- Short-lived feature branches are used for code-review and build checking (CI)
- By keeping feature branches short lived you stimulate a continuous code review ethos
- For changes that require longer effort, techniques such as branch by abstraction are required
- A build server is required to make sure commits do not break the build.
- Not breaking the build is a golden rule
Branched-based workflow: GitHub Flow, Atlassian Simple Git
- Make a new branch from main for each feature or bug fix and give it a descriptive name
- If necessary, branch from feature branches to develop subfeatures
- GitHub Flow and Simple Git are lightweight and only discuss branching from main, not from feature branches.
- Trunk-based Development strongly advocates against extra branching like subfeature branches
- Merge subfeatures into their parent feature branches
- During development, keep your branch updated with main by fetching changes from origin
- GitHub Flow uses merging (default behavior):
git pull origin main(N.B.pullcombinesfetchandmergeinto a single command) - Simple Git workflow rebases commits on top of
main:git fetch originfollowed bygit rebase origin/main- Understand the caveats of rebasing first!
- Do not rebase main onto your feature branch! E.g. see here
- Never rebase commits made by others; this also loses authorship attribution with
git blame
- GitHub Flow uses merging (default behavior):
- Only merge after a PR review
- The Simple Git workflow recommends a final rebase to clean up the history, but only after the review and not during development
- I think this should be an interactive rebase with
git rebase -i
- I think this should be an interactive rebase with
- The Simple Git workflow also recommends using an explicit non-fast forward merge of your branch work into the main branch:
git checkout main;git pull origin main;git merge --no-ff yourbranchname- This strategy always makes a separate commit merge even if the merge itself has no content
- This way the merge will be a marker that preserves the context of the feature branch even if no changes were made in the merge
- This means that you can easily revert a merge and all its commits.
- Merge requests on GitLab and GitHub automatically use this strategy, but if you merge manually you have to specify this
- The Simple Git workflow recommends a final rebase to clean up the history, but only after the review and not during development
- The only hard rule: anything in the master branch is always deployable
- No separate deploy branch is used, this means you always have to pass tests and check the code is buildable
- Important in GitHub Flow: deploy to production after review but before merging to really make sure the build is not broken
- This strategy may scale poorly to large teams because many devs may try to deploy changes to production at the same time; this becomes a bottleneck for merging
- If this is an issue, check out Release Flow.
Remarks:
- Is lightweight and compatible with PR-centric scaled Trunk-based development
- Supports development and testing of features independent from other features
- This workflow assumes there is no strict hierarchy in the team
- This workflow works well with continuous delivery because it is quite simple and does not require extra release or hotfix branches.
- but there are some scenarios in which continuous delivery is not appropriate, e.g. when you have deploy windows, but where you do want to be able to merge code
- If you use rebasing, you have to make sure that nobody else is pushing to your branch. If they are, you also need to rebase on the version of your branch that exists the remote (and has been pushed to in the meantime)
- I.e. after
git rebase origin/mainyou have to also dogit rebase origin/yourbranchname.
- I.e. after
Release Flow
- Trunk-based development like GitHub Flow, but the main difference is that the main branch is not continuously deployed
- In Release Flow you do not continuously deploy like in GitHub Flow
- In GitHub flow you need to build in production before merging, but this can become a bottleneck at scale
- Instead, make a release branch for each release, e.g. after each sprint
- Hotfixes are applied in
mainand then cherry-picked into a release branch
Gitflow
- There are two long-running primary branches,
mainfor releases anddevelopfor merging in stable and completed features developbranches frommain- Feature branches branch from
develop - Once a feature is done, merge into
develop - Once all features for a release cycle or sprint are done, make a new
releasebranch with a version tag- you can optionally publish this somewhere as a release candidate
- the idea is to verify here that this release is well tested and stable
- If necessary perform bug fixes on the
releasebranch. Pull fixes intodevelopand feature branches. - Once verified that the
releaseis stable, mergereleaseintomaster. - If a problem still persists or occurs in production code, you can apply a hotfix by branching from
mainand patchingmainby pulling in the hotfix.- Also pull in the hotfix into
develop
- Also pull in the hotfix into
Visual: The gitflow workflow in less than 5 min.
Notes:
- Con:
developis the default branch for development, but by defaultmainis the default branch (also for tools) - Con: the hotfix and release branches introduce complexity which may be unnecessary for most modern projects
- These manual steps are a cause of mistakes, e.g. merging in main but forgetting to merge into develop
- Unnecessary nowadays due to continuous integration/continuous delivery (CI/CD)!
- Con: if you deploy very often, going through the ceremony of the
releasebranch is restrictive - Pro: you may need something like Gitflow to distinguish between a deployment and development branch if you do releases on a long interval (weeks to months between releases), while still being able to do things like hot-fixes.
- When you deploy all the time, there is no need for a separate hotfix-branch because your normal process can already get commits into the main trunk in no time.
Gitlab Flow
This flow can be seen as the lightweight GitHub flow at base and taking some of the ideas from Gitflow without the extra fuss.
GitLab Flow combines feature-driven development and feature branches with best practices for issue tracking.
What's a bit confusing is that there does not seem to be "one" GitLab flow. Instead, the documentation page describes different variants and options. GitLab flow is thus quite flexible and incorporates ideas from GitHub flow, Release Flow and others.
Some main points:
mainis the main trunk, there is no defaultdevelopbranch as in GitFlow- use feature branching from
main - There can be other stable branches, which are all optional depending on your needs:
productionbranch if you need a separate branch for deployingreleasebranch if you need a public code release to the public (this allows exposing older versions too, like v1, v2 etc.)- you may make environment branches (e.g. preproduction), but the rule here is that commits flow downstream
- All merge requests should be linked to issues
Production branch?
- If needed, you can make a separate
productionbranch where code is deployed from (can be main?)- New versions can be deployed by merging
developinto the production branch (I thought) N.B. I thought there was no develop branch? Is this not just main? - No development is being done in the production branch
- Deployment time is approximately the merge commit time to the
productionbranch- Alternatively, let the deployment script automatically tag on deployment
- This saves the overhead of releasing, tagging, merging of Gitflow
- GitHub flow assumes you can deploy to production every time you merge a feature branch, but this is not always appropriate
- E.g. sometimes you want to merge code in your development branch without it being deployed (e.g. because it's outside a deployment window)
- the time of the merge commit is approximately the deploy time if you use continuous delivery (saves overhead!)
- New versions can be deployed by merging
Environment branches for pre-production
- If you have different branches for different environments, e.g.
staging,pre-productionandproduction, only merge in such a way that commits flow downstream, i.e. from staging to pre-production to production.
Release branches
- Only work with release branches if you release software to the outside world (rather than a deployment only)
- Each branch should contain a minor version, e.g.
2.3-stableor2.4-stable - Branch release branches as late from
mainas possible
- Each branch should contain a minor version, e.g.
- Upstream first policy: If you need to do bug fixes, do them in
mainand cherry-pick them into the release branch- Do do not fix bugs/work in the release branch; if you do that, you might forget to cherry pick the commits back into
mainand stumble upon the same bug later. - Increase the patch version number, cf. Semantic Versioning
- Do do not fix bugs/work in the release branch; if you do that, you might forget to cherry pick the commits back into
Pull request, merge request, merging feature branches
- Terminology:
- GitHub and Bitbucket use the term "pull request" because the first manual action is to pull the feature branch
- GitLab uses the term "merge request" because the final action is to merge the feature branch
- If working on a feature branch for more than a few hours, allow others to review your work in progress by opening a merge request without assigning it to anyone
- You can request comments by tagging people in the description or a comment
- Assign the merge request to the person who knows most about this part of the codebase
- GitLab protects the long-lived branches such as
main- So when merging into a protection branch, request review from someone with enough rights (the GitLab doc mentions the "Maintainer role")
- Remove feature branches after merging
- Unassigned merge requests are drafts. Name them as such:
[Draft]orDraft:etc. - Merge requests in GitLab always use the
no fast-forwardstrategy: this means that a separate commit is always made for merges, even when that is not necessary. - Avoid many merge commits in feature branches because this makes the history very messy
- Strategy 1: You can avoid this by rebasing their commits on top of the main branch (not the other way around!)
- Only rebase if you are the only person working on the feature branch
- Rebasing may create more work because you may need to keep resolving the same conflicts
- Strategy 2: Do not frequently merge
maininto the feature branch- GitLab prefers cherry-picking commits from main in the feature branch
- If there's a conflict, a merge commit is the default way of handling this
- Strategy 3: keep feature branches short-lived
- Strategy 1: You can avoid this by rebasing their commits on top of the main branch (not the other way around!)
Issue tracking
- Any significant change should start with an issue in the issue tracking system (significant? more than an hours work)
- Issues should describe the desired state of the system
- "As an admin, I want to remove users without receiving an error"
- This is pretty much a user story!
- Create a branch from
mainfor the issue - Reference issues in merge requests or commit messages
- In GitLab you can link and automatically close issues when a merge request is solved by mentioning
fixes #16orcloses #67.
Forking Flow
In open source projects, there's typically a select number of persons choosing the direction of a project (a BDFL). You always have the option to fork the entire repository and make changes as you please.
- Instead of merge requests, you can make a pull request to integrate the fork into the forked repository
- Alternatively, you can take the code base in a completely new direction and spin-off to make a project of your own.
- You still have the option to pull in changes from the forked repository
Example: I maintain a fork of dwm where several patches are applied to the source code.
I use the main branch to pull in changes from the forked repository and a patched branch where the patched version of the source code lives.
I then have the option to merge main into patched.
Note that this implies that main and patched have different upstream repositories (one the original dwm, the other my own forked repo).
Hierarchy and Merge Requests
If your team has a hierarchy, there are two ways in which this can be consolidated in a workflow:
- Push permissions
- Restrict pushing to certain branches, e.g. based on seniority
- Merge request
- Ask to verify made changes to a more senior developer to merge into push-protected branches.
The feature branch workflow for example assumes that there is no strict hierarchy within the development team and that all members have comparable knowledge.
In the Gitflow workflow you can lock the main branch and possibly the develop branch too, so that more senior members are responsible for preparing release candidates and final releases.