Dealing with large files in Git

@git

I'm making this note because I keep repeating the same mistake: committing a file that is too big (>100Mb) and then scratching my head how to take it out of the history again. (Where "too big" means that we can't push to github etc.)

Committed a big file, not pushed yet

git revert is useless in this scenario, because it will make a new commit to revert the changes. This means our big file will still be in our commit history, blocking the push.

Instead we want to reset our repository to a state where we did not yet make the erroneous commit. So we can use git reset. git reset is now replaced with git restore, so we use that instead. So the big difference with git revert is that git restore does not make a commit. It simply shifts the HEAD pointer of your repository.

I'm assuming HEAD is the commit with the big file, i.e. we realized our mistake immediately after committing. Then we can reset the github repo to a state just before that commit. A shorthand for this is appending ^. So we can do:

git reset --soft HEAD^

I used --soft because I'm a chicken!

--soft doesn't touch both the index and work tree, meaning that this assumes all your done work was actually correct. As a result, all work will appear as staged i.e. ready to be committed. We now explicitly need to unstage the big file we do not want to exclude in the commit.

I guess that we should just do git reset HEAD^ instead and then commit everything we do want. (Didn't try this yet!)

In any case, be wary of --hard because this also resets your working tree to a previous state, meaning all your work since then will be permanently deleted!

You can also set only the file specifically, but when I tried this the big file showed up as "deleted". Since I do not want to commit this deletion (it will again be in the history), I figured this approach did not work here.

Faulty commit is n commits back

I do not yet know what's the most gracious solution here. I figure you'd have to repeat the commits you made manually after resetting.

I recall having found a perfect page explaining these scenarios, but I just can't find it again. This answer is quite nice though.

After resetting the repo and making a new commit, the erroneous commit is not accessible anymore but is still there. This is because we just shifted the pointer of head, rather than deleting a commit or making a new one with a reverting change before shifting HEAD.

We should be able to use git reflog to trace back commit that were still ok and recommit them in the current branch. I didn't do this yet; but this is for future reference.

Staged a big file, but not committed yet

Hallelujah. Just unstage the file.

git restore --staged ./thisismyfile.huge

Other

In my use case, I do not worry about removing files on the remote repo, because my issue was that I could not push.