Tuesday, September 15, 2015

Git in fifty-odd (was thirty-odd) questions

Moving to git from another version control system like SVN, Perforce or CVS often represents teething pains that don't go away as soon as you'd like them to. But git is a fabulous tool and learning it is worth your while. The following FAQ tries to address frequently encountered scenarios that programmers face when moving to git. It assumes that you understand what a version control system is and have some experience using another system like SVN, Perforce or CVS.

In the Unix tradition, this FAQ is terse rather than elaborate. We don't do visualization of branches and branching models, etc. here. There are better tutorials dedicated to them. The expectation is that in this FAQ you will find answers to the most frequent scenarios that you encounter and know which git command to use. And you should be able to cobble together these commands to solve other scenarios that can be decomposed into these.

  1. Why is Git so complex?
    Two reasons for it:
    a. Git is completely distributed - so each developer has a copy of the entire repository and is responsible for keeping this in sync with other repositories.
    b. As a consequence of #a, operations in Git do not easily map to operations in CVS or Perforce or other popular version control systems.

  2. How do I start on a new project?
    The most common way is to clone a remote repository. Git supports http, ssh, etc. You need a git URL for a remote git repository and a local directory in which you want to create the cloned workspace. You can then issue the following command:

    $ cd local_dir
    $ git clone git_url

    This remote repository can be any repository. Usually there is a designated central repository shared by all members of a development team. But it could as well be another clone of that repository hosted on a different machine.

    The remote repository from which the local repository is cloned is conventionally called origin.

    $ git config --global user.name "Your Name"
    $ git config --global user.email "your@email.com"
    $ git config --global core.editor "path-to-your-fav-editor"

  3. How do I checkout a file if I need to change it in my workspace?
    There is no concept of checking out a file. You cannot lock any files for edit. Just edit it in your workspace.

  4. How do I keep track of which files I've modified?
    In your workspace, run the following command.

    $ git status

    It will list files that have been modified, as well as new files that have been created in your workspace.

  5. What is the index and what is meant by staging files?
    When you add a new file, it is an untracked file. When you modify an existing file, it is a modified file. To indicate your intent of committing them, you need to stage them. You stage a file by running:

    $ git add file_or_dir

    The index is simply the set of changes you've staged. It is also called the staging area. If you've removed files, to stage the removals too, use the --all switch:
    $ git add --all file_or_dir

  6. How can I unstage a staged file?

    If you added a new file, you can unstage it thus:

    $ git rm --cached file_name

    You can unstage all files in the index by running:

    $ git reset HEAD

    To unset a specific modified file, run:

    $ git reset -- file_name

  7. How do I discard my changes to a modified file?

    By discarding, you jettison your changes and replace it with the latest committed copy:

    $ git checkout -- file_name

    However, if the file has already been staged, you will need to unstage it first before being able to discard it (see #6 above).

  8. How do I discard all my local changes?
    Run the following:
    $ git checkout -- .
    If some of your changes are staged, and some unstaged, the above will discard both. If you want to discard only your unstaged changes:
    $ git stash save --keep-index
    $ git stash drop
  9. How do I check-in a file I have modified?
    The term check-in is not used in git. Instead you do a two-step synchronization of your local repository with the remote repository.

    a. First you make sure that all files you have modified or created in your workspace are committed to your local repository. You do this thus:

    $ git add filename  # necessary if it's a new file
    $ git add dirname   # recursive

    You call git add on files that are newly added, as well as those that you've modified. These files are now said to be staged.

    b. Next, you commit all these files to your local repository using git commit.

    $ git commit -a -m "Commit message"

    The -a switch ensures that any modified files are automatically added without the need for calling git add explicitly. You still have to call git add on newly added files.

    Each commit is assigned a SHA1 hash (because in a distributed system like git, adding incremental version numbers, which requires global ordering, is very difficult).

    The symbolic name HEAD refers to the last commit. HEAD^ refers to the parent of head. HEAD~N refers to Nth commit before HEAD (not including HEAD).

  10. How do I check what files I committed locally?
    $ git log

    This command prints the list of commits in your local repository including their hash values and commit notes. If you want to see the specific files committed and their diffs with the origin, run the following command.

    $ git diff origin..HEAD

    You can also check what changed between two commits:

    $ git diff commit1-sha1 commit2-sha1

  11. How do I keep my repository in sync with the remote repository?
    $ git pull remote-branch local-branch

    This would often (but not always) take the form of:

    $ git pull origin master

    Read on for more details.

  12. I have commits in my local repository and there are new commits in the upstream branch that I don't have in my local repo? How do merge everything keeping my commits?
    The following would still work.
    $ git pull remote-branch local-branch

    Under the circumstances, this would create a merge commit - marking the re-convergence point of two divergent branches in the local and the remote.

    However, this is often not desirable in many repos which generally avoid merge commits because they represent non-change commits. Instead you could rebase your local commits on top of the latest from the remote branch as you pull it.
    $ git pull --rebase

  13. How do I check the diff for staged files? What about files from a specific commit?
    To check the diff for your staged changes, use:

    $ git diff --cached

    To check the diff for a specific commit, use:

    $ git show commit-sha1

  14. To check the changes in a particular file between two commits:
    $ git show commit1-sha1 commit2-sha2 -- path_to_file

  15. I realized that my commit went wrong. How do I fix it?
    Maybe you left something out that should have made it to your commit. Or you had to make a change to a file that was already in the commit and you want to include that change. Or perhaps it is just the message you wanted to change. Or all of these? Use "git commit --amend".

    $ git add new_files
    $ git commit -a --amend   # only adds files
    $ git commit --amend -m "message"   # only changes message
    $ git commit -a --amend -m "message"  # changes message, adds files

    Amending a commit replaces it with a new commit with a new SHA1 id. Thus amending should be reserved for private branches, or else any branches based off a commit that is replaced by an amend operation would be difficult to manage.

  16. Now that I have committed my changes to my local repository, how do I check them into the remote repository?
    You don't check-in. You sync with the remote repository using git push. The general command is:
    $ git push remote-repo local-branch

  17. How can I refer to additional remote repositories, pull from them, push to them?
    First you add a remote:
    $ git remote add <remote-name> https://github.com/another-user/repo.git
    Next you can fetch from the remote:
    $ git fetch <remote-name>
    This gets you the commits from this repo. Now you can refer to branches and commits from this remote repo locally - merge changes into yours, and push your branches to this remote.
    $ git merge <remote-name>/<branch-name<remote-name>
    $ git push <remote-name> <local-branch-name<
    You can also remove all references to commits and branches from this remote repository by deleting the remote itself.
    $ git remote remove <remote-name>

  18. How do I delete or rename a file that's already checked in?

    To delete a file:$ git rm path_to_file

    To rename a file:

    $ git mv path_to_file path_to_target
    $ git rm path_to_file1 path_to_file2 ... target_dir

    You need to then commit this change and push it upstream.
  19. How do I see the history of a moved file? Git log only shows the history after it was moved.

    Use the --follow option with git log.

    $ git log --follow -p path_to_file

  20. You keep mentioning "branch", but I don't know what a branch is.
    A branch is a fork of a source tree. When you clone a repository, your local repository has a single branch called master. You can then create more branches from it as follows:

    $ git branch new-branch-name

    This one creates a branch from the HEAD of your current, but you can also create branches from other branches, or from specific commits.

    $ git branch new-branch-name commit-ref

    The commit ref here can be the name of another branch (referring to the HEAD of that branch) or to the SHA1 of a specific branch.

  21. Why do I need branches?
    Short answer: if you have used Perforce: think of them as Perforce changelists with history.

    You need branches to isolate and streamline your feature development, as well as handle releases. You should develop features and commit them to branches.

    You can switch between multiple branches in your workspace. If you want to switch to branch mybranch, use git checkout.

    $ git checkout mybranch

    This will swap the current contents of your workspace with that of the mybranch branch.

    * If there are uncommitted changes in your workspace and you switch to a different branch using checkout, the switch will fail if it required clobbering your uncommitted changes.

    You can also check all the branches by simply using:

    $ git branch

    Your current branch will be marked with an asterisk.

  22. I have some uncommitted changes in my master branch workspace but I now want to put them in a different branch. How do I do it?
    Just create a branch and switch to it. You can do both in a single command:

    $ git checkout -b new-feature-branch

  23. How do I make sure that my branch is updated with changes from another branch?
    Let us suppose you're working on branch my-branch. You can sync your master with the origin:

    $ git pull origin master

    and then merge changes in your master into your my-branch:

    $ git checkout my-branch  # switch to target branch
    $ git merge master        # merge from source branch

    Or, if you want to merge the contents of my-branch into master, you could do:

    $ git checkout master   # switch to target branch
    $ git merge my-branch   # merge from source branch

    A better way of keeping your branch updated is to use rebasing (see below).

  24. How do I resolve conflicts in git?
    Use a tool like kdiff3 or meld. Download and configure it using git config, then invoke git mergetool.

    $ git config --global merge.tool kdiff3
    $ git config --global merge.tool.cmd '"C:\\Program Files\\KDiff3\\kdiff3.exe" $BASE $LOCAL $REMOTE -o $MERGED' $ git mergetool

    $ git mergetool runs the configured mergetool on every file with merge conflicts in the current workspace.

  25. I committed something to my master branch but now want those changes in a different branch but not in master. How do I do it?

    If those changes were not followed by other changes that you would like to retain in the master, then you can do this.

    a. Create branch from master.

    $ git checkout -b new-feature-branch

    b. Reset master to the last commit before your changes.

    $ git checkout master
    $ git reset --hard last-retained-commit-sha1

    Never run reset on a branch you share with other developers. Just like git commit --amend, git reset removes some commits and if such a removed commit is the baseline for some other branch, then that presents a difficult scenario to recover from.

  26. Can I push a branch to the remote repository?
    Yes. Use git push as show below:

    $ git push origin my-local-branch

  27. How can I delete the remote branch I pushed, but retain the local branch?
    Use the colon-prefixed branch name with git push.

    $ git push origin :my-local-branch

  28. How can I delete a local branch?
    $ git branch -d my-local-branch

    The above will delete the local branch only if all commits in the branch are also part of at least another branch locally known. Sometimes, this would require you to pull other remote branches which were not locally updated to ensure that those commits are visible in other branches locally, and then retry the command. If you don't care and just want to delete the branch anyhow, then use:

    $ git branch -D my-local-branch

  29. How do I modify older commits?
    If you can, don't. Work on a single commit per branch, keep amending as needed. Push upstream when you're done. If you still have to tinker with older commits as you sometimes need to do, read the answer to the question "What is the use of rebasing?" below.

  30. How do I trace which commit by which user changed a particular line in a file?
    $ git blame file_name

    It's usually a better idea to use the -w switch to ignore whitespace changes and -M to detect moved lines.
    $ git blame -M -w file_name

  31. How can I undo a commit?
    You can always undo manually and check-in. When it is the last commit, or you can tell that undoing an older commit will not cause conflicts with later commits, then you can use git revert.

    $ git revert HEAD
    $ git revert commit-sha1

    This creates a new commit (you don't need to separately call git commit after this), that undoes the previous commit.

  32. What is the use of rebasing?
    Rebasing changes the baseline commit of a branch. It is a cool way to merge branches which cleans up commit history nicely. Features are usually developed on separate branches. It is possible that one branch gets merged into the master while development on the other branch is still in progress. When you are done on the other branch and want to merge it back to the master too, you realize that its baseline commit is quite old. At this point you either merge the master into the branch (as suggested but discouraged in #19) and then merge it back to the master, or you rebase the branch. The former approach results in extraneous commits on account of the merge and could bury your own commits in a barrage of other commits that came in the merge. This is what rebase aims to avoid. To rebase a branch, you run the following command:

    $ git checkout branch-to-rebase
    $ git rebase commit-ref

    The commit-ref identifies the baseline commit you want to rebase your branch to. It could be a commit SHA1, or a branch name, etc. Frequently, you would want to rebase your branch to the head of a parent branch, so you would simply use the name of the parent branch for commit-ref. This effectively moves the baseline of your branch to the head of its parent branch (or to whatever commit you specified). A merge into the parent following this would be a fast-forward merge and produce a clean history.

    Rebase removes some commits and creates new ones in their lieu at different points in the branch. If there are sub-branches based on any of those commits that are removed in a rebase operation, then recovering those branches could be complex.

  33. I hurriedly reset my branch back by a couple of commits and now I realize I lost some important changes? Is there a way to retrieve the lost information?
    There actually is. We can find the SHA1 hash of any deleted commits only if it is from the last 30 days, by using:

    git reflog

    From its output, we can identify the SHA1 of a commit that has since been lost due to the reset. For a slightly more detailed output that helps you identify the context of each commit, you can run $ git log -g branch_name. We can then run git reset with that SHA1 as shown:

    $ git reset --hard lost-SHA1
    In case reflog didn't show you what you were looking for (an unlikely event), try your luck by running:
    $ git fsck --full

    Then go through the listed dangling commits, and blobs (for stashes), and use git show commit-sha1 to list contents.

  34. Could I encounter merge conflicts during rebasing and what do I do then?
    Yes you could and you have the option of either resolving the conflicts (see #20) and completing the rebase, or discarding the attempt to rebase. Once you have resolved the conflicts, you use the following command to continue the rebase:

    $ git rebase --continue

    Sometimes the conflict could be bad enough that you have to ditch the attempt to rebase. You could do that with:

    $ git rebase --abort

  35. Is there a way to save my uncommitted changes and work on another feature? OR What is git stash?
    Of course there is. $ git stash provides the easiest way of doing that. When you run git stash, it saves your uncommitted changes and lets you start on clean workspace that is in sync with your HEAD. Once you're done with these changes and have committed them, you could continue on your old changes with $ git stash pop. You can also use it as a "rebase" for your uncommitted changes, if there are new changes upstream that need to be pulled into your working branch:

    $ git stash
    $ git pull upstream-branch local-branch
    $ git stash apply
    $ git stash drop

    In case git stash apply encounters a conflict, you need to manually merge and then call git stash drop. Likewise, if you have to rebase your branch and have some unfinished work that you'd like to continue on after rebasing, use the following commands:

    $ git stash
    $ git rebase parent-branch
    $ git stash apply
    $ git stash drop

    You can use git stash pop in place of the command pair git stash apply/drop. You can discard your stashed changes without applying, by simply issuing git stash drop.

  36. I have some modified files (or added files) that I am yet to stage. Trying to pull / merge on my branch fails. What do I do?
    One simple approach is to use stash.
    $ git stash   # this saves your work without committing it
    $ git pull origin master # or git merge some-branch
    $ git stash pop  # or git stash apply + git stash drop
  37. How do I see what all is stashed, peep into individual stashes, and apply stashes selectively?
    To list stashes:
    $ git stash list  # this saves your work without committing it
    stash@{0} ...
    stash@{1} ...

    Now you can check the content of a specific stash:
    $ git stash show -p stash@{1}

    You can also apply a specific stash:
    $ git stash pop stash@\{1\}  # escape the braces

    Why would you need to apply specific stashes? Stashes are added to a global list. Say you alternate between two branches A and B, and you stash your work on each branch into stashes s_A and s_B. If you now run:
    $ git stash pop

    it will always apply the last stash on whatever branch you're in. If the last stash you took was s_B on branch B, but you then call
    git stash pop
    on branch A, then it will apply s_B on branch A. This is not what you usually want. Instead, you should take care to list the stashes out, and apply the correct stash explicitly to the correct branch.
  38. I have some staged files that I'm yet to commit. At this stage I need to pull / merge / rebase on my branch, but it fails because of the staged changes what to do?
    Unstage your changes.

    $ git reset HEAD

    Then follow the above #32. However, if this was your first commit in your repo, the reset won't work as is. Instead you have to run the following command (but be very careful about running it under any other circumstances because you lose all previous commits):

    $ git update-ref -d HEAD

  39. I committed some files on my local branch but now I want to change them further and not push these commits upstream. Plus I may need to pull / merge / rebase. What do I do?
    Identify the last commit you want to retain - say last-good-commit-sha1. Run the following:
    $ git reset last-good-commit-sha1

    Your changes are now unstaged. In order to now do any merges on your branch, follow #31. If no merges are needed, just make the necessary changes and create a fresh commit. Also see, git commit --amend, git rebase and git rebase -i. Any command that changes commit history - whether git reset, git rebase or git commit --amend, should be issued only on local branches to edit commits that have not be pushed upstream.

    To push such commit history changes upstream, you have to use:

    $ git push -f ...

    In many repositories, git push -f ... is disabled and for good reason.

  40. I have some changes committed to one branch which I want to pull into another branch without pulling the whole branch?
    Identify the individual commits that you need to pull. Note their sha1 ids. Switch to the target branch and then use the git cherry-pick command:
    $ git checkout target-branch
    $ git cherry-pick -x <commit1-sha1>
    $ git cherry-pick -x <commit2-sha1>
    $ git cherry-pick -x <commit3-sha1>

    Your changes are now committed to your local branch. The commit ids are different from the ones you cherry-picked though. You now need to push your branch upstream. The -x option is vital if at a later point in time, your current branch and the branch you cherry-picked from need to be merged, etc. The -x switch avoids conflicts in such cases. If that's not your concern, leave out -x.

  41. How do I find what all I committed for this sprint?

    Use the following command:

    $ git log --author=your_user_name --since=2.weeks

    Adjust to your sprint length in weeks, days, even hours. Want to see all your diffs so far:

    $ git log --author=your_user_name --since=2.weeks | grep commit | sed -e 's/commit //' | xargs git show >changes.diff

    More generally, to check the difference between two specific points in time, you can use the following:
    $ git diff my-branch@{2015-09-30}..my-branch@{2015-09-28}

  42. How do I find out how many commits have gone into my current dev branch?
    $ git rev-list master.. --count

    That's assuming your parent branch is called master.

  43. How do I find out whether a branch contains a particular commit?

    You have to do it the other way round. You figure out the branches that contain a particular commit and check whether the specific branch you're interested in is one of them. Use the following command:

    $ git branch --contains <commit-id>

  44. How do I check the commit history of a specific file?
    $ git log -p 
  45. I want to move a file from one repository to another (say proj1 to proj2). Is there a way to do it that preserves history?

    Yes there is a way, but it won't happen with a single command and will require some manual intervention. So let us suppose you want to move file Foo.java from path src/bar under repository proj1 to path src/baz under repository proj2. Here are the commands you need to run in sequence:
    $ cd proj1
    $ git log --pretty=email --patch-with-stat --reverse – src/bar/Foo.java > ../Foo.patch
    $ vi ../Foo.patch
    ... Inside vi, run the following sed script in command mode
    :% s^/src/bar^/src/baz^g
    $ cd ../proj2
    $ git am < ../Foo.patch

    The last command recreates the entire commit history in the target repository (proj2). Now in general, moving an entire repo into another as a subdirectory is much more common. This link here shows a nice way of doing it. Just remember to use --allow-unrelated-histories during git merge.

  46. How can I import one git repository as a subdirectory of another repository?

    You need to use the git subtree command as shown below:
    $ cd target-repo
    $ git subtree add --prefix=target/subdir https://source.repo/foo/bar.git 

    In the above example, enter the target repo directory, then import source.repo under the target/repo subdirectory of your target repo. You also specify the branch from which you pull. The directories are created as needed. This will add the entire history of commits from the specified branch in source.repo in your current repo, plus create a merge commit. You would then push your changes in the target repo upstream.

    Sometimes before merging, you may need to change the committer ids of all the previous commits in case your git administrator has disabled merging commits with a different committer. You can do this by changing the committer id for all the commits on a branch (note that the author for these commits would not need to be unchanged so git blame would correctly show the author, not you). The best way is to change this at source, i.e. in the source repository on the branch that you would then pull into the destination repo.
    $ git checkout target-branch
    $ git filter-branch --env-filter '                                    
    export GIT_COMMITTER_NAME="i.am"
    export GIT_COMMITTER_EMAIL="i.am@mydomain.com"
    ' --tag-name-filter cat -- --branches --tags
  47. Something seems wrong with my working tree / local repository. Pulls / merges don't seem to be working and it is showing unstaged files I haven't even changed. What do I do?
    To start with, run:
    $ git gc

    Make sure no other programs are running git commands on the same repository in the background. This means closing Eclipse, SourceTree, any program that might be running some git commands in the background, etc. You should also periodically clean up your repository of dangling commits, etc. Run the following commands:
    $ git fsck
    $ git prune
  48. My remote url has changed (i.e. my origin has changed), do I have to clone the whole repository again from the new URL ?
    No. You can easily change the origin's URL or the remote URL of your local repository. You can do using git remote command.
    $ git remote set-url origin 
  49. How do I find out who committed how many files?
    You can run the following command:
    $ git shortlog -s -n

    But do note that the same author could show up with multiple names depending on whether their user name changed or not.

  50. $ git log -n 10 --author=amukher1

  51. How do I create a local branch and push from there into a remote branch?
    Do the following:
    $ git fetch --all
    $ git fetch -t
    $ git branch <local_branch_name> remotes/origin/<remote_branch_name>

    The last step creates a tracking branch.

  52. How do I clone a single remote branch of a repository?
    Do the following:
    $ git clone <url> --branch <branch-name>

    The last step creates a tracking branch.

  53. How do I update a local branch from its tracking branch?
    Simply type:
    $ git pull
  54. How can I add an existing repository to github?
    Create a new repository on github using the web interface. Let's say its call newrepo. Now go to your local repository root dir and run the following commands.
    $ git init
    $ git remote add origin https://github.com/username/newrepo
    $ git push -u origin master

  55. I stashed my work but have now lost it. How can I recover it?
    Run the following command:
    $ for ref in `git fsck --unreachable | grep commit | cut -d' ' -f3`; do git show --summary $ref; done | less

    This will neatly list the unreachable commits (including stash commits). Identify which one you need and do a:
    $ git show <sha1>

  56. Is there an easy way to list just file names and not their contents in a commit or diff?
    Yes. When in doubt, try the --name-only option on a git command. It will usually suffice:
    $ git show --name-only

    This will list the files in the last commit. You can easily extend that to diff:
    $ git diff --name-only

  57. How can I see the state of a file at a particular commit in the past?
    This way:
    $ git show <commit-sha1>:<file-path>

  58. How can I list all the remote branches I have created (so that I can clean them up later)?
    This way:
    $ git for-each-ref --format='%(committerdate) %09 %(authorname) %09 %(refname)' | sort -k5n -k2M -k3n -k4n| grep | awk '{print $8}'| grep origin| sed 's/refs\/remotes\///'

  59. How can I squash all my commits before merging into another branch?
    Let's say you have a branch called fix-NNN and you want to squash merge it in feature-NNN:
    $ git checkout feature-NNN
    $ git merge --squash fix-NNN    # squashes and adds to the index
    $ git commit

    Things work best and cleanest if fix-NNN was branched off feature-NNN. Another alternative (which also works on older git) is the following:
    $ git show-ref fix-NNN
    sha1... refs/heads/fix-NNN   # make a note of the SHA1 -> head-of-fix-NNN
    $ git merge feature-NNN
    $ git stash    # stash everything in progress
    $ git reset head-of-fix-NNN
    $ git add --all
    $ git commit     # effectively squashes

  60. How can I get the last common commit between two branches?
    You're in luck - git has a specific command for the purpose:
    $ git merge-base branch1 branch2

  61. I squash-merged my dev branch into my github feature branch. But I continued adding more commits after that on my dev branch. How can I now incrementally merge / rebase my dev branch?
    Let's suppose that in your dev branch, the commits are a1, a2, a3, a4, a5. You squash merged till a2 so a1 and a2 are no longer available in your feature branch. If you now tried rebasing your dev branch on the feature branch you will get conflicts due to the changes in a1, a2 being present in the feature branch with a different commit. What you should be able to do is rebase from a3 onwards on to the head of the feature branch. You do this with:

    $ git rebase --onto feature-branch a3

  62. Additional references

  1. Git from the bottom-up: https://jwiegley.github.io/git-from-the-bottom-up
  2. Git revision selection: http://git-scm.com/book/en/v2/Git-Tools-Revision-Selection
  3. Git-aware bash prompts: https://github.com/jimeh/git-aware-prompt
  4. A Hacker's Guide to Git: https://wildlyinaccurate.com/a-hackers-guide-to-git/

Read more!