Merge branch 'jc/fetch-pull-refmap'

* jc/fetch-pull-refmap:
  docs: Explain the purpose of fetch's and pull's <refspec> parameter.
  fetch: allow explicit --refmap to override configuration
  fetch doc: add a section on configured remote-tracking branches
  fetch doc: remove "short-cut" section
  fetch doc: update refspec format description
  fetch doc: on pulling multiple refspecs
  fetch doc: remove notes on outdated "mixed layout"
  fetch doc: update note on '+' in front of the refspec
  fetch doc: move FETCH_HEAD material lower and add an example
  fetch doc: update introductory part for clarity

Author: gitster

Documentation/fetch-options.txt

@@ -72,6 +72,14 @@ endif::git-pull[]
 	setting. See linkgit:git-config[1].
 
 ifndef::git-pull[]
+--refmap=<refspec>::
+	When fetching refs listed on the command line, use the
+	specified refspec (can be given more than once) to map the
+	refs to remote-tracking branches, instead of the values of
+	`remote.*.fetch` configuration variables for the remote
+	repository.  See section on "Configured Remote-tracking
+	Branches" for details.
+
 -t::
 --tags::
 	Fetch all tags from the remote (i.e., fetch remote tags

Documentation/git-fetch.txt

@@ -17,29 +17,31 @@ SYNOPSIS
 
 DESCRIPTION
 -----------
-Fetches named heads or tags from one or more other repositories,
-along with the objects necessary to complete them.
+Fetch branches and/or tags (collectively, "refs") from one or more
+other repositories, along with the objects necessary to complete their
+histories.  Remote-tracking branches are updated (see the description
+of <refspec> below for ways to control this behavior).
 
-The ref names and their object names of fetched refs are stored
-in `.git/FETCH_HEAD`.  This information is left for a later merge
-operation done by 'git merge'.
-
-By default, tags are auto-followed.  This means that when fetching
-from a remote, any tags on the remote that point to objects that exist
-in the local repository are fetched.  The effect is to fetch tags that
+By default, any tag that points into the histories being fetched is
+also fetched; the effect is to fetch tags that
 point at branches that you are interested in.  This default behavior
-can be changed by using the --tags or --no-tags options, by
-configuring remote.<name>.tagopt, or by using a refspec that fetches
-tags explicitly.
+can be changed by using the --tags or --no-tags options or by
+configuring remote.<name>.tagopt.  By using a refspec that fetches tags
+explicitly, you can fetch tags that do not point into branches you
+are interested in as well.
 
-'git fetch' can fetch from either a single named repository,
+'git fetch' can fetch from either a single named repository or URL,
 or from several repositories at once if <group> is given and
 there is a remotes.<group> entry in the configuration file.
 (See linkgit:git-config[1]).
 
 When no remote is specified, by default the `origin` remote will be used,
 unless there's an upstream branch configured for the current branch.
 
+The names of refs that are fetched, together with the object names
+they point at, are written to `.git/FETCH_HEAD`.  This information
+may be used by scripts or other git commands, such as linkgit:git-pull[1].
+
 OPTIONS
 -------
 include::fetch-options.txt[]
@@ -49,6 +51,55 @@ include::pull-fetch-param.txt[]
 include::urls-remotes.txt[]
 
 
+CONFIGURED REMOTE-TRACKING BRANCHES[[CRTB]]
+-------------------------------------------
+
+You often interact with the same remote repository by
+regularly and repeatedly fetching from it.  In order to keep track
+of the progress of such a remote repository, `git fetch` allows you
+to configure `remote.<repository>.fetch` configuration variables.
+
+Typically such a variable may look like this:
+
+------------------------------------------------
+[remote "origin"]
+	fetch = +refs/heads/*:refs/remotes/origin/*
+------------------------------------------------
+
+This configuration is used in two ways:
+
+* When `git fetch` is run without specifying what branches
+  and/or tags to fetch on the command line, e.g. `git fetch origin`
+  or `git fetch`, `remote.<repository>.fetch` values are used as
+  the refspecs---they specify which refs to fetch and which local refs
+  to update.  The example above will fetch
+  all branches that exist in the `origin` (i.e. any ref that matches
+  the left-hand side of the value, `refs/heads/*`) and update the
+  corresponding remote-tracking branches in the `refs/remotes/origin/*`
+  hierarchy.
+
+* When `git fetch` is run with explicit branches and/or tags
+  to fetch on the command line, e.g. `git fetch origin master`, the
+  <refspec>s given on the command line determine what are to be
+  fetched (e.g. `master` in the example,
+  which is a short-hand for `master:`, which in turn means
+  "fetch the 'master' branch but I do not explicitly say what
+  remote-tracking branch to update with it from the command line"),
+  and the example command will
+  fetch _only_ the 'master' branch.  The `remote.<repository>.fetch`
+  values determine which
+  remote-tracking branch, if any, is updated.  When used in this
+  way, the `remote.<repository>.fetch` values do not have any
+  effect in deciding _what_ gets fetched (i.e. the values are not
+  used as refspecs when the command-line lists refspecs); they are
+  only used to decide _where_ the refs that are fetched are stored
+  by acting as a mapping.
+
+The latter use of the `remote.<repository>.fetch` values can be
+overridden by giving the `--refmap=<refspec>` parameter(s) on the
+command line.
+
+
 EXAMPLES
 --------
 
@@ -76,6 +127,19 @@ the local repository by fetching from the branches (respectively)
 The `pu` branch will be updated even if it is does not fast-forward,
 because it is prefixed with a plus sign; `tmp` will not be.
 
+* Peek at a remote's branch, without configuring the remote in your local
+repository:
++
+------------------------------------------------
+$ git fetch git://git.kernel.org/pub/scm/git/git.git maint
+$ git log FETCH_HEAD
+------------------------------------------------
++
+The first command fetches the `maint` branch from the repository at
+`git://git.kernel.org/pub/scm/git/git.git` and the second command uses
+`FETCH_HEAD` to examine the branch with linkgit:git-log[1].  The fetched
+objects will eventually be removed by git's built-in housekeeping (see
+linkgit:git-gc[1]).
 
 BUGS
 ----

Documentation/pull-fetch-param.txt

@@ -12,9 +12,23 @@ ifndef::git-pull[]
 endif::git-pull[]
 
 <refspec>::
-	The format of a <refspec> parameter is an optional plus
-	`+`, followed by the source ref <src>, followed
-	by a colon `:`, followed by the destination ref <dst>.
+	Specifies which refs to fetch and which local refs to update.
+	When no <refspec>s appear on the command line, the refs to fetch
+	are read from `remote.<repository>.fetch` variables instead
+ifndef::git-pull[]
+	(see <<CRTB,CONFIGURED REMOTE-TRACKING BRANCHES>> below).
+endif::git-pull[]
+ifdef::git-pull[]
+	(see linkgit:git-fetch[1]).
+endif::git-pull[]
++
+The format of a <refspec> parameter is an optional plus
+`+`, followed by the source ref <src>, followed
+by a colon `:`, followed by the destination ref <dst>.
+The colon can be omitted when <dst> is empty.
++
+`tag <tag>` means the same as `refs/tags/<tag>:refs/tags/<tag>`;
+it requests fetching everything up to the given tag.
 +
 The remote ref that matches <src>
 is fetched, and if <dst> is not empty string, the local
@@ -24,55 +38,34 @@ is updated even if it does not result in a fast-forward
 update.
 +
 [NOTE]
-If the remote branch from which you want to pull is
-modified in non-linear ways such as being rewound and
-rebased frequently, then a pull will attempt a merge with
-an older version of itself, likely conflict, and fail.
-It is under these conditions that you would want to use
-the `+` sign to indicate non-fast-forward updates will
-be needed.  There is currently no easy way to determine
-or declare that a branch will be made available in a
-repository with this behavior; the pulling user simply
+When the remote branch you want to fetch is known to
+be rewound and rebased regularly, it is expected that
+its new tip will not be descendant of its previous tip
+(as stored in your remote-tracking branch the last time
+you fetched).  You would want
+to use the `+` sign to indicate non-fast-forward updates
+will be needed for such branches.  There is no way to
+determine or declare that a branch will be made available
+in a repository with this behavior; the pulling user simply
 must know this is the expected usage pattern for a branch.
-+
-[NOTE]
-You never do your own development on branches that appear
-on the right hand side of a <refspec> colon on `Pull:` lines;
-they are to be updated by 'git fetch'.  If you intend to do
-development derived from a remote branch `B`, have a `Pull:`
-line to track it (i.e. `Pull: B:remote-B`), and have a separate
-branch `my-B` to do your development on top of it.  The latter
-is created by `git branch my-B remote-B` (or its equivalent `git
-checkout -b my-B remote-B`).  Run `git fetch` to keep track of
-the progress of the remote side, and when you see something new
-on the remote branch, merge it into your development branch with
-`git pull . remote-B`, while you are on `my-B` branch.
+ifdef::git-pull[]
 +
 [NOTE]
 There is a difference between listing multiple <refspec>
 directly on 'git pull' command line and having multiple
-`Pull:` <refspec> lines for a <repository> and running
+`remote.<repository>.fetch` entries in your configuration
+for a <repository> and running a
 'git pull' command without any explicit <refspec> parameters.
-<refspec> listed explicitly on the command line are always
+<refspec>s listed explicitly on the command line are always
 merged into the current branch after fetching.  In other words,
-if you list more than one remote refs, you would be making
-an Octopus.  While 'git pull' run without any explicit <refspec>
-parameter takes default <refspec>s from `Pull:` lines, it
-merges only the first <refspec> found into the current branch,
-after fetching all the remote refs.  This is because making an
+if you list more than one remote ref, 'git pull' will create
+an Octopus merge.  On the other hand, if you do not list any
+explicit <refspec> parameter on the command line, 'git pull'
+will fetch all the <refspec>s it finds in the
+`remote.<repository>.fetch` configuration and merge
+only the first <refspec> found into the current branch.
+This is because making an
 Octopus from remote refs is rarely done, while keeping track
 of multiple remote heads in one-go by fetching more than one
 is often useful.
-+
-Some short-cut notations are also supported.
-+
-* `tag <tag>` means the same as `refs/tags/<tag>:refs/tags/<tag>`;
-  it requests fetching everything up to the given tag.
-ifndef::git-pull[]
-* A parameter <ref> without a colon fetches that ref into FETCH_HEAD,
-endif::git-pull[]
-ifdef::git-pull[]
-* A parameter <ref> without a colon merges <ref> into the current
-  branch,
 endif::git-pull[]
-  and updates the remote-tracking branches (if any).

builtin/fetch.c

@@ -45,6 +45,8 @@ static struct transport *gsecondary;
 static const char *submodule_prefix = "";
 static const char *recurse_submodules_default;
 static int shown_url = 0;
+static int refmap_alloc, refmap_nr;
+static const char **refmap_array;
 
 static int option_parse_recurse_submodules(const struct option *opt,
 				   const char *arg, int unset)
@@ -69,6 +71,19 @@ static int git_fetch_config(const char *k, const char *v, void *cb)
 	return 0;
 }
 
+static int parse_refmap_arg(const struct option *opt, const char *arg, int unset)
+{
+	ALLOC_GROW(refmap_array, refmap_nr + 1, refmap_alloc);
+
+	/*
+	 * "git fetch --refmap='' origin foo"
+	 * can be used to tell the command not to store anywhere
+	 */
+	if (*arg)
+		refmap_array[refmap_nr++] = arg;
+	return 0;
+}
+
 static struct option builtin_fetch_options[] = {
 	OPT__VERBOSITY(&verbosity),
 	OPT_BOOL(0, "all", &all,
@@ -107,6 +122,8 @@ static struct option builtin_fetch_options[] = {
 		   N_("default mode for recursion"), PARSE_OPT_HIDDEN },
 	OPT_BOOL(0, "update-shallow", &update_shallow,
 		 N_("accept refs that update .git/shallow")),
+	{ OPTION_CALLBACK, 0, "refmap", NULL, N_("refmap"),
+	  N_("specify fetch refmap"), PARSE_OPT_NONEG, parse_refmap_arg },
 	OPT_END()
 };
 
@@ -278,6 +295,9 @@ static struct ref *get_ref_map(struct transport *transport,
 	const struct ref *remote_refs = transport_get_remote_refs(transport);
 
 	if (refspec_count) {
+		struct refspec *fetch_refspec;
+		int fetch_refspec_nr;
+
 		for (i = 0; i < refspec_count; i++) {
 			get_fetch_map(remote_refs, &refspecs[i], &tail, 0);
 			if (refspecs[i].dst && refspecs[i].dst[0])
@@ -307,12 +327,21 @@ static struct ref *get_ref_map(struct transport *transport,
 		 * by ref_remove_duplicates() in favor of one of these
 		 * opportunistic entries with FETCH_HEAD_IGNORE.
 		 */
-		for (i = 0; i < transport->remote->fetch_refspec_nr; i++)
-			get_fetch_map(ref_map, &transport->remote->fetch[i],
-				      &oref_tail, 1);
+		if (refmap_array) {
+			fetch_refspec = parse_fetch_refspec(refmap_nr, refmap_array);
+			fetch_refspec_nr = refmap_nr;
+		} else {
+			fetch_refspec = transport->remote->fetch;
+			fetch_refspec_nr = transport->remote->fetch_refspec_nr;
+		}
+
+		for (i = 0; i < fetch_refspec_nr; i++)
+			get_fetch_map(ref_map, &fetch_refspec[i], &oref_tail, 1);
 
 		if (tags == TAGS_SET)
 			get_fetch_map(remote_refs, tag_refspec, &tail, 0);
+	} else if (refmap_array) {
+		die("--refmap option is only meaningful with command-line refspec(s).");
 	} else {
 		/* Use the defaults */
 		struct remote *remote = transport->remote;

t/t5510-fetch.sh

@@ -447,6 +447,43 @@ test_expect_success 'explicit pull should update tracking' '
 	)
 '
 
+test_expect_success 'explicit --refmap is allowed only with command-line refspec' '
+	cd "$D" &&
+	(
+		cd three &&
+		test_must_fail git fetch --refmap="*:refs/remotes/none/*"
+	)
+'
+
+test_expect_success 'explicit --refmap option overrides remote.*.fetch' '
+	cd "$D" &&
+	git branch -f side &&
+	(
+		cd three &&
+		git update-ref refs/remotes/origin/master base-origin-master &&
+		o=$(git rev-parse --verify refs/remotes/origin/master) &&
+		git fetch --refmap="refs/heads/*:refs/remotes/other/*" origin master &&
+		n=$(git rev-parse --verify refs/remotes/origin/master) &&
+		test "$o" = "$n" &&
+		test_must_fail git rev-parse --verify refs/remotes/origin/side &&
+		git rev-parse --verify refs/remotes/other/master
+	)
+'
+
+test_expect_success 'explicitly empty --refmap option disables remote.*.fetch' '
+	cd "$D" &&
+	git branch -f side &&
+	(
+		cd three &&
+		git update-ref refs/remotes/origin/master base-origin-master &&
+		o=$(git rev-parse --verify refs/remotes/origin/master) &&
+		git fetch --refmap="" origin master &&
+		n=$(git rev-parse --verify refs/remotes/origin/master) &&
+		test "$o" = "$n" &&
+		test_must_fail git rev-parse --verify refs/remotes/origin/side
+	)
+'
+
 test_expect_success 'configured fetch updates tracking' '
 
 	cd "$D" &&