git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
* [RFC/PoC 0/5] Provide example docbundles
@ 2022-11-02 22:48 Philip Oakley
  2022-11-02 22:48 ` [RFC/PoC 1/5] doc: provide DocBundles design document Philip Oakley
                   ` (4 more replies)
  0 siblings, 5 replies; 15+ messages in thread
From: Philip Oakley @ 2022-11-02 22:48 UTC (permalink / raw)
  To: GitList; +Cc: Self, Junio C Hamano, Taylor Blau

The man pages provide user facing examples which explain important
concepts, such as history-simplification. However users can have
difficulty in reproducing these examples to allow the exploration
of thr concepts. Let's provide these simple example, pre-packaged
as part of the documentation.

This Request for Comment (RFC) / Proof of Concept (PoC) attempts
to set out some design aims, and an outline implementation based
on using bundles for each example, and the test system for their
preparation.

Are there any obvious show-stoppers? 
Where best to hold the docbundle dir on various platforms?
How many examples to provide, and how to decide?
Usage of existing test repo examples?
etc.
--
Philip

Philip Oakley (5):
  doc: provide DocBundles design document
  Documentation Bundles: ignore artifact files from creating doc bundles
  test-lib: add GIT_BUNDLE_DIR setup
  Doc Bundles: add t6102 rev-list simple and pulls examples
  docbundle: provide the git-log History Simplication example

 .gitignore                            |   2 +
 Documentation/technical/docbundle.txt |  97 ++++++++++
 t/t6012-rev-list-simplify.sh          |   8 +
 t/t9990-history-simplification.sh     | 261 ++++++++++++++++++++++++++
 t/test-lib.sh                         |  14 ++
 5 files changed, 382 insertions(+)
 create mode 100644 Documentation/technical/docbundle.txt
 create mode 100644 t/t9990-history-simplification.sh

-- 
2.38.1.281.g83ef3ded8d


^ permalink raw reply	[flat|nested] 15+ messages in thread

* [RFC/PoC 1/5] doc: provide DocBundles design document
  2022-11-02 22:48 [RFC/PoC 0/5] Provide example docbundles Philip Oakley
@ 2022-11-02 22:48 ` Philip Oakley
  2022-11-03  9:19   ` Ævar Arnfjörð Bjarmason
  2022-11-02 22:48 ` [RFC/PoC 2/5] Documentation Bundles: ignore artifact files from creating doc bundles Philip Oakley
                   ` (3 subsequent siblings)
  4 siblings, 1 reply; 15+ messages in thread
From: Philip Oakley @ 2022-11-02 22:48 UTC (permalink / raw)
  To: GitList; +Cc: Self, Junio C Hamano, Taylor Blau

DocBundles are exemplar repositories, provided as bundles, which
contain either 'interesting' test setups for exploration, or
the described examples from within the man pages.

Signed-off-by: Philip Oakley <philipoakley@iee.email>
---
 Documentation/technical/docbundle.txt | 97 +++++++++++++++++++++++++++
 1 file changed, 97 insertions(+)
 create mode 100644 Documentation/technical/docbundle.txt

diff --git a/Documentation/technical/docbundle.txt b/Documentation/technical/docbundle.txt
new file mode 100644
index 0000000000..8c17a7847f
--- /dev/null
+++ b/Documentation/technical/docbundle.txt
@@ -0,0 +1,97 @@
+docbundle Design Document
+=========================
+
+The Git man pages contain many example setups to demonstrate aspects
+of the related Git commands, such as history simplification in the `log`
+and `show commands. However, while illustrative, these are not
+accessible, as example repositories, to regular, potentially confused,
+users who may need to understand the set up of the examples.
+
+The docbundle concept is that for each of these examples there is a
+matching `bundle` file, provided with the Documentation, which can be
+simply cloned to provide an example repository containing the example.
+
+There are also many example repositories in the Git `/t` test system
+that could be exposed to users which already demonstrate some important
+testable aspect of Git within a carefully constructed repository.
+
+
+This proposal is to provide DocBundles to users via the Git project.
+
+
+Firstly, it should be reasonably self-evident that the man page examples
+should, in themselves, be tested for accuracy, especially if provided
+to users.
+
+Secondly, the existing test system is the proper place to setup and
+test any such docbundle repositories.
+
+Third, that the bundle is an obvious pre-existing mechanism for
+transporting repositories (see also the Bundle-URI proposals).
+
+Fourthly, the preparation and distribution of the docbundle should
+require minimal maintainer effort, preferably being integrated into
+the existing automation for Documentation preparation and distribution.
+
+Fifthly, the docbudles themselves should not to be part of the git.git
+repo. They are prepared via the test suite and should be stored (locally)
+in a designated Git_DocBundle_Dir, and be .gitignored. The DocBundles
+are transferred to the maintainer's Doc repo via the automation.
+
+Sixthly, the docbundles to be named based on their `tnnnn` test number
+and a (short) descriptive name.
+
+Seven, (dev) start with an existing test, rather than add a test (most
+doc/man examples aren't tested!). Add tests for missing examples later.
+
+Eight, (option) add a `git docbundle` command to assist in locating and
+listing the docbundle examples. It also provides place to explain their
+purpose as "Tutorial Examples" (Note, many existing examples are to
+illustrate complex/difficult scenarios where the wording was insufficient;
+the docbudles are minimal examples).
+
+
+Status
+======
+
+This is an RFC/PoC (Request for Comment / Proof of Concept).
+
+The attached commits establish:
+
+Set out this design in the Documentation/technical directory.
+
+A default location for the docbundles and an env variable to allow
+flexibility by extending test-lib.
+
+The docbundles are ignored, as is the default directory.
+
+Two docbundles are generated from a 'random' test that I happened to
+have open in my editor (t6102 rev-list-simplify).
+
+The docbundle creation is protected by `test_expect_success` wrappers.
+
+
+
+ToDo
+====
+
+DocBundles - Is this the best name, or just a convenient way of
+indicating the delivery mechanism?
+
+Add Prerequisite to control if docbundles are generated (don't waste
+maintainers PC time when not needed).
+
+CI integration (i.e. whether to test the bundling aspect - should be
+reliable).
+
+How to update the man pages, perhaps by inclusion of the test comment
+(exported?), and distinguish between man page descriptions (being
+packaged by the test suite) and tests which offer users insight into
+Git operations, usually with 'interesting' commit-graph set-ups.
+
+Claim t999x as the man doc bundle test sequence ('999' is the UK
+emergency ("Help!") number; 911, 101 and 102 are taken test areas.
+
+Other stuff implicit in the eight point list
+
+
-- 
2.38.1.281.g83ef3ded8d


^ permalink raw reply related	[flat|nested] 15+ messages in thread

* [RFC/PoC 2/5] Documentation Bundles: ignore artifact files from creating doc bundles
  2022-11-02 22:48 [RFC/PoC 0/5] Provide example docbundles Philip Oakley
  2022-11-02 22:48 ` [RFC/PoC 1/5] doc: provide DocBundles design document Philip Oakley
@ 2022-11-02 22:48 ` Philip Oakley
  2022-11-03  9:12   ` Ævar Arnfjörð Bjarmason
  2022-11-02 22:48 ` [RFC/PoC 3/5] test-lib: add GIT_BUNDLE_DIR setup Philip Oakley
                   ` (2 subsequent siblings)
  4 siblings, 1 reply; 15+ messages in thread
From: Philip Oakley @ 2022-11-02 22:48 UTC (permalink / raw)
  To: GitList; +Cc: Self, Junio C Hamano, Taylor Blau

The Git documentation contains many examples that can be prepared as
repository bundles for users to clone and explore without any language
or translation confusions between the written word and intended code.

There are also a large number of example repositories in the t/ test
system can can also be exposed to users to highlight the critical
distinctions tested therein.

These documentation bundles will be prepared ar part of the test system.
Start by ignoring those parts which will be considered build artifacts,
the GIT-BUNDLE-DIR, and the `.bndl` bundle files themselves.

The artifacts themselves will later be manipulated by the `make` elements
of the documentation system.

Signed-off-by: Philip Oakley <philipoakley@iee.email>
---
 .gitignore | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/.gitignore b/.gitignore
index 6ded10067a..b96a4ce5df 100644
--- a/.gitignore
+++ b/.gitignore
@@ -3,6 +3,7 @@
 /fuzz-pack-headers
 /fuzz-pack-idx
 /GIT-BUILD-OPTIONS
+/GIT-BUNDLE-DIR
 /GIT-CFLAGS
 /GIT-LDFLAGS
 /GIT-PREFIX
@@ -250,3 +251,4 @@ Release/
 *.dSYM
 /contrib/buildsystems/out
 CMakeSettings.json
+*.bndl
-- 
2.38.1.281.g83ef3ded8d


^ permalink raw reply related	[flat|nested] 15+ messages in thread

* [RFC/PoC 3/5] test-lib: add GIT_BUNDLE_DIR setup
  2022-11-02 22:48 [RFC/PoC 0/5] Provide example docbundles Philip Oakley
  2022-11-02 22:48 ` [RFC/PoC 1/5] doc: provide DocBundles design document Philip Oakley
  2022-11-02 22:48 ` [RFC/PoC 2/5] Documentation Bundles: ignore artifact files from creating doc bundles Philip Oakley
@ 2022-11-02 22:48 ` Philip Oakley
  2022-11-03  9:14   ` Ævar Arnfjörð Bjarmason
  2022-11-02 22:48 ` [RFC/PoC 4/5] Doc Bundles: add t6102 rev-list simple and pulls examples Philip Oakley
  2022-11-02 22:48 ` [RFC/PoC 5/5] docbundle: provide the git-log History Simplification example Philip Oakley
  4 siblings, 1 reply; 15+ messages in thread
From: Philip Oakley @ 2022-11-02 22:48 UTC (permalink / raw)
  To: GitList; +Cc: Self, Junio C Hamano, Taylor Blau

The test system may output a number of documentation bundles containing
test repositories of interest to users, including those explicitly
documented.

Provide a default location for holding those bundles.
As build artefact's this location, along with `.bndl` files, was
ignored in an earlier commit.

This may need further refinement.

Signed-off-by: Philip Oakley <philipoakley@iee.email>
---
 t/test-lib.sh | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/t/test-lib.sh b/t/test-lib.sh
index 4e45bc7c4f..b18cd8a34e 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -56,6 +56,20 @@ then
 	echo "PANIC: Running in a $TEST_DIRECTORY that doesn't end in '/t'?" >&2
 	exit 1
 fi
+if test -z "$GIT_BUNDLE_DIR"
+then
+	# Similarly, override this to store the documentation bundles subdir
+	# elsewhere
+	GIT_BUNDLE_DIR=$TEST_DIRECTORY/GIT_BUNDLE_DIR
+	if test ! -d "$GIT_BUNDLE_DIR"
+	then
+		mkdir -p "$GIT_BUNDLE_DIR"
+	fi
+elif test ! -d "$GIT_BUNDLE_DIR"
+then
+	echo "PANIC: Your "$GIT_BUNDLE_DIR" doesn't exist!" >&2
+	exit 1
+fi
 
 # Prepend a string to a VAR using an arbitrary ":" delimiter, not
 # adding the delimiter if VAR or VALUE is empty. I.e. a generalized:
-- 
2.38.1.281.g83ef3ded8d


^ permalink raw reply related	[flat|nested] 15+ messages in thread

* [RFC/PoC 4/5] Doc Bundles: add t6102 rev-list simple and pulls examples
  2022-11-02 22:48 [RFC/PoC 0/5] Provide example docbundles Philip Oakley
                   ` (2 preceding siblings ...)
  2022-11-02 22:48 ` [RFC/PoC 3/5] test-lib: add GIT_BUNDLE_DIR setup Philip Oakley
@ 2022-11-02 22:48 ` Philip Oakley
  2022-11-02 22:48 ` [RFC/PoC 5/5] docbundle: provide the git-log History Simplification example Philip Oakley
  4 siblings, 0 replies; 15+ messages in thread
From: Philip Oakley @ 2022-11-02 22:48 UTC (permalink / raw)
  To: GitList; +Cc: Self, Junio C Hamano, Taylor Blau

The choice of the t6012-rev-list-simplify examples as the initial
examples is somewhat arbitrary, and based on having the example
available in my editor at the time.

The environment variable GIT_BUILD_DIR is expected to contain
the absolute path to that directory. Later commits will set that up
in the test-lib-setup.

Signed-off-by: Philip Oakley <philipoakley@iee.email>
---
 t/t6012-rev-list-simplify.sh | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/t/t6012-rev-list-simplify.sh b/t/t6012-rev-list-simplify.sh
index de1e87f162..8931102392 100755
--- a/t/t6012-rev-list-simplify.sh
+++ b/t/t6012-rev-list-simplify.sh
@@ -101,6 +101,10 @@ test_expect_success setup '
 	note L
 '
 
+test_expect_success 'bundle simple repo' '
+	git bundle create ${GIT_BUNDLE_DIR}/t6102-rev-list-simple.bndl --all
+'
+
 FMT='tformat:%P 	%H | %s'
 
 check_outcome () {
@@ -240,6 +244,10 @@ test_expect_success 'setup rebuild repo' '
 	note P
 '
 
+test_expect_success 'bundle pulls repo' '
+	git bundle create ${GIT_BUNDLE_DIR}/t6102-rev-list-pulls.bndl --all
+'
+
 check_result 'X I' -- file
 check_result 'N R X I' --show-pulls -- file
 
-- 
2.38.1.281.g83ef3ded8d


^ permalink raw reply related	[flat|nested] 15+ messages in thread

* [RFC/PoC 5/5] docbundle: provide the git-log History Simplification example
  2022-11-02 22:48 [RFC/PoC 0/5] Provide example docbundles Philip Oakley
                   ` (3 preceding siblings ...)
  2022-11-02 22:48 ` [RFC/PoC 4/5] Doc Bundles: add t6102 rev-list simple and pulls examples Philip Oakley
@ 2022-11-02 22:48 ` Philip Oakley
  2022-11-03  9:17   ` Ævar Arnfjörð Bjarmason
  4 siblings, 1 reply; 15+ messages in thread
From: Philip Oakley @ 2022-11-02 22:48 UTC (permalink / raw)
  To: GitList; +Cc: Self, Junio C Hamano, Taylor Blau

Create the History Simplification example and bundle it.

ToDo: add the tests that the man page attempts to demonstrate.

add the effects of `--graph` to the tests and man page.

/* NEEDSWORK */

Signed-off-by: Philip Oakley <philipoakley@iee.email>
---
 t/t9990-history-simplification.sh | 261 ++++++++++++++++++++++++++++++
 1 file changed, 261 insertions(+)
 create mode 100644 t/t9990-history-simplification.sh

diff --git a/t/t9990-history-simplification.sh b/t/t9990-history-simplification.sh
new file mode 100644
index 0000000000..ada06698d5
--- /dev/null
+++ b/t/t9990-history-simplification.sh
@@ -0,0 +1,261 @@
+#!/bin/sh
+
+test_description='History simplification man page'
+
+GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
+export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
+
+. ./test-lib.sh
+
+note () {
+	git tag "$1"
+}
+
+unnote () {
+	test_when_finished "rm -f tmp" &&
+	git name-rev --tags --annotate-stdin >tmp &&
+	sed -e "s|$OID_REGEX (tags/\([^)]*\)) |\1 |g" <tmp
+}
+
+#
+# Create a test repo with the man(3) History Simplification commit graph:
+#
+# We assume, for the TREESAME check,
+# that you are filtering for a file `foo` in this commit graph:
+#-----------------------------------------------------------------------
+#	  .-A---M---N---O---P---Q
+#	 /     /   /   /   /   /
+#	I     B   C   D   E   Y
+#	 \   /   /   /   /   /
+#	  `-------------'   X
+#-----------------------------------------------------------------------
+#The horizontal line of history A---Q is taken to be the first parent of
+#each merge.
+#
+# See Documentation/rev-list-options.txt
+# see https://groups.google.com/d/msgid/git-users/11286f4a-c961-4d12-91a0-f49e34d8691bn%40googlegroups.com. for git-user discussion.
+# https://github.com/ChuckTest/git-history-test
+# C:\git-sdk-64\usr\src\git-history-test
+
+test_expect_success setup '
+	# get the structure set out, then the file contents
+	echo "asdf" >foo &&
+	echo "quux" >quux &&
+	git add foo quux &&
+	test_tick && git commit -m "I: Initial commit" &&
+	note I &&
+
+	git branch B &&
+	git branch C &&
+	git branch D &&
+	git branch E &&
+
+	echo "foo" >foo &&
+	git add foo &&
+	test_tick && git commit -m "A" &&
+	note A &&
+
+	git checkout B &&
+	echo "foo" >foo &&
+	git add foo &&
+	test_tick && git commit -m "B" &&
+	note B &&
+
+	git checkout main &&
+	test_tick && git merge B -m "M: trivial merge" &&
+	note M &&
+
+	git checkout C &&
+	# is this needed?
+	#echo "other" >other &&
+	#git add other &&
+	test_tick && git commit --allow-empty -m "C: identical to I" &&
+	note C &&
+
+	git checkout main &&
+	test_tick && git merge C --no-commit -m "N: !TREESAME any parent" &&
+	echo "foobar" >foo &&
+	git add foo &&
+	git commit -m "N: !TREESAME any parent" &&
+	note N &&
+
+	git checkout D &&
+	echo "baz" >foo &&
+	git add foo &&
+	test_tick && git commit -m "D" &&
+	note D &&
+
+	git checkout main &&
+	test_tick &&
+	test_must_fail git merge D --no-commit &&
+	echo "foobarbaz" >foo &&
+	git add foo &&
+	git commit -m "O: !TREESAME any parent" &&
+	note O &&
+
+	git checkout E &&
+	echo "xyzzy" >quux &&
+	git add quux &&
+	test_tick && git commit -m "E" &&
+	note E &&
+
+	git checkout main &&
+	test_tick &&
+	git merge E --no-commit &&
+	echo "quux xyzzy" >quux &&
+	git add quux &&
+	git commit -m "P: TREESAME O; !E" &&
+	note S &&
+
+	git checkout --orphan Side &&
+	git rm -f foo quux &&
+	echo "initial" >side &&
+	git add side &&
+	git commit -m "X" &&
+	note X &&
+
+	echo "side" >side &&
+	git add side &&
+	git commit -m "Y" &&
+	note Y &&
+
+	git checkout main &&
+	test_tick && 
+	git merge Side --allow-unrelated-histories -m "Q: TREESAME P; !Y" &&
+	note Q
+'
+
+test_expect_success 'bundle history-simplification repo' '
+	git bundle create ${GIT_BUNDLE_DIR}/t9990-history-simplification.bndl --all
+'
+
+# Test it!
+# `rev-list` walks backwards through history, including or excluding
+# commits based on whether `--full-history` and/or parent rewriting
+# (via `--parents` or `--children`) are used. The following settings
+# are available.
+# 
+# Default mode::
+# 	Commits are included if they are not TREESAME to any parent
+# 	(though this can be changed, see `--sparse` below).  If the
+# 	commit was a merge, and it was TREESAME to one parent, follow
+# 	only that parent.  (Even if there are several TREESAME
+# 	parents, follow only one of them.)  Otherwise, follow all
+# 	parents.
+# +
+# This results in:
+# +
+# -----------------------------------------------------------------------
+# 	  .-A---N---O
+# 	 /     /   /
+# 	I---------D
+# -----------------------------------------------------------------------
+# +
+# Note how the rule to only follow the TREESAME parent, if one is
+# available, removed `B` from consideration entirely.  `C` was
+# considered via `N`, but is TREESAME.  Root commits are compared to an
+# empty tree, so `I` is !TREESAME.
+# +
+# Parent/child relations are only visible with `--parents`, but that does
+# not affect the commits selected in default mode, so we have shown the
+# parent lines.
+# 
+# --full-history without parent rewriting::
+# 	This mode differs from the default in one point: always follow
+# 	all parents of a merge, even if it is TREESAME to one of them.
+# 	Even if more than one side of the merge has commits that are
+# 	included, this does not imply that the merge itself is!  In
+# 	the example, we get
+# +
+# -----------------------------------------------------------------------
+# 	I  A  B  N  D  O  P  Q
+# -----------------------------------------------------------------------
+# +
+# `M` was excluded because it is TREESAME to both parents.  `E`,
+# `C` and `B` were all walked, but only `B` was !TREESAME, so the others
+# do not appear.
+# +
+# Note that without parent rewriting, it is not really possible to talk
+# about the parent/child relationships between the commits, so we show
+# them disconnected.
+# 
+# --full-history with parent rewriting::
+# 	Ordinary commits are only included if they are !TREESAME
+# 	(though this can be changed, see `--sparse` below).
+# +
+# Merges are always included.  However, their parent list is rewritten:
+# Along each parent, prune away commits that are not included
+# themselves.  This results in
+# +
+# -----------------------------------------------------------------------
+# 	  .-A---M---N---O---P---Q
+# 	 /     /   /   /   /
+# 	I     B   /   D   /
+# 	 \   /   /   /   /
+# 	  `-------------'
+# -----------------------------------------------------------------------
+# +
+# Compare to `--full-history` without rewriting above.  Note that `E`
+# was pruned away because it is TREESAME, but the parent list of P was
+# rewritten to contain `E`'s parent `I`.  The same happened for `C` and
+# `N`, and `X`, `Y` and `Q`.
+# 
+# In addition to the above settings, you can change whether TREESAME
+# affects inclusion:
+# 
+# --dense::
+# 	Commits that are walked are included if they are not TREESAME
+# 	to any parent.
+# 
+# --sparse::
+# 	All commits that are walked are included.
+# +
+# Note that without `--full-history`, this still simplifies merges: if
+# one of the parents is TREESAME, we follow only that one, so the other
+# sides of the merge are never walked.
+# 
+# --simplify-merges::
+# 	First, build a history graph in the same way that
+# 	`--full-history` with parent rewriting does (see above).
+# +
+# Then simplify each commit `C` to its replacement `C'` in the final
+# history according to the following rules:
+# +
+# --
+# * Set `C'` to `C`.
+# +
+# * Replace each parent `P` of `C'` with its simplification `P'`.  In
+#   the process, drop parents that are ancestors of other parents or that are
+#   root commits TREESAME to an empty tree, and remove duplicates, but take care
+#   to never drop all parents that we are TREESAME to.
+# +
+# * If after this parent rewriting, `C'` is a root or merge commit (has
+#   zero or >1 parents), a boundary commit, or !TREESAME, it remains.
+#   Otherwise, it is replaced with its only parent.
+# --
+# +
+# The effect of this is best shown by way of comparing to
+# `--full-history` with parent rewriting.  The example turns into:
+# +
+# -----------------------------------------------------------------------
+# 	  .-A---M---N---O
+# 	 /     /       /
+# 	I     B       D
+# 	 \   /       /
+# 	  `---------'
+# -----------------------------------------------------------------------
+# +
+# Note the major differences in `N`, `P`, and `Q` over `--full-history`:
+# +
+# --
+# * `N`'s parent list had `I` removed, because it is an ancestor of the
+#   other parent `M`.  Still, `N` remained because it is !TREESAME.
+# +
+# * `P`'s parent list similarly had `I` removed.  `P` was then
+#   removed completely, because it had one parent and is TREESAME.
+# +
+# * `Q`'s parent list had `Y` simplified to `X`. `X` was then removed, because it
+#   was a TREESAME root. `Q` was then removed completely, because it had one
+#   parent and is TREESAME.
+
+test_done
-- 
2.38.1.281.g83ef3ded8d


^ permalink raw reply related	[flat|nested] 15+ messages in thread

* Re: [RFC/PoC 2/5] Documentation Bundles: ignore artifact files from creating doc bundles
  2022-11-02 22:48 ` [RFC/PoC 2/5] Documentation Bundles: ignore artifact files from creating doc bundles Philip Oakley
@ 2022-11-03  9:12   ` Ævar Arnfjörð Bjarmason
  2022-11-03 19:50     ` Philip Oakley
  0 siblings, 1 reply; 15+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2022-11-03  9:12 UTC (permalink / raw)
  To: Philip Oakley; +Cc: GitList, Junio C Hamano, Taylor Blau


On Wed, Nov 02 2022, Philip Oakley wrote:

> The Git documentation contains many examples that can be prepared as
> repository bundles for users to clone and explore without any language
> or translation confusions between the written word and intended code.

I just find this commit entirely confusing, we're adding .gitignore
entries, but nothing generates these yet (but I'll read on). All to
describe some seeming future feature...

> There are also a large number of example repositories in the t/ test
> system can can also be exposed to users to highlight the critical
> distinctions tested therein.
>
> These documentation bundles will be prepared ar part of the test system.

s/ar/as/

> Start by ignoring those parts which will be considered build artifacts,
> the GIT-BUNDLE-DIR, and the `.bndl` bundle files themselves.
>
> The artifacts themselves will later be manipulated by the `make` elements
> of the documentation system.
>
> Signed-off-by: Philip Oakley <philipoakley@iee.email>
> ---
>  .gitignore | 2 ++
>  1 file changed, 2 insertions(+)
>
> diff --git a/.gitignore b/.gitignore
> index 6ded10067a..b96a4ce5df 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -3,6 +3,7 @@
>  /fuzz-pack-headers
>  /fuzz-pack-idx
>  /GIT-BUILD-OPTIONS
> +/GIT-BUNDLE-DIR
>  /GIT-CFLAGS
>  /GIT-LDFLAGS
>  /GIT-PREFIX
> @@ -250,3 +251,4 @@ Release/
>  *.dSYM
>  /contrib/buildsystems/out
>  CMakeSettings.json
> +*.bndl

We've been putting quite a few things in .build/ recently, so whatever
this is, if you put it in .build/test-bundles/ or whatever you don't
need to write "clean" rules for it, or add this ignore entry.

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [RFC/PoC 3/5] test-lib: add GIT_BUNDLE_DIR setup
  2022-11-02 22:48 ` [RFC/PoC 3/5] test-lib: add GIT_BUNDLE_DIR setup Philip Oakley
@ 2022-11-03  9:14   ` Ævar Arnfjörð Bjarmason
  2022-11-03 19:50     ` Philip Oakley
  0 siblings, 1 reply; 15+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2022-11-03  9:14 UTC (permalink / raw)
  To: Philip Oakley; +Cc: GitList, Junio C Hamano, Taylor Blau


On Wed, Nov 02 2022, Philip Oakley wrote:

> The test system may output a number of documentation bundles containing
> test repositories of interest to users, including those explicitly
> documented.
>
> Provide a default location for holding those bundles.
> As build artefact's this location, along with `.bndl` files, was
> ignored in an earlier commit.
>
> This may need further refinement.
>
> Signed-off-by: Philip Oakley <philipoakley@iee.email>
> ---
>  t/test-lib.sh | 14 ++++++++++++++
>  1 file changed, 14 insertions(+)
>
> diff --git a/t/test-lib.sh b/t/test-lib.sh
> index 4e45bc7c4f..b18cd8a34e 100644
> --- a/t/test-lib.sh
> +++ b/t/test-lib.sh
> @@ -56,6 +56,20 @@ then
>  	echo "PANIC: Running in a $TEST_DIRECTORY that doesn't end in '/t'?" >&2
>  	exit 1
>  fi
> +if test -z "$GIT_BUNDLE_DIR"
> +then
> +	# Similarly, override this to store the documentation bundles subdir
> +	# elsewhere
> +	GIT_BUNDLE_DIR=$TEST_DIRECTORY/GIT_BUNDLE_DIR
> +	if test ! -d "$GIT_BUNDLE_DIR"
> +	then
> +		mkdir -p "$GIT_BUNDLE_DIR"

Okey, so "mkdir -p", not "mkdir" once one thinks about it for a second:
Because we might be testing in parallel, so this will be racy.

> +	fi
> +elif test ! -d "$GIT_BUNDLE_DIR"
> +then
> +	echo "PANIC: Your "$GIT_BUNDLE_DIR" doesn't exist!" >&2
> +	exit 1
> +fi

I think it's better just to put these in the "test-results" dir.

For leak testing I have local patches to e.g. create test-results/<test
number>.leak, and then after a run I slurp that up and process it.

So, same idea, you could just add stuff there, and afterwards grab it
there. No need for specifying the dir etc., you just need a test-lib.sh
flag to say "yes, write these out".

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [RFC/PoC 5/5] docbundle: provide the git-log History Simplification example
  2022-11-02 22:48 ` [RFC/PoC 5/5] docbundle: provide the git-log History Simplification example Philip Oakley
@ 2022-11-03  9:17   ` Ævar Arnfjörð Bjarmason
  2022-11-03 19:50     ` Philip Oakley
  0 siblings, 1 reply; 15+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2022-11-03  9:17 UTC (permalink / raw)
  To: Philip Oakley; +Cc: GitList, Junio C Hamano, Taylor Blau


On Wed, Nov 02 2022, Philip Oakley wrote:

> +note () {
> +	git tag "$1"
> +}

Aren't you just...

> +test_expect_success setup '
> +	# get the structure set out, then the file contents
> +	echo "asdf" >foo &&
> +	echo "quux" >quux &&
> +	git add foo quux &&
> +	test_tick && git commit -m "I: Initial commit" &&
> +	note I &&

...re-inventing "test_commit" here (including the "echo")?

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [RFC/PoC 1/5] doc: provide DocBundles design document
  2022-11-02 22:48 ` [RFC/PoC 1/5] doc: provide DocBundles design document Philip Oakley
@ 2022-11-03  9:19   ` Ævar Arnfjörð Bjarmason
  2022-11-03 19:50     ` Philip Oakley
  0 siblings, 1 reply; 15+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2022-11-03  9:19 UTC (permalink / raw)
  To: Philip Oakley; +Cc: GitList, Junio C Hamano, Taylor Blau


On Wed, Nov 02 2022, Philip Oakley wrote:

> [...]
> diff --git a/Documentation/technical/docbundle.txt b/Documentation/technical/docbundle.txt
> new file mode 100644
> index 0000000000..8c17a7847f
> --- /dev/null
> +++ b/Documentation/technical/docbundle.txt
> @@ -0,0 +1,97 @@
> +docbundle Design Document
> +=========================
> +
> +The Git man pages contain many example setups to demonstrate aspects
> +of the related Git commands, such as history simplification in the `log`
> +and `show commands. However, while illustrative, these are not
> +accessible, as example repositories, to regular, potentially confused,
> +users who may need to understand the set up of the examples.
> +
> +The docbundle concept is that for each of these examples there is a
> +matching `bundle` file, provided with the Documentation, which can be
> +simply cloned to provide an example repository containing the example.
> +
> +There are also many example repositories in the Git `/t` test system
> +that could be exposed to users which already demonstrate some important
> +testable aspect of Git within a carefully constructed repository.
> +
> +
> +This proposal is to provide DocBundles to users via the Git project.
> +
> +
> +Firstly, it should be reasonably self-evident that the man page examples
> +should, in themselves, be tested for accuracy, especially if provided
> +to users.
> +
> +Secondly, the existing test system is the proper place to setup and
> +test any such docbundle repositories.
> +
> +Third, that the bundle is an obvious pre-existing mechanism for
> +transporting repositories (see also the Bundle-URI proposals).
> +
> +Fourthly, the preparation and distribution of the docbundle should
> +require minimal maintainer effort, preferably being integrated into
> +the existing automation for Documentation preparation and distribution.
> +
> +Fifthly, the docbudles themselves should not to be part of the git.git
> +repo. They are prepared via the test suite and should be stored (locally)
> +in a designated Git_DocBundle_Dir, and be .gitignored. The DocBundles
> +are transferred to the maintainer's Doc repo via the automation.
> +
> +Sixthly, the docbundles to be named based on their `tnnnn` test number
> +and a (short) descriptive name.
> +
> +Seven, (dev) start with an existing test, rather than add a test (most
> +doc/man examples aren't tested!). Add tests for missing examples later.
> +
> +Eight, (option) add a `git docbundle` command to assist in locating and
> +listing the docbundle examples. It also provides place to explain their
> +purpose as "Tutorial Examples" (Note, many existing examples are to
> +illustrate complex/difficult scenarios where the wording was insufficient;
> +the docbudles are minimal examples).
> +
> +
> +Status
> +======
> +
> +This is an RFC/PoC (Request for Comment / Proof of Concept).
> +
> +The attached commits establish:
> +
> +Set out this design in the Documentation/technical directory.
> +
> +A default location for the docbundles and an env variable to allow
> +flexibility by extending test-lib.
> +
> +The docbundles are ignored, as is the default directory.
> +
> +Two docbundles are generated from a 'random' test that I happened to
> +have open in my editor (t6102 rev-list-simplify).
> +
> +The docbundle creation is protected by `test_expect_success` wrappers.
> +
> +ToDo
> +====
> +
> +DocBundles - Is this the best name, or just a convenient way of
> +indicating the delivery mechanism?
> +
> +Add Prerequisite to control if docbundles are generated (don't waste
> +maintainers PC time when not needed).
> +
> +CI integration (i.e. whether to test the bundling aspect - should be
> +reliable).
> +
> +How to update the man pages, perhaps by inclusion of the test comment
> +(exported?), and distinguish between man page descriptions (being
> +packaged by the test suite) and tests which offer users insight into
> +Git operations, usually with 'interesting' commit-graph set-ups.
> +
> +Claim t999x as the man doc bundle test sequence ('999' is the UK
> +emergency ("Help!") number; 911, 101 and 102 are taken test areas.
> +
> +Other stuff implicit in the eight point list

I've skimmed this, and provided some rough comments inline.

I think the core idea sounds nice, if I'm getting it right. Let me see
if I can rephrase it, and you can point out where I'm wrong/didn't get
it:

We have stuff in our docs like e.g. this, at the top of git-rebase(1);

       Assume the following history exists and the current branch is "topic":

                     A---B---C topic
                    /
               D---E---F---G master

It would be nice if for that and other things we could add a blurb like:

	To play around with this history, do:

		git clone git-doc://git-rebase/description-1.bndl && cd description-1

Or whatever, this POC doesn't fleshen that out, but the idea is that
through <some mechanism> we would bridge the gap such that the user
could be looking at and playing with the history we're describing in our
docs.

The above assumes we'd ship those with installed git, but these could
also be hosted online, e.g.:

	git clone https://git-scm.com/test-bundles/git-rebase/description-1.bdl

So as an end-goal you could both run these locally, and we could imagine
that e.g. git-scm.com could eventually have UX similar to Redis's
website: https://redis.io/commands/hgetall/

I.e. where it drops you into a shell to play with the command.

In reply to 3/5 I said I had some "local patches", but I forgot the
relevant part was on master already (the whole subsequent leak analysis
is local): 366bd129dc3 (test-lib: add a SANITIZE=leak logging mode,
2022-07-28)

So I think you're making the whole "have the test suite make these" part
of this too complex, i.e. with a minor variation of that you can do this
in test-lib.sh:

	TEST_RESULTS_BDL_DIR="$TEST_RESULTS_DIR/$TEST_NAME.test-bdl"

Then have a helper like:

	do_my_test_bundle_stuff () {
		local name="$1" &&
		shift &&
		git bundle create "$TEST_RESULTS_BDL_DIR/$name.bdl" "$@"
	}

Then all you need to make these is:

	make test GIT_TEST_OPTS=--debug

And then:

	ls -l t/test-results/*.bdl/*

Will give you all the bundles you want.

We're then missing the "connect it with the docs" part, but that part
would be e.g.:

- Have a validation target for it that depends on such a "make test" run
- Sanity check that we did create the ones the docs expect

Anyway.

I like the end-goals, but I can't help think that bundles are the wrong
direction to go in, as opposed to:

1. We could have "make install" ship test-lib{,-functions}.sh along
   with git itself
2. Have e.g. t/doc-setup/*.sh with the snippets to set up these test
   demos, which we'd also ship.

And you'd then do e.g.:

	git test-case rebase/description-1

Which would just be a thin wrapper around (pseudocode):

	dir=$(git test--helper get-tempdir) &&
	(
		cd "$dir" &&
                # And whatever else we need to set up to say "use this local installed git"
		GIT_TEST_INSTALLED=[...] &&
		. "$(git --test-dir)"/test-lib.sh
		. "$(git --test-dir)"/doc-setup/rebase-description-1.sh"

		# This would run test_commit, and any other arbitrary setup code
		setup_rebase_description_1()
	) &&
	echo "Go play with '$dir', and have fun"

The big advantage of that is:

 1. It ships with git, so it works offline, and we won't get into the
    inevitable confusion of the user reading v2.40 docs locally, but the
    online url tracking a v2.50 example or whatever.
 2. We can represent whatever arbitrary repo state, e.g. drop the user
    into a merge conflict that's already happening.
 3. We can also represent "repo-extra" state, e.g. imagine the
    "git-gc/git-repack" docs discussing loose objects, and wanting to
    get you into some arbitrary state with regards to your loose objects
    etc.

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [RFC/PoC 1/5] doc: provide DocBundles design document
  2022-11-03  9:19   ` Ævar Arnfjörð Bjarmason
@ 2022-11-03 19:50     ` Philip Oakley
  2022-11-03 20:36       ` Ævar Arnfjörð Bjarmason
  0 siblings, 1 reply; 15+ messages in thread
From: Philip Oakley @ 2022-11-03 19:50 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: GitList, Junio C Hamano, Taylor Blau

Hi Ævar,
Thanks for the review,

On 03/11/2022 09:19, Ævar Arnfjörð Bjarmason wrote:
> On Wed, Nov 02 2022, Philip Oakley wrote:
>
>> [...]
>> diff --git a/Documentation/technical/docbundle.txt b/Documentation/technical/docbundle.txt
>> new file mode 100644
>> index 0000000000..8c17a7847f
>> --- /dev/null
>> +++ b/Documentation/technical/docbundle.txt
>> @@ -0,0 +1,97 @@
>> +docbundle Design Document
>> +=========================
>> +
>> +The Git man pages contain many example setups to demonstrate aspects
>> +of the related Git commands, such as history simplification in the `log`
>> +and `show commands. However, while illustrative, these are not
>> +accessible, as example repositories, to regular, potentially confused,
>> +users who may need to understand the set up of the examples.
>> +
>> +The docbundle concept is that for each of these examples there is a
>> +matching `bundle` file, provided with the Documentation, which can be
>> +simply cloned to provide an example repository containing the example.
>> +
>> +There are also many example repositories in the Git `/t` test system
>> +that could be exposed to users which already demonstrate some important
>> +testable aspect of Git within a carefully constructed repository.
>> +
>> +
>> +This proposal is to provide DocBundles to users via the Git project.
>> +
>> +
>> +Firstly, it should be reasonably self-evident that the man page examples
>> +should, in themselves, be tested for accuracy, especially if provided
>> +to users.
>> +
>> +Secondly, the existing test system is the proper place to setup and
>> +test any such docbundle repositories.
>> +
>> +Third, that the bundle is an obvious pre-existing mechanism for
>> +transporting repositories (see also the Bundle-URI proposals).
>> +
>> +Fourthly, the preparation and distribution of the docbundle should
>> +require minimal maintainer effort, preferably being integrated into
>> +the existing automation for Documentation preparation and distribution.
>> +
>> +Fifthly, the docbudles themselves should not to be part of the git.git
>> +repo. They are prepared via the test suite and should be stored (locally)
>> +in a designated Git_DocBundle_Dir, and be .gitignored. The DocBundles
>> +are transferred to the maintainer's Doc repo via the automation.
>> +
>> +Sixthly, the docbundles to be named based on their `tnnnn` test number
>> +and a (short) descriptive name.
>> +
>> +Seven, (dev) start with an existing test, rather than add a test (most
>> +doc/man examples aren't tested!). Add tests for missing examples later.
>> +
>> +Eight, (option) add a `git docbundle` command to assist in locating and
>> +listing the docbundle examples. It also provides place to explain their
>> +purpose as "Tutorial Examples" (Note, many existing examples are to
>> +illustrate complex/difficult scenarios where the wording was insufficient;
>> +the docbudles are minimal examples).
>> +
>> +
>> +Status
>> +======
>> +
>> +This is an RFC/PoC (Request for Comment / Proof of Concept).
>> +
>> +The attached commits establish:
>> +
>> +Set out this design in the Documentation/technical directory.
>> +
>> +A default location for the docbundles and an env variable to allow
>> +flexibility by extending test-lib.
>> +
>> +The docbundles are ignored, as is the default directory.
>> +
>> +Two docbundles are generated from a 'random' test that I happened to
>> +have open in my editor (t6102 rev-list-simplify).
>> +
>> +The docbundle creation is protected by `test_expect_success` wrappers.
>> +
>> +ToDo
>> +====
>> +
>> +DocBundles - Is this the best name, or just a convenient way of
>> +indicating the delivery mechanism?
>> +
>> +Add Prerequisite to control if docbundles are generated (don't waste
>> +maintainers PC time when not needed).
>> +
>> +CI integration (i.e. whether to test the bundling aspect - should be
>> +reliable).
>> +
>> +How to update the man pages, perhaps by inclusion of the test comment
>> +(exported?), and distinguish between man page descriptions (being
>> +packaged by the test suite) and tests which offer users insight into
>> +Git operations, usually with 'interesting' commit-graph set-ups.
>> +
>> +Claim t999x as the man doc bundle test sequence ('999' is the UK
>> +emergency ("Help!") number; 911, 101 and 102 are taken test areas.
>> +
>> +Other stuff implicit in the eight point list
> I've skimmed this, and provided some rough comments inline.
>
> I think the core idea sounds nice, if I'm getting it right. Let me see
> if I can rephrase it, and you can point out where I'm wrong/didn't get
> it:
>
> We have stuff in our docs like e.g. this, at the top of git-rebase(1);
>
>        Assume the following history exists and the current branch is "topic":
>
>                      A---B---C topic
>                     /
>                D---E---F---G master
>
> It would be nice if for that and other things we could add a blurb like:
>
> 	To play around with this history, do:
>
> 		git clone git-doc://git-rebase/description-1.bndl && cd description-1

That is generically correct. The motivating example was History
Simplification (see 5/5), where the example needed to cover a range of
setups which then confused new users in a catch 22 manner (link in the
'PoC patch). I had been aware of the issue previously, when looking at
the support I'd seen in some other tools which have example setups.
>
> Or whatever, this POC doesn't fleshen that out, but the idea is that
> through <some mechanism> we would bridge the gap such that the user
> could be looking at and playing with the history we're describing in our
> docs.

In terms of fleshing the PoC out, I didn't want to try stepping on too
many toes too soon, rather wanted to pick out a few pinch points with
suggestions.

>
> The above assumes we'd ship those with installed git, but these could
> also be hosted online, e.g.:
>
> 	git clone https://git-scm.com/test-bundles/git-rebase/description-1.bdl
>
> So as an end-goal you could both run these locally, and we could imagine
> that e.g. git-scm.com 

I was assuming that the bundles would be static documentation artefacts,
available locally and potentially on-line. They could even be hosted on
the forges..

> could eventually have UX similar to Redis's
> website: https://redis.io/commands/hgetall/
>
> I.e. where it drops you into a shell to play with the command.
I wasn't expecting to have a virtualised on-line git experience from
this, but never say never ;-)
> In reply to 3/5 I said I had some "local patches", but I forgot the
> relevant part was on master already (the whole subsequent leak analysis
> is local): 366bd129dc3 (test-lib: add a SANITIZE=leak logging mode,
> 2022-07-28)
>
> So I think you're making the whole "have the test suite make these" part
> of this too complex,

My main point was that the examples need to be covered by the test
suite, and that bundles were a simple way of capturing the tested setup.
The GIT_BUNDLE_DIR was simply a way of collecting them in one place for
ease of distribution with the documentation (as you mentioned that's a
gap in the automation step)
>  i.e. with a minor variation of that you can do this
> in test-lib.sh:
>
> 	TEST_RESULTS_BDL_DIR="$TEST_RESULTS_DIR/$TEST_NAME.test-bdl"
>
> Then have a helper like:
>
> 	do_my_test_bundle_stuff () {
> 		local name="$1" &&
> 		shift &&
> 		git bundle create "$TEST_RESULTS_BDL_DIR/$name.bdl" "$@"
> 	}
That would be a simplification
>
> Then all you need to make these is:
>
> 	make test GIT_TEST_OPTS=--debug

I didn't quite get the link in that step between the `--debug` and the
bndl prep.
>
> And then:
>
> 	ls -l t/test-results/*.bdl/*
>
> Will give you all the bundles you want.
>
> We're then missing the "connect it with the docs" part, but that part
> would be e.g.:
>
> - Have a validation target for it that depends on such a "make test" run
> - Sanity check that we did create the ones the docs expect
>
> Anyway.
>
> I like the end-goals, but I can't help think that bundles are the wrong
> direction to go in, as opposed to:
>
> 1. We could have "make install" ship test-lib{,-functions}.sh along
>    with git itself
> 2. Have e.g. t/doc-setup/*.sh with the snippets to set up these test
>    demos, which we'd also ship.

I'm not into the idea of shipping a load of extra `test` infrastructure
with Git itself. We don't ship any in the Git for Windows install.
 
I'd viewed the bundles as an adjunct to the also independent manuals and
documentation.
>
> And you'd then do e.g.:
>
> 	git test-case rebase/description-1
>
> Which would just be a thin wrapper around (pseudocode):
>
> 	dir=$(git test--helper get-tempdir) &&
> 	(
> 		cd "$dir" &&
>                 # And whatever else we need to set up to say "use this local installed git"
> 		GIT_TEST_INSTALLED=[...] &&
> 		. "$(git --test-dir)"/test-lib.sh
> 		. "$(git --test-dir)"/doc-setup/rebase-description-1.sh"
>
> 		# This would run test_commit, and any other arbitrary setup code
> 		setup_rebase_description_1()
> 	) &&
> 	echo "Go play with '$dir', and have fun"
>
> The big advantage of that is:
>
>  1. It ships with git, so it works offline, and we won't get into the
>     inevitable confusion of the user reading v2.40 docs locally, but the
>     online url tracking a v2.50 example or whatever.

Isn't this always a problem (lack of synchronisation between multiple
sources). If the user has local documentation, it should have matching
bundles locally, No?
>  2. We can represent whatever arbitrary repo state, e.g. drop the user
>     into a merge conflict that's already happening.
Isn't the manual's example already meant to describe that? And the user
then (hopefully) follows along in their throw-away copy of the repo bundle.
>  3. We can also represent "repo-extra" state, e.g. imagine the
>     "git-gc/git-repack" docs discussing loose objects, and wanting to
>     get you into some arbitrary state with regards to your loose objects
>     etc.
That's a future option, but way past the initial concept for user
on-boarding.

Thanks

Philip


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [RFC/PoC 2/5] Documentation Bundles: ignore artifact files from creating doc bundles
  2022-11-03  9:12   ` Ævar Arnfjörð Bjarmason
@ 2022-11-03 19:50     ` Philip Oakley
  0 siblings, 0 replies; 15+ messages in thread
From: Philip Oakley @ 2022-11-03 19:50 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: GitList, Junio C Hamano, Taylor Blau

On 03/11/2022 09:12, Ævar Arnfjörð Bjarmason wrote:
> On Wed, Nov 02 2022, Philip Oakley wrote:
>
>> The Git documentation contains many examples that can be prepared as
>> repository bundles for users to clone and explore without any language
>> or translation confusions between the written word and intended code.
> I just find this commit entirely confusing, we're adding .gitignore
> entries, but nothing generates these yet (but I'll read on). All to
> describe some seeming future feature...

Probably could be squashed in with the others once the PoC has matured a
bit.
>
>> There are also a large number of example repositories in the t/ test
>> system can can also be exposed to users to highlight the critical
>> distinctions tested therein.
>>
>> These documentation bundles will be prepared ar part of the test system.
> s/ar/as/

Ok
>
>> Start by ignoring those parts which will be considered build artifacts,
>> the GIT-BUNDLE-DIR, and the `.bndl` bundle files themselves.
>>
>> The artifacts themselves will later be manipulated by the `make` elements
>> of the documentation system.
>>
>> Signed-off-by: Philip Oakley <philipoakley@iee.email>
>> ---
>>  .gitignore | 2 ++
>>  1 file changed, 2 insertions(+)
>>
>> diff --git a/.gitignore b/.gitignore
>> index 6ded10067a..b96a4ce5df 100644
>> --- a/.gitignore
>> +++ b/.gitignore
>> @@ -3,6 +3,7 @@
>>  /fuzz-pack-headers
>>  /fuzz-pack-idx
>>  /GIT-BUILD-OPTIONS
>> +/GIT-BUNDLE-DIR
>>  /GIT-CFLAGS
>>  /GIT-LDFLAGS
>>  /GIT-PREFIX
>> @@ -250,3 +251,4 @@ Release/
>>  *.dSYM
>>  /contrib/buildsystems/out
>>  CMakeSettings.json
>> +*.bndl
> We've been putting quite a few things in .build/ recently, so whatever
> this is, if you put it in .build/test-bundles/ or whatever you don't
> need to write "clean" rules for it, or add this ignore entry.
Ok. I'd also hadn't considered the 'clean' rules for the make file.

Philip

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [RFC/PoC 3/5] test-lib: add GIT_BUNDLE_DIR setup
  2022-11-03  9:14   ` Ævar Arnfjörð Bjarmason
@ 2022-11-03 19:50     ` Philip Oakley
  0 siblings, 0 replies; 15+ messages in thread
From: Philip Oakley @ 2022-11-03 19:50 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: GitList, Junio C Hamano, Taylor Blau

On 03/11/2022 09:14, Ævar Arnfjörð Bjarmason wrote:
> On Wed, Nov 02 2022, Philip Oakley wrote:
>
>> The test system may output a number of documentation bundles containing
>> test repositories of interest to users, including those explicitly
>> documented.
>>
>> Provide a default location for holding those bundles.
>> As build artefact's this location, along with `.bndl` files, was
>> ignored in an earlier commit.
>>
>> This may need further refinement.
>>
>> Signed-off-by: Philip Oakley <philipoakley@iee.email>
>> ---
>>  t/test-lib.sh | 14 ++++++++++++++
>>  1 file changed, 14 insertions(+)
>>
>> diff --git a/t/test-lib.sh b/t/test-lib.sh
>> index 4e45bc7c4f..b18cd8a34e 100644
>> --- a/t/test-lib.sh
>> +++ b/t/test-lib.sh
>> @@ -56,6 +56,20 @@ then
>>  	echo "PANIC: Running in a $TEST_DIRECTORY that doesn't end in '/t'?" >&2
>>  	exit 1
>>  fi
>> +if test -z "$GIT_BUNDLE_DIR"
>> +then
>> +	# Similarly, override this to store the documentation bundles subdir
>> +	# elsewhere
>> +	GIT_BUNDLE_DIR=$TEST_DIRECTORY/GIT_BUNDLE_DIR
>> +	if test ! -d "$GIT_BUNDLE_DIR"
>> +	then
>> +		mkdir -p "$GIT_BUNDLE_DIR"
> Okey, so "mkdir -p", not "mkdir" once one thinks about it for a second:
> Because we might be testing in parallel, so this will be racy.
>
>> +	fi
>> +elif test ! -d "$GIT_BUNDLE_DIR"
>> +then
>> +	echo "PANIC: Your "$GIT_BUNDLE_DIR" doesn't exist!" >&2
>> +	exit 1
>> +fi
> I think it's better just to put these in the "test-results" dir.
>
> For leak testing I have local patches to e.g. create test-results/<test
> number>.leak, and then after a run I slurp that up and process it.
>
> So, same idea, you could just add stuff there, and afterwards grab it
> there. No need for specifying the dir etc., you just need a test-lib.sh
> flag to say "yes, write these out".
Ok, I suppose it depends on the choice between pre-slurp and
post-slurping these delivery artefact bundles.

Taylor and Junio maybe able to suggest ideas about approaches that would
fit the documentation delivery pipeline.

Philip

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [RFC/PoC 5/5] docbundle: provide the git-log History Simplification example
  2022-11-03  9:17   ` Ævar Arnfjörð Bjarmason
@ 2022-11-03 19:50     ` Philip Oakley
  0 siblings, 0 replies; 15+ messages in thread
From: Philip Oakley @ 2022-11-03 19:50 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: GitList, Junio C Hamano, Taylor Blau

On 03/11/2022 09:17, Ævar Arnfjörð Bjarmason wrote:
> On Wed, Nov 02 2022, Philip Oakley wrote:
>
>> +note () {
>> +	git tag "$1"
>> +}
> Aren't you just...
>
>> +test_expect_success setup '
>> +	# get the structure set out, then the file contents
>> +	echo "asdf" >foo &&
>> +	echo "quux" >quux &&
>> +	git add foo quux &&
>> +	test_tick && git commit -m "I: Initial commit" &&
>> +	note I &&
> ...re-inventing "test_commit" here (including the "echo")?

Probably.

It was a bit of 'monkey coding' of an earlier test without any real
study.. As you can probably tell it is just getting the spirit of the
history simplification setup in place..


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [RFC/PoC 1/5] doc: provide DocBundles design document
  2022-11-03 19:50     ` Philip Oakley
@ 2022-11-03 20:36       ` Ævar Arnfjörð Bjarmason
  0 siblings, 0 replies; 15+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2022-11-03 20:36 UTC (permalink / raw)
  To: Philip Oakley; +Cc: GitList, Junio C Hamano, Taylor Blau


On Thu, Nov 03 2022, Philip Oakley wrote:

> Hi Ævar,
> Thanks for the review,
>
> On 03/11/2022 09:19, Ævar Arnfjörð Bjarmason wrote:

> My main point was that the examples need to be covered by the test
> suite, and that bundles were a simple way of capturing the tested setup.
> The GIT_BUNDLE_DIR was simply a way of collecting them in one place for
> ease of distribution with the documentation (as you mentioned that's a
> gap in the automation step)
>>  i.e. with a minor variation of that you can do this
>> in test-lib.sh:
>>
>> 	TEST_RESULTS_BDL_DIR="$TEST_RESULTS_DIR/$TEST_NAME.test-bdl"
>>
>> Then have a helper like:
>>
>> 	do_my_test_bundle_stuff () {
>> 		local name="$1" &&
>> 		shift &&
>> 		git bundle create "$TEST_RESULTS_BDL_DIR/$name.bdl" "$@"
>> 	}
> That would be a simplification

I forgot that there needs to be a "mkdir" of the "$TEST_RESULTS_BDL_DIR"
in there somewhere (see the *.leak dir setup), but other than that
that's pretty much all there's to it.

>> Then all you need to make these is:
>>
>> 	make test GIT_TEST_OPTS=--debug
>
> I didn't quite get the link in that step between the `--debug` and the
> bndl prep.

Sorry, I was misrecalling that we wiped away "test-results" except
without "--debug", which when I spend more than 2 seconds thinking about
it makes no sense, that's just for the trash directories. Nevermind.

>> And then:
>>
>> 	ls -l t/test-results/*.bdl/*
>>
>> Will give you all the bundles you want.
>>
>> We're then missing the "connect it with the docs" part, but that part
>> would be e.g.:
>>
>> - Have a validation target for it that depends on such a "make test" run
>> - Sanity check that we did create the ones the docs expect
>>
>> Anyway.
>>
>> I like the end-goals, but I can't help think that bundles are the wrong
>> direction to go in, as opposed to:
>>
>> 1. We could have "make install" ship test-lib{,-functions}.sh along
>>    with git itself
>> 2. Have e.g. t/doc-setup/*.sh with the snippets to set up these test
>>    demos, which we'd also ship.
>
> I'm not into the idea of shipping a load of extra `test` infrastructure
> with Git itself. We don't ship any in the Git for Windows install.

We don't ship in anywhere now, but presumably GfW has the required parts
to use it, since it has git-submodule.sh and other *.sh now?

Anyway....

> I'd viewed the bundles as an adjunct to the also independent manuals and
> documentation.

...yeah me too, I'm just wondering if we won't back ourselves into a
corner with some examples, where we'll want to hand the user more
complex state.

But maybe simpler is better, and it can just be a bundle plus "run these
commands manually". The bundle can also contain a script in-tree that
performs the finishing touches, if required (unless there's no 'sh' at
runtime in GfW?).

Anyway, *if* that's a good idea (and I've got no idea, just throwing it
out there) it doesn't need to be hindered by lack of 'sh', we could just
ship compiled C code that does the required run_command() sequences to
set up the desired state.

Then in the test suite we'd call that as a test helper, or a new
built-in helper,.

>>
>> And you'd then do e.g.:
>>
>> 	git test-case rebase/description-1
>>
>> Which would just be a thin wrapper around (pseudocode):
>>
>> 	dir=$(git test--helper get-tempdir) &&
>> 	(
>> 		cd "$dir" &&
>>                 # And whatever else we need to set up to say "use this local installed git"
>> 		GIT_TEST_INSTALLED=[...] &&
>> 		. "$(git --test-dir)"/test-lib.sh
>> 		. "$(git --test-dir)"/doc-setup/rebase-description-1.sh"
>>
>> 		# This would run test_commit, and any other arbitrary setup code
>> 		setup_rebase_description_1()
>> 	) &&
>> 	echo "Go play with '$dir', and have fun"
>>
>> The big advantage of that is:
>>
>>  1. It ships with git, so it works offline, and we won't get into the
>>     inevitable confusion of the user reading v2.40 docs locally, but the
>>     online url tracking a v2.50 example or whatever.
>
> Isn't this always a problem (lack of synchronisation between multiple
> sources). If the user has local documentation, it should have matching
> bundles locally, No?

Yes, I just didn't know if that was your plan, so if that's how it works
then no problem.

It's just a depressingly common pattern these days that manual pages are
"just read about it at this link", which inevitably links to some
out-of-date documentation, even some of GCC's docs are doing that these
days.

It's good that that's not the plan here.

>>  2. We can represent whatever arbitrary repo state, e.g. drop the user
>>     into a merge conflict that's already happening.
> Isn't the manual's example already meant to describe that? And the user
> then (hopefully) follows along in their throw-away copy of the repo bundle.

Yeah, maybe that's fine.

>>  3. We can also represent "repo-extra" state, e.g. imagine the
>>     "git-gc/git-repack" docs discussing loose objects, and wanting to
>>     get you into some arbitrary state with regards to your loose objects
>>     etc.
> That's a future option, but way past the initial concept for user
> on-boarding.

^ permalink raw reply	[flat|nested] 15+ messages in thread

end of thread, other threads:[~2022-11-03 20:47 UTC | newest]

Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-11-02 22:48 [RFC/PoC 0/5] Provide example docbundles Philip Oakley
2022-11-02 22:48 ` [RFC/PoC 1/5] doc: provide DocBundles design document Philip Oakley
2022-11-03  9:19   ` Ævar Arnfjörð Bjarmason
2022-11-03 19:50     ` Philip Oakley
2022-11-03 20:36       ` Ævar Arnfjörð Bjarmason
2022-11-02 22:48 ` [RFC/PoC 2/5] Documentation Bundles: ignore artifact files from creating doc bundles Philip Oakley
2022-11-03  9:12   ` Ævar Arnfjörð Bjarmason
2022-11-03 19:50     ` Philip Oakley
2022-11-02 22:48 ` [RFC/PoC 3/5] test-lib: add GIT_BUNDLE_DIR setup Philip Oakley
2022-11-03  9:14   ` Ævar Arnfjörð Bjarmason
2022-11-03 19:50     ` Philip Oakley
2022-11-02 22:48 ` [RFC/PoC 4/5] Doc Bundles: add t6102 rev-list simple and pulls examples Philip Oakley
2022-11-02 22:48 ` [RFC/PoC 5/5] docbundle: provide the git-log History Simplification example Philip Oakley
2022-11-03  9:17   ` Ævar Arnfjörð Bjarmason
2022-11-03 19:50     ` Philip Oakley

Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).