From: Ramsay Jones <ramsay@ramsayjones.plus.com>
To: GIT Mailing-list <git@vger.kernel.org>
Cc: Junio C Hamano <gitster@pobox.com>,
Pratyush Yadav <me@yadavpratyush.com>,
Adam Dinwoodie <adam@dinwoodie.org>
Subject: [RFC PATCH 0/8] speed up 'make clean'
Date: Thu, 5 Nov 2020 21:01:08 +0000 [thread overview]
Message-ID: <e1d218bb-7658-565e-0931-2411efbb561c@ramsayjones.plus.com> (raw)
Approximately five years ago, I stopped building the documentation
locally and instead started using the 'pre-built' documentation repos
from git://git.kernel.org/pub/scm/git/git-{htmldocs,manpages}.git.
I placed them next to the git repository (ie. as siblings) and wrote
a script (mk-dist-doc.sh) to reproduce the effect of 'make dist-doc',
using git-archive, to place the distribution tarballs at the top of
the git repo worktree.
[During the creation of these patches, I actually did a 'make' in
the Documentation directory and was surprised that it succeeded!
However, it reminded me why I stopped building the docs! :-P ]
This meant that the Documentation directory was essentially unused
for build purposes, so that a 'make clean' should have nothing to
do in that directory. However, on cygwin, I noticed a long pause
during a 'make clean' in that directory and, further, it seemed to
be GEN-erating some files! I put an item on my TODO list to take a
look at that - during the v2.28.0 cycle, I finally found time to look
at this issue! :-D
This patch series is the result. Note that this was mainly about
speeding up 'make clean' on cygwin (and the performance numbers
in the patches reflect use on cygwin). Tonight, I did some timings
on Linux as well, so that I could show this table:
CYGWIN LINUX
v2.29.0 23.339s 1.709s
Patch #1 12.364s -47.02% 0.877s -48.68%
Patch #2 10.361s -16.20% 0.805s -8.21%
Patch #3 8.430s -18.64% 0.672s -16.52%
Patch #4 6.454s -23.44% 0.484s -27.98%
Patch #5 6.428s -0.40% 0.503s +3.93%
Patch #6 6.440s +0.19% 0.517s +2.78%
Patch #7 6.430s -0.16% 0.515s -0.39%
Patch #8 4.064s -36.80% 0.322s -37.48%
23.339/4.064 = 5.74 1.709/0.322 = 5.31
Note that patches 5-7 are just preparatory patches for the final
patch, and are not expected to show any improvement.
This series is marked RFC because I forgot that 'git-gui' patches
need to be separately submitted to Pratyush Yadav, the git-gui
maintainer. So, I need to drop patch #4 from this series and
submit that to Pratyush (the commit message needs to be re-written
as well). Unfortunately, this means that the following commit
messages will need to be re-written (and timings re-done).
Also, since this series is based on v2.29.0, I had anticipated some
conflicts with the 'rs/dist-doc-with-git-archive' branch when merging
this with the 'master', 'next' and 'seen' branches. I have just done
a trial merge with each branch and, to my surprise, there were no
conflicts and they (auto) merged clean. ;)
However, I can certainly rebase these onto the 'master' branch, if that
would be easier to deal with. (There has to be a v2 anyway, so ...)
I should note here that the main idea used in these patches, to use
the $(MAKECMDGOALS) variable to conditionally include some files, can
be defeated by invoking make with multiple goals. For example, if you
were to do 'make clean all', then $(MAKECMDGOALS) would be 'clean all'
and not 'clean'.
However, doing so with the top-level Makefile would only negate the
effect of the last patch, since all of the sub-makes are issued as
'make -C <dir> clean'. So, all of the conditional includes in patches
#1-4 will work as intended (and show the noted improvement).
[Yes, 'cd Documentation; make clean all' will be slower that a doing
separate 'make clean; make', but the extra 10s, or so, will be swamped
by the documentation build time! ;-) ]
ATB,
Ramsay Jones
Ramsay Jones (8):
Documentation/Makefile: conditionally include doc.dep
Documentation/Makefile: conditionally include ../GIT-VERSION-FILE
gitweb/Makefile: conditionally include ../GIT-VERSION-FILE
git-gui/Makefile: conditionally include GIT-VERSION-FILE
Makefile: don't try to clean old debian build product
Makefile: don't use a versioned temp distribution directory
Makefile: don't delete dist tarballs directly by name
Makefile: conditionally include GIT-VERSION-FILE
.gitignore | 1 +
Documentation/Makefile | 4 ++++
Makefile | 28 +++++++++++++++++++---------
git-gui/Makefile | 2 ++
gitweb/Makefile | 2 ++
5 files changed, 28 insertions(+), 9 deletions(-)
--
2.29.0
next reply other threads:[~2020-11-05 21:01 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-11-05 21:01 Ramsay Jones [this message]
2020-11-05 21:48 ` [RFC PATCH 0/8] speed up 'make clean' Junio C Hamano
2020-11-06 1:26 ` Ramsay Jones
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=e1d218bb-7658-565e-0931-2411efbb561c@ramsayjones.plus.com \
--to=ramsay@ramsayjones.plus.com \
--cc=adam@dinwoodie.org \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=me@yadavpratyush.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).