From: Denton Liu <liu.denton@gmail.com>
To: "SZEDER Gábor" <szeder.dev@gmail.com>
Cc: Junio C Hamano <gitster@pobox.com>,
Git Mailing List <git@vger.kernel.org>,
Johannes Schindelin <Johannes.Schindelin@gmx.de>,
Jeff King <peff@peff.net>
Subject: Re: [PATCH v2 3/3] Makefile: run coccicheck on more source files
Date: Fri, 13 Sep 2019 14:38:18 -0700 [thread overview]
Message-ID: <20190913213818.GA88412@dentonliu-ltm.internal.salesforce.com> (raw)
In-Reply-To: <20190913180014.GQ32087@szeder.dev>
On Fri, Sep 13, 2019 at 08:00:14PM +0200, SZEDER Gábor wrote:
> On Fri, Sep 13, 2019 at 10:14:01AM -0700, Denton Liu wrote:
> > On Fri, Sep 13, 2019 at 01:49:52PM +0200, SZEDER Gábor wrote:
> > > On Thu, Sep 12, 2019 at 11:40:36AM -0700, Junio C Hamano wrote:
> > > > Denton Liu <liu.denton@gmail.com> writes:
> > > >
> > > > > +FIND_C_SOURCES = $(filter %.c,$(shell $(FIND_SOURCE_FILES)))
> > > > > +COCCI_SOURCES = $(filter-out $(THIRD_PARTY_SOURCES),$(FIND_C_SOURCES))
> > > >
> > > > The former is somewhat misnamed. FIND_SOURCE_FILES is *not* a list
> > > > of source files---it is a procedure to list source files to its
> > > > standard output. FIND_C_SOUCRES sounds as if it is a similar
> > > > procedure, which would be implemented much like
> > > >
> > > > FIND_C_SOURCES = $(FIND_SOURCE_FILES) | sed -n -e '/\.c$/p'
> > > >
> > > > but that is not what you did and that is not what you want to have.
> > > > Perhaps call it FOUND_C_SOURCES?
> > > >
> > > > I wonder if we can get rid of FIND_SOURCE_FILES that is a mere
> > > > procedure and replace its use with a true list of source files.
> > > > Would it make the result more pleasant to work with?
> > > >
> > > > Perhaps something like the attached patch, (which would come before
> > > > this entire thing as a clean-up, and removing the need for 2/3)?
> > > >
> > > > I dunno.
> > > >
> > > > Using a procedure whose output is fed to xargs has an advantage that
> > > > a platform with very short command line limit can still work with
> > > > many source files, but the way you create and use COCCI_SOURCES in
> > > > this patch would defeat that advantage anyway,
> > >
> > > COCCI_SOURCES is only used as an input to 'xargs', so that advantage
> > > is not defeated.
> >
> > I think it still does matter; the relevant snippet is as follows:
> >
> > if ! echo $(COCCI_SOURCES) | xargs $$limit \
> > $(SPATCH) --sp-file $< $(SPATCH_FLAGS) \
> > >$@+ 2>$@.log; \
> >
> > which means that a really big COCCI_SOURCES could exceed the limit.
>
> Oh, you're both right.
>
> > That being said, COCCI_SOURCES should be smaller than the future
> > SOURCE_FILES variable since we're only taking %.c files (and filtering
> > out some of them too!).
>
> We could also argue that Coccinelle only runs on platforms that have a
> reasonably large command line arg limit, and the number of our source
> files is way below that, so it won't matter in the foreseeable future.
Good point.
>
> (Furthermore, 'echo' is often a shell builtin command, and I don't
> think that the platform's argument size limit applies to them. At
> least the 'echo' of dash, Bash, ksh, ksh93, mksh, and BusyBox sh can
> deal with at least 10 million arguments; the platform limit is
> somewhere around 147k)
>
> > > > diff --git a/Makefile b/Makefile
> > > > index f9255344ae..9dddd0e88c 100644
> > > > --- a/Makefile
> > > > +++ b/Makefile
> > > > @@ -2584,7 +2584,7 @@ perl/build/man/man3/Git.3pm: perl/Git.pm
> > > > $(QUIET_GEN)mkdir -p $(dir $@) && \
> > > > pod2man $< $@
> > > >
> > > > -FIND_SOURCE_FILES = ( \
> > > > +SOURCE_FILES = $(patsubst ./%,%,$(shell \
> > > > git ls-files \
> > > > '*.[hcS]' \
> > > > '*.sh' \
> > > > @@ -2599,19 +2599,19 @@ FIND_SOURCE_FILES = ( \
> > > > -o \( -name 'trash*' -type d -prune \) \
> > > > -o \( -name '*.[hcS]' -type f -print \) \
> > > > -o \( -name '*.sh' -type f -print \) \
> > > > - )
> > > > + ))
> > > >
> > > > $(ETAGS_TARGET): FORCE
> > > > $(RM) $(ETAGS_TARGET)
> > > > - $(FIND_SOURCE_FILES) | xargs etags -a -o $(ETAGS_TARGET)
> > > > + etags -a -o $(ETAGS_TARGET) $(SOURCE_FILES)
> > > >
> > > > tags: FORCE
> > > > $(RM) tags
> > > > - $(FIND_SOURCE_FILES) | xargs ctags -a
> > > > + ctags -a $(SOURCE_FILES)
> > > >
> > > > cscope:
> > > > $(RM) cscope*
> > > > - $(FIND_SOURCE_FILES) | xargs cscope -b
> > > > + cscope -b $(SOURCE_FILES)
>
> Here, however, the list of source files is passed as argument to
> non-builtin commands, that also might be used on
> cmdline-arg-limit-challenged platforms.
>
After doing a bit of research, I think that I agree with you. It seems
like the max command-line length for CMD on Windows is 8191 characters.
However, after running the following,
$ git ls-files '*.[hcS]' '*.sh' ':!*[tp][0-9][0-9][0-9][0-9]*' ':!contrib' | wc -c
12779
we can see that the command-line length would definitely exceed the max
length so xargs would be required. As a result, we should probably just
keep the existing xargs invocations.
next prev parent reply other threads:[~2019-09-13 21:38 UTC|newest]
Thread overview: 29+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-09-10 7:44 [PATCH 0/2] Makefile: run coccicheck on all non-upstream sources Denton Liu
2019-09-10 7:44 ` [PATCH 1/2] Makefile: define UPSTREAM_SOURCES Denton Liu
2019-09-10 7:44 ` [PATCH 2/2] Makefile: run coccicheck on more source files Denton Liu
2019-09-10 13:28 ` SZEDER Gábor
2019-09-10 16:07 ` Denton Liu
2019-09-10 14:18 ` SZEDER Gábor
2019-09-12 17:28 ` [PATCH v2 0/3] Makefile: run coccicheck on all non-upstream sources Denton Liu
2019-09-12 17:28 ` [PATCH v2 1/3] Makefile: define THIRD_PARTY_SOURCES Denton Liu
2019-09-12 21:42 ` Junio C Hamano
2019-09-12 17:28 ` [PATCH v2 2/3] Makefile: strip leading ./ in $(FIND_SOURCE_FILES) Denton Liu
2019-09-12 18:18 ` Junio C Hamano
2019-09-13 12:05 ` SZEDER Gábor
2019-09-12 17:28 ` [PATCH v2 3/3] Makefile: run coccicheck on more source files Denton Liu
2019-09-12 18:40 ` Junio C Hamano
2019-09-13 11:49 ` SZEDER Gábor
2019-09-13 17:14 ` Denton Liu
2019-09-13 18:00 ` SZEDER Gábor
2019-09-13 21:38 ` Denton Liu [this message]
2019-09-13 17:38 ` Junio C Hamano
2019-09-16 19:23 ` [PATCH v3 0/4] Makefile: run coccicheck on all non-upstream sources Denton Liu
2019-09-16 19:23 ` [PATCH v3 1/4] Makefile: strip leading ./ in $(LIB_H) Denton Liu
2019-09-16 19:23 ` [PATCH v3 2/4] Makefile: define THIRD_PARTY_SOURCES Denton Liu
2019-09-16 20:56 ` Junio C Hamano
2019-09-16 22:00 ` [PATCH] fixup! " Denton Liu
2019-09-16 19:23 ` [PATCH v3 3/4] Makefile: strip leading ./ in $(FIND_SOURCE_FILES) Denton Liu
2019-09-16 19:23 ` [PATCH v3 4/4] Makefile: run coccicheck on more source files Denton Liu
2019-09-16 20:57 ` [PATCH v3 0/4] Makefile: run coccicheck on all non-upstream sources Junio C Hamano
2019-09-17 8:18 ` SZEDER Gábor
2019-09-17 16:13 ` Junio C Hamano
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190913213818.GA88412@dentonliu-ltm.internal.salesforce.com \
--to=liu.denton@gmail.com \
--cc=Johannes.Schindelin@gmx.de \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=peff@peff.net \
--cc=szeder.dev@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).