what if i use makeheader tool to generate c header file, it would be accepted.

what if i use makeheader tool to generate c header file, it would be accepted.

[Skrab Sah]

Re: what if i use makeheader tool to generate c header file, it would be accepted.

[Ævar Arnfjörð Bjarmason]

On Mon, Sep 19 2022, Skrab Sah wrote:

Just sending a $subject is rather light on details, so I'm having to do
a lot of guessing, but here goes.

In case this is a genuine "prep question" that you're asking in
wonderining if you should even waste time on coming up with a patch to
do $subject I want to say that:

- Yes, this project would definitely be interested in the general topic
  of "fixing our headers". E.g. as I noted in passing in [1] (and this
  is just a case I happen to remember) we have obviously "bad" cases of
  headers being included when they don't need to that e.g. the iwyu tool
  spots.

- I think it's *very unlikely* that a patch that just drops in some
  external tool to generale a bunch of our code or headers would be
  accepted without a very strong case being made for its advantages.

Now, this does *not* mean that we're not interested, but just that we're
very big on a "show your work" approach to things.

So I'd think that you'd want to pursue a more incremental approach,
e.g. are there cases that makeheader spots now that can be looked at and
turned into patches? Can we perhaps get to a point whene we can run
"makeheader" in CI and spot changes that make header definitions
obsolete?

I think you'd also want to look at some prior art in this
area. E.g. COMPUTE_HEADER_DEPENDENCIES is something we use to get a
"perfect" dependency graph of headers right now.

I'm not sure how that overlaps with the makeheader use-case, but any
patches in the area should aim to review related existing work & tools,
and how a new tool is better, or fills in previous blind-spots.

For those trying to follow-along, this is the documentation for the
"makeheaders" program (which I've only skimmed, and I haven't yet used
the tool): https://fossil-scm.org/home/doc/trunk/tools/makeheaders.html

I think the biggest problem with our headers is not that we over-do
interface definitions sometimes, or that we miss some includes, but that
most things genuinely end up depending on git-compat-util.h, and
cache.h. The latter of those is a big bag of random unrelated
functions. It would be nice to split it up, so that if you e.g. need to
change just one function in cache.h we don't need to re-build the entire
project.

But that "biggest problem" isn't so big, it's only something people
working on git find occasionally annoying, and ccache mostly mitigates
it...

1. https://lore.kernel.org/git/220825.86a67s4e6r.gmgdl@evledraar.gmail.com/

Re: what if i use makeheader tool to generate c header file, it would be accepted.

[Junio C Hamano]

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

> On Mon, Sep 19 2022, Skrab Sah wrote:
>
> Just sending a $subject is rather light on details, so I'm having to do
> a lot of guessing, but here goes.
> ...
> In case this is a genuine "prep question" that you're asking in
> wonderining if you should even waste time on coming up with a patch to
> do $subject I want to say that:
> ...
> Now, this does *not* mean that we're not interested, but just that we're
> very big on a "show your work" approach to things.

Perhaps this should be made into a FAQ entry?

In general, we do not give promises or guidance, but something a
contributor finds it rewarding to work on, whether we would leter
accept in the upstream, will spread among users and developers, and
eventually we would come begging to the contributor for upstreaming.

On the other hand, a new thing that the contributor who thought of
does not feel it is worth investing their work in, if only to use
for themselves, is unlikely to be of interest to us.

So saying "if this will be accepted, I'll work on it" is counter
productive, as it is easily (mis)taken as a sign that it is the
latter case.

In other words, make it so good that we would come to you.

Thanks.

Re: what if i use makeheader tool to generate c header file, it would be accepted.

[Skrab Sah]

Let me elaborate to you, how and why I wanted to implement the
makeheaders tool in the project.

First of all, this program will automatically generate c header(.h)
files for specified c source(.c) files, which will help the developer.

Here the test shows how the tool can be implemented in different
cases: https://github.com/skrab-sah/makeheaders-test


pros:
    1. it will slightly reduce the size of the project.
    2. no need to declare anything in the header file, which is time
consuming and a headache for developers.

On Mon, 19 Sept 2022 at 23:10, Junio C Hamano <gitster@pobox.com> wrote:
>
> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
>
> > On Mon, Sep 19 2022, Skrab Sah wrote:
> >
> > Just sending a $subject is rather light on details, so I'm having to do
> > a lot of guessing, but here goes.
> > ...
> > In case this is a genuine "prep question" that you're asking in
> > wonderining if you should even waste time on coming up with a patch to
> > do $subject I want to say that:
> > ...
> > Now, this does *not* mean that we're not interested, but just that we're
> > very big on a "show your work" approach to things.
>
> Perhaps this should be made into a FAQ entry?
>
> In general, we do not give promises or guidance, but something a
> contributor finds it rewarding to work on, whether we would leter
> accept in the upstream, will spread among users and developers, and
> eventually we would come begging to the contributor for upstreaming.
>
> On the other hand, a new thing that the contributor who thought of
> does not feel it is worth investing their work in, if only to use
> for themselves, is unlikely to be of interest to us.
>
> So saying "if this will be accepted, I'll work on it" is counter
> productive, as it is easily (mis)taken as a sign that it is the
> latter case.
>
> In other words, make it so good that we would come to you.
>
> Thanks.
>

Re: what if i use makeheader tool to generate c header file, it would be accepted.

[Ævar Arnfjörð Bjarmason]

On Tue, Sep 20 2022, Skrab Sah wrote:

> Let me elaborate to you, how and why I wanted to implement the
> makeheaders tool in the project.
>
> First of all, this program will automatically generate c header(.h)
> files for specified c source(.c) files, which will help the developer.
>
> Here the test shows how the tool can be implemented in different
> cases: https://github.com/skrab-sah/makeheaders-test
>
>
> pros:
>     1. it will slightly reduce the size of the project.
>     2. no need to declare anything in the header file, which is time
> consuming and a headache for developers.

Sure, this all sound interesting in principle, and I think the answer is
definitely "we're not opposed in principle, but if you're interested
let's see patches".

But whether this is worthwhile is something that really can't be
answered until someone (i.e. you) puts in the legwork of implementing
it.

You'll then run into various trade-offs you'd have to make, and issues
you may not have forseen. Just some I can think of offhand:

 * It's unclear if you mean that we'd commit the generated files or
   not. If "not" then our Makefile will need to learn to do two-stage
   compilation. I.e. we'd ship a copy of the makeheader tool, build
   that, build the headers, and then do our "real" build.

   I happen to have an implementation of that "two-stage" compilation
   for entirely different reasons (being able to do configure/probes for
   our compilation), but *just* doing that is non-trivial.

 * The way we document various APIs now is via manually curated header
   files, e.g. how would a strbuf.h look like in this model you're
   proposing?

   Obviously we could move those comments to the *.c file, but right now
   we have a convention of implementation comments going in the *.c
   file, but the API docs going in the *.h.

   We could tell them apart with "/*" v.s. "/**" comments, as we do
   now. But part of the point of having them in the *.h file is that you
   can easily skim the docs & APi definition. Putting the docs in the
   much bigger *.c file wouldn't be nice.

 * We'd have another not-quite-compiler C-parser running on git.git,
   right now we basically have a dependency on spatch's parsing. See
   5cf88fd8b05 (git-compat-util.h: use "UNUSED", not "UNUSED(var)",
   2022-08-25).

   Is this parser smart enough to handle all the edge cases? E.g. for
   KHASH_INIT() we define interfaces via a macro-indirection, so an
   auto-generated *.h needs to resolve the macros in the *.c file.

Re: what if i use makeheader tool to generate c header file, it would be accepted.

[Skrab Sah]

1. We would not commit the generated file.
2. There are different ways we can install the makeheaders tool.
3. Makefile will first generate the header file then other compilation.
4. For API docs, there is one multi-line comment in the header file
that we can put in at the top of the .c file and also it will
automatically be copied in the .h file if we want.
5. makeheaders tools is fast and small and it will not run always,
only when its respective file is changed.
6. As I showed you before, we have different approaches to handle
different things so, it will be better to see in the patches.

I think the patches will show you a better figure.

for patches, in makefile we need to add some script for the makeheaders tool.
And the Makefile is too big and complicated to handle, it will consume time.

Is Makefile generated by another script?


For patches, I need some time and your help.

On Tue, 20 Sept 2022 at 15:43, Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
>
>
> On Tue, Sep 20 2022, Skrab Sah wrote:
>
> > Let me elaborate to you, how and why I wanted to implement the
> > makeheaders tool in the project.
> >
> > First of all, this program will automatically generate c header(.h)
> > files for specified c source(.c) files, which will help the developer.
> >
> > Here the test shows how the tool can be implemented in different
> > cases: https://github.com/skrab-sah/makeheaders-test
> >
> >
> > pros:
> >     1. it will slightly reduce the size of the project.
> >     2. no need to declare anything in the header file, which is time
> > consuming and a headache for developers.
>
> Sure, this all sound interesting in principle, and I think the answer is
> definitely "we're not opposed in principle, but if you're interested
> let's see patches".
>
> But whether this is worthwhile is something that really can't be
> answered until someone (i.e. you) puts in the legwork of implementing
> it.
>
> You'll then run into various trade-offs you'd have to make, and issues
> you may not have forseen. Just some I can think of offhand:
>
>  * It's unclear if you mean that we'd commit the generated files or
>    not. If "not" then our Makefile will need to learn to do two-stage
>    compilation. I.e. we'd ship a copy of the makeheader tool, build
>    that, build the headers, and then do our "real" build.
>
>    I happen to have an implementation of that "two-stage" compilation
>    for entirely different reasons (being able to do configure/probes for
>    our compilation), but *just* doing that is non-trivial.
>
>  * The way we document various APIs now is via manually curated header
>    files, e.g. how would a strbuf.h look like in this model you're
>    proposing?
>
>    Obviously we could move those comments to the *.c file, but right now
>    we have a convention of implementation comments going in the *.c
>    file, but the API docs going in the *.h.
>
>    We could tell them apart with "/*" v.s. "/**" comments, as we do
>    now. But part of the point of having them in the *.h file is that you
>    can easily skim the docs & APi definition. Putting the docs in the
>    much bigger *.c file wouldn't be nice.
>
>  * We'd have another not-quite-compiler C-parser running on git.git,
>    right now we basically have a dependency on spatch's parsing. See
>    5cf88fd8b05 (git-compat-util.h: use "UNUSED", not "UNUSED(var)",
>    2022-08-25).
>
>    Is this parser smart enough to handle all the edge cases? E.g. for
>    KHASH_INIT() we define interfaces via a macro-indirection, so an
>    auto-generated *.h needs to resolve the macros in the *.c file.

Re: what if i use makeheader tool to generate c header file, it would be accepted.

[Đoàn Trần Công Danh]

On 2022-09-21 13:23:11+0530, Skrab Sah <skrab.sah@gmail.com> wrote:
> 1. We would not commit the generated file.
> 2. There are different ways we can install the makeheaders tool.
> 3. Makefile will first generate the header file then other compilation.

Without seeing the patches, here is my guess, you will make us only
write source code in '*.c' and extract declaration from foo.c to foo.h
And that will save us some time to write (and update) declaration in foo.h

However, whenever foo.c is updated, foo.h will need to be
re-generated, thus all other users of foo.h will need to re-compiled.
Thus, increase compilation time. We can work-around by write the
output to foo.h+ and compare to foo.h, discard it if its content
hasn't been changed. It still takes time to analyse foo.c and write to
foo.h+

In addition, I'm not sure where would you put the struct, enum,
preprocessor declaration, and compat stuff.

If my understanding is correct, it's not worth the effort, IMHO.

-- 
Danh

Re: what if i use makeheader tool to generate c header file, it would be accepted.

[Skrab Sah]

For foo.h and foo.h+, makeheaders tool already has a feature in it.
for more info. read the manual.

The tool is fast enough to handle the situation of generating a single file.
note: we can also manually edit header file, it is not gone only it's
auto generated.

For situations like struct, enum, preprocessor declaration, and compat
stuff I have mentioned in the previous mail.

On Wed, 21 Sept 2022 at 13:41, Đoàn Trần Công Danh <congdanhqx@gmail.com> wrote:
>
> On 2022-09-21 13:23:11+0530, Skrab Sah <skrab.sah@gmail.com> wrote:
> > 1. We would not commit the generated file.
> > 2. There are different ways we can install the makeheaders tool.
> > 3. Makefile will first generate the header file then other compilation.
>
> Without seeing the patches, here is my guess, you will make us only
> write source code in '*.c' and extract declaration from foo.c to foo.h
> And that will save us some time to write (and update) declaration in foo.h
>
> However, whenever foo.c is updated, foo.h will need to be
> re-generated, thus all other users of foo.h will need to re-compiled.
> Thus, increase compilation time. We can work-around by write the
> output to foo.h+ and compare to foo.h, discard it if its content
> hasn't been changed. It still takes time to analyse foo.c and write to
> foo.h+
>
> In addition, I'm not sure where would you put the struct, enum,
> preprocessor declaration, and compat stuff.
>
> If my understanding is correct, it's not worth the effort, IMHO.
>
> --
> Danh

Re: what if i use makeheader tool to generate c header file, it would be accepted.

[Ævar Arnfjörð Bjarmason]

On Wed, Sep 21 2022, Skrab Sah wrote:

> [...]
> Is Makefile generated by another script?
> [...]
> For patches, I need some time and your help.

Aside from what Junio noted in another reply: I and others here are
generally happy to help, but when you're proposing to entirely re-do how
a part of our build process works, then you really should know the
answers to not only the basics ("is the Makefile generated?"), but also
the more complex interactions.

If you're still interested I think there's good things to work on in
this area, but starting smaller (e.g. my upthread iwyu suggestion, or
similar) would be much better.

I.e. think about the practical benefits of a proposed big rewrite are,
and whether there's ways to get some large portion of that in easier
ways.

> On Tue, 20 Sept 2022 at 15:43, Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
>>
>>
>> On Tue, Sep 20 2022, Skrab Sah wrote:
>>
>> > Let me elaborate to you, how and why I wanted to implement the
>> > makeheaders tool in the project.
>> >
>> > First of all, this program will automatically generate c header(.h)
>> > files for specified c source(.c) files, which will help the developer.
>> >
>> > Here the test shows how the tool can be implemented in different
>> > cases: https://github.com/skrab-sah/makeheaders-test
>> >
>> >
>> > pros:
>> >     1. it will slightly reduce the size of the project.
>> >     2. no need to declare anything in the header file, which is time
>> > consuming and a headache for developers.
>>
>> Sure, this all sound interesting in principle, and I think the answer is
>> definitely "we're not opposed in principle, but if you're interested
>> let's see patches".
>>
>> But whether this is worthwhile is something that really can't be
>> answered until someone (i.e. you) puts in the legwork of implementing
>> it.
>>
>> You'll then run into various trade-offs you'd have to make, and issues
>> you may not have forseen. Just some I can think of offhand:
>>
>>  * It's unclear if you mean that we'd commit the generated files or
>>    not. If "not" then our Makefile will need to learn to do two-stage
>>    compilation. I.e. we'd ship a copy of the makeheader tool, build
>>    that, build the headers, and then do our "real" build.
>>
>>    I happen to have an implementation of that "two-stage" compilation
>>    for entirely different reasons (being able to do configure/probes for
>>    our compilation), but *just* doing that is non-trivial.
>>
>>  * The way we document various APIs now is via manually curated header
>>    files, e.g. how would a strbuf.h look like in this model you're
>>    proposing?
>>
>>    Obviously we could move those comments to the *.c file, but right now
>>    we have a convention of implementation comments going in the *.c
>>    file, but the API docs going in the *.h.
>>
>>    We could tell them apart with "/*" v.s. "/**" comments, as we do
>>    now. But part of the point of having them in the *.h file is that you
>>    can easily skim the docs & APi definition. Putting the docs in the
>>    much bigger *.c file wouldn't be nice.
>>
>>  * We'd have another not-quite-compiler C-parser running on git.git,
>>    right now we basically have a dependency on spatch's parsing. See
>>    5cf88fd8b05 (git-compat-util.h: use "UNUSED", not "UNUSED(var)",
>>    2022-08-25).
>>
>>    Is this parser smart enough to handle all the edge cases? E.g. for
>>    KHASH_INIT() we define interfaces via a macro-indirection, so an
>>    auto-generated *.h needs to resolve the macros in the *.c file.

Re: what if i use makeheader tool to generate c header file, it would be accepted.

[Skrab Sah]

https://github.com/git/git/pull/1344

help me in solving this.
my spec, debian x86_64.

On Fri, 23 Sept 2022 at 13:05, Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
>
>
> On Wed, Sep 21 2022, Skrab Sah wrote:
>
> > [...]
> > Is Makefile generated by another script?
> > [...]
> > For patches, I need some time and your help.
>
> Aside from what Junio noted in another reply: I and others here are
> generally happy to help, but when you're proposing to entirely re-do how
> a part of our build process works, then you really should know the
> answers to not only the basics ("is the Makefile generated?"), but also
> the more complex interactions.
>
> If you're still interested I think there's good things to work on in
> this area, but starting smaller (e.g. my upthread iwyu suggestion, or
> similar) would be much better.
>
> I.e. think about the practical benefits of a proposed big rewrite are,
> and whether there's ways to get some large portion of that in easier
> ways.
>
> > On Tue, 20 Sept 2022 at 15:43, Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
> >>
> >>
> >> On Tue, Sep 20 2022, Skrab Sah wrote:
> >>
> >> > Let me elaborate to you, how and why I wanted to implement the
> >> > makeheaders tool in the project.
> >> >
> >> > First of all, this program will automatically generate c header(.h)
> >> > files for specified c source(.c) files, which will help the developer.
> >> >
> >> > Here the test shows how the tool can be implemented in different
> >> > cases: https://github.com/skrab-sah/makeheaders-test
> >> >
> >> >
> >> > pros:
> >> >     1. it will slightly reduce the size of the project.
> >> >     2. no need to declare anything in the header file, which is time
> >> > consuming and a headache for developers.
> >>
> >> Sure, this all sound interesting in principle, and I think the answer is
> >> definitely "we're not opposed in principle, but if you're interested
> >> let's see patches".
> >>
> >> But whether this is worthwhile is something that really can't be
> >> answered until someone (i.e. you) puts in the legwork of implementing
> >> it.
> >>
> >> You'll then run into various trade-offs you'd have to make, and issues
> >> you may not have forseen. Just some I can think of offhand:
> >>
> >>  * It's unclear if you mean that we'd commit the generated files or
> >>    not. If "not" then our Makefile will need to learn to do two-stage
> >>    compilation. I.e. we'd ship a copy of the makeheader tool, build
> >>    that, build the headers, and then do our "real" build.
> >>
> >>    I happen to have an implementation of that "two-stage" compilation
> >>    for entirely different reasons (being able to do configure/probes for
> >>    our compilation), but *just* doing that is non-trivial.
> >>
> >>  * The way we document various APIs now is via manually curated header
> >>    files, e.g. how would a strbuf.h look like in this model you're
> >>    proposing?
> >>
> >>    Obviously we could move those comments to the *.c file, but right now
> >>    we have a convention of implementation comments going in the *.c
> >>    file, but the API docs going in the *.h.
> >>
> >>    We could tell them apart with "/*" v.s. "/**" comments, as we do
> >>    now. But part of the point of having them in the *.h file is that you
> >>    can easily skim the docs & APi definition. Putting the docs in the
> >>    much bigger *.c file wouldn't be nice.
> >>
> >>  * We'd have another not-quite-compiler C-parser running on git.git,
> >>    right now we basically have a dependency on spatch's parsing. See
> >>    5cf88fd8b05 (git-compat-util.h: use "UNUSED", not "UNUSED(var)",
> >>    2022-08-25).
> >>
> >>    Is this parser smart enough to handle all the edge cases? E.g. for
> >>    KHASH_INIT() we define interfaces via a macro-indirection, so an
> >>    auto-generated *.h needs to resolve the macros in the *.c file.
>

Re: what if i use makeheader tool to generate c header file, it would be accepted.

[Junio C Hamano]

Skrab Sah <skrab.sah@gmail.com> writes:

> Let me elaborate to you, how and why I wanted to implement the

No, I won't let you ;-), and that was the most important part you
seem to have missed in the message you are responding to: we do not
work with the handwaving.

>     2. no need to declare anything in the header file, which is
> time consuming and a headache for developers.

This alone is enough to convince me that this should not be done.

Unless the tool can infer the intent while generating the headers
and place comments describing why it exists and what it and its
parameters mean next to the function definition it produces, that
is, in which case I may be OK.  As developers become more and more
experienced, they will invest more and more time in getting the
interface right.  We shouldn't target developers who find the time
to do so a waste and headache.

Thanks.