should any build system legitimately change any tracked files?

should any build system legitimately change any tracked files?

[Robert P. J. Day]

  just finished teaching a couple git courses and, after class, a
student came up and described a rather weird problem -- in short:

  1) before build, "git diff" shows nothing
  2) do the standard build
  3) suddenly, "git diff" shows some changes

that's all the info i was given, but it *seems* clear that the build
process itself was making changes to one or more tracked files.

  technically, i guess one can design a build system to do pretty
much anything, but is it fair to say that this is a really poor design
decision? admittedly, this isn't specifically a git question, but i'm
open to opinions on something that strikes me as a bad idea.

rday

RE: should any build system legitimately change any tracked files?

[Randall S. Becker]

On January 19, 2018 12:52 PM, Robert P. J. Day wrote:
>   just finished teaching a couple git courses and, after class, a student
came
> up and described a rather weird problem -- in short:
> 
>   1) before build, "git diff" shows nothing
>   2) do the standard build
>   3) suddenly, "git diff" shows some changes
> 
> that's all the info i was given, but it *seems* clear that the build
process itself
> was making changes to one or more tracked files.
> 
>   technically, i guess one can design a build system to do pretty much
> anything, but is it fair to say that this is a really poor design
decision?
> admittedly, this isn't specifically a git question, but i'm open to
opinions on
> something that strikes me as a bad idea.

Depends what you're up to.  Changing the source repository content is
probably bad. Adding tags may not be. Also, updating a separate repository
to include build information (a.k.a dependency tracking between source and
object commits) can be very useful for managing production builds and
environments.

Cheers,
Randall

-- Brief whoami:
 NonStop developer since approximately 211288444200000000
 UNIX developer since approximately 421664400
-- In my real life, I talk too much.

Re: should any build system legitimately change any tracked files?

[Theodore Ts'o]

On Fri, Jan 19, 2018 at 12:51:52PM -0500, Robert P. J. Day wrote:
> that's all the info i was given, but it *seems* clear that the build
> process itself was making changes to one or more tracked files.
> 
>   technically, i guess one can design a build system to do pretty
> much anything, but is it fair to say that this is a really poor design
> decision? admittedly, this isn't specifically a git question, but i'm
> open to opinions on something that strikes me as a bad idea.

I agree that in general it's a bad idea.  I can see how it happens,
though, which is because two things come into tension:

1) The general desire not to check in generated files into the git
repository --- including configure files generated by autoconf,
Makefiles generated by automake, libtool files, etc.

2) Wanting not to give users trying to build from source a non-hostile
experience.  Unfortunately autoconf/automake/libtool systems are
notorious for not having a stable interface, such that if you have the
wrong or outdated version of the tools, the results of generating the
configure, Makefile, etc., using a different version than what the
developer used.... well, your results may vary.

What I do is use "Maintainer mode" which means that the generated
files are *not* automatically rebuilt by the build system unless you
configure with --enable-maintainer-mode, and then I *do* check in the
generated files into git.  That way I can run with
--enable-maintainer-mode, and check in updates to Makefile, configure,
etc., as necessary when the input files change, but that way, end
users don't have to worry getting ambushed by version skew caused by
using an old (or unexpectedly newer) version of the
autoconf/autoconf/libtool tools.

Heck, I even have had config.guess/config.sub change on me in
incompatible ways(*), so I ship my own version and don't enable a blind
update of those files from the upstream FSF sources --- mainly because
I don't trust them to preserve a stable interface.  Better that I
manually pull them into the repo, and test them before I do a public
release.

					- Ted

(*) Although to be fair it's been years since I've been screwed in
this fashion.  But once bitten, twice shy....

Re: should any build system legitimately change any tracked files?

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 2010 bytes --]

On Fri, Jan 19, 2018 at 12:51:52PM -0500, Robert P. J. Day wrote:
> 
>   just finished teaching a couple git courses and, after class, a
> student came up and described a rather weird problem -- in short:
> 
>   1) before build, "git diff" shows nothing
>   2) do the standard build
>   3) suddenly, "git diff" shows some changes
> 
> that's all the info i was given, but it *seems* clear that the build
> process itself was making changes to one or more tracked files.
> 
>   technically, i guess one can design a build system to do pretty
> much anything, but is it fair to say that this is a really poor design
> decision? admittedly, this isn't specifically a git question, but i'm
> open to opinions on something that strikes me as a bad idea.
I have seen what you describe, but it had a good cause:
1. The source repo contained some intermediate generated source, 
   eg foo.x -> foo.c -> foo.o
2. The output of the tool that did foo.a -> foo.c differed due to some
   factor on the system (different version, different config in /etc etc).
3. The initial checkout caused the mtime of foo.c to be just older
   newer than foo.x, so the build system decided to regen foo.c.
4. (optional) The makefile had conditional rules to skip the regen if the tool was
   not present.

Until the tool output changed, even if the file was regenerated, it was
identical, so it didn't show up in diff.

What are the possible mistakes here?
- The intermediate source possibly should not be committed [depending on
  the tool, this isn't always an option]
- The build system scripts (makefile etc) contains a mistake.
- Some final (non-intermediate/non-source) file was committed.

I've seen similar patterns for GNU Bison, autoconf, and lots of other
tools. 

-- 
Robin Hugh Johnson
Gentoo Linux: Dev, Infra Lead, Foundation Treasurer
E-Mail   : robbat2@gentoo.org
GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 1113 bytes --]