.gitignore, .gitattributes, .gitmodules, .gitprecious?, .gitacls? etc.

.gitignore, .gitattributes, .gitmodules, .gitprecious?, .gitacls? etc.

[Dmitry Kakurin]

Git is using more and more .git* files to store metadata.
I have two comments on this:
1. It may be better to combine all these files into one (.gitmeta) with different sections
2. Storing metadata in regular source-controlled files feels wrong to me. I cannot be very specific why (call it intuition), but 
something inside me screams "bad design" :-). We've already seen some chicken-and-the-egg problems with crlf and .gitattributes. So, 
may be it would be better to keep this metadata only in repository (and index). One option would be to associate such a META object 
with every TREE (an make it optional). Then allow editing this object with something like 'git meta'.

- Dmitry 

Re: .gitignore, .gitattributes, .gitmodules, .gitprecious?, .gitacls? etc.

[Junio C Hamano]

Dmitry Kakurin <dmitry.kakurin@gmail.com> writes:

> 1. It may be better to combine all these files into one (.gitmeta) with different sections

Merging what has traditionally been known as .gitignore's
capability to attributes has been discussed, and I think it
would make sense in longer term, as 'this path pattern is to be
ignored' is just a special case of a more general attribute.
And "precious" handling would naturally fit there.  However, as
the .gitignore and .git/info/exclude has been as old as git
itself (I think it was introduced around early May 2005), I do
not see us even start talking about deprecating .gitignore.

I do not think .gitmodules fits the model of what .gitattributes
solves.  .gitattributes is about the attribute of paths, while
.gitmodules is about attribute of subprojects, and one attribute
of a subproject is where in the superproject directory hierarchy
it sits at.

I do not know what you are talking about with .gitacls.
Personally I am not interested in turning git into a back-up
program at all, so if you are talking beyond what has already
been suggested as "owner", "group" and "perms" attributes that
could be stored in .gitattributes, I do not think it belongs to
git.

> 2. Storing metadata in regular source-controlled files feels wrong to
> me.

You are free to _feel_ whatever you want without thinking, but
please keep that _feeling_ to yourself, and speak it out after
making it into an _opinion_, which would take a bit of thinking
about it first.  For example, think about what you could do
without confusing a total newbie after the initial clone.  You
cannot avoid chicken-and-egg problem.  I think reading from
index as a fallback measure when work tree file is missing is a
very good compromise we came up recently.  The wish of the user
(i.e. the owner of the work tree) overrides what is in the
index, and the index is how the repository contents are
initially propagated back to the work tree.

Re: .gitignore, .gitattributes, .gitmodules, .gitprecious?, .gitacls? etc.

[Dmitry Kakurin]

----- Original Message ----- 
From: "Junio C Hamano" <gitster@pobox.com>
 >> 2. Storing metadata in regular source-controlled files feels wrong to
>> me.
> You are free to _feel_ whatever you want without thinking, but

I did quite a bit of thinking before posting it. Not sure what made you think otherwise.

>  I think reading from
> index as a fallback measure when work tree file is missing is a
> very good compromise we came up recently.

Can you specify _exactly_ how it works now? And I'll show you a bunch of corner cases where it's broken.

- Dmitry

Re: .gitignore, .gitattributes, .gitmodules, .gitprecious?, .gitacls? etc.

[Junio C Hamano]

Dmitry Kakurin <dmitry.kakurin@gmail.com> writes:

> ----- Original Message ----- 
> From: "Junio C Hamano" <gitster@pobox.com>
>>> 2. Storing metadata in regular source-controlled files feels wrong to
>>> me.
>> You are free to _feel_ whatever you want without thinking, but
>
> I did quite a bit of thinking before posting it. Not sure what made you think otherwise.
>
>>  I think reading from
>> index as a fallback measure when work tree file is missing is a
>> very good compromise we came up recently.
>
> Can you specify _exactly_ how it works now? And I'll show you a bunch of corner cases where it's broken.

As I made it clear, it is a compromise.  I am not interested in
discussing corner cases with you -- I am sure there are.

If you are offering improvements, I am all ears.

Re: .gitignore, .gitattributes, .gitmodules, .gitprecious?, .gitacls? etc.

[Dmitry Kakurin]

----- Original Message ----- 
From: "Junio C Hamano" <gitster@pobox.com>
>> Can you specify _exactly_ how it works now? And I'll show you a bunch of corner cases where it's broken.
>
> As I made it clear, it is a compromise.  I am not interested in
> discussing corner cases with you -- I am sure there are.

Usually many corner cases == design flaw.

> If you are offering improvements, I am all ears.

I thought I did: I've observed that these problem are caused by storing metadata in regular files (that exist both in repo/index and 
in workplace).
My knowledge of Git internals is quite limited, but if *I* were to do it right now, I'd introduce a META entry in every TREE object 
that would point to a BLOB that contains combined content of .gitattributes, .gitignore etc. Then command like 'git meta' would open 
vi for this BLOB, let you edit it, and then would upload it back to index.

- Dmitry

Re: .gitignore, .gitattributes, .gitmodules, .gitprecious?, .gitacls? etc.

[Junio C Hamano]

Dmitry Kakurin <dmitry.kakurin@gmail.com> writes:

> I thought I did: I've observed that these problem are caused by
> storing metadata in regular files (that exist both in repo/index and
> in workplace).

And that observation solves the initial checkout issue how?

> My knowledge of Git internals is quite limited, but if *I* were to do
> it right now, I'd introduce a META entry in every TREE object that
> would point to a BLOB that contains combined content of
> .gitattributes, .gitignore etc.

A tree that has .gitattributes (and I am assuming in the longer
term you can use "ignore" and "precious" in .gitattributes
instead of using .gitignore) POINTS TO A BLOB already, so what
you are saying does not add anything to what we already have,
other than that you are renaming .gitattributes to "META ENTRY".

When you do "git checkout -- this-path", you are checking things
out from the index and at that point you may not have _any_ tree
yet (think "before initial commit").  A "META ENTRY" that exists
only in a tree does not work -- it has to come to index somehow
for it to work with how git works.

Re: .gitignore, .gitattributes, .gitmodules, .gitprecious?, .gitacls? etc.

[Dmitry Kakurin]

----- Original Message ----- 
From: "Junio C Hamano" <gitster@pobox.com>
> Dmitry Kakurin <dmitry.kakurin@gmail.com> writes:
> 
>> I thought I did: I've observed that these problem are caused by
>> storing metadata in regular files (that exist both in repo/index and
>> in workplace).
> 
> And that observation solves the initial checkout issue how?
There is always only one place to check for metadata - the index.

>> My knowledge of Git internals is quite limited, but if *I* were to do
>> it right now, I'd introduce a META entry in every TREE object that
>> would point to a BLOB that contains combined content of
>> .gitattributes, .gitignore etc.
> 
> A tree that has .gitattributes (and I am assuming in the longer
> term you can use "ignore" and "precious" in .gitattributes
> instead of using .gitignore) POINTS TO A BLOB already, so what
> you are saying does not add anything to what we already have,
> other than that you are renaming .gitattributes to "META ENTRY".

Almost true! The difference is: META BLOBS are not created as files in the workspace (not during checkout, not ever).
In order to edit it you'd have to use 'git meta' command.
So once again, there is only one place to check for metadata - the index.

> When you do "git checkout -- this-path", you are checking things
> out from the index and at that point you may not have _any_ tree
> yet (think "before initial commit").  A "META ENTRY" that exists
> only in a tree does not work -- it has to come to index somehow
> for it to work with how git works.

I don't fully understand the difficulty. Here is how I see the initial checkin to work:
Let's say you just did 'git init' and want to add file.txt with unusual crlf. So you don't want automatic translation.
What you'll do is:
* git meta
  In vi you put 'file.txt -crlf'
  git creates a TREE object in the index with only entry META that point to BLOB with this file.
  so it's kind of an empty tree from user's perspective.
* git add file.txt
  while adding the file git consults META BLOB from the TREE (stored in index) and does not do any crlf translation
  the TREE now contains 2 entries: META, and BLOB for file.txt
* git commit
   nothing special here

Now someone else doing initial checkout:
* git checkout
  Before writing file.txt to disk git consults META BLOB (in the index) and does not do any crlf translation.

- Dmitry

Re: .gitignore, .gitattributes, .gitmodules, .gitprecious?, .gitacls? etc.

[Petr Baudis]

On Sun, Aug 26, 2007 at 10:02:50AM CEST, Dmitry Kakurin wrote:
>>> My knowledge of Git internals is quite limited, but if *I* were to do
>>> it right now, I'd introduce a META entry in every TREE object that
>>> would point to a BLOB that contains combined content of
>>> .gitattributes, .gitignore etc.
>> A tree that has .gitattributes (and I am assuming in the longer
>> term you can use "ignore" and "precious" in .gitattributes
>> instead of using .gitignore) POINTS TO A BLOB already, so what
>> you are saying does not add anything to what we already have,
>> other than that you are renaming .gitattributes to "META ENTRY".
>
> Almost true! The difference is: META BLOBS are not created as files in the 
> workspace (not during checkout, not ever).
> In order to edit it you'd have to use 'git meta' command.
> So once again, there is only one place to check for metadata - the index.

That sounds so incredibly ugly, I really would hate to see that.

It's still not clear to me how this would help anything, though I didn't
watch late Git development. Can you explain some particular scenario
where this would improve the current situation?

-- 
				Petr "Pasky" Baudis
Ever try. Ever fail. No matter. // Try again. Fail again. Fail better.
		-- Samuel Beckett

Re: .gitignore, .gitattributes, .gitmodules, .gitprecious?, .gitacls? etc.

[Johannes Schindelin]

Hi,

On Sat, 25 Aug 2007, Dmitry Kakurin wrote:

> ----- Original Message ----- From: "Junio C Hamano" <gitster@pobox.com>
> >> 2. Storing metadata in regular source-controlled files feels wrong to 
> >> me.
> > You are free to _feel_ whatever you want without thinking, but
> 
> I did quite a bit of thinking before posting it. Not sure what made you 
> think otherwise.

Well, it certainly appears to me that your proposal to move metadata from 
the working tree (where it is visible, and easily editable with the editor 
of _your_ choice) to the index (where it is hidden, and could only be 
edited with _yet another_ git command) is not well thought through.

It certainly would make some common operation much more complicated, for 
no gain at all.

Should you still not be convinced, please find some convincing mails by 
Linus (which are  much longer than I would have the patience to write) 
where he goes into detail _why_ it is _wrong_ to _hide_ things away from 
the working tree.

(Just a small hint: git is much more powerful _because_ it keeps metadata 
visibly in the filesystem.)

As for your proposal to munge the different metadata files into sections 
of _one_ file: I doubt that this is "cleaner" or "more elegant" than what 
we have now.  For one, if a script fscks up one file, it does not fsck up 
the others.

For another, scripts do not have to jump through hoops to edit the 
metadata files, as long as they do not have sections (containing 
completely independent informations).

Ciao,
Dscho

Re: .gitignore, .gitattributes, .gitmodules, .gitprecious?, .gitacls? etc.

[Sam Vilain]

Dmitry Kakurin wrote:
>> A tree that has .gitattributes (and I am assuming in the longer
>> term you can use "ignore" and "precious" in .gitattributes
>> instead of using .gitignore) POINTS TO A BLOB already, so what
>> you are saying does not add anything to what we already have,
>> other than that you are renaming .gitattributes to "META ENTRY".
>
> Almost true! The difference is: META BLOBS are not created as files in
> the workspace (not during checkout, not ever).
> In order to edit it you'd have to use 'git meta' command.
> So once again, there is only one place to check for metadata - the index.

Can I just chime in here and express my distaste for this idea, on
several grounds, but the summary is that svn does it this way, so it
must be wrong.

These files which store metadata would be stored in a way that is "in
another dimension" to the project files, despite being a part of the
history.  That means that all tools built to deal with regular files and
directories will not be able to merge the changes to the attributes
without special support.  I think this is broken.

This is something I frequently run up against with people coming from
Subversion, which supports unversioned revision properties which can
change randomly and without trace, and per-file/directory properties
which are simply files which you can't refer to in the regular way, and
are interpreted in an application-specific way.

My question to these people, and my question to you is: why do these
files need to be served from another dimension, what value does it add?

You see, either way, their contents need to be processed in an
application-specific way.  Same thing with git's "commit properties" -
basically just RFC822.. headers used in the commit message.  People I
have talked to have described this as "more arbitrary" than conventions
for attributes which are structured.  However, when pressed I have yet
to hear a clear argument why this is the case.

As far as file properties goes, I still like Linus' idea of making these
files which are accessed by treating the file as a directory (eg
filename.txt/ACL, filename.txt/mime-type), and that approach could be
represented in git well.

Sam.

Re: .gitignore, .gitattributes, .gitmodules, .gitprecious?, .gitacls? etc.

[David Kastrup]

Sam Vilain <sam@vilain.net> writes:

> Dmitry Kakurin wrote:
>>> A tree that has .gitattributes (and I am assuming in the longer
>>> term you can use "ignore" and "precious" in .gitattributes
>>> instead of using .gitignore) POINTS TO A BLOB already, so what
>>> you are saying does not add anything to what we already have,
>>> other than that you are renaming .gitattributes to "META ENTRY".
>>
>> Almost true! The difference is: META BLOBS are not created as files in
>> the workspace (not during checkout, not ever).
>> In order to edit it you'd have to use 'git meta' command.
>> So once again, there is only one place to check for metadata - the index.
>
> Can I just chime in here and express my distaste for this idea, on
> several grounds, but the summary is that svn does it this way, so it
> must be wrong.
>
> These files which store metadata would be stored in a way that is
> "in another dimension" to the project files, despite being a part of
> the history.  That means that all tools built to deal with regular
> files and directories will not be able to merge the changes to the
> attributes without special support.  I think this is broken.

That presumes that a good way to merge attributes is to use a text
file merge algorithm, complete with finding diff context lines in a
basically unchanged order.

And I don't see that this is a sensible merge strategy at all.  No
matter where the attributes are stored, whether in a file or somewhere
else, any useful merge strategy would require an algorithm quite
different from the currently used one.

Now this might be a case for pluggable merge strategies: after all,
there might be non-git related files with similar unordered per-line
merge semantics, or files expressing some information about files.

> As far as file properties goes, I still like Linus' idea of making
> these files which are accessed by treating the file as a directory
> (eg filename.txt/ACL, filename.txt/mime-type), and that approach
> could be represented in git well.

Well, at least _some_ interesting Reiser4 idea resurfaces.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: .gitignore, .gitattributes, .gitmodules, .gitprecious?, .gitacls? etc.

[Sam Vilain]

David Kastrup wrote:
>> These files which store metadata would be stored in a way that is
>> "in another dimension" to the project files, despite being a part of
>> the history.  That means that all tools built to deal with regular
>> files and directories will not be able to merge the changes to the
>> attributes without special support.  I think this is broken.
>>     
>
> That presumes that a good way to merge attributes is to use a text
> file merge algorithm, complete with finding diff context lines in a
> basically unchanged order.
>   

Yes.  Is that not a reasonable assumption, in the absence of anything
more enlightened?

>> As far as file properties goes, I still like Linus' idea of making
>> these files which are accessed by treating the file as a directory
>> (eg filename.txt/ACL, filename.txt/mime-type), and that approach
>> could be represented in git well.
>>     
>
> Well, at least _some_ interesting Reiser4 idea resurfaces.
>   

That was in there too?  Man that Reiser4 manifesto read like the Naked
Lunch.

Sam.

Re: .gitignore, .gitattributes, .gitmodules, .gitprecious?, .gitacls? etc.

[David Kastrup]

Sam Vilain <sam@vilain.net> writes:

> David Kastrup wrote:
>>> These files which store metadata would be stored in a way that is
>>> "in another dimension" to the project files, despite being a part of
>>> the history.  That means that all tools built to deal with regular
>>> files and directories will not be able to merge the changes to the
>>> attributes without special support.  I think this is broken.
>>>     
>>
>> That presumes that a good way to merge attributes is to use a text
>> file merge algorithm, complete with finding diff context lines in a
>> basically unchanged order.
>>   
>
> Yes.  Is that not a reasonable assumption, in the absence of anything
> more enlightened?

Is that a trick question?  My comment was exactly about not throwing
away the information ("This is not arbitrary text but talks about the
files in our tree.") that would make for more enlightened use.

-- 
David Kastrup

Re: .gitignore, .gitattributes, .gitmodules, .gitprecious?, .gitacls? etc.

[Johannes Schindelin]

Hi,

On Mon, 27 Aug 2007, Sam Vilain wrote:

> David Kastrup wrote:
> >> These files which store metadata would be stored in a way that is
> >> "in another dimension" to the project files, despite being a part of
> >> the history.  That means that all tools built to deal with regular
> >> files and directories will not be able to merge the changes to the
> >> attributes without special support.  I think this is broken.
> >>     
> >
> > That presumes that a good way to merge attributes is to use a text
> > file merge algorithm, complete with finding diff context lines in a
> > basically unchanged order.
> >   
> 
> Yes.  Is that not a reasonable assumption, in the absence of anything
> more enlightened?

Umm.

It is not about _text_ file merge algorithms, but algorithms _outside of 
git_!

If you tuck the stuff away in some obscure database where it is hard to 
access, you make it more complicated and time consuming to access the data 
than it needs to be to _begin with_.

> >> As far as file properties goes, I still like Linus' idea of making 
> >> these files which are accessed by treating the file as a directory 
> >> (eg filename.txt/ACL, filename.txt/mime-type), and that approach 
> >> could be represented in git well.
> >
> > Well, at least _some_ interesting Reiser4 idea resurfaces.
> 
> That was in there too?  Man that Reiser4 manifesto read like the Naked 
> Lunch.

It is funny.  No, I mean really funny.  People criticise Reiser4, and only 
preciously few actually have ideas as good as in Reiser4.  Yes, Reiser4 
was not developed as openly as it should have been.  Yes, Hans was not the 
most diplomatic poster ever, on lkml.  No, even the stupid ideas in 
Reiser4 are not half as stupid as mistaking the working tree for a place 
where only _text_ files reside.

Ciao,
Dscho

Re: .gitignore, .gitattributes, .gitmodules, .gitprecious?, .gitacls? etc.

[martin f krafft]

[-- Attachment #1: Type: text/plain, Size: 1169 bytes --]

also sprach Junio C Hamano <gitster@pobox.com> [2007.08.26.0637 +0200]:
> > 1. It may be better to combine all these files into one
> > (.gitmeta) with different sections
> 
> Merging what has traditionally been known as .gitignore's
> capability to attributes has been discussed, and I think it would
> make sense in longer term, as 'this path pattern is to be ignored'
> is just a special case of a more general attribute.

I tried to find related threads in the archives but failed. If
someone has a pointer handy, I'd appreciate it. Alternatively,
please feel free to just bounce related messages from your own
archives to me.

I am interested in this mainly because of a somewhat related idea of
honouring .gitignore/* in case it's a directory [0].

0. http://marc.info/?l=git&m=118725982332041&w=2

-- 
martin;              (greetings from the heart of the sun.)
  \____ echo mailto: !#^."<*>"|tr "<*> mailto:" net@madduck
 
"the question of whether computers can think
 is like the question of whether submarines can swim."
                                                 -- edsgar w. dijkstra
 
spamtraps: madduck.bogus@madduck.net

[-- Attachment #2: Digital signature (see http://martin-krafft.net/gpg/) --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: .gitignore, .gitattributes, .gitmodules, .gitprecious?, .gitacls? etc.

[Sergio Callegari]

Junio C Hamano <gitster <at> pobox.com> writes:

> 
> Dmitry Kakurin <dmitry.kakurin <at> gmail.com> writes:
> 
> > 1. It may be better to combine all these files into one (.gitmeta) with
different sections
> 

Sorry about entering this discussion so late.
I am just wondering about one thing.

Couldn't all this directory/ownership/permission tracing be easily done by
using hooks?
E.g. Having a pre-status and pre-commit hook one could fire up a program/script
to collect all the extra info he wants to trace and store it somewhere
(typically in some traced file).
The other way round one could have a post-checkout hook and he could arrange
it to fire up some program to look into the extra-info file to set up
all the meta-data he wants.

This would be very flexible and would permit to manage absolutely /any/ kind
of the metadata leaving absolute freedom about how to do so.

Am I missing something here?

Sergio

Re: .gitignore, .gitattributes, .gitmodules, .gitprecious?, .gitacls? etc.

[David Kastrup]

Sergio Callegari <scallegari@arces.unibo.it> writes:

> Couldn't all this directory/ownership/permission tracing be easily
> done by using hooks?  E.g. Having a pre-status and pre-commit hook
> one could fire up a program/script to collect all the extra info he
> wants to trace and store it somewhere (typically in some traced
> file).  The other way round one could have a post-checkout hook and
> he could arrange it to fire up some program to look into the
> extra-info file to set up all the meta-data he wants.
>
> This would be very flexible and would permit to manage absolutely
> /any/ kind of the metadata leaving absolute freedom about how to do
> so.
>
> Am I missing something here?

Merging.

-- 
David Kastrup

Re: .gitignore, .gitattributes, .gitmodules, .gitprecious?, .gitacls? etc.

[Petr Baudis]

On Mon, Aug 27, 2007 at 05:48:34PM CEST, David Kastrup wrote:
> Sergio Callegari <scallegari@arces.unibo.it> writes:
> 
> > Couldn't all this directory/ownership/permission tracing be easily
> > done by using hooks?  E.g. Having a pre-status and pre-commit hook
> > one could fire up a program/script to collect all the extra info he
> > wants to trace and store it somewhere (typically in some traced
> > file).  The other way round one could have a post-checkout hook and
> > he could arrange it to fire up some program to look into the
> > extra-info file to set up all the meta-data he wants.
> >
> > This would be very flexible and would permit to manage absolutely
> > /any/ kind of the metadata leaving absolute freedom about how to do
> > so.
> >
> > Am I missing something here?
> 
> Merging.

Fetching.

-- 
				Petr "Pasky" Baudis
Early to rise and early to bed makes a male healthy and wealthy and dead.
                -- James Thurber

Re: .gitignore, .gitattributes, .gitmodules, .gitprecious?, .gitacls? etc.

[Sergio Callegari]

David Kastrup <dak <at> gnu.org> writes:

> 
> Sergio Callegari <scallegari <at> arces.unibo.it> writes:
> 
> > Couldn't all this directory/ownership/permission tracing be easily
> > done by using hooks?  E.g. Having a pre-status and pre-commit hook
> > one could fire up a program/script to collect all the extra info he
> > wants to trace and store it somewhere (typically in some traced
> > file).  The other way round one could have a post-checkout hook and
> > he could arrange it to fire up some program to look into the
> > extra-info file to set up all the meta-data he wants.
> >
> > This would be very flexible and would permit to manage absolutely
> > /any/ kind of the metadata leaving absolute freedom about how to do
> > so.
> >
> > Am I missing something here?
> 
> Merging.
> 

Sorry, maybe I am really missing something, since merging does not look to me
as an issue.

Why cannot git simply do the merging in the working tree as it normally
does, including merging of the traced metadata file generated by the metadata
helpers invoked via the hooks?
Only, again more hooks are needed and likely a post-merge hook, so that at
the end of the merge, the metadata can be applied.

Only, to have things going on smoothly, one should be so wise to assure that
the metadata helpers save metadata as nice, sorted text files in order to
minimize the burden of manual intervention if there are conflicts in
metadata merging.

BTW.  Having a post-checkout hook could also help getting rid of unwanted
empty directories, couldn't it?
 
Sergio

Re: .gitignore, .gitattributes, .gitmodules, .gitprecious?, .gitacls? etc.

[Sergio Callegari]

Petr Baudis <pasky <at> suse.cz> writes:

> 
> On Mon, Aug 27, 2007 at 05:48:34PM CEST, David Kastrup wrote:
> > Sergio Callegari <scallegari <at> arces.unibo.it> writes:
> > 
> > > Couldn't all this directory/ownership/permission tracing be easily
> > > done by using hooks?  E.g. Having a pre-status and pre-commit hook
> > > one could fire up a program/script to collect all the extra info he
> > > wants to trace and store it somewhere (typically in some traced
> > > file).  The other way round one could have a post-checkout hook and
> > > he could arrange it to fire up some program to look into the
> > > extra-info file to set up all the meta-data he wants.
> > >
> > > This would be very flexible and would permit to manage absolutely
> > > /any/ kind of the metadata leaving absolute freedom about how to do
> > > so.
> > >
> > > Am I missing something here?
> > 
> > Merging.
> 
> Fetching.
> 

Even here, I must be missing something, as I cannot see the issue.

If I need to fetch from someone who is tracing metadata, then there are 2
alternatives:

1) I am fetching only for myself and I am not interested in metadata at all.
All I need to do is to fetch. With this I will fetch a repository with
one/some extra file/files (e.g. .helper-metadata).

2) I am interested in the metadata tracing (e.g. to interact with
my origin). Then it is sufficient to first install the same set of
metadata tracing helpers as my origin and after that to do the fetch.
With this I will fetch a repository including the traced metadata files
just as above, yet these would be immediately be used by the helpers
through the hooks. For instance, as soon as anything gets checked out the
proper metadata can be applied.

Obviously, before installing any hooks, I should trust their origin.  But I
believe that if hooks get this kind of usage, rapidly we will see
the growth of trustable "standard" hooks-bundles for many tasks.

.gitignore, .gitattributes, .gitmodules, .gitprecious?,.gitacls? etc.

[Dmitry Kakurin]

----- Original Message -----
From: "Petr Baudis" <pasky@suse.cz>
> On Sun, Aug 26, 2007 at 10:02:50AM CEST, Dmitry Kakurin wrote:
>>>> My knowledge of Git internals is quite limited, but if *I* were to do
>>>> it right now, I'd introduce a META entry in every TREE object that
>>>> would point to a BLOB that contains combined content of
>>>> .gitattributes, .gitignore etc.
>>> A tree that has .gitattributes (and I am assuming in the longer
>>> term you can use "ignore" and "precious" in .gitattributes
>>> instead of using .gitignore) POINTS TO A BLOB already, so what
>>> you are saying does not add anything to what we already have,
>>> other than that you are renaming .gitattributes to "META ENTRY".
>>
>> Almost true! The difference is: META BLOBS are not created as files in the
>> workspace (not during checkout, not ever).
>> In order to edit it you'd have to use 'git meta' command.
>> So once again, there is only one place to check for metadata - the index.
>
> It's still not clear to me how this would help anything, though I didn't
> watch late Git development. Can you explain some particular scenario
> where this would improve the current situation?

Here is the problem: we need to apply crlf attributes to a file. We
could have .gitattributes both in the index and in the worktree.
Which one do we use?
In general .gitattributes file could be (U)nchanged, (C)hanged, (NP)
NotPresent in each place.
This gives us 3x3 matrix = 9 special cases to handle. When you think
about this a little more you realize that even that
is not enough and we need to take into account direction of data
movement (index -> filesystem or vice versa).
This doubles the matrix to 18 cases. Even if we come up with a very
good choice for every case (which is doubtful)
there is no way this could be *easily* explained to end user, or even memorized.
This leads to a simple idea that mostly works:
1. when files are moved from index to filesystem, then only
.gitattributes in the index is used, if it's not there == no special
attributes.
2. when files are moved from filesystem to index, then only
.gitattributes in filesystem is used, again if it's not there == no
special attributes.

Then, in any operation, only one .gitattributes is taken into account.
We could stop here, but to me this redundancy still has some room for
confusion and looks unnecessary.
Plus there is still room for unintentional abuse (by mistake).

That's why I think it's a good idea to always have only one .gitattributes.

- Dmitry

Re: .gitignore, .gitattributes, .gitmodules, .gitprecious?,.gitacls? etc.

[Jan Hudec]

[-- Attachment #1: Type: text/plain, Size: 2176 bytes --]

On Tue, Aug 28, 2007 at 09:49:47 +0100, Johannes Schindelin wrote:
> Hi,
> 
> On Mon, 27 Aug 2007, Dmitry Kakurin wrote:
> 
> > Here is the problem: we need to apply crlf attributes to a file. We
> > could have .gitattributes both in the index and in the worktree.
> > Which one do we use?
> > In general .gitattributes file could be (U)nchanged, (C)hanged, (NP)
> > NotPresent in each place.
> 
> I do not see these cases.  You can have these cases, basically:
> 
> - .gitattributes in worktree (then it does not matter what else we have),
> - .gitattributes not in the worktree, but in the index (then that is taken)
> 
> In the latter case, there could be conflicts _in_ .gitattributes, in which 
> case those .gitattributes are ignored.
> 
> I do not see any problem with that.

I do.

IMNSHO it should be the other way around:
 .gitattributes in index, than index version is used.
 .gitattributes not in index, but in worktree, than that tree version is used.

Why? Because when you check out another version, the .gitattributes commited
in that version need to be applied, since it might be different from whatever
is currently in the tree.

In the other direction, the tree version seems to make more sense, but in
reality it does not. If you do a partial commit of a single file, than the
index version gets into the commit, so that should better be the version used
to store the file. On the other hand if you change .gitattributes, the normal
commit rules are that you need to add it for commit, so needing to add it to
make it effective goes well together with it. update-index would need to
always handle .git* before other entries to make add . and commit -a work
correctly.

The case for worktree only is for cases when you for some reason want to have
local .gitattributes. Though I am not sure that should actually work, because
you couldn't have local .gitattributes if there is versioned version.
.git/info/attributes would be better for that. Which gets us back to "always
use .gitattributes from *index*" (and read it from tree before other files
when adding it).

-- 
						 Jan 'Bulb' Hudec <bulb@ucw.cz>

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: .gitignore, .gitattributes, .gitmodules, .gitprecious?, .gitacls? etc.

[Jan Hudec]

[-- Attachment #1: Type: text/plain, Size: 2797 bytes --]

On Wed, Aug 29, 2007 at 00:07:55 +0100, Johannes Schindelin wrote:
> On Wed, 29 Aug 2007, Sam Vilain wrote:
> > Johannes Schindelin wrote:
> > >> Ok, but let's say for a moment that file properties are allowed, and 
> > >> that they are stored in the Reiser4 fashion. On filesystems that did not 
> > >> support this, it would be the only way to get at them - to go through 
> > >> the index. Unless they were also mapped to regular files, or 
> > >> filesystem-specific features somehow.
> > > 
> > > Happily, file properties as well hidden as these have _no_ _place_ in 
> > > source code that needs to be tracked.
> > 
> > But you're restricting your statements to tidy, sane code bases.  Are
> > there any particular reasons that git shouldn't be able to track insane
> > code bases, with attributes etc?  It sure would shut up a whole load of
> > people.
> 
> To the contrary.  People having those insane setups seem to be unable to 
> admit it.  And I'm sure you saw some on this very list, like me.  They 
> never shut up, they only get louder.

Whether it's insane depends on whether you want to keep git purely for
source control -- in which case they are definitely insane -- or want to
allow git to grow to a tool useful for other cases, like tracking content of
/etc, whole filesystem images and similar stuff -- in which case most of
those setups are not insane at all.

Personally I would vote for a middle ground. To keep core git simple, but to
provide enough hooks to build the other tools on top of it. Enough hooks in
this case would mean (at least the way I can imagine as workable) hooks to
run when transfering files between worktree and index and place for the hooks
to store the extra tracked information. This place could be some of special
named file (.gitxattrs or anything) which would be updated in index by the
hook, special index entries somehow related to individual filenames and/or
headers prepended to the blobs and trees. Any of those methods can be used
for storing permissions, acl, extended attributes and other random stuff and
different ones could be used by different hooks at the same time.

Each of those methods has it's advantages and disadvantages. The special
files are easiest to store if the hook is not available or can't represent
the data in given worktree -- simply make the special files regular. The
entries attached to files (which are equivalent of extended attributes) would
work with standard merge on them (because they are handled individually) and
the headers to files would probably require least changes to git (I believe
they are implementable for files with git as is using the hook provided for
keyword expansion (or is it not there?)).

-- 
						 Jan 'Bulb' Hudec <bulb@ucw.cz>

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: .gitignore, .gitattributes, .gitmodules, .gitprecious?, .gitacls? etc.

[Jan Hudec]

[-- Attachment #1: Type: text/plain, Size: 2425 bytes --]

On Mon, Aug 27, 2007 at 17:07:34 +0000, Sergio Callegari wrote:
> David Kastrup <dak <at> gnu.org> writes:
> 
> > 
> > Sergio Callegari <scallegari <at> arces.unibo.it> writes:
> > 
> > > Couldn't all this directory/ownership/permission tracing be easily
> > > done by using hooks?  E.g. Having a pre-status and pre-commit hook
> > > one could fire up a program/script to collect all the extra info he
> > > wants to trace and store it somewhere (typically in some traced
> > > file).  The other way round one could have a post-checkout hook and
> > > he could arrange it to fire up some program to look into the
> > > extra-info file to set up all the meta-data he wants.
> > >
> > > This would be very flexible and would permit to manage absolutely
> > > /any/ kind of the metadata leaving absolute freedom about how to do
> > > so.
> > >
> > > Am I missing something here?
> > 
> > Merging.
> > 
> 
> Sorry, maybe I am really missing something, since merging does not look to me
> as an issue.
> 
> Why cannot git simply do the merging in the working tree as it normally
> does, including merging of the traced metadata file generated by the metadata
> helpers invoked via the hooks?
> Only, again more hooks are needed and likely a post-merge hook, so that at
> the end of the merge, the metadata can be applied.
> 
> Only, to have things going on smoothly, one should be so wise to assure that
> the metadata helpers save metadata as nice, sorted text files in order to
> minimize the burden of manual intervention if there are conflicts in
> metadata merging.

The post-checkout (no need for post-merge -- after in-index merge is done,
the files are checked out to worktree, so post-checkout would run anyway)
could actually apply any custom merge strategy required to avoid/clean up
spurious conflicts in the metadata file (eg. adding two files that go after
each other would be a textual conflict). The relevant versions are stored in
index stages at that point.

> BTW.  Having a post-checkout hook could also help getting rid of unwanted
> empty directories, couldn't it?

Probably not. I would imagine it would actually only run for the files being
checked out -- and there is nothing checked out in empty directories. (Well,
it would run once or once per directory with list of checked out files on
standard input).

-- 
						 Jan 'Bulb' Hudec <bulb@ucw.cz>

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: .gitignore, .gitattributes, .gitmodules, .gitprecious?,.gitacls? etc.

[Dmitry Kakurin]

On 9/4/07, Jan Hudec <bulb@ucw.cz> wrote:
> On Tue, Aug 28, 2007 at 09:49:47 +0100, Johannes Schindelin wrote:
> > On Mon, 27 Aug 2007, Dmitry Kakurin wrote:
> >
> > > Here is the problem: we need to apply crlf attributes to a file. We
> > > could have .gitattributes both in the index and in the worktree.
> > > Which one do we use?
> > > In general .gitattributes file could be (U)nchanged, (C)hanged, (NP)
> > > NotPresent in each place.
> >
> > I do not see these cases.  You can have these cases, basically:
> >
> > - .gitattributes in worktree (then it does not matter what else we have),
> > - .gitattributes not in the worktree, but in the index (then that is taken)
> >
> > In the latter case, there could be conflicts _in_ .gitattributes, in which
> > case those .gitattributes are ignored.
> >
> > I do not see any problem with that.
>
> I do.
>
> IMNSHO it should be the other way around:
>  .gitattributes in index, than index version is used.
>  .gitattributes not in index, but in worktree, than that tree version is used.

Consider scenario when my commit #1 has .gitattributes:
    a.txt -nocrlf
and file a.txt
You pull it.
Now I make some changes to a.txt and realize that a.txt *is* a text
file now. I remove the entry from .gitattributes and notice that it
becomes empty. So I just remove .gitattributes file all together. It
becomes commit#2.
Now you pull it again. There *is* .gitattributes in local directory,
but index does not have it (because I've removed it on purpose). What
should happen?
I assert that since index does not have .gitattributes the one from
local directory should not be used.

Think about dedicated build machine scenario: I have a machine that
always does sync + build. After every sync the local directory should
always be identical to what-was-committed.
With every commit 3 things could happen: .gitattributes could appear,
disappear or change. In every case "build machine" must produce the
exact copy of what-was-checked-in. The only way I see this happening
is by using *only* index version of .gitattributes when files are
moved index -> workspace.

A similar reasoning works for other direction (workplace -> index).
-- 
- Dmitry

Re: .gitignore, .gitattributes, .gitmodules, .gitprecious?,.gitacls? etc.

[Junio C Hamano]

"Dmitry Kakurin" <dmitry.kakurin@gmail.com> writes:

> I assert that since index does not have .gitattributes the one from
> local directory should not be used.
>
> Think about dedicated build machine scenario: I have a machine that
> always does sync + build. After every sync the local directory should
> always be identical to what-was-committed.

Thinking about the reason _why_ .gitattributes may be updated,
one would notice that it is because somebody did this command
sequence:

	git checkout		;# now work tree is clean
	edit .gitattributes	;# modify the attributes of a file
	edit file		;# edit the file attributes talks about
	git add file		;# this can be affected by .gitattributes
	git add .gitattributes	;# this is changed in the same commit
	git commit

Now, should we always take .gitattributes from the index?

Re: .gitignore, .gitattributes, .gitmodules, .gitprecious?,.gitacls? etc.

[Dmitry Kakurin]

On 9/5/07, Junio C Hamano <gitster@pobox.com> wrote:
> "Dmitry Kakurin" <dmitry.kakurin@gmail.com> writes:
>
> > I assert that since index does not have .gitattributes the one from
> > local directory should not be used.
> >
> > Think about dedicated build machine scenario: I have a machine that
> > always does sync + build. After every sync the local directory should
> > always be identical to what-was-committed.
>
> Thinking about the reason _why_ .gitattributes may be updated,
> one would notice that it is because somebody did this command
> sequence:
>
>        git checkout            ;# now work tree is clean
>        edit .gitattributes     ;# modify the attributes of a file
>        edit file               ;# edit the file attributes talks about
>        git add file            ;# this can be affected by .gitattributes
>        git add .gitattributes  ;# this is changed in the same commit
>        git commit
>
> Now, should we always take .gitattributes from the index?
No, from couple of my emails back:

> This leads to a simple idea that mostly works:
> 1. when files are moved from index to filesystem, then only .gitattributes in the index is used, if it's not there == no special attributes.
> 2. when files are moved from filesystem to index, then only .gitattributes in filesystem is used, again if it's not there == no special attributes.
>
> Then, in any operation, only one .gitattributes is taken into account.

This is a must-do change (IMHO).

To go one step further (again IMHO) is to eliminate workspace version
of .gitattributes all together:
> We could stop here, but to me this redundancy still has some room for confusion and looks unnecessary.
> Plus there is still room for unintentional abuse (by mistake).
> That's why I think it's a good idea to always have only one .gitattributes (in the index).

-- 
- Dmitry

Re: .gitignore, .gitattributes, .gitmodules, .gitprecious?,.gitacls? etc.

[Jan Hudec]

[-- Attachment #1: Type: text/plain, Size: 1982 bytes --]

On Wed, Sep 05, 2007 at 01:14:44 -0700, Junio C Hamano wrote:
> "Dmitry Kakurin" <dmitry.kakurin@gmail.com> writes:
> 
> > I assert that since index does not have .gitattributes the one from
> > local directory should not be used.
> >
> > Think about dedicated build machine scenario: I have a machine that
> > always does sync + build. After every sync the local directory should
> > always be identical to what-was-committed.
> 
> Thinking about the reason _why_ .gitattributes may be updated,
> one would notice that it is because somebody did this command
> sequence:
> 
> 	git checkout		;# now work tree is clean
> 	edit .gitattributes	;# modify the attributes of a file
> 	edit file		;# edit the file attributes talks about
> 	git add file		;# this can be affected by .gitattributes
> 	git add .gitattributes	;# this is changed in the same commit
> 	git commit
> 
> Now, should we always take .gitattributes from the index?

Yes, they should:

$ git checkout
$ edit .gitattributes
$ edit file
$ git add file
$ git commit ;# this does NOT have the changes to .gitattributes

the above case is a user error that can (at some cost) be detected:

$ git checkout
$ edit .gitattributes
$ edit file
$ git add file
$ git add .gitattributes
Warning! Changes to gitattributes affects handling of files scheduled for
commit. Please add following files again before commit:
  file
$

It would be possible to special-case .gitattributes in add to:
 - do diff between the old and new value of .gitattributes in index,
 - list files changed in index compared to HEAD,
 - match each of them to all patterns in the diff,
 - if any matches, print the warning and list of matches.
It might be even possible to actually inspect the changes and apply those
that can be automatically (and not ask user to re-add), but some filters
loose information, so user interaction is needed to add good version.

-- 
						 Jan 'Bulb' Hudec <bulb@ucw.cz>

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]