ident hash usage question

ident hash usage question

[Eugene Sajine]

Hi,

Please correct me if I’m missing something, but:

I was able to get the $Id$ keyword to show the SHA-1 hash of the file
content, which is done using ident in gitattributes. Now I’m trying to
find a way to get more info about the file using only this hash.

Could you , please, advise if there is a way to get file name and path
inside the repo by having only this Blob hash?

Thank you,
Eugene

Re: ident hash usage question

[Alex Riesen]

On Tue, Oct 20, 2009 at 19:24, Eugene Sajine <euguess@gmail.com> wrote:
> I was able to get the $Id$ keyword to show the SHA-1 hash of the file
> content, which is done using ident in gitattributes. Now I’m trying to
> find a way to get more info about the file using only this hash.
>
> Could you , please, advise if there is a way to get file name and path
> inside the repo by having only this Blob hash?

It is not possible.

But you can save the commits hash in your blob. Then you'll have everything.
That how git knows it's version.

Re: ident hash usage question

[Alex Riesen]

On Tue, Oct 20, 2009 at 20:16, Alex Riesen <raa.lkml@gmail.com> wrote:
> On Tue, Oct 20, 2009 at 19:24, Eugene Sajine <euguess@gmail.com> wrote:
>> I was able to get the $Id$ keyword to show the SHA-1 hash of the file
>> content, which is done using ident in gitattributes. Now I’m trying to
>> find a way to get more info about the file using only this hash.
>>
>> Could you , please, advise if there is a way to get file name and path
>> inside the repo by having only this Blob hash?
>
> It is not possible.
>

Well, not exactly impossible, but you can end up with multiple paths,
some of which may not have anything to do the original path.

Just run git log --no-abbrev --raw --all and grep for the SHA-1

Re: ident hash usage question

[Eugene Sajine]

>
> Well, not exactly impossible, but you can end up with multiple paths,
> some of which may not have anything to do the original path.
>
> Just run git log --no-abbrev --raw --all and grep for the SHA-1
>

This is exactly what I was looking for! Thank you!

I understand that in some cases this can give me two paths instead of
one, but it will only demonstrate that I have absolute copy of a file
inside my repo, which is also good to diagnose (because the
probability of having two meaningful files with the same hash, but
different content is way too low).

So, this means that after few little tricks the keyword expansion
problem may be resolved by only using $Id$ keyword.
Because after having this hash one can build up all necessary info from it:

#finding blobs with SHA indicated in $Id$ keword
$ git log --no-abbrev --raw --all | grep SHA-1

# little script or regexp here (don’t have it)
$ pull out path from result

# last commit for the path with all corresponding info
$ git log -1 HEAD path

So, this seems to cover most of the needs of people who would like to
use keywords expansion, if they are not ready to forget about them…

Does it make sense?

Thanks,
Eugene

Re: ident hash usage question

[Alex Riesen]

On Tue, Oct 20, 2009 at 20:59, Eugene Sajine <euguess@gmail.com> wrote:
>>
>> Well, not exactly impossible, but you can end up with multiple paths,
>> some of which may not have anything to do the original path.
>>
>> Just run git log --no-abbrev --raw --all and grep for the SHA-1
>
> This is exactly what I was looking for! Thank you!
>

Very likely it isn't, but it's your choice.

> Because after having this hash one can build up all necessary info from it:

Depending on your definition of "necessary".

> #finding blobs with SHA indicated in $Id$ keword
> $ git log --no-abbrev --raw --all | grep SHA-1

yeah. These are all starting from commit which introduced
the hash under a specific path, ending at the commit where
the path contains another SHA-1.

> # little script or regexp here (don’t have it)
> $ pull out path from result
>
> # last commit for the path with all corresponding info
> $ git log -1 HEAD path
>
> So, this seems to cover most of the needs of people who would like to
> use keywords expansion, if they are not ready to forget about them…
>
> Does it make sense?

Not much. You'll always get a long list of commits which didn't
change the damned blob. And you have absolutely no way
to find out exactly which of the commits have produced
the blob you're looking at (because you decided to do away
with the information).

What's so hard with storing the SHA-1 of the *commit*, anyway?

Re: ident hash usage question

[Johannes Sixt]

On Dienstag, 20. Oktober 2009, Alex Riesen wrote:
> What's so hard with storing the SHA-1 of the *commit*, anyway?

The hard part is that you get a cycle: commit SHA1 depends on contents depends 
on commit SHA1.

-- Hannes

Re: ident hash usage question

[Eugene Sajine]

First, thank you for your answers!

>
> Very likely it isn't, but it's your choice.

;)

>
>> Because after having this hash one can build up all necessary info from it:
>
> Depending on your definition of "necessary".

I'm trying to find a way to get to state where i can do

$ git log <path>

Or any other variants of it without introducing any non-default
scripts/features/keywords and limiting keywords to one to avoid any
related problem...

One of my friends said that git is not working for their development
model... C++ development with static linking across the board, where
they need to see exactly which version of the file has got to the
executable. Roughly, they are using CVS' keywords and revision numbers
and a script wich matches them between two versions of the
executables.

I've got curious if Git can support it and how it can be done with
minimal changes to workflow.

>
>> #finding blobs with SHA indicated in $Id$ keword
>> $ git log --no-abbrev --raw --all | grep SHA-1
>
> yeah. These are all starting from commit which introduced
> the hash under a specific path, ending at the commit where
> the path contains another SHA-1.

Actually, grep "SHA-1 A" will show added path(s) only, so this is resolved

>
>> # little script or regexp here (don’t have it)
>> $ pull out path from result
>>
>> # last commit for the path with all corresponding info
>> $ git log -1 HEAD path
>>
>> So, this seems to cover most of the needs of people who would like to
>> use keywords expansion, if they are not ready to forget about them…
>>
>> Does it make sense?
>
> Not much. You'll always get a long list of commits which didn't
> change the damned blob. And you have absolutely no way
> to find out exactly which of the commits have produced
> the blob you're looking at (because you decided to do away
> with the information).

How is that? It seams to me that git log <path> will show only commits
where <path> was changed/committed? Considering the fact that I've got
the initial path from the blob, i should get the exact commit history
(or last commit in my example) for the file(s) (Files if renaming
occurred without content change).

Thanks,
Eugene

Re: ident hash usage question

[Alex Riesen]

On Tue, Oct 20, 2009 at 22:22, Johannes Sixt <j6t@kdbg.org> wrote:
> On Dienstag, 20. Oktober 2009, Alex Riesen wrote:
>> What's so hard with storing the SHA-1 of the *commit*, anyway?
>
> The hard part is that you get a cycle: commit SHA1 depends on contents depends
> on commit SHA1.

Don't store it in the repo. Store it in the output.

Re: ident hash usage question

[Junio C Hamano]

Eugene Sajine <euguess@gmail.com> writes:

> One of my friends said that git is not working for their development
> model... C++ development with static linking across the board, where
> they need to see exactly which version of the file has got to the
> executable. Roughly, they are using CVS' keywords and revision numbers
> and a script wich matches them between two versions of the
> executables.
>
> I've got curious if Git can support it and how it can be done with
> minimal changes to workflow.

If the project is already arranged to be compiled with decent automation,
I do not think you need any change to the workflow.

You would have a version.cc file with

	static char program_version[] = "My Program " VERSION_STRING;

in it, and teach the build procedure how to compile and link this file.
Something like:

    version.o: version.cc
    	$(CXX) -o $@ -DVERSION_STRING=\""$(git describe HEAD)"\" $?

Re: ident hash usage question

[Alex Riesen]

On Tue, Oct 20, 2009 at 22:30, Eugene Sajine <euguess@gmail.com> wrote:
> One of my friends said that git is not working for their development
> model... C++ development with static linking across the board, where
> they need to see exactly which version of the file has got to the
> executable. Roughly, they are using CVS' keywords and revision numbers
> and a script wich matches them between two versions of the
> executables.

As soon as your friend understand, that a commit describes the complete
state of the repository on the moment of commit, he/she will notice that
the commit allows to find each what content each file in the product
had at the moment of compilation and linking (assuming they weren't
compiling uncommitted tree, which RCS/CVS/SVN/Perforce mindset
tends to encourage).

> I've got curious if Git can support it and how it can be done with
> minimal changes to workflow.

Depends on workflow, I afraid. And I personally wouldn't bother.

>>> # little script or regexp here (don’t have it)
>>>
>>> Does it make sense?
>>
>> Not much. You'll always get a long list of commits which didn't
>> change the damned blob. And you have absolutely no way
>> to find out exactly which of the commits have produced
>> the blob you're looking at (because you decided to do away
>> with the information).
>
> How is that? It seams to me that git log <path> will show only commits
> where <path> was changed/committed? Considering the fact that I've got
> the initial path from the blob, i should get the exact commit history
> (or last commit in my example) for the file(s) (Files if renaming
> occurred without content change).

The blob is present in each commit since it was introduced. Except
when your project contains only that one blob, isn't the state of
the other parts of an interest?

Re: ident hash usage question

[Eugene Sajine]

> If the project is already arranged to be compiled with decent automation,
> I do not think you need any change to the workflow.
>
> You would have a version.cc file with
>
>        static char program_version[] = "My Program " VERSION_STRING;
>
> in it, and teach the build procedure how to compile and link this file.
> Something like:
>
>    version.o: version.cc
>        $(CXX) -o $@ -DVERSION_STRING=\""$(git describe HEAD)"\" $?
>

Please, correct me if I'm mistaken and forgive me if I'm not correct
in using C++ terms.

Your solution proposes to have a version file which will carry the
info about last state the program was built from.
But as I understand in case of static linking the executable will get
only obj files from a library, which are necessary and everything
irrelevant will be thrown away by linker. In this circumstances the
program comparing two executable will not notice any differences
between let's say existing production and new development version of
the executable. In case of several libraries linked this gets even
worse. That's what their basis is for keywords expansion use.

If there is no other solution in VCS terms how to understand which
particular version of file has got linked to the executable except
keywords expansion, then I'm talking about the way of reducing the
amount of keywords expanded to a minimum - 1.

If we can agree on this, then it seems that having the content/blob
hash in the file $Id$ is enough to get any other relevant info about
the file.

At least by hash you can find the path added/modified. When the path
is on hands you can easily realize the history for the file. It seems
to me that using content hash will provide unique results enough to
make conclusions, because the probability of having two files with
same hash but different content is too low, and only scenario is to
get the same hash is to get the absolute copy of the file, but such
file existence is questionable...

OTOH, Alex was right that there are situation where this is not
working as expected from the first run:
If the file was modified its blob hash is going to change, but there
will be no entry with letter A in the log. So, if

$ git log --no-abbrev --raw --all | grep "SHA-1 A"

Returns empty string then, we should do that without "A".
As I understand this *second* run should always return:

1. only one entry if this is latest hash - one path. Goal reached.
2. two entries if there was a change afterwards - One path. Goal reached
3. three entries if rename was committed separately and then renamed
file was changed - two paths. Enough to make conclusion about the
files.

In any case the  goal is to get the path as correct as possible and
then proceed with matching and other stuff.


Thanks,
Eugene

Re: ident hash usage question

[Eugene Sajine]

>> How is that? It seams to me that git log <path> will show only commits
>> where <path> was changed/committed? Considering the fact that I've got
>> the initial path from the blob, i should get the exact commit history
>> (or last commit in my example) for the file(s) (Files if renaming
>> occurred without content change).
>
> The blob is present in each commit since it was introduced. Except
> when your project contains only that one blob, isn't the state of
> the other parts of an interest?
>
I would question this statement. It seems to me that hash of the file
content is logged only for the commit when it was touched.
Therefore there is very limited amount of actual commits where the
same hash can be met.

Thanks,
Eugene

Re: ident hash usage question

[Junio C Hamano]

Eugene Sajine <euguess@gmail.com> writes:

>> If the project is already arranged to be compiled with decent automation,
>> I do not think you need any change to the workflow.
>>
>> You would have a version.cc file with
>>
>>        static char program_version[] = "My Program " VERSION_STRING;
>>
>> in it, and teach the build procedure how to compile and link this file.
>> Something like:
>>
>>    version.o: version.cc
>>        $(CXX) -o $@ -DVERSION_STRING=\""$(git describe HEAD)"\" $?
>>
>
> Please, correct me if I'm mistaken and forgive me if I'm not correct
> in using C++ terms.
>
> Your solution proposes to have a version file which will carry the
> info about last state the program was built from.
> But as I understand in case of static linking the executable will get
> only obj files from a library, which are necessary and everything
> irrelevant will be thrown away by linker.

I am not sure what you mean by "static linking" anymore.  Usually the word
means that everything you tell the linker to link to the executable is
linked, together with objects from libraries.  The resulting executable is
usable on its own and it does not change behaviour regardless of which
version of dynamic libraries you depend on happen to be installed on the
target system (because by definition a statically linked executable does
not depend on dynamic libraries---that is the whole point of static
linking).

So as long as you make sure version.o is linked to the final executable,
you will be fine.  One way to do so might be to have

	printf("Program version: %s", program_version);

at the beginning of main() ;-)

If your product ships as one main executable _dynamically_ linked with two
dynamic libraries, and all three components are built from the source
material under your source control, obviously you would need to make sure
that the above version.o or some equivalent of what embeds output from
"git describe HEAD" are linked to the main executable and to the two
libraries, but the idea is the same.

Re: ident hash usage question

[Eugene Sajine]

> I am not sure what you mean by "static linking" anymore.  Usually the word
> means that everything you tell the linker to link to the executable is
> linked, together with objects from libraries.  The resulting executable is
> usable on its own and it does not change behaviour regardless of which
> version of dynamic libraries you depend on happen to be installed on the
> target system (because by definition a statically linked executable does
> not depend on dynamic libraries---that is the whole point of static
> linking).
>

There seems to be no misunderstanding in the static linking - i meant
the same thing. But let me put an example:

I have a program.exe

This program.exe is built basing on two statically linked libraries
lib1.lib and lib2.lib

I'm not developing any of those libraries, but only my own code of the
program.exe

Now, somebody changed 2 files in lib1.lib and 5 files in lib2.lib. But
i don't know that they are changed because it is different CVS module
or because I'm building against latest released libs or for whatever
reason...

When i rebuild my program the build supposed to pick up changes from
the libraries I'm using and relink, that will include 7 changed obj
files.

How can i say which exactly files are changed in my new version of
executable comparing to the previous version?

Currently they can take a look at the revision number of every
particular file included into the executable, which is put there by
CVS and compare it with the production. If the version is different,
then you know which files are changed and you can get diffs on them...
They also have file path and date and other stuff expanded...

Please note, my personal goal is *to prove that git can do that
better*, with less intrusion into the code sources, not other way
around.So, while keeping the info they want to have they might get
some benefits of git. Although, i understand that there might be no
cure for this state already, you can tell me that and I will close the
topic, but I just keep hoping;)


Thanks a lot,
Eugene

Re: ident hash usage question

[Nanako Shiraishi]

Quoting Junio C Hamano <gitster@pobox.com>

> If your product ships as one main executable _dynamically_ linked with two
> dynamic libraries, and all three components are built from the source
> material under your source control, obviously you would need to make sure
> that the above version.o or some equivalent of what embeds output from
> "git describe HEAD" are linked to the main executable and to the two
> libraries, but the idea is the same.

Doesn't the above strategy also apply to the case where the 
libraries are linked statically, too? You get the version 
string from the main program and two version strings from the 
libraries embedded in the final product.

-- 
Nanako Shiraishi
http://ivory.ap.teacup.com/nanako3/

Re: ident hash usage question

[Daniel Barkalow]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 4248 bytes --]

On Tue, 20 Oct 2009, Eugene Sajine wrote:

> > I am not sure what you mean by "static linking" anymore.  Usually the word
> > means that everything you tell the linker to link to the executable is
> > linked, together with objects from libraries.  The resulting executable is
> > usable on its own and it does not change behaviour regardless of which
> > version of dynamic libraries you depend on happen to be installed on the
> > target system (because by definition a statically linked executable does
> > not depend on dynamic libraries---that is the whole point of static
> > linking).
> >
> 
> There seems to be no misunderstanding in the static linking - i meant
> the same thing. But let me put an example:
> 
> I have a program.exe
> 
> This program.exe is built basing on two statically linked libraries
> lib1.lib and lib2.lib
> 
> I'm not developing any of those libraries, but only my own code of the
> program.exe
> 
> Now, somebody changed 2 files in lib1.lib and 5 files in lib2.lib. But
> i don't know that they are changed because it is different CVS module
> or because I'm building against latest released libs or for whatever
> reason...
> 
> When i rebuild my program the build supposed to pick up changes from
> the libraries I'm using and relink, that will include 7 changed obj
> files.
> 
> How can i say which exactly files are changed in my new version of
> executable comparing to the previous version?

As Nanako points out, you want a version string from each library, 
probably with the name of the library in the symbol. So:

static char *lib1_version = "" VERSION_STRING

in lib1/version.cc.

> Currently they can take a look at the revision number of every
> particular file included into the executable, which is put there by
> CVS and compare it with the production. If the version is different,
> then you know which files are changed and you can get diffs on them...
> They also have file path and date and other stuff expanded...

Once you've got the version strings from each of the components, and 
you've got the repository for each library, you can compare the version in 
the string with other versions, and that one hash expands to the hashes 
for all of the files, including things like the Makefile that don't 
actually become object files but affect them.

You can also do things like: when you build an executable, write it as a 
blob to the repository and make a tag (actually, a note will be 
better, but that's newer than I've done this) pointing to it that lists 
everything used to build it. If you encounter the executable again later, 
it'll have the same hash, and git can find the tag with the other 
information.

I've actually done something where I tagged the main tree and a few 
libraries with matching tags, built the executable with a 20-byte 
sequence of zeros, put it into the repository, then wrote its sha1 to 
those 20 bytes, and tagged the executable with a tag based on the source 
tag. Then I put it on a microcontroller, sealed it in epoxy, mixed it with 
other devices with different firmware revisions, and sent it out. When a 
device had a problem, I could ask it for those 20 bytes, ask git for the 
tag that pointed to those 20 bytes, and check out exactly the source used 
to build it.

> Please note, my personal goal is *to prove that git can do that
> better*, with less intrusion into the code sources, not other way
> around.So, while keeping the info they want to have they might get
> some benefits of git. Although, i understand that there might be no
> cure for this state already, you can tell me that and I will close the
> topic, but I just keep hoping;)

You're not going to be able to avoid having different tracking things in 
the source, but git does give you the major advantage that you can have 
the build system produce one hash that identifies every single file in the 
project, rather than having to have each file identify itself (and 
probably overlook files that don't directly produce object code). It's a 
big help the day you're trying to figure out if an executable came from 
before or after someone's commit to change the compiler options, and it's 
fewer strings than you need with CVS.

	-Daniel
*This .sig left intentionally blank*

Re: ident hash usage question

[Alex Riesen]

On Wed, Oct 21, 2009 at 00:19, Eugene Sajine <euguess@gmail.com> wrote:
>>> How is that? It seams to me that git log <path> will show only commits
>>> where <path> was changed/committed? Considering the fact that I've got
>>> the initial path from the blob, i should get the exact commit history
>>> (or last commit in my example) for the file(s) (Files if renaming
>>> occurred without content change).
>>
>> The blob is present in each commit since it was introduced. Except
>> when your project contains only that one blob, isn't the state of
>> the other parts of an interest?
>>
> I would question this statement. It seems to me that hash of the file
> content is logged only for the commit when it was touched.
> Therefore there is very limited amount of actual commits where the
> same hash can be met.

git rev-list HEAD | while read sha; do git ls-tree $sha; done | grep <blob-sha>

Re: ident hash usage question

[Eugene Sajine]

Many thanks to everybody for your time and comments!

I have plenty of info to think about now...;)

With best regards,
Eugene