[GSoC] Hello Git

[GSoC] Hello Git

[ZheNing Hu]

Hello，Git!

### Self introduction

I'm ZheNing Hu, I am very lucky to participate in the GSoC
git project this year. Many people in the git community already
have given me a lot of help in the past few months.
Junio, Christian, Peff, Eric, Denton..., it's great to get along with
you guys! Your review and guidance have greatly benefited
my growth.

In these days before GSoC, I learned some simple command
implementation and data structures of git, learned strict coding
standards and learned how to test, More importantly, I learned how
to communicate with these kind-hearted reviewers.

Of course, there are still many difficulties for me:
* My poor English sometimes can’t express the meaning clearly.
* The Git architecture is very large, even a small command may
require very complex logic to implement, I often feel very difficult
to find clues.

Therefore, during GSoC, I hope to overcome these difficulties and
learn more in-depth knowledge in git.

### How to complete my GSoC project

I don’t want to be rejected by Git after I have completed a lot of
content, as Olga once did, Therefore, I think it is best to move in
the right direction step by step under everyone's supervision.

Current condition:
1. In order to use a unified interface containing both short-named
and full-named atoms, inspired by the short name option and long name
option of "OPT_*" in `parse-options.h`,  I thought I should add a new short
name atom to `struct valid_atom` for expansion in `ref-filter.c`, In [1]:
Junio thinks that `--format` with two-letters short atoms should not be
implemented in `ref-filter.c`
and provided for the `for-each-ref` family. Instead, let the "log" family
learn the full-named atom. To ensure the consistency of `log --format`
and `for-each-ref --format`. But for now, It seems that the parsing of
full-named atoms in `pretty.c` is not very elegant.
E.g.

   if (skip_prefix(placeholder, "(describe", &arg)) {
   ...
   }

       if (skip_prefix(placeholder, "(trailers", &arg)) {
   ...
   }

We should have an atom table like `valid_atom` in `ref-filter.c`,
but this is missing in `pretty.c`. So how do we complete the
unification of the two types of atoms now?

2. "50cfe1f([GSOC] ref-filter: introduce enum atom_type)" has
been merged into the `next` branch, should we focus on the
performance optimization of `ref-filter.c`? Finally, we can build a
`format_cat_file_item()` similar to `format_ref_array_item()`,
which should not be difficult. And we have to make sure that the
performance of `cat-file --batch` is not lower than before.

### The connection between Git and the file system

I am currently studying operating system course, git as a
file system on top of the file system.
There are many differences and similarities with ordinary
file systems. I am very curious about this knowledge, do
not know if there are any relevant reading materials?

### Motto

I remember I saw an interview about Junio and Dan Kogai before [2]:

Dan: What do you think is a "good engineer"?

Junio: When I took over the maintainer of Git, Linus told me that star
engineers have three qualities. The most important thing is to be able
to keep doing one thing all the time. In that sense, you can't be an
alpha geek. It's sharp and you can jump to a new one right away,
but don't get bored right away.

In other words, it’s one thing to have enthusiasm, but it’s more
important to persevere.

When I read the Git source code, I can see the code written by Peff
15 years ago and the code written by Peff a few months ago, which
is very touching. It's as if you guys can travel through time and space.

So, I hope I can also persevere and fight for interest. After 15 years,
let’s take a look at the code I wrote today :)

[1]:https://lore.kernel.org/git/xmqqk0nz72on.fsf@gitster.g/
[2]:https://gihyo.jp/dev/serial/01/alpha-geek/0040?page=2

Re: [GSoC] Hello Git

[Christian Couder]

Hi ZheNing,

On Tue, May 18, 2021 at 8:40 AM ZheNing Hu <adlternative@gmail.com> wrote:
>
> Hello，Git!
>
> ### Self introduction
>
> I'm ZheNing Hu, I am very lucky to participate in the GSoC
> git project this year. Many people in the git community already
> have given me a lot of help in the past few months.
> Junio, Christian, Peff, Eric, Denton..., it's great to get along with
> you guys! Your review and guidance have greatly benefited
> my growth.

We are very happy with your application to the GSoC with us, and with
your work so far!

> In these days before GSoC, I learned some simple command
> implementation and data structures of git, learned strict coding
> standards and learned how to test, More importantly, I learned how
> to communicate with these kind-hearted reviewers.
>
> Of course, there are still many difficulties for me:
> * My poor English sometimes can’t express the meaning clearly.
> * The Git architecture is very large, even a small command may
> require very complex logic to implement, I often feel very difficult
> to find clues.
>
> Therefore, during GSoC, I hope to overcome these difficulties and
> learn more in-depth knowledge in git.

Sure!

> ### How to complete my GSoC project
>
> I don’t want to be rejected by Git after I have completed a lot of
> content, as Olga once did, Therefore, I think it is best to move in
> the right direction step by step under everyone's supervision.

Sometimes it's very difficult to know in advance what will work well
enough to be accepted. And if something doesn't work well enough we
can still learn from that, and sometimes use it as a base to implement
something better.

> Current condition:
> 1. In order to use a unified interface containing both short-named
> and full-named atoms, inspired by the short name option and long name
> option of "OPT_*" in `parse-options.h`,  I thought I should add a new short
> name atom to `struct valid_atom` for expansion in `ref-filter.c`, In [1]:
> Junio thinks that `--format` with two-letters short atoms should not be
> implemented in `ref-filter.c` and provided for the `for-each-ref` family.
> Instead, let the "log" family learn the full-named atom. To ensure the
> consistency of `log --format` and `for-each-ref --format`.

Yeah.

> But for now,
> It seems that the parsing of full-named atoms in `pretty.c` is not very elegant.
> E.g.
>
>    if (skip_prefix(placeholder, "(describe", &arg)) {
>    ...
>    }
>
>        if (skip_prefix(placeholder, "(trailers", &arg)) {
>    ...
>    }
>
> We should have an atom table like `valid_atom` in `ref-filter.c`,
> but this is missing in `pretty.c`. So how do we complete the
> unification of the two types of atoms now?

Yeah, unifying pretty.c and ref-filter.c was the purpose of Hariom's
GSoC last year, and there is still work to do on this, but you might
want to focus on the format code used by git cat-file first, as that's
what your project is about.

> 2. "50cfe1f([GSOC] ref-filter: introduce enum atom_type)" has
> been merged into the `next` branch, should we focus on the
> performance optimization of `ref-filter.c`? Finally, we can build a
> `format_cat_file_item()` similar to `format_ref_array_item()`,
> which should not be difficult. And we have to make sure that the
> performance of `cat-file --batch` is not lower than before.

I wonder if format_ref_array_item() could not just be used by
batch_object_write() like Olga did it in:

https://github.com/git/git/pull/568/commits/f911b342ae6503dc9f6f8e3c7df316aa45d97bdf

> ### The connection between Git and the file system
>
> I am currently studying operating system course, git as a
> file system on top of the file system.
> There are many differences and similarities with ordinary
> file systems. I am very curious about this knowledge, do
> not know if there are any relevant reading materials?

You might be interested in https://ostreedev.github.io/ostree/

Best,
Christian.

Re: [GSoC] Hello Git

[ZheNing Hu]

Christian Couder <christian.couder@gmail.com> 于2021年5月18日周二 下午10:32写道：
>
> Hi ZheNing,
>
> On Tue, May 18, 2021 at 8:40 AM ZheNing Hu <adlternative@gmail.com> wrote:
> >
> > Hello，Git!
> >
> > ### Self introduction
> >
> > I'm ZheNing Hu, I am very lucky to participate in the GSoC
> > git project this year. Many people in the git community already
> > have given me a lot of help in the past few months.
> > Junio, Christian, Peff, Eric, Denton..., it's great to get along with
> > you guys! Your review and guidance have greatly benefited
> > my growth.
>
> We are very happy with your application to the GSoC with us, and with
> your work so far!
>

Thanks!

> > But for now,
> > It seems that the parsing of full-named atoms in `pretty.c` is not very elegant.
> > E.g.
> >
> >    if (skip_prefix(placeholder, "(describe", &arg)) {
> >    ...
> >    }
> >
> >        if (skip_prefix(placeholder, "(trailers", &arg)) {
> >    ...
> >    }
> >
> > We should have an atom table like `valid_atom` in `ref-filter.c`,
> > but this is missing in `pretty.c`. So how do we complete the
> > unification of the two types of atoms now?
>
> Yeah, unifying pretty.c and ref-filter.c was the purpose of Hariom's
> GSoC last year, and there is still work to do on this, but you might
> want to focus on the format code used by git cat-file first, as that's
> what your project is about.
>

Oh, what you said makes sense. The refactoring of `cat-file --batch`
using `ref-filter` logic seems to have little to do with the task of letting
`ref-filter` learn short-name atoms at the time. I will shift the focus of
my work

> > 2. "50cfe1f([GSOC] ref-filter: introduce enum atom_type)" has
> > been merged into the `next` branch, should we focus on the
> > performance optimization of `ref-filter.c`? Finally, we can build a
> > `format_cat_file_item()` similar to `format_ref_array_item()`,
> > which should not be difficult. And we have to make sure that the
> > performance of `cat-file --batch` is not lower than before.
>
> I wonder if format_ref_array_item() could not just be used by
> batch_object_write() like Olga did it in:
>
> https://github.com/git/git/pull/568/commits/f911b342ae6503dc9f6f8e3c7df316aa45d97bdf
>

Thanks for the tip! Olga uses `format_cat_file_item()` in
batch_object_write() , which may be the correct approach.
Before that, we have to make ref-filter support "%(contents:raw)"
(similar to Olga's "%(raw)") and "%(rest)".

> > ### The connection between Git and the file system
> >
> > I am currently studying operating system course, git as a
> > file system on top of the file system.
> > There are many differences and similarities with ordinary
> > file systems. I am very curious about this knowledge, do
> > not know if there are any relevant reading materials?
>
> You might be interested in https://ostreedev.github.io/ostree/
>
> Best,
> Christian.

Thanks!
--
ZheNing Hu

Re: [GSoC] Hello Git

[Christian Couder]

On Thu, May 20, 2021 at 9:14 AM ZheNing Hu <adlternative@gmail.com> wrote:
>
> Christian Couder <christian.couder@gmail.com> 于2021年5月18日周二 下午10:32写道：

> > > 2. "50cfe1f([GSOC] ref-filter: introduce enum atom_type)" has
> > > been merged into the `next` branch, should we focus on the
> > > performance optimization of `ref-filter.c`? Finally, we can build a
> > > `format_cat_file_item()` similar to `format_ref_array_item()`,
> > > which should not be difficult. And we have to make sure that the
> > > performance of `cat-file --batch` is not lower than before.
> >
> > I wonder if format_ref_array_item() could not just be used by
> > batch_object_write() like Olga did it in:
> >
> > https://github.com/git/git/pull/568/commits/f911b342ae6503dc9f6f8e3c7df316aa45d97bdf
>
> Thanks for the tip! Olga uses `format_cat_file_item()` in

You mean `format_ref_array_item()` not `format_cat_file_item()`, right?

> batch_object_write() , which may be the correct approach.
> Before that, we have to make ref-filter support "%(contents:raw)"
> (similar to Olga's "%(raw)") and "%(rest)".

Yeah, please take a look, if you haven't already, at
https://github.com/git/git/pull/568 where Olga already worked on that.
It would likely help if you can reuse her work.

Re: [GSoC] Hello Git

[ZheNing Hu]

Christian Couder <christian.couder@gmail.com> 于2021年5月20日周四 下午4:00写道：
>
> On Thu, May 20, 2021 at 9:14 AM ZheNing Hu <adlternative@gmail.com> wrote:
> >
> > Christian Couder <christian.couder@gmail.com> 于2021年5月18日周二 下午10:32写道：
>
> > > > 2. "50cfe1f([GSOC] ref-filter: introduce enum atom_type)" has
> > > > been merged into the `next` branch, should we focus on the
> > > > performance optimization of `ref-filter.c`? Finally, we can build a
> > > > `format_cat_file_item()` similar to `format_ref_array_item()`,
> > > > which should not be difficult. And we have to make sure that the
> > > > performance of `cat-file --batch` is not lower than before.
> > >
> > > I wonder if format_ref_array_item() could not just be used by
> > > batch_object_write() like Olga did it in:
> > >
> > > https://github.com/git/git/pull/568/commits/f911b342ae6503dc9f6f8e3c7df316aa45d97bdf
> >
> > Thanks for the tip! Olga uses `format_cat_file_item()` in
>
> You mean `format_ref_array_item()` not `format_cat_file_item()`, right?
>

Yes. `format_ref_array_item()`.

> > batch_object_write() , which may be the correct approach.
> > Before that, we have to make ref-filter support "%(contents:raw)"
> > (similar to Olga's "%(raw)") and "%(rest)".
>
> Yeah, please take a look, if you haven't already, at
> https://github.com/git/git/pull/568 where Olga already worked on that.
> It would likely help if you can reuse her work.

I have check them, I want to use "%(contents:raw)" instead of %(raw),
https://github.com/gitgitgadget/git/pull/958 tell the reason,
I will submit it later.

--
ZheNing Hu

Re: [GSoC] Hello Git

[Christian Couder]

On Thu, May 20, 2021 at 10:09 AM ZheNing Hu <adlternative@gmail.com> wrote:

> Christian Couder <christian.couder@gmail.com> 于2021年5月20日周四 下午4:00写道：
> > > Before that, we have to make ref-filter support "%(contents:raw)"
> > > (similar to Olga's "%(raw)") and "%(rest)".
> >
> > Yeah, please take a look, if you haven't already, at
> > https://github.com/git/git/pull/568 where Olga already worked on that.
> > It would likely help if you can reuse her work.
>
> I have check them, I want to use "%(contents:raw)" instead of %(raw),
> https://github.com/gitgitgadget/git/pull/958 tell the reason,
> I will submit it later.

Great, thanks!

Re: [GSoC] Hello Git

[Jiang Xin]

ZheNing Hu <adlternative@gmail.com> 于2021年5月18日周二 下午2:40写道：
>
> Hello，Git!
>
> ### Self introduction
>
> I'm ZheNing Hu, I am very lucky to participate in the GSoC
> git project this year. Many people in the git community already
> have given me a lot of help in the past few months.
> Junio, Christian, Peff, Eric, Denton..., it's great to get along with
> you guys! Your review and guidance have greatly benefited
> my growth.

I admire you so much that you can join the open source community at a
very young age. I will continue to offer my help both online and
offline, in your favorite language (Chinese of course).

> In these days before GSoC, I learned some simple command
> implementation and data structures of git, learned strict coding
> standards and learned how to test, More importantly, I learned how
> to communicate with these kind-hearted reviewers.
>
> Of course, there are still many difficulties for me:
> * My poor English sometimes can’t express the meaning clearly.

There are too many differences between English and Chinese, but don't
worry. Remember, practice is perfect. You can also practice your
English by reviewing Git l10n in Chinese, see: "po/README.md".

> ### The connection between Git and the file system
>
> I am currently studying operating system course, git as a
> file system on top of the file system.
> There are many differences and similarities with ordinary
> file systems. I am very curious about this knowledge, do
> not know if there are any relevant reading materials?

Git objects are stored in ".git/objects" directory in two forms: loose
object and packfile.  These two types of files are deflated in zlib
format. In order to increase the speed of traversing these objects in
this object store, many meta files are created. For example, index
files and reverse index files for packfiles, bitmap file, commit-graph
files, etc.

Docs for these files:

 * Documentation/technical/pack-format.txt
 * Documentation/technical/index-format.txt
 * Documentation/technical/chunk-format.txt
 * Documentation/technical/multi-pack-index.txt
 * Documentation/technical/bitmap-format.txt
 * Documentation/technical/commit-graph-format.txt
 * Documentation/technical/commit-graph.txt

--
Jiang Xin

Re: [GSoC] Hello Git

[ZheNing Hu]

Jiang Xin <worldhello.net@gmail.com> 于2021年5月19日周三 下午3:19写道：
>
> > I'm ZheNing Hu, I am very lucky to participate in the GSoC
> > git project this year. Many people in the git community already
> > have given me a lot of help in the past few months.
> > Junio, Christian, Peff, Eric, Denton..., it's great to get along with
> > you guys! Your review and guidance have greatly benefited
> > my growth.
>
> I admire you so much that you can join the open source community at a
> very young age. I will continue to offer my help both online and
> offline, in your favorite language (Chinese of course).
>

Participating in open source is a very fun thing! Thanks for your help haha!

> > In these days before GSoC, I learned some simple command
> > implementation and data structures of git, learned strict coding
> > standards and learned how to test, More importantly, I learned how
> > to communicate with these kind-hearted reviewers.
> >
> > Of course, there are still many difficulties for me:
> > * My poor English sometimes can’t express the meaning clearly.
>
> There are too many differences between English and Chinese, but don't
> worry. Remember, practice is perfect. You can also practice your
> English by reviewing Git l10n in Chinese, see: "po/README.md".
>

I have noticed that the language translation comparison table is in *.po,
which really helps me a lot sometimes.

> > ### The connection between Git and the file system
> >
> > I am currently studying operating system course, git as a
> > file system on top of the file system.
> > There are many differences and similarities with ordinary
> > file systems. I am very curious about this knowledge, do
> > not know if there are any relevant reading materials?
>
> Git objects are stored in ".git/objects" directory in two forms: loose
> object and packfile.  These two types of files are deflated in zlib
> format. In order to increase the speed of traversing these objects in
> this object store, many meta files are created. For example, index
> files and reverse index files for packfiles, bitmap file, commit-graph
> files, etc.
>
> Docs for these files:
>
>  * Documentation/technical/pack-format.txt
>  * Documentation/technical/index-format.txt
>  * Documentation/technical/chunk-format.txt
>  * Documentation/technical/multi-pack-index.txt
>  * Documentation/technical/bitmap-format.txt
>  * Documentation/technical/commit-graph-format.txt
>  * Documentation/technical/commit-graph.txt
>

This is what I want to know: how git objects are stored!
This can help me understand git itself at a deeper level.

> --
> Jiang Xin

Thanks!
--
ZheNing Hu