[GSoC] Exploring Previous year Projects

[GSoC] Exploring Previous year Projects

[Shourya Shukla]

Hello,

I was looking at the previous year projects[1] and a project intrigued me, namely:
"Convert scripts to builtins".

Following from Christian's advice[2], I have decided to focus on my project proposal.
I noticed that various commands such as "git bisect", "git web--browse"(it particularly
interests me) are still in their "shell" form and will be needed to be converted into
their "C" form as per the project description.

I want to contribute in this project for GSoC '20, hence I want to know in which direction
should I proceed.

Thanks,
Shourya Shukla

[1]: https://git.github.io/SoC-2019-Ideas/
[2]: https://lore.kernel.org/git/CAP8UFD2Fo=2suQDLwzLM-Wg3ZzXpqHw-x0brPtPV0d4dRsgs9A@mail.gmail.com/

Re: [GSoC] Exploring Previous year Projects

[Jakub Narebski]

Shourya Shukla <shouryashukla.oo@gmail.com> writes:

> Hello,
>
> I was looking at the previous year projects[1] and a project intrigued me, namely:
> "Convert scripts to builtins".
>
> Following from Christian's advice[2], I have decided to focus on my project proposal.
> I noticed that various commands such as "git bisect", "git web--browse"(it particularly
> interests me) are still in their "shell" form and will be needed to be converted into
> their "C" form as per the project description.
[...]
> [1]: https://git.github.io/SoC-2019-Ideas/
> [2]: https://lore.kernel.org/git/CAP8UFD2Fo=2suQDLwzLM-Wg3ZzXpqHw-x0brPtPV0d4dRsgs9A@mail.gmail.com/

As far as I know, "git bisect" is currently being converted from shell
to C by Miriam Rubio for Outreachy project [3], so I am not sure if it
would be feasible as GSoC 2020 project.

I'm not sure if it would be possible and if it would make sense to
convert "git instaweb" and/or it's helper script "git web--browse" from
shell to C.

I think trying to convert either "git stash" or "git submodule" to C
would make more sense.

[3]: https://public-inbox.org/git/20200128144026.53128-1-mirucam@gmail.com/t/#u

Best,
-- 
Jakub Narębski

Converting scripted commands to built-ins, was Re: [GSoC] Exploring Previous year Projects

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 3586 bytes --]

Hi Kuba,

On Wed, 29 Jan 2020, Jakub Narebski wrote:

> Shourya Shukla <shouryashukla.oo@gmail.com> writes:
>
> > Hello,
> >
> > I was looking at the previous year projects[1] and a project intrigued me, namely:
> > "Convert scripts to builtins".
> >
> > Following from Christian's advice[2], I have decided to focus on my project proposal.
> > I noticed that various commands such as "git bisect", "git web--browse"(it particularly
> > interests me) are still in their "shell" form and will be needed to be converted into
> > their "C" form as per the project description.
> [...]
> > [1]: https://git.github.io/SoC-2019-Ideas/
> > [2]: https://lore.kernel.org/git/CAP8UFD2Fo=2suQDLwzLM-Wg3ZzXpqHw-x0brPtPV0d4dRsgs9A@mail.gmail.com/
>
> As far as I know, "git bisect" is currently being converted from shell
> to C by Miriam Rubio for Outreachy project [3], so I am not sure if it
> would be feasible as GSoC 2020 project.

Indeed. That one "is already taken".

> I'm not sure if it would be possible and if it would make sense to
> convert "git instaweb" and/or it's helper script "git web--browse" from
> shell to C.

I agree, both of those scripts seem not to benefit all that much from
being converted, while some people would still consider them to be
developed easier as shell scripts.

> I think trying to convert either "git stash" or "git submodule" to C
> would make more sense.

Oh, but `git stash` is already converted. The only two remainining shell
scripts for which I would consider a conversion to C to make sense are
`git submodule` and `git mergetool`.

Large parts of `git submodule` are already implemented in `git
submodule--helper`, so that's a head start (thanks Stephan Beller!).

The `git mergetool` command is a bit trickier, as it consists of three
scripts, really, with the `difftool--helper` depending on
`mergetool--lib`.

Realistically, I think that it would be possible for a GSoC student who is
already very familiar with the code base and with submodules to finish the
conversion of `git submodule` in one season.

The same is probably not true for `git mergetool`: it would require a
couple of seasons to convert, and a good chunk of the first month would be
taken by planning a conversion strategy.

As of the current `master`, the `git-*.sh` scripts are:

# In the process of being converted

git-bisect.sh

# Candidates for being converted

git-difftool--helper.sh
git-mergetool--lib.sh
git-mergetool.sh
git-submodule.sh

# Trivial conversions

git-merge-octopus.sh
git-merge-one-file.sh
git-merge-resolve.sh

# Already deprecated

git-filter-branch.sh
git-legacy-stash.sh
git-rebase--preserve-merges.sh

# Is this even needed anymore?

git-quiltimport.sh

# Probably better to leave them as shell scripts

git-instaweb.sh
git-request-pull.sh
git-web--browse.sh

# Not even Git commands

git-parse-remote.sh
git-sh-i18n.sh
git-sh-setup.sh

The situation of the Perl scripts to be converted is much nicer:

# Already in code review

git-add--interactive.perl

# Too complex/too dependent on the Perl packages

git-send-email.perl
git-svn.perl

# Support for legacy SCMs that are less and less used

git-archimport.perl
git-cvsexportcommit.perl
git-cvsimport.perl
git-cvsserver.perl

So: after `git add -i`, I think we're done with the conversion of the Perl
scripts. Took long enough ;-)

Ciao,
Dscho

>
> [3]: https://public-inbox.org/git/20200128144026.53128-1-mirucam@gmail.com/t/#u
>
> Best,
> --
> Jakub Narębski
>

Re: Converting scripted commands to built-ins, was Re: [GSoC] Exploring Previous year Projects

[Derrick Stolee]

On 1/30/2020 6:10 AM, Johannes Schindelin wrote:
> Large parts of `git submodule` are already implemented in `git
> submodule--helper`, so that's a head start (thanks Stephan Beller!).

As I recall, a bit part of what prevented more going into submodule--helper
was isolating the_repository and allowing multiple repositories in-memory
at once. That effort has stalled since Stephan has moved to other things.
(I will also admit that perhaps I am wrong: is it complete, rather than
stalled?)

I would be happy to see more progress in this area, and it could be a good
way to get familiar with the codebase and submitting patches.

> Realistically, I think that it would be possible for a GSoC student who is
> already very familiar with the code base and with submodules to finish the
> conversion of `git submodule` in one season.

Even assuming the most generous definition of "very familiar" I'm not sure
this is the case. But I also don't meet the bar of "very familiar" when it
comes to the submodule code.
 
So, I'm sending this message just to say "DRAGONS BE HERE" and recommend
taking on a smaller project.

Thanks,
-Stolee

Re: Converting scripted commands to built-ins, was Re: [GSoC] Exploring Previous year Projects

[Johannes Schindelin]

Hi Stolee,

On Thu, 30 Jan 2020, Derrick Stolee wrote:

> On 1/30/2020 6:10 AM, Johannes Schindelin wrote:
> > Large parts of `git submodule` are already implemented in `git
> > submodule--helper`, so that's a head start (thanks Stephan Beller!).
>
> As I recall, a bit part of what prevented more going into submodule--helper
> was isolating the_repository and allowing multiple repositories in-memory
> at once. That effort has stalled since Stephan has moved to other things.
> (I will also admit that perhaps I am wrong: is it complete, rather than
> stalled?)

There was definitely a push toward using `the_repository` in more places,
essentially making the entire code base a bit more object-oriented. My
impression is that this effort has largely stalled since Stephan and Duy
moved to other things.

However, this push is not exactly necessary to make `git submodule` a
built-in, much like `git push` can be a built-in even if it has to work on
two different repositories when run locally: it simply spawns a `git
receive-pack` in the repository to push to, rather than handling two
repositories in the same memory space.

The same is true for `git submodule`, of course: it can very easily spawn
processes for all the submodules. And that's what the `submodule--helper`
already does ;-)

> I would be happy to see more progress in this area, and it could be a
> good way to get familiar with the codebase and submitting patches.

I am of two minds here. On the one hand, I am eager to get rid of issues
such as https://github.com/git-for-windows/git/issues/1661, i.e. issues
that are solely due to Git requiring a Unix shell interpreter to run some
subcommands.

On the other hand... it is well known, I believe, what I think of
the design of Git's submodules feature. Putting an effort into making them
easier to use will not change this design.

In my mind, it would make more sense to focus on making partial clone and
sparse checkout a thing, and then combine it with a built-in version of
`git subtree` to include dependencies' commit history. Yes, I do suspect
that git-subtrac is going the wrong direction from git-subtree.

> > Realistically, I think that it would be possible for a GSoC student
> > who is already very familiar with the code base and with submodules to
> > finish the conversion of `git submodule` in one season.
>
> Even assuming the most generous definition of "very familiar" I'm not sure
> this is the case. But I also don't meet the bar of "very familiar" when it
> comes to the submodule code.

Hmm. I guess you're right, and it actually matches my earlier assessment:
https://lore.kernel.org/git/nycvar.QRO.7.76.6.1904031957480.41@tvgsbejvaqbjf.bet/

While it would be more satisfying for a prolific student to "complete" the
conversion in 3 months, it does not _have_ to be rushed that way. There
are tons of things to learn along the way, which is also part of GSoC's
mission.

The good news is that it is not an all-or-nothing project. Just like the
conversion of `git rebase` to C, which took several years, and the effort
of several contributors (implementing merge-recursive in C, then the
sequencer, then letting most of `rebase -i` be performed by the sequencer,
then the built-in `git am`, then the two GSoC projects to convert `git
rebase` and `git rebase -i`, extracting and deprecating `rebase -p` and of
course implementing the replacement for it: `rebase -r`), `git submodule`
_can_ and should be converted incrementally.

> So, I'm sending this message just to say "DRAGONS BE HERE" and recommend
> taking on a smaller project.

I can get behind that pretty good advice ;-)

Thanks for weighing in,
Dscho

Re: Converting scripted commands to built-ins, was Re: [GSoC] Exploring Previous year Projects

[Johannes Schindelin]

Hi again,

On Fri, 31 Jan 2020, Johannes Schindelin wrote:

> On Thu, 30 Jan 2020, Derrick Stolee wrote:
>
> > On 1/30/2020 6:10 AM, Johannes Schindelin wrote:
> > > Large parts of `git submodule` are already implemented in `git
> > > submodule--helper`, so that's a head start (thanks Stephan Beller!).
> > >
> > > [...]
> > >
> > > Realistically, I think that it would be possible for a GSoC student
> > > who is already very familiar with the code base and with submodules to
> > > finish the conversion of `git submodule` in one season.
> >
> > Even assuming the most generous definition of "very familiar" I'm not sure
> > this is the case. But I also don't meet the bar of "very familiar" when it
> > comes to the submodule code.
>
> Hmm. I guess you're right, and it actually matches my earlier assessment:
> https://lore.kernel.org/git/nycvar.QRO.7.76.6.1904031957480.41@tvgsbejvaqbjf.bet/

I vaguely remembered that I had described this project in a lot more
detail in a Git for Windows ticket, and I just spent a couple of minutes
to coagulate it into a more concrete proposal here:
https://github.com/gitgitgadget/git/issues/541

Ciao,
Dscho