[GSoC] My Git Dev Blog – Week 8

[GSoC] My Git Dev Blog – Week 8

[Atharva Raykar]

Here's my weekly update:
https://atharvaraykar.me/gitnotes/week8

I am currently blocked by trying to pass a super-prefix parameter
to another command, when calling it from within C.

If you are interested in helping with that, this will take you to
right to it:
https://atharvaraykar.me/gitnotes/week8#path-pains

Have a nice day.

---
Atharva Raykar
ಅಥರ್ವ ರಾಯ್ಕರ್
अथर्व रायकर

Re: [GSoC] My Git Dev Blog – Week 8

[Kaartic Sivaraam]

Hi Atharva,

On 11/07/21 4:48 pm, Atharva Raykar wrote:
> Here's my weekly update:
> https://atharvaraykar.me/gitnotes/week8
> 
> I am currently blocked by trying to pass a super-prefix parameter
> to another command, when calling it from within C.
>

 From the blog:

> When queried by the get_super_prefix() function, the answer is (null).
> This boggles my mind to no end (see update). The implementation is basically
> the same getenv() call?

Good to see that you were able to identify the reason yourself.

> I am not sure how to tell Git that the environment variable has in fact
> been modified, and that it needs to be reinitialized. Maybe I am going
> about this whole thing wrong?

I get the same feeling too. I took a brief look at how the issue could be
fixed and it seems to me that you exploring to set super-prefix might not
lead us to the solution. Alternatively, you could explore how other sub-commands
handle recursing into submodules. To me it looks like they handle it by spawning
a sub-process is likely the easiest approach for achieving recursion. That would
solve the super-prefix problem as you have observed.

Unfortunately, there's not yet proper support for handling recursion of submodules
which calls for working with the data of multiple Git repositories in the same
Git process. There was an effort[1] few years ago to make working with
other Git repositories simpler without having to spawn a sub-process. The state
of the effort is unclear to me. As far as I know, it has been stalled. I hope
others could provide more details about it.

So, you could try the approach of spawning of sub-process for now. In case there's
a better approach than spawning sub-process others might be able to point during
review. In the meanwhile, I'll try to take a better look later and see if I could
find anything.

[1]: https://public-inbox.org/git/20180205235508.216277-1-sbeller@google.com/

-- 
Sivaraam

Re: [GSoC] My Git Dev Blog – Week 8

[Atharva Raykar]

On 13-Jul-2021, at 13:48, Kaartic Sivaraam <kaartic.sivaraam@gmail.com> wrote:
> 
> Hi Atharva,
> 
> On 11/07/21 4:48 pm, Atharva Raykar wrote:
>> Here's my weekly update:
>> https://atharvaraykar.me/gitnotes/week8
>> I am currently blocked by trying to pass a super-prefix parameter
>> to another command, when calling it from within C.
>> 
> 
> From the blog:
> 
>> When queried by the get_super_prefix() function, the answer is (null).
>> This boggles my mind to no end (see update). The implementation is basically
>> the same getenv() call?
> 
> Good to see that you were able to identify the reason yourself.
> 
>> I am not sure how to tell Git that the environment variable has in fact
>> been modified, and that it needs to be reinitialized. Maybe I am going
>> about this whole thing wrong?
> 
> I get the same feeling too. I took a brief look at how the issue could be
> fixed and it seems to me that you exploring to set super-prefix might not
> lead us to the solution. Alternatively, you could explore how other sub-commands
> handle recursing into submodules. To me it looks like they handle it by spawning
> a sub-process is likely the easiest approach for achieving recursion. That would
> solve the super-prefix problem as you have observed.

Yes, I was hoping I would not have to spawn a subprocess for recursing on
update, and it does look theoretically possible--it would require changing
a lot of of existing code to use functions taking a repository objects
rather than assuming 'the_repository'. But since that was outside the scope
of my project, I did it like the other implementations that spawned a new
process for the recursion of 'update'.

> Unfortunately, there's not yet proper support for handling recursion of submodules
> which calls for working with the data of multiple Git repositories in the same
> Git process. There was an effort[1] few years ago to make working with
> other Git repositories simpler without having to spawn a sub-process. The state
> of the effort is unclear to me. As far as I know, it has been stalled. I hope
> others could provide more details about it.
> 
> So, you could try the approach of spawning of sub-process for now. In case there's
> a better approach than spawning sub-process others might be able to point during
> review. In the meanwhile, I'll try to take a better look later and see if I could
> find anything.

I was seeing if it was possible to at least save another spawn for calling
init when '--init' is provided for an update. The current implementation
does not spawn a separate process for this, so I was hoping I don't add
more overhead in the conversion, but it's looking hard to avoid at the
moment.

> [1]: https://public-inbox.org/git/20180205235508.216277-1-sbeller@google.com/
> 
> -- 
> Sivaraam

Re: [GSoC] My Git Dev Blog – Week 8

[Christian Couder]

On Tue, Jul 13, 2021 at 10:46 AM Atharva Raykar <raykar.ath@gmail.com> wrote:
>
> On 13-Jul-2021, at 13:48, Kaartic Sivaraam <kaartic.sivaraam@gmail.com> wrote:

> > ... Alternatively, you could explore how other sub-commands
> > handle recursing into submodules. To me it looks like they handle it by spawning
> > a sub-process is likely the easiest approach for achieving recursion. That would
> > solve the super-prefix problem as you have observed.
>
> Yes, I was hoping I would not have to spawn a subprocess for recursing on
> update, and it does look theoretically possible--it would require changing
> a lot of of existing code to use functions taking a repository objects
> rather than assuming 'the_repository'. But since that was outside the scope
> of my project, I did it like the other implementations that spawned a new
> process for the recursion of 'update'.

Yeah, it's definitely outside the scope of your project. So it's ok to
spawn a new process.

Re: [GSoC] My Git Dev Blog – Week 8

[Kaartic Sivaraam]

Hi Atharva,

On 13/07/21 2:16 pm, Atharva Raykar wrote:
> 
> I was seeing if it was possible to at least save another spawn for calling
> init when '--init' is provided for an update. The current implementation
> does not spawn a separate process for this, so I was hoping I don't add
> more overhead in the conversion, but it's looking hard to avoid at the
> moment.
> 

I'm having some difficulties understanding this. I tried to take a look
at the existing code[2][3] in 'git-submodule.sh', I could only see that
if '--init' is passed to update, it calls the 'cmd_update' shell function
which in turn does invoke 'git submodule--helper init'. OTOH, if '--recursive'
is passed 'cmd_update' itself is called recursively called after setting
the 'prefix' and 'wt_prefix' shell variables.

So, I'm not sure how you mean to say that a sub-process was not spawned
when '--init' is passed. Could you clarify this a little?

>> [1]: https://public-inbox.org/git/20180205235508.216277-1-sbeller@google.com/
>>

[2]: https://github.com/git/git/blob/abb21c7263616f01c5e950861a29279ab21cb02f/git-submodule.sh#L530-L533

[3]: https://github.com/git/git/blob/abb21c7263616f01c5e950861a29279ab21cb02f/git-submodule.sh#L651-L673

-- 
Sivaraam

Re: [GSoC] My Git Dev Blog – Week 8

[Atharva Raykar]

On 18-Jul-2021, at 23:09, Kaartic Sivaraam <kaartic.sivaraam@gmail.com> wrote:
> 
> Hi Atharva,
> 
> On 13/07/21 2:16 pm, Atharva Raykar wrote:
>> I was seeing if it was possible to at least save another spawn for calling
>> init when '--init' is provided for an update. The current implementation
>> does not spawn a separate process for this, so I was hoping I don't add
>> more overhead in the conversion, but it's looking hard to avoid at the
>> moment.
> 
> I'm having some difficulties understanding this. I tried to take a look
> at the existing code[2][3] in 'git-submodule.sh', I could only see that
> if '--init' is passed to update, it calls the 'cmd_update' shell function
> which in turn does invoke 'git submodule--helper init'. OTOH, if '--recursive'
> is passed 'cmd_update' itself is called recursively called after setting
> the 'prefix' and 'wt_prefix' shell variables.
> 
> So, I'm not sure how you mean to say that a sub-process was not spawned
> when '--init' is passed. Could you clarify this a little?
> 
>>> [1]: https://public-inbox.org/git/20180205235508.216277-1-sbeller@google.com/
>>> 
> 
> [2]: https://github.com/git/git/blob/abb21c7263616f01c5e950861a29279ab21cb02f/git-submodule.sh#L530-L533
> 
> [3]: https://github.com/git/git/blob/abb21c7263616f01c5e950861a29279ab21cb02f/git-submodule.sh#L651-L673

My mistake. When I wrote that paragraph, I had considered the shell call
in my head to be "git submodule init" and not "git submodule--helper init".

In the former case, there would be an extra subprocess because
'submodule init' will in turn spawn 'submodule--helper init'. So in short,
there would have been no difference between the two versions had I gone
down that route.