git@vger.kernel.org list mirror (unofficial, one of many)
 help / color / Atom feed
* [GSoC] My project blog
@ 2019-07-03  4:34 Matheus Tavares Bernardino
  2019-07-14  7:38 ` Matheus Tavares Bernardino
                   ` (5 more replies)
  0 siblings, 6 replies; 15+ messages in thread
From: Matheus Tavares Bernardino @ 2019-07-03  4:34 UTC (permalink / raw)
  To: git
  Cc: Christian Couder,
	Оля
	Тележная,
	Thomas Gummerer, Elijah Newren, Rohit Ashiwal

Hi, everyone

I've been posting about the progress of my GSoC project at
https://matheustavares.gitlab.io/gsoc/ I should have sent the link
earlier but, please, let me know if you have any comments on my
previous posts.

You can read my latest update here:
https://matheustavares.gitlab.io/posts/week-7-race-conditions-on-delta-base-cache
I've been working to allow parallel decompression when reading objects
for git-grep.

Thanks,
Matheus

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [GSoC] My project blog
  2019-07-03  4:34 [GSoC] My project blog Matheus Tavares Bernardino
@ 2019-07-14  7:38 ` Matheus Tavares Bernardino
  2019-07-14 11:00   ` Christian Couder
  2019-07-23  1:13 ` Matheus Tavares Bernardino
                   ` (4 subsequent siblings)
  5 siblings, 1 reply; 15+ messages in thread
From: Matheus Tavares Bernardino @ 2019-07-14  7:38 UTC (permalink / raw)
  To: git
  Cc: Christian Couder,
	Оля
	Тележная,
	Thomas Gummerer, Elijah Newren, Rohit Ashiwal

Hi, everyone

I just posted a new update about my GSoC project here:
https://matheustavares.gitlab.io/posts/week-8-a-working-parallel-inflation
Please, feel free to leave any comments. I've finally solved the race
condition problem and now we have a working parallel inflation :) But
there's still some tasks to be done on it before I have a proper
patchset.

Thanks,
Matheus

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [GSoC] My project blog
  2019-07-14  7:38 ` Matheus Tavares Bernardino
@ 2019-07-14 11:00   ` Christian Couder
  2019-07-14 16:38     ` Matheus Tavares Bernardino
  0 siblings, 1 reply; 15+ messages in thread
From: Christian Couder @ 2019-07-14 11:00 UTC (permalink / raw)
  To: Matheus Tavares Bernardino
  Cc: git,
	Оля
	Тележная,
	Thomas Gummerer, Elijah Newren, Rohit Ashiwal

Hi Matheus,

On Sun, Jul 14, 2019 at 9:38 AM Matheus Tavares Bernardino
<matheus.bernardino@usp.br> wrote:
>
> I just posted a new update about my GSoC project here:
> https://matheustavares.gitlab.io/posts/week-8-a-working-parallel-inflation
> Please, feel free to leave any comments. I've finally solved the race
> condition problem and now we have a working parallel inflation :) But
> there's still some tasks to be done on it before I have a proper
> patchset.

Great that you solved the race condition!

I agree with the tasks you list in the next steps. I would even
suggest that, right after splitting the changes into logical patches,
you ask your mentors (Olga and me) to review it, and then send it as
an RFC patch series to the mailing list. When you do that though,
please list the other tasks and limitations of your patch series in
the cover letter.

Thanks,
Christian.

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [GSoC] My project blog
  2019-07-14 11:00   ` Christian Couder
@ 2019-07-14 16:38     ` Matheus Tavares Bernardino
  0 siblings, 0 replies; 15+ messages in thread
From: Matheus Tavares Bernardino @ 2019-07-14 16:38 UTC (permalink / raw)
  To: Christian Couder
  Cc: git,
	Оля
	Тележная,
	Thomas Gummerer, Elijah Newren, Rohit Ashiwal

On Sun, Jul 14, 2019 at 8:00 AM Christian Couder
<christian.couder@gmail.com> wrote:
>
> Hi Matheus,
>
> On Sun, Jul 14, 2019 at 9:38 AM Matheus Tavares Bernardino
> <matheus.bernardino@usp.br> wrote:
> >
> > I just posted a new update about my GSoC project here:
> > https://matheustavares.gitlab.io/posts/week-8-a-working-parallel-inflation
> > Please, feel free to leave any comments. I've finally solved the race
> > condition problem and now we have a working parallel inflation :) But
> > there's still some tasks to be done on it before I have a proper
> > patchset.
>
> Great that you solved the race condition!
>
> I agree with the tasks you list in the next steps. I would even
> suggest that, right after splitting the changes into logical patches,
> you ask your mentors (Olga and me) to review it, and then send it as
> an RFC patch series to the mailing list. When you do that though,
> please list the other tasks and limitations of your patch series in
> the cover letter.

OK, I will do that.

Thanks,
Matheus

> Thanks,
> Christian.

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [GSoC] My project blog
  2019-07-03  4:34 [GSoC] My project blog Matheus Tavares Bernardino
  2019-07-14  7:38 ` Matheus Tavares Bernardino
@ 2019-07-23  1:13 ` Matheus Tavares Bernardino
  2019-07-29 20:41 ` Matheus Tavares Bernardino
                   ` (3 subsequent siblings)
  5 siblings, 0 replies; 15+ messages in thread
From: Matheus Tavares Bernardino @ 2019-07-23  1:13 UTC (permalink / raw)
  To: git
  Cc: Christian Couder,
	Оля
	Тележная,
	Thomas Gummerer, Elijah Newren, Rohit Ashiwal

Hi, everyone

Unfortunately, not quite much to report this week :( I'm attending
DebConf until July 28th but I'm trying to conciliate that with the
work on my project. You can see my brief report of last week here:
https://matheustavares.gitlab.io/posts/week-9-an-rfc-on-parallel-inflation

As always, every comment is highly appreciated.

Thanks,
Matheus

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [GSoC] My project blog
  2019-07-03  4:34 [GSoC] My project blog Matheus Tavares Bernardino
  2019-07-14  7:38 ` Matheus Tavares Bernardino
  2019-07-23  1:13 ` Matheus Tavares Bernardino
@ 2019-07-29 20:41 ` Matheus Tavares Bernardino
  2019-08-06  2:54 ` Matheus Tavares Bernardino
                   ` (2 subsequent siblings)
  5 siblings, 0 replies; 15+ messages in thread
From: Matheus Tavares Bernardino @ 2019-07-29 20:41 UTC (permalink / raw)
  To: git
  Cc: Christian Couder,
	Оля
	Тележная,
	Thomas Gummerer, Elijah Newren, Rohit Ashiwal

Hi, everyone

I just posted an update about my project here:
https://matheustavares.gitlab.io/posts/week-10-a-bug-in-git-grep-submodules
This one is focused on a patch to correct a bug at git-grep
--recurse-submodules and some tests for the parallel inflation
patchset. Please, feel free to leave any comments.

Thanks,
Matheus

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [GSoC] My project blog
  2019-07-03  4:34 [GSoC] My project blog Matheus Tavares Bernardino
                   ` (2 preceding siblings ...)
  2019-07-29 20:41 ` Matheus Tavares Bernardino
@ 2019-08-06  2:54 ` Matheus Tavares Bernardino
  2019-08-06  7:16   ` Christian Couder
  2019-08-13  4:06 ` Matheus Tavares Bernardino
  2019-08-20  4:58 ` Matheus Tavares Bernardino
  5 siblings, 1 reply; 15+ messages in thread
From: Matheus Tavares Bernardino @ 2019-08-06  2:54 UTC (permalink / raw)
  To: git
  Cc: Christian Couder,
	Оля
	Тележная,
	Thomas Gummerer, Elijah Newren, Rohit Ashiwal,
	Nguyễn Thái Ngọc Duy

Hi,

Here's my report from last week:
https://matheustavares.gitlab.io/posts/week-11-wip-grep-protecting-textconv-and-submodules
I'm working to protect the operations I left behind on the first
version of the patchset[1]. And for that, I used a lot of the code Duy
provided[2] me as an example in the early days of this project. The
race conditions are now majorly gone, but the patches still need some
refactoring and there are still some problems to overcome.

Thanks,
Matheus

[1]: https://public-inbox.org/git/cover.1563570204.git.matheus.bernardino@usp.br/
[2]: https://gitlab.com/snippets/1834613#L3

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [GSoC] My project blog
  2019-08-06  2:54 ` Matheus Tavares Bernardino
@ 2019-08-06  7:16   ` Christian Couder
  2019-08-08 14:22     ` Matheus Tavares Bernardino
  0 siblings, 1 reply; 15+ messages in thread
From: Christian Couder @ 2019-08-06  7:16 UTC (permalink / raw)
  To: Matheus Tavares Bernardino
  Cc: git,
	Оля
	Тележная,
	Thomas Gummerer, Elijah Newren, Rohit Ashiwal,
	Nguyễn Thái Ngọc Duy

Hi Matheus,

On Tue, Aug 6, 2019 at 4:54 AM Matheus Tavares Bernardino
<matheus.bernardino@usp.br> wrote:
>
> Here's my report from last week:
> https://matheustavares.gitlab.io/posts/week-11-wip-grep-protecting-textconv-and-submodules

Thank you for another great report!

> I'm working to protect the operations I left behind on the first
> version of the patchset[1]. And for that, I used a lot of the code Duy
> provided[2] me as an example in the early days of this project. The
> race conditions are now majorly gone, but the patches still need some
> refactoring and there are still some problems to overcome.

This looks good to me! I think you are now way past the point where
you know much more about this than me, but anyway the following might
give you some ideas or perhaps in some other ways help you:

  - About "Try to remove the obj_read_lock", which is the last point
on your TODO list, it doesn't look necessary before you submit a patch
series, unless there is no performance improvement without it. You
could, as far as I can tell, do it in a later patch series.

  - I guess " --recurse-submodules" is not used very often with git
grep. For example I never use it on the Git repo even when I perhaps
should. So if it would simplify things to just disable the
improvements you made when " --recurse-submodules" is used, then you
might consider doing that. You could later send another patch series
that would enable your improvements when " --recurse-submodules" is
used.

Thanks,
Christian.

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [GSoC] My project blog
  2019-08-06  7:16   ` Christian Couder
@ 2019-08-08 14:22     ` Matheus Tavares Bernardino
  0 siblings, 0 replies; 15+ messages in thread
From: Matheus Tavares Bernardino @ 2019-08-08 14:22 UTC (permalink / raw)
  To: Christian Couder
  Cc: git,
	Оля
	Тележная,
	Thomas Gummerer, Elijah Newren, Rohit Ashiwal,
	Nguyễn Thái Ngọc Duy

On Tue, Aug 6, 2019 at 4:16 AM Christian Couder
<christian.couder@gmail.com> wrote:
>
> Hi Matheus,
>
> On Tue, Aug 6, 2019 at 4:54 AM Matheus Tavares Bernardino
> <matheus.bernardino@usp.br> wrote:
> >
> > Here's my report from last week:
> > https://matheustavares.gitlab.io/posts/week-11-wip-grep-protecting-textconv-and-submodules
>
> Thank you for another great report!
>
> > I'm working to protect the operations I left behind on the first
> > version of the patchset[1]. And for that, I used a lot of the code Duy
> > provided[2] me as an example in the early days of this project. The
> > race conditions are now majorly gone, but the patches still need some
> > refactoring and there are still some problems to overcome.
>
> This looks good to me! I think you are now way past the point where
> you know much more about this than me, but anyway the following might
> give you some ideas or perhaps in some other ways help you:
>
>   - About "Try to remove the obj_read_lock", which is the last point
> on your TODO list, it doesn't look necessary before you submit a patch
> series, unless there is no performance improvement without it. You
> could, as far as I can tell, do it in a later patch series.
>
>   - I guess " --recurse-submodules" is not used very often with git
> grep. For example I never use it on the Git repo even when I perhaps
> should. So if it would simplify things to just disable the
> improvements you made when " --recurse-submodules" is used, then you
> might consider doing that. You could later send another patch series
> that would enable your improvements when " --recurse-submodules" is
> used.

Thanks for the suggestions, Christian. So, I think I could finish and
send the patchset without the optimization for --recurse-submodules
and --textconv, while I don't finish support for them, right? And I
think 'obj_read_lock' may be removed in a future patch, as you said,
because we already have good speedup without removing it. Finally,
should I also add a "NOTES" section to Documentation/git-grep.txt
saying which options --threads is not compatible with?

Thanks,
Matheus

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [GSoC] My project blog
  2019-07-03  4:34 [GSoC] My project blog Matheus Tavares Bernardino
                   ` (3 preceding siblings ...)
  2019-08-06  2:54 ` Matheus Tavares Bernardino
@ 2019-08-13  4:06 ` Matheus Tavares Bernardino
  2019-08-13 10:51   ` Christian Couder
  2019-08-20  4:58 ` Matheus Tavares Bernardino
  5 siblings, 1 reply; 15+ messages in thread
From: Matheus Tavares Bernardino @ 2019-08-13  4:06 UTC (permalink / raw)
  To: git
  Cc: Christian Couder,
	Оля
	Тележная,
	Thomas Gummerer, Elijah Newren, Rohit Ashiwal

Hi, everyone

I just posted a new update on my project:
https://matheustavares.gitlab.io/posts/simplified-version-of-parallel-inflation

This week I sent a simplified version of the series I was working on
last week (which would still take some time to finish). And I'm
working to improve it with support to --textconv and
--recurse-submodules in the added optimizations. I've been also
working on a patch to make git-grep stop adding submodules to the
alternates list.

Thanks,
Matheus

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [GSoC] My project blog
  2019-08-13  4:06 ` Matheus Tavares Bernardino
@ 2019-08-13 10:51   ` Christian Couder
  0 siblings, 0 replies; 15+ messages in thread
From: Christian Couder @ 2019-08-13 10:51 UTC (permalink / raw)
  To: Matheus Tavares Bernardino
  Cc: git,
	Оля
	Тележная,
	Thomas Gummerer, Elijah Newren, Rohit Ashiwal

Hi Matheus,

On Tue, Aug 13, 2019 at 6:06 AM Matheus Tavares Bernardino
<matheus.bernardino@usp.br> wrote:
>
> I just posted a new update on my project:
> https://matheustavares.gitlab.io/posts/simplified-version-of-parallel-inflation

Great blog post as usual!

> This week I sent a simplified version of the series I was working on
> last week (which would still take some time to finish). And I'm
> working to improve it with support to --textconv and
> --recurse-submodules in the added optimizations. I've been also
> working on a patch to make git-grep stop adding submodules to the
> alternates list.

I don't know what refactoring you think is needed on your patch:

https://github.com/matheustavares/git/commit/05536a503b3315d36ad13276af7728adf674ed51

I think it's worth sending to the list with or without addressing the
small nits that I commented on.

Thanks,
Christian.

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [GSoC] My project blog
  2019-07-03  4:34 [GSoC] My project blog Matheus Tavares Bernardino
                   ` (4 preceding siblings ...)
  2019-08-13  4:06 ` Matheus Tavares Bernardino
@ 2019-08-20  4:58 ` Matheus Tavares Bernardino
  2019-08-20 11:27   ` Olga Telezhnaya
  5 siblings, 1 reply; 15+ messages in thread
From: Matheus Tavares Bernardino @ 2019-08-20  4:58 UTC (permalink / raw)
  To: git
  Cc: Christian Couder,
	Оля
	Тележная,
	Thomas Gummerer, Elijah Newren, Rohit Ashiwal

Hi, everyone

I just posted the penultimate report on my project:
https://matheustavares.gitlab.io/posts/going-for-a-too-big-step This
week I’ve been working on a v2 of threaded git-grep w/ parallel
inflation, to allow threads when grepping submodules. I also tried
some more optimizations along the way.

As always, any comments/suggestions will be highly appreciated.

Thanks,
Matheus

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [GSoC] My project blog
  2019-08-20  4:58 ` Matheus Tavares Bernardino
@ 2019-08-20 11:27   ` Olga Telezhnaya
  2019-08-20 12:16     ` Christian Couder
  0 siblings, 1 reply; 15+ messages in thread
From: Olga Telezhnaya @ 2019-08-20 11:27 UTC (permalink / raw)
  To: Matheus Tavares Bernardino
  Cc: git, Christian Couder, Thomas Gummerer, Elijah Newren, Rohit Ashiwal

вт, 20 авг. 2019 г. в 07:59, Matheus Tavares Bernardino
<matheus.bernardino@usp.br>:
>
> Hi, everyone
>
> I just posted the penultimate report on my project:
> https://matheustavares.gitlab.io/posts/going-for-a-too-big-step This
> week I’ve been working on a v2 of threaded git-grep w/ parallel
> inflation, to allow threads when grepping submodules. I also tried
> some more optimizations along the way.

Thank you for great blog post! You have done so many things, it is impressive.

I absolutely agree with your plans to structure the job and split it
to smaller pieces, it should definitely help. Moreover, if you want,
you can make something like a documentation and delegate some parts.

I hope you enjoyed this summer :) Thank you for your readiness to
continue contributing after GSoC. It is not mandatory, you do not have
to finish the project, but if you want - all the community will be so
glad to see you as the contributor.

Thank you so much,
Olga

>
> As always, any comments/suggestions will be highly appreciated.
>
> Thanks,
> Matheus

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [GSoC] My project blog
  2019-08-20 11:27   ` Olga Telezhnaya
@ 2019-08-20 12:16     ` Christian Couder
  2019-08-20 22:41       ` Matheus Tavares Bernardino
  0 siblings, 1 reply; 15+ messages in thread
From: Christian Couder @ 2019-08-20 12:16 UTC (permalink / raw)
  To: Matheus Tavares Bernardino
  Cc: git, Olga Telezhnaya, Thomas Gummerer, Elijah Newren, Rohit Ashiwal

Hi Matheus,

On Tue, Aug 20, 2019 at 1:28 PM Olga Telezhnaya
<olyatelezhnaya@gmail.com> wrote:
>
> вт, 20 авг. 2019 г. в 07:59, Matheus Tavares Bernardino
> <matheus.bernardino@usp.br>:
> >
> > I just posted the penultimate report on my project:
> > https://matheustavares.gitlab.io/posts/going-for-a-too-big-step This
> > week I’ve been working on a v2 of threaded git-grep w/ parallel
> > inflation, to allow threads when grepping submodules. I also tried
> > some more optimizations along the way.
>
> Thank you for great blog post! You have done so many things, it is impressive.

Yeah, great blog post again!

> I absolutely agree with your plans to structure the job and split it
> to smaller pieces, it should definitely help. Moreover, if you want,
> you can make something like a documentation and delegate some parts.

I also agree with the above. I think it would be really nice for
example if we could have something in Documentation/technical/
describing how multi-threading in `git grep` works and how it could be
improved. (You can of course copy paste from your blog.)

About static function-scope variables I wonder if we could have one
day some .h and .c files with only thread safe functions in them.

> I hope you enjoyed this summer :) Thank you for your readiness to
> continue contributing after GSoC. It is not mandatory, you do not have
> to finish the project, but if you want - all the community will be so
> glad to see you as the contributor.

Thank you for your work and for considering continuing after the GSoC,
Christian.

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [GSoC] My project blog
  2019-08-20 12:16     ` Christian Couder
@ 2019-08-20 22:41       ` Matheus Tavares Bernardino
  0 siblings, 0 replies; 15+ messages in thread
From: Matheus Tavares Bernardino @ 2019-08-20 22:41 UTC (permalink / raw)
  To: Christian Couder
  Cc: git, Olga Telezhnaya, Thomas Gummerer, Elijah Newren, Rohit Ashiwal

On Tue, Aug 20, 2019 at 9:16 AM Christian Couder
<christian.couder@gmail.com> wrote:
>
> Hi Matheus,
>
> On Tue, Aug 20, 2019 at 1:28 PM Olga Telezhnaya
> <olyatelezhnaya@gmail.com> wrote:
> >
> > вт, 20 авг. 2019 г. в 07:59, Matheus Tavares Bernardino
> > <matheus.bernardino@usp.br>:
> > >
> > > I just posted the penultimate report on my project:
> > > https://matheustavares.gitlab.io/posts/going-for-a-too-big-step This
> > > week I’ve been working on a v2 of threaded git-grep w/ parallel
> > > inflation, to allow threads when grepping submodules. I also tried
> > > some more optimizations along the way.
> >
> > Thank you for great blog post! You have done so many things, it is impressive.
>
> Yeah, great blog post again!

Thank you both for the always encouraging comments :)

> > I absolutely agree with your plans to structure the job and split it
> > to smaller pieces, it should definitely help. Moreover, if you want,
> > you can make something like a documentation and delegate some parts.
>
> I also agree with the above. I think it would be really nice for
> example if we could have something in Documentation/technical/
> describing how multi-threading in `git grep` works and how it could be
> improved. (You can of course copy paste from your blog.)

Hm, good idea! I would have to revisit the important points and better
formulate them, but that could hopefully help in future optimizations
to git-grep.

> About static function-scope variables I wonder if we could have one
> day some .h and .c files with only thread safe functions in them.

Yes, that would probably require quite some work but would definitely
be very helpful!

> > I hope you enjoyed this summer :) Thank you for your readiness to
> > continue contributing after GSoC. It is not mandatory, you do not have
> > to finish the project, but if you want - all the community will be so
> > glad to see you as the contributor.
>
> Thank you for your work and for considering continuing after the GSoC,

Thank you both, again. It's been a great summer of code! (although
it's winter here haha) I do hope to finish the project and keep
contributing with whatever I can :)

Best,
Matheus

^ permalink raw reply	[flat|nested] 15+ messages in thread

end of thread, back to index

Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-07-03  4:34 [GSoC] My project blog Matheus Tavares Bernardino
2019-07-14  7:38 ` Matheus Tavares Bernardino
2019-07-14 11:00   ` Christian Couder
2019-07-14 16:38     ` Matheus Tavares Bernardino
2019-07-23  1:13 ` Matheus Tavares Bernardino
2019-07-29 20:41 ` Matheus Tavares Bernardino
2019-08-06  2:54 ` Matheus Tavares Bernardino
2019-08-06  7:16   ` Christian Couder
2019-08-08 14:22     ` Matheus Tavares Bernardino
2019-08-13  4:06 ` Matheus Tavares Bernardino
2019-08-13 10:51   ` Christian Couder
2019-08-20  4:58 ` Matheus Tavares Bernardino
2019-08-20 11:27   ` Olga Telezhnaya
2019-08-20 12:16     ` Christian Couder
2019-08-20 22:41       ` Matheus Tavares Bernardino

git@vger.kernel.org list mirror (unofficial, one of many)

Archives are clonable:
	git clone --mirror https://public-inbox.org/git
	git clone --mirror http://ou63pmih66umazou.onion/git
	git clone --mirror http://czquwvybam4bgbro.onion/git
	git clone --mirror http://hjrcffqmbrq6wope.onion/git

Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.version-control.git
	nntp://ou63pmih66umazou.onion/inbox.comp.version-control.git
	nntp://czquwvybam4bgbro.onion/inbox.comp.version-control.git
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.version-control.git
	nntp://news.gmane.org/gmane.comp.version-control.git

 note: .onion URLs require Tor: https://www.torproject.org/

AGPL code for this site: git clone https://public-inbox.org/ public-inbox