[OUTREACHY][DRAFT Proposal] Unify ref-filter formats with other --pretty formats

[OUTREACHY][DRAFT Proposal] Unify ref-filter formats with other --pretty formats

[Debra Obondo]

Hello Team,
This is a draft proposal for the Outreachy Internship, could I kindly
have your reviews on it.
https://docs.google.com/document/d/11HX9dkEH--ZfXBTRWJ8rhGnlf3Bj_n_4U2Z-qgbDYVM/edit?usp=sharing

Below is its basic structure:

UNIFY ref-filter FORMATS WITH OTHER --pretty FORMATS

Personal Information

Name: Debra Obondo

Email: debraobondo@gmail.com

University: Jomo Kenyatta University of Agriculture and Technology

Telephone Number: +254743635117

Course: B.Sc Electronic and Computer Engineering

Github: https://github.com/fobiasic07

LinkedIn: https://www.linkedin.com/in/debra-obondo

Timezone: Nairobi (UTC+3)

Location: Nairobi, Kenya


Abstract

Git has an old problem of duplicated implementations of some logic
such as the four different implementations for formatting command
outputs for different commands. Ref-filter {.c.h} and pretty-lib
{.c.h} formatting logic have been no different and require(/d)
unification, which will help to simplify adding new functionality as
Git continues to develop.


This project was initially proposed by Hariom Verma following his FOSS
contributions to Git and Olga<olyatelezhnaya@gmail.com>’s work on
‘Unifying Git’s format languages’.


In his final report, he had completed almost all pretty format
implementation with a few challenges :

30% Log related failing tests on branch pretty-lib-2.0.2

Inability of pretty-lib {.c.h} to handle unknown formatting options.

Mailmap logic in ref-filter requires modification due to the new email
options which do not allow rebasing and consequently affects the MBOX
commit format.

MBOX commit format.

Completing the work and Rebasing a few other pretty-lib branches with
features such as fix-graph3 using Junio's way to pretty-lib-2.0.2


He also brought attention to :

Olga’s remaining work https://github.com/telezhnaya/git/commits/format


The purpose of this Internship Project is to solve the challenges,
submit well-functioning patches and increase functionality and
simplicity in pretty-lib {.c.h}.

Deliverables

The purpose of this project having been to improve and easen the
functionality of pretty, I aim to deliver in the following:

Working on failing formats to pass tests and maintain desired functionality.

Easen error identification when handling unknown formats.

Update MBOX commit format so that it still holds initial settings with
additional functionality.

Modifying mailmap logic to allow additional application.

Finishing work in other pretty-lib branches and merging with original
once tests are passed successfully.

Update documentation for ease of understanding

Final report on the improvements made and their importance.


Work that may be done if the above is completed before the end of the
project timeline:

Understanding Olga’s work in depth.

Working on the formats in ref-filter that were being adopted from cat-file.

Write and run passing tests for the formats when using ref-filter.

Working on expanded and added cat-file formats.

Update documentation on both cat-file and ref-filter work related to
Olga’s work.


In the case of the outreachy timeline having been passed, I plan to
work on this during the two months left in my long after-semester
break to further my knowledge on git and remain an active member of
the git community.


Project Implementation

The project will be done using Git, Bash Scripting and C (and likely Google;-)


Since I am about to finish my current semester, my exams (5th - 16th
December)  are likely to clash with the start of the internship, but
not to worry, since after that, I will be able to give my full-time
concentration to this project, about 45-50 Hours or more a week, since
I will be having an academic year break between then and 8th May 2023
before my final academic year begins.. Below is my programming plan.


Programming Plan

I have divided my programming plan into 5 parts, which would be as follows:

PART 1(Pre- Program);

This is before the official start of the program, between now and 5th Dec :

I plan to familiarize myself better with the community, submit a few
patches using 'git send-email', as it enables ease of communication on
various contributions and projects as compared to gitgitgadget.
Learning and understanding the logic and format functionality of both
the pretty and ref-filter commands is a priority to easen working on
the project later on. If accepted, plan for the required weekly
mentor-mentee meeting time(s). I also plan to understand the function
die() by going through this project: Issue #635

PART 2 (Phase 1); 5th Dec -2nd Jan:

This is during the first third of the program:

Working with mentors to identify the log-related failing test formats.

 Reworking the failing formats to pass the tests, while still
functioning in the required unified logic.

Share the first two blogs via my social media handles.

 Update Documentation.

Start on 'handling unknown formatting options'

 Feedback #1 submission

PART 3 (Phase 2); 3rd Jan - 1st Feb:

During the second third of the program:

 Working on the die() function, to figure out the best way to handle
unknown formatting options by pretty, be it creating a new 'die'/end
process function with more options,, mentor and community discussion
will help.

 Feedback #2 submission

 Modifying mailmap logic in ref-filter to allow rebasing, in turn,
working on the MBOX commit format.

Share my learning progress and challenges in other two blogs.

 Update Documentation.

 Feedback #3 submission.

PART 4 (Phase 3): 2nd Feb - 1st March:

During the final third of the program:

 Working on the other pretty lib branches (yet to go through them) and
rebasing/ merging the features to pretty-lib 2.0.2.

 Sharing my experience with Git as an FOSS community and giving my
final report through blogs.

 Fix any bugs that may be left and 'polish' the work on --pretty and ref-filter.

 Update Documentation.

Feedback #4 submission.

Part 5(Post-program):

After gaining many skills from working on the above, I believe I will
have the capacity to take on Olga's work during the remaining 2 month
break that I will be left with. In the case of similar projects during
GSoC 23';-) or Outreachy May to August, I will also be willing to
mentor, since teaching helps us learn more.

Perceived Obstacles and Challenges

Test Fails

This is always a possible occurrence in programming, therefore,
together with my mentors and the git community, solutions are highly
likely to emerge.

Maintaining compatibility for both newer and older systems

This is majorly in the mbox commit format and mailmap logic issue.
Further research and various opinions will lead to us finding the best
way forward.

Overflow of work past the internship period

This is likely to occur with rebasing of the other pretty-lib branches
and Olga’s work and can easily be dealt with by me giving more time in
the case of not reaching my timeline personal set deadlines.

About Me

I am a fourth year B.Sc student taking Electronics and Computer
Engineering at Jomo Kenyatta University of Agriculture and Technology,
Kenya. I have also been a member of the ALX-Africa-2022 Software
Programme.


I recently decided to venture into software as my career path. This
was during my attachment at Gearbox Academy , where I gained skills in
Web Development, Microcontroller processing and 3D Design.


I have experience with C from my coursework and ALX practice and level
3 skills and knowledge in BASH scripting and Git, which I would like
to further improve on. I believe contributing to this open source
project will help me to further grow my skills and widen my knowledge
on FOSS as a whole.


Equipped with this chance, I shall also use the skills learnt to help
future developers on similar projects and to continue contributing to
git and other open-source projects.


Requirements


This is the micro project required while submitting the proposal for
the Outreachy 2022 program:
https://lore.kernel.org/git/pull.1372.git.git.1667150441883.gitgitgadget@gmail.com


Description:

Older test scripts have used the command 'test -' to verify the
presence or absence of features such as files, directories and
symbolic links. This however requires slightly complicated uses, such
as 'test ! -f '. The helper functions 'test_path_is_' located in
t/test-lib-functions.sh have taken into account all scenarios of the
'test -*' to reduce errors. This was a microproject to replace them
with the helper functions.


Test script to verify the presence/absence of files, paths,
directories, symboliclinks and many other features in mv command are
using the command format


'test -(e|f|s|h|...)’


Replace them with helper functions of format


'test_path_is_*'


At the time of submission, this patch has been queued into ‘seen’
preparation for merging into next and eventually master: t7001-mv.sh
commit

Motivation

Git members are welcoming, I feel it will be a good learning community
for me to grow as an Open source contributor and software programmer,
which is one of my personal targets. During my contribution, I
received understandable feedback in a short period of time and was
able to make ‘polished’ and ‘out-of-the-blue’ patches as required.
(It’s part of the conversation thread in my microproject) This brought
to my awareness that git is an easily approachable community for any
programming challenges faced.

References

Git SoC 2022 Ideas : https://git.github.io/SoC-2022-Ideas/

Git Soc 2022 Applicant Microprojects :
https://git.github.io/SoC-2022-Microprojects/

The Final Report by Hariom Verma:
https://harry-hov.github.io/blogs/posts/the-final-report

Looking forward to your responses.

Kind regards,
Debra obondo.

Re: [OUTREACHY][DRAFT Proposal] Unify ref-filter formats with other --pretty formats

[Christian Couder]

Hi,

On Tue, Nov 8, 2022 at 11:55 AM Debra Obondo <debraobondo@gmail.com> wrote:
>
> Hello Team,
> This is a draft proposal for the Outreachy Internship, could I kindly
> have your reviews on it.
> https://docs.google.com/document/d/11HX9dkEH--ZfXBTRWJ8rhGnlf3Bj_n_4U2Z-qgbDYVM/edit?usp=sharing

I am going to review it even if it's a bit late to be taken into
account. (I think the Contribution Period ended on Nov 4, and today,
Nov 9, is the last day for us to select interns.)

Thank you anyway for being interested in this project and for sending
this proposal!

> Below is its basic structure:
>
> UNIFY ref-filter FORMATS WITH OTHER --pretty FORMATS

[...]

> Abstract
>
> Git has an old problem of duplicated implementations of some logic
> such as the four different implementations for formatting command
> outputs for different commands.

I am not sure if there are still four different implementations. As
far as I remember, we have been trying to unify:

  - `git for-each-ref`, `git branch` and `git tag` formats into the
ref-filter formats:

done by Karthik Nayak (GSoC 2015)

  -  `git cat-file` formats and the ref-filter formats:

started by Olga Telezhnaya (Outreachy 20217-2018),
continued by ZheNing Hu (GSoC 2021),
but still not finished due to tricky performance issues

  - ref-filter formats and pretty formats:

started by Hariom Verma (GSoC 2020)
continued a bit by Jaydeep Das (GSoC 2022)
but still not finished (although there doesn't seem to be hard issues),
so proposed for Outreachy 2022-2023

> Ref-filter {.c.h} and pretty-lib
> {.c.h} formatting logic

I am not sure it's worth talking about filenames as early in this
proposal. Also, on the current master branch, code for the pretty
formats seems to be in pretty.{c,h} not pretty-lib.{c,h}.

> have been no different and require(/d)
> unification, which will help to simplify adding new functionality as
> Git continues to develop.
>
> This project was initially proposed by Hariom Verma following his FOSS
> contributions to Git and Olga<olyatelezhnaya@gmail.com>’s work on
> ‘Unifying Git’s format languages’.

See above for more details. (Yeah, Olga worked on a project named
‘Unifying Git’s format languages’, but it was actually about using
ref-filter formats in `git cat-file`.)

> In his final report, he had completed almost all pretty format
> implementation with a few challenges :
>
> 30% Log related failing tests on branch pretty-lib-2.0.2
>
> Inability of pretty-lib {.c.h} to handle unknown formatting options.

Yeah, Hariom wanted to create pretty-lib.{c,h} files, but I think this
was not necessary so it wasn't done.

> Mailmap logic in ref-filter requires modification due to the new email
> options which do not allow rebasing and consequently affects the MBOX
> commit format.
>
> MBOX commit format.
>
> Completing the work and Rebasing a few other pretty-lib branches with
> features such as fix-graph3 using Junio's way to pretty-lib-2.0.2

I think it would have been better to summarize things a bit better,
and to avoid talking about low level details that are sometimes not
relevant anymore, like pretty-lib.

> He also brought attention to :
>
> Olga’s remaining work https://github.com/telezhnaya/git/commits/format
>
> The purpose of this Internship Project is to solve the challenges,
> submit well-functioning patches and increase functionality and
> simplicity in pretty-lib {.c.h}.

Here also I am not sure it's a good idea to talk about pretty-lib.

> Deliverables
>
> The purpose of this project having been to improve and easen the
> functionality of pretty, I aim to deliver in the following:
>
> Working on failing formats to pass tests and maintain desired functionality.
>
> Easen error identification when handling unknown formats.
>
> Update MBOX commit format so that it still holds initial settings with
> additional functionality.
>
> Modifying mailmap logic to allow additional application.
>
> Finishing work in other pretty-lib branches and merging with original
> once tests are passed successfully.
>
> Update documentation for ease of understanding
>
> Final report on the improvements made and their importance.

I think it would be nice if the different pretty formats for which
some work is needed could be listed along with the work needed.

> Work that may be done if the above is completed before the end of the
> project timeline:
>
> Understanding Olga’s work in depth.
>
> Working on the formats in ref-filter that were being adopted from cat-file.

This project is not about using ref-filter formats in `git cat-file`.
Yeah, it would be very nice if you could do it, but it's very unlikely
that you could do it on top of unifying the ref-filter and pretty
formats. (This has been discussed recently on the mailing list between
ZheNing Hu and me.)

[...]

> In the case of the outreachy timeline having been passed, I plan to
> work on this during the two months left in my long after-semester
> break to further my knowledge on git and remain an active member of
> the git community.

Nice!

[...]

> Below is my programming plan.
>
> Programming Plan
>
> I have divided my programming plan into 5 parts, which would be as follows:

I have basically the same comments for this section as for the previous section.

[...]

> This is the micro project required while submitting the proposal for
> the Outreachy 2022 program:
> https://lore.kernel.org/git/pull.1372.git.git.1667150441883.gitgitgadget@gmail.com

Thanks for working on this micro-project!

> Motivation
>
> Git members are welcoming, I feel it will be a good learning community
> for me to grow as an Open source contributor and software programmer,
> which is one of my personal targets. During my contribution, I
> received understandable feedback in a short period of time and was
> able to make ‘polished’ and ‘out-of-the-blue’ patches as required.
> (It’s part of the conversation thread in my microproject) This brought
> to my awareness that git is an easily approachable community for any
> programming challenges faced.

Thanks for the kind words!

Christian.

Re: [OUTREACHY][DRAFT Proposal] Unify ref-filter formats with other --pretty formats

[Debra Obondo]

Hello Christian

On Wed, Nov 9, 2022 at 2:49 PM Christian Couder
<christian.couder@gmail.com> wrote:
>
> Hi,
>
> On Tue, Nov 8, 2022 at 11:55 AM Debra Obondo <debraobondo@gmail.com> wrote:
> >
> > Hello Team,
> > This is a draft proposal for the Outreachy Internship, could I kindly
> > have your reviews on it.
> > https://docs.google.com/document/d/11HX9dkEH--ZfXBTRWJ8rhGnlf3Bj_n_4U2Z-qgbDYVM/edit?usp=sharing
>
> I am going to review it even if it's a bit late to be taken into
> account. (I think the Contribution Period ended on Nov 4, and today,
> Nov 9, is the last day for us to select interns.)

Thank you for taking your time to review my proposal despite its
lateness, I sincerely appreciate it. I've been at the period in the
semester where balance gets a little bit 'tricky', so, I was putting
in my last minute efforts. I had, however, submitted some of the
'required' details in the Outreachy portal before the deadline .
>
> Thank you anyway for being interested in this project and for sending
> this proposal!
>
> > Below is its basic structure:
> >
> > UNIFY ref-filter FORMATS WITH OTHER --pretty FORMATS
>
> [...]
>
> > Abstract
> >
> > Git has an old problem of duplicated implementations of some logic
> > such as the four different implementations for formatting command
> > outputs for different commands.
>
> I am not sure if there are still four different implementations. As
> far as I remember, we have been trying to unify:
>
>   - `git for-each-ref`, `git branch` and `git tag` formats into the
> ref-filter formats:
>
> done by Karthik Nayak (GSoC 2015)
>
>   -  `git cat-file` formats and the ref-filter formats:
>
> started by Olga Telezhnaya (Outreachy 20217-2018),
> continued by ZheNing Hu (GSoC 2021),
> but still not finished due to tricky performance issues
>
>   - ref-filter formats and pretty formats:
>
> started by Hariom Verma (GSoC 2020)
> continued a bit by Jaydeep Das (GSoC 2022)
> but still not finished (although there doesn't seem to be hard issues),
> so proposed for Outreachy 2022-2023
>
> > Ref-filter {.c.h} and pretty-lib
> > {.c.h} formatting logic
>
> I am not sure it's worth talking about filenames as early in this
> proposal. Also, on the current master branch, code for the pretty
> formats seems to be in pretty.{c,h} not pretty-lib.{c,h}.
>
I understand, I am still doing further research on this project, to
understand it better. I was also following Hariom's commits and
branches, which could have caused the confusion of me naming it
pretty-lib(as he has) instead of pretty (as is in git). I am yet to
begin looking through Jaydeep's work as well.

> > have been no different and require(/d)
> > unification, which will help to simplify adding new functionality as
> > Git continues to develop.
> >
> > This project was initially proposed by Hariom Verma following his FOSS
> > contributions to Git and Olga<olyatelezhnaya@gmail.com>’s work on
> > ‘Unifying Git’s format languages’.
>
> See above for more details. (Yeah, Olga worked on a project named
> ‘Unifying Git’s format languages’, but it was actually about using
> ref-filter formats in `git cat-file`.)
>
> > In his final report, he had completed almost all pretty format
> > implementation with a few challenges :
> >
> > 30% Log related failing tests on branch pretty-lib-2.0.2
> >
> > Inability of pretty-lib {.c.h} to handle unknown formatting options.
>
> Yeah, Hariom wanted to create pretty-lib.{c,h} files, but I think this
> was not necessary so it wasn't done.
>
> > Mailmap logic in ref-filter requires modification due to the new email
> > options which do not allow rebasing and consequently affects the MBOX
> > commit format.
> >
> > MBOX commit format.
> >
> > Completing the work and Rebasing a few other pretty-lib branches with
> > features such as fix-graph3 using Junio's way to pretty-lib-2.0.2
>
> I think it would have been better to summarize things a bit better,
> and to avoid talking about low level details that are sometimes not
> relevant anymore, like pretty-lib.
>
> > He also brought attention to :
> >
> > Olga’s remaining work https://github.com/telezhnaya/git/commits/format
> >
> > The purpose of this Internship Project is to solve the challenges,
> > submit well-functioning patches and increase functionality and
> > simplicity in pretty-lib {.c.h}.
>
> Here also I am not sure it's a good idea to talk about pretty-lib.
>
> > Deliverables
> >
> > The purpose of this project having been to improve and easen the
> > functionality of pretty, I aim to deliver in the following:
> >
> > Working on failing formats to pass tests and maintain desired functionality.
> >
> > Easen error identification when handling unknown formats.
> >
> > Update MBOX commit format so that it still holds initial settings with
> > additional functionality.
> >
> > Modifying mailmap logic to allow additional application.
> >
> > Finishing work in other pretty-lib branches and merging with original
> > once tests are passed successfully.
> >
> > Update documentation for ease of understanding
> >
> > Final report on the improvements made and their importance.
>
> I think it would be nice if the different pretty formats for which
> some work is needed could be listed along with the work needed.
>
I will continue to edit my proposal as I do more research, despite the
passed deadline (today), because it will help me, if not for this
particular project, maybe in future, I will be able to apply the
skills, knowledge, understanding and thought process gained from this
one.

> > Work that may be done if the above is completed before the end of the
> > project timeline:
> >
> > Understanding Olga’s work in depth.
> >
> > Working on the formats in ref-filter that were being adopted from cat-file.
>
> This project is not about using ref-filter formats in `git cat-file`.
> Yeah, it would be very nice if you could do it, but it's very unlikely
> that you could do it on top of unifying the ref-filter and pretty
> formats. (This has been discussed recently on the mailing list between
> ZheNing Hu and me.)
>
I will look deeper into it, since well, I will still be having the 4
months holiday's time.
> [...]
>
> > In the case of the outreachy timeline having been passed, I plan to
> > work on this during the two months left in my long after-semester
> > break to further my knowledge on git and remain an active member of
> > the git community.
>
> Nice!
>
> [...]
>
> > Below is my programming plan.
> >
> > Programming Plan
> >
> > I have divided my programming plan into 5 parts, which would be as follows:
>
> I have basically the same comments for this section as for the previous section.
>
> [...]
>
> > This is the micro project required while submitting the proposal for
> > the Outreachy 2022 program:
> > https://lore.kernel.org/git/pull.1372.git.git.1667150441883.gitgitgadget@gmail.com
>
> Thanks for working on this micro-project!
>
> > Motivation
> >
> > Git members are welcoming, I feel it will be a good learning community
> > for me to grow as an Open source contributor and software programmer,
> > which is one of my personal targets. During my contribution, I
> > received understandable feedback in a short period of time and was
> > able to make ‘polished’ and ‘out-of-the-blue’ patches as required.
> > (It’s part of the conversation thread in my microproject) This brought
> > to my awareness that git is an easily approachable community for any
> > programming challenges faced.
>
> Thanks for the kind words!
>
> Christian.

Thank you for the feedback. I would still love to be considered in the
case that Outreachy creates space for companies to accept more
interns. I will also send a V2 for the proposal soon.

Thanks,
Debra