Re: [PATCH v3 1/3] factor out refresh_and_write_cache function

[Johannes Schindelin <Johannes.Schindelin@gmx.de>]

Hi Thomas,

On Fri, 6 Sep 2019, Thomas Gummerer wrote:

> On 09/05, Junio C Hamano wrote:
> > Thomas Gummerer <t.gummerer@gmail.com> writes:
> >
> > > Getting the lock for the index, refreshing it and then writing it is a
> > > pattern that happens more than once throughout the codebase, and isn't
> > > trivial to get right.  Factor out the refresh_and_write_cache function
> > > from builtin/am.c to read-cache.c, so it can be re-used in other
> > > places in a subsequent commit.
> > >
> > > Note that we return different error codes for failing to refresh the
> > > cache, and failing to write the index.  The current caller only cares
> > > about failing to write the index.  However for other callers we're
> > > going to convert in subsequent patches we will need this distinction.
> > >
> > > Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
> > > ---
> > >  builtin/am.c | 16 ++--------------
> > >  cache.h      | 16 ++++++++++++++++
> > >  read-cache.c | 19 +++++++++++++++++++
> > >  3 files changed, 37 insertions(+), 14 deletions(-)
> >
> > I think this goes in the right direction, but obviously conflicts
> > with what Dscho wants to do in the builtin-add-i series, and needs
> > to be reconciled by working better together.
>
> Oops, I didn't realize there was another series in flight that also
> introduces 'repo_refresh_and_write_index'.  Probably should have done
> a test merge of this with pu.

Yep, our patches clash. I would not mind placing my patch series on top
of yours, provided that you can make a few changes that I need ;-)

> > For now, I'll eject builtin-add-i and queue this for a few days to
> > give it a bit more exposure, but after that requeue builtin-add-i
> > and discard these three patches.  By that time, hopefully you two
> > would have a rerolled version of this one and builtin-add-i that
> > agree what kind of refresh-and-write-index behaviour they both want.
> >
> > The differences I see that need reconciling are:
>
> Thanks for writing these down.
>
> >  - builtin-add-i seems to allow 'gentle' and allow returning an
> >    error when we cannot open the index for writing by passing false
> >    to 'gentle'; this feature is not used yet, though.
>
> Right, and if gentle is set to false, it avoids writing the index,
> which seems fine from my perspective.

This also suggests that it would make sense to avoid
`LOCK_DIE_ON_ERROR`, _in particular_ because this is supposed to be a
library function, not just a helper function for a one-shot built-in
(don't you like how this idea "it is okay to use exit() to clean up
after us, we don't care" comes back to bite us?).

> >  - This version allows to pass pathspec, seen and header_msg, while
> >    the one in builtin-add-i cannot limit the part of the index
> >    getting refreshed with pathspec.  It wouldn't be a brain surgery
> >    to use this version and adjust the caller (there only is one) in
> >    the builtin-add-i topic.
>
> 'pathspec', 'seen' and 'header_msg' are not used in my version either,
> I just implemented it for completeness and compatibility.  So I'd be
> fine to do without them.

Oh, why not keep them? I'd rather keep them and adjust the caller in
`builtin-add-i`.

> >  - This version does not write the index back when refresh_index()
> >    returns non-zero, but the one in builtin-add-i ignores the
> >    returned value.  I think, as a performance measure, it probably
> >    is a better idea to write it back, even when the function returns
> >    non-zero (the local variable's name is has_errors, but having an
> >    entry in the index that does not get refreshed is *not* an error;
> >    e.g. an unmerged entry is a normal thing in the index, and as
> >    long as we refreshed other entries while having an unmerged and
> >    unrefreshable entry, we are making progress that is worth writing
> >    out).
>
> I'm happy with writing the index back even if there are errors.
> However I think we still need the option to get the return code from
> 'refresh_index()', as some callers where I'm using
> 'refresh_and_write_index()' in this series behave differently
> depending on its return code.
>
> There's two more differences between the versions:
>
>  - The version in my series allows passing in write_flags to be passed
>    to write_locked_index, which is required to convert the callers in
>    builtin/merge.c.

I can always pass in 0 as `write_flags`.

>  - Dscho's version also calls 'repo_read_index_preload()', which I
>    don't do in mine.  Some callers don't need to do that, so I think it
>    would be nice to keep that outside of the
>    'repo_refresh_and_write_index()' function.

Agreed.

> I can think of a few ways forward here:
>
>  - I incorporate features that are needed for the builtin-add-i series
>    here, and that is rebased on top of this series.

I'd prefer this way forward. The `builtin-add-i` patch series is
evolving more slowly than yours.

>  - We drop the first two patches of this series, so we only fix the
>    problems in 'git stash' for now.  Later we can have a refactoring
>    series that uses repo_refresh_and_write_index in the places we
>    converted here, once the dust of the builtin-add-i series settled.
>
>  - I rebase this on top of builtin-add-i.
>
> I'm happy with either of the first two, but less so with the last
> option.  I was hoping this series could potentially go to maint as it
> was a bugfix, which we obviously can't do with that option.
>
> Dscho, what do you think? :)

See above ;-)

Thank you!
Dscho

>
> > Thanks.
> >
> > > +int repo_refresh_and_write_index(struct  repository *repo,
> > > +				 unsigned int refresh_flags,
> > > +				 unsigned int write_flags,
> > > +				 const struct pathspec *pathspec,
> > > +				 char *seen, const char *header_msg)
> > > +{
> > > +	struct lock_file lock_file = LOCK_INIT;
> > > +
> > > +	repo_hold_locked_index(repo, &lock_file, LOCK_DIE_ON_ERROR);
> > > +	if (refresh_index(repo->index, refresh_flags, pathspec, seen, header_msg)) {
> > > +		rollback_lock_file(&lock_file);
> > > +		return 1;
> > > +	}
> > > +	if (write_locked_index(repo->index, &lock_file, COMMIT_LOCK | write_flags))
> > > +		return -1;
> > > +	return 0;
> > > +}
> > > +
> > > +
> > >  int refresh_index(struct index_state *istate, unsigned int flags,
> > >  		  const struct pathspec *pathspec,
> > >  		  char *seen, const char *header_msg)
>