git@vger.kernel.org list mirror (unofficial, one of many)
 help / color / Atom feed
* [BUG] rebase --interactive silently overwrites ignored files
@ 2019-04-11 23:56 wh
  2019-04-12 16:30 ` Phillip Wood
  0 siblings, 1 reply; 8+ messages in thread
From: wh @ 2019-04-11 23:56 UTC (permalink / raw)
  To: git

I'm using git 2.20.1 from Debian. Git is usually careful not to
overwrite untracked files, including ignored files. But interactive
rebase doesn't detect this (non-interactive rebase works fine).

Reproduction:
-----

#!/bin/sh
mkdir upstream
cd upstream
git init
echo 1 >feature-1
git add feature-1
git commit -m "feature 1"

cd ..
git clone upstream local
cd local
# write some tools for our own convenience
echo ours >tools
echo /tools >>.git/info/exclude
# start working on a feature
git checkout -b f2
echo wip >feature-2
git add feature-2
git commit -m "wip"

cd ../upstream
# official tools are available
echo theirs >tools
git add tools
git commit -m "tools"

cd ../local
git fetch ../upstream master

# this would be okay
#git rebase FETCH_HEAD

# problem: overwrites tools silently
GIT_EDITOR=true git rebase -i FETCH_HEAD

cat tools

-----

Expected: `git rebase -i` fails because it would have to overwrite the
untracked "tools" file. Contents of tools file remains `ours`.

Actual: Contents of tools file becomes `theirs`.

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [BUG] rebase --interactive silently overwrites ignored files
  2019-04-11 23:56 [BUG] rebase --interactive silently overwrites ignored files wh
@ 2019-04-12 16:30 ` Phillip Wood
  2019-04-14  1:59   ` wh
  0 siblings, 1 reply; 8+ messages in thread
From: Phillip Wood @ 2019-04-12 16:30 UTC (permalink / raw)
  To: wh, git

Hi

On 12/04/2019 00:56, wh wrote:
> I'm using git 2.20.1 from Debian. Git is usually careful not to
> overwrite untracked files, including ignored files.

Git normally overwrites ignored files, so I think in your example rebase 
-i is working as expected, I'm surprised that the am based rebase does 
not overwrite the ignored file. There has been some discussion about 
introducing 'precious' files that are ignored but protected in the same 
way as untracked files [1].

Best Wishes

Phillip

[1] https://public-inbox.org/git/20190409102649.22115-1-pclouds@gmail.com/

  But interactive
> rebase doesn't detect this (non-interactive rebase works fine).
> 
> Reproduction:
> -----
> 
> #!/bin/sh
> mkdir upstream
> cd upstream
> git init
> echo 1 >feature-1
> git add feature-1
> git commit -m "feature 1"
> 
> cd ..
> git clone upstream local
> cd local
> # write some tools for our own convenience
> echo ours >tools
> echo /tools >>.git/info/exclude
> # start working on a feature
> git checkout -b f2
> echo wip >feature-2
> git add feature-2
> git commit -m "wip"
> 
> cd ../upstream
> # official tools are available
> echo theirs >tools
> git add tools
> git commit -m "tools"
> 
> cd ../local
> git fetch ../upstream master
> 
> # this would be okay
> #git rebase FETCH_HEAD
> 
> # problem: overwrites tools silently
> GIT_EDITOR=true git rebase -i FETCH_HEAD
> 
> cat tools
> 
> -----
> 
> Expected: `git rebase -i` fails because it would have to overwrite the
> untracked "tools" file. Contents of tools file remains `ours`.
> 
> Actual: Contents of tools file becomes `theirs`.
> 

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [BUG] rebase --interactive silently overwrites ignored files
  2019-04-12 16:30 ` Phillip Wood
@ 2019-04-14  1:59   ` wh
  2019-04-14 13:24     ` Phillip Wood
  2019-05-02 15:45     ` Phillip Wood
  0 siblings, 2 replies; 8+ messages in thread
From: wh @ 2019-04-14  1:59 UTC (permalink / raw)
  To: phillip.wood; +Cc: git

Thanks for the info about the upcoming "precious" attribute. Looks useful.

I didn't get the impression that Git normally overwrites ignored
files. I ran some more experiments:

git rebase FETCH_HEAD        # bails
git rebase -i FETCH_HEAD     # overwrites
git merge FETCH_HEAD         # bails
git reset --keep FETCH_HEAD  # bails
git reset --merge FETCH_HEAD # bails
git checkout FETCH_HEAD      # overwrites
# without feature 2 locally:
git merge FETCH_HEAD         # overwrites, fast-forwards
git merge --no-ff FETCH_HEAD # bails

On Fri, Apr 12, 2019 at 9:30 AM Phillip Wood <phillip.wood123@gmail.com> wrote:
>
> Hi
>
> On 12/04/2019 00:56, wh wrote:
> > I'm using git 2.20.1 from Debian. Git is usually careful not to
> > overwrite untracked files, including ignored files.
>
> Git normally overwrites ignored files, so I think in your example rebase
> -i is working as expected, I'm surprised that the am based rebase does
> not overwrite the ignored file. There has been some discussion about
> introducing 'precious' files that are ignored but protected in the same
> way as untracked files [1].
>
> Best Wishes
>
> Phillip
>
> [1] https://public-inbox.org/git/20190409102649.22115-1-pclouds@gmail.com/
>
>   But interactive
> > rebase doesn't detect this (non-interactive rebase works fine).
> >
> > Reproduction:
> > -----
> >
> > #!/bin/sh
> > mkdir upstream
> > cd upstream
> > git init
> > echo 1 >feature-1
> > git add feature-1
> > git commit -m "feature 1"
> >
> > cd ..
> > git clone upstream local
> > cd local
> > # write some tools for our own convenience
> > echo ours >tools
> > echo /tools >>.git/info/exclude
> > # start working on a feature
> > git checkout -b f2
> > echo wip >feature-2
> > git add feature-2
> > git commit -m "wip"
> >
> > cd ../upstream
> > # official tools are available
> > echo theirs >tools
> > git add tools
> > git commit -m "tools"
> >
> > cd ../local
> > git fetch ../upstream master
> >
> > # this would be okay
> > #git rebase FETCH_HEAD
> >
> > # problem: overwrites tools silently
> > GIT_EDITOR=true git rebase -i FETCH_HEAD
> >
> > cat tools
> >
> > -----
> >
> > Expected: `git rebase -i` fails because it would have to overwrite the
> > untracked "tools" file. Contents of tools file remains `ours`.
> >
> > Actual: Contents of tools file becomes `theirs`.
> >

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [BUG] rebase --interactive silently overwrites ignored files
  2019-04-14  1:59   ` wh
@ 2019-04-14 13:24     ` Phillip Wood
  2019-05-02 15:45     ` Phillip Wood
  1 sibling, 0 replies; 8+ messages in thread
From: Phillip Wood @ 2019-04-14 13:24 UTC (permalink / raw)
  To: wh, phillip.wood; +Cc: git

On 14/04/2019 02:59, wh wrote:
> Thanks for the info about the upcoming "precious" attribute. Looks useful.

Hmm, unfortunately it's not looking so hopeful now [1]

> I didn't get the impression that Git normally overwrites ignored
> files. I ran some more experiments:
> 
> git rebase FETCH_HEAD        # bails
> git rebase -i FETCH_HEAD     # overwrites
> git merge FETCH_HEAD         # bails
> git reset --keep FETCH_HEAD  # bails
> git reset --merge FETCH_HEAD # bails
> git checkout FETCH_HEAD      # overwrites
> # without feature 2 locally:
> git merge FETCH_HEAD         # overwrites, fast-forwards
> git merge --no-ff FETCH_HEAD # bails

Thanks for sharing that, I was assuming the other commands behaved like 
checkout, I'm intrigued that merge behaves differently to rebase -i as 
they use the same underlying machinery, I'm a bit snowed under at the 
moment but I'll try and take a look at what's going sometime in the not 
too distant future.

Best Wishes

Phillip

[1] 
https://public-inbox.org/git/CACsJy8AEZ-Lz6zgEsuNukvphB9TTa9FAC1gK05fhnie2xtfc9w@mail.gmail.com/T/#u
> On Fri, Apr 12, 2019 at 9:30 AM Phillip Wood <phillip.wood123@gmail.com> wrote:
>>
>> Hi
>>
>> On 12/04/2019 00:56, wh wrote:
>>> I'm using git 2.20.1 from Debian. Git is usually careful not to
>>> overwrite untracked files, including ignored files.
>>
>> Git normally overwrites ignored files, so I think in your example rebase
>> -i is working as expected, I'm surprised that the am based rebase does
>> not overwrite the ignored file. There has been some discussion about
>> introducing 'precious' files that are ignored but protected in the same
>> way as untracked files [1].
>>
>> Best Wishes
>>
>> Phillip
>>
>> [1] https://public-inbox.org/git/20190409102649.22115-1-pclouds@gmail.com/
>>
>>    But interactive
>>> rebase doesn't detect this (non-interactive rebase works fine).
>>>
>>> Reproduction:
>>> -----
>>>
>>> #!/bin/sh
>>> mkdir upstream
>>> cd upstream
>>> git init
>>> echo 1 >feature-1
>>> git add feature-1
>>> git commit -m "feature 1"
>>>
>>> cd ..
>>> git clone upstream local
>>> cd local
>>> # write some tools for our own convenience
>>> echo ours >tools
>>> echo /tools >>.git/info/exclude
>>> # start working on a feature
>>> git checkout -b f2
>>> echo wip >feature-2
>>> git add feature-2
>>> git commit -m "wip"
>>>
>>> cd ../upstream
>>> # official tools are available
>>> echo theirs >tools
>>> git add tools
>>> git commit -m "tools"
>>>
>>> cd ../local
>>> git fetch ../upstream master
>>>
>>> # this would be okay
>>> #git rebase FETCH_HEAD
>>>
>>> # problem: overwrites tools silently
>>> GIT_EDITOR=true git rebase -i FETCH_HEAD
>>>
>>> cat tools
>>>
>>> -----
>>>
>>> Expected: `git rebase -i` fails because it would have to overwrite the
>>> untracked "tools" file. Contents of tools file remains `ours`.
>>>
>>> Actual: Contents of tools file becomes `theirs`.
>>>

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [BUG] rebase --interactive silently overwrites ignored files
  2019-04-14  1:59   ` wh
  2019-04-14 13:24     ` Phillip Wood
@ 2019-05-02 15:45     ` Phillip Wood
  2019-05-02 16:22       ` Duy Nguyen
  2019-05-05  4:02       ` Junio C Hamano
  1 sibling, 2 replies; 8+ messages in thread
From: Phillip Wood @ 2019-05-02 15:45 UTC (permalink / raw)
  To: wh, phillip.wood
  Cc: git, Junio C Hamano, Johannes Schindelin,
	Nguyễn Thái Ngọc Duy

Hi wh

On 14/04/2019 02:59, wh wrote:
> Thanks for the info about the upcoming "precious" attribute. Looks useful.
> 
> I didn't get the impression that Git normally overwrites ignored
> files. I ran some more experiments:
> 
> git rebase FETCH_HEAD        # bails
> git rebase -i FETCH_HEAD     # overwrites
> git merge FETCH_HEAD         # bails
> git reset --keep FETCH_HEAD  # bails
> git reset --merge FETCH_HEAD # bails
> git checkout FETCH_HEAD      # overwrites
> # without feature 2 locally:
> git merge FETCH_HEAD         # overwrites, fast-forwards
> git merge --no-ff FETCH_HEAD # bails

I've had a look at the rebase -i code and I think it only overwrites 
ignored files when it is fast-forwarding. This matches what merge does 
when fast-forwarding but I'm not convinced either of them should be 
doing this by default (I think checkout doing it is probably asking for 
trouble), as you point out most operations preserve ignored files. When 
pull fast-forwards it does not overwrite ignored files.

Both checkout and merge have an undocumented --overwrite-ignore option 
which is on by default. Checkout and fast-forward merges overwriting 
ignored files dates back a long way to commit 1127148089 ("Loosen 
"working file will be lost" check in Porcelain-ish", 2006-12-04). There 
does not seem to have been much discussion of changing the semantics of 
ignored files around the patches that implemented this [1,2]. There is a 
brief mention of doing it in this thread [3] but no one seems to comment 
on the idea as far as I can see. Before this change ignored files were 
just ignored by status and add but otherwise treated like any other 
untracked file.

I'll put a patch together to fix rebase -i, I'd like to see the defaults 
for merge and checkout changed but I'm not sure that would be popular. 
It does seem like surprising behavior though when most operations try to 
preserve untracked files.

Best Wishes

Phillip

[1] 
https://public-inbox.org/git/7vlklnkv39.fsf@assigned-by-dhcp.cox.net/T/#u
[2] 
https://public-inbox.org/git/7vbqmjkuzd.fsf@assigned-by-dhcp.cox.net/T/#u
[3]https://public-inbox.org/git/Pine.LNX.4.64.0610081657400.3952@g5.osdl.org/

> On Fri, Apr 12, 2019 at 9:30 AM Phillip Wood <phillip.wood123@gmail.com> wrote:
>>
>> Hi
>>
>> On 12/04/2019 00:56, wh wrote:
>>> I'm using git 2.20.1 from Debian. Git is usually careful not to
>>> overwrite untracked files, including ignored files.
>>
>> Git normally overwrites ignored files, so I think in your example rebase
>> -i is working as expected, I'm surprised that the am based rebase does
>> not overwrite the ignored file. There has been some discussion about
>> introducing 'precious' files that are ignored but protected in the same
>> way as untracked files [1].
>>
>> Best Wishes
>>
>> Phillip
>>
>> [1] https://public-inbox.org/git/20190409102649.22115-1-pclouds@gmail.com/
>>
>>    But interactive
>>> rebase doesn't detect this (non-interactive rebase works fine).
>>>
>>> Reproduction:
>>> -----
>>>
>>> #!/bin/sh
>>> mkdir upstream
>>> cd upstream
>>> git init
>>> echo 1 >feature-1
>>> git add feature-1
>>> git commit -m "feature 1"
>>>
>>> cd ..
>>> git clone upstream local
>>> cd local
>>> # write some tools for our own convenience
>>> echo ours >tools
>>> echo /tools >>.git/info/exclude
>>> # start working on a feature
>>> git checkout -b f2
>>> echo wip >feature-2
>>> git add feature-2
>>> git commit -m "wip"
>>>
>>> cd ../upstream
>>> # official tools are available
>>> echo theirs >tools
>>> git add tools
>>> git commit -m "tools"
>>>
>>> cd ../local
>>> git fetch ../upstream master
>>>
>>> # this would be okay
>>> #git rebase FETCH_HEAD
>>>
>>> # problem: overwrites tools silently
>>> GIT_EDITOR=true git rebase -i FETCH_HEAD
>>>
>>> cat tools
>>>
>>> -----
>>>
>>> Expected: `git rebase -i` fails because it would have to overwrite the
>>> untracked "tools" file. Contents of tools file remains `ours`.
>>>
>>> Actual: Contents of tools file becomes `theirs`.
>>>

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [BUG] rebase --interactive silently overwrites ignored files
  2019-05-02 15:45     ` Phillip Wood
@ 2019-05-02 16:22       ` Duy Nguyen
  2019-05-05  4:02       ` Junio C Hamano
  1 sibling, 0 replies; 8+ messages in thread
From: Duy Nguyen @ 2019-05-02 16:22 UTC (permalink / raw)
  To: Phillip Wood; +Cc: wh, Git Mailing List, Junio C Hamano, Johannes Schindelin

On Thu, May 2, 2019 at 10:45 PM Phillip Wood <phillip.wood123@gmail.com> wrote:
>
> Hi wh
>
> On 14/04/2019 02:59, wh wrote:
> > Thanks for the info about the upcoming "precious" attribute. Looks useful.
> >
> > I didn't get the impression that Git normally overwrites ignored
> > files.

Please do not include me in these discussions again. Thank you.
-- 
Duy

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [BUG] rebase --interactive silently overwrites ignored files
  2019-05-02 15:45     ` Phillip Wood
  2019-05-02 16:22       ` Duy Nguyen
@ 2019-05-05  4:02       ` Junio C Hamano
  2019-05-07 14:03         ` Phillip Wood
  1 sibling, 1 reply; 8+ messages in thread
From: Junio C Hamano @ 2019-05-05  4:02 UTC (permalink / raw)
  To: Phillip Wood
  Cc: wh, phillip.wood, git, Johannes Schindelin,
	Nguyễn Thái Ngọc Duy

Phillip Wood <phillip.wood123@gmail.com> writes:

> I've had a look at the rebase -i code and I think it only overwrites
> ignored files when it is fast-forwarding. This matches what merge does
> when fast-forwarding but I'm not convinced either of them should be
> doing this by default (I think checkout doing it is probably asking
> for trouble)...

> I'll put a patch together to fix rebase -i, I'd like to see the
> defaults for merge and checkout changed but I'm not sure that would be
> popular.

I suspect that such a change in behaviour for "rebase -i" is not a
fix but a regression.  Shouldn't it be consistent with checkout and
merge?

> It does seem like surprising behavior though when most
> operations try to preserve untracked files.

Are you conflating untracked and ignored?

Because we haven't adopted 'precious' (or whatever the final name
would be), which is "ignored but not expendable", ignored files are
by definition "ignored and expendable".  When checkout (and merge
that is its more general form) needs to match the working tree to
the index contents and an ingored file is in the way, it should
overwrite it.  

Until we introduce "ignored but not expendable" class, that is.

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [BUG] rebase --interactive silently overwrites ignored files
  2019-05-05  4:02       ` Junio C Hamano
@ 2019-05-07 14:03         ` Phillip Wood
  0 siblings, 0 replies; 8+ messages in thread
From: Phillip Wood @ 2019-05-07 14:03 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: wh, phillip.wood, git, Johannes Schindelin,
	Nguyễn Thái Ngọc Duy

Hi Junio

On 05/05/2019 05:02, Junio C Hamano wrote:
> Phillip Wood <phillip.wood123@gmail.com> writes:
> 
>> I've had a look at the rebase -i code and I think it only overwrites
>> ignored files when it is fast-forwarding. This matches what merge does
>> when fast-forwarding but I'm not convinced either of them should be
>> doing this by default (I think checkout doing it is probably asking
>> for trouble)...
> 
>> I'll put a patch together to fix rebase -i, I'd like to see the
>> defaults for merge and checkout changed but I'm not sure that would be
>> popular.
> 
> I suspect that such a change in behaviour for "rebase -i" is not a
> fix but a regression.  Shouldn't it be consistent with checkout and
> merge?

My problem is that I don't really understand why some operations should 
overwrite ignored files and others shouldn't. Having nearly all rebase 
operations preserve ignored files and only overwriting them for a 
fast-forward seems confusing and probably surprising to users. Is it a 
bug that pull does not overwrite ignored files when fast-forwarding but 
merge does? Why is it ok to overwrite ignored files when merge 
fast-forwards but not otherwise?

>> It does seem like surprising behavior though when most
>> operations try to preserve untracked files.
> 
> Are you conflating untracked and ignored?
> 
> Because we haven't adopted 'precious' (or whatever the final name
> would be), which is "ignored but not expendable", ignored files are
> by definition "ignored and expendable".  

My confusion is that they are only "ignored and expendable" in certain 
circumstances. Many git operations actually fail if they are going to 
overwrite an ignored file [1] - I don't understand if that is 
intentional or not.

Best Wishes

Phillip

[1] 
https://public-inbox.org/git/CAL_tzDFQQtDYMStN+RDVYN_TzJmO+kufMhG9PGHwvsUWREpgWQ@mail.gmail.com/

> When checkout (and merge
> that is its more general form) needs to match the working tree to
> the index contents and an ingored file is in the way, it should
> overwrite it.
> 
> Until we introduce "ignored but not expendable" class, that is.
> 

^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, back to index

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-04-11 23:56 [BUG] rebase --interactive silently overwrites ignored files wh
2019-04-12 16:30 ` Phillip Wood
2019-04-14  1:59   ` wh
2019-04-14 13:24     ` Phillip Wood
2019-05-02 15:45     ` Phillip Wood
2019-05-02 16:22       ` Duy Nguyen
2019-05-05  4:02       ` Junio C Hamano
2019-05-07 14:03         ` Phillip Wood

git@vger.kernel.org list mirror (unofficial, one of many)

Archives are clonable:
	git clone --mirror https://public-inbox.org/git
	git clone --mirror http://ou63pmih66umazou.onion/git
	git clone --mirror http://czquwvybam4bgbro.onion/git
	git clone --mirror http://hjrcffqmbrq6wope.onion/git

Example config snippet for mirrors

Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.version-control.git
	nntp://ou63pmih66umazou.onion/inbox.comp.version-control.git
	nntp://czquwvybam4bgbro.onion/inbox.comp.version-control.git
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.version-control.git
	nntp://news.gmane.org/gmane.comp.version-control.git

 note: .onion URLs require Tor: https://www.torproject.org/

AGPL code for this site: git clone https://public-inbox.org/ public-inbox