Git Rename Detection Bug

[Jeremy Pridmore <jpridmore@rdt.co.uk>]

Thank you for filling out a Git bug report!
Please answer the following questions to help us understand your issue.

What did you do before the bug happened? (Steps to reproduce your issue)
I have two GIT repositories (A and B). Both migrated from the same TFS server using git-tfs tool. I migrated code into A and made lots of changes, including moving 50,000+ files from folder "/Landscape" to "/Landscape/src".  B contains the same code but with various other changes made since my original migration from TFS to A.  All the files in B are still in the "/Landscape" folder.  I recently needed to merge my changes from A to B, so I added A as a remote to B and then performed a number of cherry-picks from A to B, but got stuck when trying to cherry-pick the commit containing the results of moving all files into "/Landscape/src".

What did you expect to happen? (Expected behavior)
I expected the git rename detection to match all files in A "/Landscape" to files in B "/Landscape/src".

What happened instead? (Actual behavior)
Although many files were matched successfully, git mismatched over two dozen similarly named files, e.g.

Incorrect path match: Landscape/Services/uiServices/Complaints/Interfaces/IAccountsIntegration.vb -> Landscape/src/Complaints/Rdt.Complaints.UI/Interfaces/IAccountsIntegration.vb
Incorrect path match: Landscape/Services/uiServices/Complaints/Interfaces/IDocumentIntegration.vb -> Landscape/src/Complaints/Rdt.Complaints.UI/Interfaces/IDocumentIntegration.vb
Incorrect path match: Landscape/Deployment/PowershellScripts/pre-req/Rdt.BatchProcessingService.Setup/pre-req.ps1 -> Landscape/src/Deployment/PowershellScripts/pre-req/Landscape.Net/pre-req.ps1
Incorrect path match: Landscape/Deployment/PowershellScripts/pre-req/Workflow/pre-req.ps1 -> Landscape/src/Deployment/PowershellScripts/pre-req/Rdt.BatchProcessingService.Setup/pre-req.ps1
Incorrect name match: Landscape/Documentation/Rdt.Documentation.UI/Properties/licenses.licx -> Landscape/src/Deployment/PowershellScripts/pre-req/Workflow/pre-req.ps1
Incorrect path match: Landscape/Documentation/uiDocumentation/licenses.licx -> Landscape/src/Documentation/Rdt.Documentation.UI/Properties/licenses.licx
Incorrect path match: Landscape/Import/uiImport/My Project/licenses.licx -> Landscape/src/Documentation/uiDocumentation/licenses.licx
Incorrect path match: Landscape/Main/uiMain.Workflow/My Project/licenses.licx -> Landscape/src/Import/uiImport/My Project/licenses.licx
Incorrect path match: Landscape/Main/uiMain/My Project/licenses.licx -> Landscape/src/Main/uiMain.Workflow/My Project/licenses.licx
Incorrect path match: Landscape/LandscapeApiService.Setup/Setup/UIContent/RDT_Logo.ico -> Landscape/src/Main/uiMain.Workflow/Resources/RDT_Logo.ico
Incorrect path match: Landscape/Policy/Rdt.Policy.UI.Templates/Properties/licenses.licx -> Landscape/src/Main/uiMain/My Project/licenses.licx
Incorrect path match: Landscape/Main/uiMain.Workflow/Resources/RDT_Logo.ico -> Landscape/src/Main/uiMain/Resources/RDT_Logo.ico
Incorrect path match: Landscape/Policy/Rdt.Policy.UI/Properties/licenses.licx -> Landscape/src/Policy/Rdt.Policy.UI.Templates/Properties/licenses.licx
Incorrect path match: Landscape/Rates/uiRates/My Project/licenses.licx -> Landscape/src/Policy/Rdt.Policy.UI/Properties/licenses.licx
Incorrect path match: Landscape/Rdt.Claim.UI/Properties/licenses.licx -> Landscape/src/Rates/uiRates/My Project/licenses.licx
Incorrect path match: Landscape/Rdt.Landscape.UI.Templates.Workflow/Properties/licenses.licx -> Landscape/src/Rdt.Claim.UI/Properties/licenses.licx
Incorrect path match: Landscape/Rdt.Landscape.UI.Templates/Properties/licenses.licx -> Landscape/src/Rdt.Landscape.UI.Templates.Workflow/Properties/licenses.licx
Incorrect path match: Landscape/Rdt.Landscape.UI.Workflow/Properties/licenses.licx -> Landscape/src/Rdt.Landscape.UI.Templates/Properties/licenses.licx
Incorrect path match: Landscape/Rdt.Landscape.UI/Properties/licenses.licx -> Landscape/src/Rdt.Landscape.UI.Workflow/Properties/licenses.licx
Incorrect path match: Landscape/StandardLetters/uiStandardLetters/My Project/licenses.licx -> Landscape/src/Rdt.Landscape.UI/Properties/licenses.licx
Incorrect path match: Landscape/Complaints/Rdt.Complaints.UI/Interfaces/IDocumentIntegration.vb -> Landscape/src/Services/uiServices/Complaints/Interfaces/IDocumentIntegration.vb
Incorrect path match: Landscape/SystemEvents/uiSystemEvents/My Project/licenses.licx -> Landscape/src/StandardLetters/uiStandardLetters/My Project/licenses.licx
Incorrect path match: Landscape/Services/busServices/RDT_Logo.ico -> Landscape/src/Startup/uiStartup.Workflow/Resources/RDT_Logo.ico
Incorrect path match: Landscape/Startup/uiStartup.Workflow/Resources/RDT_Logo.ico -> Landscape/src/Startup/uiStartup/Resources/RDT_Logo.ico
Incorrect path match: Landscape/Startup/uiStartup/Resources/RDT_Logo.ico -> Landscape/src/Startup/uiStartup32/RDT_Logo.ico
Incorrect path match: Landscape/Startup/uiStartup/Resources/newrdlogogradiant48shad.ico -> Landscape/src/Startup/uiStartup32/newrdlogogradiant48shad.ico
Incorrect path match: Landscape/Templates/uiTemplates.Workflow/My Project/licenses.licx -> Landscape/src/SystemEvents/uiSystemEvents/My Project/licenses.licx
Incorrect path match: Landscape/Utils/Rdt.Utils.UI/Properties/licenses.licx -> Landscape/src/Templates/uiTemplates.Workflow/My Project/licenses.licx
Incorrect path match: Landscape/Utils/uiUtils/My Project/licenses.licx -> Landscape/src/Utils/Rdt.Utils.UI/Properties/licenses.licx
Incorrect name match: Landscape/WebServices/ServiceFabric/Policy/Rdt.Policy.Repository.Service.Fabric.Host/PackageRoot/Data/Swagger/Examples/POST_UKSTasks_Response.json -> Landscape/src/Utils/uiUtils/My Project/licenses.licx


What's different between what you expected and what actually happened?

As you can see, although the filenames (and content) are the same, there are a few files that have been incorrectly matched to files with teh same name, but in a different sub folder.  In some cases, it seems that the catalyst has been git thinking that a file from B has been deleted from A, when in fact it has not actually been deleted at all.
For example, the file Landscape/Deployment/PowershellScripts/pre-req/Landscape.Net/pre-req.ps1 has not been deleted in A or B, therefore git should not have attempted to rename Landscape/Deployment/PowershellScripts/pre-req/Rdt.BatchProcessingService.Setup/pre-req.ps1 to Landscape/Deployment/PowershellScripts/pre-req/Landscape.Net/pre-req.ps1, especially as it then attempts to rename Landscape/Deployment/PowershellScripts/pre-req/Workflow/pre-req.ps1 to Landscape/src/Deployment/PowershellScripts/pre-req/Rdt.BatchProcessingService.Setup/pre-req.ps1 and so on.

Git status contains, for example:
        deleted by them: Landscape/Deployment/PowershellScripts/pre-req/Landscape.Net/pre-req.ps1
        renamed:    Landscape/Deployment/PowershellScripts/pre-req/Rdt.BatchProcessingService.Setup/pre-req.ps1 -> Landscape/src/Deployment/PowershellScripts/pre-req/Landscape.Net/pre-req.ps1

The correct renames should have been:
git mv "Landscape/Complaints/Rdt.Complaints.UI/Interfaces/IAccountsIntegration.vb" "Landscape/src/Complaints/Rdt.Complaints.UI/Interfaces/IAccountsIntegration.vb"
git mv "Landscape/Complaints/Rdt.Complaints.UI/Interfaces/IDocumentIntegration.vb" "Landscape/src/Complaints/Rdt.Complaints.UI/Interfaces/IDocumentIntegration.vb"
git mv "Landscape/Deployment/PowershellScripts/pre-req/Rdt.BatchProcessingService.Setup/pre-req.ps1" "Landscape/src/Deployment/PowershellScripts/pre-req/Rdt.BatchProcessingService.Setup/pre-req.ps1"
git mv "Landscape/Deployment/PowershellScripts/pre-req/Workflow/pre-req.ps1" "Landscape/src/Deployment/PowershellScripts/pre-req/Workflow/pre-req.ps1"
git mv "Landscape/Documentation/Rdt.Documentation.UI/Properties/licenses.licx" "Landscape/src/Documentation/Rdt.Documentation.UI/Properties/licenses.licx"
git mv "Landscape/Documentation/uiDocumentation/licenses.licx" "Landscape/src/Documentation/uiDocumentation/licenses.licx"
git mv "Landscape/Import/uiImport/My Project/licenses.licx" "Landscape/src/Import/uiImport/My Project/licenses.licx"
git mv "Landscape/Main/uiMain.Workflow/My Project/licenses.licx" "Landscape/src/Main/uiMain.Workflow/My Project/licenses.licx"
git mv "Landscape/Main/uiMain.Workflow/Resources/RDT_Logo.ico" "Landscape/src/Main/uiMain.Workflow/Resources/RDT_Logo.ico"
git mv "Landscape/Main/uiMain/My Project/licenses.licx" "Landscape/src/Main/uiMain/My Project/licenses.licx"
git mv "Landscape/Main/uiMain/Resources/RDT_Logo.ico" "Landscape/src/Main/uiMain/Resources/RDT_Logo.ico"
git mv "Landscape/Policy/Rdt.Policy.UI.Templates/Properties/licenses.licx" "Landscape/src/Policy/Rdt.Policy.UI.Templates/Properties/licenses.licx"
git mv "Landscape/Policy/Rdt.Policy.UI/Properties/licenses.licx" "Landscape/src/Policy/Rdt.Policy.UI/Properties/licenses.licx"
git mv "Landscape/Rdt.Claim.UI/Properties/licenses.licx" "Landscape/src/Rdt.Claim.UI/Properties/licenses.licx"
git mv "Landscape/Rdt.Landscape.UI.Templates.Workflow/Properties/licenses.licx" "Landscape/src/Rdt.Landscape.UI.Templates.Workflow/Properties/licenses.licx"
git mv "Landscape/Rdt.Landscape.UI.Templates/Properties/licenses.licx" "Landscape/src/Rdt.Landscape.UI.Templates/Properties/licenses.licx"
git mv "Landscape/Rdt.Landscape.UI.Workflow/Properties/licenses.licx" "Landscape/src/Rdt.Landscape.UI.Workflow/Properties/licenses.licx"
git mv "Landscape/Rdt.Landscape.UI/Properties/licenses.licx" "Landscape/src/Rdt.Landscape.UI/Properties/licenses.licx"
git mv "Landscape/Services/busServices/RDT_Logo.ico" "Landscape/src/Services/busServices/RDT_Logo.ico"
git mv "Landscape/Services/uiServices/Complaints/Interfaces/IAccountsIntegration.vb" "Landscape/src/Services/uiServices/Complaints/Interfaces/IAccountsIntegration.vb"
git mv "Landscape/Services/uiServices/Complaints/Interfaces/IDocumentIntegration.vb" "Landscape/src/Services/uiServices/Complaints/Interfaces/IDocumentIntegration.vb"
git mv "Landscape/StandardLetters/uiStandardLetters/My Project/licenses.licx" "Landscape/src/StandardLetters/uiStandardLetters/My Project/licenses.licx"
git mv "Landscape/Startup/uiStartup.Workflow/Resources/RDT_Logo.ico" "Landscape/src/Startup/uiStartup.Workflow/Resources/RDT_Logo.ico"
git mv "Landscape/Startup/uiStartup/Resources/RDT_Logo.ico" "Landscape/src/Startup/uiStartup/Resources/RDT_Logo.ico"
git mv "Landscape/SystemEvents/uiSystemEvents/My Project/licenses.licx" "Landscape/src/SystemEvents/uiSystemEvents/My Project/licenses.licx"
git mv "Landscape/Templates/uiTemplates.Workflow/My Project/licenses.licx" "Landscape/src/Templates/uiTemplates.Workflow/My Project/licenses.licx"
git mv "Landscape/Utils/Rdt.Utils.UI/Properties/licenses.licx" "Landscape/src/Utils/Rdt.Utils.UI/Properties/licenses.licx"
git mv "Landscape/Utils/uiUtils/My Project/licenses.licx" "Landscape/src/Utils/uiUtils/My Project/licenses.licx"


Anything else you want to add:
I can't help but think that this is related to changes made by Palantir:
https://blog.palantir.com/optimizing-gits-merge-machinery-1-127ceb0ef2a1

I have tried to unstage these renames using "git restore --staged <file_name>" so I can then apply the correct "git mv" commands, but bizzarely, this then results in "git status" reporting a different, smaller set of mismatched names:

Incorrect name match: Landscape/Services/busServices/Service References/DataCollectorService/busServices.DataCollectorService.GetDataResponse.datasource -> Landscape/src/Deployment/PowershellScripts/pre-req/Landscape.Net/pre-req.ps1
Incorrect name match: Landscape/Services/busServices/Service References/GlobalGatewayService/busServices.GlobalGatewayService.OrderResponse.datasource -> Landscape/src/Deployment/PowershellScripts/pre-req/Rdt.BatchProcessingService.Setup/pre-req.ps1
Incorrect name match: Landscape/WebServices/ServiceFabric/Policy/Rdt.Policy.Repository.Service.Fabric.Host/PackageRoot/Data/Swagger/Examples/POST_UKSTasks_Response.json -> Landscape/src/Deployment/PowershellScripts/pre-req/Workflow/pre-req.ps1
Incorrect path match: Landscape/LandscapeApiService.Setup/Setup/UIContent/RDT_Logo.ico -> Landscape/src/Main/uiMain.Workflow/Resources/RDT_Logo.ico
Incorrect path match: Landscape/WebServices/WCFServices/Landscape WCF Service/wsWcfLandscapeServices/RDT_Logo.ico -> Landscape/src/Main/uiMain/Resources/RDT_Logo.ico
Incorrect path match: Landscape/WindowsServices/Rdt.BatchProcessingService/Rdt.BatchProcessingService.Integration/RDT_Logo.ico -> Landscape/src/Services/busServices/RDT_Logo.ico
Incorrect path match: Landscape/WindowsServices/Rdt.BatchProcessingService/Rdt.BatchProcessingService.Setup/UIContent/RDT_Logo.ico -> Landscape/src/Startup/uiStartup.Workflow/Resources/RDT_Logo.ico
Incorrect path match: Landscape/WindowsServices/Rdt.BatchProcessingService/Rdt.BatchProcessingService/RDT_Logo.ico -> Landscape/src/Startup/uiStartup/Resources/RDT_Logo.ico
Incorrect path match: Landscape/_Tests/Rdt.BatchProcessingService.Tests/RDT_Logo.ico -> Landscape/src/Startup/uiStartup32/RDT_Logo.ico
Incorrect path match: Landscape/Startup/uiStartup/Resources/newrdlogogradiant48shad.ico -> Landscape/src/Startup/uiStartup32/newrdlogogradiant48shad.ico
Incorrect path match: Landscape/uiStartup.Setup/Setup/UIContent/RDT_Logo.ico -> Landscape/src/WebServices/WCFServices/Landscape WCF Service/wsWcfLandscapeServices/RDT_Logo.ico
Incorrect path match: Landscape/wisWorkflow.Setup/Setup/UIContent/RDT_Logo.ico -> Landscape/src/WindowsServices/Rdt.BatchProcessingService/Rdt.BatchProcessingService.Integration/RDT_Logo.ico
Incorrect path match: Landscape/wsXmlLandscapeServices.Setup/Setup/Binary/RDT_Logo.ico -> Landscape/src/WindowsServices/Rdt.BatchProcessingService/Rdt.BatchProcessingService.Setup/UIContent/RDT_Logo.ico

Notice how "busServices.DataCollectorService.GetDataResponse.datasource" is now renamed to "Landscape.Net/pre-req.ps1", whereas the datasource file wasn't even listed previously?

Please help.  I've been on this for a couple of weeks now and I'm running out of ideas.

Please review the rest of the bug report below.
You can delete any lines you don't wish to share.


[System Info]
git version:
git version 2.42.0.windows.2
cpu: x86_64
built from commit: 2f819d1670fff9a1818f63b6722e9959405378e3
sizeof-long: 4
sizeof-size_t: 8
shell-path: /bin/sh
feature: fsmonitor--daemon
uname: Windows 10.0 19044
compiler info: gnuc: 13.2
libc info: no libc information available
$SHELL (typically, interactive shell): <unset>


[Enabled Hooks]
<none>


Regards,

Jeremy Pridmore
Lead Solution Architect

________________________________

DISCLAIMER This email is confidential. It should only be read by those persons to whom it is addressed. RDT Ltd accept no liability for the consequences of any person acting, or refraining from acting, on any information contained within this e-mail or any attached documents prior to the receipt by those persons of subsequent written confirmation of that information. If you think this e-mail may not be intended for you, do not use, pass on or copy the transmission in any way. While all reasonable precautions are taken to minimise the risk of transmitting software viruses we advise you to carry out your own virus checks on any attachment to this message. We cannot accept liability for any loss or damage caused by software viruses.