From: Philip McGraw <Philip.McGraw@bentley.com>
To: "ahippo@yandex.ru" <ahippo@yandex.ru>
Cc: "git@vger.kernel.org" <git@vger.kernel.org>,
"luke@diamand.org" <luke@diamand.org>
Subject: RE: [PATCH] git-p4: close temporary file before removing
Date: Wed, 31 Jul 2019 21:51:22 +0000 [thread overview]
Message-ID: <BL0PR1901MB209790A0A8F5F9C8EFB8B3F0FFDF0@BL0PR1901MB2097.namprd19.prod.outlook.com> (raw)
In-Reply-To: <2835251564582156@myt6-4218ece6190d.qloud-c.yandex.net>
2019.07.31 10:09 Andrey <ahippo@yandex.ru>
>31.07.2019, 09:53, "Philip McGraw" <philip.mcgraw@bentley.com>:
>>> 30.07.2019, 13:37, "Philip McGraw" <philip.mcgraw@bentley.com>:
>>> > python os.remove() throws exceptions on Windows platform when attempting
>>> > to remove file while it is still open. Need to grab filename while file open,
>>> > close file handle, then remove by name. Apparently other platforms are more
>>> > permissive of removing files while busy.
>>> > reference:
>>> > ---
>>> > git-p4.py | 4 +++-
>>> > 1 file changed, 3 insertions(+), 1 deletion(-)
>>> >
>>> > diff --git a/git-p4.py b/git-p4.py
>>> > index c71a6832e2..6b9d2a8317 100755
>>> > --- a/git-p4.py
>>> > +++ b/git-p4.py
>>> > @@ -1161,12 +1161,14 @@ def exceedsLargeFileThreshold(self, relPath, contents):
>>> > return False
>>> > contentTempFile = self.generateTempFile(contents)
>>> > compressedContentFile = tempfile.NamedTemporaryFile(prefix='git-p4-large-file', delete=False)
>>> > + compressedContentFileName = compressedContentFile.name
>>> > zf = zipfile.ZipFile(compressedContentFile.name, mode='w')
>>> > zf.write(contentTempFile, compress_type=zipfile.ZIP_DEFLATED)
>>> > zf.close()
>>> > compressedContentsSize = zf.infolist()[0].compress_size
>>> > os.remove(contentTempFile)
>>> > - os.remove(compressedContentFile.name)
>>> > + compressedContentFile.close()
>>> > + os.remove(compressedContentFileName)
>>>
>>> I'm not sure why NamedTemporaryFile() is called with delete=False above,
>>> but it appears to me that it can have delete=True instead,
>>> so that there is no need to call os.remove() explicitly
>>> and thus worry about remove vs close ordering at all.
>>>
>>> > if compressedContentsSize > gitConfigInt('git-p4.largeFileCompressedThreshold'):
>>> > return True
>>> > return False
>>> > --
>>> > 2.21.0.windows.1
>>>
>>> Thank you,
>>> Andrey.
>>
>> Thanks Andrey; simpler is certainly better! I will test and re-submit v2 of patch with that approach.
>
>Thank you, that would be great!
>
>--
>Andrey.
Unfortunately it wasn't as simple it seemed: upon testing with only changing delete=True,
found that the problem was not solved. Upon further debugging, recoded/refactored slightly adding
allocateTempFileName() locally scoped function to try to clarify how the NamedTemporaryFile()
was actually being used.
We can't depend on the delete-on-close because the NamedTemporaryFile() is merely allocating
a temporary name for real use by the zipfile open-for-write which fails (on Windows) if file
was not explicitly closed first.
Hopefully the new patch (https://github.com/gitgitgadget/git/pull/301) will make this more clear.
Open to other suggestions if still not clear.
Thanks again,
Philip
next prev parent reply other threads:[~2019-07-31 21:51 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-07-30 17:37 [PATCH] git-p4: close temporary file before removing Philip McGraw
2019-07-31 1:48 ` Andrey
2019-07-31 13:53 ` Philip McGraw
2019-07-31 14:09 ` Andrey
2019-07-31 21:51 ` Philip McGraw [this message]
2019-08-01 1:34 ` Andrey
2019-08-01 15:30 ` Philip McGraw
2019-08-02 3:50 ` Andrey
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=BL0PR1901MB209790A0A8F5F9C8EFB8B3F0FFDF0@BL0PR1901MB2097.namprd19.prod.outlook.com \
--to=philip.mcgraw@bentley.com \
--cc=ahippo@yandex.ru \
--cc=git@vger.kernel.org \
--cc=luke@diamand.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).