From: "Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
To: Johannes Sixt <j6t@kdbg.org>
Cc: "Jeff King" <peff@peff.net>, "Junio C Hamano" <gitster@pobox.com>,
"Krzysztof Żelechowski" <giecrilj@stegny.2a.pl>,
git@vger.kernel.org,
"Hamza Mahfooz" <someguy@effective-light.com>
Subject: Re: *Really* noisy encoding warnings post-v2.33.0
Date: Sun, 10 Oct 2021 17:43:42 +0200 [thread overview]
Message-ID: <87sfx8lw8j.fsf@evledraar.gmail.com> (raw)
In-Reply-To: <5eca71b7-e4df-92a1-35bf-5a99550e558e@kdbg.org>
On Sun, Oct 10 2021, Johannes Sixt wrote:
> Am 09.10.21 um 04:36 schrieb Jeff King:
>> On Sat, Oct 09, 2021 at 02:58:10AM +0200, Ævar Arnfjörð Bjarmason wrote:
>>
>>> I ran into this while testing the grep coloring patch[1] (but it's
>>> unrelated). Before this commit e.g.:
>>>
>>> LC_ALL=C ~/g/git/git -P -c i18n.commitEncoding=ascii log --author=Ævar -100|wc -l
>>> 28333
>>>
>>> So ~3k lines for my last 100 commits, but then:
>>>
>>> $ LC_ALL=C ~/g/git/git -P -c i18n.commitEncoding=ascii log --author=Ævar -100 2>&1|grep -c ^warning
>>> 299
>>>
>>> At first I thought it was spewing warnings for every failed re-encoded
>>> line in some cases, because I get hundreds at a time sometimes, but it's
>>> because stderr and stdout I/O buffering is different (a common
>>> case). Adding a "fflush(stderr)" "fixes" that.
>>
>> I don't think the buffering is the issue. By default stderr flushes on
>> lines, and we flush commits after showing them. If you take away "-P"
>> (or look at the combined 2>&1 output in order), you'll see that they are
>> grouped.
>>
>> Now one thing you might notice is that there may be multiple warnings
>> between output commits. But that's because we really are re-encoding
>> each of those intermediate commits to do your --author grep. And if that
>> re-encoding fails, we may well be producing the wrong output, because
>> the matching won't be correct (in your case, presumably the correct
>> output should be _nothing_, because Æ is not an ascii character).
>
> I don't understand why i18n.commitEncoding plays a role here. Isn't it
> an instruction "when you make a commit, mark the commit message having
> this encoding". But grep does not make a commit.
>
> If this were i18n.logOuputEncoding it would make much more sense.
>
> Have I misunderstood the meaning of the two options?
It doesn't, see my later <871r4umfnm.fsf@evledraar.gmail.com> for when I
got it right.
For the amount of warnings etc. it's the same, whether we call iconv
because it's e.g. ascii->utf-8 and that triggers iconv() issues, or
(with i18n.logOuputEncoding) utf-8->ascii.
next prev parent reply other threads:[~2021-10-10 15:45 UTC|newest]
Thread overview: 33+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-08-24 9:00 git log --encoding=HTML is not supported Krzysztof Żelechowski
2021-08-24 10:31 ` Bagas Sanjaya
2021-08-24 10:33 ` Krzysztof Żelechowski
2021-08-24 10:46 ` Bagas Sanjaya
2021-08-24 19:11 ` Junio C Hamano
2021-08-25 0:57 ` Jeff King
2021-08-25 16:31 ` Junio C Hamano
2021-08-27 18:30 ` Jeff King
2021-08-27 18:32 ` Jeff King
2021-08-27 19:47 ` Junio C Hamano
2021-10-09 0:58 ` *Really* noisy encoding warnings post-v2.33.0 Ævar Arnfjörð Bjarmason
2021-10-09 1:29 ` Ævar Arnfjörð Bjarmason
2021-10-09 2:36 ` Jeff King
2021-10-09 2:42 ` Jeff King
2021-10-09 13:47 ` Ævar Arnfjörð Bjarmason
2021-10-27 11:03 ` Jeff King
2021-10-29 10:47 ` Ævar Arnfjörð Bjarmason
2021-10-29 20:40 ` Jeff King
2021-10-29 20:45 ` Junio C Hamano
2021-10-29 20:52 ` Junio C Hamano
2021-10-29 21:10 ` Jeff King
2021-10-22 22:58 ` Ævar Arnfjörð Bjarmason
2021-10-10 13:53 ` Johannes Sixt
2021-10-10 15:43 ` Ævar Arnfjörð Bjarmason [this message]
2021-08-25 23:00 ` git log --encoding=HTML is not supported Krzysztof Żelechowski
2021-08-27 18:33 ` Jeff King
2021-08-25 23:28 ` Krzysztof Żelechowski
2021-08-25 23:47 ` Bryan Turner
2021-08-26 15:37 ` Junio C Hamano
2021-08-26 20:52 ` Krzysztof Żelechowski
2021-08-27 15:59 ` Junio C Hamano
2021-08-27 18:37 ` Jeff King
2021-08-27 21:51 ` Junio C Hamano
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87sfx8lw8j.fsf@evledraar.gmail.com \
--to=avarab@gmail.com \
--cc=giecrilj@stegny.2a.pl \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=j6t@kdbg.org \
--cc=peff@peff.net \
--cc=someguy@effective-light.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).