[PATCH v9 0/2] strftime: Improve the width of alternative representation for year [BZ #23758][BZ #24096]

[PATCH v9 0/2] strftime: Improve the width of alternative representation for year [BZ #23758][BZ #24096]

[TAMUKI Shoichi]

Hello,

This is the new set of patches.  I have split the patch into 2 parts.

Note, Patch 1 is same as v8.

Changes include:

Patch 1: strftime: Set the default width of "%Ey" to 2
Patch 2: strftime: Pass the additional flags from "%EY" to "%Ey"

Regards,
TAMUKI Shoichi

[PATCH v9 1/2] strftime: Set the default width of "%Ey" to 2 [BZ #23758]

[TAMUKI Shoichi]

In Japanese locales, strftime's alternative year format (%Ey) produces
a year numbered within a time period called an _era_.  A new era
typically begins when a new emperor is enthroned.  The result of "%Ey"
is therefore usually a one- or two-digit number.

Many programs that display Japanese era dates assume that the era year
is two digits wide.  To improve how these programs display dates
during the first nine years of a new era, change "%Ey" to pad one-
digit numbers on the left with a zero.  This change applies to all
locales.  It is expected to be harmless for other locales that use the
alternative year format (e.g. lo_LA and th_TH, in which "%Ey" produces
the year of the Buddhist calendar) as those calendars' year numbers
are already more than two digits wide, and this is not expected to
change.

This change needs to be in place before 2019-05-01 CE, as a new era is
scheduled to begin on that date.

ChangeLog:

	[BZ #23758]
	* manual/time.texi (strftime): Document "%Ey".
	* time/strftime_l.c (__strftime_internal): Set the default width
	padding with zero of "%Ey" to 2.
---
 NEWS              | 8 ++++++++
 manual/time.texi  | 7 +++++++
 time/strftime_l.c | 2 +-
 3 files changed, 16 insertions(+), 1 deletion(-)

diff --git a/NEWS b/NEWS
index d863d80f3a..aec18367a4 100644
--- a/NEWS
+++ b/NEWS
@@ -52,6 +52,14 @@ Major new features:
     - C-SKY ABIV2 soft-float little-endian
     - C-SKY ABIV2 hard-float little-endian
 
+* strftime's default formatting of a locale's alternative year (%Ey)
+  has been changed to zero-pad the year to a minimum of two digits,
+  like "%y".  This improves the display of Japanese era years during
+  the first nine years of a new era, and is expected to be harmless
+  for all other locales (only Japanese locales regularly have
+  alternative year numbers less than 10).  Zero-padding can be
+  overridden with the '_' or '-' flags (which are GNU extensions).
+
 Deprecated and removed features, and other changes affecting compatibility:
 
 * The glibc.tune tunable namespace has been renamed to glibc.cpu and the
diff --git a/manual/time.texi b/manual/time.texi
index fd7781c531..03a8a0e10f 100644
--- a/manual/time.texi
+++ b/manual/time.texi
@@ -1568,6 +1568,13 @@ The preferred time of day representation for the current locale.
 The year without a century as a decimal number (range @code{00} through
 @code{99}).  This is equivalent to the year modulo 100.
 
+If the @code{E} modifier is specified (@code{%Ey}), instead produces
+the year number according to a locale-specific alternative calendar.
+Unlike @code{%y}, the number is @emph{not} reduced modulo 100.
+However, by default it is zero-padded to a minimum of two digits (this
+can be overridden by an explicit field width or by the @code{_} and
+@code{-} flags).
+
 @item %Y
 The year as a decimal number, using the Gregorian calendar.  Years
 before the year @code{1} are numbered @code{0}, @code{-1}, and so on.
diff --git a/time/strftime_l.c b/time/strftime_l.c
index 7ba4179de3..cbe08e7afb 100644
--- a/time/strftime_l.c
+++ b/time/strftime_l.c
@@ -1294,7 +1294,7 @@ __strftime_internal (CHAR_T *s, size_t maxsize, const CHAR_T *format,
 	      if (era)
 		{
 		  int delta = tp->tm_year - era->start_date[0];
-		  DO_NUMBER (1, (era->offset
+		  DO_NUMBER (2, (era->offset
 				 + delta * era->absolute_direction));
 		}
 #else
-- 
2.12.2

Re: [PATCH v9 1/2] strftime: Set the default width of "%Ey" to 2 [BZ #23758]

[Zack Weinberg]

On Mon, Jan 21, 2019 at 8:52 PM TAMUKI Shoichi <tamuki@linet.gr.jp> wrote:
> In Japanese locales, strftime's alternative year format (%Ey) produces
> a year numbered within a time period called an _era_.  A new era
> typically begins when a new emperor is enthroned.  The result of "%Ey"
> is therefore usually a one- or two-digit number.
...

I asked a professor at a local university who teaches both Japanese as
a second language, and Japanese literature, whether there were better
words to use in English for either the eras, or the calendar itself.
He said there were not, "era" is the usual term in English and the
calendar doesn't have a more formal name than "the traditional
Japanese calendar"/

Based on that, I think this patch is now good to go in.

Reviewed-by: Zack Weinberg <zackw@panix.com>

zw

Re: [PATCH v9 1/2] strftime: Set the default width of "%Ey" to 2 [BZ #23758]

[Rafal Luzynski]

23.01.2019 22:29 Zack Weinberg <zackw@panix.com> wrote:
> 
> [...]
> Based on that, I think this patch is now good to go in.
> 
> Reviewed-by: Zack Weinberg <zackw@panix.com>

Thank you.

Reviewed-by: Rafal Luzynski <digitalfreak@lingonborough.com>

Tamuki Shoichi, please add these two "Reviewed-by" lines to
your commit message and push this patch to master.

Regards,

Rafal

[PATCH v9 2/2] strftime: Pass the additional flags from "%EY" to "%Ey" [BZ #24096]

[TAMUKI Shoichi]

The full representation of the alternative calendar year (%EY)
typically includes an internal use of "%Ey".  As a GNU extension,
apply any flags on "%EY" (e.g. "%_EY", "%-EY") to the internal "%Ey",
allowing users of "%EY" to control how the year is padded.

ChangeLog:

	[BZ #24096]
	* manual/time.texi (strftime): Document "%EC" and "%EY".
	* time/Makefile (tests): Add tst-strftime2.
	(LOCALES): Add ja_JP.UTF-8, lo_LA.UTF-8, and th_TH.UTF-8.
	* time/strftime_l.c (__strftime_internal): Add argument yr_spec to
	override padding for "%Ey".
	If an optional flag ('_' or '-') is specified to "%EY", interpret the
	"%Ey" in the subformat as if decorated with that flag.
	* time/tst-strftime2.c: New file.
---
 NEWS                 |   4 ++
 manual/time.texi     |  11 +++++
 time/Makefile        |   5 +-
 time/strftime_l.c    |  18 ++++---
 time/tst-strftime2.c | 132 +++++++++++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 161 insertions(+), 9 deletions(-)
 create mode 100644 time/tst-strftime2.c

diff --git a/NEWS b/NEWS
index aec18367a4..76679f3393 100644
--- a/NEWS
+++ b/NEWS
@@ -60,6 +60,10 @@ Major new features:
   alternative year numbers less than 10).  Zero-padding can be
   overridden with the '_' or '-' flags (which are GNU extensions).
 
+* As a GNU extension, the '_' and '-' flags can now be applied to
+  "%EY" to control how the year number is formatted; they have the
+  same effect that they would on "%Ey".
+
 Deprecated and removed features, and other changes affecting compatibility:
 
 * The glibc.tune tunable namespace has been renamed to glibc.cpu and the
diff --git a/manual/time.texi b/manual/time.texi
index 03a8a0e10f..6d398bfe1f 100644
--- a/manual/time.texi
+++ b/manual/time.texi
@@ -1393,6 +1393,10 @@ The preferred calendar time representation for the current locale.
 The century of the year.  This is equivalent to the greatest integer not
 greater than the year divided by 100.
 
+If the @code{E} modifier is specified (@code{%EC}), instead produces
+the name of the period for the year (e.g.@: an era name) in the
+locale's alternative calendar.
+
 This format was first standardized by POSIX.2-1992 and by @w{ISO C99}.
 
 @item %d
@@ -1579,6 +1583,13 @@ can be overridden by an explicit field width or by the @code{_} and
 The year as a decimal number, using the Gregorian calendar.  Years
 before the year @code{1} are numbered @code{0}, @code{-1}, and so on.
 
+If the @code{E} modifier is specified (@code{%EY}), instead produces a
+complete representation of the year according to the locale's
+alternative calendar.  Generally this will be some combination of the
+information produced by @code{%EC} and @code{Ey}.  As a GNU extension,
+the formatting flags @code{_} or @code{-} may be used with this
+conversion specifier; they affect how the year number is printed.
+
 @item %z
 @w{RFC 822}/@w{ISO 8601:1988} style numeric time zone (e.g.,
 @code{-0600} or @code{+0100}), or nothing if no time zone is
diff --git a/time/Makefile b/time/Makefile
index d23ba2dee6..5c6304ece1 100644
--- a/time/Makefile
+++ b/time/Makefile
@@ -43,13 +43,14 @@ tests	:= test_time clocktest tst-posixtz tst-strptime tst_wcsftime \
 	   tst-getdate tst-mktime tst-mktime2 tst-ftime_l tst-strftime \
 	   tst-mktime3 tst-strptime2 bug-asctime bug-asctime_r bug-mktime1 \
 	   tst-strptime3 bug-getdate1 tst-strptime-whitespace tst-ftime \
-	   tst-tzname tst-y2039 bug-mktime4
+	   tst-tzname tst-y2039 bug-mktime4 tst-strftime2
 
 include ../Rules
 
 ifeq ($(run-built-tests),yes)
 LOCALES := de_DE.ISO-8859-1 en_US.ISO-8859-1 ja_JP.EUC-JP fr_FR.UTF-8 \
-	   es_ES.UTF-8 pl_PL.UTF-8 ru_RU.UTF-8
+	   es_ES.UTF-8 pl_PL.UTF-8 ru_RU.UTF-8 \
+	   ja_JP.UTF-8 lo_LA.UTF-8 th_TH.UTF-8
 include ../gen-locales.mk
 
 $(objpfx)tst-ftime_l.out: $(gen-locales)
diff --git a/time/strftime_l.c b/time/strftime_l.c
index cbe08e7afb..cb3b8d36f6 100644
--- a/time/strftime_l.c
+++ b/time/strftime_l.c
@@ -434,7 +434,7 @@ static CHAR_T const month_name[][10] =
 #endif
 
 static size_t __strftime_internal (CHAR_T *, size_t, const CHAR_T *,
-				   const struct tm *, bool *
+				   const struct tm *, int, bool *
 				   ut_argument_spec
 				   LOCALE_PARAM) __THROW;
 
@@ -457,7 +457,7 @@ my_strftime (CHAR_T *s, size_t maxsize, const CHAR_T *format,
   tp = &tmcopy;
 #endif
   bool tzset_called = false;
-  return __strftime_internal (s, maxsize, format, tp, &tzset_called
+  return __strftime_internal (s, maxsize, format, tp, 0, &tzset_called
 			      ut_argument LOCALE_ARG);
 }
 #ifdef _LIBC
@@ -466,7 +466,7 @@ libc_hidden_def (my_strftime)
 
 static size_t
 __strftime_internal (CHAR_T *s, size_t maxsize, const CHAR_T *format,
-		     const struct tm *tp, bool *tzset_called
+		     const struct tm *tp, int yr_spec, bool *tzset_called
 		     ut_argument_spec LOCALE_PARAM)
 {
 #if defined _LIBC && defined USE_IN_EXTENDED_LOCALE_MODEL
@@ -838,11 +838,11 @@ __strftime_internal (CHAR_T *s, size_t maxsize, const CHAR_T *format,
 	  {
 	    CHAR_T *old_start = p;
 	    size_t len = __strftime_internal (NULL, (size_t) -1, subfmt,
-					      tp, tzset_called ut_argument
-					      LOCALE_ARG);
+					      tp, yr_spec, tzset_called
+					      ut_argument LOCALE_ARG);
 	    add (len, __strftime_internal (p, maxsize - i, subfmt,
-					   tp, tzset_called ut_argument
-					   LOCALE_ARG));
+					   tp, yr_spec, tzset_called
+					   ut_argument LOCALE_ARG));
 
 	    if (to_uppcase)
 	      while (old_start < p)
@@ -1273,6 +1273,8 @@ __strftime_internal (CHAR_T *s, size_t maxsize, const CHAR_T *format,
 # else
 		  subfmt = era->era_format;
 # endif
+		  if (pad != 0)
+		    yr_spec = pad;
 		  goto subformat;
 		}
 #else
@@ -1294,6 +1296,8 @@ __strftime_internal (CHAR_T *s, size_t maxsize, const CHAR_T *format,
 	      if (era)
 		{
 		  int delta = tp->tm_year - era->start_date[0];
+		  if (yr_spec != 0)
+		    pad = yr_spec;
 		  DO_NUMBER (2, (era->offset
 				 + delta * era->absolute_direction));
 		}
diff --git a/time/tst-strftime2.c b/time/tst-strftime2.c
new file mode 100644
index 0000000000..57d2144c83
--- /dev/null
+++ b/time/tst-strftime2.c
@@ -0,0 +1,132 @@
+/* Verify the behavior of strftime on alternative representation for
+   year.
+
+   Copyright (C) 2019 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <http://www.gnu.org/licenses/>.  */
+
+#include <array_length.h>
+#include <locale.h>
+#include <time.h>
+#include <stdio.h>
+#include <string.h>
+
+static const char *locales[] = { "ja_JP.UTF-8", "lo_LA.UTF-8", "th_TH.UTF-8" };
+
+static const char *formats[] = { "%EY", "%_EY", "%-EY" };
+
+static const struct
+{
+  const int d, m, y;
+} dates[] =
+  {
+    { 1, 3, 88 },
+    { 7, 0, 89 },
+    { 8, 0, 89 },
+    { 1, 3, 90 },
+    { 1, 3, 97 },
+    { 1, 3, 98 }
+  };
+
+static char ref[3][3][6][100];
+
+static void
+mkreftable (void)
+{
+  int i, j, k;
+  char era[10];
+  static const int yrj[] = { 63, 64, 1, 2, 9, 10 };
+  static const int yrb[] = { 2531, 2532, 2532, 2533, 2540, 2541 };
+
+  for (i = 0; i < array_length (locales); i++)
+    for (j = 0; j < array_length (formats); j++)
+      for (k = 0; k < array_length (dates); k++)
+	{
+	  if (i == 0)
+	    {
+	      sprintf (era, "%s", (k < 2) ? "\xe6\x98\xad\xe5\x92\x8c"
+					  : "\xe5\xb9\xb3\xe6\x88\x90");
+	      if (yrj[k] == 1)
+		sprintf (ref[i][j][k], "%s\xe5\x85\x83\xe5\xb9\xb4", era);
+	      else
+		{
+		  if (j == 0)
+		    sprintf (ref[i][j][k], "%s%02d\xe5\xb9\xb4", era, yrj[k]);
+		  else if (j == 1)
+		    sprintf (ref[i][j][k], "%s%2d\xe5\xb9\xb4", era, yrj[k]);
+		  else
+		    sprintf (ref[i][j][k], "%s%d\xe5\xb9\xb4", era, yrj[k]);
+		}
+	    }
+	  else if (i == 1)
+	    {
+	      sprintf (era, "\xe0\xba\x9e\x2e\xe0\xba\xaa\x2e ");
+	      sprintf (ref[i][j][k], "%s%d", era, yrb[k]);
+	    }
+	  else
+	    {
+	      sprintf (era, "\xe0\xb8\x9e\x2e\xe0\xb8\xa8\x2e ");
+	      sprintf (ref[i][j][k], "%s%d", era, yrb[k]);
+	    }
+	}
+}
+
+static int
+do_test (void)
+{
+  int i, j, k, result = 0;
+  struct tm ttm;
+  char date[11], buf[100];
+  size_t r, e;
+
+  mkreftable ();
+  for (i = 0; i < array_length (locales); i++)
+    {
+      if (setlocale (LC_ALL, locales[i]) == NULL)
+	{
+	  printf ("locale %s does not exist, skipping...\n", locales[i]);
+	  continue;
+	}
+      printf ("[%s]\n", locales[i]);
+      for (j = 0; j < array_length (formats); j++)
+	{
+	  for (k = 0; k < array_length (dates); k++)
+	    {
+	      ttm.tm_mday = dates[k].d;
+	      ttm.tm_mon  = dates[k].m;
+	      ttm.tm_year = dates[k].y;
+	      strftime (date, sizeof (date), "%F", &ttm);
+	      r = strftime (buf, sizeof (buf), formats[j], &ttm);
+	      e = strlen (ref[i][j][k]);
+	      printf ("%s\t\"%s\"\t\"%s\"", date, formats[j], buf);
+	      if (strcmp (buf, ref[i][j][k]) != 0)
+		{
+		  printf ("\tshould be \"%s\"", ref[i][j][k]);
+		  if (r != e)
+		    printf ("\tgot: %zu, expected: %zu", r, e);
+		  result = 1;
+		}
+	      else
+		printf ("\tOK");
+	      putchar ('\n');
+	    }
+	  putchar ('\n');
+	}
+    }
+  return result;
+}
+
+#include <support/test-driver.c>
-- 
2.12.2

Re: [PATCH v9 2/2] strftime: Pass the additional flags from "%EY" to "%Ey" [BZ #24096]

[Rafal Luzynski]

22.01.2019 02:53 TAMUKI Shoichi <tamuki@linet.gr.jp> wrote:
> 
> The full representation of the alternative calendar year (%EY)
> typically includes an internal use of "%Ey".  As a GNU extension,
> apply any flags on "%EY" (e.g. "%_EY", "%-EY") to the internal "%Ey",
> allowing users of "%EY" to control how the year is padded.
> 
> ChangeLog:
> 
> 	[BZ #24096]
> 	* manual/time.texi (strftime): Document "%EC" and "%EY".
> 	* time/Makefile (tests): Add tst-strftime2.
> 	(LOCALES): Add ja_JP.UTF-8, lo_LA.UTF-8, and th_TH.UTF-8.
> 	* time/strftime_l.c (__strftime_internal): Add argument yr_spec to
> 	override padding for "%Ey".
> 	If an optional flag ('_' or '-') is specified to "%EY", interpret the
> 	"%Ey" in the subformat as if decorated with that flag.
> 	* time/tst-strftime2.c: New file.
> ---
>  NEWS                 |   4 ++
>  manual/time.texi     |  11 +++++
>  time/Makefile        |   5 +-
>  time/strftime_l.c    |  18 ++++---
>  time/tst-strftime2.c | 132
> +++++++++++++++++++++++++++++++++++++++++++++++++++
>  5 files changed, 161 insertions(+), 9 deletions(-)
>  create mode 100644 time/tst-strftime2.c

The code in this patch is OK for me.

Reviewed-by: Rafal Luzynski <digitalfreak@lingonborough.com>

Zack, can you please verify for the last time that the patch [1] is correct
and give your final OK and "Reviewed-by"?  Thank you.

Regards,

Rafal


[1] https://sourceware.org/ml/libc-alpha/2019-01/msg00553.html

Re: [PATCH v9 2/2] strftime: Pass the additional flags from "%EY" to "%Ey" [BZ #24096]

[Zack Weinberg]

On Wed, Jan 23, 2019 at 4:47 PM Rafal Luzynski
<digitalfreak@lingonborough.com> wrote:
> 22.01.2019 02:53 TAMUKI Shoichi <tamuki@linet.gr.jp> wrote:
> > The full representation of the alternative calendar year (%EY)
> > typically includes an internal use of "%Ey".  As a GNU extension,
> > apply any flags on "%EY" (e.g. "%_EY", "%-EY") to the internal "%Ey",
> > allowing users of "%EY" to control how the year is padded.
...
> The code in this patch is OK for me.
>
> Reviewed-by: Rafal Luzynski <digitalfreak@lingonborough.com>

Reviewed-by: Zack Weinberg <zackw@panix.com>

> Zack, can you please verify for the last time that the patch [1] is correct
> and give your final OK and "Reviewed-by"?  Thank you.

Thanks for the pointer, I didn't receive message [1] for some reason.
I read over the patch in the list archive and I confirm the
documentation is good.

zw

Re: [PATCH v9 0/2] strftime: Improve the width of alternative representation for year [BZ #23758][BZ #24096]

[Rafal Luzynski]

22.01.2019 02:44 TAMUKI Shoichi <tamuki@linet.gr.jp> wrote:
> 
> Hello,
> 
> This is the new set of patches.  I have split the patch into 2 parts.
> 
> Note, Patch 1 is same as v8.
> 
> Changes include:
> 
> Patch 1: strftime: Set the default width of "%Ey" to 2
> Patch 2: strftime: Pass the additional flags from "%EY" to "%Ey"

Both patches look good to me.  Especially the first one since it
is the same as the previous version which I already said was OK,
and since this patch actually changes only one line of the code
and other lines change just the documentation.

My suggestion: please wait a while to let the other maintainers
express their final opinion and push them to the repository if
nobody replies or if anybody asks you to push immediately.

I think that waiting 24 hours from now or maximum 48 hours is enough.

Please remember to update the ChangeLog when pushing and to close
the bug reports.  And then please port these changes to Gnulib.

Again thank you for working on this.

Regards,

Rafal

Re: [PATCH v9 0/2] strftime: Improve the width of alternative representation for year [BZ #23758][BZ #24096]

[TAMUKI Shoichi]

Hello all,

Late last night in Japan time, I have commited both patches and push
them to master.  Thank you Florian, Rafal, Paul, Siddhesh, Carlos,
Joseph, Andreas, Zack, and everyone else for your reviews,
documentation updates, assistance, etc.

From: Rafal Luzynski <digitalfreak@lingonborough.com>
Subject: Re: [PATCH v9 0/2] strftime: Improve the width of alternative representation for year [BZ #23758][BZ #24096]
Date: Wed, 23 Jan 2019 00:43:22 +0100 (CET)

> Please remember to update the ChangeLog when pushing and to close
> the bug reports.  And then please port these changes to Gnulib.

I set both bugs to be RESOLVED as FIXED in Bugzilla, but I could not
change the target milestone to 2.29 (Joseph did it).  Parhaps, it
seems I do not have the editbugs group access. [1]

[1] https://sourceware.org/glibc/wiki/Bugzilla%20Procedures

Can I get the ability to edit bugs?

Thank you again for your help.

Regards,
TAMUKI Shoichi

Re: [PATCH v9 0/2] strftime: Improve the width of alternative representation for year [BZ #23758][BZ #24096]

[Florian Weimer]

* TAMUKI Shoichi:

> I set both bugs to be RESOLVED as FIXED in Bugzilla, but I could not
> change the target milestone to 2.29 (Joseph did it).  Parhaps, it
> seems I do not have the editbugs group access. [1]
>
> [1] https://sourceware.org/glibc/wiki/Bugzilla%20Procedures
>
> Can I get the ability to edit bugs?

I have assigned editbugs privileges to you.

Thanks,
Florian