Question regarding test suite build

Question regarding test suite build

[Lukasz Majewski]

[-- Attachment #1: Type: text/plain, Size: 2166 bytes --]

Dear Community,

I've thought about using yocto's ptest utility to check correctness of
glibc on for example ARMv7 or i386.

The idea would be simple - copy cross compiled tests - e.g
test-clock_settime to the target's rootfs when created in yocto.

Then, after booting up the board (in for example qemu) run those tests
with ptest-runner.

Advantages of this approach:

1. DO NOT use SSH and test-wrapper script to copy those binaries

2. Tests would be cross compiled on powerfull HOST machine

3. Run tests on target device - can be qemu or custom board (which
would help a lot with validation)


The problem I'm facing now is that make check does everything at once (
it cross compiles tests, executes them and prints results).
I would probably need to add new make rule (like make tst or such)

The first step would be just cross compile tests. The
'host-built-program-cmd' from Makeconfig seems to be responsible for it.


Questions:
----------

1. I'm adjusting Rules and then execute:
rm ./time/tst-clock_settime*
make V=1 --debug=b PARALLELMFLAGS="-j8" subdirs=time check

but the changes from Rules or Makeconfig are not applied (at least not
in logs).

For example I've added to Rules (line 297)
$(objpfx)%.out:·%.input·$(objpfx)%
	@echo·FOO$
	@echo·$(make-test-out)·>·$@·<·$(word·1,$^);

to see how tests are built, but no output is visible.


2. What is the purpose of 'all-testsuite' variable in Makerules (line
1255)?

Is a separate (helper) library (libtestsuite) built to have all eligible
tests in it?

3. It looks like some env variables GCONV_PATH and LOCPATH are set when
tests are built (Makeconfig line 745) in run-program-env.

I'm wondering how I could avoid setting it on target (store in bash.rc
when image is created?) and just call the test itself (like
tst-clock_settime)?


Thanks in advance for help.


Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: lukma@denx.de

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: Question regarding test suite build

[Joseph Myers]

On Wed, 21 Jul 2021, Lukasz Majewski wrote:

> The problem I'm facing now is that make check does everything at once (
> it cross compiles tests, executes them and prints results).
> I would probably need to add new make rule (like make tst or such)

You can build tests without running them simply by running "make check" 
for a cross-compiled glibc without test-wrapper set.  (I've done a variant 
of that - "make math/tests" with some -j option to build the math/ tests 
in parallel, then "make regen-ulps" with test-wrapper set to run those 
tests needed for ulps regeneration.)

Many tests depend on aspects of the test environment and aren't designed 
to be installed and run without a glibc source and build tree present.

-- 
Joseph S. Myers
joseph@codesourcery.com