From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS22989 208.118.235.0/24 X-Spam-Status: No, score=-4.2 required=3.0 tests=AWL,BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by dcvr.yhbt.net (Postfix) with ESMTPS id 5AE661F405 for ; Sat, 29 Dec 2018 15:11:45 +0000 (UTC) Received: from localhost ([127.0.0.1]:38770 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gdGH9-0001MA-SW for normalperson@yhbt.net; Sat, 29 Dec 2018 10:11:43 -0500 Received: from eggs.gnu.org ([208.118.235.92]:53038) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gdGGx-0001Jz-De for bug-gnulib@gnu.org; Sat, 29 Dec 2018 10:11:36 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gdG9R-0005YX-KY for bug-gnulib@gnu.org; Sat, 29 Dec 2018 10:03:48 -0500 Received: from vmicros1.altlinux.org ([194.107.17.57]:47414) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gdG9R-0005Qi-3V for bug-gnulib@gnu.org; Sat, 29 Dec 2018 10:03:45 -0500 Received: from imap.altlinux.org (imap.altlinux.org [194.107.17.38]) by vmicros1.altlinux.org (Postfix) with ESMTP id 5CC8F72CC6C; Sat, 29 Dec 2018 18:03:42 +0300 (MSK) Received: by imap.altlinux.org (Postfix, from userid 620) id 4EEAB4A4AEB; Sat, 29 Dec 2018 18:03:42 +0300 (MSK) Received: from localhost (localhost [127.0.0.1]) by imap.altlinux.org (Postfix) with ESMTP id 396714A4A16; Sat, 29 Dec 2018 18:03:42 +0300 (MSK) Date: Sat, 29 Dec 2018 18:03:42 +0300 (MSK) From: Ivan Zakharyaschev To: Bruno Haible Subject: Re: [RFC PATCH] test-c-stack2.sh: skip if the platform sent SIGILL on an invalid address. In-Reply-To: <1553285.vUXQ5OdejE@omega> Message-ID: References: <20181215125047.2071-1-imz@altlinux.org> <1787335.ZHZPG2Za76@omega> <1553285.vUXQ5OdejE@omega> User-Agent: Alpine 2.20 (LFD 67 2015-01-07) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Received-From: 194.107.17.57 X-BeenThere: bug-gnulib@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Gnulib discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: bircoph@altlinux.org, bug-gnulib@gnu.org Errors-To: bug-gnulib-bounces+normalperson=yhbt.net@gnu.org Sender: "bug-gnulib" Hi Bruno, On Sat, 29 Dec 2018, Bruno Haible wrote: > > "system in development" is the one which suits > > Linux/E2k better. The port to E2K (MCST Elbrus general purpose hardware > > architecture) is quite mature, but not yet released publicly. > > Thanks for the info. Based on it, I found a couple of other pointers as well: > [1][2]. > [1] https://linux.slashdot.org/story/99/03/31/2324218/linus-will-move-to-moscow-to-work-with-elbrus [1] is fun. :) > > As for the SIGILL peculiarity, it has a reason in the Elbrus architecture. > > ... > > And it's not a segmentation fault. Meanwhile, I have found out that my explanations about it being the consequence of tagged memory (at least, in this specific case of test-c-stack.c) were largely incorrect. I'm sorry for that misleading information. I've studied the assembler code and found the other true reason in this specific case: these are faults "hidden" in an explicitly "speculative" computation which utltimately result in SIGILL. (The E2K ISA is reminiscent of IA64; this can help get the idea.) The specific kind of the fault is "forgotten", unfortunately. Bruno, this discovery makes your claims even more strong and relevant: this kind of fault is expected by all programs to be SIGSEGV normally, and they can't care whether the computation was done speculatively or not (i.e., with immediate effects). > I believe you should make it signal a SIGSEGV or SIGBUS, not SIGILL, for > the following reasons: > > * Look at the second table in > http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/signal.h.html. > It defines a couple of signal codes for SIGILL, SIGSEGV, and SIGBUS. > It implies that SIGILL means an invalid instruction (and "illegal operand" > means an invalid operand that is in the instruction stream). > Whereas SIGSEGV and SIGBUS mean a problem with an instruction in combination > with a memory address. Thanks for the explanation concerning "illegal operand"! This was a rebuttal more relevant given my first imagined explanation, but not the actual one. But anyway important to know. > * The main users of SIGSEGV and SIGBUS are catching stack overflow, garbage > collection, and similar (e.g. by use of GNU libsigsegv). The fact that > you observe an incompatibility between your Linux adaptation and > application programs that work fine across Linux/BSD/AIX/Solaris is a sure > indication that you will encounter similar incompatibilities along the lines, > until you fix that port, to produce SIGSEGV or SIGBUS instead of SIGILL. That's what I'm feeling now, too. It only remains a question concerning the hardware: whether it can save the type of the fault that happened in a speculative computation to give it back when the result of the speculative computation is actually needed. > This reminds the segmented architectures, such as the ones used by AIX > and Linux/ia64. In these OSes, SIGSEGV is produced when a memory address > is used that does not fit with the instruction. Thanks for the information about similar conditions (to what I wrote about tagged memory) in other OSes! Besides, in many aspects including the newly mentioned by me explicitly speculative instructions, E2K reminds IA64. And it'd be interesting to have a look how they treat faults coming from speculative computations in Linux/ia64 to get an idea whether it can be done in a manner with better conformance to POSIX. * * * Here are the actual facts about what happens on E2k (and little bit on IA64) with a set of minimal contrasting examples: Here are four example programs; the first two write to the memory, the latter two first read from the memory. (There is an amazing difference between the last two examples.) Probably, the demonstrated contrasts do not cover all conditions under which SIGILL can occur. $ cc -Wall -xc - && ./a.out; echo $? int main(int argc, char ** argv) { *(char*)0 = 175; return 0; } Segmentation fault 139 $ cc -Wall -xc - && ./a.out; echo $? int main(int argc, char ** argv) { if (0 < argc) *(char*)0 = 175; return 0; } Segmentation fault 139 $ cc -Wall -xc - && ./a.out; echo $? int main(int argc, char ** argv) { if (0 < argc) ++*(char*)0; return 0; } Illegal instruction 132 $ cc -Wall -xc - && ./a.out; echo $? int main(int argc, char ** argv) { ++*(char*)0; return 0; } Segmentation fault 139 $ cc --version lcc:1.23.12:Aug--6-2018:e2k-v4-linux gcc (GCC) 5.5.0 compatible $ This leads to a suspicion that not only the direction of the memory access matters (read or write), but also the speculative execution of the memory access instruction (in the third example) -- for the sake of optimization, something is done before the actual value of the condition is computed. (Otherwise, without a speculative computation, it's unclear how a redundant condition can affect anything.) The speculative instructions are written explicitly in E2K ISA (and this is also like this in IA64 AFAIR). Let's have a look at the assembler code of these four examples: $ cc -Wall -xc - -S -o a.s && cat a.s int main(int argc, char ** argv) { *(char*)0xbad = 175; return 0xbeef; } .file "-" .ignore ld_st_style .ignore strict_delay .text .global main .type main, #function .align 8 main: { setwd wsz = 0x4, nfx = 0x1, dbl = 0x0 return %ctpr3 adds,0 0x0, _f16s,_lts1lo 0xaf, %g16 addd,1 0x0, _f32s,_lts2 0xbeef, %r0 } { nop 4 stb,2 0x0, _f16s,_lts0lo 0xbad, %g16 } { ct %ctpr3 } .size main, .- main .weak elbrus_optimizing_compiler_v1.23.12_Aug__6_2018 elbrus_optimizing_compiler_v1.23.12_Aug__6_2018 = 0x0 The curly braces group instructions to be executed simultaneously in a single VLIW. "adds" is employed to put the constant 175 (0xaf) into register g16. "stb" STores a Byte into memory address 0xbad from register g16. $ cc -Wall -xc - -S -o b.s && cat b.s int main(int argc, char ** argv) { if (0 < argc) *(char*)0xbad = 175; return 0xbeef; } .file "-" .ignore ld_st_style .ignore strict_delay .text .global main .type main, #function .align 8 main: { setwd wsz = 0x4, nfx = 0x1, dbl = 0x0 return %ctpr3 adds,0,sm 0x0, _f16s,_lts1lo 0xaf, %g16 } { nop 1 cmplsb,0 0x0, %r0, %pred0 addd,1 0x0, _f32s,_lts0 0xbeef, %r0 } { nop 2 stb,2 0x0, _f16s,_lts0lo 0xbad, %g16 ? %pred0 } { ct %ctpr3 } .size main, .- main .weak elbrus_optimizing_compiler_v1.23.12_Aug__6_2018 elbrus_optimizing_compiler_v1.23.12_Aug__6_2018 = 0x0 "cmplsb" puts the comparison result between 0 and register r0 (the first function argument must reside here) in to the predicate register pred0. The "stb" instruction to store a value into memory is like in the previous example, but is executed "predicatively" (expressed by the tail "? %pred0"), i.e., only under the given condition. (The "adds,0,sm" instruction is employed to put the constant 175 (0xaf) into register g16 and has the "speculative" flag: ",sm"; but in this example, we don't perceive anything special because of this. This is the way to mark instructions, which are executed before it is actually known whether their execution is needed.) $ cc -Wall -xc - -S -o c.s && cat c.s int main(int argc, char ** argv) { if (0 < argc) ++*(char*)0xbad; return 0xbeef; } .file "-" .ignore ld_st_style .ignore strict_delay .text .global main .type main, #function .align 8 main: { setwd wsz = 0x4, nfx = 0x1, dbl = 0x0 return %ctpr3 ldb,2,sm 0x0, _f16s,_lts1lo 0xbad, %g16 } { nop 1 cmplsb,0 0x0, %r0, %pred0 addd,1 0x0, _f32s,_lts0 0xbeef, %r0 } { adds,0,sm %g16, 0x1, %g16 } { nop 1 stb,2 0x0, _f16s,_lts0lo 0xbad, %g16 ? %pred0 } { ct %ctpr3 } .size main, .- main .weak elbrus_optimizing_compiler_v1.23.12_Aug__6_2018 elbrus_optimizing_compiler_v1.23.12_Aug__6_2018 = 0x0 "ldb,2,sm" LoaDs a Byte from memory address 0xbad into register g16; the ",sm" flags mean that it is executed "speculatively", i.e., without immediate side effects (faults), because it's not known yet at this moment whether these actions will be needed. Then, in this register g16, there is a speculative computation "adds,0,sm" (to increase by one; it also has the "speculative" flags). Then, "stb" needs to store the result from g16 into the real memory (non-speculatively). It is remembered that g16 holds a result of a speculative computation, and hence ,if it was with a delayed fault, a real fault occurs at this moment (but not a SIGSEGV, rather a SIGILL -- a signal that the speculative computation is invalid). $ cc -Wall -xc - -S -o d.s && cat d.s int main(int argc, char ** argv) { ++*(char*)0xbad; return 0xbeef; } .file "-" .ignore ld_st_style .ignore strict_delay .text .global main .type main, #function .align 8 main: { nop 2 setwd wsz = 0x4, nfx = 0x1, dbl = 0x0 return %ctpr3 ldb,0 0x0, _f16s,_lts1lo 0xbad, %g16 addd,1 0x0, _f32s,_lts2 0xbeef, %r0 } { adds,0 %g16, 0x1, %g16 } { nop 1 stb,2 0x0, _f16s,_lts0lo 0xbad, %g16 } { ct %ctpr3 } .size main, .- main .weak elbrus_optimizing_compiler_v1.23.12_Aug__6_2018 elbrus_optimizing_compiler_v1.23.12_Aug__6_2018 = 0x0 Here we see same things as in the previous example, but without the comparison and without any speculative computations. The fault that occurs is a SIGSEGV (immediately, not delayed). Why is there a difference between the second and the third examples? Well, it happened so that nothing faulty is done in the speculative computation in the second example, and because of that, the execution comes to the memory write and its fault. In contrast, in the third example, the fault of the memory write can occur only after bringing the result of the speculative computation into reality and this already causes a fault. We can think of it as if register g16 was marked as invalid during the speculative computation, and this caused a SIGILL an an attempt to use it; however, the reason why it was marked invalid has been forgotten. It's interesting to test whether the type of the fault that happens in a speculative computation is really forgotten and any type of fault gives a SIGILL in case there is an attempt to use its result non-speculatively. It would mean that Linux/e2k can hardly conform to POSIX well, as Bruno said, because POSIX requires different signals for different cases and incompatibilities can't be forgiven on the reason of speculative computations in the CPU. Here is a test demonstrating that SIGFPE may hide beside SIGILL: $ cc -Wall -xc - && ./a.out; echo $? int main(int argc, char ** argv) { if (0 < argc) *(char*)0xbad = 175 / (argc - 1); return 0xbeef; } Illegal instruction 132 $ cc -Wall -xc - && ./a.out; echo $? int main(int argc, char ** argv) { *(char*)0xbad = 175 / (argc - 1); return 0xbeef; } Floating point exception 136 I've inspected the assembler code in these two examples, and can confirm that the division is actually done speculatively in the second program. * * * BTW, saving and forgetting the type of the original fault doesn't seem to be something expensive to implement (after some thought): when a register is marked as invalid, it shouldn't matter anymore what value it holds. So, the same register can be used to save the information about the type of the fault. * * * I wanted to see how Linux/ia64 handles these complications arising from speculative computations possibly causing a fault; and powered on such a machine, and had a look at the above examples with SIGILL on E2K: the third one, and the fifth one (speculative division by zero). The third example from above: imz@rx2620:~/test-speculative-SIGSEGV$ cc -Wall -O3 -xc - -S -o c.s && cat c.s int main(int argc, char ** argv) { if (0 < argc) ++*(char*)0xbad; return 0xbeef; } .file "" .pred.safe_across_calls p1-p5,p16-p63 .section .text.startup,"ax",@progbits .align 16 .align 64 .global main# .type main#, @function .proc main# main: .prologue .body .mmi cmp4.ge p6, p7 = 0, r32 addl r14 = 2989, r0 addl r8 = 48879, r0 ;; .mmi (p7) ld1 r15 = [r14] ;; (p7) adds r15 = 1, r15 nop 0 ;; .mib (p7) st1 [r14] = r15 nop 0 br.ret.sptk.many b0 .endp main# .ident "GCC: (Debian 4.6.3-14) 4.6.3" .section .note.GNU-stack,"",@progbits imz@rx2620:~/test-speculative-SIGSEGV$ cc -Wall -O3 c.s && ./a.out; echo $? Segmentation fault 139 The fifth example from above: imz@rx2620:~/test-speculative-SIGSEGV$ cc -Wall -O3 -xc - -S -o e.s && cat e.s int main(int argc, char ** argv) { if (0 < argc) *(char*)0xbad = 175 / (argc - 1); return 0xbeef; } .file "" .pred.safe_across_calls p1-p5,p16-p63 .global __divdi3# .section .text.startup,"ax",@progbits .align 16 .align 64 .global main# .type main#, @function .proc main# main: .prologue 12, 33 .mib .save ar.pfs, r34 alloc r34 = ar.pfs, 1, 3, 2, 0 .save rp, r33 mov r33 = b0 nop 0 .mmi adds r37 = -1, r32 mov r35 = r1 .body cmp4.ge p6, p7 = 0, r32 ;; .mii nop 0 sxt4 r37 = r37 nop 0 .mmb addl r36 = 175, r0 addl r8 = 48879, r0 (p6) br.cond.dpnt .L2 ;; .mib nop 0 nop 0 br.call.sptk.many b0 = __divdi3# ;; .mmi addl r14 = 2989, r0 nop 0 mov r1 = r35 ;; .mmi nop 0 st1 [r14] = r8 addl r8 = 48879, r0 .L2: .mii nop 0 mov ar.pfs = r34 nop 0 ;; .mib nop 0 mov b0 = r33 br.ret.sptk.many b0 .endp main# .ident "GCC: (Debian 4.6.3-14) 4.6.3" .section .note.GNU-stack,"",@progbits imz@rx2620:~/test-speculative-SIGSEGV$ cc -Wall -O3 e.s && ./a.out; echo $? Floating point exception 136 Notes on the assembler: the possible groupings into VLIWs are separated by double semicolons (";;"). Predicative execution of instructions is marked by a prefix with the corresponding predicate register in parentheses, like "(p7)" in the code above: .mmi (p7) ld1 r15 = [r14] ;; (p7) adds r15 = 1, r15 nop 0 ;; .mib (p7) st1 [r14] = r15 These are the "load", "add", and "store" instructions corresponding to: ++*(char*)0xbad All this shows that gcc-4.6 on IA-64 doesn't generate speculative computations for the same examples that had speculative computations on E2K. Unfortunately, this means that we couldn't compare the interesting bits of the behavior between Linux/e2k and Linux/ia64 quickly. Perhaps, editing the IA64 assembler code can give a desired example. -- Best regards, Ivan