* Re: [libvirt] Fwd: libvirtd failing on MacOS in setgroups [not found] ` <20190821183405.kp3usu4m55tlgk5t@SPB-NB-133.local> @ 2019-08-30 16:39 ` Eric Blake 2019-08-31 1:33 ` Bruno Haible 0 siblings, 1 reply; 9+ messages in thread From: Eric Blake @ 2019-08-30 16:39 UTC (permalink / raw) To: Roman Bolshakov, Daniel P. Berrangé Cc: libvir-list, Gnulib bugs, Marcus Furlong [-- Attachment #1.1: Type: text/plain, Size: 1450 bytes --] [adding gnulib] On 8/21/19 1:34 PM, Roman Bolshakov wrote: >>>>> I get the following error when running libvirtd on MacOS as root: >>>>> >>>>> 2019-07-11 00:12:33.673+0000: 123145573953536: error : >>>>> qemuProcessQMPLaunch:8501 : internal error: Failed to start QEMU >>>>> binary /usr/local/bin/qemu-system-x86_64 for probing: libvirt: error >>>>> : cannot set supplemental groups: Invalid argument >>>> >> +++ b/src/util/virutil.c >> @@ -1043,6 +1043,7 @@ virSetUIDGID(uid_t uid, gid_t gid, gid_t *groups ATTRIBUTE_UNUSED, >> } >> >> # if HAVE_SETGROUPS >> + VIR_DEBUG("setgroups %d max %d", ngroups, NGROUPS_MAX); >> if (gid != (gid_t)-1 && setgroups(ngroups, groups) < 0) { >> virReportSystemError(errno, "%s", >> _("cannot set supplemental groups")); >> >> > > Yes, there's an overflow: > 2019-08-21 18:25:37.943+0000: 123145413914624: debug : virSetUIDGID:1046 : setgroups 23 max 16 > > Related samba ticket (it also has references to the python and dovecot > issues): > https://bugzilla.samba.org/show_bug.cgi?id=8773 I wonder if gnulib could provide a workaround setgroups() that overcomes this issue (it's better to maintain such a patch there, where it benefits multiple programs, rather than just in libvirt). -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [libvirt] Fwd: libvirtd failing on MacOS in setgroups 2019-08-30 16:39 ` [libvirt] Fwd: libvirtd failing on MacOS in setgroups Eric Blake @ 2019-08-31 1:33 ` Bruno Haible 2019-09-27 15:38 ` Marcus Furlong 0 siblings, 1 reply; 9+ messages in thread From: Bruno Haible @ 2019-08-31 1:33 UTC (permalink / raw) To: bug-gnulib Cc: libvir-list, Daniel P. Berrangé, Roman Bolshakov, Eric Blake, Marcus Furlong Hi Eric, > I wonder if gnulib could provide a workaround setgroups() that overcomes > this issue I don't see how a workaround could look like. The problem is not the value of NGROUPS_MAX in user-space, but the same value NGROUPS_MAX in the kernel. More precisely, in the Darwin kernel file bsd/kern/kern_prot.c there is a function 'setgroups1', that contains the common implementation of the setgroups() and initgroups() system call, and this function fails with EINVAL if the number of groups in the set is > NGROUPS. In the kernel sources, NGROUPS is defined as NGROUPS_MAX, and NGROUPS_MAX is defined as 16. So, the situation on macOS has not changed since this page was written: https://www.j3e.de/ngroups.html What kind of workaround are you imagining? That we override open(), access(), eaccess() to call setgroups() first, in an intelligent way? That would be quite gross. For what purpose is libvirt or QEMU using setgroups()? Bruno ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [libvirt] Fwd: libvirtd failing on MacOS in setgroups 2019-08-31 1:33 ` Bruno Haible @ 2019-09-27 15:38 ` Marcus Furlong 2019-09-28 11:36 ` Bruno Haible 0 siblings, 1 reply; 9+ messages in thread From: Marcus Furlong @ 2019-09-27 15:38 UTC (permalink / raw) To: Bruno Haible Cc: libvir-list, Eric Blake, Roman Bolshakov, bug-gnulib, Daniel P. Berrangé On Fri, 30 Aug 2019 at 21:33, Bruno Haible <bruno@clisp.org> wrote: > > Hi Eric, > > > I wonder if gnulib could provide a workaround setgroups() that overcomes > > this issue > > I don't see how a workaround could look like. The problem is not the value > of NGROUPS_MAX in user-space, but the same value NGROUPS_MAX in the kernel. > More precisely, in the Darwin kernel file bsd/kern/kern_prot.c there is a > function 'setgroups1', that contains the common implementation of the > setgroups() and initgroups() system call, and this function fails with EINVAL > if the number of groups in the set is > NGROUPS. In the kernel sources, > NGROUPS is defined as NGROUPS_MAX, and NGROUPS_MAX is defined as 16. > > So, the situation on macOS has not changed since this page was written: > https://www.j3e.de/ngroups.html > > What kind of workaround are you imagining? That we override open(), > access(), eaccess() to call setgroups() first, in an intelligent way? > That would be quite gross. > > For what purpose is libvirt or QEMU using setgroups()? FWIW I compiled libvirt without the setgroups code on Mac and it worked as expected. Not sure what the implications of that are though? Marcus. -- Marcus Furlong ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [libvirt] Fwd: libvirtd failing on MacOS in setgroups 2019-09-27 15:38 ` Marcus Furlong @ 2019-09-28 11:36 ` Bruno Haible 2019-09-30 9:02 ` Daniel P. Berrangé 0 siblings, 1 reply; 9+ messages in thread From: Bruno Haible @ 2019-09-28 11:36 UTC (permalink / raw) To: Marcus Furlong Cc: libvir-list, Eric Blake, Roman Bolshakov, bug-gnulib, Daniel P. Berrangé Marcus Furlong wrote: > FWIW I compiled libvirt without the setgroups code on Mac and it > worked as expected. Not sure what the implications of that are though? OK, then the fix would be to not use setgroups on Mac, and nothing to do in gnulib. Right? Bruno ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [libvirt] Fwd: libvirtd failing on MacOS in setgroups 2019-09-28 11:36 ` Bruno Haible @ 2019-09-30 9:02 ` Daniel P. Berrangé 2019-09-30 12:06 ` Bruno Haible 0 siblings, 1 reply; 9+ messages in thread From: Daniel P. Berrangé @ 2019-09-30 9:02 UTC (permalink / raw) To: Bruno Haible Cc: libvir-list, Eric Blake, Roman Bolshakov, bug-gnulib, Marcus Furlong On Sat, Sep 28, 2019 at 01:36:15PM +0200, Bruno Haible wrote: > Marcus Furlong wrote: > > FWIW I compiled libvirt without the setgroups code on Mac and it > > worked as expected. Not sure what the implications of that are though? > > OK, then the fix would be to not use setgroups on Mac, and nothing to do > in gnulib. Right? Not calling setgroups means the QEMU process doesn't run with any of the supplementary groups associated with its user account, so this is not really a working solution. It re-introduces the bug that the setgroups call was added to fix. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [libvirt] Fwd: libvirtd failing on MacOS in setgroups 2019-09-30 9:02 ` Daniel P. Berrangé @ 2019-09-30 12:06 ` Bruno Haible 2019-09-30 12:16 ` Daniel P. Berrangé 0 siblings, 1 reply; 9+ messages in thread From: Bruno Haible @ 2019-09-30 12:06 UTC (permalink / raw) To: Daniel P. Berrangé Cc: libvir-list, Eric Blake, Roman Bolshakov, bug-gnulib, Marcus Furlong Daniel P. Berrangé wrote: > > > FWIW I compiled libvirt without the setgroups code on Mac and it > > > worked as expected. Not sure what the implications of that are though? > > > > OK, then the fix would be to not use setgroups on Mac, and nothing to do > > in gnulib. Right? > > Not calling setgroups means the QEMU process doesn't run with any of > the supplementary groups associated with its user account, so this is > not really a working solution. It re-introduces the bug that the > setgroups call was added to fix. For what purpose is libvirt or QEMU using setgroups()? What goes wrong if setgroups() fails? The problem is that the Darwin kernel does not support setting more than NGROUPS_MAX (= 16) groups. So - What happens when you have a user account which is in more than 16 groups? What do other processes do in this sitation? - Is using the first 16 groups and ignoring the extra ones an acceptable solution? Bruno ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [libvirt] Fwd: libvirtd failing on MacOS in setgroups 2019-09-30 12:06 ` Bruno Haible @ 2019-09-30 12:16 ` Daniel P. Berrangé 2019-09-30 20:05 ` Bruno Haible 0 siblings, 1 reply; 9+ messages in thread From: Daniel P. Berrangé @ 2019-09-30 12:16 UTC (permalink / raw) To: Bruno Haible Cc: libvir-list, Eric Blake, Roman Bolshakov, bug-gnulib, Marcus Furlong On Mon, Sep 30, 2019 at 02:06:07PM +0200, Bruno Haible wrote: > Daniel P. Berrangé wrote: > > > > FWIW I compiled libvirt without the setgroups code on Mac and it > > > > worked as expected. Not sure what the implications of that are though? > > > > > > OK, then the fix would be to not use setgroups on Mac, and nothing to do > > > in gnulib. Right? > > > > Not calling setgroups means the QEMU process doesn't run with any of > > the supplementary groups associated with its user account, so this is > > not really a working solution. It re-introduces the bug that the > > setgroups call was added to fix. > > For what purpose is libvirt or QEMU using setgroups()? What goes wrong if > setgroups() fails? QEMU potentially needs access to files owned by a supplementary group. On Linux for example, /dev/kvm is often owned by 'kvm' group, but the 'qemu' user on Fedora has 'qemu' group as its primary group. So QEMU would be unable to open /dev/kvm without the setgroups call to set up supplementary groups. > The problem is that the Darwin kernel does not support setting more than > NGROUPS_MAX (= 16) groups. So > - What happens when you have a user account which is in more than 16 > groups? What do other processes do in this sitation? Samba appears to use initgroups on Darwin, while clamping to 16 groups only: https://github.com/samba-team/samba/blob/v4-11-stable/source3/smbd/sec_ctx.c#L248 > - Is using the first 16 groups and ignoring the extra ones an acceptable > solution? Certainly that's better than just ignoring groups entirely, as it will work for many more cases, even if not perfect. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [libvirt] Fwd: libvirtd failing on MacOS in setgroups 2019-09-30 12:16 ` Daniel P. Berrangé @ 2019-09-30 20:05 ` Bruno Haible 2019-10-15 16:07 ` Marcus Furlong 0 siblings, 1 reply; 9+ messages in thread From: Bruno Haible @ 2019-09-30 20:05 UTC (permalink / raw) To: Daniel P. Berrangé Cc: libvir-list, Eric Blake, Roman Bolshakov, bug-gnulib, Marcus Furlong Daniel P. Berrangé wrote: > > For what purpose is libvirt or QEMU using setgroups()? What goes wrong if > > setgroups() fails? > > QEMU potentially needs access to files owned by a supplementary group. > On Linux for example, /dev/kvm is often owned by 'kvm' group, but the > 'qemu' user on Fedora has 'qemu' group as its primary group. So QEMU > would be unable to open /dev/kvm without the setgroups call to set up > supplementary groups. Ah, it's libvirt which calls setgroups and qemu which needs the groups. Then my suggested workaround that consists of overriding setgroups() and open() won't work. > > - Is using the first 16 groups and ignoring the extra ones an acceptable > > solution? > > Certainly that's better than just ignoring groups entirely, as it will > work for many more cases, even if not perfect. Hmm. If the group of /dev/kvm comes at 17th group, it will still not work. I.e. it will be unreliable. Then, how about if libvirt collects the set of groups that qemu might need for accessing devices (surely less than 16), then fills up the remaining up to 16 slots with secondary groups? Admittedly it makes qemu less self-contained. But given that setgroups() works only for root on macOS [1] I see no better way. Bruno [1] https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man2/setgroups.2.html ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [libvirt] Fwd: libvirtd failing on MacOS in setgroups 2019-09-30 20:05 ` Bruno Haible @ 2019-10-15 16:07 ` Marcus Furlong 0 siblings, 0 replies; 9+ messages in thread From: Marcus Furlong @ 2019-10-15 16:07 UTC (permalink / raw) To: Bruno Haible Cc: libvir-list, bug-gnulib, Roman Bolshakov, Daniel P. Berrangé, Eric Blake On Mon, 30 Sep 2019 at 21:05, Bruno Haible <bruno@clisp.org> wrote: > > Daniel P. Berrangé wrote: > > > For what purpose is libvirt or QEMU using setgroups()? What goes wrong if > > > setgroups() fails? On macOS, as far as I can see, everything works as expected without it. So not sure if it's actually needed? > > QEMU potentially needs access to files owned by a supplementary group. > > On Linux for example, /dev/kvm is often owned by 'kvm' group, but the > > 'qemu' user on Fedora has 'qemu' group as its primary group. So QEMU > > would be unable to open /dev/kvm without the setgroups call to set up > > supplementary groups. > > Ah, it's libvirt which calls setgroups and qemu which needs the groups. > Then my suggested workaround that consists of overriding setgroups() and > open() won't work. > > > > - Is using the first 16 groups and ignoring the extra ones an acceptable > > > solution? > > > > Certainly that's better than just ignoring groups entirely, as it will > > work for many more cases, even if not perfect. > > Hmm. If the group of /dev/kvm comes at 17th group, it will still not work. > I.e. it will be unreliable. > > Then, how about if libvirt collects the set of groups that qemu might need > for accessing devices (surely less than 16), then fills up the remaining > up to 16 slots with secondary groups? Admittedly it makes qemu less > self-contained. But given that setgroups() works only for root on macOS [1] > I see no better way. Note that /dev/kvm is for linux and does not exist on macOS. Unless we identify specific devices on macOS that qemu requires access to, then something like the following might work? https://github.com/furlongm/libvirt/commit/01a1d3d0e37c7f81a04da2e9707ac1c39f4642b9 Seems to work correctly for me (virsh capabilities now returns the correct output, and VMs run). -- Marcus Furlong ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2019-10-15 16:08 UTC | newest] Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- [not found] <CAH4CTx4edbdM_ifM=aXchcXKxP7wqgrQ9GtypfKYf2JqGj3XTg@mail.gmail.com> [not found] ` <CAH4CTx45uyozD8C5SKv-U7NWA41PxER98LNXB4BdC39xL==4BA@mail.gmail.com> [not found] ` <20190821122314.GF29327@redhat.com> [not found] ` <CAH4CTx5OYHg+B+P-p+AJZoHAD89dCyRQLqiRveSPyZe+biTzAw@mail.gmail.com> [not found] ` <20190821165551.GI29327@redhat.com> [not found] ` <20190821183405.kp3usu4m55tlgk5t@SPB-NB-133.local> 2019-08-30 16:39 ` [libvirt] Fwd: libvirtd failing on MacOS in setgroups Eric Blake 2019-08-31 1:33 ` Bruno Haible 2019-09-27 15:38 ` Marcus Furlong 2019-09-28 11:36 ` Bruno Haible 2019-09-30 9:02 ` Daniel P. Berrangé 2019-09-30 12:06 ` Bruno Haible 2019-09-30 12:16 ` Daniel P. Berrangé 2019-09-30 20:05 ` Bruno Haible 2019-10-15 16:07 ` Marcus Furlong
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).