* [PATCH] strbuf: support long paths w/o read rights in strbuf_getcwd() on FreeBSD @ 2017-03-26 13:43 René Scharfe 2017-03-27 0:40 ` Junio C Hamano 2017-03-28 21:15 ` Christian Couder 0 siblings, 2 replies; 10+ messages in thread From: René Scharfe @ 2017-03-26 13:43 UTC (permalink / raw) To: Git List; +Cc: Zenobiusz Kunegunda, Junio C Hamano, Jeff King FreeBSD implements getcwd(3) as a syscall, but falls back to a version based on readdir(3) if it fails for some reason. The latter requires permissions to read and execute path components, while the former does not. That means that if our buffer is too small and we're missing rights we could get EACCES, but we may succeed with a bigger buffer. Keep retrying if getcwd(3) indicates lack of permissions until our buffer can fit PATH_MAX bytes, as that's the maximum supported by the syscall on FreeBSD anyway. This way we do what we can to be able to benefit from the syscall, but we also won't loop forever if there is a real permission issue. This fixes a regression introduced with 7333ed17 (setup: convert setup_git_directory_gently_1 et al. to strbuf, 2014-07-28) for paths longer than 127 bytes with components that miss read or execute permissions (e.g. 0711 on /home for privacy reasons); we used a fixed PATH_MAX-sized buffer before. Reported-by: Zenobiusz Kunegunda <zenobiusz.kunegunda@interia.pl> Signed-off-by: Rene Scharfe <l.s.r@web.de> --- strbuf.c | 11 +++++++++++ t/t0001-init.sh | 14 ++++++++++++++ 2 files changed, 25 insertions(+) diff --git a/strbuf.c b/strbuf.c index ace58e7367..00457940cf 100644 --- a/strbuf.c +++ b/strbuf.c @@ -449,6 +449,17 @@ int strbuf_getcwd(struct strbuf *sb) strbuf_setlen(sb, strlen(sb->buf)); return 0; } + + /* + * If getcwd(3) is implemented as a syscall that falls + * back to a regular lookup using readdir(3) etc. then + * we may be able to avoid EACCES by providing enough + * space to the syscall as it's not necessarily bound + * to the same restrictions as the fallback. + */ + if (errno == EACCES && guessed_len < PATH_MAX) + continue; + if (errno != ERANGE) break; } diff --git a/t/t0001-init.sh b/t/t0001-init.sh index e424de5363..5f81fbe07c 100755 --- a/t/t0001-init.sh +++ b/t/t0001-init.sh @@ -315,6 +315,20 @@ test_expect_success 'init with separate gitdir' ' test_path_is_dir realgitdir/refs ' +test_expect_success 'init in long base path' ' + # exceed initial buffer size of strbuf_getcwd() + component=123456789abcdef && + test_when_finished "chmod 0700 $component; rm -rf $component" && + p31=$component/$component && + p127=$p31/$p31/$p31/$p31 && + mkdir -p $p127 && + chmod 0111 $component && + ( + cd $p127 && + git init newdir + ) +' + test_expect_success 're-init on .git file' ' ( cd newdir && git init ) ' -- 2.12.2 ^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH] strbuf: support long paths w/o read rights in strbuf_getcwd() on FreeBSD 2017-03-26 13:43 [PATCH] strbuf: support long paths w/o read rights in strbuf_getcwd() on FreeBSD René Scharfe @ 2017-03-27 0:40 ` Junio C Hamano 2017-03-27 5:55 ` Zenobiusz Kunegunda 2017-03-28 21:15 ` Christian Couder 1 sibling, 1 reply; 10+ messages in thread From: Junio C Hamano @ 2017-03-27 0:40 UTC (permalink / raw) To: René Scharfe; +Cc: Git List, Zenobiusz Kunegunda, Jeff King René Scharfe <l.s.r@web.de> writes: > FreeBSD implements getcwd(3) as a syscall, but falls back to a version > based on readdir(3) if it fails for some reason. The latter requires > permissions to read and execute path components, while the former does > not. That means that if our buffer is too small and we're missing > rights we could get EACCES, but we may succeed with a bigger buffer. WOW. Just WOW. Looking at the debugging exchange from the sideline, I didn't expect this end result. > Keep retrying if getcwd(3) indicates lack of permissions until our > buffer can fit PATH_MAX bytes, as that's the maximum supported by the > syscall on FreeBSD anyway. This way we do what we can to be able to > benefit from the syscall, but we also won't loop forever if there is a > real permission issue. > > This fixes a regression introduced with 7333ed17 (setup: convert > setup_git_directory_gently_1 et al. to strbuf, 2014-07-28) for paths > longer than 127 bytes with components that miss read or execute > permissions (e.g. 0711 on /home for privacy reasons); we used a fixed > PATH_MAX-sized buffer before. > > Reported-by: Zenobiusz Kunegunda <zenobiusz.kunegunda@interia.pl> > Signed-off-by: Rene Scharfe <l.s.r@web.de> > --- Nicely analysed and fixed (or is the right word "worked around"?) Thanks, will queue. ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] strbuf: support long paths w/o read rights in strbuf_getcwd() on FreeBSD 2017-03-27 0:40 ` Junio C Hamano @ 2017-03-27 5:55 ` Zenobiusz Kunegunda 2017-03-27 18:40 ` Junio C Hamano 0 siblings, 1 reply; 10+ messages in thread From: Zenobiusz Kunegunda @ 2017-03-27 5:55 UTC (permalink / raw) To: Junio C Hamano, René Scharfe; +Cc: Git List, Jeff King Od: "Junio C Hamano" <gitster@pobox.com> Do: "René Scharfe" <l.s.r@web.de>; Wysłane: 2:40 Poniedziałek 2017-03-27 Temat: Re: [PATCH] strbuf: support long paths w/o read rights in strbuf_getcwd() on FreeBSD > > Nicely analysed and fixed (or is the right word "worked around"?) > > Thanks, will queue. > Is this patch going to be included in next git version ( or sooner ) by any chance? Thank you, everyone, for your attention to the problem. ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] strbuf: support long paths w/o read rights in strbuf_getcwd() on FreeBSD 2017-03-27 5:55 ` Zenobiusz Kunegunda @ 2017-03-27 18:40 ` Junio C Hamano 0 siblings, 0 replies; 10+ messages in thread From: Junio C Hamano @ 2017-03-27 18:40 UTC (permalink / raw) To: Zenobiusz Kunegunda; +Cc: René Scharfe, Git List, Jeff King Zenobiusz Kunegunda <zenobiusz.kunegunda@interia.pl> writes: > Od: "Junio C Hamano" <gitster@pobox.com> > Do: "René Scharfe" <l.s.r@web.de>; > Wysłane: 2:40 Poniedziałek 2017-03-27 > Temat: Re: [PATCH] strbuf: support long paths w/o read rights in strbuf_getcwd() on FreeBSD > >> >> Nicely analysed and fixed (or is the right word "worked around"?) >> >> Thanks, will queue. > > Is this patch going to be included in next git version ( or sooner ) by any chance? Absolutely. Thanks for your initial report and sticking with us during the session to identify the root cause that led to this solution. Again, René, thanks for your superb analysis and solution. ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] strbuf: support long paths w/o read rights in strbuf_getcwd() on FreeBSD 2017-03-26 13:43 [PATCH] strbuf: support long paths w/o read rights in strbuf_getcwd() on FreeBSD René Scharfe 2017-03-27 0:40 ` Junio C Hamano @ 2017-03-28 21:15 ` Christian Couder 2017-03-28 21:24 ` Stefan Beller 2017-03-28 21:49 ` Jeff King 1 sibling, 2 replies; 10+ messages in thread From: Christian Couder @ 2017-03-28 21:15 UTC (permalink / raw) To: René Scharfe Cc: Git List, Zenobiusz Kunegunda, Junio C Hamano, Jeff King On Sun, Mar 26, 2017 at 3:43 PM, René Scharfe <l.s.r@web.de> wrote: > FreeBSD implements getcwd(3) as a syscall, but falls back to a version > based on readdir(3) if it fails for some reason. The latter requires > permissions to read and execute path components, while the former does > not. That means that if our buffer is too small and we're missing > rights we could get EACCES, but we may succeed with a bigger buffer. > > Keep retrying if getcwd(3) indicates lack of permissions until our > buffer can fit PATH_MAX bytes, as that's the maximum supported by the > syscall on FreeBSD anyway. This way we do what we can to be able to > benefit from the syscall, but we also won't loop forever if there is a > real permission issue. Sorry to be late and maybe I missed something obvious, but the above and the patch seem complex to me compared with something like: diff --git a/strbuf.c b/strbuf.c index ace58e7367..25eadcbedc 100644 --- a/strbuf.c +++ b/strbuf.c @@ -441,7 +441,7 @@ int strbuf_readlink(struct strbuf *sb, const char *path, size_t hint) int strbuf_getcwd(struct strbuf *sb) { size_t oldalloc = sb->alloc; - size_t guessed_len = 128; + size_t guessed_len = PATH_MAX > 128 ? PATH_MAX : 128; for (;; guessed_len *= 2) { strbuf_grow(sb, guessed_len); ^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH] strbuf: support long paths w/o read rights in strbuf_getcwd() on FreeBSD 2017-03-28 21:15 ` Christian Couder @ 2017-03-28 21:24 ` Stefan Beller 2017-03-28 21:47 ` Christian Couder 2017-03-28 21:49 ` Jeff King 1 sibling, 1 reply; 10+ messages in thread From: Stefan Beller @ 2017-03-28 21:24 UTC (permalink / raw) To: Christian Couder Cc: René Scharfe, Git List, Zenobiusz Kunegunda, Junio C Hamano, Jeff King On Tue, Mar 28, 2017 at 2:15 PM, Christian Couder <christian.couder@gmail.com> wrote: > On Sun, Mar 26, 2017 at 3:43 PM, René Scharfe <l.s.r@web.de> wrote: >> FreeBSD implements getcwd(3) as a syscall, but falls back to a version >> based on readdir(3) if it fails for some reason. The latter requires >> permissions to read and execute path components, while the former does >> not. That means that if our buffer is too small and we're missing >> rights we could get EACCES, but we may succeed with a bigger buffer. >> >> Keep retrying if getcwd(3) indicates lack of permissions until our >> buffer can fit PATH_MAX bytes, as that's the maximum supported by the >> syscall on FreeBSD anyway. This way we do what we can to be able to >> benefit from the syscall, but we also won't loop forever if there is a >> real permission issue. > > Sorry to be late and maybe I missed something obvious, but the above > and the patch seem complex to me compared with something like: > > diff --git a/strbuf.c b/strbuf.c > index ace58e7367..25eadcbedc 100644 > --- a/strbuf.c > +++ b/strbuf.c > @@ -441,7 +441,7 @@ int strbuf_readlink(struct strbuf *sb, const char > *path, size_t hint) > int strbuf_getcwd(struct strbuf *sb) > { > size_t oldalloc = sb->alloc; > - size_t guessed_len = 128; > + size_t guessed_len = PATH_MAX > 128 ? PATH_MAX : 128; > > for (;; guessed_len *= 2) { > strbuf_grow(sb, guessed_len); From f22a76e911 (strbuf: add strbuf_getcwd(), 2014-07-28): Because it doesn't use a fixed-size buffer it supports arbitrarily long paths, provided the platform's getcwd() does as well. At least on Linux and FreeBSD it handles paths longer than PATH_MAX just fine. So with your patch, we'd still see the original issue for paths > PATH_MAX IIUC. Thanks, Stefan ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] strbuf: support long paths w/o read rights in strbuf_getcwd() on FreeBSD 2017-03-28 21:24 ` Stefan Beller @ 2017-03-28 21:47 ` Christian Couder 0 siblings, 0 replies; 10+ messages in thread From: Christian Couder @ 2017-03-28 21:47 UTC (permalink / raw) To: Stefan Beller Cc: René Scharfe, Git List, Zenobiusz Kunegunda, Junio C Hamano, Jeff King On Tue, Mar 28, 2017 at 11:24 PM, Stefan Beller <sbeller@google.com> wrote: > On Tue, Mar 28, 2017 at 2:15 PM, Christian Couder > <christian.couder@gmail.com> wrote: >> On Sun, Mar 26, 2017 at 3:43 PM, René Scharfe <l.s.r@web.de> wrote: >>> FreeBSD implements getcwd(3) as a syscall, but falls back to a version >>> based on readdir(3) if it fails for some reason. The latter requires >>> permissions to read and execute path components, while the former does >>> not. That means that if our buffer is too small and we're missing >>> rights we could get EACCES, but we may succeed with a bigger buffer. >>> >>> Keep retrying if getcwd(3) indicates lack of permissions until our >>> buffer can fit PATH_MAX bytes, as that's the maximum supported by the >>> syscall on FreeBSD anyway. This way we do what we can to be able to >>> benefit from the syscall, but we also won't loop forever if there is a >>> real permission issue. >> >> Sorry to be late and maybe I missed something obvious, but the above >> and the patch seem complex to me compared with something like: >> >> diff --git a/strbuf.c b/strbuf.c >> index ace58e7367..25eadcbedc 100644 >> --- a/strbuf.c >> +++ b/strbuf.c >> @@ -441,7 +441,7 @@ int strbuf_readlink(struct strbuf *sb, const char >> *path, size_t hint) >> int strbuf_getcwd(struct strbuf *sb) >> { >> size_t oldalloc = sb->alloc; >> - size_t guessed_len = 128; >> + size_t guessed_len = PATH_MAX > 128 ? PATH_MAX : 128; >> >> for (;; guessed_len *= 2) { >> strbuf_grow(sb, guessed_len); > > From f22a76e911 (strbuf: add strbuf_getcwd(), 2014-07-28): > Because it doesn't use a fixed-size buffer it supports > arbitrarily long paths, provided the platform's getcwd() does as well. > At least on Linux and FreeBSD it handles paths longer than PATH_MAX > just fine. Well René's patch says: >>> Keep retrying if getcwd(3) indicates lack of permissions until our >>> buffer can fit PATH_MAX bytes, as that's the maximum supported by the >>> syscall on FreeBSD anyway. So it says that FreeBSD syscall doesn't handle paths longer than PATH_MAX. > So with your patch, we'd still see the original issue for paths > PATH_MAX > IIUC. Also, René's patch just adds: + if (errno == EACCES && guessed_len < PATH_MAX) + continue; so if the length of the path is > PATH_MAX, then guessed_len will have to be > PATH_MAX and then René's patch will do nothing. ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] strbuf: support long paths w/o read rights in strbuf_getcwd() on FreeBSD 2017-03-28 21:15 ` Christian Couder 2017-03-28 21:24 ` Stefan Beller @ 2017-03-28 21:49 ` Jeff King 2017-03-29 4:54 ` Christian Couder 1 sibling, 1 reply; 10+ messages in thread From: Jeff King @ 2017-03-28 21:49 UTC (permalink / raw) To: Christian Couder Cc: René Scharfe, Git List, Zenobiusz Kunegunda, Junio C Hamano On Tue, Mar 28, 2017 at 11:15:12PM +0200, Christian Couder wrote: > On Sun, Mar 26, 2017 at 3:43 PM, René Scharfe <l.s.r@web.de> wrote: > > FreeBSD implements getcwd(3) as a syscall, but falls back to a version > > based on readdir(3) if it fails for some reason. The latter requires > > permissions to read and execute path components, while the former does > > not. That means that if our buffer is too small and we're missing > > rights we could get EACCES, but we may succeed with a bigger buffer. > > > > Keep retrying if getcwd(3) indicates lack of permissions until our > > buffer can fit PATH_MAX bytes, as that's the maximum supported by the > > syscall on FreeBSD anyway. This way we do what we can to be able to > > benefit from the syscall, but we also won't loop forever if there is a > > real permission issue. > > Sorry to be late and maybe I missed something obvious, but the above > and the patch seem complex to me compared with something like: > > diff --git a/strbuf.c b/strbuf.c > index ace58e7367..25eadcbedc 100644 > --- a/strbuf.c > +++ b/strbuf.c > @@ -441,7 +441,7 @@ int strbuf_readlink(struct strbuf *sb, const char > *path, size_t hint) > int strbuf_getcwd(struct strbuf *sb) > { > size_t oldalloc = sb->alloc; > - size_t guessed_len = 128; > + size_t guessed_len = PATH_MAX > 128 ? PATH_MAX : 128; > > for (;; guessed_len *= 2) { > strbuf_grow(sb, guessed_len); I think the main reason is just that we do not have to pay the price to allocate PATH_MAX-sized buffers when they are rarely used. I doubt it matters all that much in practice, though. -Peff ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] strbuf: support long paths w/o read rights in strbuf_getcwd() on FreeBSD 2017-03-28 21:49 ` Jeff King @ 2017-03-29 4:54 ` Christian Couder 2017-03-30 18:01 ` René Scharfe 0 siblings, 1 reply; 10+ messages in thread From: Christian Couder @ 2017-03-29 4:54 UTC (permalink / raw) To: Jeff King Cc: René Scharfe, Git List, Zenobiusz Kunegunda, Junio C Hamano On Tue, Mar 28, 2017 at 11:49 PM, Jeff King <peff@peff.net> wrote: > On Tue, Mar 28, 2017 at 11:15:12PM +0200, Christian Couder wrote: > >> On Sun, Mar 26, 2017 at 3:43 PM, René Scharfe <l.s.r@web.de> wrote: >> > FreeBSD implements getcwd(3) as a syscall, but falls back to a version >> > based on readdir(3) if it fails for some reason. The latter requires >> > permissions to read and execute path components, while the former does >> > not. That means that if our buffer is too small and we're missing >> > rights we could get EACCES, but we may succeed with a bigger buffer. >> > >> > Keep retrying if getcwd(3) indicates lack of permissions until our >> > buffer can fit PATH_MAX bytes, as that's the maximum supported by the >> > syscall on FreeBSD anyway. This way we do what we can to be able to >> > benefit from the syscall, but we also won't loop forever if there is a >> > real permission issue. >> >> Sorry to be late and maybe I missed something obvious, but the above >> and the patch seem complex to me compared with something like: >> >> diff --git a/strbuf.c b/strbuf.c >> index ace58e7367..25eadcbedc 100644 >> --- a/strbuf.c >> +++ b/strbuf.c >> @@ -441,7 +441,7 @@ int strbuf_readlink(struct strbuf *sb, const char >> *path, size_t hint) >> int strbuf_getcwd(struct strbuf *sb) >> { >> size_t oldalloc = sb->alloc; >> - size_t guessed_len = 128; >> + size_t guessed_len = PATH_MAX > 128 ? PATH_MAX : 128; >> >> for (;; guessed_len *= 2) { >> strbuf_grow(sb, guessed_len); > > I think the main reason is just that we do not have to pay the price to > allocate PATH_MAX-sized buffers when they are rarely used. > > I doubt it matters all that much in practice, though. Yeah, I can understand that. But I also wonder if René's patch relies on PATH_MAX being a multiple of 128 on FreeBSD and what would happen for a path between 129 and PATH_MAX if PATH_MAX was something like 254. ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] strbuf: support long paths w/o read rights in strbuf_getcwd() on FreeBSD 2017-03-29 4:54 ` Christian Couder @ 2017-03-30 18:01 ` René Scharfe 0 siblings, 0 replies; 10+ messages in thread From: René Scharfe @ 2017-03-30 18:01 UTC (permalink / raw) To: Christian Couder, Jeff King; +Cc: Git List, Zenobiusz Kunegunda, Junio C Hamano Am 29.03.2017 um 06:54 schrieb Christian Couder: > On Tue, Mar 28, 2017 at 11:49 PM, Jeff King <peff@peff.net> wrote: >> On Tue, Mar 28, 2017 at 11:15:12PM +0200, Christian Couder wrote: >> >>> On Sun, Mar 26, 2017 at 3:43 PM, René Scharfe <l.s.r@web.de> wrote: >>>> FreeBSD implements getcwd(3) as a syscall, but falls back to a version >>>> based on readdir(3) if it fails for some reason. The latter requires >>>> permissions to read and execute path components, while the former does >>>> not. That means that if our buffer is too small and we're missing >>>> rights we could get EACCES, but we may succeed with a bigger buffer. >>>> >>>> Keep retrying if getcwd(3) indicates lack of permissions until our >>>> buffer can fit PATH_MAX bytes, as that's the maximum supported by the >>>> syscall on FreeBSD anyway. This way we do what we can to be able to >>>> benefit from the syscall, but we also won't loop forever if there is a >>>> real permission issue. >>> >>> Sorry to be late and maybe I missed something obvious, but the above >>> and the patch seem complex to me compared with something like: >>> >>> diff --git a/strbuf.c b/strbuf.c >>> index ace58e7367..25eadcbedc 100644 >>> --- a/strbuf.c >>> +++ b/strbuf.c >>> @@ -441,7 +441,7 @@ int strbuf_readlink(struct strbuf *sb, const char >>> *path, size_t hint) >>> int strbuf_getcwd(struct strbuf *sb) >>> { >>> size_t oldalloc = sb->alloc; >>> - size_t guessed_len = 128; >>> + size_t guessed_len = PATH_MAX > 128 ? PATH_MAX : 128; >>> >>> for (;; guessed_len *= 2) { >>> strbuf_grow(sb, guessed_len); >> >> I think the main reason is just that we do not have to pay the price to >> allocate PATH_MAX-sized buffers when they are rarely used. >> >> I doubt it matters all that much in practice, though. > > Yeah, I can understand that. Right, using PATH_MAX as initial size (no ?: operator needed) is the easiest fix. And yes, I wanted to avoid wasting memory just to fix an odd special case. Given that the function isn't called very often the impact is probably not that high either way, but increasing memory usage by 3200% on Linux just didn't sit right with me -- even though 4KB isn't much in absolute terms, of course. > But I also wonder if René's patch relies on PATH_MAX being a multiple > of 128 on FreeBSD and what would happen for a path between 129 and > PATH_MAX if PATH_MAX was something like 254. We can use any value bigger than zero to initialize guessed_len. getcwd() won't have any problems fitting e.g. a path with a length of 130 bytes into a buffer of 256 bytes in the second round of the loop. René ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2017-03-30 18:02 UTC | newest] Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2017-03-26 13:43 [PATCH] strbuf: support long paths w/o read rights in strbuf_getcwd() on FreeBSD René Scharfe 2017-03-27 0:40 ` Junio C Hamano 2017-03-27 5:55 ` Zenobiusz Kunegunda 2017-03-27 18:40 ` Junio C Hamano 2017-03-28 21:15 ` Christian Couder 2017-03-28 21:24 ` Stefan Beller 2017-03-28 21:47 ` Christian Couder 2017-03-28 21:49 ` Jeff King 2017-03-29 4:54 ` Christian Couder 2017-03-30 18:01 ` René Scharfe
Code repositories for project(s) associated with this public inbox https://80x24.org/mirrors/git.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).