From: Matheus Tavares <matheus.bernardino@usp.br>
To: git@vger.kernel.org
Cc: "Thomas Gummerer" <t.gummerer@gmail.com>,
"Ævar Arnfjörð Bjarmason" <avarab@gmail.com>,
"Christian Couder" <christian.couder@gmail.com>,
"Nguyễn Thái Ngọc Duy" <pclouds@gmail.com>,
"SZEDER Gábor" <szeder.dev@gmail.com>,
kernel-usp@googlegroups.com
Subject: [GSoC][PATCH v6 00/10] clone: dir-iterator refactoring with tests
Date: Thu, 2 May 2019 11:48:19 -0300 [thread overview]
Message-ID: <20190502144829.4394-1-matheus.bernardino@usp.br> (raw)
In-Reply-To: <20190330224907.3277-1-matheus.bernardino@usp.br>
This patchset contains:
- a replacement of explicit recursive dir iteration at
copy_or_link_directory for the dir-iterator API;
- some refactoring and behaviour changes at local clone, mainly to
take care of symlinks and hidden files at .git/objects, together
with tests for this types of files.
- dir-iterator refactoring and feature adding with tests.
Changes since v5:
- Add tests for the dir-iterator API
- Refactor the dir-iterator state machine model, simplifying its
mechanics to improve readability.
- Change warning() to warning_errno() at dir-iterator.c
- Add a recursive symlinks check for dir_iterator_advance() in order
to avoid unwanted recursions with DIR_ITERATOR_FOLLOW_SYMLIKS
- Add tests for the dir-iterator flags feature
- Make warnings be emitted both when DIR_ITERATOR_PEDANTIC is
supplied and when it's not. It contains more relevant information
on the error, so I thought it should be always printed.
- Make dir_iterator_begin() check if the given argument is a valid
path to a directory.
- Adjusted some minor codestyle problems and commit messages
- Address Thomas comments in v5
v5: https://public-inbox.org/git/20190330224907.3277-1-matheus.bernardino@usp.br/
travis build: https://travis-ci.org/MatheusBernardino/git/builds/527176611
Note: I tried to use --range-diff as Thomas suggested but I'm not sure
the output is as desired. Please, let me know if I did something wrong
using it.
Daniel Ferreira (1):
dir-iterator: add tests for dir-iterator API
Matheus Tavares (8):
clone: better handle symlinked files at .git/objects/
dir-iterator: use warning_errno when possible
dir-iterator: refactor state machine model
dir-iterator: add flags parameter to dir_iterator_begin
clone: copy hidden paths at local clone
clone: extract function from copy_or_link_directory
clone: use dir-iterator to avoid explicit dir traversal
clone: replace strcmp by fspathcmp
Ævar Arnfjörð Bjarmason (1):
clone: test for our behavior on odd objects/* content
Makefile | 1 +
builtin/clone.c | 75 +++++----
dir-iterator.c | 289 +++++++++++++++++++++--------------
dir-iterator.h | 59 +++++--
refs/files-backend.c | 17 ++-
t/helper/test-dir-iterator.c | 58 +++++++
t/helper/test-tool.c | 1 +
t/helper/test-tool.h | 1 +
t/t0066-dir-iterator.sh | 163 ++++++++++++++++++++
t/t5604-clone-reference.sh | 133 ++++++++++++++++
10 files changed, 634 insertions(+), 163 deletions(-)
create mode 100644 t/helper/test-dir-iterator.c
create mode 100755 t/t0066-dir-iterator.sh
Range-diff against v5:
1: 3d422dd4de = 1: a630b1a129 clone: test for our behavior on odd objects/* content
2: 35819e6ed1 ! 2: 51e06687fc clone: better handle symlinked files at .git/objects/
@@ -45,10 +45,7 @@
die_errno(_("failed to unlink '%s'"), dest->buf);
if (!option_no_hardlinks) {
- if (!link(src->buf, dest->buf))
-+ char *resolved_path = real_pathdup(src->buf, 1);
-+ int status = link(resolved_path, dest->buf);
-+ free(resolved_path);
-+ if (!status)
++ if (!link(real_path(src->buf), dest->buf))
continue;
if (option_local > 0)
die_errno(_("failed to create link '%s'"), dest->buf);
3: 2afe3208a4 < -: ---------- dir-iterator: add flags parameter to dir_iterator_begin
-: ---------- > 3: c8a860e3a5 dir-iterator: add tests for dir-iterator API
-: ---------- > 4: b975351080 dir-iterator: use warning_errno when possible
-: ---------- > 5: 0fdbd1633e dir-iterator: refactor state machine model
-: ---------- > 6: 7b2a9ae947 dir-iterator: add flags parameter to dir_iterator_begin
4: 71d64e6278 = 7: b9f298cbc6 clone: copy hidden paths at local clone
5: 35e36756db = 8: 0e7b1e49e2 clone: extract function from copy_or_link_directory
6: 1bfda87879 ! 9: f726ce2733 clone: use dir-iterator to avoid explicit dir traversal
@@ -8,10 +8,14 @@
copy_or_link_directory.
This process also makes copy_or_link_directory call die() in case of an
- error on readdir or stat, inside dir_iterator_advance. Previously it
+ error on readdir or stat inside dir_iterator_advance. Previously it
would just print a warning for errors on stat and ignore errors on
readdir, which isn't nice because a local git clone could succeed even
- though the .git/objects copy didn't fully succeed.
+ though the .git/objects copy didn't fully succeed. Also, with the
+ dir-iterator API, recursive symlinks will be detected and skipped. This
+ is another behavior improvement, since the current version would
+ continue to copy the same content over and over until stat() returned an
+ ELOOP error.
Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br>
@@ -44,12 +48,15 @@
- die_errno(_("failed to open '%s'"), src->buf);
+ struct dir_iterator *iter;
+ int iter_status;
-+ unsigned flags;
++ unsigned int flags;
mkdir_if_missing(dest->buf, 0777);
+ flags = DIR_ITERATOR_PEDANTIC | DIR_ITERATOR_FOLLOW_SYMLINKS;
+ iter = dir_iterator_begin(src->buf, flags);
++
++ if (!iter)
++ die_errno(_("failed to start iterator over '%s'"), src->buf);
+
strbuf_addch(src, '/');
src_len = src->len;
7: 3861b30108 = 10: 6a57bb3887 clone: replace strcmp by fspathcmp
--
2.20.1
next prev parent reply other threads:[~2019-05-02 14:48 UTC|newest]
Thread overview: 127+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-02-26 5:17 [WIP RFC PATCH v2 0/5] clone: dir iterator refactoring with tests Matheus Tavares
2019-02-26 5:18 ` [WIP RFC PATCH v2 1/5] dir-iterator: add flags parameter to dir_iterator_begin Matheus Tavares
2019-02-26 12:01 ` Duy Nguyen
2019-02-27 13:59 ` Matheus Tavares Bernardino
2019-02-26 5:18 ` [WIP RFC PATCH v2 2/5] clone: test for our behavior on odd objects/* content Matheus Tavares
2019-02-26 5:18 ` [WIP RFC PATCH v2 3/5] clone: copy hidden paths at local clone Matheus Tavares
2019-02-26 12:13 ` Duy Nguyen
2019-02-26 5:18 ` [WIP RFC PATCH v2 4/5] clone: extract function from copy_or_link_directory Matheus Tavares
2019-02-26 12:18 ` Duy Nguyen
2019-02-27 17:30 ` Matheus Tavares Bernardino
2019-02-27 22:45 ` Thomas Gummerer
2019-02-27 22:50 ` Matheus Tavares Bernardino
2019-02-26 5:18 ` [WIP RFC PATCH v2 5/5] clone: use dir-iterator to avoid explicit dir traversal Matheus Tavares
2019-02-26 11:35 ` Ævar Arnfjörð Bjarmason
2019-02-26 12:32 ` Duy Nguyen
2019-02-26 12:50 ` Ævar Arnfjörð Bjarmason
2019-02-27 17:40 ` Matheus Tavares Bernardino
2019-02-28 7:13 ` Duy Nguyen
2019-02-28 7:53 ` Ævar Arnfjörð Bjarmason
2019-02-26 11:36 ` [WIP RFC PATCH v2 0/5] clone: dir iterator refactoring with tests Ævar Arnfjörð Bjarmason
2019-02-26 12:20 ` Duy Nguyen
2019-02-26 12:28 ` [RFC PATCH v3 " Ævar Arnfjörð Bjarmason
2019-02-26 20:56 ` Matheus Tavares Bernardino
2019-03-22 23:22 ` [GSoC][PATCH v4 0/7] clone: dir-iterator " Matheus Tavares
2019-03-22 23:22 ` [GSoC][PATCH v4 1/7] clone: test for our behavior on odd objects/* content Matheus Tavares
2019-03-24 18:09 ` Matheus Tavares Bernardino
2019-03-24 20:56 ` SZEDER Gábor
2019-03-26 19:43 ` Matheus Tavares Bernardino
2019-03-28 21:49 ` Thomas Gummerer
2019-03-29 14:06 ` Matheus Tavares Bernardino
2019-03-29 19:31 ` Thomas Gummerer
2019-03-29 19:42 ` SZEDER Gábor
2019-03-30 2:49 ` Matheus Tavares Bernardino
2019-03-22 23:22 ` [GSoC][PATCH v4 2/7] clone: better handle symlinked files at .git/objects/ Matheus Tavares
2019-03-28 22:10 ` Thomas Gummerer
2019-03-29 8:38 ` Ævar Arnfjörð Bjarmason
2019-03-29 20:15 ` Thomas Gummerer
2019-03-29 14:27 ` Matheus Tavares Bernardino
2019-03-29 20:05 ` Thomas Gummerer
2019-03-30 5:32 ` Matheus Tavares Bernardino
2019-03-30 19:27 ` Thomas Gummerer
2019-04-01 3:56 ` Matheus Tavares Bernardino
2019-03-29 15:40 ` Johannes Schindelin
2019-03-22 23:22 ` [GSoC][PATCH v4 3/7] dir-iterator: add flags parameter to dir_iterator_begin Matheus Tavares
2019-03-28 22:19 ` Thomas Gummerer
2019-03-29 13:16 ` Matheus Tavares Bernardino
2019-03-22 23:22 ` [GSoC][PATCH v4 4/7] clone: copy hidden paths at local clone Matheus Tavares
2019-03-22 23:22 ` [GSoC][PATCH v4 5/7] clone: extract function from copy_or_link_directory Matheus Tavares
2019-03-22 23:22 ` [GSoC][PATCH v4 6/7] clone: use dir-iterator to avoid explicit dir traversal Matheus Tavares
2019-03-22 23:22 ` [GSoC][PATCH v4 7/7] clone: Replace strcmp by fspathcmp Matheus Tavares
2019-03-30 22:49 ` [GSoC][PATCH v5 0/7] clone: dir-iterator refactoring with tests Matheus Tavares
2019-03-30 22:49 ` [GSoC][PATCH v5 1/7] clone: test for our behavior on odd objects/* content Matheus Tavares
2019-03-30 22:49 ` [GSoC][PATCH v5 2/7] clone: better handle symlinked files at .git/objects/ Matheus Tavares
2019-03-31 17:40 ` Thomas Gummerer
2019-04-01 3:59 ` Matheus Tavares Bernardino
2019-03-30 22:49 ` [GSoC][PATCH v5 3/7] dir-iterator: add flags parameter to dir_iterator_begin Matheus Tavares
2019-03-31 18:12 ` Thomas Gummerer
2019-04-10 20:24 ` Matheus Tavares Bernardino
2019-04-11 21:09 ` Thomas Gummerer
2019-04-23 17:07 ` Matheus Tavares Bernardino
2019-04-24 18:36 ` Thomas Gummerer
2019-04-26 4:13 ` Matheus Tavares Bernardino
2019-03-30 22:49 ` [GSoC][PATCH v5 4/7] clone: copy hidden paths at local clone Matheus Tavares
2019-03-30 22:49 ` [GSoC][PATCH v5 5/7] clone: extract function from copy_or_link_directory Matheus Tavares
2019-03-30 22:49 ` [GSoC][PATCH v5 6/7] clone: use dir-iterator to avoid explicit dir traversal Matheus Tavares
2019-03-30 22:49 ` [GSoC][PATCH v5 7/7] clone: replace strcmp by fspathcmp Matheus Tavares
2019-03-31 18:16 ` [GSoC][PATCH v5 0/7] clone: dir-iterator refactoring with tests Thomas Gummerer
2019-04-01 13:56 ` Matheus Tavares Bernardino
2019-05-02 14:48 ` Matheus Tavares [this message]
2019-05-02 14:48 ` [GSoC][PATCH v6 01/10] clone: test for our behavior on odd objects/* content Matheus Tavares
2019-05-02 14:48 ` [GSoC][PATCH v6 02/10] clone: better handle symlinked files at .git/objects/ Matheus Tavares
2019-05-02 14:48 ` [GSoC][PATCH v6 03/10] dir-iterator: add tests for dir-iterator API Matheus Tavares
2019-05-02 14:48 ` [GSoC][PATCH v6 04/10] dir-iterator: use warning_errno when possible Matheus Tavares
2019-05-02 14:48 ` [GSoC][PATCH v6 05/10] dir-iterator: refactor state machine model Matheus Tavares
2019-05-02 14:48 ` [GSoC][PATCH v6 06/10] dir-iterator: add flags parameter to dir_iterator_begin Matheus Tavares
2019-05-02 14:48 ` [GSoC][PATCH v6 07/10] clone: copy hidden paths at local clone Matheus Tavares
2019-05-02 14:48 ` [GSoC][PATCH v6 08/10] clone: extract function from copy_or_link_directory Matheus Tavares
2019-05-02 14:48 ` [GSoC][PATCH v6 09/10] clone: use dir-iterator to avoid explicit dir traversal Matheus Tavares
2019-05-02 14:48 ` [GSoC][PATCH v6 10/10] clone: replace strcmp by fspathcmp Matheus Tavares
2019-06-18 23:27 ` [GSoC][PATCH v7 00/10] clone: dir-iterator refactoring with tests Matheus Tavares
2019-06-18 23:27 ` [GSoC][PATCH v7 01/10] clone: test for our behavior on odd objects/* content Matheus Tavares
2019-06-18 23:27 ` [GSoC][PATCH v7 02/10] clone: better handle symlinked files at .git/objects/ Matheus Tavares
2019-06-18 23:27 ` [GSoC][PATCH v7 03/10] dir-iterator: add tests for dir-iterator API Matheus Tavares
2019-06-18 23:27 ` [GSoC][PATCH v7 04/10] dir-iterator: use warning_errno when possible Matheus Tavares
2019-06-18 23:27 ` [GSoC][PATCH v7 05/10] dir-iterator: refactor state machine model Matheus Tavares
2019-06-18 23:27 ` [GSoC][PATCH v7 06/10] dir-iterator: add flags parameter to dir_iterator_begin Matheus Tavares
2019-06-25 18:00 ` Junio C Hamano
2019-06-25 18:11 ` Matheus Tavares Bernardino
2019-06-26 13:34 ` Johannes Schindelin
2019-06-26 18:04 ` Junio C Hamano
2019-06-27 9:20 ` Duy Nguyen
2019-06-27 17:23 ` Matheus Tavares Bernardino
2019-06-27 18:48 ` Johannes Schindelin
2019-06-27 19:33 ` Matheus Tavares Bernardino
2019-06-28 12:51 ` Johannes Schindelin
2019-06-28 14:16 ` Matheus Tavares Bernardino
2019-07-01 12:15 ` Johannes Schindelin
2019-07-03 8:57 ` SZEDER Gábor
2019-07-08 22:21 ` Matheus Tavares Bernardino
2019-06-18 23:27 ` [GSoC][PATCH v7 07/10] clone: copy hidden paths at local clone Matheus Tavares
2019-06-18 23:27 ` [GSoC][PATCH v7 08/10] clone: extract function from copy_or_link_directory Matheus Tavares
2019-06-18 23:27 ` [GSoC][PATCH v7 09/10] clone: use dir-iterator to avoid explicit dir traversal Matheus Tavares
2019-06-18 23:27 ` [GSoC][PATCH v7 10/10] clone: replace strcmp by fspathcmp Matheus Tavares
2019-06-19 4:36 ` [GSoC][PATCH v7 00/10] clone: dir-iterator refactoring with tests Matheus Tavares Bernardino
2019-06-20 20:18 ` Junio C Hamano
2019-06-21 13:41 ` Matheus Tavares Bernardino
2019-07-10 23:58 ` [GSoC][PATCH v8 " Matheus Tavares
2019-07-10 23:58 ` [GSoC][PATCH v8 01/10] clone: test for our behavior on odd objects/* content Matheus Tavares
2019-07-10 23:58 ` [GSoC][PATCH v8 02/10] clone: better handle symlinked files at .git/objects/ Matheus Tavares
2019-07-10 23:58 ` [GSoC][PATCH v8 03/10] dir-iterator: add tests for dir-iterator API Matheus Tavares
2019-07-10 23:58 ` [GSoC][PATCH v8 04/10] dir-iterator: use warning_errno when possible Matheus Tavares
2019-07-10 23:58 ` [GSoC][PATCH v8 05/10] dir-iterator: refactor state machine model Matheus Tavares
2019-07-10 23:59 ` [GSoC][PATCH v8 06/10] dir-iterator: add flags parameter to dir_iterator_begin Matheus Tavares
2019-07-10 23:59 ` [GSoC][PATCH v8 07/10] clone: copy hidden paths at local clone Matheus Tavares
2019-07-10 23:59 ` [GSoC][PATCH v8 08/10] clone: extract function from copy_or_link_directory Matheus Tavares
2019-07-10 23:59 ` [GSoC][PATCH v8 09/10] clone: use dir-iterator to avoid explicit dir traversal Matheus Tavares
2019-07-10 23:59 ` [GSoC][PATCH v8 10/10] clone: replace strcmp by fspathcmp Matheus Tavares
2019-07-11 11:56 ` [GSoC][PATCH v8 00/10] clone: dir-iterator refactoring with tests Johannes Schindelin
2019-07-11 15:24 ` Matheus Tavares Bernardino
2019-02-26 12:28 ` [RFC PATCH v3 1/5] clone: test for our behavior on odd objects/* content Ævar Arnfjörð Bjarmason
2019-02-28 21:19 ` Matheus Tavares Bernardino
2019-03-01 13:49 ` Ævar Arnfjörð Bjarmason
2019-03-13 3:17 ` Matheus Tavares
2019-02-26 12:28 ` [RFC PATCH v3 2/5] dir-iterator: add flags parameter to dir_iterator_begin Ævar Arnfjörð Bjarmason
2019-02-26 12:28 ` [RFC PATCH v3 3/5] clone: copy hidden paths at local clone Ævar Arnfjörð Bjarmason
2019-02-26 12:28 ` [RFC PATCH v3 4/5] clone: extract function from copy_or_link_directory Ævar Arnfjörð Bjarmason
2019-02-26 12:28 ` [RFC PATCH v3 5/5] clone: use dir-iterator to avoid explicit dir traversal Ævar Arnfjörð Bjarmason
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190502144829.4394-1-matheus.bernardino@usp.br \
--to=matheus.bernardino@usp.br \
--cc=avarab@gmail.com \
--cc=christian.couder@gmail.com \
--cc=git@vger.kernel.org \
--cc=kernel-usp@googlegroups.com \
--cc=pclouds@gmail.com \
--cc=szeder.dev@gmail.com \
--cc=t.gummerer@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).