From: Christian Couder <email@example.com> To: "Ævar Arnfjörð Bjarmason" <firstname.lastname@example.org>, Alireza <email@example.com> Cc: Konstantin Ryabitsev <firstname.lastname@example.org>, "brian m. carlson" <email@example.com>, git <firstname.lastname@example.org> Subject: Partial clone demo for large files (Re: Why Git LFS is not a built-in feature) Date: Wed, 18 Nov 2020 11:20:20 +0100 [thread overview] Message-ID: <CAP8UFD35kk10FpUnPpiAUzTHJbm=SJ-76OTmkTwBstGFe3Zgdw@mail.gmail.com> (raw) In-Reply-To: <email@example.com> On Sat, Nov 14, 2020 at 7:25 PM Ævar Arnfjörð Bjarmason <firstname.lastname@example.org> wrote: > > > On Sat, Nov 14 2020, Konstantin Ryabitsev wrote: > > > On Sat, Nov 14, 2020 at 12:29:02AM +0000, brian m. carlson wrote: > >> Additionally, in many cases, projects can avoid the need for storing > >> large files at all by using repository best practices, like not storing > >> build products or binary dependencies in the repository and instead > >> using an artifact server or a standard packaging system. If possible, > >> that will almost always provide a better experience than any solution > >> for storing large files in the repository. > > > > Well, I would argue that if the goal is ongoing archival and easy > > replication, then storing objects in a repository like git makes a lot > > more sense than keeping them on a central server that may or may not be > > there a few years down the line. Having large file support native in git > > is a laudable goal and I quite often wish that it existed. > > That native support does exist right now in the form of partial clones, > the packfile-uris support, core.bigFileThreshold etc. > > It's got a lot of rough edges currently, but if it's something you're > interested in you should try it out and see if the subset of features > that works well now is something that would work for you. I have been working on a partial clone demo that stores large files on an HTTP server: https://gitlab.com/chriscool/partial-clone-demo/-/blob/master/http-promisor/demo.txt It has a lot of rough edges indeed. Fetching from the HTTP server promisor remote is very slow. For the last part you need a hacked Git. The scripts have a lot of bugs and limitations and are not finished (to say the least). The goal for now is just to give people (especially product managers, developers and managers inside GitLab) an outlook about how it could work.
next prev parent reply other threads:[~2020-11-18 10:23 UTC|newest] Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-11-13 9:45 Why Git LFS is not a built-in feature Alireza 2020-11-14 0:29 ` brian m. carlson 2020-11-14 16:27 ` Konstantin Ryabitsev 2020-11-14 18:20 ` Ævar Arnfjörð Bjarmason 2020-11-18 10:20 ` Christian Couder [this message] 2020-11-14 19:15 ` brian m. carlson
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style List information: http://vger.kernel.org/majordomo-info.html * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to='CAP8UFD35kk10FpUnPpiAUzTHJbm=SJ-76OTmkTwBstGFe3Zgdw@mail.gmail.com' \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --subject='Partial clone demo for large files (Re: Why Git LFS is not a built-in feature)' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
Code repositories for project(s) associated with this inbox: https://80x24.org/mirrors/git.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).