Re: Cloning submodules on github actions very slow

Re: Cloning submodules on github actions very slow

[विश्वासो वासुकिजः (Vishvas Vasuki)]

(A bug report - resending as plain text as the html one bounces.
kindly reply-all/ cc me if I am mistaken, since I am not subscribed to
git@vger.kernel.org .)

Why does submodule cloning pull in unrelated branches? Trying the
below did not work (the second command fails with " Unable to find
refs/remotes/origin/content revision in submodule path 'content'"):

        git submodule update  --single-branch --init --recursive
        git submodule update --remote --merge --recursive

For background info, please see
https://github.com/vishvAsa/purANam/issues/5 or the thread below.


>
>
>
> On Thu, 2 Feb 2023 at 19:19, Shreevatsa R <shreevatsa@gmail.com> wrote:
>>
>> Yes, the size of the repo has gone from ~8 GB to ~120MB now, so it's faster now.
>>
>> The size of the repository depends on all the objects in it, which includes the history of all branches. It need not have any direct relationship with the number of files.
>>
>> On Thu, 2 Feb 2023 at 02:28, विश्वासो वासुकिजः (Vishvas Vasuki) <vishvas.vasuki@gmail.com> wrote:
>>>
>>>
>>>
>>> On Thu, 2 Feb 2023 at 13:39, विश्वासो वासुकिजः (Vishvas Vasuki) <vishvas.vasuki@gmail.com> wrote:
>>>>
>>>>
>>>>
>>>> On Thu, 2 Feb 2023 at 11:39, Shreevatsa R <shreevatsa@gmail.com> wrote:
>>>>>
>>>>> AFAICT the issue has nothing to do with submodules or GitHub actions; the repository in question is just massive.
>>>>>
>>>> I thought so too initially, but it doesn't seem so. Observe the timestamps on the following lines
>>>>
>>>> 2023-02-01T17:34:32.3793039Z Cloning into '/home/runner/work/purANam/purANam/content'...
>>>> 2023-02-01T17:47:40.0953104Z Cloning into '/home/runner/work/purANam/purANam/static'...
>>>> 2023-02-01T18:00:36.8478598Z Cloning into '/home/runner/work/purANam/purANam/themes/sanskrit-documentation-theme-hugo'...
>>>>
>>>> Now, you see it took 13 minutes for the static_files branch. And, that branch has - just a few files, and just two commits - https://github.com/vishvAsa/purANam/tree/static_files .
>>>
>>>
>>>
>>> Oh - It seems like I had to squash ALL branches (and ensure that the deploy-branch history does not keep growing over time).
>>>
>>> I could not figure out a way to make sure that submodule initialization/ updation works without peeking into unrelated branches.
>>>
>>>
>>>>
>>>>
>>>> To add to this - This is in contrast with 27s for purANam_vaiShNavam, where the number of files is quite large.
>>>>
>>>>
>>>>
>>>>
>>>>>
>>>>> Just ran git clone git@github.com:vishvAsa/purANam.git and it (still running) seems to be about 8 GiB in size!
>>>>>
>>>>> See this Gitlab help page for some ideas on reducing repository size (basically rewrite history). (If you want to understand what's going on, this blog post I wrote a few years ago (needs to be updated for Python 3) may be helpful though it's been a while and I haven't thought about how relevant it is in this case...)
>>>>
>>>>
>>>>
>>>> I already tried squashing commits ( https://github.com/vishvAsa/purANam/issues/5#issuecomment-1412653883 ) and force pushing - did not help.
>>>>
>>>> (Something like git rebase -i 4d7ca0eaf2583d8454b36671fed09b7ab2501680  followed by squashing all but a few commits, followed by force-push)
>>>>
>>>>
>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Wed, 1 Feb 2023 at 20:13, विश्वासो वासुकिजः (Vishvas Vasuki) <vishvas.vasuki@gmail.com> wrote:
>>>>>>
>>>>>> ( Using github actions to build a sanskrit related site. )
>>>>>>
>>>>>> Any ideas regarding https://stackoverflow.com/questions/75318595/github-actions-cloning-submodules-very-slow ?
>>>>>>
>>>>>> --
>>>>>> --
>>>>>> Vishvas /विश्वासः
>>>>>>
>>>>>> --
>>>>>> You received this message because you are subscribed to the Google Groups "sanskrit-programmers" group.
>>>>>> To unsubscribe from this group and stop receiving emails from it, send an email to sanskrit-programmers+unsubscribe@googlegroups.com.
>>>>>> To view this discussion on the web visit https://groups.google.com/d/msgid/sanskrit-programmers/CAFY6qgHNvXQq1Fjg4K3E5aPwr5JMxegjSJymgcGus-iw8L4riA%40mail.gmail.com.
>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google Groups "sanskrit-programmers" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send an email to sanskrit-programmers+unsubscribe@googlegroups.com.
>>>>> To view this discussion on the web visit https://groups.google.com/d/msgid/sanskrit-programmers/CAKEM%3DPMnBYqW-0TkB_P6Tav5JonEkefs71fOXr9s7hVpJ0uK6A%40mail.gmail.com.
>>>>
>>>>
>>>>
>>>> --
>>>> --
>>>> Vishvas /विश्वासः
>>>>
>>>
>>>
>>> --
>>> --
>>> Vishvas /विश्वासः
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups "sanskrit-programmers" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an email to sanskrit-programmers+unsubscribe@googlegroups.com.
>>> To view this discussion on the web visit https://groups.google.com/d/msgid/sanskrit-programmers/CAFY6qgG%2BYTX63ND68zdF98SqTK87%2B4kKu3aQUVCX-dsAVwqGAg%40mail.gmail.com.
>>
>> --
>> You received this message because you are subscribed to the Google Groups "sanskrit-programmers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an email to sanskrit-programmers+unsubscribe@googlegroups.com.
>> To view this discussion on the web visit https://groups.google.com/d/msgid/sanskrit-programmers/CAKEM%3DPOA6L4pZCeWgYiqgamaAt4OG6Fg50QSRnXBbFEekmOXSQ%40mail.gmail.com.
>
>
>
> --
> --
> Vishvas /विश्वासः
>


-- 
--
Vishvas /विश्वासः