git-svn too slow, contacts upstream svn repo

git-svn too slow, contacts upstream svn repo

[Quark]

hello list,

I have been using git-svn in an corporate environment where svn repo has lot of branches, (lot means > 100). To avoid cloning all branches my config looks as below

[svn-remote "svn"]
        url = svn+ssh://url
        fetch = srcroot/trunk:refs/remotes/trunk
        branches = srcroot/branches/{branch_1, branch_2, branch_3}:refs/remotes/*

Now, when a new branch of my interest is added in svn repo, I had like it to be in my git-repo as well. In an ideal world, one shall simply add that branch name to list in curly braces above, but that does not work. I had love to be proved wrong here. Somebody on stackoverflow.com suggested (to else's question, not mine) to add one more "fetch" as show below.

[svn-remote "svn"]
        url = svn+ssh://url
        fetch = srcroot/trunk:refs/remotes/trunk
        fetch = srcroot/branch_4:refs/remotes/* # NEW BRANCH
        branches = srcroot/branches/{branch_1, branch_2, branch_3}:refs/remotes/*

haven't tried it, neither do I like it.

I do have a solution which WORKS, it is modified version of an example from \doc\git\html\git-svn.html page. Below I have pasted the example and modified it to reflect my specifics.

# assume an existing git-svn repo
        D:\sourcecode
# Clone locally - make sure the refs/remotes/ space matches the server
        mkdir project
        cd project
        git init
        git remote add origin file:///D:/sourcecode # file:// is delibrate, want true n/w behaviour
        git config --replace-all remote.origin.fetch '+refs/remotes/*:refs/remotes/*'
        git fetch
# Prevent fetch/pull from local git repo,
# we only want to use git svn for form here
       git config --remove-section remote.origin
# Create a local branch from one of the branches just fetched
        git checkout -b master FETCH_HEAD
# Initialize 'git svn' locally (be sure to use the same URL and -T/-b/-t options as were used on server)
        git svn init http://svn.example.com/project

# edit config to include newly added branch in curly braces
# Pull the latest changes from Subversion
        git svn fetch -r <recentish rev, say BEGIN>:HEAD

Above works exactly as I want, except the last step of "git-svn fetch" connects to svn repo for each revision from BEGIN to HEAD. I want fairly long history (for pretty blame). Which implies my BEGIN is not so recentish adterall. Hence, it takes awfully long to finish, 2 days typically.

When GIT_TRACE is 1, below is output received

trace: exec: 'git-svn' 'fetch'
trace: run_command: 'git-svn' 'fetch'
trace: built-in: git 'config' '--bool' '--get' 'svn.fetchall'
trace: built-in: git 'config' '--bool' '--get' 'svn.parent'
trace: built-in: git 'config' '--bool' '--get' 'svn.noauthcache'
trace: built-in: git 'config' '--get' 'svn.revision'
trace: built-in: git 'config' '--bool' '--get' 'svn.nocheckout'
trace: built-in: git 'config' '--get' 'svn.authorsprog'
trace: built-in: git 'config' '--bool' '--get' 'svn.followparent'
trace: built-in: git 'config' '--get' 'svn.authorsfile'
trace: built-in: git 'config' '--bool' '--get' 'svn.useSvmProps'
trace: built-in: git 'config' '--get' 'svn.username'
trace: built-in: git 'config' '--get' 'svn.repackflags'
trace: built-in: git 'config' '--bool' '--get' 'svn.localtime'
trace: built-in: git 'config' '--int' '--get' 'svn.repack'
trace: built-in: git 'config' '--get' 'svn.ignorepaths'
trace: built-in: git 'config' '--int' '--get' 'svn.logwindowsize'
trace: built-in: git 'config' '--bool' '--get' 'svn.quiet'
trace: built-in: git 'config' '--get' 'svn.ignorerefs'
trace: built-in: git 'config' '--get' 'svn.configdir'
trace: built-in: git 'config' '--bool' '--get' 'svn.addauthorfrom'
trace: built-in: git 'config' '--bool' '--get' 'svn.useSvnsyncProps'
trace: built-in: git 'config' '--bool' '--get' 'svn.noMetadata'
trace: built-in: git 'config' '--bool' '--get' 'svn.uselogauthor'
trace: built-in: git 'rev-parse' '--symbolic' '--all'
trace: built-in: git 'config' '-l'
trace: built-in: git 'config' '-l'
trace: built-in: git 'config' '--bool' 'svn.useSvmProps'
trace: built-in: git 'config' '-l'
trace: built-in: git 'config' 'svn-remote.svn.branches-maxRev' '524908'
trace: built-in: git 'rev-list' '--pretty=raw' '--reverse' '9f1414be94ab007b62ace31bf4d210a069276127..refs/remotes/branch_1' '--'
trace: built-in: git 'rev-list' '--pretty=raw' '--reverse' '5eb0a454bcf066a8199b851add9ec07cde80119d..refs/remotes/branch_2' '--'
trace: built-in: git 'rev-list' '--pretty=raw' '--reverse' '383b68b8514010a71efe10821e5ccc3541903ceb..refs/remotes/branch_3' '--'
trace: built-in: git 'rev-list' '--pretty=raw' '--reverse' '89fe1a1d2cfca0886003f043c408fb5afadfec93..refs/remotes/trunk' '--'

I keep an watch for "svn-remote.svn.branches-maxRev", this serves as my progress. This wait is too much for a simple branch. Though, I must point out, day feels fresh once it finishes :) Does any one know any for this?

If you paid attention this is Windows machine
git version 1.8.1.msysgit.1


thanks,
Quark

Re: git-svn too slow, contacts upstream svn repo

[Quark]

----- Original Message -----
> From: Quark <unixuser2000-fbsd@yahoo.com>
> To: Git List <git@vger.kernel.org>
> Cc: 
> Sent: Monday, 27 May 2013 8:55 PM
> Subject: git-svn too slow, contacts upstream svn repo
> 
> hello list,
> 
> I have been using git-svn in an corporate environment where svn repo has lot of 
> branches, (lot means > 100). To avoid cloning all branches my config looks as 
> below
> 
> [svn-remote "svn"]
>         url = svn+ssh://url
>         fetch = srcroot/trunk:refs/remotes/trunk
>         branches = srcroot/branches/{branch_1, branch_2, 
> branch_3}:refs/remotes/*
> 
> Now, when a new branch of my interest is added in svn repo, I had like it to be 
> in my git-repo as well. In an ideal world, one shall simply add that branch name 
> to list in curly braces above, but that does not work. I had love to be proved 
> wrong here. Somebody on stackoverflow.com suggested (to else's question, not 
> mine) to add one more "fetch" as show below.
> 
> [svn-remote "svn"]
>         url = svn+ssh://url
>         fetch = srcroot/trunk:refs/remotes/trunk
>         fetch = srcroot/branch_4:refs/remotes/* # NEW BRANCH
>         branches = srcroot/branches/{branch_1, branch_2, 
> branch_3}:refs/remotes/*
> 
> haven't tried it, neither do I like it.
> 
> I do have a solution which WORKS, it is modified version of an example from 
> \doc\git\html\git-svn.html page. Below I have pasted the example 
> and modified it to reflect my specifics.
> 
> # assume an existing git-svn repo
>         D:\sourcecode
> # Clone locally - make sure the refs/remotes/ space matches the server
>         mkdir project
>         cd project
>         git init
>         git remote add origin file:///D:/sourcecode # file:// is delibrate, want 
> true n/w behaviour
>         git config --replace-all remote.origin.fetch 
> '+refs/remotes/*:refs/remotes/*'
>         git fetch
> # Prevent fetch/pull from local git repo,
> # we only want to use git svn for form here
>        git config --remove-section remote.origin
> # Create a local branch from one of the branches just fetched
>         git checkout -b master FETCH_HEAD
> # Initialize 'git svn' locally (be sure to use the same URL and -T/-b/-t 
> options as were used on server)
>         git svn init http://svn.example.com/project
> 
> # edit config to include newly added branch in curly braces
> # Pull the latest changes from Subversion
>         git svn fetch -r <recentish rev, say BEGIN>:HEAD
> 
> Above works exactly as I want, except the last step of "git-svn fetch" 
> connects to svn repo for each revision from BEGIN to HEAD. I want fairly long 
> history (for pretty blame). Which implies my BEGIN is not so recentish adterall. 
> Hence, it takes awfully long to finish, 2 days typically.
> 
> When GIT_TRACE is 1, below is output received
> 
> trace: exec: 'git-svn' 'fetch'
> trace: run_command: 'git-svn' 'fetch'
> trace: built-in: git 'config' '--bool' '--get' 
> 'svn.fetchall'
> trace: built-in: git 'config' '--bool' '--get' 
> 'svn.parent'
> trace: built-in: git 'config' '--bool' '--get' 
> 'svn.noauthcache'
> trace: built-in: git 'config' '--get' 'svn.revision'
> trace: built-in: git 'config' '--bool' '--get' 
> 'svn.nocheckout'
> trace: built-in: git 'config' '--get' 'svn.authorsprog'
> trace: built-in: git 'config' '--bool' '--get' 
> 'svn.followparent'
> trace: built-in: git 'config' '--get' 'svn.authorsfile'
> trace: built-in: git 'config' '--bool' '--get' 
> 'svn.useSvmProps'
> trace: built-in: git 'config' '--get' 'svn.username'
> trace: built-in: git 'config' '--get' 'svn.repackflags'
> trace: built-in: git 'config' '--bool' '--get' 
> 'svn.localtime'
> trace: built-in: git 'config' '--int' '--get' 
> 'svn.repack'
> trace: built-in: git 'config' '--get' 'svn.ignorepaths'
> trace: built-in: git 'config' '--int' '--get' 
> 'svn.logwindowsize'
> trace: built-in: git 'config' '--bool' '--get' 
> 'svn.quiet'
> trace: built-in: git 'config' '--get' 'svn.ignorerefs'
> trace: built-in: git 'config' '--get' 'svn.configdir'
> trace: built-in: git 'config' '--bool' '--get' 
> 'svn.addauthorfrom'
> trace: built-in: git 'config' '--bool' '--get' 
> 'svn.useSvnsyncProps'
> trace: built-in: git 'config' '--bool' '--get' 
> 'svn.noMetadata'
> trace: built-in: git 'config' '--bool' '--get' 
> 'svn.uselogauthor'
> trace: built-in: git 'rev-parse' '--symbolic' '--all'
> trace: built-in: git 'config' '-l'
> trace: built-in: git 'config' '-l'
> trace: built-in: git 'config' '--bool' 'svn.useSvmProps'
> trace: built-in: git 'config' '-l'
> trace: built-in: git 'config' 'svn-remote.svn.branches-maxRev' 
> '524908'
> trace: built-in: git 'rev-list' '--pretty=raw' 
> '--reverse' 
> '9f1414be94ab007b62ace31bf4d210a069276127..refs/remotes/branch_1' 
> '--'
> trace: built-in: git 'rev-list' '--pretty=raw' 
> '--reverse' 
> '5eb0a454bcf066a8199b851add9ec07cde80119d..refs/remotes/branch_2' 
> '--'
> trace: built-in: git 'rev-list' '--pretty=raw' 
> '--reverse' 
> '383b68b8514010a71efe10821e5ccc3541903ceb..refs/remotes/branch_3' 
> '--'
> trace: built-in: git 'rev-list' '--pretty=raw' 
> '--reverse' 
> '89fe1a1d2cfca0886003f043c408fb5afadfec93..refs/remotes/trunk' 
> '--'
> 
> I keep an watch for "svn-remote.svn.branches-maxRev", this serves as 
> my progress. This wait is too much for a simple branch. Though, I must point 
> out, day feels fresh once it finishes :) Does any one know any for this?
> 
> If you paid attention this is Windows machine
> git version 1.8.1.msysgit.1
> 
> 
> thanks,
> Quark
> 
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

is this not right forum?

thanks,
Quark

Re: git-svn too slow, contacts upstream svn repo

[Konstantin Khomoutov]

On Tue, 28 May 2013 15:42:57 +0800 (SGT)
Quark <unixuser2000-fbsd@yahoo.com> wrote:

> > I have been using git-svn in an corporate environment where svn
> > repo has lot of branches, (lot means > 100). To avoid cloning all
> > branches my config looks as below
[...]
> is this not right forum?

As a matter of fact, this mailing list is the only correct place to ask
questions like yours.  But this is free software after all -- people
who could answer your question may be busy/absent at the moment or even
not involved in the project anymore (in the worst case).  So be
prepared to wait some time.  Also be prepared for your particular
trouble not being solved.

In the meantime, I think Thomas provided you with valuable suggestions
in reply to your mirror post on git-users, so you could possibly
explore them.

Re: git-svn too slow, contacts upstream svn repo

[Quark]

----- Original Message -----
> From: Konstantin Khomoutov <kostix+git@007spb.ru>
> To: Quark <unixuser2000-fbsd@yahoo.com>
> Cc: Git List <git@vger.kernel.org>
> Sent: Tuesday, 28 May 2013 5:24 PM
> Subject: Re: git-svn too slow, contacts upstream svn repo
> 
> On Tue, 28 May 2013 15:42:57 +0800 (SGT)
> Quark <unixuser2000-fbsd@yahoo.com> wrote:
> 
>>  > I have been using git-svn in an corporate environment where svn
>>  > repo has lot of branches, (lot means > 100). To avoid cloning all
>>  > branches my config looks as below
> [...]
>>  is this not right forum?
> 
> As a matter of fact, this mailing list is the only correct place to ask
> questions like yours.  But this is free software after all -- people
> who could answer your question may be busy/absent at the moment or even
> not involved in the project anymore (in the worst case).  So be
> prepared to wait some time.  Also be prepared for your particular
> trouble not being solved.
> 
> In the meantime, I think Thomas provided you with valuable suggestions
> in reply to your mirror post on git-users, so you could possibly
> explore them.
>

I was impatient not because feeling-of-entitlement, but I thought it might get lost in pile of e-mails
Secondly was not sure if this is right forum, I might be making noise after all, thanks for your clarification.
Now will you please excuse, I have to check Thomas's suggestions? :)