git-svn "Temp file with moniker 'svn_delta' already in use" and skelta mode

git-svn "Temp file with moniker 'svn_delta' already in use" and skelta mode

[David Rothenberger]

I recently upgraded my Subversion server to 1.8.0 and started
receiving the following error from "git svn fetch":

Temp file with moniker 'svn_delta' already in use at /usr/lib/perl5/vendor_perl/5.10/Git.pm line 1024.

This occurs only when using an http:// URL; svn:// URLs work fine.

I think this is related to skelta mode [1] with serf. When I add
"SVNAllowBulkUpdates Prefer" to mod_dav configuration, the error
goes away. (Similarly, if I add "http-bulk-updates = yes" to
~/.subversion/servers.)

The server is running WanDisco's 1.8.0 distribution on 64-bit Debian
7.0. The client is Cygwin: svn version 1.8.0 and git version
1.8.3.2.

I initially reported this to the subversion users mailing list, but
was (not so politely) told to report this to the git-svn authors.
I'm not so sure, since the problem goes away simply by downgrading
the subversion-perl bindings back to 1.7.

[1] http://subversion.apache.org/docs/release-notes/1.8.html#serf-skelta-default

-- 
David Rothenberger  ----  daveroth@acm.org

Teutonic:
        Not enough gin.

Re: git-svn "Temp file with moniker 'svn_delta' already in use" and skelta mode

[Kyle McKay]

On Jul 5, 2013, at 13:48, David Rothenberger wrote:
> I recently upgraded my Subversion server to 1.8.0 and started
> receiving the following error from "git svn fetch":
>
> Temp file with moniker 'svn_delta' already in use at /usr/lib/perl5/ 
> vendor_perl/5.10/Git.pm line 1024.
>
> This occurs only when using an http:// URL; svn:// URLs work fine.
[snip]
> The client is Cygwin: svn version 1.8.0 and git version
> 1.8.3.2.

The subversion 1.8 release has removed the neon library, all svn  
client http access now always goes through the serf library.  If you  
put "http-library = serf" in the "[global]" section of the  
"~/.subversion/servers" file you will get the 'svn_delta' error with  
git-svn when running earlier versions of the svn client as well.

> I initially reported this to the subversion users mailing list, but
> was (not so politely) told to report this to the git-svn authors.
> I'm not so sure, since the problem goes away simply by downgrading
> the subversion-perl bindings back to 1.7.

That changes the default http client access back to neon which doesn't  
exhibit the problem.  Adding "http-library = serf" as described above  
will make the problem reappear with the 1.7 subversion-perl bindings  
(and the 1.6 as well).

Kyle

Re: git-svn "Temp file with moniker 'svn_delta' already in use" and skelta mode

[David Rothenberger]

On 7/5/2013 3:58 PM, Kyle McKay wrote:
> On Jul 5, 2013, at 13:48, David Rothenberger wrote:
>> I recently upgraded my Subversion server to 1.8.0 and started
>> receiving the following error from "git svn fetch":
>>
>> Temp file with moniker 'svn_delta' already in use at
>> /usr/lib/perl5/vendor_perl/5.10/Git.pm line 1024.
>>
>> This occurs only when using an http:// URL; svn:// URLs work fine.
> [snip]
>> The client is Cygwin: svn version 1.8.0 and git version
>> 1.8.3.2.
> 
> The subversion 1.8 release has removed the neon library, all svn client
> http access now always goes through the serf library.  If you put
> "http-library = serf" in the "[global]" section of the
> "~/.subversion/servers" file you will get the 'svn_delta' error with
> git-svn when running earlier versions of the svn client as well.

That was not my experience. I did try this with the 1.7 perl bindings
and libraries and with "http-library = serf" in by servers and it worked
fine. I confirmed that serf was being used with a Wireshark trace.

-- 
David Rothenberger  ----  daveroth@acm.org

To downgrade the human mind is bad theology.
                -- C. K. Chesterton

Re: git-svn "Temp file with moniker 'svn_delta' already in use" and skelta mode

[Kyle McKay]

On Jul 5, 2013, at 16:07, David Rothenberger wrote:
> On 7/5/2013 3:58 PM, Kyle McKay wrote:
>> On Jul 5, 2013, at 13:48, David Rothenberger wrote:
>>> I recently upgraded my Subversion server to 1.8.0 and started
>>> receiving the following error from "git svn fetch":
>>>
>>> Temp file with moniker 'svn_delta' already in use at
>>> /usr/lib/perl5/vendor_perl/5.10/Git.pm line 1024.
>>>
>>> This occurs only when using an http:// URL; svn:// URLs work fine.
>> [snip]
>>> The client is Cygwin: svn version 1.8.0 and git version
>>> 1.8.3.2.
>>
>> The subversion 1.8 release has removed the neon library, all svn  
>> client
>> http access now always goes through the serf library.  If you put
>> "http-library = serf" in the "[global]" section of the
>> "~/.subversion/servers" file you will get the 'svn_delta' error with
>> git-svn when running earlier versions of the svn client as well.
>
> That was not my experience. I did try this with the 1.7 perl bindings
> and libraries and with "http-library = serf" in by servers and it  
> worked
> fine. I confirmed that serf was being used with a Wireshark trace.

I had it always fail right away from my home ISP (kinda slow  
connection), but when running on a host with quite a different  
internet connection it would sometimes run for a while before  
generating the error (maybe a couple hundred revisions fetched) for  
some sources, but it would *always* eventually fail.

For this reason I also believe the problem is timing sensitive.

Try doing "git svn clone --quiet http://dev.heuristiclab.com/svn/hl/ 
core" with "~/.subversion/servers" section "[global]" including  
"http_library = serf".  I find this one always seems to fail right  
away for me with git-svn over serf but will clone fine over neon (if  
you can stand to wait long enough).

I have been looking into using serf for git-svn clones ever since I  
discovered how very much faster snvrdump works on an http url over  
serf as compared to neon.

Kyle

Re: git-svn "Temp file with moniker 'svn_delta' already in use" and skelta mode

[David Rothenberger]

On 7/5/2013 6:01 PM, Kyle McKay wrote:
> On Jul 5, 2013, at 16:07, David Rothenberger wrote:
>> On 7/5/2013 3:58 PM, Kyle McKay wrote:
>>> On Jul 5, 2013, at 13:48, David Rothenberger wrote:
>>>> I recently upgraded my Subversion server to 1.8.0 and started
>>>> receiving the following error from "git svn fetch":
>>>>
>>>> Temp file with moniker 'svn_delta' already in use at
>>>> /usr/lib/perl5/vendor_perl/5.10/Git.pm line 1024.
>>>>
>>>> This occurs only when using an http:// URL; svn:// URLs work fine.
>>> [snip]
>>>> The client is Cygwin: svn version 1.8.0 and git version
>>>> 1.8.3.2.
>>>
>>> The subversion 1.8 release has removed the neon library, all svn client
>>> http access now always goes through the serf library.  If you put
>>> "http-library = serf" in the "[global]" section of the
>>> "~/.subversion/servers" file you will get the 'svn_delta' error with
>>> git-svn when running earlier versions of the svn client as well.
>>
>> That was not my experience. I did try this with the 1.7 perl bindings
>> and libraries and with "http-library = serf" in by servers and it worked
>> fine. I confirmed that serf was being used with a Wireshark trace.
> 
> I had it always fail right away from my home ISP (kinda slow
> connection), but when running on a host with quite a different internet
> connection it would sometimes run for a while before generating the
> error (maybe a couple hundred revisions fetched) for some sources, but
> it would *always* eventually fail.
> 
> For this reason I also believe the problem is timing sensitive.
> 
> Try doing "git svn clone --quiet
> http://dev.heuristiclab.com/svn/hl/core" with "~/.subversion/servers"
> section "[global]" including "http_library = serf".  I find this one
> always seems to fail right away for me with git-svn over serf but will
> clone fine over neon (if you can stand to wait long enough).

Yes, you're right. That one fails for me too with 1.7, although I note
that the server itself is running 1.5. Still, the reason is the same --
a non-depth-first drive of the delta editor from serf.

-- 
David Rothenberger  ----  daveroth@acm.org

"If Diet Coke did not exist it would have been necessary to invent it."
                -- Karl Lehenbauer

Re: git-svn "Temp file with moniker 'svn_delta' already in use" and skelta mode

[David Rothenberger]

On 7/5/2013 1:48 PM, David Rothenberger wrote:
> I recently upgraded my Subversion server to 1.8.0 and started
> receiving the following error from "git svn fetch":
> 
> Temp file with moniker 'svn_delta' already in use at /usr/lib/perl5/vendor_perl/5.10/Git.pm line 1024.
> 
> This occurs only when using an http:// URL; svn:// URLs work fine.

I traced git-svn and discovered that the error is due to a known
problem in the SVN APIs. ra_serf does not drive the delta editor in
a depth-first manner as required by the API [1]. Instead, the calls
come in this order:

 1. open_root
 2. open_directory
 3. add_file
 4. apply_textdelta
 5. add_file
 6. apply_textdelta

This is a known issue [2] and one that the Subversion folks have
elected not to fix [3].

[1]
http://subversion.apache.org/docs/api/latest/structsvn__delta__editor__t.html#details
[2] http://subversion.tigris.org/issues/show_bug.cgi?id=2932
[3] http://subversion.tigris.org/issues/show_bug.cgi?id=3831

-- 
David Rothenberger  ----  daveroth@acm.org

management, n.:
        The art of getting other people to do all the work.

Re: git-svn "Temp file with moniker 'svn_delta' already in use" and skelta mode

[Kyle McKay]

On Jul 5, 2013, at 16:14, David Rothenberger wrote:
> On 7/5/2013 1:48 PM, David Rothenberger wrote:
>> I recently upgraded my Subversion server to 1.8.0 and started
>> receiving the following error from "git svn fetch":
>>
>> Temp file with moniker 'svn_delta' already in use at /usr/lib/perl5/ 
>> vendor_perl/5.10/Git.pm line 1024.
>>
>> This occurs only when using an http:// URL; svn:// URLs work fine.

I have created a patch (separate emails) that seems to work.  However,  
if the server being fetched against is not configured well for use  
with serf then "Error retrieving REPORT (54): Connection reset by peer  
Git/SVN/Ra.pm line 282" errors can occur and/or the fetch can be  
horribly slow compared to using neon.  If the "Connection reset" error  
occurs, the fetch can be restarted to get more revisions until it dies  
again.

Kyle

Re: git-svn "Temp file with moniker 'svn_delta' already in use" and skelta mode

[Branko Čibej]

[-- Attachment #1: Type: text/plain, Size: 1489 bytes --]

[Copying dev@ because it's related to a known issue that we document
more loudly.]

On 06.07.2013 00:51, David Rothenberger wrote:
> I cannot reproduce this problem using command-line tools, but I was
> able to do a trace of git-svn when it is failing and it looks to me
> that the problem is in the order in which the SVN::Delta::Editor
> (svn_delta_editor_t) function are being called.
>
> The order I see is:
>  1. open_root
>  2. open_directory
>  3. add_file
>  4. apply_textdelta
>  5. add_file
>  6. apply_textdelta
>
> The git-svn code is expecting a close_file call before the add_file
> call in #5. It appears to me that the svn_delta_editor_t API [1]
> requires this close_file call. It looks to me like this is Issue
> #2932 [2].

Indeed it is.

And it is actually documented here:

https://svn.apache.org/repos/asf/subversion/trunk/notes/api-errata/1.7/ra001.txt

and mentioned here:

http://subversion.apache.org/docs/release-notes/1.7.html#svnrdump
In other words, this is a limitation of the Serf-based backend that has
been around since Subversion 1.4. I'm aware that it isn't documented as
well as it should be, but the bulk-mode workaround exists in part as a
workaround for that, effectively disabling the more efficient HTTPv2
protocol.

In the meantime, it might be a good idea to relax the restrictions in
git-svn to account for the way the HTTPv2 protocol works.

-- Brane

-- 
Branko Čibej | Director of Subversion
WANdisco // Non-Stop Data
e. brane@wandisco.com

[-- Attachment #2: Type: text/html, Size: 2699 bytes --]

Re: git-svn "Temp file with moniker 'svn_delta' already in use" and skelta mode

[Daniel Shahaf]

On Sat, Jul 06, 2013 at 02:34:23AM +0200, Branko Čibej wrote:
> http://subversion.apache.org/docs/release-notes/1.7.html#svnrdump
> In other words, this is a limitation of the Serf-based backend that has
> been around since Subversion 1.4. I'm aware that it isn't documented as
> well as it should be, but the bulk-mode workaround exists in part as a
> workaround for that, effectively disabling the more efficient HTTPv2
> protocol.

Is it possible to set "SVNAllowBulkUpdates Prefer" on a per-client basis?

That would require two things: (1) for git-svn to identify itself as such via
the User-Agent string, (2) for httpd to support making the SVNAllowBulkUpdates
directive conditional on a request header.

Re: git-svn "Temp file with moniker 'svn_delta' already in use" and skelta mode

[Daniel Shahaf]

On Sat, Jul 06, 2013 at 02:04:07AM +0000, Daniel Shahaf wrote:
> On Sat, Jul 06, 2013 at 02:34:23AM +0200, Branko Čibej wrote:
> > http://subversion.apache.org/docs/release-notes/1.7.html#svnrdump
> > In other words, this is a limitation of the Serf-based backend that has
> > been around since Subversion 1.4. I'm aware that it isn't documented as
> > well as it should be, but the bulk-mode workaround exists in part as a
> > workaround for that, effectively disabling the more efficient HTTPv2
> > protocol.
> 
> Is it possible to set "SVNAllowBulkUpdates Prefer" on a per-client basis?

Actually, I'm approaching this from the wrong direction.  We have an
'http-bulk-updates' config knob, so an immediate workaround is for git-svn
to install --config-option=servers:global:http-bulk-updates=on in their client
context.

Re: git-svn "Temp file with moniker 'svn_delta' already in use" and skelta mode

[Branko Čibej]

[-- Attachment #1: Type: text/plain, Size: 799 bytes --]

On 06.07.2013 02:34, Branko Čibej wrote:
> In the meantime, it might be a good idea to relax the restrictions in
> git-svn to account for the way the HTTPv2 protocol works.

By the way, this section of the 1.8 release notes is relevant:

http://subversion.apache.org/docs/release-notes/1.8.html#neon-deleted

In 1.8 there is a client-side configuration option called
http-bulk-updates that controls how the client will request data from
the server. It can be set in the ~/.subversion/servers file, or on the
comand-line by the option

    --config-option=servers:global:http-bulk-updates=on

or, of course, in the client API context. git-svn should probably do the
latter as a simple workaround.

-- Brane

-- 
Branko Čibej | Director of Subversion
WANdisco // Non-Stop Data
e. brane@wandisco.com

[-- Attachment #2: Type: text/html, Size: 1845 bytes --]

Re: git-svn "Temp file with moniker 'svn_delta' already in use" and skelta mode

[Bert Huijben]

[-- Attachment #1: Type: text/plain, Size: 2024 bytes --]

Note that the commit logic in libsvn_client uses exactly the same driver pattern, but even in a more extreme way: it opens all nodes before closing the first file that will receive content changes.


Serf was only the first driver to do it this way in the other direction.


Bert






Sent from Windows Mail





From: Branko Čibej
Sent: ‎Saturday‎, ‎July‎ ‎6‎, ‎2013 ‎2‎:‎34‎ ‎AM
To: users@subversion.apache.org
Cc: Subversion Development; git@vger.kernel.org




[Copying dev@ because it's related to a known issue that we document more loudly.]

On 06.07.2013 00:51, David Rothenberger wrote: 

I cannot reproduce this problem using command-line tools, but I was
able to do a trace of git-svn when it is failing and it looks to me
that the problem is in the order in which the SVN::Delta::Editor
(svn_delta_editor_t) function are being called.

The order I see is:
 1. open_root
 2. open_directory
 3. add_file
 4. apply_textdelta
 5. add_file
 6. apply_textdelta

The git-svn code is expecting a close_file call before the add_file
call in #5. It appears to me that the svn_delta_editor_t API [1]
requires this close_file call. It looks to me like this is Issue
#2932 [2].

Indeed it is.

And it is actually documented here:

https://svn.apache.org/repos/asf/subversion/trunk/notes/api-errata/1.7/ra001.txt

and mentioned here:

http://subversion.apache.org/docs/release-notes/1.7.html#svnrdump


In other words, this is a limitation of the Serf-based backend that has been around since Subversion 1.4. I'm aware that it isn't documented as well as it should be, but the bulk-mode workaround exists in part as a workaround for that, effectively disabling the more efficient HTTPv2 protocol.

In the meantime, it might be a good idea to relax the restrictions in git-svn to account for the way the HTTPv2 protocol works.

-- Brane


-- 
Branko Čibej | Director of Subversion 
WANdisco // Non-Stop Data 
e. brane@wandisco.com

[-- Attachment #2: Type: text/html, Size: 4988 bytes --]