Re: git-svn bridge and line endings

Re: git-svn bridge and line endings

[Lucian Smith]

I'm attempting to use the git-svn bridge, and am having problems with
line endings on Windows.

The setup is that we have a git repository on github, and I've checked
out a branch on my Windows machine using Tortoise svn.  I make
changes, commit them, and the branch is updated.  In general, this
works fine.

If this was just SVN, I could set the 'eol-style' for files to
'native' to let it know to expect Windows/linux/mac line endings for
particular files.  This seems to be handled in git by using the
'.gitattributes' file instead.  Unfortunately, the git/svn bridge
doesn't seem to be translate the information in the .gitattributes
file to appropriate eol-style settings in SVN.  Checking out a file
using SVN on Windows leaves me with a file without CRLF's, and if I
check in a CRLF file, that's the way it goes into the repository.
Differences in CRLF alone show up as 'real' differences that can be
checked in, and, if this happens, this causes problems with other
people's repositories.

Am I doing something wrong; is there another way to handle this; or
can I file this as a bug report/feature request?

Thank you!

-Lucian Smith

Re: git-svn bridge and line endings

[Eric Wong]

Adding Alfred to the Cc:, more below...

Lucian Smith <lucianoelsmitho@gmail.com> wrote:
> I'm attempting to use the git-svn bridge, and am having problems with
> line endings on Windows.
> 
> The setup is that we have a git repository on github, and I've checked
> out a branch on my Windows machine using Tortoise svn.  I make
> changes, commit them, and the branch is updated.  In general, this
> works fine.
> 
> If this was just SVN, I could set the 'eol-style' for files to
> 'native' to let it know to expect Windows/linux/mac line endings for
> particular files.  This seems to be handled in git by using the
> '.gitattributes' file instead.  Unfortunately, the git/svn bridge
> doesn't seem to be translate the information in the .gitattributes
> file to appropriate eol-style settings in SVN.  Checking out a file
> using SVN on Windows leaves me with a file without CRLF's, and if I
> check in a CRLF file, that's the way it goes into the repository.
> Differences in CRLF alone show up as 'real' differences that can be
> checked in, and, if this happens, this causes problems with other
> people's repositories.
> 
> Am I doing something wrong; is there another way to handle this; or
> can I file this as a bug report/feature request?
> 
> Thank you!
> 
> -Lucian Smith

Lucian: Which version of git are you using?

As of git v2.3.0 and thanks to Alfred, the "git svn propset"
command exists for setting new props via git; however it
only got documented recently in v2.9.1.

It also seems to support setting props via "svn-properties"
in the .gitattributes file based on reading the code;
but I'm not familiar with this area, so I defer to Alfred.

Alfred: "svn-properties" isn't documented anywhere, yet;
is this something that should/could be documented?

Thanks.

cf. commit 83c9433e679635f8fbf8961081ea3581c93ca778
    ("git-svn: support for git-svn propset")
    https://bogomips.org/git-svn.git/commit/?id=83c9433e6796

    commit 19a7f24b6f8aa89ea5899c928c2fa350f4b1521e
    ("git-svn: document the 'git svn propset' command")
    https://bogomips.org/git-svn.git/commit/?id=19a7f24b6f8a

    https://public-inbox.org/git/?q=s:%22git-svn+propset%22

Anyways I've never used SVN props much myself, so don't much
have experience in this area; nor do I have much experience
with git-side gitattributes.

Re: git-svn bridge and line endings

[Alfred Perlstein]

On 8/22/16 8:07 PM, Eric Wong wrote:
> Adding Alfred to the Cc:, more below...
>
> Lucian Smith <lucianoelsmitho@gmail.com> wrote:
>> I'm attempting to use the git-svn bridge, and am having problems with
>> line endings on Windows.
>>
>> The setup is that we have a git repository on github, and I've checked
>> out a branch on my Windows machine using Tortoise svn.  I make
>> changes, commit them, and the branch is updated.  In general, this
>> works fine.
>>
>> If this was just SVN, I could set the 'eol-style' for files to
>> 'native' to let it know to expect Windows/linux/mac line endings for
>> particular files.  This seems to be handled in git by using the
>> '.gitattributes' file instead.  Unfortunately, the git/svn bridge
>> doesn't seem to be translate the information in the .gitattributes
>> file to appropriate eol-style settings in SVN.  Checking out a file
>> using SVN on Windows leaves me with a file without CRLF's, and if I
>> check in a CRLF file, that's the way it goes into the repository.
>> Differences in CRLF alone show up as 'real' differences that can be
>> checked in, and, if this happens, this causes problems with other
>> people's repositories.
>>
>> Am I doing something wrong; is there another way to handle this; or
>> can I file this as a bug report/feature request?
>>
>> Thank you!
>>
>> -Lucian Smith
> Lucian: Which version of git are you using?
>
> As of git v2.3.0 and thanks to Alfred, the "git svn propset"
> command exists for setting new props via git; however it
> only got documented recently in v2.9.1.
>
> It also seems to support setting props via "svn-properties"
> in the .gitattributes file based on reading the code;
> but I'm not familiar with this area, so I defer to Alfred.
>
> Alfred: "svn-properties" isn't documented anywhere, yet;
> is this something that should/could be documented?
>
> Thanks.
>
> cf. commit 83c9433e679635f8fbf8961081ea3581c93ca778
>      ("git-svn: support for git-svn propset")
>      https://bogomips.org/git-svn.git/commit/?id=83c9433e6796
>
>      commit 19a7f24b6f8aa89ea5899c928c2fa350f4b1521e
>      ("git-svn: document the 'git svn propset' command")
>      https://bogomips.org/git-svn.git/commit/?id=19a7f24b6f8a
>
>      https://public-inbox.org/git/?q=s:%22git-svn+propset%22
>
> Anyways I've never used SVN props much myself, so don't much
> have experience in this area; nor do I have much experience
> with git-side gitattributes.
>
Thank you.  I'm going to need some time to look into this.  The addition 
of svn-properties support was mostly to facilitate those that needed 
attributes set for other svn consumers or "repo meister" (the person 
that runs VCS).  An example being FreeBSD's 'Keyword expansion' on 
checkout facility, that says whether or not to expand $FreeBSD$.

I hadn't anticipated there be to translation between svn props and 
.gitattributes, it sounds a bit messy but possible, that said, is it not 
possible to commit .gitattribute files to the svn repo?  Even in FreeBSD 
land such small token files are permitted.

As far as documenting svn-properties, most of the properties are used on 
the Subversion side either by subversion itself, or by scripts in the 
subversion repository.  Perhaps a blurb "see the subversion 
documentation and/or your local subversion administrator's guide for 
properties and their uses." would suffice?

Opinions?  Happy to look into it.

-Alfred

Re: git-svn bridge and line endings

[Eric Wong]

Alfred Perlstein <alfred@freebsd.org> wrote:
> I hadn't anticipated there be to translation between svn props and
> .gitattributes, it sounds a bit messy but possible, that said, is it
> not possible to commit .gitattribute files to the svn repo?  Even in
> FreeBSD land such small token files are permitted.

I'm not sure if an automatic translation is necessary or
desired (because of a corruption risk).

Perhaps Lucian can clarify the situation for his repo.

> As far as documenting svn-properties, most of the properties are
> used on the Subversion side either by subversion itself, or by
> scripts in the subversion repository.  Perhaps a blurb "see the
> subversion documentation and/or your local subversion
> administrator's guide for properties and their uses." would suffice?

Yes, perhaps with a workable example Lucian can use today with
any git v2.3.0 or later.

Thanks for the quick response!

> Opinions?  Happy to look into it.

Re: git-svn bridge and line endings

[Lucian Smith]

Thanks for the quick responses!

My situation is that the git side is entirely whatever github.org is
running; presumably the latest stable version?  They provide a URL for
repositories hosted there that can be accessed by an SVN
client--somewhere on github is the 'git-svn bridge' (as I understand
it): something that receives SVN requests, translates them to
git-speak, and replies with what SVN expects.

There is indeed a .gitattributes file in the repository, but the SVN
client doesn't know what to do with it.  My hope was that something in
the bridge code, that translated SVN requests to git and back, could
take the SVN request, "Please give me this file; I'm on Windows" look
at the .gitattributes file in the repository, and hand out a file with
CR/LF's in it.  Conversely, when SVN tells git "Here is the new
version of the file to check in," the bridge could look at the file,
realize it had CR/LF's in it, look at the .gitattributes file to know
if it needed to be converted, and then convert it appropriately.

I can imagine a full-blown bridge that could even translate the SVN
EOL propset back and forth appropriately, but I'm not sure if going
that far is necessary and/or helpful.

I don't know if this is the right mailing list for that particular bit
of software, but it at least seemed like a good place to start.  Thank
you!

-Lucian

On Mon, Aug 22, 2016 at 10:54 PM, Eric Wong <e@80x24.org> wrote:
> Alfred Perlstein <alfred@freebsd.org> wrote:
>> I hadn't anticipated there be to translation between svn props and
>> .gitattributes, it sounds a bit messy but possible, that said, is it
>> not possible to commit .gitattribute files to the svn repo?  Even in
>> FreeBSD land such small token files are permitted.
>
> I'm not sure if an automatic translation is necessary or
> desired (because of a corruption risk).
>
> Perhaps Lucian can clarify the situation for his repo.
>
>> As far as documenting svn-properties, most of the properties are
>> used on the Subversion side either by subversion itself, or by
>> scripts in the subversion repository.  Perhaps a blurb "see the
>> subversion documentation and/or your local subversion
>> administrator's guide for properties and their uses." would suffice?
>
> Yes, perhaps with a workable example Lucian can use today with
> any git v2.3.0 or later.
>
> Thanks for the quick response!
>
>> Opinions?  Happy to look into it.

Re: git-svn bridge and line endings

[Torsten Bögershausen]

On Mon, Aug 22, 2016 at 05:04:47PM -0700, Lucian Smith wrote:
> I'm attempting to use the git-svn bridge, and am having problems with
> line endings on Windows.
> 
> The setup is that we have a git repository on github, and I've checked
> out a branch on my Windows machine using Tortoise svn.  I make
> changes, commit them, and the branch is updated.  In general, this
> works fine.

Just to make sure:
The repo is in git format.
Is it a public repo ?
Or could you make a piblic demo repo ?
Do I understand it right: Tortoise SVN talks directly to the Git server ?
Isn't Tortoise SVN a client to talk to SVN server?
What goes over the wire to the remote Git server, git or SVN ?

To my understanding, "git svn" can use Git locally, and talk to an SVN server.
What do I miss ?

> 
> If this was just SVN, I could set the 'eol-style' for files to
> 'native' to let it know to expect Windows/linux/mac line endings for
> particular files.  This seems to be handled in git by using the
> '.gitattributes' file instead.  Unfortunately, the git/svn bridge
> doesn't seem to be translate the information in the .gitattributes
> file to appropriate eol-style settings in SVN.  Checking out a file
> using SVN on Windows leaves me with a file without CRLF's, and if I
> check in a CRLF file, that's the way it goes into the repository.
> Differences in CRLF alone show up as 'real' differences that can be
> checked in, and, if this happens, this causes problems with other
> people's repositories.
> 
> Am I doing something wrong; is there another way to handle this; or
> can I file this as a bug report/feature request?

Re: git-svn bridge and line endings

[Lucian Smith]

On Tue, Aug 23, 2016 at 9:43 AM, Torsten Bögershausen <tboegi@web.de> wrote:
> On Mon, Aug 22, 2016 at 05:04:47PM -0700, Lucian Smith wrote:
>> I'm attempting to use the git-svn bridge, and am having problems with
>> line endings on Windows.
>>
>> The setup is that we have a git repository on github, and I've checked
>> out a branch on my Windows machine using Tortoise svn.  I make
>> changes, commit them, and the branch is updated.  In general, this
>> works fine.
>
> Just to make sure:
> The repo is in git format.
> Is it a public repo ?

Yes, it is.  In fact, it's https://github.com/sbmlteam/sbml-test-suite

> Or could you make a piblic demo repo ?
> Do I understand it right: Tortoise SVN talks directly to the Git server ?

This is the part that is a little bit of black magic for me, but yes:
I point TortoiseSVN at
https://github.com/sbmlteam/test-suite.git/branches/develop and tell
it to check that out, update, commit, etc.

> Isn't Tortoise SVN a client to talk to SVN server?
> What goes over the wire to the remote Git server, git or SVN ?

That's a good question ;-)  I think SVN!

> To my understanding, "git svn" can use Git locally, and talk to an SVN server.
> What do I miss ?

I assume something is using "git svn" on the github side of things,
but I don't know for sure, and I'm not sure who's in charge of that,
either.

-Lucian

>> If this was just SVN, I could set the 'eol-style' for files to
>> 'native' to let it know to expect Windows/linux/mac line endings for
>> particular files.  This seems to be handled in git by using the
>> '.gitattributes' file instead.  Unfortunately, the git/svn bridge
>> doesn't seem to be translate the information in the .gitattributes
>> file to appropriate eol-style settings in SVN.  Checking out a file
>> using SVN on Windows leaves me with a file without CRLF's, and if I
>> check in a CRLF file, that's the way it goes into the repository.
>> Differences in CRLF alone show up as 'real' differences that can be
>> checked in, and, if this happens, this causes problems with other
>> people's repositories.
>>
>> Am I doing something wrong; is there another way to handle this; or
>> can I file this as a bug report/feature request?
>

Re: git-svn bridge and line endings

[Junio C Hamano]

Lucian Smith <lucianoelsmitho@gmail.com> writes:

> The setup is that we have a git repository on github, and I've checked
> out a branch on my Windows machine using Tortoise svn.  I make
> changes, commit them, and the branch is updated.  In general, this
> works fine.

Hmph, doesn't Windows port ship with core.autocrlf these days?

Re: git-svn bridge and line endings

[Lucian Smith]

Windows doesn't have any problem reading files with any form of crlf.
The main issue we have run into thus far is that if I create a new
file in Windows, it's going to have crlf, and if I check it in that
way through this svn process, the file is then stored with crlfs,
which git does not expect.  A wide variety of problems results.

-Lucian

On Tue, Aug 23, 2016 at 10:26 AM, Junio C Hamano <gitster@pobox.com> wrote:
> Lucian Smith <lucianoelsmitho@gmail.com> writes:
>
>> The setup is that we have a git repository on github, and I've checked
>> out a branch on my Windows machine using Tortoise svn.  I make
>> changes, commit them, and the branch is updated.  In general, this
>> works fine.
>
> Hmph, doesn't Windows port ship with core.autocrlf these days?

Re: git-svn bridge and line endings

[Julian Phillips]

On 23/08/2016 17:14, Lucian Smith wrote:
> Thanks for the quick responses!
> 
> My situation is that the git side is entirely whatever github.org is
> running; presumably the latest stable version?  They provide a URL for
> repositories hosted there that can be accessed by an SVN
> client--somewhere on github is the 'git-svn bridge' (as I understand
> it): something that receives SVN requests, translates them to
> git-speak, and replies with what SVN expects.

The ability to use a Subversion client is functionality provided by 
GitHub, and not native to git itself.  So unless someone for the 
appropriate GitHub team happens to read this thread I expect there isn't 
much we can do to help.  I don't know if they've even provided any real 
detail of how they implemented the bridge.

> There is indeed a .gitattributes file in the repository, but the SVN
> client doesn't know what to do with it.  My hope was that something in
> the bridge code, that translated SVN requests to git and back, could
> take the SVN request, "Please give me this file; I'm on Windows" look
> at the .gitattributes file in the repository, and hand out a file with
> CR/LF's in it.  Conversely, when SVN tells git "Here is the new
> version of the file to check in," the bridge could look at the file,
> realize it had CR/LF's in it, look at the .gitattributes file to know
> if it needed to be converted, and then convert it appropriately.
> 
> I can imagine a full-blown bridge that could even translate the SVN
> EOL propset back and forth appropriately, but I'm not sure if going
> that far is necessary and/or helpful.
> 
> I don't know if this is the right mailing list for that particular bit
> of software, but it at least seemed like a good place to start.  Thank
> you!

Supported properties are listed here: 
https://help.github.com/articles/subversion-properties-supported-by-github/

You'll need to ask GitHub to implement support for the svn:eol-style 
property I expect.

Might be easier to just use Tortoise Git?

> -Lucian
> 
> On Mon, Aug 22, 2016 at 10:54 PM, Eric Wong <e@80x24.org> wrote:
>> Alfred Perlstein <alfred@freebsd.org> wrote:
>>> I hadn't anticipated there be to translation between svn props and
>>> .gitattributes, it sounds a bit messy but possible, that said, is it
>>> not possible to commit .gitattribute files to the svn repo?  Even in
>>> FreeBSD land such small token files are permitted.
>> 
>> I'm not sure if an automatic translation is necessary or
>> desired (because of a corruption risk).
>> 
>> Perhaps Lucian can clarify the situation for his repo.
>> 
>>> As far as documenting svn-properties, most of the properties are
>>> used on the Subversion side either by subversion itself, or by
>>> scripts in the subversion repository.  Perhaps a blurb "see the
>>> subversion documentation and/or your local subversion
>>> administrator's guide for properties and their uses." would suffice?
>> 
>> Yes, perhaps with a workable example Lucian can use today with
>> any git v2.3.0 or later.
>> 
>> Thanks for the quick response!
>> 
>>> Opinions?  Happy to look into it.
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

-- 
Julian

Re: git-svn bridge and line endings

[Lucian Smith]

On Tue, Aug 23, 2016 at 10:36 AM, Julian Phillips
<julian@quantumfyre.co.uk> wrote:
> On 23/08/2016 17:14, Lucian Smith wrote:
>>
>> Thanks for the quick responses!
>>
>> My situation is that the git side is entirely whatever github.org is
>> running; presumably the latest stable version?  They provide a URL for
>> repositories hosted there that can be accessed by an SVN
>> client--somewhere on github is the 'git-svn bridge' (as I understand
>> it): something that receives SVN requests, translates them to
>> git-speak, and replies with what SVN expects.
>
>
> The ability to use a Subversion client is functionality provided by GitHub,
> and not native to git itself.  So unless someone for the appropriate GitHub
> team happens to read this thread I expect there isn't much we can do to
> help.  I don't know if they've even provided any real detail of how they
> implemented the bridge.

All right, that makes sense.  And yeah, it was hard to find any
information about the bridge, which is why I ended up here...

>> There is indeed a .gitattributes file in the repository, but the SVN
>> client doesn't know what to do with it.  My hope was that something in
>> the bridge code, that translated SVN requests to git and back, could
>> take the SVN request, "Please give me this file; I'm on Windows" look
>> at the .gitattributes file in the repository, and hand out a file with
>> CR/LF's in it.  Conversely, when SVN tells git "Here is the new
>> version of the file to check in," the bridge could look at the file,
>> realize it had CR/LF's in it, look at the .gitattributes file to know
>> if it needed to be converted, and then convert it appropriately.
>>
>> I can imagine a full-blown bridge that could even translate the SVN
>> EOL propset back and forth appropriately, but I'm not sure if going
>> that far is necessary and/or helpful.
>>
>> I don't know if this is the right mailing list for that particular bit
>> of software, but it at least seemed like a good place to start.  Thank
>> you!
>
>
> Supported properties are listed here:
> https://help.github.com/articles/subversion-properties-supported-by-github/
>
> You'll need to ask GitHub to implement support for the svn:eol-style
> property I expect.

Thanks for finding that!  That even has an 'ask a human' button, which
I expect is my next step.

> Might be easier to just use Tortoise Git?

It may be!  But thanks for the responses anyway.

-Lucian

Re: git-svn bridge and line endings

[Torsten Bögershausen]

On 23.08.16 19:50, Lucian Smith wrote:
> On Tue, Aug 23, 2016 at 10:36 AM, Julian Phillips
> <julian@quantumfyre.co.uk> wrote:
>> On 23/08/2016 17:14, Lucian Smith wrote:
>>>
>>> Thanks for the quick responses!
>>>
>>> My situation is that the git side is entirely whatever github.org is
>>> running; presumably the latest stable version?  They provide a URL for
>>> repositories hosted there that can be accessed by an SVN
>>> client--somewhere on github is the 'git-svn bridge' (as I understand
>>> it): something that receives SVN requests, translates them to
>>> git-speak, and replies with what SVN expects.
>>
>>
>> The ability to use a Subversion client is functionality provided by GitHub,
>> and not native to git itself.  So unless someone for the appropriate GitHub
>> team happens to read this thread I expect there isn't much we can do to
>> help.  I don't know if they've even provided any real detail of how they
>> implemented the bridge.
> 
> All right, that makes sense.  And yeah, it was hard to find any
> information about the bridge, which is why I ended up here...
> 
>>> There is indeed a .gitattributes file in the repository, but the SVN
>>> client doesn't know what to do with it.  My hope was that something in
>>> the bridge code, that translated SVN requests to git and back, could
>>> take the SVN request, "Please give me this file; I'm on Windows" look
>>> at the .gitattributes file in the repository, and hand out a file with
>>> CR/LF's in it.  Conversely, when SVN tells git "Here is the new
>>> version of the file to check in," the bridge could look at the file,
>>> realize it had CR/LF's in it, look at the .gitattributes file to know
>>> if it needed to be converted, and then convert it appropriately.
>>>
>>> I can imagine a full-blown bridge that could even translate the SVN
>>> EOL propset back and forth appropriately, but I'm not sure if going
>>> that far is necessary and/or helpful.
>>>
>>> I don't know if this is the right mailing list for that particular bit
>>> of software, but it at least seemed like a good place to start.  Thank
>>> you!
>>
>>
>> Supported properties are listed here:
>> https://help.github.com/articles/subversion-properties-supported-by-github/
>>
>> You'll need to ask GitHub to implement support for the svn:eol-style
>> property I expect.
> 
> Thanks for finding that!  That even has an 'ask a human' button, which
> I expect is my next step.
> 
>> Might be easier to just use Tortoise Git?
> 
> It may be!  But thanks for the responses anyway.
Most text-files have been commited with LF:
/tmp/ttt/sbml-test-suite> git ls-files --eol | grep "i/lf" | wc -l
   10266
Some have been commited with CRLF:
/tmp/ttt/sbml-test-suite> git ls-files --eol | grep "i/crlf" | wc -l
    1620


The whole repo deserves to be normalized and equipped with a .gitattributes file,
see

https://www.kernel.org/pub/software/scm/git/docs/gitattributes.html