Git Community Book

Git Community Book

[Scott Chacon]

So I wanted to develop a really nice, easy to follow book for Git
newcomers to learn git quickly and easily.  One of the issues I
remember having when learning Git is that there is a lot of great
material in the User Guide, Tutorial, Tutorial 2, Everyday Git, etc -
but they're all huge long documents that are sometimes difficult to
come back to and remember where you were, and I didn't know which one
to start with or where to find what I was looking for, etc.

So, what I've started to do is pull material from all of them into a
single book which will be available in online HTML (one page per
chapter) and downloadable PDF form.  I'm trying to give it a very
organized flow that will hopefully be a bit easier to follow and
digest than the current formats, and including a number of diagrams,
illustrations and screencasts to supplement the text.  Where possible,
I am also trying to simplify the explanations a bit to be a tad more
digestible for beginning users, at least in the first couple dozen
chapters. I have put the current html output of this book here:

http://book.git-scm.com

It is not complete - the grey links are chapters that are very short
or completely empty - but it is a start.  Please let me know what you
think, and if anyone is interested in helping with the project, give
me a shout.

Also, for credit, I have generated an Authors page I will be linking
to the site soon that lists everyone that contributed a patch to any
of the Git User Guide, Git Tutorials, etc.  It is in the PDF right
now, but not in the HTML version yet (and the PDF is not yet linked to
the site).

Thanks,
Scott

Re: Git Community Book

[Miklos Vajna]

[-- Attachment #1: Type: text/plain, Size: 436 bytes --]

On Tue, Jul 29, 2008 at 09:20:20AM -0700, Scott Chacon <schacon@gmail.com> wrote:
> It is not complete - the grey links are chapters that are very short
> or completely empty - but it is a start.  Please let me know what you
> think, and if anyone is interested in helping with the project, give
> me a shout.

At http://github.com/schacon/learn-github/wikis/how-to-contribute, there
is a typo: you want 'git checkout origin/book'. ;-)

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: Git Community Book

[Petr Baudis]

On Tue, Jul 29, 2008 at 09:20:20AM -0700, Scott Chacon wrote:
> So, what I've started to do is pull material from all of them into a
> single book which will be available in online HTML (one page per
> chapter) and downloadable PDF form.  I'm trying to give it a very
> organized flow that will hopefully be a bit easier to follow and
> digest than the current formats, and including a number of diagrams,
> illustrations and screencasts to supplement the text.  Where possible,
> I am also trying to simplify the explanations a bit to be a tad more
> digestible for beginning users, at least in the first couple dozen
> chapters. I have put the current html output of this book here:
> 
> http://book.git-scm.com

I think what most of the people here would be also interested in is

	http://github.com/schacon/learn-github/wikis/how-to-contribute

There is no license in the source code - what are the copying terms?

It is maybe somewhat unfortunate that this is in a different format that
the standard git choice asciidoc, but the formats do look rather similar
so I assume it should not be hard to even convert from one to another if
needed.

Unfortunately, I probably won't have enough time to review the content
in details anytime soon, so I can only say that that the site looks
pretty. :-) I have skimmed through the Introduction part only, but
frankly, my feelings are somewhat mixed; I think the "direct dive-in"
you take in the Database and Index section is controversial at best, and
I personally much prefer the gentle approach of user manual, which does
not hurl details on git's objects model on the user right away. To me,
it would make sense to move this all somewhere between chapter four and
five. (Incidentally, only after writing this, I have looked at the
actual structure of the User Manual and I think it makes more sense than
your approach.)

So my confusion still is - where does this stand wrt. the user manual?
Why didn't you just start with the manual and work on that? I thought
you were planning to do that, but apparently we misunderstood each other
in the last mails.

Which goals are different between the Git Community Book and the User
Manual? It seems to me that the intent is the same in both cases, and if
the User Manual is not sufficiently digestible and easy to understand
for a newcomer, wouldn't it make more sense to make it so?

The thought of yet another Git resource _in addition_ to the existing
ones just makes me nervous. This isn't only about your time that I feel
is being spent unnecessarily ineffectively by not building upon the
existing text, but also about the _community_ resources - the user
manual has a great benefit that it was actually reviewed by the mailing
list so it will probably have quite smaller error rate than anything
you or me would write on our own, no matter how big Git expert you are.

I'm not saying you don't have good reasons to make the choice you did,
I just don't understand them yet - please help me here.

> So I wanted to develop a really nice, easy to follow book for Git
> newcomers to learn git quickly and easily.  One of the issues I
> remember having when learning Git is that there is a lot of great
> material in the User Guide, Tutorial, Tutorial 2, Everyday Git, etc -
> but they're all huge long documents that are sometimes difficult to
> come back to and remember where you were, and I didn't know which one
> to start with or where to find what I was looking for, etc.

So, one of your arguments is that the current material are huge long
documents that are difficult to come back to and remember where you
were. But if I'd split the User Manaul TOC to the same layout you use
for the Community Book, what is the difference here? It seems to me that
both would appear pretty much the same. Should I do a proof of concept?
;-)

> Also, for credit, I have generated an Authors page I will be linking
> to the site soon that lists everyone that contributed a patch to any
> of the Git User Guide, Git Tutorials, etc.  It is in the PDF right
> now, but not in the HTML version yet (and the PDF is not yet linked to
> the site).

So, right now you are basically taking existing material and rearranging
it? By what rules? What is the underlying idea of your approach, and why
is it better than the current structure of the user manual? Have you
considered how to perform this all so that you can easily get further
updates and corrections to the user manual?

-- 
				Petr "Pasky" Baudis
As in certain cults it is possible to kill a process if you know
its true name.  -- Ken Thompson and Dennis M. Ritchie

Re: Git Community Book

[Junio C Hamano]

"Scott Chacon" <schacon@gmail.com> writes:

> So I wanted to develop a really nice, easy to follow book for Git
> newcomers to learn git quickly and easily.  One of the issues I
> remember having when learning Git is that there is a lot of great
> material in the User Guide, Tutorial, Tutorial 2, Everyday Git, etc -
> but they're all huge long documents that are sometimes difficult to
> come back to and remember where you were, and I didn't know which one
> to start with or where to find what I was looking for, etc.

Interesting.  A few comments, before I get dragged into my day job fully.

[overall]

 - Some people mentioned that the necessity of reading through large
   volume of documentation can be reduced if they were divided by
   developer roles (similar to how Everyday does), e.g. people in
   individual contributor role does not have to learn integrator tools
   such as "am" in their first pass on the documentation.  Has the
   approach considered while developing this book?

 - The order of sections in "Working with Git" chapter somehow does not
   feel quite right, except that I'd agree that "Git on Windows" at the
   beginning is a very good idea (disclaimer. I do not use Windows
   myself). "StGIT" coming next was very understandable, but then
   "Capistrano"????  And no CVS section next to Subversion section?  Ruby
   before Perl or Python (I would have listed Perl, Python and then Ruby
   to avoid language wars.  That's the language age order, and it is even
   alphabetical)???

   Above "Capistrano" and "Ruby" comment shows the bias this TOC has (and
   my bias being different from the TOC's bias).  I'd imagine that
   Ruby-minded folks won't share the same reaction as I had.  What's the
   target audience of this book?  Git users in general, or primarily
   Ruby-minded subset?  If the latter, labeling this as "Community Book"
   may be misleading.

[http://book.git-scm.com/1_the_git_object_database.html]

 - The color of "blob" does not match the blob that is committed to eat
   trees at the top of your site ;-)

 - In a recent thread on the list, quite a lot of people seem to have
   found that teaching the low level details and plumbing first to the new
   people is detrimental.  Do you have response to that thread?

Re: Git Community Book

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> "Scott Chacon" <schacon@gmail.com> writes:
>
>> So I wanted to develop a really nice, easy to follow book for Git
>> newcomers to learn git quickly and easily.  One of the issues I
>> remember having when learning Git is that there is a lot of great
>> material in the User Guide, Tutorial, Tutorial 2, Everyday Git, etc -
>> but they're all huge long documents that are sometimes difficult to
>> come back to and remember where you were, and I didn't know which one
>> to start with or where to find what I was looking for, etc.
>
> Interesting.  A few comments, before I get dragged into my day job fully.

(cont'd)

I was planning to comment on the contents (i.e. text) but it appears that
most of the stuff was borrowed from the User Manual, so I won't.

	Side note: I wonder if this makes the whole "Community Book"
	GPLv2.  What would happen to the part that includes your own
	screencast?  You do not mind it be contaminated by our licence?

But there seems to be some stuff User Manual does not talk about.

[3_basic_branching_and_merging.html]

 - You've talked about low-level individual objects in an earlier section
   but you stopped at showing a single commit pointing at a tree.  People
   would find branching and merging very hard to get, without
   understanding the commit DAG.  On the other hand, you can explain
   commit DAG without going into details down to trees and blobs in the
   earlier section.  The user manual has "understanding reachability"
   section early on for this exact reason.

[5_creating_new_empty_branches.html]

 - As I repeatedly said on the list, I do not think teaching this is
   useful.  Multiple roots may happen as a result of pushing (or pulling)
   from a repository with unrelated root, but it is not something you
   would want to actively aim for.  At least there needs an explanation
   for the reason why making disjoint roots in the same repository is
   (sometimes) a good thing to do, and what its downsides are.

Re: Git Community Book

[Scott Chacon]

>
> There is no license in the source code - what are the copying terms?
>

I copied in the COPYING file from Git - GPL2.

> It is maybe somewhat unfortunate that this is in a different format that
> the standard git choice asciidoc, but the formats do look rather similar
> so I assume it should not be hard to even convert from one to another if
> needed.

I simply didn't want to get asciidoc working locally - it's always
been a bit of a pain to compile (I've heard it referred to more than
once as the only 'nightmare dependancy' in git), and I don't need to
make man pages or anything, so it seemed Markdown would be a better
choice for my output targets.  There are a number of good Markdown
interpreters and they're easy to get running.

>
> Unfortunately, I probably won't have enough time to review the content
> in details anytime soon, so I can only say that that the site looks
> pretty. :-) I have skimmed through the Introduction part only, but
> frankly, my feelings are somewhat mixed; I think the "direct dive-in"
> you take in the Database and Index section is controversial at best, and
> I personally much prefer the gentle approach of user manual, which does
> not hurl details on git's objects model on the user right away. To me,
> it would make sense to move this all somewhere between chapter four and
> five. (Incidentally, only after writing this, I have looked at the
> actual structure of the User Manual and I think it makes more sense than
> your approach.)
>
> So my confusion still is - where does this stand wrt. the user manual?
> Why didn't you just start with the manual and work on that? I thought
> you were planning to do that, but apparently we misunderstood each other
> in the last mails.
>

I was originally planning on doing that, but the problem is the
graphics, diagrams and screencasts.  Unless I am mistaken, there is
not a single outside media reference in any of these guides - the
diagrams that are there are all ascii drawings.  I'm assuming there is
a reason for that. If I wanted to add images and screencast embeds
into the guide, how would that work?

Also, the user guide seems much more technical than I wanted - I
wanted to simplify a lot of the explanations, especially at the
beginning, and I don't want to screw up all the existing text.  I
thought that the best solution would be to have the Community Book as
more of a book format, and the User Guide as more of an advanced
technical guide.  We don't want to put 'Git and Capistrano' or 'Using
Git in Perl' in the User Guide, do we?  I just wanted to copy the
sections that were already well written that need to be in both, so
that I don't have to re-write them.

> Which goals are different between the Git Community Book and the User
> Manual? It seems to me that the intent is the same in both cases, and if
> the User Manual is not sufficiently digestible and easy to understand
> for a newcomer, wouldn't it make more sense to make it so?

I think the goals are a bit different.  I think the User Manual is
helpful for people coming from the Linux/Perl hacker communities that
are more used to guides like that - who like things explained more
technically and possibly even think screencasts are stupid and an
ascii graph is just as understandable as a pretty one with rounded
corners and pastel colors.

I think my goal with the book is to create a book.  The length of a
book, readable one chapter at a time over several days, etc.  Also,
eventually, I want to make it bookmarkable, maybe add some interactive
quizzes at the end of each chapter, maybe add a comments section to
the end of each chapter, add a live search box, etc.  That just seems
so much different than the User Guide and Tutorials that it warrants a
different project, but so much of the content in the Guide is quality
that I didn't want to reinvent the wheel yet again.

> The thought of yet another Git resource _in addition_ to the existing
> ones just makes me nervous. This isn't only about your time that I feel
> is being spent unnecessarily ineffectively by not building upon the
> existing text, but also about the _community_ resources - the user
> manual has a great benefit that it was actually reviewed by the mailing
> list so it will probably have quite smaller error rate than anything
> you or me would write on our own, no matter how big Git expert you are.

Well, that's what the point of this is - to ask everyone to help me
review it, and possibly help me add to it.  The user manual is great,
but even I don't reference it very often because I find it difficult
to find content in it I need quickly.  As Git becomes more and more
popular, more and more resources will continue to come out - I did the
Peepcode mini-book, which sold over a thousand copies already, and
Pragmatic Programmers and O'Reilly both have Git books in the works,
too.  I was planning on a second book with Peepcode, but I thought it
would be better to do this instead.

I would love to develop a book that is totally open and rivals all of
those and is consistently up to date and allows the community to
interact.  I don't think it's really possible to get the User Guide
there very easily except in this way.


> So, one of your arguments is that the current material are huge long
> documents that are difficult to come back to and remember where you
> were. But if I'd split the User Manaul TOC to the same layout you use
> for the Community Book, what is the difference here? It seems to me that
> both would appear pretty much the same. Should I do a proof of concept?
> ;-)

Again, I started to do this, but the image references, screencast
embeds, and general different goal of the book, both in length and
scope, makes me think that is not the best way to go.

> So, right now you are basically taking existing material and rearranging
> it? By what rules? What is the underlying idea of your approach, and why
> is it better than the current structure of the user manual? Have you
> considered how to perform this all so that you can easily get further
> updates and corrections to the user manual?

I have thought about this a lot, and it comes from the talks and
training I've done with Git and the feedback I've gotten from that.
For one, I think it's very helpful to split up the chapters into
sections ('First Time', 'Basic Usage', 'Advanced Usage', etc) so users
of different skill levels can easily see which chapters may have
something for them at a glance.

The specific order I choose is very different from the User Guide and
is likely to bother a number of people, which you mentioned (and I'm
sure Dscho will _hate_) because I introduce the object model at the
beginning.  (I'm still working on that section, trying to simplify it
and add in some other diagrams and a short screencast I have that I
think will be helpful)  This is because I have had a lot of positive
feedback that primary frustration from people comes from them thinking
of Git as a super-better Subversion.  I would venture to say that
_most_ of the users coming to Git now are currently fluent in
Subversion.  Even if they are from Perforce or CVS (the other two ones
I will occasionally run into), their mental model of what an SCM does
is the same - delta storage.  I've found that by ridding them of that
notion off the bat, they have _far_ fewer problems and frustrations
with Git than when I just try to show them the first 10 commands in
sort of a cookbook style.  It's not a complicated model, it doesn't
take long to teach, and in _my personal_ experience (which is not to
say it's necessarily correct), it helps people the most in picking it
up and really loving the tool.

The book is built so that it is just as easy to start in the 'Basic
Usage' section and go back later, but if you're going to sit down and
just start reading, I think it would be better to explain why Git is
different at a fundamental level right off the bat.

Scott

Re: Git Community Book

[Junio C Hamano]

"Scott Chacon" <schacon@gmail.com> writes:

>>
>> There is no license in the source code - what are the copying terms?
>>
>
> I copied in the COPYING file from Git - GPL2.
>
>> It is maybe somewhat unfortunate that this is in a different format that
>> the standard git choice asciidoc, but the formats do look rather similar
>> so I assume it should not be hard to even convert from one to another if
>> needed.
>
> I simply didn't want to get asciidoc working locally - it's always
> been a bit of a pain to compile (I've heard it referred to more than
> once as the only 'nightmare dependancy' in git), and I don't need to
> make man pages or anything, so it seemed Markdown would be a better
> choice for my output targets.  There are a number of good Markdown
> interpreters and they're easy to get running.

I personally like markdown, but doesn't your refusal to work with existing
practices pose a significant problem, unless:

 (0) you do not consider it a goal to keep the documentation shipped with
     git and your book in sync; or

 (1) you have either markdown to asciidoc (or the other way around)
     converter; the book is written in markdown, and its conversion back
     to asciidoc is fed to Documentation as patches (or the other way
     around); or

 (2) somebody tries to find markdown to manpage, and we convert
     Documentation/ to markdown.

Or is this, "fork once and borrow reviewer's time, but never be able to
contribute back to the original text because the result is so different"
approach?

Re: Git Community Book

[Julian Phillips]

On Tue, 29 Jul 2008, Junio C Hamano wrote:

> "Scott Chacon" <schacon@gmail.com> writes:
>
>>>
>>> There is no license in the source code - what are the copying terms?
>>>
>>
>> I copied in the COPYING file from Git - GPL2.
>>
>>> It is maybe somewhat unfortunate that this is in a different format that
>>> the standard git choice asciidoc, but the formats do look rather similar
>>> so I assume it should not be hard to even convert from one to another if
>>> needed.
>>
>> I simply didn't want to get asciidoc working locally - it's always
>> been a bit of a pain to compile (I've heard it referred to more than
>> once as the only 'nightmare dependancy' in git), and I don't need to
>> make man pages or anything, so it seemed Markdown would be a better
>> choice for my output targets.  There are a number of good Markdown
>> interpreters and they're easy to get running.
>
> I personally like markdown, but doesn't your refusal to work with existing
> practices pose a significant problem, unless:
>
> (0) you do not consider it a goal to keep the documentation shipped with
>     git and your book in sync; or
>
> (1) you have either markdown to asciidoc (or the other way around)
>     converter; the book is written in markdown, and its conversion back
>     to asciidoc is fed to Documentation as patches (or the other way
>     around); or
>
> (2) somebody tries to find markdown to manpage, and we convert
>     Documentation/ to markdown.

Haven't used it personally, and without commenting on the "political" side 
of such an approach - there does exist at least one tool that claims to be 
able to convert from markdown to man: http://johnmacfarlane.net/pandoc/

> Or is this, "fork once and borrow reviewer's time, but never be able to
> contribute back to the original text because the result is so different"
> approach?
>
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

-- 
Julian

  ---
[The French Riviera is] a sunny place for shady people.
 		-- Somerset Maugham

Re: Git Community Book

[Junio C Hamano]

Julian Phillips <julian@quantumfyre.co.uk> writes:

> On Tue, 29 Jul 2008, Junio C Hamano wrote:
>
>> "Scott Chacon" <schacon@gmail.com> writes:
>>
>>> I simply didn't want to get asciidoc working locally - it's always
>>> been a bit of a pain to compile (I've heard it referred to more than
>>> once as the only 'nightmare dependancy' in git), and I don't need to
>>> make man pages or anything, so it seemed Markdown would be a better
>>> choice for my output targets.  There are a number of good Markdown
>>> interpreters and they're easy to get running.
>>
>> I personally like markdown, but doesn't your refusal to work with existing
>> practices pose a significant problem, unless:
>> ...
>> (2) somebody tries to find markdown to manpage, and we convert
>>     Documentation/ to markdown.
> 
> Haven't used it personally, and without commenting on the "political"
> side of such an approach - there does exist at least one tool that
> claims to be able to convert from markdown to man:
> http://johnmacfarlane.net/pandoc/

Oh, there is nothing political about this.  It is not like some of us is
employed by AsciiDoc company and defecting to markdown would cost
somebody's job or life ;-)

It is good to know that an option is availble to make it easier to go
back-and-forth, when/if it becomes necessary.

Re: Git Community Book

[Scott Chacon]

On Tue, Jul 29, 2008 at 10:43 AM, Junio C Hamano <gitster@pobox.com> wrote:
> "Scott Chacon" <schacon@gmail.com> writes:
>
>> So I wanted to develop a really nice, easy to follow book for Git
>> newcomers to learn git quickly and easily.  One of the issues I
>> remember having when learning Git is that there is a lot of great
>> material in the User Guide, Tutorial, Tutorial 2, Everyday Git, etc -
>> but they're all huge long documents that are sometimes difficult to
>> come back to and remember where you were, and I didn't know which one
>> to start with or where to find what I was looking for, etc.
>
> Interesting.  A few comments, before I get dragged into my day job fully.
>
> [overall]
>
>  - Some people mentioned that the necessity of reading through large
>   volume of documentation can be reduced if they were divided by
>   developer roles (similar to how Everyday does), e.g. people in
>   individual contributor role does not have to learn integrator tools
>   such as "am" in their first pass on the documentation.  Has the
>   approach considered while developing this book?
>

Not really - I'm assuming that everyone will have to be one of those
roles at some point - I'm mostly aiming at the smaller developers like
myself, and probably 90% of the Git users, who have 20 git projects
that they work on with 1-5 other people.  I am not aiming at the Linux
or Git developers that have to deal with a project with hundreds of
users - everyone is going to have to be a developer, participant,
integrator and administrator to some degree, so I wanted to introduce
those commands when you need them.  IE, 'gc' and 'fsck' are rarely
_needed_ by most users - you can work just fine for a really long time
without ever needing to run them, but they are first in the Everyday
list.  I'm ordering it roughly in the order that I've seen people need
certain commands.  I could be convinced otherwise on any of them,
though.

>  - The order of sections in "Working with Git" chapter somehow does not
>   feel quite right, except that I'd agree that "Git on Windows" at the
>   beginning is a very good idea (disclaimer. I do not use Windows
>   myself). "StGIT" coming next was very understandable, but then
>   "Capistrano"????  And no CVS section next to Subversion section?  Ruby
>   before Perl or Python (I would have listed Perl, Python and then Ruby
>   to avoid language wars.  That's the language age order, and it is even
>   alphabetical)???
>

This is basically just notes at this point.  I will likely re-arrange
them as they are written.  However, I would argue that there are
likely more Git people using Ruby than there are using Python, though
Perl might rival it.  Nearly every major Ruby project out there is now
using Git, whereas very few Python ones seem to be (possibly because
Mercurial is written in python) - however, in all honesty, I don't
really care what order they are in.

As for the Capistrano section - again it is demand.  I have had tons
and tons of questions about Capistrano and Git, and many thousands of
people use that combination or are beginning to.  Again though, I
don't care where it is - I would be happy to put it at the bottom of
the section.

>   Above "Capistrano" and "Ruby" comment shows the bias this TOC has (and
>   my bias being different from the TOC's bias).  I'd imagine that
>   Ruby-minded folks won't share the same reaction as I had.  What's the
>   target audience of this book?  Git users in general, or primarily
>   Ruby-minded subset?  If the latter, labeling this as "Community Book"
>   may be misleading.

The target audience are users being convinced by their friends to use
Git and I want to impress them with a well thought out and laid out,
comprehensive, easy to use website and book as their first experience,
and show them an easy and smooth path to switch their mind from
thinking in SVN/Perforce to thinking in Git.  The Ruby community is a
very large part of the current surge to Git right now, but I want the
book to be easily accessible and acceptable to all communities that
are doing that.

> [http://book.git-scm.com/1_the_git_object_database.html]
>
>  - The color of "blob" does not match the blob that is committed to eat
>   trees at the top of your site ;-)
>
>  - In a recent thread on the list, quite a lot of people seem to have
>   found that teaching the low level details and plumbing first to the new
>   people is detrimental.  Do you have response to that thread?
>

I think I addressed this in a previous response.  As for the blob
color, a number of diagrams I am planning to introduce initially are
from a talk I gave at RailsConf on Git, and I will likely go back over
them a bit later.

Thanks,
Scott

Re: Git Community Book

[Scott Chacon]

On Tue, Jul 29, 2008 at 11:25 AM, Junio C Hamano <gitster@pobox.com> wrote:
>
> (cont'd)
>
> I was planning to comment on the contents (i.e. text) but it appears that
> most of the stuff was borrowed from the User Manual, so I won't.
>
>        Side note: I wonder if this makes the whole "Community Book"
>        GPLv2.  What would happen to the part that includes your own
>        screencast?  You do not mind it be contaminated by our licence?
>
> But there seems to be some stuff User Manual does not talk about.

Yes, I'm using text from the User Manual and beginning to convert a
bunch of the text to make more sense in the new context, but it is
still under GPL2 and the screencasts are being linked to, not included
and distributed, so they shouldn't be affected.  However, they are
MIT, so you can pretty much do whatever you want with them.

> [3_basic_branching_and_merging.html]
>
>  - You've talked about low-level individual objects in an earlier section
>   but you stopped at showing a single commit pointing at a tree.  People
>   would find branching and merging very hard to get, without
>   understanding the commit DAG.  On the other hand, you can explain
>   commit DAG without going into details down to trees and blobs in the
>   earlier section.  The user manual has "understanding reachability"
>   section early on for this exact reason.
>
> [5_creating_new_empty_branches.html]
>
>  - As I repeatedly said on the list, I do not think teaching this is
>   useful.  Multiple roots may happen as a result of pushing (or pulling)
>   from a repository with unrelated root, but it is not something you
>   would want to actively aim for.  At least there needs an explanation
>   for the reason why making disjoint roots in the same repository is
>   (sometimes) a good thing to do, and what its downsides are.
>

Thank you for your feedback, I'll try to address both of these points
as I revise the book.

Scott

Re: Git Community Book

[Scott Chacon]

On Tue, Jul 29, 2008 at 11:42 AM, Junio C Hamano <gitster@pobox.com> wrote:
> "Scott Chacon" <schacon@gmail.com> writes:
>
>>>
>>> There is no license in the source code - what are the copying terms?
>>>
>>
>> I copied in the COPYING file from Git - GPL2.
>>
>>> It is maybe somewhat unfortunate that this is in a different format that
>>> the standard git choice asciidoc, but the formats do look rather similar
>>> so I assume it should not be hard to even convert from one to another if
>>> needed.
>>
>> I simply didn't want to get asciidoc working locally - it's always
>> been a bit of a pain to compile (I've heard it referred to more than
>> once as the only 'nightmare dependancy' in git), and I don't need to
>> make man pages or anything, so it seemed Markdown would be a better
>> choice for my output targets.  There are a number of good Markdown
>> interpreters and they're easy to get running.
>
> I personally like markdown, but doesn't your refusal to work with existing
> practices pose a significant problem, unless:
>
>  (0) you do not consider it a goal to keep the documentation shipped with
>     git and your book in sync; or
>
>  (1) you have either markdown to asciidoc (or the other way around)
>     converter; the book is written in markdown, and its conversion back
>     to asciidoc is fed to Documentation as patches (or the other way
>     around); or
>
>  (2) somebody tries to find markdown to manpage, and we convert
>     Documentation/ to markdown.
>
> Or is this, "fork once and borrow reviewer's time, but never be able to
> contribute back to the original text because the result is so different"
> approach?
>

The book is basically a fork of all three of the guides I mentioned
(User Manual and both Tutorials), and with the scope and goals I
currently have in mind, will not be kept in sync - it's just not going
to be possible.  I think in the end, the goals of the texts are so
very different that sections it will simply not make sense to try to
keep them in sync in some sort of automated fashion.  That's one of
the reasons why I choose Markdown - I saw no need to use asciidoc, as
the book will not be shipped around with Git or built using the same
processes, and I had no need for the advantages of asciidoc in my
project.  I don't think it makes much sense to have the book be a man
page at all.

However, I will watch the manual and guides and try to incorporate
changes to them as appropriate, and I will likely have some updates to
them myself as I've been more closely scrutinizing them.

Scott

Re: Git Community Book

[Junio C Hamano]

"Scott Chacon" <schacon@gmail.com> writes:

> The book is basically a fork of all three of the guides I mentioned
> (User Manual and both Tutorials), and with the scope and goals I
> currently have in mind, will not be kept in sync - it's just not going
> to be possible.  I think in the end, the goals of the texts are so
> very different that sections it will simply not make sense to try to
> keep them in sync in some sort of automated fashion.  That's one of
> the reasons why I choose Markdown - I saw no need to use asciidoc, as
> the book will not be shipped around with Git or built using the same
> processes, and I had no need for the advantages of asciidoc in my
> project.  I don't think it makes much sense to have the book be a man
> page at all.
>
> However, I will watch the manual and guides and try to incorporate
> changes to them as appropriate, and I will likely have some updates to
> them myself as I've been more closely scrutinizing them.

If that is the approach you decided for your book, I am Ok with that.

Not that you need my blessing to do your own book.  It was unclear what
your goals were, and if one of the goals were to keep the hassle of
maintaining shared materials in both manuals up-to-date, choice of
markdown seemed suboptimal to me, hence my comments.

Thanks.

... /me goes back to work after lunch break ...

Re: Git Community Book

[Daniel Barkalow]

On Tue, 29 Jul 2008, Scott Chacon wrote:

> So I wanted to develop a really nice, easy to follow book for Git
> newcomers to learn git quickly and easily.  One of the issues I
> remember having when learning Git is that there is a lot of great
> material in the User Guide, Tutorial, Tutorial 2, Everyday Git, etc -
> but they're all huge long documents that are sometimes difficult to
> come back to and remember where you were, and I didn't know which one
> to start with or where to find what I was looking for, etc.

It would be good to include stuff from 
http://eagain.net/articles/git-for-computer-scientists/

Maybe only in inspiration, since it doesn't have an obvious license and 
it's stylisticly more technical. But it would be nice to have diagrams of 
"this is what git thinks of as history", possibly even arranging them like 
gitk shows things (older downward, refs pointing in from the side).

In particular, I think it's really useful to show a commit graph with 
branching and merging, and introduce refs as movable pointers to commits 
in the graph, and local branches as refs that you move and tracking refs 
as refs that copy values in other repositories.

I think you can even gloss of details of blobs and trees because they 
pretty much work just like files and directories in a filesystem (except 
that they take up much less storage in large quantities than you'd think). 
The only potentially interesting things are (1) a blob names the inode, 
not the dentry, so it's the file contents, not the name, mode, etc; and 
(2) the permission bits are just 'x', we've got symlinks, there are no 
owner/group or other attributes and "see also Submodules".

But I think that the section:
  http://eagain.net/articles/git-for-computer-scientists/#history
should have an equivalent in any git documentation that can have diagrams, 
and introducing a history diagram style early means that you can do a 
bunch of simple pictures to explain operations like "git checkout -b foo" 
or "git reset --hard HEAD^^" or "git checkout origin/master".

	-Daniel
*This .sig left intentionally blank*

Re: Git Community Book

[Junio C Hamano]

Daniel Barkalow <barkalow@iabervon.org> writes:

> In particular, I think it's really useful to show a commit graph with 
> branching and merging, and introduce refs as movable pointers to commits 
> in the graph, and local branches as refs that you move and tracking refs 
> as refs that copy values in other repositories.

I'd very strongly second this.  If somebody is really into screencasts
(and especially from the Ruby circle, I would guess), this may be worth
a look:

    http://excess.org/article/2008/07/ogre-git-tutorial/

I saw a couple of technical inaccuracies in the presentation (I do not
expect any presentation or screencast to be perfect; I've never seen one
without any technical error anyway, perhaps other than my own at OLS a few
years ago), but otherwise it was very well done.  Espcially the part that
builds the commit ancestry chains I was very happy to see it taught like
so.

Re: Git Community Book

[Bart Trojanowski]

* Junio C Hamano <gitster@pobox.com> [080729 18:48]:
> I'd very strongly second this.  If somebody is really into screencasts
> (and especially from the Ruby circle, I would guess), this may be worth
> a look:
> 
>     http://excess.org/article/2008/07/ogre-git-tutorial/

Thank you very much for the plug, Junio.

> I saw a couple of technical inaccuracies in the presentation (I do not
> expect any presentation or screencast to be perfect; I've never seen one
> without any technical error anyway, perhaps other than my own at OLS a few
> years ago)

Could you let me know what the biggest inaccuracies were?  I would like
to correct my mistakes and update the slides.

Cheers,
-Bart

-- 
				WebSig: http://www.jukie.net/~bart/sig/

markdown 2 man, was Re: Git Community Book

[Johannes Schindelin]

Hi,

On Tue, 29 Jul 2008, Julian Phillips wrote:

> there does exist at least one tool that claims to be able
> to convert from markdown to man: http://johnmacfarlane.net/pandoc/

Just want to mention that it is written in Haskell, so chances are that it 
is even harder to install than asciidoc (I am thinking about non-Linux, 
of course).

Note also that Markdown cannot create TOCs automatically, AFAICT.  So 
probably it would be not all that easy to convert the User Manual to that 
format.

If at all, I would have preferred a format switch to Wiki syntax so that 
we can use the same source on the Git wiki as in our Documentation/ 
directory.  But such a switch could only come after a consensus of the 
Community anyway.

Ciao,
Dscho

Re: Git Community Book

[Bart Trojanowski]

* Junio C Hamano <gitster@pobox.com> [080729 18:48]:
> I'd very strongly second this.  If somebody is really into screencasts
> (and especially from the Ruby circle, I would guess), this may be worth
> a look:
> 
>     http://excess.org/article/2008/07/ogre-git-tutorial/

BTW, if anyone is interested in the SVGs used for the slides...

        git clone git://tachyon.jukie.net/intro-to-git.git/

You will need inkscape and latex-beamer to build the PDF.

Feel free to use them as you wish with attribution.  Although my may
want to wait till I fix the mistakes that Junio mentioned.

-Bart

-- 
				WebSig: http://www.jukie.net/~bart/sig/

Re: Git Community Book

[Junio C Hamano]

Bart Trojanowski <bart@jukie.net> writes:

> Could you let me know what the biggest inaccuracies were?  I would like
> to correct my mistakes and update the slides.

The one I offhand can recall was in your spoken part not on slides ("git
add -u" does not notice new files but does notice removed ones).  Nothing
major, really.

Re: markdown 2 man, was Re: Git Community Book

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> Note also that Markdown cannot create TOCs automatically, AFAICT.  So 
> probably it would be not all that easy to convert the User Manual to that 
> format.

The use of markdown may mean updates to the User Manual won't be merged
back to his book without effort and manual porting on his side, and the
side porting in the other direction has the same issue as well, but the
contents and the way materials are presented will be vastly different in
future versions anyway; not being able to side-port new material verbatim
may not be an issue.  Discussion with Scott seems to suggest that the
overall philosophy of his book is "this is a different book targetted for
different audiences; its initial text happens to heavily borrow from the
existing documents but expected to become vastly improved", in other
words, fork-and-never-return.

That's one valid approach.  I or you might have taken a different avenue,
but after all, it's his book, not mine, not yours, nor git list's book.

As I am not in "graphics and screencast" camp, I may probably not be able
to offer much help improving his book, and I suspect some people on this
list might feel the same way.  But that's is Ok --- we are not dumping the
User Manual.

We originally hoped (well, at least I did) that Scott's effort on his book
might help us in improving the User Manual as well, but the approach seems
to make it unlikely.  But that is nothing to hold against him --- he is
doing his own thing in a way he feels is the best, and that's perfectly
fine.  We lost nothing, perhaps except for a chance to cooperate a bit
better and to widen the community.

> If at all, I would have preferred a format switch to Wiki syntax so that 
> we can use the same source on the Git wiki as in our Documentation/ 
> directory.

Yeah, that's also true.  I seem to recall markdown was used in ikiwiki?

Re: Git Community Book

[J. Bruce Fields]

> > So my confusion still is - where does this stand wrt. the user manual?
> > Why didn't you just start with the manual and work on that? I thought
> > you were planning to do that, but apparently we misunderstood each other
> > in the last mails.
> >
On Tue, Jul 29, 2008 at 11:30:55AM -0700, Scott Chacon wrote:
> 
> I was originally planning on doing that, but the problem is the
> graphics, diagrams and screencasts.  Unless I am mistaken, there is
> not a single outside media reference in any of these guides - the
> diagrams that are there are all ascii drawings.  I'm assuming there is
> a reason for that. If I wanted to add images and screencast embeds
> into the guide, how would that work?
> 

Yeah, some possible obstacles:

	- Size: People probably won't want large binary blobs added to
	  the git repository.
	- Editability: We want to be able to keep the materials up to
	  date and accurate.
	- Source readability: the current documentation can all be read
	  in place without doing a build.
	- Build requirements: I seem to recall complaints about the
	  toolchain required to build the existing documentation.

At least for simple diagrams it might be possible to solve most of those
problems with an appropriate diagram-description-language that could be
compiled into image files.  Screencasts are probably totally out,
though.

In cases where you do find you're working with the same material, any
improvements you could contribute back to the in-tree documentation
would of course be appreciated.

> Well, that's what the point of this is - to ask everyone to help me
> review it, and possibly help me add to it.  The user manual is great,
> but even I don't reference it very often because I find it difficult
> to find content in it I need quickly.

If you had notes on any particular examples (I looked for X in place Y,
then place Z, and finally found it where I least expected it in place
Q...), they'd be appreciated.

> The specific order I choose is very different from the User Guide and
> is likely to bother a number of people, which you mentioned (and I'm
> sure Dscho will _hate_) because I introduce the object model at the
> beginning.  (I'm still working on that section, trying to simplify it
> and add in some other diagrams and a short screencast I have that I
> think will be helpful)  This is because I have had a lot of positive
> feedback that primary frustration from people comes from them thinking
> of Git as a super-better Subversion.
>
> I would venture to say that
> _most_ of the users coming to Git now are currently fluent in
> Subversion.  Even if they are from Perforce or CVS (the other two ones
> I will occasionally run into), their mental model of what an SCM does
> is the same - delta storage.  I've found that by ridding them of that
> notion off the bat, they have _far_ fewer problems and frustrations
> with Git than when I just try to show them the first 10 commands in
> sort of a cookbook style.  It's not a complicated model, it doesn't
> take long to teach, and in _my personal_ experience (which is not to
> say it's necessarily correct), it helps people the most in picking it
> up and really loving the tool.

I've considered doing the same for the user manual, actually, for some
of the same reasons--my main concern would be that it be done very
quickly, so as not to make people feel like it was a big obstacle on
their way to actually doing what they need to do.

So, anyway, that's to say that suggestions for reorganization of the
in-tree documentation (as opposed to just smaller-scale fixes) would
also be welcomed....

--b.

> 
> The book is built so that it is just as easy to start in the 'Basic
> Usage' section and go back later, but if you're going to sit down and
> just start reading, I think it would be better to explain why Git is
> different at a fundamental level right off the bat.
> 
> Scott
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: markdown 2 man, was Re: Git Community Book

[Wincent Colaiuta]

El 30/7/2008, a las 21:32, Junio C Hamano escribió:

> That's one valid approach.  I or you might have taken a different  
> avenue,
> but after all, it's his book, not mine, not yours, nor git list's  
> book.

Funnily enough, he chose to title it the "Git Community Book". Hard to  
match Scott's enthusiasm; this is the second major initiative we've  
seen from him in the last few days (the other being git-scm.com  
itself) which to the casual onlooker might look like the "official"  
Git homepage and documentation, but in both cases development occurred  
behind the scenes and the list was only notified after the fact.  
Better late than never I suppose.

> We originally hoped (well, at least I did) that Scott's effort on  
> his book
> might help us in improving the User Manual as well, but the approach  
> seems
> to make it unlikely.  But that is nothing to hold against him --- he  
> is
> doing his own thing in a way he feels is the best, and that's  
> perfectly
> fine.  We lost nothing, perhaps except for a chance to cooperate a bit
> better and to widen the community.

Even though there might not be an automated way to get changes back  
from the fork, if there are clear improvements made then there is at  
least no legal obstacle to incorporating them back in, the only  
obstacle would be time and willingness to do so manually.

>
Cheers,
Wincent

Re: markdown 2 man, was Re: Git Community Book

[Scott Chacon]

On Wed, Jul 30, 2008 at 4:48 PM, Wincent Colaiuta <win@wincent.com> wrote:
> El 30/7/2008, a las 21:32, Junio C Hamano escribió:
>
>> That's one valid approach.  I or you might have taken a different avenue,
>> but after all, it's his book, not mine, not yours, nor git list's book.
>
> Funnily enough, he chose to title it the "Git Community Book". Hard to match
> Scott's enthusiasm; this is the second major initiative we've seen from him
> in the last few days (the other being git-scm.com itself) which to the
> casual onlooker might look like the "official" Git homepage and
> documentation, but in both cases development occurred behind the scenes and
> the list was only notified after the fact. Better late than never I suppose.

Not sure what else I could have done - I announced that I was starting
a documentation project like this about a week ago on this list, then
I started the book 3 days ago
(http://github.com/schacon/gitscm/commits/book) and announced it here
for initial review yesterday.  I haven't told very many people about
it yet and I haven't linked to it from git-scm.com yet either.  It's
been open source from the first minute on GitHub, and the link to the
source was on the website I posted here.

Same for the git-scm site - I started it on the 23rd and emailed Pasky
about it the next day, and the day after that he began submitting
patches to me for it and I announced it on this list.  Am I missing
something here?  Do you think I've been working on these secretly for
months, or something?  If there is a better communication workflow, I
would be happy to do so.

I appreciate that you notice my enthusiasm, though. :)

Scott



>> We originally hoped (well, at least I did) that Scott's effort on his book
>> might help us in improving the User Manual as well, but the approach seems
>> to make it unlikely.  But that is nothing to hold against him --- he is
>> doing his own thing in a way he feels is the best, and that's perfectly
>> fine.  We lost nothing, perhaps except for a chance to cooperate a bit
>> better and to widen the community.
>
> Even though there might not be an automated way to get changes back from the
> fork, if there are clear improvements made then there is at least no legal
> obstacle to incorporating them back in, the only obstacle would be time and
> willingness to do so manually.
>
>>
> Cheers,
> Wincent
>
>

Re: markdown 2 man, was Re: Git Community Book

[Junio C Hamano]

Wincent Colaiuta <win@wincent.com> writes:

> Funnily enough, he chose to title it the "Git Community Book". Hard to  
> match Scott's enthusiasm; this is the second major initiative we've  
> seen from him in the last few days (the other being git-scm.com  
> itself) which to the casual onlooker might look like the "official"  
> Git homepage and documentation, but in both cases development occurred  
> behind the scenes and the list was only notified after the fact.  

I think your "Behind the scenes, after the fact" is being unnecessarily
harsh.

What counts is what happens now after the launch, when there are issues
identified that he could address on his side if he wanted to work with the
community.  "Ignore and fork forever" may be to further fracture the
community, but for a book like his that has quite different aim than the
official manual set, it might be a sensible approach.  You have to weigh
the pros and cons.

We've seen other comments raised to both the book and the git-scm.com site
on this list after they were announced.  We'll see how they are addressed
in coming weeks.  I think it is not too late to voice your negative
judgements only after seeing what happens.

Re: markdown 2 man, was Re: Git Community Book

[Abdelrazak Younes]

Hello,

Sorry for this irruption on this list. I am just a git user and casual 
reader of this list. I thought I could share my thoughts about this as I 
know a bit about document creation. Please ignore if this is not 
appropriate.

Disclaimer: I am involved in LyX development, so anything I said will be 
biased :-)

Junio C Hamano wrote:
> As I am not in "graphics and screencast" camp, I may probably not be able
> to offer much help improving his book, and I suspect some people on this
> list might feel the same way.  But that's is Ok --- we are not dumping the
> User Manual.

IMHO, documentation is best written by users, not developer. So, again 
IMHO, anything that could accommodate the _user_ for document writing 
should be done. An enthusiastic user is more likely to spend time 
writing documentation than a developer. For example, within the LyX 
project, most writers and translator are not developer.

Asciidoc or Markdown are tools that accommodate the _developer_, not the 
user. I understand that these markup language are ideally suited for in 
source documentation (thought I personally much prefer Doxygen). I also 
understand that launching a different application just to modify a line 
or two in the user manual seems cumbersome for the developer but IMHO, 
if you're serious about working on the documentation, you are not going 
to change a line or two and launching an external application is no big 
deal.

Now, about my shameless plug: LyX is ideally suited for structured 
documentation writing :-)

Abdel.

Re: markdown 2 man, was Re: Git Community Book

[Stephan Beyer]

Hi,

Abdelrazak Younes wrote:
> Please ignore if this is not appropriate.

Well, so I should've ignored, but I think this is worth some correction.

> Asciidoc or Markdown are tools that accommodate the _developer_, not the  
> user. I understand that these markup language are ideally suited for in  
> source documentation (thought I personally much prefer Doxygen).

http://www.methods.co.nz/asciidoc/ says
 ``AsciiDoc is a text document format for writing short documents,
   articles, books and UNIX man pages. AsciiDoc files can be translated to
   HTML and DocBook markups using the asciidoc(1) command.''

http://daringfireball.net/projects/markdown/ says
 ``Markdown is a text-to-HTML conversion tool for web writers. Markdown
   allows you to write using an easy-to-read, easy-to-write plain text
   format, then convert it to structurally valid XHTML (or HTML).''

So those are not suited for in-source documentation.

They're "lightweight" markup for documentation, very easy to read and somehow
easy to write for non-developers.
The user manual can give you an impression:
	http://repo.or.cz/w/git.git?a=blob;f=Documentation/user-manual.txt

I think, this is easier than LyX for users and developers..

Regards,
  Stephan

-- 
Stephan Beyer <s-beyer@gmx.net>, PGP 0x6EDDD207FCC5040F

Re: markdown 2 man, was Re: Git Community Book

[Abdelrazak Younes]

Stephan Beyer wrote:
> Hi,
>
> Abdelrazak Younes wrote:
>    
>> Please ignore if this is not appropriate.
>>      
>
> Well, so I should've ignored, but I think this is worth some correction.
>    
Thanks for the corrections :-)

> They're "lightweight" markup for documentation, very easy to read and somehow
> easy to write for non-developers.
> The user manual can give you an impression:
> 	http://repo.or.cz/w/git.git?a=blob;f=Documentation/user-manual.txt
>
> I think, this is easier than LyX for users and developers..
>    

Well, easier for short document writing maybe, better suited I don't 
think so, at least if you want to keep track of contents, structure, 
links, references, citations, etc. Bug again this is IMHO.

Abdel.

Re: markdown 2 man, was Re: Git Community Book

[Abdelrazak Younes]

Stephan Beyer wrote:
> They're "lightweight" markup for documentation, very easy to read and somehow
> easy to write for non-developers.
> The user manual can give you an impression:
> 	http://repo.or.cz/w/git.git?a=blob;f=Documentation/user-manual.txt
>
> I think, this is easier than LyX for users and developers..

I just had a look at the user manual and, well unless you have a special 
emacs mode or whatever that can automate the markup tag insertion, I 
wonder how can anybody think that writing with this markup language is 
easier than within LyX, really (genuine question, not sarcasm).

Abdel.

Re: markdown 2 man, was Re: Git Community Book

[Miklos Vajna]

[-- Attachment #1: Type: text/plain, Size: 616 bytes --]

On Thu, Jul 31, 2008 at 04:33:24PM +0200, Abdelrazak Younes <younes@lyx.org> wrote:
> I just had a look at the user manual and, well unless you have a special 
> emacs mode or whatever that can automate the markup tag insertion, I wonder 
> how can anybody think that writing with this markup language is easier than 
> within LyX, really (genuine question, not sarcasm).

People usually find it easy to contribute to a wiki, due to its easy
markup language.

asciidoc's markup is configurable, but the default one is really similar
to a wiki syntax, so at the end, people find it easy, including myself.

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: markdown 2 man, was Re: Git Community Book

[Abdelrazak Younes]

Hi Miklos,

Miklos Vajna wrote:
> On Thu, Jul 31, 2008 at 04:33:24PM +0200, Abdelrazak Younes<younes@lyx.org>  wrote:
>    
>> I just had a look at the user manual and, well unless you have a special
>> emacs mode or whatever that can automate the markup tag insertion, I wonder
>> how can anybody think that writing with this markup language is easier than
>> within LyX, really (genuine question, not sarcasm).
>>      
>
> People usually find it easy to contribute to a wiki, due to its easy
> markup language.
>    

I understand that but my point is that writing a book or a manual is too 
big a task for a wiki.

Anyway, if there is an interest to switch to LyX for the user manual, 
just let me know. Ascii has a LateX backend* and LyX can import LateX so 
the task should be easy.

* http://www.methods.co.nz/asciidoc/latex-backend.html

Thanks for answering :-)
Abdel.

Re: markdown 2 man, was Re: Git Community Book

[Miklos Vajna]

[-- Attachment #1: Type: text/plain, Size: 645 bytes --]

On Thu, Jul 31, 2008 at 05:29:13PM +0200, Abdelrazak Younes <younes@lyx.org> wrote:
> I understand that but my point is that writing a book or a manual is too 
> big a task for a wiki.

That's probably subjective. There is http://wikibooks.org/, after all.
;-)

> Anyway, if there is an interest to switch to LyX for the user manual, just 
> let me know. Ascii has a LateX backend* and LyX can import LateX so the 
> task should be easy.
> 
> * http://www.methods.co.nz/asciidoc/latex-backend.html

Last time I checked it was actually broken, but dblatex can transform
asciidoc's docbook output to latex, if that's really wished.

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: markdown 2 man, was Re: Git Community Book

[Jan Krüger]

Hi,

> Disclaimer: I am involved in LyX development, so anything I said will
> be biased :-)

I think that's fine since I consider LaTeX (and therefore LyX as the
best graphical editor for it that I know) a choice always worth
considering when it comes to projects that have the size of a book.

> Now, about my shameless plug: LyX is ideally suited for structured 
> documentation writing :-)

That may well be, but it gets really complicated once you want to
get your document into other markup-based formats while preserving all
the important aspects of formatting. I know this because I started
using LaTeX for a project that was supposed to be available in HTML
form along with, say, PDF. I've found that the only converter that
comes close to being useful for somewhat more ambitious sources
(including, perhaps, custom environments and stuff like that) without
spending a ridiculous amount of time trying to understand it is hevea.
Of course, hevea only translates to HTML, so, for example, generating
manpages or plain text is an entirely different matter of considerable
difficulty.

In addition to that, I suspect that LyX files might be difficult to
deal with in forky Git situations. For example, what if two
separately contributed patches need merging into a LyX source file?
This will only work automatically if the LyX source, treated as plain
text, has a really low chance of randomly changing in other places than
what the patch is supposed to touch. Also, if a merge does cause a
conflict, I imagine it would be difficult to resolve that.

Finally, it's pretty much a given that Git's manpages continue to use
AsciiDoc because there are few other things that can generate actual
manpages. I'm not sure it would be a good idea to keep half of Git's
documentation in one format and the rest in another. And AsciiDoc is --
by far! -- not the worst choice. I'm tempted to say it's the best that
I know.

-Jan

Re: markdown 2 man, was Re: Git Community Book

[Junio C Hamano]

Abdelrazak Younes <younes@lyx.org> writes:

> I just had a look at the user manual and, well unless you have a
> special emacs mode or whatever that can automate the markup tag
> insertion, I wonder how can anybody think that writing with this
> markup language is easier than within LyX, really (genuine question,
> not sarcasm).

How greppable and "log -p"-able is the documentation written in LyX?

 * Where in the documentation do I have to change the description of
   "--parents" option?

 * When did the description of "--cc" for diff families last changed, by
   whom and why?

Eas of doing these is mostly why we chose AsciiDoc to begin with.  Any
alternative you are going to suggest should not make these two things
impossible or very harder to do.

Re: markdown 2 man, was Re: Git Community Book

[Abdelrazak Younes]

[-- Attachment #1: Type: text/plain, Size: 1496 bytes --]

Junio C Hamano wrote:
> Abdelrazak Younes<younes@lyx.org>  writes:
>
>    
>> I just had a look at the user manual and, well unless you have a
>> special emacs mode or whatever that can automate the markup tag
>> insertion, I wonder how can anybody think that writing with this
>> markup language is easier than within LyX, really (genuine question,
>> not sarcasm).
>>      
>
> How greppable and "log -p"-able is the documentation written in LyX?
>    

LyX format is plain text, loosely based on LateX. Here's attached a 
sample .lyx file FYI. We have one tag per line and a maximum of 80 char 
per line so that the format is easily parsable. Advanced users often use 
unix tools (grep, sed, etc) to modify the .lyx file manually.

>   * Where in the documentation do I have to change the description of
>     "--parents" option?
>    

You mean in a text editor, not within LyX? Just look for the string :-)

>   * When did the description of "--cc" for diff families last changed, by
>     whom and why?
>    

Ditto.

> Eas of doing these is mostly why we chose AsciiDoc to begin with.  Any
> alternative you are going to suggest should not make these two things
> impossible or very harder to do.
>    

If you ignore the LyX tags, you can just do what you are used to do 
without problem using a plain text editor. If you want to do something 
more complicated stuff like choosing a different environment or creating 
a nested enumerate list, it is easier to do that within LyX.

Abdel.


[-- Attachment #2: FAQ.lyx --]
[-- Type: application/lyx, Size: 35350 bytes --]

Re: markdown 2 man, was Re: Git Community Book

[Abdelrazak Younes]

Hi Jan,

Jan Krüger wrote:
>> Now, about my shameless plug: LyX is ideally suited for structured
>> documentation writing :-)
>
> That may well be, but it gets really complicated once you want to
> get your document into other markup-based formats while preserving all
> the important aspects of formatting. I know this because I started
> using LaTeX for a project that was supposed to be available in HTML
> form along with, say, PDF. I've found that the only converter that
> comes close to being useful for somewhat more ambitious sources
> (including, perhaps, custom environments and stuff like that) without
> spending a ridiculous amount of time trying to understand it is hevea.

I had good success with htlatex (the default converter within LyX). I 
just modified the css and was done with it. All cross-references etc 
were correctly handled.

> Of course, hevea only translates to HTML, so, for example, generating
> manpages or plain text is an entirely different matter of considerable
> difficulty.

LyX has an excellent plain text export. You can use the export method of 
LyX at the command line without launching it graphically by the way. You 
don't even need an X server, just use 'lyx -e text mydocument.lyx'

For man page, LyX does not support it natively I'm afraid, but I guess 
there are LateX to man converter, aren't there?

> In addition to that, I suspect that LyX files might be difficult to
> deal with in forky Git situations. For example, what if two
> separately contributed patches need merging into a LyX source file?
> This will only work automatically if the LyX source, treated as plain
> text, has a really low chance of randomly changing in other places than
> what the patch is supposed to touch. Also, if a merge does cause a
> conflict, I imagine it would be difficult to resolve that.

Not really. As I said to Junio, .lyx files are using a plain text utf8 
format. They are easily mergeable as LyX preserves the structure of the 
file: if the two collaborators modify two different parts of the 
document there is basically zero chance to have a conflict. On the rare 
occasion where I had  a conflict with svn, it was very easy to solve 
manually by removing the conflict tags inserted by svn. With git, I 
never had a single conflict ;-)

> Finally, it's pretty much a given that Git's manpages continue to use
> AsciiDoc because there are few other things that can generate actual
> manpages. I'm not sure it would be a good idea to keep half of Git's
> documentation in one format and the rest in another.

That's a good argument. My personal opinion is that users prefer to use 
'-help' for short help and to read the tutorial or the user guide for 
more in-depth information. I never use man personally... OK, that's 
probably because I use Windows :-)

> And AsciiDoc is --
> by far! -- not the worst choice. I'm tempted to say it's the best that
> I know.

AsciiDoc is indeed excellent if you want to write in a plain text 
editor. But LyX is easier to use and more porwerful :-)

Thanks,
Abdel

Re: markdown 2 man, was Re: Git Community Book

[Thomas Rast]

[-- Attachment #1: Type: text/plain, Size: 1250 bytes --]

You wrote:
> Junio C Hamano wrote:
> >
> > How greppable and "log -p"-able is the documentation written in LyX?
> 
> LyX format is plain text, loosely based on LateX. Here's attached a 
> sample .lyx file FYI. We have one tag per line and a maximum of 80 char 
> per line so that the format is easily parsable. Advanced users often use 
> unix tools (grep, sed, etc) to modify the .lyx file manually.

Is it just me or is the format very hard to read?  For example, line
492ff spells a list of quoted items as

    \begin_layout Standard
    Generally, you would send email to lyx-foo-subscribe@lists.lyx.org to subscribe
     to these lists or to lyx-foo-unsubscribe@lists.lyx.org to unsubscribe, where
     
    \begin_inset Quotes eld
    \end_inset

    foo
    \begin_inset Quotes erd
    \end_inset

     is one of 
    \begin_inset Quotes eld
    \end_inset

    announce
    \begin_inset Quotes erd
    \end_inset

etc.  Of course I can "parse" the language, but my untrained eye is
unable to fluently read the text hiding behind it.

Also, if I made a commit changing the "announce", you would have to
turn up diff context to at least 13 lines to get any _semantic_
context of the change.

- Thomas


[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: markdown 2 man, was Re: Git Community Book

[Abdelrazak Younes]

Thomas Rast wrote:
> You wrote:
>    
>> Junio C Hamano wrote:
>>      
>>> How greppable and "log -p"-able is the documentation written in LyX?
>>>        
>> LyX format is plain text, loosely based on LateX. Here's attached a
>> sample .lyx file FYI. We have one tag per line and a maximum of 80 char
>> per line so that the format is easily parsable. Advanced users often use
>> unix tools (grep, sed, etc) to modify the .lyx file manually.
>>      
>
> Is it just me or is the format very hard to read?  For example, line
> 492ff spells a list of quoted items as
>    

Right, quote is a special case in lyx format because we have to take 
care of locale differences. So, as you guessed, quotes are not really 
written with the ascii quote character. But the format is not that hard 
in general. If needed, I could modify this special case so that it's 
easier to read though.
Don't get me wrong, I don't pretend that LyX is easy to read for the 
untrained eyes, it is not. But simple modifications like Junio's example 
is definitely possible. For non simple text insertion, it is better to 
launch LyX and to type the modification within LyX. But maybe this is a 
showstopper for you, and so is maybe our treatment of quotes. In which 
case I'll stop arguying :-)

Abdel.

Re: markdown 2 man, was Re: Git Community Book

[Dmitry Potapov]

On Fri, Aug 1, 2008 at 11:50 AM, Abdelrazak Younes <younes@lyx.org> wrote:
>
> AsciiDoc is indeed excellent if you want to write in a plain text editor.
> But LyX is easier to use and more porwerful :-)

What is really powerful is TeX. As to LyX, it is leaky abstraction over it.
I have never been able to use without ending up saying, it is so much easier
and much more powerful to use Latex than trying to do the same with LyX. Of
course, LyX looks much better nowadays than used to be, so I decided to give
it another try, and here is my fifteen minutes experience with it.

First, I tried to open FAQ.lyx that you attached to your previous email, and
here is what I see:

===
/tmp/FAW.lix is from a different version of LyX, but the lex2lex script failed
to covert it.
===

This is result was received with two LyX versions that I tried:
LyX Version 1.4.3 (21/09/2006)
LyX 1.5.5 (Sun, May 11, 2008)

Now, I see, that your FAQ was created with LyX 1.6.0svn, which is not released
yet. So, I hope that this issue will be correctly before it will be released.
Otherwise, anyone opening document with 1.6.0 will make it unaccessible to users
of previous versions.

Then I tried to use Formatted reference and everything looks okay until I tried
to generate DVI file, where I was welcome but the following error:
===
Paragraph ended before \@prettyref was complete.
===

What is \@prettyref? What is wrong with my paragraph? Actually, my paragraph is
fine, it is just when you use Formatted reference, you should know that it is
implemented using prettyref TeX package, which requires three letter prefix in
name of each label. Why did not LyX warn me about that? BTW, is really prettyref
is the best package for this job anyway? I remember some TeX experts recommended
some other packages for references.

Finally, I still have not figured out how to the same what AsciiDoc does:
Chapter #, $CHAPTER_NAME
It does not look like that LyX can produce references in this format.

The I tried to insert some verbatim text, and I cannot find the standard way
to do that in LyX. Sure, I can press CTRL-L and type in TeX:
\begin{verbatim}
        # git itself (approx. 10MB download):
$ git clone git://git.kernel.org/pub/scm/git/git.git
        # the linux kernel (approx. 150MB download):
$ git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git
\end{verbatim}

but I don't think that having a lot TeX code is going to help us with
having good formatted HTML version.

BTW, it is really annoying to see TeX code displayed in proportional
fonts and formatted with full adjustment. For instance, the last line
was displayed like this:

$                               git                              clone
git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git


Another rather surprising experience for those who got used to HTML:
Left-click on a reference produces its properties, while the right
click means to go to the label, and once you jump on it, there is
no way to jump back (at least, I was not able to find how to do that).

Well, I wrote all above only because I hope that LyX will continue to
improve. It looks much better now than before. Yet, I will rather stay
with plain text editors for now. Some of them are much more powerful
than Notepad :)


Dmitry

Re: markdown 2 man, was Re: Git Community Book

[Abdelrazak Younes]

Hi Dimitry,

Dmitry Potapov wrote:
>  On Fri, Aug 1, 2008 at 11:50 AM, Abdelrazak Younes <younes@lyx.org>
>  wrote:
> > AsciiDoc is indeed excellent if you want to write in a plain text
> > editor. But LyX is easier to use and more porwerful :-)
>
>  What is really powerful is TeX. As to LyX, it is leaky abstraction
>  over it. I have never been able to use without ending up saying, it
>  is so much easier and much more powerful to use Latex than trying to
>  do the same with LyX. Of course, LyX looks much better nowadays than
>  used to be, so I decided to give it another try, and here is my
>  fifteen minutes experience with it.

I was afraid this thread will turn into a pro and con of LyX versus 
plain LateX :-)


>  First, I tried to open FAQ.lyx that you attached to your previous
>  email, and here is what I see:
...
>  Now, I see, that your FAQ was created with LyX 1.6.0svn, which is not
>  released yet. So, I hope that this issue will be correctly before it
>  will be released.

Of course. Sorry, as I use the pre-release I didn't think that about 
that. FYI, we will release one last version of 1.5.x that is able to 
read 1.6 format. 1.6 will is of course able to read all previous format.

>  Otherwise, anyone opening document with 1.6.0 will make it
>  unaccessible to users of previous versions.
>
>  Then I tried to use Formatted reference and everything looks okay
>  until I tried to generate DVI file, where I was welcome but the
>  following error: === Paragraph ended before \@prettyref was
>  complete. ===
>
>  What is \@prettyref? What is wrong with my paragraph? Actually, my
>  paragraph is fine, it is just when you use Formatted reference, you
>  should know that it is implemented using prettyref TeX package, which
>  requires three letter prefix in name of each label. Why did not LyX
>  warn me about that? BTW, is really prettyref is the best package for
>  this job anyway? I remember some TeX experts recommended some other
>  packages for references.

Aha, yes you're right. LyX will automatically insert those three letters 
(eg. 'cha' for chapter). This is the reason why I never came across this 
bug. We'll try to fix that, thanks!

>
>  Finally, I still have not figured out how to the same what AsciiDoc
>  does: Chapter #, $CHAPTER_NAME It does not look like that LyX can
>  produce references in this format.

You can choose among a number of document class. If you want the 
"Chapter" prefixing, choose the 'Book' document class. The default,  
document class is 'Article', for with you don't have level 1 sections.

>  The I tried to insert some verbatim text, and I cannot find the
>  standard way to do that in LyX.

There are at least two:
- The LyX-code environment
- The listing inset

The listing inset supports a number of languages so you'll be able to 
have syntax highlighting and cloring for your language of choice.

>  Sure, I can press CTRL-L and type in
>  TeX: \begin{verbatim} # git itself (approx. 10MB download): $ git
>  clone git://git.kernel.org/pub/scm/git/git.git # the linux kernel
>  (approx. 150MB download): $ git clone
>  git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git
>  \end{verbatim}
>
>  but I don't think that having a lot TeX code is going to help us
>  with having good formatted HTML version.

No, either LyX-code (To choose from the Layout combo box) or preferable 
the Listing inset (Menu Insert -> Program Listing). Of course, all these 
action have keyboard shortcuts.

>
>  BTW, it is really annoying to see TeX code displayed in proportional
>  fonts and formatted with full adjustment. For instance, the last
>  line was displayed like this:
>
>  $                               git
>  clone
>  git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git

Yes I know, this will be better in 1.6 due out this month in principle.

>  Another rather surprising experience for those who got used to HTML:
>  Left-click on a reference produces its properties, while the right
>  click means to go to the label, and once you jump on it, there is no
>  way to jump back (at least, I was not able to find how to do that).

There is one 'Ctrl-0' but this is more or less hidden feature. 1.6 will 
have context menu so all the above actions will be a lot more consistant 
and easier.

>  Well, I wrote all above only because I hope that LyX will continue
>  to improve. It looks much better now than before.

Thanks for the comments :-)

>  Yet, I will rather
>  stay with plain text editors for now. Some of them are much more
>  powerful than Notepad :)

It's a matter of choice. I have to confess that I don't use plain text 
editor anymore because I am so used to LyX keybindings.

Thanks,
Abdel.

Git Community Book

[Scott Chacon]

Hey all,

I just wanted to let those of you who are interested know that I've
been making a lot of progress on the Git Community Book
(http://book.git-scm.com)  I was wondering if anyone was interested in
helping me with a few parts.  For one, there are some sections that I
personally have very little experience with, and was looking for some
notes/blog posts/personal experiences on, namely Advanced History
Modification (filter-branch, advanced rebasing, etc), Corruption
Recovery, Branch Tracking, Subversion Integration, Git with
Perl/Python/PHP, and Using Git with Editors (especially
NetBeans/Eclipse).

Also, the last section of the book is on some of the plumbing - mostly
stuff I've found difficult to pick up with the existing documentation
while re-implementing stuff in Ruby.  I would really appreciate it if
someone could proofread some of these chapters for errors:

http://book.git-scm.com/7_the_packfile.html
http://book.git-scm.com/7_raw_git.html
http://book.git-scm.com/7_transfer_protocols.html

Some of the next things I'm interested in producing is a cookbook
style guide and some searching tools for all the online documentation,
just to keep everyone up to date on where I'm going with the project.
Also, there is now a simple PDF downloadable version of the book
available and being kept up to date with the html version.

Thanks,
Scott

Re: Git Community Book

[Thomas Adam]

2008/9/5 Scott Chacon <schacon@gmail.com>:
> Hey all,
>
> I just wanted to let those of you who are interested know that I've
> been making a lot of progress on the Git Community Book
> (http://book.git-scm.com)  I was wondering if anyone was interested in

I'm going to bite and ask the obvious questions:

1.  How does what you're producing differ from the current Git Users' Manual?
2.  Is this project of yours aiming to obsolete the Git Users' Manual
with "official" sanctioning from people involved with Git?
3.  Assuming 2 is a "no", patches to the Users' Guide would be nice.  :)

-- Thomas Adam

Re: Git Community Book

[Junio C Hamano]

"Scott Chacon" <schacon@gmail.com> writes:

> Also, the last section of the book is on some of the plumbing - mostly
> stuff I've found difficult to pick up with the existing documentation
> while re-implementing stuff in Ruby.  I would really appreciate it if
> someone could proofread some of these chapters for errors:
>
> http://book.git-scm.com/7_the_packfile.html

Nice pictures.  You might also want to know that code for reading pack idx
version 2 was backported to 1.4.4.5 for people who are stuck on 1.4.4
series for whatever reason.

What is the target audience of this section?  If it is written for a mere
curious type, or if it is written to give "here is the general idea, for
more details read the source", the level of detail here would be Ok.

If you are writing for people who want to (re)implement something that
produces these files, you might want to at least say that offset/sha1[]
table is sorted by sha1[] values (this is to allow binary search of this
table), and fanout[] table points at the offset/sha1[] table in a specific
way (so that part of the latter table that covers all hashes that start
with a given byte can be found to avoid 8 iterations of the binary
search).

<data> part is just zlib stream for non-delta object types; for the two
delta object representations, the <data> portion contains something that
identifies which base object this delta representation depends on, and the
delta to apply on the base object to resurrect this object.  ref-delta
uses 20-byte hash of the base object at the beginning of <data>, while
ofs-delta stores an offset within the same packfile to identify the base
object.  In either case, two important constraints a reimplementor must
adhere to are:

 * delta representation must be based on some other object within the same
   packfile;

 * the base object must be of the same underlying type (blob, tree, commit
   or tag);

> http://book.git-scm.com/7_raw_git.html

I am guessing this is for Porcelain writers who use plumbing.  Please
don't teach echoing into .git/refs/...  but DO teach using update-ref with
the -m option.  We do not want people's random Porcelains flipping the tip
of branches without leaving trail in reflog for users to use to recover
from mistakes.

Re: Git Community Book

[Linus Torvalds]

On Fri, 5 Sep 2008, Scott Chacon wrote:
> 
> http://book.git-scm.com/7_the_packfile.html 

The checksums in the index file "trailers" are all claiming to be 4 bytes, 
and that's wrong - they're full SHA1 sums at 20 bytes each.

The v2 pack-file _also_ has per-object CRC's, and those are indeed just 4 
bytes each, and are correctly listed as such.

The pack-file itself also has a few more things there, it's not just the 
"PACK" string and then the objects. It has two more 32-bit words: a pack 
file version number and the number of entries in the pack-file (all 
network byte order). It also has its own checksum at the end (20-byte SHA1 
again).

But looks good otherwise from a quick look.

		Linus

Re: Git Community Book

[Scott Chacon]

On Fri, Sep 5, 2008 at 12:15 PM, Thomas Adam <thomas.adam22@gmail.com> wrote:
> 2008/9/5 Scott Chacon <schacon@gmail.com>:
>> Hey all,
>>
>> I just wanted to let those of you who are interested know that I've
>> been making a lot of progress on the Git Community Book
>> (http://book.git-scm.com)  I was wondering if anyone was interested in
>
> I'm going to bite and ask the obvious questions:

Just for reference, a lot of this was discussed here a while back:

http://thread.gmane.org/gmane.comp.version-control.git/90653

however, I would be happy to answer these for you.

>
> 1.  How does what you're producing differ from the current Git Users' Manual?

I'm going for a different audience with this project.  I'd like for it
to be a lot more user-friendly, easily digestible, and to include
images, diagrams and screencasts.

> 2.  Is this project of yours aiming to obsolete the Git Users' Manual
> with "official" sanctioning from people involved with Git?

I think there will be people who prefer the Users Manual format, who
think screencasts are wussy :)
Also, I'm not sure an "official" sanctioning would do much of anything
- because of the images and screencasts, this will never be included
in the git source like the UM is, but it's also open source so if
people want to take content from it to improve the UM, that's cool.

> 3.  Assuming 2 is a "no", patches to the Users' Guide would be nice.  :)

I would love to do this, but I don't know what exactly the community
thinks is missing/lacking.  My ideas about what is helpful is rarely
the same as the git lists :)  However, if someone pointed to one of
the chapters I wrote and said "that would be great in the UM", I would
happily convert it.

Scott

Re: Git Community Book

[Scott Chacon]

On Fri, Sep 5, 2008 at 12:41 PM, Junio C Hamano <gitster@pobox.com> wrote:
> "Scott Chacon" <schacon@gmail.com> writes:
>
>> Also, the last section of the book is on some of the plumbing - mostly
>> stuff I've found difficult to pick up with the existing documentation
>> while re-implementing stuff in Ruby.  I would really appreciate it if
>> someone could proofread some of these chapters for errors:
>>
>> http://book.git-scm.com/7_the_packfile.html
>
> Nice pictures.  You might also want to know that code for reading pack idx
> version 2 was backported to 1.4.4.5 for people who are stuck on 1.4.4
> series for whatever reason.
>
> What is the target audience of this section?  If it is written for a mere
> curious type, or if it is written to give "here is the general idea, for
> more details read the source", the level of detail here would be Ok.
>
> If you are writing for people who want to (re)implement something that
> produces these files, you might want to at least say that offset/sha1[]
> table is sorted by sha1[] values (this is to allow binary search of this
> table), and fanout[] table points at the offset/sha1[] table in a specific
> way (so that part of the latter table that covers all hashes that start
> with a given byte can be found to avoid 8 iterations of the binary
> search).
>
> <data> part is just zlib stream for non-delta object types; for the two
> delta object representations, the <data> portion contains something that
> identifies which base object this delta representation depends on, and the
> delta to apply on the base object to resurrect this object.  ref-delta
> uses 20-byte hash of the base object at the beginning of <data>, while
> ofs-delta stores an offset within the same packfile to identify the base
> object.  In either case, two important constraints a reimplementor must
> adhere to are:
>
>  * delta representation must be based on some other object within the same
>   packfile;
>
>  * the base object must be of the same underlying type (blob, tree, commit
>   or tag);
>
>> http://book.git-scm.com/7_raw_git.html
>
> I am guessing this is for Porcelain writers who use plumbing.  Please
> don't teach echoing into .git/refs/...  but DO teach using update-ref with
> the -m option.  We do not want people's random Porcelains flipping the tip
> of branches without leaving trail in reflog for users to use to recover
> from mistakes.
>

I've implemented all of these and Linus's fixes and suggestions.
Thanks for the feedback.

To answer your earlier question, these docs are basically for people
working on bindings/re-implementations in other languages, since there
is no real linked library available yet, as a primer before they dig
into the source, or possibly so they don't have to.

I'm not fantastic at C, so it took me a while in some cases - figuring
out that the size listed in the object header was not the actual size
of the data, but the size of it when expanded, for example, was not
very easy to do.  I've been doing a lot of work on re-implementations
in Ruby and ObjC because I can't easily make real bindings, so I
thought I would add things that I could not easily find in the docs
for others that are trying in other languages.

If you want, I could create a patch for any of this stuff to
Documentation/ (that goes for the whole book), but someone will have
to tell me which parts might be useful to add.

Thanks again for taking the time!
Scott

Re: Git Community Book

[Felipe Contreras]

On Sat, Sep 6, 2008 at 12:34 AM, Scott Chacon <schacon@gmail.com> wrote:
> On Fri, Sep 5, 2008 at 12:41 PM, Junio C Hamano <gitster@pobox.com> wrote:
>> "Scott Chacon" <schacon@gmail.com> writes:
>>
>>> Also, the last section of the book is on some of the plumbing - mostly
>>> stuff I've found difficult to pick up with the existing documentation
>>> while re-implementing stuff in Ruby.  I would really appreciate it if
>>> someone could proofread some of these chapters for errors:
>>>
>>> http://book.git-scm.com/7_the_packfile.html
>>
>> Nice pictures.  You might also want to know that code for reading pack idx
>> version 2 was backported to 1.4.4.5 for people who are stuck on 1.4.4
>> series for whatever reason.
>>
>> What is the target audience of this section?  If it is written for a mere
>> curious type, or if it is written to give "here is the general idea, for
>> more details read the source", the level of detail here would be Ok.
>>
>> If you are writing for people who want to (re)implement something that
>> produces these files, you might want to at least say that offset/sha1[]
>> table is sorted by sha1[] values (this is to allow binary search of this
>> table), and fanout[] table points at the offset/sha1[] table in a specific
>> way (so that part of the latter table that covers all hashes that start
>> with a given byte can be found to avoid 8 iterations of the binary
>> search).
>>
>> <data> part is just zlib stream for non-delta object types; for the two
>> delta object representations, the <data> portion contains something that
>> identifies which base object this delta representation depends on, and the
>> delta to apply on the base object to resurrect this object.  ref-delta
>> uses 20-byte hash of the base object at the beginning of <data>, while
>> ofs-delta stores an offset within the same packfile to identify the base
>> object.  In either case, two important constraints a reimplementor must
>> adhere to are:
>>
>>  * delta representation must be based on some other object within the same
>>   packfile;
>>
>>  * the base object must be of the same underlying type (blob, tree, commit
>>   or tag);
>>
>>> http://book.git-scm.com/7_raw_git.html
>>
>> I am guessing this is for Porcelain writers who use plumbing.  Please
>> don't teach echoing into .git/refs/...  but DO teach using update-ref with
>> the -m option.  We do not want people's random Porcelains flipping the tip
>> of branches without leaving trail in reflog for users to use to recover
>> from mistakes.
>>
>
> I've implemented all of these and Linus's fixes and suggestions.
> Thanks for the feedback.
>
> To answer your earlier question, these docs are basically for people
> working on bindings/re-implementations in other languages, since there
> is no real linked library available yet, as a primer before they dig
> into the source, or possibly so they don't have to.
>
> I'm not fantastic at C, so it took me a while in some cases - figuring
> out that the size listed in the object header was not the actual size
> of the data, but the size of it when expanded, for example, was not
> very easy to do.  I've been doing a lot of work on re-implementations
> in Ruby and ObjC because I can't easily make real bindings, so I
> thought I would add things that I could not easily find in the docs
> for others that are trying in other languages.

I have experience mixing C and Ruby code if you are interested, it's
actually quite easy.

I also think a shared library would make sense.

Keep up the good work ;)

-- 
Felipe Contreras

Re: Git Community Book

[Stephan Beyer]

Hi,

Scott Chacon wrote:
> I just wanted to let those of you who are interested know that I've
> been making a lot of progress on the Git Community Book
> (http://book.git-scm.com)  I was wondering if anyone was interested in
> helping me with a few parts.

I just had a very quick look over the PDF, meaning only looking at
pictures and headlines.

Just nitpicking about one thing:
I was wondering if "Stash Queue" is the right headline, because I
usually use

	git stash save	# oh, an interrupt, have to do something else now

and after this is done:

	git stash pop	# back to the real work

And if you are interrupted in an interrupt, you want the last stash
being the first one to pop, which is a stack-like (last in, first out)
behavior.

Of course, there may be cases where you want the queuing behavior that
you advertise in the book.
I use it rather seldomly. But perhaps it is just me :-)

Regards,
  Stephan

-- 
Stephan Beyer <s-beyer@gmx.net>, PGP 0x6EDDD207FCC5040F

Re: Git Community Book

[Shawn O. Pearce]

Scott Chacon <schacon@gmail.com> wrote:
> On Fri, Sep 5, 2008 at 12:41 PM, Junio C Hamano <gitster@pobox.com> wrote:
> > "Scott Chacon" <schacon@gmail.com> writes:
> >
> >> Also, the last section of the book is on some of the plumbing - mostly
> >> stuff I've found difficult to pick up with the existing documentation
> >> while re-implementing stuff in Ruby.  I would really appreciate it if
> >> someone could proofread some of these chapters for errors:
> >>
> >> http://book.git-scm.com/7_the_packfile.html

OK, time for me to throw in comments.  ;-)

I do like this book, its organized and concise.  Thanks for doing it.


http://book.git-scm.com/7_how_git_stores_objects.html:

The loose object formatting of

 header = "#{type} #{size}#body"
 store = header + content

I can't read Ruby so I'm not sure what the header value computes
out to here.  #body should be a \0.  I'm also not sure that the
prior line setting size = content.length.to_s is very clear for
the non-Ruby people to understand how a size is formatted.

If the code shown here is the Ruby implementation I'm a little
concerned about it writing directly into the loose object.  If the
write is partial then you have a partial object which is at the
right name, but is unusable.  That can give you corruption that
is difficult to track down and fix.  C Git and JGit both write
to temporary files then atomically move the temporary file into
position under its proper name only after it has been fully written.

If an implementor is implementing they should be offered this advice,
and probably do so right here in this section of the book.

"When objects are written to disk, it is often in the loose format,
since that format is less expensive to access."

I'm not sure that statement is true.  Access from packs tends
to scream compared to access from loose objects.  The overheads
of opening and closing the file descriptors, even on Linux, is
what kills performance for data access.  However Git writes to
loose objects first and packs later for _safety_ not efficiency.
Although it is a lot more efficient to write a 2 KB loose object
and avoid rewriting a 50 MB pack, but its also less likely to fail
and make you lose your work.


http://book.git-scm.com/7_the_git_index.html:

I wouldn't say that the index stores permissions.  More like it
stores the "class" or "type" of the thing located at that path.
There are 4 major classes:

	- regular file
	- executable file
	- symbolic link
	- git submodule

The 5th class is the subtree, but only appears in trees and not
in the index since the index file is actually flat.


http://book.git-scm.com/7_the_packfile.html:

You should probably point out that the .idx file uses network byte
order for the numeric fields like the version number and the file
offsets.

I'd also point out that the offsets in index v1 are unsigned and
from the start of the pack file.  The offsets in index v2 are
also unsigned, but the 1<<31 is tested in the 32 bit offset to
see if a 64 bit offset is used.  The algorithm there is:

	if offset32 & 1<<31:
		offset = ofs64_table[offset32 & ~(1<<31)]
	else
		offset = offset32

Its also rather unclear how the fan out table can be used to limit
the binary search.  What you are missing is describing that fanout[X]
holds the number of objects whose first byte of their SHA-1 is <= X.
Hence fanout[0] has the number of objects whose SHA-1 starts with
"00" and fanout[0x15] has the number of objects whose SHA-1 starts
with "15", "14", "13", ..., "00".  Thus fanout[0xff] has the total
number of objects in the pack.

In the pack file section I'd also point out the version and entry
count are unsigned network byte order.  This is not clear from the
Ruby code, although one can guess at it if one knows the git.git
code very very well (like I do).

"After that, you get a series of packed objects, in order of thier SHAs"

Aside from s/thier/their/ this is not a correct statement _AT ALL_.

The ordering of objects in the packfile is very carefully planned
by the packer to maximize data locality from most recent -> least
recent information, making the most recent revisions of a project
the fastest to access.  This has _NOTHING_ to do with their SHA-1
names.

Technically a pack may store objects in any random order.  Heck,
you can wire up an RNG to the packer to always produce a different
ordering each time you pack.  Practically an implementation shouldn't
be that stupid and should instead try to order objects by recency,
like git.git and JGit both do.

"At the end of the packfile is a 20-byte SHA1 sum of all the shas
(in sorted order) in that packfile."

Also incorrect.  The 20-byte checksum at the end of the pack file
is a checksum of all bytes preceeding the checksum itself.  We use
it as an end-to-end data integrity check, especially on the network
transport to verify that every bit sent by the one side is received
correctly on the other side.

BTW, can I just say, I love the graphics in this book.  They are
quite well done.  Very worthwhile.


http://book.git-scm.com/7_transfer_protocols.html:

You might as well explain that the stream returned by upload-pack
uses the same 4 byte line length framing to form "packets", with
the 5th byte (really first byte of the payload) indicating the
"stream":

	- stream 1 ('\001') is the PACK data
	- stream 2 ('\002') is progress data/information
	- stream 3 ('\003') is the OH S**T we are aborting, died, dead

You may also want to explain that the way you know the end of the
pack is to read the header, get the entry count, and then read that
many objects from the stream, and then verify the pack checksum.

-- 
Shawn.

Re: Git Community Book

[Scott Chacon]

Thanks a ton for this, I'll incorporate all of this.

On Fri, Sep 5, 2008 at 11:33 PM, Shawn O. Pearce <spearce@spearce.org> wrote:
> Scott Chacon <schacon@gmail.com> wrote:
>> On Fri, Sep 5, 2008 at 12:41 PM, Junio C Hamano <gitster@pobox.com> wrote:
>> > "Scott Chacon" <schacon@gmail.com> writes:
>> >
>> >> Also, the last section of the book is on some of the plumbing - mostly
>> >> stuff I've found difficult to pick up with the existing documentation
>> >> while re-implementing stuff in Ruby.  I would really appreciate it if
>> >> someone could proofread some of these chapters for errors:
>> >>
>> >> http://book.git-scm.com/7_the_packfile.html
>
> OK, time for me to throw in comments.  ;-)
>
> I do like this book, its organized and concise.  Thanks for doing it.
>
>
> http://book.git-scm.com/7_how_git_stores_objects.html:
>
> The loose object formatting of
>
>  header = "#{type} #{size}#body"
>  store = header + content
>
> I can't read Ruby so I'm not sure what the header value computes
> out to here.  #body should be a \0.  I'm also not sure that the
> prior line setting size = content.length.to_s is very clear for
> the non-Ruby people to understand how a size is formatted.
>

Sorry, the markdown thingy is translating all the '\0's to '#body' for
some freaking reason unless I write it as '\\0'.  I'll fix this - it's
difficult for me to find these sometimes.  As for the rest of the ruby
stuff, I think I'll add some comments.

> If the code shown here is the Ruby implementation I'm a little
> concerned about it writing directly into the loose object.  If the
> write is partial then you have a partial object which is at the
> right name, but is unusable.  That can give you corruption that
> is difficult to track down and fix.  C Git and JGit both write
> to temporary files then atomically move the temporary file into
> position under its proper name only after it has been fully written.

That is a good idea - I don't do it that way and I certainly will
change the implementation to do so and modify these docs to reflect
that advice.

> "When objects are written to disk, it is often in the loose format,
> since that format is less expensive to access."
>
> I'm not sure that statement is true.  Access from packs tends
> to scream compared to access from loose objects.  The overheads
> of opening and closing the file descriptors, even on Linux, is
> what kills performance for data access.  However Git writes to
> loose objects first and packs later for _safety_ not efficiency.
> Although it is a lot more efficient to write a 2 KB loose object
> and avoid rewriting a 50 MB pack, but its also less likely to fail
> and make you lose your work.

Thanks for the clarification.  I write to loose objects first largely
because it's so much easier to do.  But also because I don't mmap
objects, so packfile access is not faster for implementations that
can't do that very well.  Also, I had originally meant "less expensive
to write", but I can see that is not clear.


> http://book.git-scm.com/7_the_git_index.html:
>
> I wouldn't say that the index stores permissions.  More like it
> stores the "class" or "type" of the thing located at that path.
> There are 4 major classes:
>
>        - regular file
>        - executable file
>        - symbolic link
>        - git submodule
>
> The 5th class is the subtree, but only appears in trees and not
> in the index since the index file is actually flat.

Interesting.  This documentation is actually from the User Manual -
I'll update this chapter first and if it looks better, I'll submit a
patch to the UM, too.

> http://book.git-scm.com/7_the_packfile.html:
>
> You should probably point out that the .idx file uses network byte
> order for the numeric fields like the version number and the file
> offsets.

Will do.

>
> I'd also point out that the offsets in index v1 are unsigned and
> from the start of the pack file.  The offsets in index v2 are
> also unsigned, but the 1<<31 is tested in the 32 bit offset to
> see if a 64 bit offset is used.  The algorithm there is:
>
>        if offset32 & 1<<31:
>                offset = ofs64_table[offset32 & ~(1<<31)]
>        else
>                offset = offset32
>
> Its also rather unclear how the fan out table can be used to limit
> the binary search.  What you are missing is describing that fanout[X]
> holds the number of objects whose first byte of their SHA-1 is <= X.
> Hence fanout[0] has the number of objects whose SHA-1 starts with
> "00" and fanout[0x15] has the number of objects whose SHA-1 starts
> with "15", "14", "13", ..., "00".  Thus fanout[0xff] has the total
> number of objects in the pack.
>
> In the pack file section I'd also point out the version and entry
> count are unsigned network byte order.  This is not clear from the
> Ruby code, although one can guess at it if one knows the git.git
> code very very well (like I do).
>
> "After that, you get a series of packed objects, in order of thier SHAs"
>
> Aside from s/thier/their/ this is not a correct statement _AT ALL_.
>
> The ordering of objects in the packfile is very carefully planned
> by the packer to maximize data locality from most recent -> least
> recent information, making the most recent revisions of a project
> the fastest to access.  This has _NOTHING_ to do with their SHA-1
> names.
>
> Technically a pack may store objects in any random order.  Heck,
> you can wire up an RNG to the packer to always produce a different
> ordering each time you pack.  Practically an implementation shouldn't
> be that stupid and should instead try to order objects by recency,
> like git.git and JGit both do.
>
> "At the end of the packfile is a 20-byte SHA1 sum of all the shas
> (in sorted order) in that packfile."
>
> Also incorrect.  The 20-byte checksum at the end of the pack file
> is a checksum of all bytes preceeding the checksum itself.  We use
> it as an end-to-end data integrity check, especially on the network
> transport to verify that every bit sent by the one side is received
> correctly on the other side.
>

I'm an idiot.  I say this because I actually implemented a bunch of
this stuff (in Ruby) and ran into most of these issues when trying to
implement it.  So I knew these things not 3 weeks ago, but I still
wrote it this way.  Dur.  Thanks for the corrections, I'll update
everything accordingly.

> BTW, can I just say, I love the graphics in this book.  They are
> quite well done.  Very worthwhile.

Thanks.

>
>
> http://book.git-scm.com/7_transfer_protocols.html:
>
> You might as well explain that the stream returned by upload-pack
> uses the same 4 byte line length framing to form "packets", with
> the 5th byte (really first byte of the payload) indicating the
> "stream":
>
>        - stream 1 ('\001') is the PACK data
>        - stream 2 ('\002') is progress data/information
>        - stream 3 ('\003') is the OH S**T we are aborting, died, dead
>
> You may also want to explain that the way you know the end of the
> pack is to read the header, get the entry count, and then read that
> many objects from the stream, and then verify the pack checksum.
>
> --
> Shawn.
>

Thanks again for all the time it must have taken to review all of this
- I'll make sure it gets into the book, and where appropriate, back
into the UM or other internal git docs.

Scott

Re: Git Community Book

[Christos Τrochalakis]

On Fri, Sep 5, 2008 at 10:08 PM, Scott Chacon <schacon@gmail.com> wrote:
> Hey all,
>
> I just wanted to let those of you who are interested know that I've
> been making a lot of progress on the Git Community Book
> (http://book.git-scm.com)
> ...

Hello Scott!

Nice book, I just started reading it and I have a recommendation to
make, at "Chapter 4: Git Treeishes" you write

---------
http://book.git-scm.com/4_git_treeishes.html
Range

Finally, you can specify a range of commits with the range spec. This
will give you all the commits between 7b593b5 and 51bea1 (where 51bea1
is most recent), excluding 7b593b5 but including 51bea1:

7b593b5..51bea1

This will include every commit since 7b593b:

7b593b..
---------

This in not quite correct. "commits between A and B" cannot really
apply here. I believe that "commits reachable from B and not from A"
is more precise. Actually you are already using the "reachability"
explanation at the start of "Chapter 3: Basic usage".

This issue is also described at the rev-parse man page.

Apart from that, you could also include "a...b" syntax for completeness.

-christos