Re: [PATCH 4/5] fast-export: Introduce --inline-blobs

[Jonathan Nieder <jrnieder@gmail.com>]

Ramkumar Ramachandra wrote:

> Agreed. I wouldn't like to introduce an extra dependency either. I was
> talking about using it for prototyping- if the final version includes
> an extra dependency, it's unlikely to get merged into git.git :) The
> final design will probably use an in-memory B+ tree, but I haven't
> thought about that hard enough.

Immediate reaction:

Please no.  There is a value to simplicity.

As you mention, the final form is a way off, so as long as people are
careful not to get locked into bad implementation decisions, I think
it is okay.  I have refrained from nitpicking the implementation so
far because the design and interface are not obvious yet.

In this particular case, I am dreaming that we will discover a hidden
"mkdir -p" node-action in the dumpfile format so the list of
directories will not be needed. ;-)

Re Junio's critique:

is it possible to use

1) a table with callbacks?  See the source code to unifdef for
   inspiration.

2) separate code paths for different input states?  fast-import.c
   does this.

3) separate "parsing" and "acting" code?  That can open the door to a
   little paralellism, though not necessarily enough to matter.

	parser                              actor

	read a chunk
	determine the first "thing to do"
	                                    pick up the first "thing to do"
	                                    do it
	determine the next "thing to do"
	                                    pick up the next "thing to do"
	                                    do it

   There is potential parallelism because the parser can keep
   chugging along if the actor is blocked, say, writing its
   output to the network.

   Syntactically: the actor function (write_dump) calls the parser
   function (next_command) to ask what to do next.  If wanted, a later
   refactoring could make that parser function just grab an action off
   of a queue, while the parser proper runs in the background.
  
   And of course if the "thing to do" data structure is simple enough,
   this can also make the code easier to read.

I mention these ideas because some of them (especially #2) could make
the prototyping a lot easier as well as resulting in code that is
easier to review.

Re error handling:

Writing robust code (e.g., checking for errors) is also a lot easier
when done from the start.  The svn-fe error handling is known to be a
problem (see BUGS in contrib/svn-fe/svn-fe.txt).  So yes, I also
consider avoiding segfaults and deadlocks and catching parse errors to
be worthwhile things.

Thanks.
Jonathan