scripting speedups [was: [Summit topic] Crazy (and not so crazy) ideas]

[Eric Wong <e@80x24.org>]

Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> * Test suite is slow. Shell scripts and process forking.
> 
>    * What if we had a special shell that interpreted the commands in a
>      single process?
> 
>    * Even Git commands like rev-parse and hash-object, as long as that’s
>      not the command you’re trying to test

This is something I've wanted in a very long time as a scripter.
fast-import has been great over the years, as is
"cat-file --batch(-check)", but there's gaps should be filled
(preferably without fragile linkage of shared libraries into a
script process)

>    * Dscho wants to slip in a C-based solution
> 
>    * Jonathan tan commented: going back to your custom shell for tests
>      idea, one thing we could do is have a custom command that generates
>      the repo commits that we want (and that saves process spawns and
>      might make the tests simpler too)

Perhaps a not-seriously-proposed patch from 2006 could be
modernized for our now-libified internals:

https://yhbt.net/lore/git/Pine.LNX.4.64.0602232229340.3771@g5.osdl.org/

>       * We could replace several “setup repo” steps with “git fast-import”
>         instead.
> 
>    * Dscho measured: 0.5 sec - 30 sec in setup steps. Can use fast-import,
>      or can make a new format that helps us set up the test scenario

0.5s - 30s across the whole suite or individual tests?

Having a way to disable fsync globally should further improve
things, especially for people on slower storage.  libeatmydata
is available, but perhaps not widely available/known.

>    * Elijah: test-lib-functions helpers could be built ins