Re: too many open files

[Eric Wong <normalperson@yhbt.net>]

"Peter P." <peterparker@fastmail.com> wrote:
> I managed to get things to work this way. Nevertheless Eric, if you have
> a moment to peek into the code and see why sox is keeping files open, it
> would be fantastic and could prevent similar questions coming up again.

Most of the sox input methods are parallel (mix/merge/etc), so
those files all need to be kept open for the duration of the
invocation.

Based on a quick look, it seems serial methods
(concatenate/sequence) can handle closing just fine;
it might make the code a little more complex.

Off the top of my head, "gain -n" needs rewindability, but
that seems to use a temporary file, anyways (since sox is
equipped to handle pipes).

One major issue of delaying fopen() is the files may disappear
from the filesystem immediately after sox opens them started; so
delaying opening them cannot be the default behavior, and needs
to be made an option, instead...

I know I often unlink files ASAP after opening them when I'm
testing different effects on a small filesystem (tmpfs).


But yeah, even command-line args are likely to exceed the limits
of your OS command size, probably before nofile limits (ulimit
-n), so I would favor find + xargs, instead of globbing;
something like:

	export FMT='-ts32 -c2 -r48000'
	find $DIR -name '*.part*.wav' -type f -print0 | \
		xargs -0 -n1 sh -c 'sox "$@" $FMT -' -- |
		sox $FMT - out.wav

This avoids nofile limits as well as any command-line length
limitations the OS might have.

(I'm embarrassed for not thinking of this from my original reply :x)

------------------------------------------------------------------------------