[PATCH] introduce config core.binaryCheckFirstBytes for xdiff-interface

[PATCH] introduce config core.binaryCheckFirstBytes for xdiff-interface

[Gerrit Pape]

xdiff-interface uses a hardcoded value of 8000 bytes to check from
the top of data whether to handle it as binary content.  If a NULL
character appears after the first 8000 bytes, git won't notice,
which for example breaks the git format-patch | git am pipeline in
git rebase, as reported by Jamey Sharp through
 http://bugs.debian.org/436182

This patch makes the hardcoded value configurable, so that users can
adjust the behavior when tracking binary files of huge sizes, e.g.
pdf's, in a repository.

Signed-off-by: Gerrit Pape <pape@smarden.org>
---
 Documentation/config.txt |    6 ++++++
 cache.h                  |    1 +
 config.c                 |    8 ++++++++
 environment.c            |    1 +
 xdiff-interface.c        |    5 ++---
 5 files changed, 18 insertions(+), 3 deletions(-)

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 866e053..6c450ee 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -275,6 +275,12 @@ You probably do not need to adjust this value.
 +
 Common unit suffixes of 'k', 'm', or 'g' are supported.
 
+core.binaryCheckFirstBytes::
+	Number of bytes of the top of data that should be checked to
+	decide whether the data must be handled as binary content, or
+	can be interpreted as text.  The default is 8000, when set to
+	zero, all available data is checked.
+
 core.excludesfile::
 	In addition to '.gitignore' (per-directory) and
 	'.git/info/exclude', git looks into this file for patterns
diff --git a/cache.h b/cache.h
index 70abbd5..a9ca846 100644
--- a/cache.h
+++ b/cache.h
@@ -313,6 +313,7 @@ extern size_t packed_git_window_size;
 extern size_t packed_git_limit;
 extern size_t delta_base_cache_limit;
 extern int auto_crlf;
+extern int binary_first_bytes;
 
 #define GIT_REPO_VERSION 0
 extern int repository_format_version;
diff --git a/config.c b/config.c
index dc3148d..892b2ff 100644
--- a/config.c
+++ b/config.c
@@ -395,6 +395,14 @@ int git_default_config(const char *var, const char *value)
 		return 0;
 	}
 
+	if (!strcmp(var, "core.binarycheckfirstbytes")) {
+		int bytes = git_config_int(var, value);
+		if (bytes < 0)
+			die("bad value for core.binaryCheckFirstBytes %d", bytes);
+		binary_first_bytes = bytes;
+		return 0;
+	}
+
 	if (!strcmp(var, "user.name")) {
 		strlcpy(git_default_name, value, sizeof(git_default_name));
 		return 0;
diff --git a/environment.c b/environment.c
index b5a6c69..dd072b9 100644
--- a/environment.c
+++ b/environment.c
@@ -35,6 +35,7 @@ int pager_in_use;
 int pager_use_color = 1;
 char *editor_program;
 int auto_crlf = 0;	/* 1: both ways, -1: only when adding git objects */
+int binary_first_bytes = 8000;
 
 /* This is set by setup_git_dir_gently() and/or git_default_config() */
 char *git_work_tree_cfg;
diff --git a/xdiff-interface.c b/xdiff-interface.c
index be866d1..dba0232 100644
--- a/xdiff-interface.c
+++ b/xdiff-interface.c
@@ -122,11 +122,10 @@ int read_mmfile(mmfile_t *ptr, const char *filename)
 	return 0;
 }
 
-#define FIRST_FEW_BYTES 8000
 int buffer_is_binary(const char *ptr, unsigned long size)
 {
-	if (FIRST_FEW_BYTES < size)
-		size = FIRST_FEW_BYTES;
+	if (binary_first_bytes && (binary_first_bytes < size))
+		size = binary_first_bytes;
 	return !!memchr(ptr, 0, size);
 }
 
-- 
1.5.3.1

Re: [PATCH] introduce config core.binaryCheckFirstBytes for xdiff-interface

[Junio C Hamano]

Gerrit Pape <pape@smarden.org> writes:

> xdiff-interface uses a hardcoded value of 8000 bytes to check from
> the top of data whether to handle it as binary content.  If a NULL
> character appears after the first 8000 bytes, git won't notice,

If the user has to set this to suit git better for a particular
project, I think the same effort is better spent on setting up
the attribute for that file so that git does not have to guess
the type, no?