[PATCH] Win32: simplify loading of DLL functions

[PATCH] Win32: simplify loading of DLL functions

[Johannes Schindelin]

Dynamic loading of DLL functions is duplicated in several places in Git
for Windows' source code.

This patch adds a pair of macros to simplify the process: the
DECLARE_PROC_ADDR(<dll>, <return-type>, <function-name>,
...<function-parameter-types>...) macro to be used at the beginning of a
code block, and the INIT_PROC_ADDR(<function-name>) macro to call before
using the declared function. The return value of the INIT_PROC_ADDR()
call has to be checked; If it is NULL, the function was not found in the
specified DLL.

Example:

        DECLARE_PROC_ADDR(kernel32.dll, BOOL, CreateHardLinkW,
                          LPCWSTR, LPCWSTR, LPSECURITY_ATTRIBUTES);

        if (!INIT_PROC_ADDR(CreateHardLinkW))
                return error("Could not find CreateHardLinkW() function";

	if (!CreateHardLinkW(source, target, NULL))
		return error("could not create hardlink from %S to %S",
			     source, target);
	return 0;

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---

	So far, there are no users (except in Git for Windows). Ben
	promised to make use of it in his fsmonitor patch series.

Published-As: https://github.com/dscho/git/releases/tag/lazyload-v1
Fetch-It-Via: git fetch https://github.com/dscho/git lazyload-v1
 compat/win32/lazyload.h | 44 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 44 insertions(+)
 create mode 100644 compat/win32/lazyload.h

diff --git a/compat/win32/lazyload.h b/compat/win32/lazyload.h
new file mode 100644
index 00000000000..91c10dad2fb
--- /dev/null
+++ b/compat/win32/lazyload.h
@@ -0,0 +1,44 @@
+#ifndef LAZYLOAD_H
+#define LAZYLOAD_H
+
+/* simplify loading of DLL functions */
+
+struct proc_addr {
+	const char *const dll;
+	const char *const function;
+	FARPROC pfunction;
+	unsigned initialized : 1;
+};
+
+/* Declares a function to be loaded dynamically from a DLL. */
+#define DECLARE_PROC_ADDR(dll, rettype, function, ...) \
+	static struct proc_addr proc_addr_##function = \
+	{ #dll, #function, NULL, 0 }; \
+	static rettype (WINAPI *function)(__VA_ARGS__)
+
+/*
+ * Loads a function from a DLL (once-only).
+ * Returns non-NULL function pointer on success.
+ * Returns NULL + errno == ENOSYS on failure.
+ */
+#define INIT_PROC_ADDR(function) \
+	(function = get_proc_addr(&proc_addr_##function))
+
+static inline void *get_proc_addr(struct proc_addr *proc)
+{
+	/* only do this once */
+	if (!proc->initialized) {
+		HANDLE hnd;
+		proc->initialized = 1;
+		hnd = LoadLibraryExA(proc->dll, NULL,
+				     LOAD_LIBRARY_SEARCH_SYSTEM32);
+		if (hnd)
+			proc->pfunction = GetProcAddress(hnd, proc->function);
+	}
+	/* set ENOSYS if DLL or function was not found */
+	if (!proc->pfunction)
+		errno = ENOSYS;
+	return proc->pfunction;
+}
+
+#endif

base-commit: 9ddaf86b06a8078420f59aec8cab6daa93cf1a91
-- 
2.14.1.windows.1.1020.g03faabc8bc8.dirty

Re: [PATCH] Win32: simplify loading of DLL functions

[Jonathan Nieder]

Hi,

Johannes Schindelin wrote:

> Dynamic loading of DLL functions is duplicated in several places in Git
> for Windows' source code.
>
> This patch adds a pair of macros to simplify the process: the
> DECLARE_PROC_ADDR(<dll>, <return-type>, <function-name>,
> ...<function-parameter-types>...) macro to be used at the beginning of a
> code block, and the INIT_PROC_ADDR(<function-name>) macro to call before
> using the declared function. The return value of the INIT_PROC_ADDR()
> call has to be checked; If it is NULL, the function was not found in the
> specified DLL.
>
> Example:
>
>         DECLARE_PROC_ADDR(kernel32.dll, BOOL, CreateHardLinkW,
>                           LPCWSTR, LPCWSTR, LPSECURITY_ATTRIBUTES);
>
>         if (!INIT_PROC_ADDR(CreateHardLinkW))
>                 return error("Could not find CreateHardLinkW() function";
>
> 	if (!CreateHardLinkW(source, target, NULL))
> 		return error("could not create hardlink from %S to %S",
> 			     source, target);
> 	return 0;

nit: whitespace is a bit strange here (mixture of tabs and spaces).

Could this example go near the top of the header instead?  That way,
it's easier for people reading the header to see how to use it.

> Signed-off-by: Karsten Blees <blees@dcon.de>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---

Just curious: what was Karsten's contribution?  (I ask mostly because
I'm interested in kinds of collaboration git metadata is failing to
capture correctly --- e.g. pair programming.)

>         So far, there are no users (except in Git for Windows). Ben
>         promised to make use of it in his fsmonitor patch series.
>
>  compat/win32/lazyload.h | 44 ++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 44 insertions(+)
>  create mode 100644 compat/win32/lazyload.h

Are any of the Git for Windows users something that could go upstream
along with this patch?  That would help illustrate what a good caller
looks like, which should help with reviewing future patches that use
this code.

> --- /dev/null
> +++ b/compat/win32/lazyload.h
> @@ -0,0 +1,44 @@
[...]
> +/* Declares a function to be loaded dynamically from a DLL. */
> +#define DECLARE_PROC_ADDR(dll, rettype, function, ...) \
> +	static struct proc_addr proc_addr_##function = \
> +	{ #dll, #function, NULL, 0 }; \
> +	static rettype (WINAPI *function)(__VA_ARGS__)
> +
> +/*
> + * Loads a function from a DLL (once-only).
> + * Returns non-NULL function pointer on success.
> + * Returns NULL + errno == ENOSYS on failure.
> + */
> +#define INIT_PROC_ADDR(function) \
> +	(function = get_proc_addr(&proc_addr_##function))

Probably worth mentioning in the doc comment that this is not thread
safe, so a caller that wants to lazy-init in a threaded context is
responsible for doing their own locking.

> +
> +static inline void *get_proc_addr(struct proc_addr *proc)
> +{
> +	/* only do this once */
> +	if (!proc->initialized) {
> +		HANDLE hnd;
> +		proc->initialized = 1;
> +		hnd = LoadLibraryExA(proc->dll, NULL,
> +				     LOAD_LIBRARY_SEARCH_SYSTEM32);
> +		if (hnd)
> +			proc->pfunction = GetProcAddress(hnd, proc->function);
> +	}
> +	/* set ENOSYS if DLL or function was not found */
> +	if (!proc->pfunction)
> +		errno = ENOSYS;
> +	return proc->pfunction;
> +}

strerror(ENOSYS) is "Function not implemented".  Cute.

Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>

Thanks,
Jonathan

Re: [PATCH] Win32: simplify loading of DLL functions

[Junio C Hamano]

Jonathan Nieder <jrnieder@gmail.com> writes:

> ...
> strerror(ENOSYS) is "Function not implemented".  Cute.
>
> Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>

Thanks, both.  Will queue.

Re: [PATCH] Win32: simplify loading of DLL functions

[Johannes Schindelin]

Hi Jonathan,

On Tue, 19 Sep 2017, Jonathan Nieder wrote:

> Johannes Schindelin wrote:
> 
> > Dynamic loading of DLL functions is duplicated in several places in Git
> > for Windows' source code.
> >
> > This patch adds a pair of macros to simplify the process: the
> > DECLARE_PROC_ADDR(<dll>, <return-type>, <function-name>,
> > ...<function-parameter-types>...) macro to be used at the beginning of a
> > code block, and the INIT_PROC_ADDR(<function-name>) macro to call before
> > using the declared function. The return value of the INIT_PROC_ADDR()
> > call has to be checked; If it is NULL, the function was not found in the
> > specified DLL.
> >
> > Example:
> >
> >         DECLARE_PROC_ADDR(kernel32.dll, BOOL, CreateHardLinkW,
> >                           LPCWSTR, LPCWSTR, LPSECURITY_ATTRIBUTES);
> >
> >         if (!INIT_PROC_ADDR(CreateHardLinkW))
> >                 return error("Could not find CreateHardLinkW() function";
> >
> > 	if (!CreateHardLinkW(source, target, NULL))
> > 		return error("could not create hardlink from %S to %S",
> > 			     source, target);
> > 	return 0;
> 
> nit: whitespace is a bit strange here (mixture of tabs and spaces).

Heh, I never thought that this would be a problem in *commit messages*.
TBH I simply copy-edited the code from Git for Windows' source code.

> Could this example go near the top of the header instead?  That way,
> it's easier for people reading the header to see how to use it.

Funny, I am *so* used to examples being at the very end, from tutorials to
man pages.

If my experience is any indication, I would rather keep this order.

> > Signed-off-by: Karsten Blees <blees@dcon.de>
> > Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> > ---
> 
> Just curious: what was Karsten's contribution?

Essentially all of the code. I just moved it around, separated it out from
the patch introducing it, used it elsewhere (e.g. in compat/poll/, not yet
contributed to git/git) and replaced the LoadLibrary() call by
LoadLibraryEx() for more precise control over the operation.

> (I ask mostly because I'm interested in kinds of collaboration git
> metadata is failing to capture correctly --- e.g. pair programming.)

Well, then I have this challenge for you: make use of the history of Git
for Windows' master branch. It is rebased to git/git's maint (or for .0
versions, master) using the "merging rebase" strategy (i.e. first merging
the to-be-rebased history using the "ours" strategy, then applying the
patches on top, so that the previous commit history is part of the new
history, yet there are still rebased patches ready to be prepared for
submission to git/git). This strategy means that multiple versions of the
same patches exist.

> >         So far, there are no users (except in Git for Windows). Ben
> >         promised to make use of it in his fsmonitor patch series.
> >
> >  compat/win32/lazyload.h | 44 ++++++++++++++++++++++++++++++++++++++++++++
> >  1 file changed, 44 insertions(+)
> >  create mode 100644 compat/win32/lazyload.h
> 
> Are any of the Git for Windows users something that could go upstream
> along with this patch?  That would help illustrate what a good caller
> looks like, which should help with reviewing future patches that use
> this code.

I do not currently have the time to do that, that's why I did not
accompany the patch by any user.

However, having said that, Ben's patch series will make for an *excellent*
user, fulfilling your wish.

> > --- /dev/null
> > +++ b/compat/win32/lazyload.h
> > @@ -0,0 +1,44 @@
> [...]
> > +/* Declares a function to be loaded dynamically from a DLL. */
> > +#define DECLARE_PROC_ADDR(dll, rettype, function, ...) \
> > +	static struct proc_addr proc_addr_##function = \
> > +	{ #dll, #function, NULL, 0 }; \
> > +	static rettype (WINAPI *function)(__VA_ARGS__)
> > +
> > +/*
> > + * Loads a function from a DLL (once-only).
> > + * Returns non-NULL function pointer on success.
> > + * Returns NULL + errno == ENOSYS on failure.
> > + */
> > +#define INIT_PROC_ADDR(function) \
> > +	(function = get_proc_addr(&proc_addr_##function))
> 
> Probably worth mentioning in the doc comment that this is not thread
> safe, so a caller that wants to lazy-init in a threaded context is
> responsible for doing their own locking.

True.

> > +static inline void *get_proc_addr(struct proc_addr *proc)
> > +{
> > +	/* only do this once */
> > +	if (!proc->initialized) {
> > +		HANDLE hnd;
> > +		proc->initialized = 1;
> > +		hnd = LoadLibraryExA(proc->dll, NULL,
> > +				     LOAD_LIBRARY_SEARCH_SYSTEM32);
> > +		if (hnd)
> > +			proc->pfunction = GetProcAddress(hnd, proc->function);
> > +	}
> > +	/* set ENOSYS if DLL or function was not found */
> > +	if (!proc->pfunction)
> > +		errno = ENOSYS;
> > +	return proc->pfunction;
> > +}
> 
> strerror(ENOSYS) is "Function not implemented".  Cute.
> 
> Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>

Okay, I'll add that for v2. Will wait a couple of days in case more stuff
crops up.

Ciao,
Dscho

Re: [PATCH] Win32: simplify loading of DLL functions

[Jonathan Nieder]

Johannes Schindelin wrote:
> On Tue, 19 Sep 2017, Jonathan Nieder wrote:

>> Could this example go near the top of the header instead?  That way,
>> it's easier for people reading the header to see how to use it.
>
> Funny, I am *so* used to examples being at the very end, from tutorials to
> man pages.
>
> If my experience is any indication, I would rather keep this order.

Sorry for the lack of clarity.  I meant "near the top of the header
*file*".

[...]
>> Are any of the Git for Windows users something that could go upstream
>> along with this patch?  That would help illustrate what a good caller
>> looks like, which should help with reviewing future patches that use
>> this code.
>
> I do not currently have the time to do that, that's why I did not
> accompany the patch by any user.
>
> However, having said that, Ben's patch series will make for an *excellent*
> user, fulfilling your wish.

Ok.  I think what you are saying is "go ahead --- anyone is welcome to
grab some patches from git for windows and upstream them", which is a
perfectly reasonable answer.

[...]
>> Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
>
> Okay, I'll add that for v2. Will wait a couple of days in case more stuff
> crops up.

FWIW nothing I noticed came to the level of requiring a v2 imho.  If any
of the ideas I mentioned seems good, they can go in patches on top.

The patch is in Junio's jch branch and I wouldn't be surprised if it
hits "next" soon.

Thanks again,
Jonathan

Re: [PATCH] Win32: simplify loading of DLL functions

[Junio C Hamano]

Jonathan Nieder <jrnieder@gmail.com> writes:

> Johannes Schindelin wrote:
>> On Tue, 19 Sep 2017, Jonathan Nieder wrote:
>
>>> Could this example go near the top of the header instead?  That way,
>>> it's easier for people reading the header to see how to use it.
>>
>> Funny, I am *so* used to examples being at the very end, from tutorials to
>> man pages.
>>
>> If my experience is any indication, I would rather keep this order.
>
> Sorry for the lack of clarity.  I meant "near the top of the header
> *file*".

Yeah, I think it would help to have something like that near the top
of the header file for the potential users of the helper.

> FWIW nothing I noticed came to the level of requiring a v2 imho.  If any
> of the ideas I mentioned seems good, they can go in patches on top.

True, too, and additional "doc in header" can also be done as a
follow-up.

But as I heard "v2 in a few days", I'll throw it in the "Expecting a
reroll" bin for now.

Thanks, both.

[PATCH v2 0/1] Simplify loading of DLL functions on Windows

[Johannes Schindelin]

Changes since v1:

- repeated the example from the commit message in the .h file

- mentioned that the function is not thread-safe.

- added Jonathan's Reviewed-by: footer


Johannes Schindelin (1):
  Win32: simplify loading of DLL functions

 compat/win32/lazyload.h | 57 +++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 57 insertions(+)
 create mode 100644 compat/win32/lazyload.h


base-commit: 28996cec80690d2322359d3650a57e8de6e01eb6
Published-As: https://github.com/dscho/git/releases/tag/lazyload-v2
Fetch-It-Via: git fetch https://github.com/dscho/git lazyload-v2

Interdiff vs v1:
 diff --git a/compat/win32/lazyload.h b/compat/win32/lazyload.h
 index 91c10dad2fb..9e631c8593f 100644
 --- a/compat/win32/lazyload.h
 +++ b/compat/win32/lazyload.h
 @@ -1,7 +1,19 @@
  #ifndef LAZYLOAD_H
  #define LAZYLOAD_H
  
 -/* simplify loading of DLL functions */
 +/*
 + * A pair of macros to simplify loading of DLL functions. Example:
 + *
 + *   DECLARE_PROC_ADDR(kernel32.dll, BOOL, CreateHardLinkW,
 + *                     LPCWSTR, LPCWSTR, LPSECURITY_ATTRIBUTES);
 + *
 + *   if (!INIT_PROC_ADDR(CreateHardLinkW))
 + *           return error("Could not find CreateHardLinkW() function";
 + *
 + *   if (!CreateHardLinkW(source, target, NULL))
 + *           return error("could not create hardlink from %S to %S",
 + *                        source, target);
 + */
  
  struct proc_addr {
  	const char *const dll;
 @@ -20,6 +32,7 @@ struct proc_addr {
   * Loads a function from a DLL (once-only).
   * Returns non-NULL function pointer on success.
   * Returns NULL + errno == ENOSYS on failure.
 + * This function is not thread-safe.
   */
  #define INIT_PROC_ADDR(function) \
  	(function = get_proc_addr(&proc_addr_##function))
-- 
2.14.1.windows.1.1024.gf2dea585d74.dirty

[PATCH v2 1/1] Win32: simplify loading of DLL functions

[Johannes Schindelin]

Dynamic loading of DLL functions is duplicated in several places in Git
for Windows' source code.

This patch adds a pair of macros to simplify the process: the
DECLARE_PROC_ADDR(<dll>, <return-type>, <function-name>,
...<function-parameter-types>...) macro to be used at the beginning of a
code block, and the INIT_PROC_ADDR(<function-name>) macro to call before
using the declared function. The return value of the INIT_PROC_ADDR()
call has to be checked; If it is NULL, the function was not found in the
specified DLL.

Example:

        DECLARE_PROC_ADDR(kernel32.dll, BOOL, CreateHardLinkW,
                          LPCWSTR, LPCWSTR, LPSECURITY_ATTRIBUTES);

        if (!INIT_PROC_ADDR(CreateHardLinkW))
                return error("Could not find CreateHardLinkW() function";

	if (!CreateHardLinkW(source, target, NULL))
		return error("could not create hardlink from %S to %S",
			     source, target);
	return 0;

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 compat/win32/lazyload.h | 57 +++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 57 insertions(+)
 create mode 100644 compat/win32/lazyload.h

diff --git a/compat/win32/lazyload.h b/compat/win32/lazyload.h
new file mode 100644
index 00000000000..9e631c8593f
--- /dev/null
+++ b/compat/win32/lazyload.h
@@ -0,0 +1,57 @@
+#ifndef LAZYLOAD_H
+#define LAZYLOAD_H
+
+/*
+ * A pair of macros to simplify loading of DLL functions. Example:
+ *
+ *   DECLARE_PROC_ADDR(kernel32.dll, BOOL, CreateHardLinkW,
+ *                     LPCWSTR, LPCWSTR, LPSECURITY_ATTRIBUTES);
+ *
+ *   if (!INIT_PROC_ADDR(CreateHardLinkW))
+ *           return error("Could not find CreateHardLinkW() function";
+ *
+ *   if (!CreateHardLinkW(source, target, NULL))
+ *           return error("could not create hardlink from %S to %S",
+ *                        source, target);
+ */
+
+struct proc_addr {
+	const char *const dll;
+	const char *const function;
+	FARPROC pfunction;
+	unsigned initialized : 1;
+};
+
+/* Declares a function to be loaded dynamically from a DLL. */
+#define DECLARE_PROC_ADDR(dll, rettype, function, ...) \
+	static struct proc_addr proc_addr_##function = \
+	{ #dll, #function, NULL, 0 }; \
+	static rettype (WINAPI *function)(__VA_ARGS__)
+
+/*
+ * Loads a function from a DLL (once-only).
+ * Returns non-NULL function pointer on success.
+ * Returns NULL + errno == ENOSYS on failure.
+ * This function is not thread-safe.
+ */
+#define INIT_PROC_ADDR(function) \
+	(function = get_proc_addr(&proc_addr_##function))
+
+static inline void *get_proc_addr(struct proc_addr *proc)
+{
+	/* only do this once */
+	if (!proc->initialized) {
+		HANDLE hnd;
+		proc->initialized = 1;
+		hnd = LoadLibraryExA(proc->dll, NULL,
+				     LOAD_LIBRARY_SEARCH_SYSTEM32);
+		if (hnd)
+			proc->pfunction = GetProcAddress(hnd, proc->function);
+	}
+	/* set ENOSYS if DLL or function was not found */
+	if (!proc->pfunction)
+		errno = ENOSYS;
+	return proc->pfunction;
+}
+
+#endif
-- 
2.14.1.windows.1.1024.gf2dea585d74.dirty

Re: [PATCH v2 1/1] Win32: simplify loading of DLL functions

[Jonathan Nieder]

Johannes Schindelin wrote:

> Dynamic loading of DLL functions is duplicated in several places in Git
> for Windows' source code.
>
> This patch adds a pair of macros to simplify the process: the
> DECLARE_PROC_ADDR(<dll>, <return-type>, <function-name>,
> ...<function-parameter-types>...) macro to be used at the beginning of a
> code block, and the INIT_PROC_ADDR(<function-name>) macro to call before
> using the declared function. The return value of the INIT_PROC_ADDR()
> call has to be checked; If it is NULL, the function was not found in the
> specified DLL.
>
> Example:
>
>         DECLARE_PROC_ADDR(kernel32.dll, BOOL, CreateHardLinkW,
>                           LPCWSTR, LPCWSTR, LPSECURITY_ATTRIBUTES);
>
>         if (!INIT_PROC_ADDR(CreateHardLinkW))
>                 return error("Could not find CreateHardLinkW() function";
>
> 	if (!CreateHardLinkW(source, target, NULL))
> 		return error("could not create hardlink from %S to %S",
> 			     source, target);
> 	return 0;
>
> Signed-off-by: Karsten Blees <blees@dcon.de>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>

Yep, this is indeed

Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>

> ---
>  compat/win32/lazyload.h | 57 +++++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 57 insertions(+)
>  create mode 100644 compat/win32/lazyload.h

Thanks,
Jonathan