Location limits on development, staging and production environments

Location limits on development, staging and production environments

[H]

I am a newcomer to git looking to set up a web development environment where individual computers are used for development, the development.git, staging.git and production.git repositories are stored on an external server reachable by password-less ssh and the staging and production websites are on yet another server, also reachable by password-less ssh from the git-server (and the development machines).

Locating the three git repositories on an external server works fine but I have not been able to have the staging and production deployment files on another server. I believe this is what is referred by GIT_WORK_TREE and based on what I found on the web I created a post-receive hook of staging.git with the two lines:

#!/bin/sh
GIT_WORK_TREE=user@1.2.3.4:/var/www/html/dev.whatever git checkout -f master

I believe this should deploy the files from the development work tree.

The above, however, fails. Should it work? I am running git 1.7.1 on CentOS 6.

Re: Location limits on development, staging and production environments

[Bryan Turner]

On Mon, Jan 29, 2018 at 11:08 AM, H <agents@meddatainc.com> wrote:
> I am a newcomer to git looking to set up a web development environment where individual computers are used for development, the development.git, staging.git and production.git repositories are stored on an external server reachable by password-less ssh and the staging and production websites are on yet another server, also reachable by password-less ssh from the git-server (and the development machines).
>
> Locating the three git repositories on an external server works fine but I have not been able to have the staging and production deployment files on another server. I believe this is what is referred by GIT_WORK_TREE and based on what I found on the web I created a post-receive hook of staging.git with the two lines:
>
> #!/bin/sh
> GIT_WORK_TREE=user@1.2.3.4:/var/www/html/dev.whatever git checkout -f master
>
> I believe this should deploy the files from the development work tree.
>
> The above, however, fails. Should it work? I am running git 1.7.1 on CentOS 6.

No, I wouldn't expect that to work. GIT_WORK_TREE is not remote-aware
in that way. It's expected to be a normal-ish filesystem path.

Based on your description, and the hook you've written, it seems like
your intention is for the source to automatically be fetched and
checked out on the staging environment after each push. (This is
dangerous, and likely _not_ what you actually want, but I'll get to
that in a moment.)

One option would be to setup something like NFS, so the git-server can
mount the filesystems from the staging and production nodes.

A different, likely better, option would be to have the post-receive
script on the git-server use straight ssh to trigger a checkout script
on the staging server, e.g.:
#!/bin/sh
ssh example@staging-server -C /opt/deploy-staging.sh

Your deploy-staging script would then do something like:
#!/bin/sh
GIT_WORK_TREE=/var/www/html/dev.whatever git pull origin

That said, though, having such a simple script is dangerous because
Git is fully capable of having receiving multiple pushes concurrently,
and they can all succeed as long as they're updating different
branches. Since your script isn't considering what branches were
changed by the push, it could end up triggering simultaneous git
processes on the staging server all attempting to deploy concurrently.

The stdin for the post-receive hook receives details about which refs
were changed, and you'll likely want to update your script to parse
stdin and only try to deploy staging if a specific, relevant branch
(master in your example) has changed.

Lastly, I'll note that using post-receive will make the pushing
(remote) user wait while the staging server is deployed. If that
process is likely to take very long, you might want to decouple the
two somehow.

Hope this helps!

Re: Location limits on development, staging and production environments

[H]

On 01/29/2018 10:02 PM, Bryan Turner wrote:
> On Mon, Jan 29, 2018 at 11:08 AM, H <agents@meddatainc.com> wrote:
>> I am a newcomer to git looking to set up a web development environment where individual computers are used for development, the development.git, staging.git and production.git repositories are stored on an external server reachable by password-less ssh and the staging and production websites are on yet another server, also reachable by password-less ssh from the git-server (and the development machines).
>>
>> Locating the three git repositories on an external server works fine but I have not been able to have the staging and production deployment files on another server. I believe this is what is referred by GIT_WORK_TREE and based on what I found on the web I created a post-receive hook of staging.git with the two lines:
>>
>> #!/bin/sh
>> GIT_WORK_TREE=user@1.2.3.4:/var/www/html/dev.whatever git checkout -f master
>>
>> I believe this should deploy the files from the development work tree.
>>
>> The above, however, fails. Should it work? I am running git 1.7.1 on CentOS 6.
> No, I wouldn't expect that to work. GIT_WORK_TREE is not remote-aware
> in that way. It's expected to be a normal-ish filesystem path.
>
> Based on your description, and the hook you've written, it seems like
> your intention is for the source to automatically be fetched and
> checked out on the staging environment after each push. (This is
> dangerous, and likely _not_ what you actually want, but I'll get to
> that in a moment.)
>
> One option would be to setup something like NFS, so the git-server can
> mount the filesystems from the staging and production nodes.
>
> A different, likely better, option would be to have the post-receive
> script on the git-server use straight ssh to trigger a checkout script
> on the staging server, e.g.:
> #!/bin/sh
> ssh example@staging-server -C /opt/deploy-staging.sh
>
> Your deploy-staging script would then do something like:
> #!/bin/sh
> GIT_WORK_TREE=/var/www/html/dev.whatever git pull origin
>
> That said, though, having such a simple script is dangerous because
> Git is fully capable of having receiving multiple pushes concurrently,
> and they can all succeed as long as they're updating different
> branches. Since your script isn't considering what branches were
> changed by the push, it could end up triggering simultaneous git
> processes on the staging server all attempting to deploy concurrently.
>
> The stdin for the post-receive hook receives details about which refs
> were changed, and you'll likely want to update your script to parse
> stdin and only try to deploy staging if a specific, relevant branch
> (master in your example) has changed.
>
> Lastly, I'll note that using post-receive will make the pushing
> (remote) user wait while the staging server is deployed. If that
> process is likely to take very long, you might want to decouple the
> two somehow.
>
> Hope this helps!

I should perhaps also have mentioned that although I am the only developer, I may use different computers to develop on. IOW, there should not be any conflict due to code being pushed by multiple developers.

Let's see if I understand this correctly:

- Unless NFS is used, the git archive and the deployment of the website code in this case should reside on the same computer.

- The combination of the checkout script and the deploy-staging script should work provided not multiple updates to the same branch are pushed at the same time.

I will try this later today but any other hints or suggestions you may have would be greatly appreciated!

Re: Location limits on development, staging and production environments

[H]

On 01/30/2018 03:48 PM, H wrote:
> On 01/29/2018 10:02 PM, Bryan Turner wrote:
>> On Mon, Jan 29, 2018 at 11:08 AM, H <agents@meddatainc.com> wrote:
>>> I am a newcomer to git looking to set up a web development environment where individual computers are used for development, the development.git, staging.git and production.git repositories are stored on an external server reachable by password-less ssh and the staging and production websites are on yet another server, also reachable by password-less ssh from the git-server (and the development machines).
>>>
>>> Locating the three git repositories on an external server works fine but I have not been able to have the staging and production deployment files on another server. I believe this is what is referred by GIT_WORK_TREE and based on what I found on the web I created a post-receive hook of staging.git with the two lines:
>>>
>>> #!/bin/sh
>>> GIT_WORK_TREE=user@1.2.3.4:/var/www/html/dev.whatever git checkout -f master
>>>
>>> I believe this should deploy the files from the development work tree.
>>>
>>> The above, however, fails. Should it work? I am running git 1.7.1 on CentOS 6.
>> No, I wouldn't expect that to work. GIT_WORK_TREE is not remote-aware
>> in that way. It's expected to be a normal-ish filesystem path.
>>
>> Based on your description, and the hook you've written, it seems like
>> your intention is for the source to automatically be fetched and
>> checked out on the staging environment after each push. (This is
>> dangerous, and likely _not_ what you actually want, but I'll get to
>> that in a moment.)
>>
>> One option would be to setup something like NFS, so the git-server can
>> mount the filesystems from the staging and production nodes.
>>
>> A different, likely better, option would be to have the post-receive
>> script on the git-server use straight ssh to trigger a checkout script
>> on the staging server, e.g.:
>> #!/bin/sh
>> ssh example@staging-server -C /opt/deploy-staging.sh
>>
>> Your deploy-staging script would then do something like:
>> #!/bin/sh
>> GIT_WORK_TREE=/var/www/html/dev.whatever git pull origin
>>
>> That said, though, having such a simple script is dangerous because
>> Git is fully capable of having receiving multiple pushes concurrently,
>> and they can all succeed as long as they're updating different
>> branches. Since your script isn't considering what branches were
>> changed by the push, it could end up triggering simultaneous git
>> processes on the staging server all attempting to deploy concurrently.
>>
>> The stdin for the post-receive hook receives details about which refs
>> were changed, and you'll likely want to update your script to parse
>> stdin and only try to deploy staging if a specific, relevant branch
>> (master in your example) has changed.
>>
>> Lastly, I'll note that using post-receive will make the pushing
>> (remote) user wait while the staging server is deployed. If that
>> process is likely to take very long, you might want to decouple the
>> two somehow.
>>
>> Hope this helps!
> I should perhaps also have mentioned that although I am the only developer, I may use different computers to develop on. IOW, there should not be any conflict due to code being pushed by multiple developers.
>
> Let's see if I understand this correctly:
>
> - Unless NFS is used, the git archive and the deployment of the website code in this case should reside on the same computer.
>
> - The combination of the checkout script and the deploy-staging script should work provided not multiple updates to the same branch are pushed at the same time.
>
> I will try this later today but any other hints or suggestions you may have would be greatly appreciated!
>
I modified post-receive in the hook directory of the staging git archive on server 1 as per your first example and created the deploy-staging.sh script on server 2 where the staging website resides in /var/www/html/dev.whatever (as well as the live website in /var/www/html/whatever. However, when trying out the deploy-staging.sh script, it fails with the error message "...not a git repository...". I do have git on server 2 but what else would I need to set up to run this successfully? Note that the git repository is on server 1 and I am deploying to server 2.

Thanks!