While chatting with @Leniy about auto-deploying with git, @Leniy asked how to set it up. Since there's interest, let me ramble a bit.
Git is, without doubt, a wonderful version control system — simple, easy to use, and gentle on the hands. And if you set up a git repo on a VPS, you can deploy directly to the environment on every push, without having to SSH in and fumble around against a XXXms latency. Note: I'm just giving an example that meets the "auto-deploy" requirement — there's much more you can do with it.
First, the requirements:
- Both the client and the server need to have git installed.
- You need to be able to push to the server.
- You need to be able to modify the server environment — FTP access or SSH access both work, actually. For safety, though, it's best to keep the git directory as the parent of the web root.
Second, the principle:
Auto-deployment with git makes use of hooks (Chinese version); you can also refer to this.
Simply put, in the hooks folder on the server, there are several files; when you perform various operations, specific bash scripts get triggered, and git then executes them.
Inside GIT_DIR/hooks/ you'll find various xxx.sample files. If you're interested, you can study how they were written.
Since this is just a test, I'm doing everything locally, but in reality one side is the client and the other is the server, with operations on both ends. You'll need to keep track of which end you're on.
Alright, enough preamble — let's walk through a complete example.
1. Initialize the server-side git
On the server, I create a repo folder under $HOME/ and initialize a test.git
[yu@argcv-com ~]$ cd ~
[yu@argcv-com ~]$ mkdir repo
[yu@argcv-com ~]$ cd repo/
[yu@argcv-com repo]$ git init --bare test.git
Initialized empty Git repository in /home/yu/repo/test.git/
[yu@argcv-com repo]$ 2. Initialize the client
I create a client under /tmp/client
[yu@argcv-com ~]$ cd /tmp/
[yu@argcv-com tmp]$ mkdir client
[yu@argcv-com tmp]$ cd client/
[yu@argcv-com client]$ git init
Initialized empty Git repository in /tmp/client/.git/
[yu@argcv-com client]$ git remote add origin yu@localhost:repo/test.git
[yu@argcv-com client]$ touch README.md
[yu@argcv-com client]$ git add .
[yu@argcv-com client]$ git commit -m "init"
[master (root-commit) 00da9a9] init
1 file changed, 0 insertions(+), 0 deletions(-)
create mode 100644 README.md
[yu@argcv-com client]$ git push origin --all
Counting objects: 3, done.
Writing objects: 100% (3/3), 210 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
To yu@localhost:repo/test.git
* [new branch] master -> master
[yu@argcv-com client]$ 3. Initialize the deployment environment on the server
The server deploys to /tmp/deploy/
[yu@argcv-com ~]$ cd /tmp/
[yu@argcv-com tmp]$ mkdir deploy
[yu@argcv-com tmp]$ cd deploy/
[yu@argcv-com deploy]$ git clone $HOME/repo/test.git .
Cloning into '.'...
done.A special reminder: at this point there's a hidden .git folder under /tmp/deploy
For safety, don't put your web root on the same level as it — it should be a subdirectory of /tmp/deploy.
4. Add hooks on the server
As mentioned earlier, the server-side git lives in $HOME/repo/test.git/
Add the file $HOME/repo/test.git/hooks/post-receive to do the deployment.
Note that there are several triggers under hooks, corresponding to actions taken at different stages.
For example, "update" actually corresponds to an action before the update happens; this bash script could be used for backups. If you use it for auto-deployment, every deployment will be one step behind... which would be rather awkward.
[yu@argcv-com repo]$ pwd
/home/yu/repo
[yu@argcv-com repo]$ cd test.git/
[yu@argcv-com test.git]$ cd hooks/
[yu@argcv-com hooks]$ cat > post-receive <<'EOF'
#!/bin/sh
unset GIT_DIR
NowPath=`pwd`
echo "now path is :"$NowPath
DeployPath="/tmp/deploy"
echo "deploy path is :"$DeployPath
cd $DeployPath
echo "cd deploy path"
git add . -A && git stash # remove local changes
git pull origin master # pull data from master
# the follow line is also ok:
# git add . && git fetch origin && git reset --hard origin/master
echo "deploy done"
cd $NowPath
echo "fine"
# --- Finished
exit 0
EOF
[yu@argcv-com hooks]$ chmod +x post-receiveA hook is just a bash script, and it's self-explanatory: go to the deployment directory, run git pull or fetch && reset, and the deployment is done. The reason for "git add . -A" is that you might create some untracked files, and if they aren't added, version control may not be able to modify them.
Of course, you could also run actions like backups.
Like any ordinary bash script, make it executable.
Just like with a Makefile, the working directory is fixed on every line — you need to unset the GIT_DIR environment variable to move around freely.
Of course, I could also use
(cd $DeployPath && git add . -A && git stash && git pull origin master )as a replacement, if things are simple enough.
About data files.
Operations like stash and reset will wipe out all modifications.
Generally speaking, we shouldn't keep data files together with code files. In terms of engineering design, we should place data files elsewhere anyway. So this logic doesn't have any big problems. But in practice, for compatibility reasons, third-party upstream code, spur-of-the-moment decisions, configuration issues, and so on, we may have to put something under the project directory. In that case, we need to configure a gitignore file in the project — files whitelisted in it will be ignored.
5. Testing
First, on the client, add a file, commit it, and push
[yu@argcv-com client]$ touch newfile
[yu@argcv-com client]$ git add .
[yu@argcv-com client]$ git commit -m "add new file"
[master 68efdcf] add new file
1 file changed, 0 insertions(+), 0 deletions(-)
create mode 100644 newfile
[yu@argcv-com client]$ git push origin
Counting objects: 4, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 384 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
remote: now path is :/home/yu/repo/test.git
remote: deploy path is :/tmp/deploy
remote: cd deploy path
remote: From /home/yu/repo/test
remote: 6627165..5ac80ec master -> origin/master
remote: Updating 6627165..5ac80ec
remote: Fast-forward
remote: newfile | 0
remote: 1 file changed, 0 insertions(+), 0 deletions(-)
remote: create mode 100644 newfile
remote: deploy done
remote: fine
To yu@localhost:repo/test.git
b3a32f2..5ac80ec master -> master
[yu@argcv-com client]$ Then check the files deployed on the server
[yu@argcv-com deploy]$ ls -al
total 0
drwxrwxr-x 3 yu yu 100 Apr 26 17:29 .
drwxrwxrwt 22 root root 460 Apr 26 17:29 ..
drwxrwxr-x 8 yu yu 280 Apr 26 17:29 .git
-rw-rw-r-- 1 yu yu 0 Apr 26 17:29 newfile
-rw-rw-r-- 1 yu yu 0 Apr 26 17:13 README.mdThat's the basic principle.
6. Related notes
User permission issues
In actual use, some people have reported Permission Denied–type problems.
This mostly happens when the user your Web Service runs as differs from the user doing the git push. The usual approach to solving it goes like this.
First, add the git user to the web user group, for example with the command
sudo usermod -a -G web gitIf you've put your target directory under /var/www/your_git, then you should check whether the permissions on /var, /var/www, /var/www, and /var/www/your_git are open enough.
Generally speaking, all parent directories need at least read permission for the git user, while the git directory needs write permission. For me, /var is shared space that everyone can usually see anyway, so I won't go into detail. Under /var/www/ there might be several projects being managed, so one approach worth considering is setting its owner to the root user and the web group. Then we can apply appropriate restrictions to the /var/www directory by controlling group permissions. For example, here I'd suggest setting /var/www to 750 (sudo chmod 750 /var/www). FAQ: How do directory permissions in Linux work?, and then setting /var/www/your_git to 770.
Then you also need to consider whether SELinux is getting in the way. Usually after modifying permissions, running sudo restorecon -R /var/www will do the trick.
You can also try checking the web directory as the git user with the command
sudo -u git > /var/www/your_git/test_fileThis command attempts to write an empty file named test_file in the /var/www/your_git/ directory as the git user.
Controlling the update frequency
A good practice is to use multiple branches. For example, if you normally push to the remote master branch, you can create a branch named publish. Change the script above to use "origin/publish".
Then when you run
git push origin masternothing will update on the remote. But when you run
git push origin master:publishthe remote publish branch will be updated, and the hook will put the latest state of publish into the running environment.
webhooks and CI tools
For projects hosted on github or gitlab, the servers come with all kinds of hooks built in. For security reasons, it's hard for us as users to access the underlying scripts. Generally we're recommended to use webhooks. We need to provide a web endpoint on our own server that's exposed externally. After configuring the project, whenever a specific event occurs, we'll receive a push event containing lots of relevant information, which we can use to perform follow-up actions.
In addition, we can also make good use of tools like travis ci, which are actually higher-level applications also based on webhooks. Even for public projects, they allow us to set some private environment variables. Then whenever a deployment succeeds, we can run some scripts.
If we want to host on github but don't want to waste the deployment scripts we've already written, we can also write a deploy script in travis's config file, pushing the latest code back to our server.
deploy: # 只有前面测试脚本通过后才执行它
- provider: script
keep-history: true
skip-cleanup: false
script: echo $ID_RSA > $HOME/id_rsa && chmod 0600 $HOME/id_rsa && GIT_SSH_COMMAND="ssh -i $HOME/id_rsa" git push git@myserver:proj master --force
on:
branch: master # 只有 master 文件才会执行 pushFor example, you can use a script like the one above. In travis's global environment variables, set ID_RSA to the contents of your private key, and then just push directly to your own server. From then on, you only need to push code to github.
gitlab and others also have built-in CI. In addition, we can also set up our own CI service, such as drone. Either way, the basic logic is one thing: when we push some code, leave the rest to the various "when foo happens, do bar" hooks — don't make trouble for yourself.
7. Conclusion
The original intent of this post was to give a crude, simple introduction to some considerations around decoupling. How do we monitor the occurrence of an event? Besides bombarding the other side's state management tool with heartbeat packets day and night, we can choose another approach: when the event occurs, let the event tell us.
This is actually a very common practice. In programs there's callback, and in libraries there's libevent. Because when you need to handle a large number of events, polling has no future :(. And the hooks mentioned in this article are essentially a general-purpose tool provided by git, letting us take advantage of this kind of model at a higher level of abstraction. Hopefully this helps everyone escape repetitive work more efficiently.
Comments · 44