rsync的配置应用

转载

kang50055 2018-05-31 07:38:11

文章标签 linux rsync 文章分类 运维

高可用软件：keepalived 负载均衡：nginx/lvs/haproxy web服务器：nginx/apache 数据库：MySQL/MongoDB/Oracle 存储：NFS/FastdFS/GFS/MFS 备份：Rsync+定时任务实时复制：Rsync+inotify/sersync 缓存：Redis/Memcached 秘密通道：Open×××/××× 跳板机：Shell/JumpServer/CreazyEYE/GateONE 批量管理：Ansible/SaltStack/Pssh 监控：Zabbix/Nagios+cacti 网站的更新过程（代码上线流程）：办公室测试环境-线上测试环境-线上环境 1.从安装系统开始准备安装虚拟机过程中添加网卡（网络适配器2 NAT）

网络适配器2，LAN区段：172.16.1.0/24

eth0 ip 10.0.0.210 netmask 24 gateway 10.0.0.254 DNS 223.5.5.5

eth1 ip 172.16.1.210 netmask 24

磁盘分区如下：

安装包选择如下：

2.统一网络环境网关都是 10.0.0.254

本机我的电脑===>网络连接

2.5 服务器ip地址规划服务器说明外网IP（NAT）内网IP（LAN区段/Host-only）主机名规划 A1-nginx负载服务器 01 10.0.0.5/24 172.16.1.5/24 lb01 A1-nginx负载服务器 02 10.0.0.6/24 172.16.1.6/24 lb02 B2-nginx web服务器10.0.0.7/24 172.16.1.7/24 web02 B2-nginx web服务器10.0.0.8/24 172.16.1.8/24 web01 C3-mysql数据库服务器10.0.0.51/24 172.16.1.51/24 db01 C1-NFS存储服务器10.0.0.31/24 172.16.1.31/24 nfs01 C2-rsync存储服务器10.0.0.41/24 172.16.1.41/24 backup
X-管理服务器10.0.0.61/24 172.16.1.61/24 m01

3.系统优化

#所有服务器模板机的基本优化 cp /etc/hosts{,.bak} cat >/etc/hosts<<EOF 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 172.16.1.5 lb01 172.16.1.6 lb02 172.16.1.7 web02 172.16.1.8 web01 172.16.1.51 db01 db01.etiantian.org 172.16.1.31 nfs01 172.16.1.41 backup 172.16.1.61 m01 EOF #0、更改yum源 mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.backup mv /etc/yum.repos.d/epel.repo /etc/yum.repos.d/epel.repo.backup mv /etc/yum.repos.d/epel-testing.repo /etc/yum.repos.d/epel-testing.repo.backup wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-6.repo wget -O /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-6.repo #yum makecache #1、关闭selinux sed -i.bak 's/SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/config grep SELINUX=disabled /etc/selinux/config setenforce 0 getenforce #2、关闭iptables /etc/init.d/iptables stop /etc/init.d/iptables stop chkconfig iptables off #3、精简开机自启动服务 export LANG=en chkconfig|egrep -v "crond|sshd|network|rsyslog|sysstat"|awk '{print"chkconfig",$1,"off"}'|bash chkconfig --list|grep 3:on #4、提权oldboy可以sudo useradd oldboy echo 123456|passwd --stdin oldboy \cp /etc/sudoers /etc/sudoers.ori echo "oldboy ALL=(ALL) NOPASSWD: ALL " >>/etc/sudoers visudo -c #5、英文字符集 cp /etc/sysconfig/i18n /etc/sysconfig/i18n.ori echo 'LANG="en_US.UTF-8"' >/etc/sysconfig/i18n source /etc/sysconfig/i18n echo $LANG #6、时间同步 echo '#time sync by lidao at 2017-03-08' >>/var/spool/cron/root echo '/5 * * * * /usr/sbin/ntpdate ntp1.aliyun.com >/dev/null 2>&1' >>/var/spool/cron/root crontab -l #8、加大文件描述 echo ' -nofile 65535 ' >>/etc/security/limits.conf tail -1 /etc/security/limits.conf #9、内核优化 cat >>/etc/sysctl.conf<<EOF net.ipv4.tcp_fin_timeout = 2 net.ipv4.tcp_tw_reuse = 1 net.ipv4.tcp_tw_recycle = 1 net.ipv4.tcp_syncookies = 1 net.ipv4.tcp_keepalive_time = 600 net.ipv4.ip_local_port_range = 4000 65000 net.ipv4.tcp_max_syn_backlog = 16384 net.ipv4.tcp_max_tw_buckets = 36000 net.ipv4.route.gc_timeout = 100 net.ipv4.tcp_syn_retries = 1 net.ipv4.tcp_synack_retries = 1 net.core.somaxconn = 16384 net.core.netdev_max_backlog = 16384 net.ipv4.tcp_max_orphans = 16384 #以下参数是对iptables防火墙的优化，防火墙不开会提示，可以忽略不理。 net.nf_conntrack_max = 25000000 net.netfilter.nf_conntrack_max = 25000000 net.netfilter.nf_conntrack_tcp_timeout_established= 180 net.netfilter.nf_conntrack_tcp_timeout_time_wait = 120 net.netfilter.nf_conntrack_tcp_timeout_close_wait = 60 net.netfilter.nf_conntrack_tcp_timeout_fin_wait = 120 EOF sysctl -p #10、安装其他小软件 yum install lrzsz nmap tree dos2unix nc telnet sl -y #11、ssh连接速度慢优化 sed -i.bak 's@#UseDNS yes@UseDNS no@g;s@^GSSAPIAuthentication yes@GSSAPIAuthentication no@g' /etc/ssh/sshd_config /etc/init.d/sshd reload

4.学会克隆虚拟机 !!!!!!!!!! 克隆前准备 1清2删 1清

/etc/udev/rules.d/70-persistent-net.rules

2删 sed -ri '/UUID|HWADDR/d' /etc/sysconfig/network-scripts/ifcfg-eth*

修改IP地址、修改主机名、重启网卡 sed -i '/IPADDR/s#210$#31#g' /etc/sysconfig/network-scripts/ifcfg-eth* vi /etc/sysconfig/network /etc/init.d/network restart 20180526 完成

Rsync 的企业工作场景说明

1.）定时备份 1.1.生产场景集群架构服务器备份方案项目借助cron+rsync把所有客户服务器数据同步到备份服务器

1.针对公司重要数据备份混乱状况和领导提出备份全网数据的解决方案。 2.通过本地打包备份，然后rsync结合inotify应用把全网数据统一备份到一个固定存储服务器，然后在存储服务器上通过脚本检查并报警管理员备份结果。 3.定期将IDC机房的数据备份公司的内部不服务器，防止机房地震及火灾问题导致数据丢失。

Rsync的工作方式：最常用的：以守护进程（socket）的方式传输数据（这个是rsync自身的重要功能）。一般用推送

2.）实时复制

本地数据传输模式（local-only mode） /oldboy/ 与 /oldboy /oldboy 表示oldboy目录及oldboy目录下面的内容 /oldboy/ 只表示oldboy目录下面的内容 rsync /etc/hosts /opt/ rsync -avz /etc /opt/ --delete 复制的时候保持目标与源目录的内容一模一样例子：# rsync -avz --delete /opt/ /tmp/

通过远程shell进行数据传输 scp命令例子：# scp -rP 22 /etc/ 172.16.1.31:/tmp rsync命令 -v 详细模式输出 -z 传输时进行压缩 -a 以递归方式传输文件，并保持所有文件属性

例子：# rsync -avz /etc/sysconfig 172.16.1.31:/tmp/ （推送，用的最多）

--bwlimit=RATE limit socket I/O bandwidth（案例：某DBA做数据同步，带宽占满，导致用户无法访问网站）

rsync服务端backup配置

[root@backup ~]# mkdir /backup [root@backup ~]# useradd -s /sbin/nologin -M rsync [root@backup ~]# chown -R rsync.rsync /backup [root@backup ~]# ls -ld /backup drwxr-xr-x 2 rsync rsync 4096 Jan 18 23:18 /backup [root@backup ~]# echo 'rsync_backup:123456' >/etc/rsync.password [root@backup ~]# cat /etc/rsync.password rsync_backup:123456 [root@backup ~]# chmod 600 /etc/rsync.password [root@backup ~]# ll /etc/rsync.password -rw------- 1 root root 20 Jan 18 23:32 /etc/rsync.password

[root@backup ~]# vim /etc/rsyncd.conf ######rsync_config_______________start #created by hkping 15:01 2018-5-27 ##rsyncd.conf start## uid = rsync gid = rsync use chroot = no max connections = 200 timeout = 300 pid file = /var/run/rsyncd.pid lock file = /var/run/rsync.lock log file = /var/log/rsyncd.log [backup] path = /backup/ ignore errors read only = false list = false hosts allow = 172.16.1.0/24 #hosts deny = 0.0.0.0/32 auth users = rsync_backup secrets file = /etc/rsync.password

[root@backup /]# rsync --daemon [root@backup ~]# ps -ef |grep rsync root 24946 1 0 23:34 ? 00:00:00 rsync --daemonroot 24948 1787 0 23:34 pts/1 00:00:00 grep rsync [root@backup ~]# ss -lntup |grep rsync tcp LISTEN 0 5 :::873 :::* users:(("rsync",24946,5)) tcp LISTEN 0 5 *:873 : users:(("rsync",24946,4))

客户端 [root@nfs01 tmp]# echo '123456' > /etc/rsync.password [root@nfs01 tmp]# chmod 600 /etc/rsync.password [root@nfs01 tmp]# rsync -avzP /etc/hosts rsync_backup@172.16.1.41::backup --password-file=/etc/rsync.password sending incremental file list sent 26 bytes received 8 bytes 68.00 bytes/sec total size is 349 speedup is 10.26 [root@nfs01 /]# rsync -avzP --delete /tmp/ rsync_backup@172.16.1.41::backup --password-file=/etc/rsync.password #--delete 无差异同步

更改完/etc/rsyncd.conf配置文件后，重启rsync --daemon [root@backup ~]# killall rsync [root@backup ~]# killall rsync rsync: no process killed [root@backup ~]# ps -ef|grep rsync root 26693 26671 0 19:13 pts/0 00:00:00 grep rsync [root@backup ~]# rsync --daemon [root@backup ~]# ps -ef|grep rsync root 26695 1 0 19:13 ? 00:00:00 rsync --daemon root 26697 26671 0 19:13 pts/0 00:00:00 grep rsync

rsync配置多个模块 [root@backup ~]# cat /etc/rsyncd.conf ######rsync_config_______________start #created by oldboy 15:01 2007-6-5 #QQ 31333741 blog:http://oldboy.blog.51cto.com ##rsyncd.conf start## uid = rsync gid = rsync use chroot = no max connections = 200 timeout = 300 pid file = /var/run/rsyncd.pid lock file = /var/run/rsync.lock log file = /var/log/rsyncd.log

ignore errors read only = false list = false hosts allow = 172.16.1.0/24 #hosts deny = 0.0.0.0/32 auth users = rsync_backup secrets file = /etc/rsync.password

[backup] path = /backup/ [nfsbackup] path = /nfsbackup/

更改完/etc/rsyncd.conf配置文件后，重启rsync --daemon，新建nfsbackup，更改目录权限 [root@backup backup]# killall rsync rsync: no process killed [root@backup backup]# ps -ef|grep rsync root 25681 1844 0 23:21 pts/0 00:00:00 grep rsync [root@backup backup]# rsync --daemon [root@backup /]# mkdir -p /nfsbackup [root@backup /]# chown -R rsync.rsync /nfsbackup/ [root@backup /]# ll -d /nfsbackup/ drwxr-xr-x 2 rsync rsync 4096 May 27 23:35 /nfsbackup/

某公司里有一台Web服务器，里面的数据很重要，但是如果硬盘坏了，数据就会丢失，现在领导要求你把数据在其他机器上做一个周期性定时备份。要求如下：每天晚上00点整在Web服务器A(web01)上打包备份网站程序目录并通过rsync命令推送到服务器B(backup)上备份保留（备份思路可以是先在本地按日期打包，然后再利用rsync推到备份服务器上）。具体要求如下： 1)Web服务器A和备份服务器B的备份目录必须都为/backup。2)Web服务器站点目录假定为(/var/www/html)。3)Web服务器本地仅保留7天内的备份。4)备份服务器上每周六的数据都保留，其他备份仅保留180天备份。 5)备份服务器上检查备份结果是否正常，并将每天的备份结果发给管理员信箱。 rsync服务端 [root@backup ~]# killall rsync [root@backup ~]# killall rsyncrsync : no process killed [root@backup ~]# rsync --daemon [root@backup ~]#[root@backup ~]#[root@backup ~]# ps -ef |grep rsyncroot 26755 1 0 19:45 ? 00:00:00 rsync --daemonroot 26757 26671 0 19:45 pts/0 00:00:00 grep rsync [root@backup ~]# cat /etc/rsyncd.conf######rsync_config_______________start#created by oldboy 15:01 2007-6-5 #QQ 31333741 blog:http://oldboy.blog.51cto.com##rsyncd.conf start##uid = rsyncgid = rsyncuse chroot = no max connections = 200 timeout = 300 pid file = /var/run/rsyncd.pidlock file = /var/run/rsync.locklog file = /var/log/rsyncd.log ignore errors read only = false list = false hosts allow = 172.16.1.0/24 #hosts deny = 0.0.0.0/32 auth users = rsync_backup secrets file = /etc/rsync.password

[backup] path = /backup/

[root@backup ~]# ll /etc/rsync.password -rw------- 1 root root 20 Jan 18 23:32 /etc/rsync.password [root@backup ~]# cat /etc/rsync.password rsync_backup:123456 [root@backup ~]# ll /backup/ total 0 [root@backup ~]#

[root@web01 ~]# echo '123456'>/etc/rsync.password [root@web01 ~]# chmod 600 /etc/rsync.password [root@web01 ~]# ll /etc/rsync.password -rw------- 1 root root 7 Jan 19 19:50 /etc/rsync.password [root@web01 ~]# [root@web01 ~]# [root@web01 ~]# [root@web01 ~]# [root@web01 ~]# rsync -avz /etc/hosts rsync_backup@172.16.1.41::backup --password-file=/etc/rsync.password sending incremental file list hosts

sent 196 bytes received 27 bytes 446.00 bytes/sectotal size is 349 speedup is 1.57 web01脚本 #backup file && conf tar zchf /backup/web01-$(date +%F_%w).tar.gz /var/www/html/ /etc/rc.local /var/spool/cron/

#make md5sum cd /backup/ md5sum web01-$(date +%F_%w).tar.gz >check.log

#rsync file to backup rsync -avz /backup/ rsync_backup@172.16.1.41::backup --password-file=/etc/rsync.password

#delete 7 day ago find /backup/ -type f -name "*.tar.gz" -mtime +7|xargs rm

#!/bin/bash #desc: backup html && conf

Date=$(date +%F_%w)

#backup file && conf cd / &&
tar zchf /backup/web01-${Date}.tar.gz var/www/html/ etc/rc.local var/spool/cron/

#make md5sum cd /backup/ &&
md5sum web01-${Date}.tar.gz >check.log

#rsync file to backup rsync -az /backup/ rsync_backup@172.16.1.41::backup --password-file=/etc/rsync.password

#delete 7 day ago find /backup/ -type f -name "*.tar.gz" -mtime +7|xargs rm

backup 脚本 [root@backup backup]# cat /server/scripts/check.sh #!/bin/bash

#del 180 day file keep every 6 find /backup/ -type f -name ".tar.gz" -mtime +180 ! -name "_6.tar.gz"|xargs rm

#check cd /backup/ && md5sum -c check.log >/backup/result.log if [ $? -ne 0 ];then

mail -s "md5sum wrong" 918391635@qq.com </backup/result.log fi #需要开启postfix[root@backup backup]# /etc/init.d/postfix startStarting postfix: [ OK ][root@backup backup]# chkconfig postfix on mail -s "md5sum wrong" kang50055@163.com </etc/hosts [root@backup backup]# crontab -l |tail -2 #check & send result lee at 2017-01-01 00 03 * * * /bin/sh /server/scripts/check.sh >/dev/null 2>&1 图片集合中小型网站搭建---大酒店https://www.processon.com/view/link/5a5dfca1e4b05a8ff305c36f 老男孩教育-链接克隆与完整克隆区别https://www.processon.com/view/link/5a609837e4b0abe85d60af61 老男孩教育-rsync守护进程模式部署流程https://www.processon.com/view/link/5a60b7f8e4b010a6e7231d10 老男孩教育-全网备份项目逻辑图https://www.processon.com/view/link/5a61d763e4b05a8ff30e5df7