搭建composer镜像搭建镜像站

转载

架构领航博主 2024-03-07 14:42:00

文章标签 搭建composer镜像 centos linux CentOS elastic 文章分类 云原生云计算

1 前言

我的服务器上安装了ESXI，我在上面建虚拟机进行测试，这些虚拟机的生命周期很短，在重装系统的时候，也需要反复的下载和安装软件。

我主要使用CentOS 7这款Linux系统，之前主要是使用国内的源镜像站来进行软件包的安装，虽然比直接去国外的镜像站下载已经快了很多，但每次安装虚拟机都会占用我的互联网带宽，导致网络变慢。所以我决定搭建自己的开源镜像站。

之前主要用的开源镜像站如下：

2 准备

先要准备一台最小化安装的CentOS 7, 并安装nginx和rsync，并且这台机器要能连接互联网。我这里使用一台安装了CentOS的虚拟机。

yum -y install nginx
yum -y install rsync

新建目录用于存放镜像文件

mkdir -p /data/mirror

配置nginx, 修改/etc/nginx/nginx.conf配置文件的server段

server {
        listen 80;
        server_name mirrors.faye.cool;
        root /data/mirror;

        location / {
                root   /data/mirror;
                autoindex on;
                autoindex_exact_size off;
                autoindex_localtime on;
                index  index.html index.htm;
        }
}

3 rsync同步

我主要同步的源及同步网站:

centos7 - 中国科技大学开源镜像站
epel - 中国科技大学开源镜像站
mariadb - 中国科技大学开源镜像站
zabbix - 清华大学开源镜像站
elasticsearch - 清华大学开源镜像站

编写同步脚本
rsync_centos7.sh

#!/bin/bash
#镜像同步脚本

#centos7-base
mkdir -pv /data/mirror/centos/7/os/x86_64/
rsync -avzP --bwlimit=30720 --timeout=120 --contimeout=120 --exclude-from=/opt/rsync/exclude.list rsync://rsync.mirrors.ustc.edu.cn/repo/centos/7/os/x86_64/ /data/mirror/centos/7/os/x86_64/
 
#centos7-updates
mkdir -pv /data/mirror/centos/7/updates/x86_64/
rsync -avzP --bwlimit=30720 --timeout=120 --contimeout=120 --exclude-from=/opt/rsync/exclude.list rsync://rsync.mirrors.ustc.edu.cn/repo/centos/7/updates/x86_64/ /data/mirror/centos/7/updates/x86_64/
 
#centos7-extras
mkdir -pv /data/mirror/centos/7/extras/x86_64/
rsync -avzP --bwlimit=30720 --timeout=120 --contimeout=120 --exclude-from=/opt/rsync/exclude.list rsync://rsync.mirrors.ustc.edu.cn/repo/centos/7/extras/x86_64/ /data/mirror/centos/7/extras/x86_64/
 
#centos7-centosplus
mkdir -pv /data/mirror/centos/7/centosplus/x86_64/
rsync -avzP --bwlimit=30720 --timeout=120 --contimeout=120 --exclude-from=/opt/rsync/exclude.list rsync://rsync.mirrors.ustc.edu.cn/repo/centos/7/centosplus/x86_64/ /data/mirror/centos/7/centosplus/x86_64/

rsync_epel.sh

#!/bin/bash
#epel
mkdir -pv /data/mirror/epel/7/x86_64/
rsync -avzP --bwlimit=30720 --timeout=120 --contimeout=120 --exclude-from=/opt/rsync/exclude.list rsync://rsync.mirrors.ustc.edu.cn/repo/epel/7/x86_64/ /data/mirror/epel/7/x86_64/
#rm -rf /data/mirror/epel/7/x86_64/.olddata
#createrepo /data/mirror/epel/7/x86_64/

rsync_zabbix.sh

#!/bin/bash
#epel
mkdir -pv /data/mirror/zabbix/zabbix/4.0/rhel/7/x86_64/
mkdir -pv /data/mirror/zabbix/non-supported/rhel/7/x86_64/
rsync -avzP --bwlimit=30720 --timeout=120 --contimeout=120 --exclude-from=/opt/rsync/exclude.list rsync://mirrors.tuna.tsinghua.edu.cn/zabbix/zabbix/4.0/rhel/7/x86_64/ /data/mirror/zabbix/zabbix/4.0/rhel/7/x86_64/
rsync -avzP --bwlimit=30720 --timeout=120 --contimeout=120 --exclude-from=/opt/rsync/exclude.list rsync://mirrors.tuna.tsinghua.edu.cn/zabbix/non-supported/rhel/7/x86_64/ /data/mirror/zabbix/non-supported/rhel/7/x86_64/
rsync -avzP --bwlimit=30720 --timeout=120 --contimeout=120 --exclude-from=/opt/rsync/exclude.list rsync://mirrors.tuna.tsinghua.edu.cn/zabbix/RPM-GPG-KEY-ZABBIX* /data/mirror/zabbix/
#rm -rf /data/mirror/epel/7/x86_64/.olddata
#createrepo /data/mirror/epel/7/x86_64/

rsync_mariadb.sh

#!/bin/bash
#epel
mkdir -pv /data/mirror/mariadb/10.2/centos7-amd64/
rsync -avzP --bwlimit=30720 --timeout=120 --contimeout=120 --exclude-from=/opt/rsync/exclude.list rsync://rsync.mirrors.ustc.edu.cn/repo/mariadb/yum/10.2/centos7-amd64/ /data/mirror/mariadb/10.2/centos7-amd64/
rsync -avzP --bwlimit=30720 --timeout=120 --contimeout=120 --exclude-from=/opt/rsync/exclude.list rsync://rsync.mirrors.ustc.edu.cn/repo/mariadb/yum/RPM-GPG-KEY-MariaDB /data/mirror/mariadb/RPM-GPG-KEY-MariaDB
#rm -rf /data/mirror/epel/7/x86_64/.olddata
#createrepo /data/mirror/epel/7/x86_64/

rsync_elastic.sh

#!/bin/bash
#epel
mkdir -pv /data/mirror/elastic/6.x/yum/
curl https://packages.elastic.co/GPG-KEY-elasticsearch > /data/mirror/elastic/GPG-KEY-elasticsearch
rsync -avzP --bwlimit=30720 --timeout=120 --contimeout=120 --exclude-from=/opt/rsync/exclude.list rsync://mirrors.tuna.tsinghua.edu.cn/elasticstack/6.x/yum/ /data/mirror/elastic/6.x/yum/

rsync.sh 调用以上的脚本进行镜像同步

#!/bin/bash
#
/data/mirror/rsync_centos_7.sh
/data/mirror/rsync_epel.sh
/data/mirror/rsync_zabbix.sh
/data/mirror/rsync_mariadb.sh
/data/mirror/rsync_elastic.sh
#/mnt/nas/rsync_centos_6.sh

#当前时间
time=`date`
echo $time"--> rsync success" >> /tmp/rsync.log

手动调用方法：(首次调用运行时间会比较长)

./rsync.sh

自动定时调用方法：

crontab -e
# 加入如下内容
15 2 * * * /data/mirror/rsync.sh > /dev/null

在以上这些脚本中，我使用–bwlimit=30720参数限制rsync最大同步带宽为300Mbps左右，我家的总带宽为500Mbps，还有一定的带宽给其它使用。而且我安排每天凌晨02:12分自动运行脚本，安排在没有人使用网络的时候。
另外我注释掉了createrepo命令，因为重新createrepo会导致centos的yum groupinstall 命令不能使用。

4 客户机使用镜像源

我写好了各镜像的.repo文件，并且也放到我内部镜像服务器的Nginx Web目录下，即/data/mirror目录下。
repo文件中 mirrors.faye.cool 是我内部DNS解析到源镜像服务器的域名

centos7.repo

# CentOS-Base.repo
#
# The mirror system uses the connecting IP address of the client and the
# update status of each mirror to pick mirrors that are updated to and
# geographically close to the client.  You should use this for CentOS updates
# unless you are manually picking other mirrors.
#
# If the mirrorlist= does not work for you, as a fall back you can try the
# remarked out baseurl= line instead.
#
#

[base]
name=CentOS-$releasever - Base
baseurl=http://mirrors.faye.cool/centos/$releasever/os/$basearch/
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=os
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7

#released updates
[updates]
name=CentOS-$releasever - Updates
baseurl=http://mirrors.faye.cool/centos/$releasever/updates/$basearch/
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=updates
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7

#additional packages that may be useful
[extras]
name=CentOS-$releasever - Extras
baseurl=http://mirrors.faye.cool/centos/$releasever/extras/$basearch/
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=extras
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7

#additional packages that extend functionality of existing packages
[centosplus]
name=CentOS-$releasever - Plus
baseurl=http://mirrors.faye.cool/centos/$releasever/centosplus/$basearch/
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=centosplus
gpgcheck=1
enabled=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7

epel.repo

[epel]
name=Extra Packages for Enterprise Linux 7 - $basearch
baseurl=http://mirrors.faye.cool/epel/7/$basearch
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7

zabbix.repo

[zabbix]
name=Zabbix Official Repository - $basearch
baseurl=http://mirrors.faye.cool/zabbix/zabbix/4.0/rhel/7/$basearch/
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-ZABBIX-A14FE591

[zabbix-non-supported]
name=Zabbix Official Repository non-supported - $basearch 
baseurl=http://mirrors.faye.cool/zabbix/non-supported/rhel/7/$basearch/
enabled=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-ZABBIX
gpgcheck=1

mariadb.repo

# MariaDB 10.3 CentOS repository list - created 2018-12-13 11:32 UTC
# http://downloads.mariadb.org/mariadb/repositories/
[mariadb]
name = MariaDB
baseurl = http://mirrors.faye.cool/mariadb/10.2/centos7-amd64
gpgkey=http://mirrors.faye.cool/mariadb/RPM-GPG-KEY-MariaDB
gpgcheck=1

elastic.repo

[elastic-6.x]
name=Elastic repository for 6.x packages
baseurl=http://mirrors.faye.cool/elastic/6.x/yum/
gpgcheck=1
gpgkey=http://mirrors.faye.cool/elastic/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

这样使用如下的命令，我就可以完成客户机系统的镜像源替换

cd /etc/yum.repos.d/ && mkdir bak && mv -f *.repo bak/
  curl http://mirrors.faye.cool/CentOS-Base.repo > /etc/yum.repos.d/CentOS-Base.repo
  yum -y install epel-release
  mv -f /etc/yum.repos.d/epel* /etc/yum.repos.d/bak
  curl http://mirrors.faye.cool/epel.repo > /etc/yum.repos.d/epel.repo
  curl http://mirrors.faye.cool/mariadb.repo > /etc/yum.repos.d/mariadb.repo
  curl http://mirrors.faye.cool/elastic.repo > /etc/yum.repos.d/elastic.repo
  rpm -ivh http://mirrors.faye.cool/zabbix/zabbix/4.0/rhel/7/x86_64/zabbix-release-4.0-1.el7.noarch.rpm
  mv -f /etc/yum.repos.d/zabbix.repo /etc/yum.repos.d/bak
  curl http://mirrors.faye.cool/zabbix.repo > /etc/yum.repos.d/zabbix.repo
  yum clean all && yum makecache