openstack swift 嵌入式 openstack swift部署

转载
mob64ca14068b0b 2024-05-21 14:04:36
文章标签 openstack swift 嵌入式 swift python bash 文章分类 OpenStack 云计算
参考官方文档，及一些国内同行的博客 
 
我的装系统是分了个lvm,/dev/sda7 
 

注意：2.1~2.5必须以root身份进行操作 
 
2.1、安装依赖包 
 

apt-get install python-software-properties 
 

add-apt-repository ppa:swift-core/ppa 
 

apt-get update 
 

apt-get install curl gcc bzr memcached python-configobj python-coverage python-dev python-nose python-setuptools python-simplejson python-xattr sqlite3 xfsprogs pythonwebob python-eventlet python-greenlet python-pastedeploy python-netifaces 
 


2.2、创建swift用户 
 

创建swift用户 
 

sudo useradd -mk /home/swift -s /bin/bash swift 
 

sudo password swift 
 

  
 

编辑/etc/sudoers，添加以下一行 
 

# User alias specification 
 

swift   ALL=(ALL) NOPASSWD:ALL 
 

Using a partition for storage 
 

If you are going to use a separate partition for Swift data, be sure to add another device when creating the VM, and follow these instructions. 
 

        mkfs.xfs -i size=1024 /dev/sda7 
 

        Edit /etc/fstab and add 
 

            /dev/sda7 /mnt/sda7 xfs noatime,nodiratime,nobarrier,logbufs=8 0 0 
 

        mkdir /mnt/sda7 
 

        mount /mnt/sda7 
 

        mkdir /mnt/sda7/1 /mnt/sda7/2 /mnt/sda7/3 /mnt/sda7/4 
 

        chown swift:swift /mnt/sda7/* 
 

        mkdir /srv 
 

        for x in {1..4}; do ln -s /mnt/sda7/$x /srv/$x; done 
 

        mkdir -p /etc/swift/object-server /etc/swift/container-server /etc/swift/account-server /srv/1/node/sdb1 /srv/2/node/sdb2 /srv/3/node/sdb3 /srv/4/node/sdb4 /var/run/swift 
 

        chown -R swift:swift /etc/swift /srv/[1-4]/ /var/run/swift 
 
    – Make sure to include the trailing slash after /srv/[1-4]/ 
 

        Add to /etc/rc.local (before the exit 0): 
 

        mkdir -p /var/cache/swift /var/cache/swift2 /var/cache/swift3 /var/cache/swift4 
 
        chown swift:swift /var/cache/swift* 
 
        mkdir -p /var/run/swift 
 
        chown swift:swift /var/run/swift 
 


设置rsync 
 

创建文件：/etc/rsyncd.conf: 
 

uid = swift 
 

gid = swift 
 

log file = /var/log/rsyncd.log 
 

pid file = /var/run/rsyncd.pid 
 

address = 127.0.0.1 
 

[account6012] 
 

max connections = 25 
 

path = /srv/1/node/ 
 

read only = false 
 

lock file = /var/lock/account6012.lock 
 

[account6022] 
 

max connections = 25 
 

path = /srv/2/node/ 
 

read only = false 
 

lock file = /var/lock/account6022.lock 
 

[account6032] 
 

max connections = 25 
 

path = /srv/3/node/ 
 

read only = false 
 

lock file = /var/lock/account6032.lock 
 

[account6042] 
 

max connections = 25 
 

path = /srv/4/node/ 
 

read only = false 
 

lock file = /var/lock/account6042.lock 
 

[container6011] 
 

max connections = 25 
 

path = /srv/1/node/ 
 

read only = false 
 

lock file = /var/lock/container6011.lock 
 

[container6021] 
 

max connections = 25 
 

path = /srv/2/node/ 
 

read only = false 
 

lock file = /var/lock/container6021.lock 
 

[container6031] 
 

max connections = 25 
 

path = /srv/3/node/ 
 

read only = false 
 

lock file = /var/lock/container6031.lock 
 

[container6041] 
 

max connections = 25 
 

path = /srv/4/node/ 
 

read only = false 
 

lock file = /var/lock/container6041.lock 
 

[object6010] 
 

max connections = 25 
 

path = /srv/1/node/ 
 

read only = false 
 

lock file = /var/lock/object6010.lock 
 

[object6020] 
 

max connections = 25 
 

path = /srv/2/node/ 
 

read only = false 
 

lock file = /var/lock/object6020.lock 
 

[object6030] 
 

max connections = 25 
 

path = /srv/3/node/ 
 

read only = false 
 

lock file = /var/lock/object6030.lock 
 

[object6040] 
 

max connections = 25 
 

path = /srv/4/node/ 
 

read only = false 
 

lock file = /var/lock/object6040.lock 
 

编辑/etc/default/rsync: 
 

RSYNC_ENABLE设置为true 
 

重启服务： 
 

service rsync restart 
 

2.5、日志设置（可选） 
 

编辑/etc/rsyslog.d/10-swift.conf: 
 

# Uncomment the following to have a log containing all logs together 
 

#local1,local2,local3,local4,local5.* /var/log/swift/all.log 
 

# Uncomment the following to have hourly proxy logs for stats processing 
 

#$template HourlyProxyLog,"/var/log/swift/hourly/%$YEAR%%$MONTH%%$DAY%% 
 

$HOUR%" 
 

#local1.*;local1.!notice ?HourlyProxyLog 
 

local1.*;local1.!notice /var/log/swift/proxy.log 
 

local1.notice /var/log/swift/proxy.error 
 

local1.* ~ 
 

local2.*;local2.!notice /var/log/swift/storage1.log 
 

local2.notice /var/log/swift/storage1.error 
 

local2.* ~ 
 

local3.*;local3.!notice /var/log/swift/storage2.log 
 

local3.notice /var/log/swift/storage2.error 
 

local3.* ~ 
 

local4.*;local4.!notice /var/log/swift/storage3.log 
 

local4.notice /var/log/swift/storage3.error 
 

local4.* ~ 
 

local5.*;local5.!notice /var/log/swift/storage4.log 
 

local5.notice /var/log/swift/storage4.error 
 

local5.* ~ 
 

  
 

修改/etc/rsyslog.conf： 
 

$PrivDropToGroup adm 
 

  
 

mkdir -p /var/log/swift/hourly 
 

chown -R syslog.adm /var/log/swift 
 

service rsyslog restart 
 

  
 

注意：2.6~2.9必须以swift身份进行操作 
 

2.6、获取代码和设置测试环境 
 

su - swift 
 

mkdir ~/bin 
 

        Check out the swift repo with 
 
git clone https:///openstack/swift.git 
 

        Build a development installation of swift, for example: 
 
 cd ~/swift; sudo python setup.py develop 
 

        Check out the python-swiftclient repo with 
 
 cd ~ 
 
 git clone https:///openstack/python-swiftclient.git 
 

        Build a development installation of python-swiftclient, for example: 
 
cd ~/python-swiftclient; sudo python setup.py develop 
 

        Edit ~/.bashrc and add to the end: 
 

        export SWIFT_TEST_CONFIG_FILE=/etc/swift/test.conf 
 
        export PATH=${PATH}:~/bin 
 

        . ~/.bashrc 
 

编辑~/.bashrc，在最后加入： 
 

export SWIFT_TEST_CONFIG_FILE=/etc/swift/func_test.conf 
 

export PATH=${PATH}:~/bin 
 

  
 
chmod a+x .bashrc 
 
. ~/.bashrc 
 

2.7、配置各个节点(以下这些文件不存在的话，就需要自己创建) 
 

1）/etc/swift/proxy-server.conf 
 

[DEFAULT] 
 

bind_port = 8080 
 

user = swift 
 

log_facility = LOG_LOCAL1 
 

[pipeline:main] 
 

pipeline = healthcheck cache swauth proxy-server 
 

[app:proxy-server] 
 

use = egg:swift#proxy 
 

allow_account_management = true 
 

[filter:swauth] 
 

use = egg:swift#swauth 
 

# Highly recommended to change this. 
 

super_admin_key = swauthkey 
 

[filter:healthcheck] 
 

use = egg:swift#healthcheck 
 

[filter:cache] 
 

use = egg:swift#memcache 
 

  
 

2）/etc/swift/swift.conf 
 

[swift-hash] 
 

# random unique string that can never change (DO NOT LOSE) 
 

swift_hash_path_suffix = 91d23be389715dab   //可以是任意的单字符串 
 

  
 

3）/etc/swift/account-server/1.conf 
 

[DEFAULT] 
 

devices = /srv/1/node 
 

mount_check = false 
 

bind_port = 6012 
 

user = swift 
 

log_facility = LOG_LOCAL2 
 

[pipeline:main] 
 

pipeline = account-server 
 

[app:account-server] 
 

use = egg:swift#account 
 

[account-replicator] 
 

vm_test_mode = yes 
 

[account-auditor] 
 

[account-reaper] 
 

  
 

4）/etc/swift/account-server/2.conf 
 

[DEFAULT] 
 

devices = /srv/2/node 
 

mount_check = false 
 

bind_port = 6022 
 

user = swift 
 

log_facility = LOG_LOCAL3 
 

[pipeline:main] 
 

pipeline = account-server 
 

[app:account-server] 
 

use = egg:swift#account 
 

[account-replicator] 
 

vm_test_mode = yes 
 

[account-auditor] 
 

[account-reaper] 
 

  
 

5）/etc/swift/account-server/3.conf 
 

[DEFAULT] 
 

devices = /srv/3/node 
 

mount_check = false 
 

bind_port = 6032 
 

user = swift 
 

log_facility = LOG_LOCAL4 
 

[pipeline:main] 
 

pipeline = account-server 
 

[app:account-server] 
 

use = egg:swift#account 
 

[account-replicator] 
 

vm_test_mode = yes 
 

[account-auditor] 
 

[account-reaper] 
 

6）/etc/swift/account-server/4.conf 
 

[DEFAULT] 
 

devices = /srv/4/node 
 

mount_check = false 
 

bind_port = 6042 
 

user = swift 
 

log_facility = LOG_LOCAL5 
 

[pipeline:main] 
 

pipeline = account-server 
 

[app:account-server] 
 

use = egg:swift#account 
 

[account-replicator] 
 

vm_test_mode = yes 
 

[account-auditor] 
 

[account-reaper] 
 

 7）/etc/swift/container-server/1.conf 
 

[DEFAULT] 
 

devices = /srv/1/node 
 

mount_check = false 
 

bind_port = 6011 
 

user = swift 
 

log_facility = LOG_LOCAL2 
 

[pipeline:main] 
 

pipeline = container-server 
 

[app:container-server] 
 

use = egg:swift#container 
 

[container-replicator] 
 

vm_test_mode = yes 
 

[container-updater] 
 

[container-auditor] 
 

[container-sync] 
 

8）/etc/swift/container-server/2.conf 
 

[DEFAULT] 
 

devices = /srv/2/node 
 

mount_check = false 
 

bind_port = 6021 
 

user = swift 
 

log_facility = LOG_LOCAL3 
 

[pipeline:main] 
 

pipeline = container-server 
 

[app:container-server] 
 

use = egg:swift#container 
 

[container-replicator] 
 

vm_test_mode = yes 
 

[container-updater] 
 

[container-auditor] 
 

[container-sync] 
 

 9）/etc/swift/container-server/3.conf 
 

[DEFAULT] 
 

devices = /srv/3/node 
 

mount_check = false 
 

bind_port = 6031 
 

user = swift 
 

log_facility = LOG_LOCAL4 
 

[pipeline:main] 
 

pipeline = container-server 
 

[app:container-server] 
 

use = egg:swift#container 
 

[container-replicator] 
 

vm_test_mode = yes 
 

[container-updater] 
 

[container-auditor] 
 

[container-sync] 
 

10）/etc/swift/container-server/4.conf 
 

[DEFAULT] 
 

devices = /srv/4/node 
 

mount_check = false 
 

bind_port = 6041 
 

user = swift 
 

log_facility = LOG_LOCAL5 
 

[pipeline:main] 
 

pipeline = container-server 
 

[app:container-server] 
 

use = egg:swift#container 
 

[container-replicator] 
 

vm_test_mode = yes 
 

[container-updater] 
 

[container-auditor] 
 

[container-sync] 
 

11）/etc/swift/object-server/1.conf 
 

[DEFAULT] 
 

devices = /srv/1/node 
 

mount_check = false 
 

bind_port = 6010 
 

user = swift 
 

log_facility = LOG_LOCAL2 
 

[pipeline:main] 
 

pipeline = object-server 
 

[app:object-server] 
 

use = egg:swift#object 
 

[object-replicator] 
 

vm_test_mode = yes 
 

[object-updater] 
 

[object-auditor] 
 

12）/etc/swift/object-server/2.conf 
 

[DEFAULT] 
 

devices = /srv/2/node 
 

mount_check = false 
 

bind_port = 6020 
 

user = swift 
 

log_facility = LOG_LOCAL3 
 

[pipeline:main] 
 

pipeline = object-server 
 

[app:object-server] 
 

use = egg:swift#object 
 

[object-replicator] 
 

vm_test_mode = yes 
 

[object-updater] 
 

[object-auditor] 
 

 13）/etc/swift/object-server/3.conf 
 

[DEFAULT] 
 

devices = /srv/3/node 
 

mount_check = false 
 

bind_port = 6030 
 

user = swift 
 

log_facility = LOG_LOCAL4 
 

[pipeline:main] 
 

pipeline = object-server 
 

[app:object-server] 
 

use = egg:swift#object 
 

[object-replicator] 
 

vm_test_mode = yes 
 

[object-updater] 
 

[object-auditor] 
 

  
 

14）/etc/swift/object-server/4.conf 
 

[DEFAULT] 
 

devices = /srv/4/node 
 

mount_check = false 
 

bind_port = 6040 
 

user = swift 
 

log_facility = LOG_LOCAL5 
 

[pipeline:main] 
 

pipeline = object-server 
 

[app:object-server] 
 

use = egg:swift#object 
 

[object-replicator] 
 

vm_test_mode = yes 
 

[object-updater] 
 

[object-auditor] 
 

2.8、创建运行swift运行的脚本 
 

1) ~/bin/resetswift 
 

#!/bin/bash 
 

swift-init all stop 
 

sudo umount /mnt/sdb1 
 

sudo mkfs.xfs -f -i size=1024 /srv/swift-disk 
 

sudo mount /mnt/sdb1 
 

sudo mkdir /mnt/sdb1/1 /mnt/sdb1/2 /mnt/sdb1/3 /mnt/sdb1/4 
 

sudo chown swift:swift /mnt/sdb1/* 
 

mkdir -p /srv/1/node/sdb1 /srv/2/node/sdb2 /srv/3/node/sdb3 /srv/4/node/sdb4 
 

sudo rm -f /var/log/debug /var/log/messages /var/log/rsyncd.log /var/log/syslog 
 

sudo service rsyslog restart 
 

sudo service memcached restart 
 

  
 

2) ~/bin/remakerings 
 

#!/bin/bash 
 

cd /etc/swift 
 

rm -f *.builder *.ring.gz backups/*.builder backups/*.ring.gz 
 

swift-ring-builder object.builder create 18 3 1 
 

swift-ring-builder object.builder add z1-127.0.0.1:6010/sdb1 1 
 

swift-ring-builder object.builder add z2-127.0.0.1:6020/sdb2 1 
 

swift-ring-builder object.builder add z3-127.0.0.1:6030/sdb3 1 
 

swift-ring-builder object.builder add z4-127.0.0.1:6040/sdb4 1 
 

swift-ring-builder object.builder rebalance 
 

swift-ring-builder container.builder create 18 3 1 
 

swift-ring-builder container.builder add z1-127.0.0.1:6011/sdb1 1 
 

swift-ring-builder container.builder add z2-127.0.0.1:6021/sdb2 1 
 

swift-ring-builder container.builder add z3-127.0.0.1:6031/sdb3 1 
 

swift-ring-builder container.builder add z4-127.0.0.1:6041/sdb4 1 
 

swift-ring-builder container.builder rebalance 
 

swift-ring-builder account.builder create 18 3 1 
 

swift-ring-builder account.builder add z1-127.0.0.1:6012/sdb1 1 
 

swift-ring-builder account.builder add z2-127.0.0.1:6022/sdb2 1 
 

swift-ring-builder account.builder add z3-127.0.0.1:6032/sdb3 1 
 

swift-ring-builder account.builder add z4-127.0.0.1:6042/sdb4 1 
 

swift-ring-builder account.builder rebalance 
 

 3) ~/bin/startmain 
 

#!/bin/bash 
 

swift-init main start 
 

  
 

4) ~/bin/startrest 
 
        #!/bin/bash 
 

swift-init rest start 
 

2.9、脚本测试 
 

1）添加~/bin/*的执行权限 
 

# chmod +x ~/bin/* 
 

2）重新产生rings 
 

# remakerings 
 

输出以下信息： 
 

Device z1-127.0.0.1:6010/sdb1_"" with 1.0 weight got id 0 
 

Device z2-127.0.0.1:6020/sdb2_"" with 1.0 weight got id 1 
 

Device z3-127.0.0.1:6030/sdb3_"" with 1.0 weight got id 2 
 

Device z4-127.0.0.1:6040/sdb4_"" with 1.0 weight got id 3 
 

Reassigned 262144 (100.00%) partitions. Balance is now 0.00. 
 

…… 
 

3）单元测试 
 

# cd ~/swift; ./.unittests 
 

输出以下信息说明单元测试通过： 
 

…… 
 

TOTAL                                   7622   5983    78%   
 

---------------------------------------------------------------------- 
 

Ran 632 tests in 40.013s 
 

 OK (SKIP=8) 
 

  
 

4）启动startmain 
 
cd ~/bin 
 
#startmain 
 

##################################################################### 
 
       运行startmain可能会报错，此网页可以解决 ： https://answers.launchpad.net/swift/+question/162957 
 
##################################################################### 
 

 5）获取一个 X-Storage-Url 和 X-Auth-Token: 
 

# curl -v -H 'X-Storage-User: test:tester' -H 'X-Storage-Pass: testing' http://127.0.0.1:8080/auth/v1.0 
 

 6）检查账户： 
 

# curl -v -H 'X-Auth-Token: <token-from-x-auth-token-above>'  <url-from-x-storage-url-above> 
 

 7）检查swift工作: 
 

# swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing stat 
 

正确情况下，应该输出以下信息： 
 

 Account: AUTH_test 
 

 Containers: 0 
 

 Objects: 0 
 

 Bytes: 0 
 

 Accept-Ranges: bytes 
 

 8）# cp ~/swift/test/functional/sample.conf /etc/swift/func_test.conf 
 

 9）功能测试 
 

# cd ~/swift; ./.functests 
 

正确情况下，输出类似于3）单元测试的结果。 
 

 10）probe测试 
 

# cd ~/swift; ./.probetests 
 

该测试脚本会在每次测试前，调用resetswift脚本 
 


三、上传/下载文件测试 
 


3.1、获得test用户的Storage_Auth_Token和URL 
 
       # curl -v -H 'X-Storage-User: test:tester' -H 'X-Storage-Pass: testing' http://127.0.0.1:8080/auth/v1.0 
 

得到以下回复信息： 
 

* About to connect() to 127.0.0.1 port 8080 (#0) 
 

*   Trying 127.0.0.1... connected 
 

* Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0) 
 

> GET /auth/v1.0 HTTP/1.1 
 

> User-Agent: curl/7.19.7 (i486-pc-linux-gnu) libcurl/7.19.7 OpenSSL/0.9.8k zlib/1.2.3.3 libidn/1.15 
 

> Host: 127.0.0.1:8080 
 

> Accept: */* 
 

> X-Storage-User: test:tester 
 

> X-Storage-Pass: testing 
 

< HTTP/1.1 200 OK 
 

< X-Storage-Url: http://127.0.0.1:8080/v1/AUTH_test 
 

< X-Storage-Token: AUTH_tk6474e5ee0cb04832b9d2a168e1a164d8 
 

< X-Auth-Token: AUTH_tk6474e5ee0cb04832b9d2a168e1a164d8 
 

< Content-Length: 0 
 

< Date: Mon, 04 Jul 2011 01:36:57 GMT 
 

* Connection #0 to host 127.0.0.1 left intact 
 

Closing connection #0 
 

3.2、创建container 
 

创建一个名称为container_test的container（目录） 
 

 # curl -X PUT -D -  -H "X-Auth_Token:AUTH_tk6474e5ee0cb04832b9d2a168e1a164d8" http://127.0.0.1:8080/v1/AUTH_test/container_test 
 

//正确情况下，应该得到以下信息： 
 

HTTP/1.1 201 Created 
 

Content-Length: 18 
 

Content-Type: text/html; charset=UTF-8 
 

Date: Mon, 04 Jul 2011 01:39:38 GMT 
 

  
 

查看test用户的container列表，发现只有一个目录：container_test： 
 

# swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing list 
 

container_test 
 

3.3、上传Object（文件） 
 

# swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing upload container_test ./cvs/CVSROOT/config            //上传config文件到ljl1目录中 
 

cvs/CVSROOT/config 
 

# swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing list container_test   //查看container_tese目录内的信息 
 

bin/startmain     
 

cvs/CVSROOT/config 
 

3.4、下载Object（文件） 
 

swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing download  container_test 
 

//将ljl1目录内的所有内容下载到目前所在目录 
 

  
 
        注意：如果上传文件的时候，有嵌套目录，则上传到swift上后，也是以嵌套目录的形式存在，下载后，会在当前目录创建一样的嵌套目录。 
 

四、遇到的问题 
 

1、上传文件失败，查看log文件， 
 

 File "/usr/lib/pymodules/python2.6/xattr/__init__.py", line 81, in set#012    self._set(name, value, 0, options | self.options)#012  File "/usr/lib/pymodules/python2.6/xattr/__init__.py", line 16, in _func#012    return func(first, *args)#012IOError: [Errno 95] Operation not supported 
 

出错原因：查看/srv/下面的文件，正确的显示信息应该如下所示：可是我的四个目录显示是没有链接信息的，删除这四个目录，1，2，3，4，重新进行软连接，并修改目录的属主即可 
 

swift@alen:/srv$ ls -l 
 

lrwxrwxrwx 1 swift swift         11 2011-07-04 15:13 1 -> /mnt/sdb1/1 
 

lrwxrwxrwx 1 swift swift         11 2011-07-04 15:13 2 -> /mnt/sdb1/2 
 

lrwxrwxrwx 1 swift swift         11 2011-07-04 15:13 3 -> /mnt/sdb1/3 
 

lrwxrwxrwx 1 swift swift         11 2011-07-04 15:13 4 -> /mnt/sdb1/4 
 

-rw-r--r-- 1 swift swift 1024000000 2011-07-04 15:50 swift-disk
本文章为转载内容，我们尊重原作者对文章享有的著作权。如有内容错误或侵权问题，欢迎原作者联系我们进行内容更正或删除文章。