樽中酒不空
分类: 云计算
2015-08-03 11:19:47
首先要感谢liu大神的翻译与试验,
简介
本文详细描述使用单台服务器部署一个swift开发版本的过程,单台服务器运行所有swift服务,并模拟运行具有4个结点的swift集群。其中,这里的“服务器”可以是物理机也可以是虚拟机。
已验证的部署环境:
本文档基于:
请确保安装的swift版本和本文档中的版本相同。如有问题,请参考凯发k8官网下载客户端中心官网的更新文档。
1. 安装依赖包和核心代码
1. 以root身份登录。
# sudo su - root |
2. 安装必须的依赖包、库。
# apt-get install python-software-properties # add-apt-repository ppa:swift-core/release # apt-get update # apt-get install curl gcc git-core memcached python-coverage python-dev python-nose python-setuptools python-simplejson python-xattr sqlite3 xfsprogs python-eventlet python-greenlet python-pastedeploy python-netifaces python-pip # pip install mock |
3. 创建swift用户和组。在此,我们直接使用root作为用户名和组名。
接下来,需要选择使用一个分区作为存储(using a partition for storage)或使用一个回环设备作为存储(using a loopback device for storage),本文档使用回环设备作为存储。若希望使用一个单独的分区作为存储,请参阅。
1. 创建存储文件夹。
# mkdir /srv |
2. 调整seek的值,可以创建一个更大或更小的分区。
# dd if=/dev/zero of=/srv/swift-disk bs=1024 count=0 seek=1000000 # mkfs.xfs -i size=1024 /srv/swift-disk |
3. 编辑/etc/fstab并添加如下一行。
/srv/swift-disk /mnt/sdb1 xfs loop,noatime,nodiratime,nobarrier,logbufs=8 0 0 |
4. 创建挂载文件夹,并挂载。
# mkdir /mnt/sdb1 # mount /mnt/sdb1 |
5. 创建4个子节点文件夹。
# mkdir /mnt/sdb1/1 /mnt/sdb1/2 /mnt/sdb1/3 /mnt/sdb1/4 |
6. 改变权限,root:root对应
# chown root:root /mnt/sdb1/* |
7. 为4个子节点创建同步链接。
# for x in {1..4}; do ln -s /mnt/sdb1/$x /srv/$x; done |
8. 创建server等文件夹,并改编权限。注意,不要忘记/etc/swift /srv/[1-4]/最后一个“/”。
# mkdir -p /etc/swift/object-server /etc/swift/container-server /etc/swift/account-server /srv/1/node/sdb1 /srv/2/node/sdb2 /srv/3/node/sdb3 /srv/4/node/sdb4 /var/run/swift # chown -r root:root /etc/swift /srv/[1-4]/ /var/run/swift |
9. 编辑文件/etc/rc.local,在exit 0 之前添加如下4行。
# mkdir -p /var/cache/swift /var/cache/swift2 /var/cache/swift3 /var/cache/swift4 # chown root:root /var/cache/swift* # mkdir -p /var/run/swift # chown root:root /var/run/swift |
1. 创建文件/etc/rsyncd.conf,内容如下:
uid = root gid = root log file = /var/log/rsyncd.log pid file = /var/run/rsyncd.pid address = 127.0.0.1
[account6012] max connections = 25 path = /srv/1/node/ read only = false lock file = /var/lock/account6012.lock
[account6022] max connections = 25 path = /srv/2/node/ read only = false lock file = /var/lock/account6022.lock
[account6032] max connections = 25 path = /srv/3/node/ read only = false lock file = /var/lock/account6032.lock
[account6042] max connections = 25 path = /srv/4/node/ read only = false lock file = /var/lock/account6042.lock
[container6011] max connections = 25 path = /srv/1/node/ read only = false lock file = /var/lock/container6011.lock
[container6021] max connections = 25 path = /srv/2/node/ read only = false lock file = /var/lock/container6021.lock
[container6031] max connections = 25 path = /srv/3/node/ read only = false lock file = /var/lock/container6031.lock
[container6041] max connections = 25 path = /srv/4/node/ read only = false lock file = /var/lock/container6041.lock
[object6010] max connections = 25 path = /srv/1/node/ read only = false lock file = /var/lock/object6010.lock
[object6020] max connections = 25 path = /srv/2/node/ read only = false lock file = /var/lock/object6020.lock
[object6030] max connections = 25 path = /srv/3/node/ read only = false lock file = /var/lock/object6030.lock
[object6040] max connections = 25 path = /srv/4/node/ read only = false lock file = /var/lock/object6040.lock |
2. 编辑文件/etc/default/rsync,设置参数rsync_enable为true。
rsync_enable=true |
3. 启动rsync服务
# service rsync restart |
1. 创建文件/etc/rsyslog.d/10-swift.conf,内容如下:
# uncomment the following to have a log containing all logs together #local1,local2,local3,local4,local5.* /var/log/swift/all.log
# uncomment the following to have hourly proxy logs for stats processing #$template hourlyproxylog,"/var/log/swift/hourly/%$year%%$month%%$day%%$hour%" #local1.*;local1.!notice ?hourlyproxylog
local1.*;local1.!notice /var/log/swift/proxy.log local1.notice /var/log/swift/proxy.error local1.* ~
local2.*;local2.!notice /var/log/swift/storage1.log local2.notice /var/log/swift/storage1.error local2.* ~
local3.*;local3.!notice /var/log/swift/storage2.log local3.notice /var/log/swift/storage2.error local3.* ~
local4.*;local4.!notice /var/log/swift/storage3.log local4.notice /var/log/swift/storage3.error local4.* ~
local5.*;local5.!notice /var/log/swift/storage4.log local5.notice /var/log/swift/storage4.error local5.* ~ |
2. 编辑文件/etc/rsyslog.conf,更改参数$privdroptogroup为adm。
$privdroptogroup adm |
3. 创建swift每小时的文件夹。
# mkdir -p /var/log/swift/hourly |
4. 更改swift的log文件夹权限。
# chown -r syslog.adm /var/log/swift # chmod -r g w /var/log/swift |
5. 重启rsyslog服务。
# service rsyslog restart |
若不打算在全root权限下安装swift,则需要切换回guest权限。此处,我们全部使用root安装。
1. 创建文件夹。
# mkdir ~/bin |
2. 从git上获取swift代码,下载到本地。
# git clone |
3. 安装swift的开发版本。
# cd ~/swift # python setup.py develop # cd .. |
4. 从git上获取python-swiftclient代码,下载到本地。
# git clone |
5. 安装python-swiftclient的开发版本。
# cd ~/python-swiftclient # python setup.py develop # cd .. |
6. 编辑文件~/.bashrc,并在文件尾添加如下内容:
export swift_test_config_file=/etc/swift/test.conf export path=${path}:~/bin |
7. 执行。
# . ~/.bashrc |
1. 创建文件/etc/swift/proxy-server.conf,文件内容如下:
[default] bind_port = 8080 user = root log_facility = log_local1 eventlet_debug = true
[pipeline:main] pipeline = healthcheck cache tempauth proxy-logging proxy-server
[app:proxy-server] use = egg:swift#proxy allow_account_management = true account_autocreate = true
[filter:tempauth] use = egg:swift#tempauth user_admin_admin = admin .admin .reseller_admin user_test_tester = testing .admin user_test2_tester2 = testing2 .admin user_test_tester3 = testing3
[filter:healthcheck] use = egg:swift#healthcheck
[filter:cache] use = egg:swift#memcache
[filter:proxy-logging] use = egg:swift#proxy_logging |
2. 创建文件/etc/swift/swift.conf,文件内容如下:
[swift-hash] # random unique string that can never change (do not lose) swift_hash_path_suffix = jtangfs |
3. 创建文件/etc/swift/account-server/1.conf,文件内容如下:
[default] devices = /srv/1/node mount_check = false disable_fallocate = true bind_port = 6012 user = root log_facility = log_local2 recon_cache_path = /var/cache/swift eventlet_debug = true
[pipeline:main] pipeline = recon account-server
[app:account-server] use = egg:swift#account
[filter:recon] use = egg:swift#recon
[account-replicator] vm_test_mode = yes
[account-auditor]
[account-reaper] |
4. 创建文件/etc/swift/account-server/2.conf,文件内容如下:
[default] devices = /srv/2/node mount_check = false disable_fallocate = true bind_port = 6022 user = root log_facility = log_local3 recon_cache_path = /var/cache/swift2 eventlet_debug = true
[pipeline:main] pipeline = recon account-server
[app:account-server] use = egg:swift#account
[filter:recon] use = egg:swift#recon
[account-replicator] vm_test_mode = yes
[account-auditor]
[account-reaper] |
5. 创建文件/etc/swift/account-server/3.conf,文件内容如下:
[default] devices = /srv/3/node mount_check = false disable_fallocate = true bind_port = 6032 user = root log_facility = log_local4 recon_cache_path = /var/cache/swift3 eventlet_debug = true
[pipeline:main] pipeline = recon account-server
[app:account-server] use = egg:swift#account
[filter:recon] use = egg:swift#recon
[account-replicator] vm_test_mode = yes
[account-auditor]
[account-reaper] |
6. 创建文件/etc/swift/account-server/4.conf,文件内容如下:
[default] devices = /srv/4/node mount_check = false disable_fallocate = true bind_port = 6042 user = root log_facility = log_local5 recon_cache_path = /var/cache/swift4 eventlet_debug = true
[pipeline:main] pipeline = recon account-server
[app:account-server] use = egg:swift#account
[filter:recon] use = egg:swift#recon
[account-replicator] vm_test_mode = yes
[account-auditor]
[account-reaper] |
7. 创建文件/etc/swift/container-server/1.conf,文件内容如下:
[default] devices = /srv/1/node mount_check = false disable_fallocate = true bind_port = 6011 user = root log_facility = log_local2 recon_cache_path = /var/cache/swift eventlet_debug = true
[pipeline:main] pipeline = recon container-server
[app:container-server] use = egg:swift#container
[filter:recon] use = egg:swift#recon
[container-replicator] vm_test_mode = yes
[container-updater]
[container-auditor]
[container-sync] |
8. 创建文件/etc/swift/container-server/2.conf,文件内容如下:
[default] devices = /srv/2/node mount_check = false disable_fallocate = true bind_port = 6021 user = root log_facility = log_local3 recon_cache_path = /var/cache/swift2 eventlet_debug = true
[pipeline:main] pipeline = recon container-server
[app:container-server] use = egg:swift#container
[filter:recon] use = egg:swift#recon
[container-replicator] vm_test_mode = yes
[container-updater]
[container-auditor]
[container-sync] |
9. 创建文件/etc/swift/container-server/3.conf,文件内容如下:
[default] devices = /srv/3/node mount_check = false disable_fallocate = true bind_port = 6031 user = root log_facility = log_local4 recon_cache_path = /var/cache/swift3 eventlet_debug = true
[pipeline:main] pipeline = recon container-server
[app:container-server] use = egg:swift#container
[filter:recon] use = egg:swift#recon
[container-replicator] vm_test_mode = yes
[container-updater]
[container-auditor]
[container-sync] |
10. 创建文件/etc/swift/container-server/4.conf,文件内容如下:
[default] devices = /srv/4/node mount_check = false disable_fallocate = true bind_port = 6041 user = root log_facility = log_local5 recon_cache_path = /var/cache/swift4 eventlet_debug = true
[pipeline:main] pipeline = recon container-server
[app:container-server] use = egg:swift#container
[filter:recon] use = egg:swift#recon
[container-replicator] vm_test_mode = yes
[container-updater]
[container-auditor]
[container-sync] |
11. 创建文件/etc/swift/object-server/1.conf,文件内容如下:
[default] devices = /srv/1/node mount_check = false disable_fallocate = true bind_port = 6010 user = root log_facility = log_local2 recon_cache_path = /var/cache/swift eventlet_debug = true
[pipeline:main] pipeline = recon object-server
[app:object-server] use = egg:swift#object
[filter:recon] use = egg:swift#recon
[object-replicator] vm_test_mode = yes
[object-updater]
[object-auditor] |
12. 创建文件/etc/swift/object-server/2.conf,文件内容如下:
[default] devices = /srv/2/node mount_check = false disable_fallocate = true bind_port = 6020 user = root log_facility = log_local3 recon_cache_path = /var/cache/swift2 eventlet_debug = true
[pipeline:main] pipeline = recon object-server
[app:object-server] use = egg:swift#object
[filter:recon] use = egg:swift#recon
[object-replicator] vm_test_mode = yes
[object-updater]
[object-auditor] |
13. 创建文件/etc/swift/object-server/3.conf,文件内容如下:
[default] devices = /srv/3/node mount_check = false disable_fallocate = true bind_port = 6030 user = root log_facility = log_local4 recon_cache_path = /var/cache/swift3 eventlet_debug = true
[pipeline:main] pipeline = recon object-server
[app:object-server] use = egg:swift#object
[filter:recon] use = egg:swift#recon
[object-replicator] vm_test_mode = yes
[object-updater]
[object-auditor] |
14. 创建文件/etc/swift/object-server/4.conf,文件内容如下:
[default] devices = /srv/4/node mount_check = false disable_fallocate = true bind_port = 6040 user = root log_facility = log_local5 recon_cache_path = /var/cache/swift4 eventlet_debug = true
[pipeline:main] pipeline = recon object-server
[app:object-server] use = egg:swift#object
[filter:recon] use = egg:swift#recon
[object-replicator] vm_test_mode = yes
[object-updater]
[object-auditor] |
1. 创建脚本~/bin/resetswift,内容如下。注意,如果使用的是单独分区存储需要将/srv/swift-disk替换为/dev/sdb1;如果没有创建rsyslog作为独立日志,则需要移除find /var/log/swift... 这一行。
#!/bin/bash
swift-init all stop find /var/log/swift -type f -exec rm -f {} \; sudo umount /mnt/sdb1 sudo mkfs.xfs -f -i size=1024 /srv/swift-disk sudo mount /mnt/sdb1 sudo mkdir /mnt/sdb1/1 /mnt/sdb1/2 /mnt/sdb1/3 /mnt/sdb1/4 sudo chown root:root /mnt/sdb1/* mkdir -p /srv/1/node/sdb1 /srv/2/node/sdb2 /srv/3/node/sdb3 /srv/4/node/sdb4 sudo rm -f /var/log/debug /var/log/messages /var/log/rsyncd.log /var/log/syslog find /var/cache/swift* -type f -name *.recon -exec rm -f {} \; sudo service rsyslog restart sudo service memcached restart |
2. 创建脚本~/bin/remakerings,内容如下:
#!/bin/bash
cd /etc/swift
rm -f *.builder *.ring.gz backups/*.builder backups/*.ring.gz
swift-ring-builder object.builder create 18 3 1 swift-ring-builder object.builder add z1-127.0.0.1:6010/sdb1 1 swift-ring-builder object.builder add z2-127.0.0.1:6020/sdb2 1 swift-ring-builder object.builder add z3-127.0.0.1:6030/sdb3 1 swift-ring-builder object.builder add z4-127.0.0.1:6040/sdb4 1 swift-ring-builder object.builder rebalance swift-ring-builder container.builder create 18 3 1 swift-ring-builder container.builder add z1-127.0.0.1:6011/sdb1 1 swift-ring-builder container.builder add z2-127.0.0.1:6021/sdb2 1 swift-ring-builder container.builder add z3-127.0.0.1:6031/sdb3 1 swift-ring-builder container.builder add z4-127.0.0.1:6041/sdb4 1 swift-ring-builder container.builder rebalance swift-ring-builder account.builder create 18 3 1 swift-ring-builder account.builder add z1-127.0.0.1:6012/sdb1 1 swift-ring-builder account.builder add z2-127.0.0.1:6022/sdb2 1 swift-ring-builder account.builder add z3-127.0.0.1:6032/sdb3 1 swift-ring-builder account.builder add z4-127.0.0.1:6042/sdb4 1 swift-ring-builder account.builder rebalance |
3. 创建脚本~/bin/startmain,内容如下:
#!/bin/bash
swift-init main start |
4. 创建脚本~/bin/startrest,内容如下:
#!/bin/bash
swift-init rest start |
5. 更改脚本权限。
# chmod x ~/bin/* |
6. 创建rings。
# remakerings |
7. 执行功能单元测试,出现“unable to read test config /etc/swift/test.conf – file not found”,可不必理会,或手动复制过去(配置文件在swift/test/sample.conf)。
# cd ~/swift # ./.unittests |
8. 运行swift,出现“unable to increase file descriptor limit. running as non-root?”警告为正常现象,不必理会。
# startmain |
完成安装部署后,可以使用swift --help命令查看swift帮助信息。
# swift --help
usage: swift
commands: stat [container] [object] displays information for the account, container, or object depending on the args given (if any). list [options] [container] lists the containers for the account or the objects for a container. -p or --prefix is an option that will only list items beginning with that prefix. -d or --delimiter is option (for container listings only) that will roll up items with the given delimiter (see cloud files general documentation for what this means). upload [options] container file_or_directory [file_or_directory] [...] uploads to the given container the files and directories specified by the remaining args. -c or --changed is an option that will only upload files
that have changed since the last upload. -s and --leave-segments are options as well (see --help for more). post [options] [container] [object] updates meta information for the account, container, or object depending on the args given. if the container is not found, it will be created automatically; but this is not true for accounts and objects. containers also allow the -r (or --read-acl) and -w (or --write-acl) options. the -m or --meta option is allowed on all and used to define the user meta data items to set in the form name:value. this option can be repeated. example: post -m color:blue -m size:large download --all or download container [options] [object] [object] ... downloads everything in the account (with --all), or everything in a container, or a list of objects depending on the args given. for a single
object download, you may use the -o [--output] redirect the output to a specific file or if "-" then just redirect to stdout. delete [options] --all or delete container [options] [object] [object] ... deletes everything in the account (with --all), or everything in a container, or a list of objects depending on the args given. segments of manifest objects will be deleted as well, unless you specify the --leave-segments option.
example: swift -a -u user -k key stat
options: --version show program's version number and exit -h, --help show this help message and exit -s, --snet use servicenet internal network -v, --verbose print more info -q, --quiet suppress status output -a auth, --auth=auth url for obtaining an auth token -v auth_version, --auth-version=auth_version specify a version for authentication. defaults to 1.0. -u user, --user=user user name for obtaining an auth token. -k key, --key=key key for obtaining an auth token.
--os-username= openstack username. defaults to env[os_username].
--os-password= openstack password. defaults to env[os_password].
--os-tenant-id= openstack tenant id. defaults to env[os_tenant_id]
--os-tenant-name= openstack tenant name. defaults to env[os_tenant_name].
--os-auth-url= openstack auth url. defaults to env[os_auth_url].
--os-auth-token= openstack token. defaults to env[os_auth_token]
--os-storage-url= openstack storage url. defaults to env[os_storage_url]
--os-region-name= openstack region name. defaults to env[os_region_name]
--os-service-type= openstack service type. defaults to env[os_service_type]
--os-endpoint-type= openstack endpoint type. defaults to env[os_endpoint_type] --insecure allow swiftclient to access insecure keystone server. the keystone's certificate will not be verified. |
1. 获取一个 x-storage-url 和 x-auth-token。
# curl -v -h 'x-storage-user: test:tester' -h 'x-storage-pass: testing' |
可以得到以下返回信息:
* about to connect() to 127.0.0.1 port 8080 (#0) * trying 127.0.0.1... connected * connected to 127.0.0.1 (127.0.0.1) port 8080 (#0) > get /auth/v1.0 http/1.1 > user-agent: curl/7.19.7 (i486-pc-linux-gnu) libcurl/7.19.7 openssl/0.9.8k zlib/1.2.3.3 libidn/1.15 > host: 127.0.0.1:8080 > accept: */* > x-storage-user: test:tester > x-storage-pass: testing
< http/1.1 200 ok < x-storage-url: < x-storage-token: auth_tk6474e5ee0cb04832b9d2a168e1a164d8 < x-auth-token: auth_tk6474e5ee0cb04832b9d2a168e1a164d8 < content-length: 0 < date: mon, 04 jul 2011 01:36:57 gmt * connection #0 to host 127.0.0.1 left intact *closing connection #0 |
2. 检查账户。注意,x-auth-token:与
# curl -v -h 'x-auth-token: |
3. 检查swift工作。
# swift -a -u test:tester -k testing stat |
正确情况下,应该输出以下信息:
account: auth_test containers: 0 objects: 0 bytes: 0 accept-ranges: bytes |
4. 创建container,创建一个名称为container_test的container(目录),注意使用当前的token。
# curl -x put -d - -h "x-auth_token:auth_tk6474e5ee0cb04832b9d2a168e1a164d8" |
正确情况下,应该得到以下信息:
http/1.1 201 created content-length: 0 content-type: text/html; charset=utf-8 date: mon, 04 jul 2011 01:39:38 gmt |
5. 查看test用户的container列表,发现只有一个目录:container_test。
# swift -a -u test:tester -k testing list |
container_test |
6. 上传object(文件),上传hello文件到container_test目录中。
# swift -a -u test:tester -k testing upload container_test /root/hello |
root/hello |
7. 查看目录中的内容,查看container_test目录中的内容。
# swift -a -u test:tester -k testing list container_test |
root/hello |
8. 下载object(文件)下载container_test下的object。
# swift -a -u test:tester -k testing download container_test |
将container_test目录内的所有内容下载到目前所在目录。注意:如果上传文件的时候,有嵌套目录,则上传到swift上后,也是以嵌套目录的形式存在,下载后,会在当前目录创建一样的嵌套目录。
补充说明:
现在已经完成saio的部署了,下面介绍一下系统环境的改变:
1. ~/swift目录下存放swift与python-swiftclient的源代码与执行脚本。
2. /usr/local/lib/python2.7/dist-packages、/usr/share/pyshared、/usr/bin、/usr/local/bin等目录下存放swift与python-swiftclient安装后的程序文件。
3. swift的回环设备挂载在/mnt/sdb1目录上。
4. 回环设备对应的文件为/srv/swift-disk。
5. 四个存储节点对应的实际存储空间(即共享上述回环设备)——/mnt/sdb1/1、/mnt/sdb1/2、/mnt/sdb1/3、/mnt/sdb1/4。
6. 四个存储节点指向实际存储空间的软链接——/srv/1、/srv/2、/srv/3、/srv/4。
7. 四个存储节点的对应的虚拟挂载点——/srv/1/node/sdb1、/srv/2/node/sdb2、/srv/2/node/sdb2、/srv/2/node/sdb2。
8. swift运行时所需目录/var/run/swift,用于存放各个服务进程的pid文件等内容。
9. swift临时缓存的目录(姑且这么理解)——/var/cache/swift、/var/cache/swift2、/var/cache/swift3、/var/cache/swift4。
10. /etc/swift目录中存放swift自身的配置文件。
11. /var/log/swift目录下存放rsyslog独立日志工具输出的日志文件。
12. 其他被创建的文件:
13. 其他被修改的文件: