如何部署使用Intel开源的对象存储测试套件CosBench,很多新手对此不是很清楚,为了帮助大家解决这个难题,下面小编将为大家详细讲解,有这方面需求的人可以来学习下,希望你能有所收获。
1.引言
我最近一直使用Ceph的对象存储,测试方法也是五花八门,有自己用Python写过测试脚本,也有用C++写过。觉得既然S3是一个标准的、通用的对象存储服务,应该有一个测试套件来进行测试。后来在社区中看到大家采用Intel的开源软件Cosbench进行测试,于是我也进行了部署及使用。
2.部署过程
2.1 机器准备 我使用了七台机器,host1~host7 角色: host1:兼任controller driver host2~6 drvier
2.2 安装包部署 将Cosbench安装包分别解压到各自的记录,安装包可以在如下地址下载: Cosbench说明
2.3 基本信息配置
[controller]
drivers = 7
log_level = INFO
log_file = log/system.log
archive_dir = archive
[driver1]
name = driver1
url = http://*.11:18088/driver
[driver2]
name = driver2
url = http://*.12:18088/driver
[driver3]
name = driver3
url = http://*.13:18088/driver
[driver4]
name = driver4
url = http://*.14:18088/driver
[driver5]
name = driver5
url = http://*.15:18088/driver
[driver6]
name = driver6
url = http://*.16:18088/driver
[driver7]
name = driver7
url = http://*.17:18088/driver
[driver]
name=hostip:18088
url=http://hostip:18088/driver
2.4 进程启动
在host1中,启动脚本start-all.sh 在host2~7中,启动脚本start-driver.sh
2.5 界面展示
##3 Cosbench使用 Cosbench的使用核心在于测试的配置文件编写,工具根据配置文件对整体集群进行读写测试。此处上一个配置文件示例:
<?xml version="1.0" encoding="UTF-8" ?>
<workload name="s3-sample" description="sample benchmark for s3">
<storage type="s3" config="accesskey= ;secretkey= ;proxyhost=;proxyport=;endpoint=" />
<workflow>
<workstage name="init">
<work type="init" workers="1" config="cprefix=s3testqwer;containers=r(1,2)" />
</workstage>
<workstage name="prepare">
<work type="prepare" workers="7" config="cprefix=s3testqwer;containers=r(1,2);objects=r(1,10);sizes=c(500)KB" />
</workstage>
<workstage name="main">
<work name="main" workers="8" runtime="30">
<!-- <operation type="read" ratio="100" config="cprefix=s3testqwer;containers=u(1,2);objects=u(1,10)" />-->
<operation type="write" ratio="100" config="cprefix=s3testqwer;containers=u(1,2);objects=u(11,20);sizes=c(500)KB" />
</work>
</workstage>
<workstage name="cleanup">
<work type="cleanup" workers="1" config="cprefix=s3testqwer;containers=r(1,2);objects=r(1,20)" />
</workstage>
<workstage name="dispose">
<work type="dispose" workers="1" config="cprefix=s3testqwer;containers=r(1,2)" />
</workstage>
</workflow>
</workload>
3.1 写测试 此处我针对写测试进行一个简单的说明 在main阶段配置wokrer 数量,控制输入压力, runtime控制程序运行时间,在operation的SIZE处设置文件大小。
<workstage name="main">
<work name="main" workers="200" runtime="100">
<operation type="write" ratio="100" config="cprefix=s3testqwer;containers=u(1,2);objects=u(11,20);sizes=c(500)KB" />
</work>
</workstage>
3.2 写结果
在配置文件的endpoint属性中配置时,请写Http://ip:port的方式,不过不标明http,他会默认采用https的方式,我之前没有配置Http时,日志报错如下:
2016-02-24 11:51:34,005 [INFO] [Log4jLogManager] - will append log to file /home/xuwenping/cosbench/cosbench/0.4.2.c3/log/mission/M2116744222.log
2016-02-24 11:51:34,225 [INFO] [NoneStorage] - performing PUT at /s3testqwer1
2016-02-24 11:51:38,436 [WARN] [S3Storage] - below exception encountered when creating bucket s3testqwer1: Unable to execute HTTP request: Connection to https://ip refused
2016-02-24 11:51:38,442 [INFO] [NoneStorage] - performing PUT at /s3testqwer2
2016-02-24 11:51:42,650 [WARN] [S3Storage] - below exception encountered when creating bucket s3testqwer2: Unable to execute HTTP request: Connection to https://ip refused
日志中写道,连接https://...... 时被拒绝。
看完上述内容是否对您有帮助呢?如果还想对相关知识有进一步的了解或阅读更多相关文章,请关注天达云行业资讯频道,感谢您对天达云的支持。