当前位置：移动技术网 > IT编程>开发语言>C/C++ > 【分析】Ceph编程实例接口Librbd(C++)--映像创建与数据读写

【分析】Ceph编程实例接口Librbd(C++)--映像创建与数据读写

2018年03月01日 | 移动技术网IT编程 | 我要评论

连锁加盟店餐饮,女体化,林祖銮

目前我们有两种路径使用Ceph的块存储：?
- 利用QEMU/KVM通过librbd与 Ceph 块设备交互，主要为虚拟机提供块存储设备，如下图所示；?

- 利用kernel module与Host kernel交互，主要为物理机提供块设备支持。

Librbd 是Ceph提供的块存储接口的抽象，它提供C/C++、Python等多种接口。对于C++，最主要的两个类就是RBD?和?Image。?RBD?主要负责创建、删除、克隆映像等操作，而Image?类负责映像的读写等操作。

`准备工作`

对于任何客户端应用，都需要首先连接到一个运行良好的Ceph集群。

`获取集群句柄`

//声明Rados对象，并初始化
librados::Rados rados;
ret = rados.init("admin"); // just use the client.admin keyring
if (ret < 0) { // let's handle any error that might have come back
    std::cerr << "couldn't initialize rados! err " << ret << std::endl;
    ret = EXIT_FAILURE;
    return EXIT_FAILURE;
} else {
    std::cout << "we just set up a rados cluster object" << std::endl;
}

//获取配置文件信息: /etc/ceph/ceph.conf
// 1. 根据命令行参数
/*
ret = rados.conf_parse_argv(argc, argv);
if (ret < 0) {
    // This really can't happen, but we need to check to be a good citizen.
    std::cerr << "failed to parse config options! error " << ret << std::endl;
    ret = EXIT_FAILURE;
    return EXIT_FAILURE;
} else {
    std::cout << "we just parsed our config options" << std::endl;
    // We also want to apply the config file if the user specified
    // one, and conf_parse_argv won't do that for us.
    for (int i = 0; i < argc; ++i) {
        if ((strcmp(argv[i], "-c") == 0) || (strcmp(argv[i], "--conf") == 0)) {
            ret = rados.conf_read_file(argv[i+1]);
            if (ret < 0) {
                // This could fail if the config file is malformed, but it'd be hard.
                std::cerr << "failed to parse config file " << argv[i+1]
                        << "! error" << ret << std::endl;
                ret = EXIT_FAILURE;
                return EXIT_FAILURE;
            }
            break;
        }
    }
}
*/
// 2. 程序里面指定
ret = rados.conf_read_file("/etc/ceph/ceph.conf");
if (ret < 0) {
    // This could fail if the config file is malformed, but it'd be hard.
    std::cerr << "failed to parse config file! err " << ret << std::endl;
    ret = EXIT_FAILURE;
    return EXIT_FAILURE;
}

`连接集群`

ret = rados.connect();
if (ret < 0) {
    std::cerr << "couldn't connect to cluster! err " << ret << std::endl;
    ret = EXIT_FAILURE;
    return EXIT_FAILURE;
} else {
    std::cout << "we just connected to the rados cluster" << std::endl;
}

`创建I/O上下文环境`

如果没有存储池，需要先新建一个存储池。

`新建存储池`

const char *pool_name = "gnar";
ret = rados.pool_create(pool_name);
if (ret < 0) {
    std::cerr << "couldn't create pool! error " << ret << std::endl;
    ret = EXIT_FAILURE;
    rados.shutdown(); //断开集群连接
    return EXIT_FAILURE;
} else {
    std::cout << "we just created a new pool named " << pool_name << std::endl;
}

`新建I/O上下文环境`

librados::IoCtx io_ctx; //I/O上下文
const char *pool_name = "gnar";
ret = rados.ioctx_create(pool_name, io_ctx);
if (ret < 0) {
    std::cerr << "couldn't setup ioctx! err " << ret << std::endl;
    ret = EXIT_FAILURE;
    rados.shutdown(); //断开集群连接
    return EXIT_FAILURE; 
} else {
    std::cout << "we just created an ioctx for our pool" << std::endl;
}

`若干RBD映像API`

`声明RBD对象，创建rbd映像`

librbd::RBD rbd;
const char *image_name = "rumboo";
uint64_t init_size = (uint64_t) 200 * 1024 * 1024; //映像初始化大小200MB
uint64_t features = 1; //影响feature个数
int order = 22; //默认值为22, 即4MB (1 << 22)
ret = rbd.create2(io_ctx, image_name, init_size, features, &order);
if (ret < 0) {
    std::cerr << "couldn't create rbd image! err " << ret << std::endl;
    ret = EXIT_FAILURE;
    io_ctx.close(); //关闭I/O上下文
    rados.shutdown(); //断开集群连接
    return EXIT_FAILURE;
} else {
    std::cout << "We just created an rbd image" << std::endl;
}

`打开rbd映像`

librbd::RBD rbd;
const char *image_name = "rumboo";
librbd::Image image;
ret = rbd.open(io_ctx, image, image_name);
if (ret < 0) {
    std::cerr << "couldn't open rbd image! err " << ret << std::endl;
    ret = EXIT_FAILURE;
    io_ctx.close(); //关闭I/O上下文
    rados.shutdown(); //断开集群连接
    return EXIT_FAILURE;
} else {
    std::cout << "We just opened an rbd image" << std::endl;
}

`查看映像大小`

uint64_t size = 0;
ret = image.size(&size);
if (ret < 0) {
    std::cerr << "couldn't get image size! err " << ret << std::endl;
    ret = EXIT_FAILURE;
    return EXIT_FAILURE;
} else {
    std::cout << "The size of the image is " << size << std::endl;
}

`调整映像大小`

size = (uint64_t) 500 * 1024 * 1024; //调整映像大小500MB
ret = image.resize(size);
if (ret < 0) {
    std::cerr << "couldn't change the size of the image! err " << ret << std::endl;
    ret = EXIT_FAILURE;
    return EXIT_FAILURE;
} else {
    std::cout << "We just change the size of the image" << std::endl;
}

`查看映像ID`

std::string id;
ret = image.get_id(&id);
if (ret < 0) {
    std::cerr << "couldn't get image ID! err " << ret << std::endl;
    ret = EXIT_FAILURE;
    return EXIT_FAILURE;
} else {
    std::cout << "The ID of the image is " << id << std::endl;
}

`查看映像features`

features = 0;
ret = image.features(&features);
if (ret < 0) {
    std::cerr << "couldn't get image features! err " << ret << std::endl;
    ret = EXIT_FAILURE;
    return EXIT_FAILURE;
} else {
    std::cout << "The features of the image are " << features << std::endl;
}

`查看映像状态信息`

librbd::image_info_t info;
ret = image.stat(info, sizeof(info));
if (ret < 0) {
    std::cerr << "couldn't get image stat_info! err " << ret << std::endl;
    ret = EXIT_FAILURE;
    return EXIT_FAILURE;
} else {
    std::cout << "info.size is " << info.size << std::endl;
    std::cout << "info.obj_size is " << info.obj_size << std::endl;
    std::cout << "info.num_objs is " << info.num_objs << std::endl;
    std::cout << "info.order is " << info.order << std::endl;
    std::cout << "info.block_name_prefix is " << info.block_name_prefix << std::endl;
}

`查看存储池ID`

std::cout << "data pool id is " << image.get_data_pool_id() << std::endl;

1

`查看block_name_prefix`

std::cout << "block name prefix is " << image.get_block_name_prefix() << std::endl;

`查看flags`

uint64_t flags = 0;
ret = image.get_flags(&flags);
if (ret < 0) {
    std::cerr << "couldn't get image flags! err " << ret << std::endl;
    ret = EXIT_FAILURE;
    return EXIT_FAILURE;
} else {
    std::cout << "image flags is " << flags << std::endl;
}

`查看条带化参数`

std::cout << "image stripe unit is " << image.get_stripe_unit() << std::endl;
std::cout << "image stripe count is " << image.get_stripe_count() << std::endl;

`RBD映像数据读写`

`数据读写 – synchronous`

uint64_t ofs_w = (uint64_t) 0; //读写偏移量
uint64_t ofs_r = (uint64_t) 0;
size_t len_w = 100; //读写长度
size_t len_r = 100;
ceph::bufferlist bl_w; //读写bufferlist
ceph::bufferlist bl_r;
const char *fn_i = "input"; //读写文件名
const char *fn_o = "output";
std::string error;

ret = bl_r.read_file(fn_i, &error);
std::cout << "read file ret = " << ret << std::endl;
if (ret < 0) {
    std::cerr << "couldn't read file! err " << ret << std::endl;
    ret = EXIT_FAILURE;
    image.close(); //关闭rbd映像
    io_ctx.close(); //关闭I/O上下文
    rados.shutdown(); //断开集群连接
    return EXIT_FAILURE;
} else {
    std::cout << "We just read a file" << std::endl;
}
ssize_t ret_w = image.write2(ofs_w, len_w, bl_r, 0);

ssize_t ret_r = image.read2(ofs_r, len_r, bl_w, 0);
ret = bl_w.write_file(fn_o, 0644);
std::cout << "write file ret = " << ret << std::endl;
if (ret < 0) {
    std::cerr << "couldn't write file! err " << ret << std::endl;
    ret = EXIT_FAILURE;
    image.close(); //关闭rbd映像
    io_ctx.close(); //关闭I/O上下文
    rados.shutdown(); //断开集群连接
    return EXIT_FAILURE;
} else {
    std::cout << "We just wrote a file" << std::endl;
}

`数据读写 – asynchronous`

std::string data = "foo"; 
uint64_t ofs_aiow = (uint64_t) 100; //读写偏移量
uint64_t ofs_aior = (uint64_t) 100;
size_t len_aiow = 600; //读写长度
size_t len_aior = 600;
ceph::bufferlist bl_aiow; //读写bufferlist
ceph::bufferlist bl_aior;
librbd::RBD::AioCompletion *write_completion = new librbd::RBD::AioCompletion(
    NULL, (librbd::callback_t) simple_write_cb); //读写AioCompletion
librbd::RBD::AioCompletion *read_completion = new librbd::RBD::AioCompletion(
    NULL, (librbd::callback_t) simple_read_cb);

for (int i = 0; i < 200; ++i) {
    bl_aior.append(data);
}
std::cout << bl_aior.to_str() << std::endl;

ret = image.aio_write2(ofs_aiow, len_aiow, bl_aior, write_completion, 0);
if (ret < 0) {
    std::cerr << "couldn't start write! error " << ret << std::endl;
    ret = EXIT_FAILURE;
    image.close(); //关闭rbd映像
    io_ctx.close(); //关闭I/O上下文
    rados.shutdown(); //断开集群连接
    return EXIT_FAILURE;
}
write_completion->wait_for_complete(); //等待写完成
ret_w = write_completion->get_return_value();
if (ret_w < 0) {
    std::cerr << "couldn't write! error " << ret << std::endl;
    ret_w = EXIT_FAILURE;
    image.close(); //关闭rbd映像
    io_ctx.close(); //关闭I/O上下文
    rados.shutdown(); //断开集群连接
    return EXIT_FAILURE;
} else {
    std::cout << "we just write data successfully, return value is " << ret_w << std::endl;
}

ret = image.aio_read2(ofs_aior, len_aior, bl_aiow, read_completion, 0);
if (ret < 0) {
    std::cerr << "couldn't start read! error " << ret << std::endl;
    ret = EXIT_FAILURE;
    image.close(); //关闭rbd映像
    io_ctx.close(); //关闭I/O上下文
    rados.shutdown(); //断开集群连接
    return EXIT_FAILURE;
}
read_completion->wait_for_complete(); //等待读完成
ret_r = read_completion->get_return_value();
if (ret_r < 0) {
    std::cerr << "couldn't read! error " << ret << std::endl;
    ret_r = EXIT_FAILURE;
    image.close(); //关闭rbd映像
    io_ctx.close(); //关闭I/O上下文
    rados.shutdown(); //断开集群连接
    return EXIT_FAILURE;
} else {
    std::cout << "we just read data successfully, return value is " << ret_r << std::endl;
}
std::cout << bl_aiow.to_str() << std::endl;

write_completion->release();
read_completion->release();

void simple_write_cb(librbd::completion_t cb, void *arg) {
    std::cout << "write completion cb called!" << std::endl;
}

//简单的回调函数，用于librbd::RBD::AioCompletion
void simple_read_cb(librbd::completion_t cb, void *arg) {
    std::cout << "read completion cb called!" << std::endl;
}

`收尾工作`

在最后，一定不能忘记关闭rbd映像、I/O上下文，断开集群连接。

ret = image.close(); //关闭rbd映像
if (ret < 0) {
    std::cerr << "couldn't close rbd image! err " << ret << std::endl;
    ret = EXIT_FAILURE;
    return EXIT_FAILURE;
} else {
    std::cout << "we just closed an rbd image" << std::endl;
}
io_ctx.close(); //关闭I/O上下文
rados.shutdown(); //断开集群连接
return EXIT_SUCCESS;

您可能感兴趣的文章:

如对本文有疑问，请在下面进行留言讨论，广大热心网友会与你互动！！点击进行留言回复

C++ 作用域

作用域：名称在翻译单元（包括文件）的可见范围局部：只在定义它的代码块中可用，如自动变量全局（文件作用域）：从定义位置到文件结尾都可用注意：静... [阅读全文]
Prim计算最小生成树权值 C语言

题目描述现在，你被委托在一个广阔区域里面为某些确定的结点设计连接网络。首先，你会给定在区域里面的一系列结点，和连接这些结点的一组线路。对于每条可能使用... [阅读全文]
C打印楼梯，同时在楼梯上方打印两个笑脸

题目：打印楼梯，同时在楼梯上方打印两个笑脸。程序分析：用 ASCII 1 来输出笑脸；用i控制行，j来控制列，j根据i的变化来控制输出黑方格的个数。 ... [阅读全文]
算法笔记刷题6 ( PAT 1003我要通过 )

算法笔记刷题6 ( PAT1003我要通过 ) 题目本体 “ 答案正确 ”是自动判题系统给出的最令人欢喜的回复。本题属于 PAT 的“ 答案正确 ”大派... [阅读全文]
聚合类型与POD类型

Lippman在《深度探索C++对象模型》的前言中写道： I have heard a number of people over the years ... [阅读全文]
GetAsyncKeyState 获取键盘按键消息

1 #include <Windows.h> 2 #include <iostream> 3 using namespace s... [阅读全文]
c primer plus(中文版)第一章的一处错误

c primer plus(中文版)1.8.8 第三段第三段第一句：“UNIX系统内置Mac OS X”。这句话讲不通，UNIX系统内置MAC OS ... [阅读全文]
如何在没有core文件的情况下用dmesg+addr2line定位段错误

前言在现网环境下，程序奔溃后不一定会留下core文件，原因有很多，比如存储空间不足就是其中一个常见的原因。此时我们只能依据linux记录的错误日志来定位... [阅读全文]
用QT制作3D点云显示器——QtDataVisualization

因为QT的三维显示模块QtDataVisualization已经对个人开发免费开放了，所以在制作点云，地图，表格之类的东西的时候，其实我们都不需要使用Q... [阅读全文]
判断一个数是否为素数（质数）

质数是指在大于1的自然数中，除了1和它本身以外不再有其他因数的自然数。 ... [阅读全文]

网友评论


验证码：

【分析】Ceph编程实例接口Librbd(C++)--映像创建与数据读写

2018年03月01日 | 移动技术网IT编程 | 我要评论

准备工作

获取集群句柄

连接集群

创建I/O上下文环境

新建存储池

新建I/O上下文环境

若干RBD映像API

声明RBD对象，创建rbd映像

打开rbd映像

查看映像大小

调整映像大小

查看映像ID

查看映像features

查看映像状态信息

查看存储池ID

查看block_name_prefix

查看flags

查看条带化参数

RBD映像数据读写

数据读写 – synchronous

数据读写 – asynchronous

收尾工作