当前位置: 移动技术网 > IT编程>开发语言>Java > Elasticsearch Java高级客户端

Elasticsearch Java高级客户端

2019年01月08日  | 移动技术网IT编程  | 我要评论

1.  概述

java rest client 有两种风格:

  • java low level rest client :用于elasticsearch的官方低级客户端。它允许通过http与elasticsearch集群通信。将请求编排和响应反编排留给用户自己处理。它兼容所有的elasticsearch版本。(ps:学过webservice的话,对编排与反编排这个概念应该不陌生。可以理解为对请求参数的封装,以及对响应结果的解析)
  • java high level rest client :用于elasticsearch的官方高级客户端。它是基于低级客户端的,它提供很多api,并负责请求的编排与响应的反编排。(ps:就好比是,一个是传自己拼接好的字符串,并且自己解析返回的结果;而另一个是传对象,返回的结果也已经封装好了,直接是对象,更加规范了参数的名称以及格式,更加面对对象一点)

(ps:所谓低级与高级,我觉得一个很形象的比喻是,面向过程编程与面向对象编程)

在 elasticsearch 7.0 中不建议使用transportclient,并且在8.0中会完全删除transportclient。因此,官方更建议我们用java high level rest client,它执行http请求,而不是序列号的java请求。既然如此,这里就直接用高级了。

 

2.  java high level rest client (高级rest客户端)

2.1.  maven仓库

<dependency>
    <groupid>org.elasticsearch.client</groupid>
    <artifactid>elasticsearch-rest-high-level-client</artifactid>
    <version>6.5.4</version>
</dependency>

2.2.  依赖

  • org.elasticsearch.client:elasticsearch-rest-client
  • org.elasticsearch:elasticsearch

2.3.  初始化

resthighlevelclient client = new resthighlevelclient(
        restclient.builder(
                new httphost("localhost", 9200, "http"),
                new httphost("localhost", 9201, "http")));

高级客户端内部会创建低级客户端用于基于提供的builder执行请求。低级客户端维护一个连接池,并启动一些线程,因此当你用完以后应该关闭高级客户端,并且在内部它将会关闭低级客户端,以释放这些资源。关闭客户端可以使用close()方法:

client.close();

 

2.4.  文档api

2.4.1.  添加文档

indexrequest

indexrequest request = new indexrequest("posts", "doc", "1");
string jsonstring = "{\"user\":\"kimchy\",\"postdate\":\"2013-01-30\",\"message\":\"trying out elasticsearch\"}";
request.source(jsonstring, xcontenttype.json);

提供文档source的方式还有很多,比如:

通过map的方式提供文档source

通过xcontentbuilder方式提供source

通过object的方式(键值对)提供source

可选参数

同步执行

异步执行

你也可以异步执行 indexrequest,为此你需要指定一个监听器来处理这个异步响应结果:

一个典型的监听器看起来是这样的:

indexresponse

如果有版本冲突,将会抛出elasticsearchexception

同样的异常也有可能发生在当optype设置为create的时候,且相同索引、相同类型、相同id的文档已经存在时。例如:

 

2.4.2.  查看文档

get request

可选参数

同步执行

异步执行

get response

当索引不存在,或者指定的文档的版本不存在时,响应状态吗是404,并且抛出elasticsearchexception

 

2.4.3.  文档是否存在

 

2.4.4.  删除文档

delete request

可选参数

同添加

 

2.5.  搜索api

search request

基本格式是这样的:

大多数查询参数被添加到 searchsourcebuilder

可选参数

searchsourcebuilder

控制检索行为的大部分选项都可以在searchsourcebuilder中设置。下面是一个常见选项的例子:

在这个例子中,我们首先创建了一个searchsourcebuilder对象,并且带着默认选项。然后设置了一个term查询,接着设置检索的位置和数量,最后设置超时时间

在设置完这些选项以后,我们只需要把searchsourcebuilder加入到searchrequest中即可

 

构建query

用querybuilder来创建serarch query。querybuilder支持elasticsearch dsl中每一种query

例如:

还可以通过querybuilders工具类来创建querybuilder对象,例如:

无论是用哪种方式创建,最后一定要把querybuilder添加到searchsourcebuilder中

 

排序

searchsourcebuilder 可以添加一个或多个 sortbuilder

sortbuilder有四种实现:fieldsortbuilder、geodistancesortbuilder、scoresortbuilder、scriptsortbuilder

 

聚集函数

同步执行

异步执行

从查询响应中取出文档

 

3.  示例

3.1.  准备数据

3.1.1.  安装ik分词器插件

./bin/elasticsearch-plugin install https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v6.5.4/elasticsearch-analysis-ik-6.5.4.zip

3.1.2.  创建索引

curl -x put "localhost:9200/book" -h 'content-type: application/json' -d'
{
    "mappings":{
        "_doc":{
            "properties":{
                "id":{
                    "type":"integer"
                },
                "name":{
                    "type":"text",
                    "analyzer":"ik_max_word",
                    "search_analyzer":"ik_max_word"
                },
                "author":{
                    "type":"text",
                    "analyzer":"ik_max_word",
                    "search_analyzer":"ik_max_word"
                },
                "category":{
                    "type":"integer"
                },
                "price":{
                    "type":"double"
                },
                "status":{
                    "type":"short"
                },
                "sellreason":{
                    "type":"text",
                    "analyzer":"ik_max_word",
                    "search_analyzer":"ik_max_word"
                },
                "selltime":{
                    "type":"date",
                    "format":"yyyy-mm-dd"
                }
            }
        }
    }
}
'

3.1.3.  数据预览

 

3.2.  示例代码

3.2.1.  完整的pom.xml

<?xml version="1.0" encoding="utf-8"?>
<project xmlns="http://maven.apache.org/pom/4.0.0" xmlns:xsi="http://www.w3.org/2001/xmlschema-instance"
         xsi:schemalocation="http://maven.apache.org/pom/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelversion>4.0.0</modelversion>
    <parent>
        <groupid>org.springframework.boot</groupid>
        <artifactid>spring-boot-starter-parent</artifactid>
        <version>2.1.1.release</version>
        <relativepath/> <!-- lookup parent from repository -->
    </parent>
    <groupid>com.cjs.example</groupid>
    <artifactid>elasticsearch-demo</artifactid>
    <version>0.0.1-snapshot</version>
    <name>elasticsearch-demo</name>
    <description></description>

    <properties>
        <java.version>1.8</java.version>
    </properties>

    <dependencies>
        <dependency>
            <groupid>org.springframework.boot</groupid>
            <artifactid>spring-boot-starter-web</artifactid>
        </dependency>
        <dependency>
            <groupid>org.springframework.boot</groupid>
            <artifactid>spring-boot-starter-thymeleaf</artifactid>
        </dependency>

        <dependency>
            <groupid>org.elasticsearch.client</groupid>
            <artifactid>elasticsearch-rest-high-level-client</artifactid>
            <version>6.5.4</version>
        </dependency>

        <dependency>
            <groupid>org.apache.commons</groupid>
            <artifactid>commons-lang3</artifactid>
            <version>3.8</version>
        </dependency>
        <dependency>
            <groupid>com.alibaba</groupid>
            <artifactid>fastjson</artifactid>
            <version>1.2.54</version>
        </dependency>
        <dependency>
            <groupid>ch.qos.logback</groupid>
            <artifactid>logback-core</artifactid>
            <version>1.2.3</version>
        </dependency>
        <dependency>
            <groupid>org.projectlombok</groupid>
            <artifactid>lombok</artifactid>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupid>org.springframework.boot</groupid>
            <artifactid>spring-boot-starter-test</artifactid>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupid>org.springframework.boot</groupid>
                <artifactid>spring-boot-maven-plugin</artifactid>
            </plugin>
        </plugins>
    </build>

</project>

3.2.2.  配置

package com.cjs.example.elasticsearch.config;

import org.apache.http.httphost;
import org.elasticsearch.client.restclient;
import org.elasticsearch.client.resthighlevelclient;
import org.springframework.context.annotation.bean;
import org.springframework.context.annotation.configuration;

/**
 * @author chengjiansheng
 * @date 2019-01-07
 */
@configuration
public class elasticsearchclientconfig {

    @bean
    public resthighlevelclient resthighlevelclient() {
        resthighlevelclient client = new resthighlevelclient(
                restclient.builder(
                        new httphost("localhost", 9200, "http")));
        return client;
    }

}

3.2.3.  domain

package com.cjs.example.elasticsearch.domain.model;

import lombok.data;

import java.io.serializable;

/**
 * 图书
 * @author chengjiansheng
 * @date 2019-01-07
 */
@data
public class bookmodel implements serializable {


    private integer id;         //  图书id

    private string name;        //  图书名称

    private string author;      //  作者

    private integer category;   //  图书分类

    private double price;       //  图书价格

    private string sellreason;  //  上架理由

    private string selltime;      //  上架时间

    private integer status;     //  状态(1:可售,0:不可售)

}

3.2.4.  controller

package com.cjs.example.elasticsearch.controller;

import com.alibaba.fastjson.json;
import com.cjs.example.elasticsearch.domain.common.baseresult;
import com.cjs.example.elasticsearch.domain.common.page;
import com.cjs.example.elasticsearch.domain.model.bookmodel;
import com.cjs.example.elasticsearch.domain.vo.bookrequestvo;
import com.cjs.example.elasticsearch.service.bookservice;
import lombok.extern.slf4j.slf4j;
import org.springframework.beans.factory.annotation.autowired;
import org.springframework.web.bind.annotation.*;

/**
 * 文档操作
 * @author chengjiansheng
 * @date 2019-01-07
 */
@slf4j
@restcontroller
@requestmapping("/book")
public class bookcontroller {

    @autowired
    private bookservice bookservice;

    /**
     * 列表分页查询
     */
    @getmapping("/list")
    public baseresult list(bookrequestvo bookrequestvo) {
        page<bookmodel> page = bookservice.list(bookrequestvo);
        if (null == page) {
            return baseresult.error();
        }
        return baseresult.ok(page);
    }

    /**
     * 查看文档
     */
    @getmapping("/detail")
    public baseresult detail(integer id) {
        if (null == id) {
            return baseresult.error("id不能为空");
        }
        bookmodel book = bookservice.detail(id);
        return baseresult.ok(book);
    }

    /**
     * 添加文档
     */
    @postmapping("/add")
    public baseresult add(@requestbody bookmodel bookmodel) {
        bookservice.save(bookmodel);
        log.info("插入文档成功!请求参数: {}", json.tojsonstring(bookmodel));
        return baseresult.ok();
    }

    /**
     * 修改文档
     */
    @postmapping("/update")
    public baseresult update(@requestbody bookmodel bookmodel) {
        integer id = bookmodel.getid();
        if (null == id) {
            return baseresult.error("id不能为空");
        }
        bookmodel book = bookservice.detail(id);
        if (null == book) {
            return baseresult.error("记录不存在");
        }
        bookservice.update(bookmodel);
        log.info("更新文档成功!请求参数: {}", json.tojsonstring(bookmodel));
        return baseresult.ok();
    }

    /**
     * 删除文档
     */
    @getmapping("/delete")
    public baseresult delete(integer id) {
        if (null == id) {
            return baseresult.error("id不能为空");
        }
        bookservice.delete(id);
        return baseresult.ok();
    }

}

3.2.5.  service

package com.cjs.example.elasticsearch.service.impl;

import com.alibaba.fastjson.json;
import com.cjs.example.elasticsearch.domain.common.page;
import com.cjs.example.elasticsearch.domain.model.bookmodel;
import com.cjs.example.elasticsearch.domain.vo.bookrequestvo;
import com.cjs.example.elasticsearch.service.bookservice;
import lombok.extern.slf4j.slf4j;
import org.apache.commons.lang3.stringutils;
import org.elasticsearch.action.actionlistener;
import org.elasticsearch.action.docwriteresponse;
import org.elasticsearch.action.delete.deleterequest;
import org.elasticsearch.action.delete.deleteresponse;
import org.elasticsearch.action.get.getrequest;
import org.elasticsearch.action.get.getresponse;
import org.elasticsearch.action.index.indexrequest;
import org.elasticsearch.action.index.indexresponse;
import org.elasticsearch.action.search.searchrequest;
import org.elasticsearch.action.search.searchresponse;
import org.elasticsearch.action.support.replication.replicationresponse;
import org.elasticsearch.action.update.updaterequest;
import org.elasticsearch.action.update.updateresponse;
import org.elasticsearch.client.requestoptions;
import org.elasticsearch.client.resthighlevelclient;
import org.elasticsearch.common.unit.timevalue;
import org.elasticsearch.index.query.boolquerybuilder;
import org.elasticsearch.index.query.querybuilders;
import org.elasticsearch.rest.reststatus;
import org.elasticsearch.search.searchhit;
import org.elasticsearch.search.searchhits;
import org.elasticsearch.search.builder.searchsourcebuilder;
import org.elasticsearch.search.sort.fieldsortbuilder;
import org.elasticsearch.search.sort.sortorder;
import org.springframework.beans.factory.annotation.autowired;
import org.springframework.stereotype.service;
import org.springframework.util.collectionutils;

import java.io.ioexception;
import java.util.*;
import java.util.stream.collectors;

/**
 * @author chengjiansheng
 * @date 2019-01-07
 */
@slf4j
@service
public class bookserviceimpl implements bookservice {

    private static final string index_name = "book";
    private static final string index_type = "_doc";

    @autowired
    private resthighlevelclient client;


    @override
    public page<bookmodel> list(bookrequestvo bookrequestvo) {
        int pageno = bookrequestvo.getpageno();
        int pagesize = bookrequestvo.getpagesize();

        searchsourcebuilder sourcebuilder = new searchsourcebuilder();
        sourcebuilder.from(pageno - 1);
        sourcebuilder.size(pagesize);
        sourcebuilder.sort(new fieldsortbuilder("id").order(sortorder.asc));
//        sourcebuilder.query(querybuilders.matchallquery());

        boolquerybuilder boolquerybuilder = querybuilders.boolquery();

        if (stringutils.isnotblank(bookrequestvo.getname())) {
            boolquerybuilder.must(querybuilders.matchquery("name", bookrequestvo.getname()));
        }
        if (stringutils.isnotblank(bookrequestvo.getauthor())) {
            boolquerybuilder.must(querybuilders.matchquery("author", bookrequestvo.getauthor()));
        }
        if (null != bookrequestvo.getstatus()) {
            boolquerybuilder.must(querybuilders.termquery("status", bookrequestvo.getstatus()));
        }
        if (stringutils.isnotblank(bookrequestvo.getselltime())) {
            boolquerybuilder.must(querybuilders.termquery("selltime", bookrequestvo.getselltime()));
        }
        if (stringutils.isnotblank(bookrequestvo.getcategories())) {
            string[] categoryarr = bookrequestvo.getcategories().split(",");
            list<integer> categorylist = arrays.aslist(categoryarr).stream().map(e->integer.valueof(e)).collect(collectors.tolist());
            boolquerybuilder categoryboolquerybuilder = querybuilders.boolquery();
            for (integer category : categorylist) {
                categoryboolquerybuilder.should(querybuilders.termquery("category", category));
            }
            boolquerybuilder.must(categoryboolquerybuilder);
        }

        sourcebuilder.query(boolquerybuilder);

        searchrequest searchrequest = new searchrequest();
        searchrequest.indices(index_name);
        searchrequest.source(sourcebuilder);

        try {
            searchresponse searchresponse = client.search(searchrequest, requestoptions.default);

            reststatus reststatus = searchresponse.status();
            if (reststatus != reststatus.ok) {
                return null;
            }

            list<bookmodel> list = new arraylist<>();
            searchhits searchhits = searchresponse.gethits();
            for (searchhit hit : searchhits.gethits()) {
                string source = hit.getsourceasstring();
                bookmodel book = json.parseobject(source, bookmodel.class);
                list.add(book);
            }

            long totalhits = searchhits.gettotalhits();

            page<bookmodel> page = new page<>(pageno, pagesize, totalhits, list);

            timevalue took = searchresponse.gettook();
            log.info("查询成功!请求参数: {}, 用时{}毫秒", searchrequest.source().tostring(), took.millis());

            return page;
        } catch (ioexception e) {
            log.error("查询失败!原因: {}", e.getmessage(), e);
        }

        return null;
    }

    @override
    public void save(bookmodel bookmodel) {
        map<string, object> jsonmap = new hashmap<>();
        jsonmap.put("id", bookmodel.getid());
        jsonmap.put("name", bookmodel.getname());
        jsonmap.put("author", bookmodel.getauthor());
        jsonmap.put("category", bookmodel.getcategory());
        jsonmap.put("price", bookmodel.getprice());
        jsonmap.put("selltime", bookmodel.getselltime());
        jsonmap.put("sellreason", bookmodel.getsellreason());
        jsonmap.put("status", bookmodel.getstatus());

        indexrequest indexrequest = new indexrequest(index_name, index_type, string.valueof(bookmodel.getid()));
        indexrequest.source(jsonmap);

        client.indexasync(indexrequest, requestoptions.default, new actionlistener<indexresponse>() {
            @override
            public void onresponse(indexresponse indexresponse) {
                string index = indexresponse.getindex();
                string type = indexresponse.gettype();
                string id = indexresponse.getid();
                long version = indexresponse.getversion();

                log.info("index: {}, type: {}, id: {}, version: {}", index, type, id, version);

                if (indexresponse.getresult() == docwriteresponse.result.created) {
                    log.info("写入文档");
                } else if (indexresponse.getresult() == docwriteresponse.result.updated) {
                    log.info("修改文档");
                }
                replicationresponse.shardinfo shardinfo = indexresponse.getshardinfo();
                if (shardinfo.gettotal() != shardinfo.getsuccessful()) {
                    log.warn("部分分片写入成功");
                }
                if (shardinfo.getfailed() > 0) {
                    for (replicationresponse.shardinfo.failure failure : shardinfo.getfailures()) {
                        string reason = failure.reason();
                        log.warn("失败原因: {}", reason);
                    }
                }
            }

            @override
            public void onfailure(exception e) {
                log.error(e.getmessage(), e);
            }
        });
    }

    @override
    public void update(bookmodel bookmodel) {
        map<string, object> jsonmap = new hashmap<>();
        jsonmap.put("sellreason", bookmodel.getsellreason());
        updaterequest request = new updaterequest(index_name, index_type, string.valueof(bookmodel.getid()));
        request.doc(jsonmap);
        try {
            updateresponse updateresponse = client.update(request, requestoptions.default);
        } catch (ioexception e) {
            log.error("更新失败!原因: {}", e.getmessage(), e);
        }
    }

    @override
    public void delete(int id) {
        deleterequest request = new deleterequest(index_name, index_type, string.valueof(id));
        try {
            deleteresponse deleteresponse = client.delete(request, requestoptions.default);
            if (deleteresponse.status() == reststatus.ok) {
                log.info("删除成功!id: {}", id);
            }
        } catch (ioexception e) {
            log.error("删除失败!原因: {}", e.getmessage(), e);
        }
    }

    @override
    public bookmodel detail(int id) {
        getrequest getrequest = new getrequest(index_name, index_type, string.valueof(id));
        try {
            getresponse getresponse = client.get(getrequest, requestoptions.default);
            if (getresponse.isexists()) {
                string source = getresponse.getsourceasstring();
                bookmodel book = json.parseobject(source, bookmodel.class);
                return book;
            }
        } catch (ioexception e) {
            log.error("查看失败!原因: {}", e.getmessage(), e);
        }
        return null;
    }
}

3.2.6.  页面

<!doctype html>
<html lang="zh">
<head>
    <meta charset="utf-8">
    <title>图书列表</title>

    <link rel="stylesheet" href="/bootstrap-4/css/bootstrap.min.css">
    <link rel="stylesheet" href="/bootstrap-table/bootstrap-table.css">

    <script src="jquery-3.3.1.min.js"></script>
    <script src="/bootstrap-4/js/bootstrap.min.js"></script>
    <script src="/bootstrap-table/bootstrap-table.js"></script>
    <script src="/bootstrap-table/locale/bootstrap-table-zh-cn.js"></script>
    <script>
        $(function(){

            $('#table').bootstraptable({
                url: '/book/list',
                method: 'get',
                sidepagination: 'server',
                responsehandler: function(res) {  // 加载服务器数据之前的处理程序,可以用来格式化数据。参数:res为从服务器请求到的数据。
                    var result = {};
                    result.total = res.data.totalcount;
                    result.rows = res.data.pagelist;
                    return result;
                },
                pagination: true,
                pagesize: 3, // 初始pagesize
                queryparams: function(params) {
                    var req = {
                        pagesize: params.limit,
                        pageno: params.offset + 1
                    };
                    return req;
                },
                striped: true,
                search: true,
                columns: [{
                    field: 'id',
                    title: 'id'
                }, {
                    field: 'name',
                    title: '名称'
                }, {
                    field: 'author',
                    title: '作者'
                }, {
                    field: 'price',
                    title: '单价'
                }, {
                    field: 'selltime',
                    title: '上架时间'
                }, {
                    field: 'status',
                    title: '状态',
                    formatter: function(value) {
                        if (value == 1) {
                            return '<span style="color: green">可售</span>';
                        } else {
                            return '<span style="color: red">不可售</span>';
                        }
                    }
                }, {
                    field: 'category',
                    title: '分类',
                    formatter: function(value) {
                        if (value == 10010) {
                            return '中国当代小说';
                        } else if (value == 10011) {
                            return '武侠小说';
                        } else if (value == 10012) {
                            return '爱情小说';
                        } else if (value == 10013) {
                            return '中国当代随笔';
                        }
                    }
                }, {
                    field: 'sellreason',
                    title: '上架理由'
                }, {
                    title: '操作',
                    formatter: function() {
                        return '<a href="#">修改</a> <a href="#">删除</a>';
                    }
                }
                ]
            });

        });
    </script>
</head>
<body>
    <div class="table-responsive" style="padding: 10px 30px">
        <table id="table" class="table text-nowrap"></table>
    </div>
</body>
</html>

3.3.  演示

重点演示几个查询

返回结果:

{
    "code": 200,
    "success": true,
    "msg": "success",
    "data": {
        "pagenumber": 1,
        "pagesize": 10,
        "totalcount": 2,
        "pagelist": [
            {
                "id": 2,
                "name": "倚天屠龙记(全四册)",
                "author": "金庸",
                "category": 10011,
                "price": 70.4,
                "sellreason": "武林至尊,宝刀屠龙,号令天下,莫敢不从。",
                "selltime": "2018-11-11",
                "status": 1
            },
            {
                "id": 3,
                "name": "神雕侠侣",
                "author": "金庸",
                "category": 10011,
                "price": 70,
                "sellreason": "风陵渡口初相遇,一见杨过误终身",
                "selltime": "2018-11-11",
                "status": 1
            }
        ]
    }
}

上面的查询对应的elasticsearch dsl是这样的:

{
    "from":0,
    "size":10,
    "query":{
        "bool":{
            "must":[
                {
                    "match":{
                        "author":{
                            "query":"金庸",
                            "operator":"or",
                            "prefix_length":0,
                            "max_expansions":50,
                            "fuzzy_transpositions":true,
                            "lenient":false,
                            "zero_terms_query":"none",
                            "auto_generate_synonyms_phrase_query":true,
                            "boost":1
                        }
                    }
                },
                {
                    "term":{
                        "status":{
                            "value":1,
                            "boost":1
                        }
                    }
                },
                {
                    "bool":{
                        "should":[
                            {
                                "term":{
                                    "category":{
                                        "value":10010,
                                        "boost":1
                                    }
                                }
                            },
                            {
                                "term":{
                                    "category":{
                                        "value":10011,
                                        "boost":1
                                    }
                                }
                            },
                            {
                                "term":{
                                    "category":{
                                        "value":10012,
                                        "boost":1
                                    }
                                }
                            }
                        ],
                        "adjust_pure_negative":true,
                        "boost":1
                    }
                }
            ],
            "adjust_pure_negative":true,
            "boost":1
        }
    },
    "sort":[
        {
            "id":{
                "order":"asc"
            }
        }
    ]
}

3.4.  工程结构

 

4.  参考

 

5.  其它相关

elasticsearch 分词器

elasticsearch document

elasticsearch search api

elasticsearch查询

elasticsearch mapping

springboot+elasticsearch

elk快速搭建日志平台

 

如对本文有疑问, 点击进行留言回复!!

相关文章:

验证码:
移动技术网