百度360必应搜狗淘宝本站头条
当前位置:网站首页 > 技术教程 > 正文

ElasticSearch进阶:一文全览各种ES查询在Java中的实现

csdh11 2025-04-10 22:04 2 浏览

ElasticSearch多种查询操作

  • 前言
  • 1 词条查询
  • 1.1 等值查询-term1.2 多值查询-terms1.3 范围查询-range1.4 前缀查询-prefix1.5 通配符查询-wildcard
  • 2 复合查询
  • 2.1 布尔查询2.2 Filter查询
  • 3 聚合查询
  • 3.1 最值、平均值、求和3.2 去重查询3.3 分组聚合3.3.1 单条件分组3.3.2 多条件分组3.4 过滤聚合

前言

  • ElasticSearch第一篇:ElasticSearch基础:从倒排索引说起,快速认知ES

这篇博文的主题是ES的查询,因此我整理了尽可能齐全的ES查询场景,形成下面的图:

本文基于elasticsearch 7.13.2版本,es从7.0以后,发生了很大的更新。7.3以后,已经不推荐使用TransportClient这个client,取而代之的是Java High Level REST Client

测试使用的数据示例

首先是,Mysql中的部分测试数据:

id

name

age

sex

address

sect

skill

power

create_time

modify_time

1

张无忌

18

光明顶

明教

九阳神功

99

2021-05-14 16:50:33

2021-06-29 16:48:56

2

周芷若

17

峨眉山

峨嵋派

九阴真经

88

2021-05-14 11:37:07

2021-06-29 16:56:40

3

赵敏

14

大都

朝廷

40

2021-05-14 11:37:07

2021-06-29 15:22:24

Mysql中的一行数据在ES中以一个文档形式存在:

{  "_index" : "person",  "_type" : "_doc",  "_id" : "4",  "_score" : 1.0,  "_source" : {    "address" : "峨眉山",    "modifyTime" : "2021-06-29 19:46:25",    "createTime" : "2021-05-14 11:37:07",    "sect" : "峨嵋派",    "sex" : "男",    "skill" : "降龙十八掌",    "name" : "宋青书",    "id" : 4,    "power" : 50,    "age" : 21  }}

简单梳理了一下ES JavaAPI的相关体系,感兴趣的可以自己研读一下源码。

接下来,我们用十几个实例,迅速上手ES的查询操作,每个示例将提供SQL语句、ES语句和Java代码。

1 词条查询

所谓词条查询,也就是ES不会对查询条件进行分词处理,只有当词条和查询字符串完全匹配时,才会被查询到。

1.1 等值查询-term

等值查询,即筛选出一个字段等于特定值得所有记录。

SQL:

select * from person where name = '张无忌';

而使用ES查询语句却很不一样(注意查询字段带上keyword):

GET /person/_search{"query": {"term": {"name.keyword": {"value": "张无忌","boost": 1.0}}}}

ElasticSearch 5.0以后,string类型有重大变更,移除了string类型,string字段被拆分成两种新的数据类型: text用于全文搜索的,而keyword用于关键词搜索的。

查询结果:

{  "took" : 0,  "timed_out" : false,  "_shards" : { // 分片信息    "total" : 1, // 总计分片数    "successful" : 1, // 查询成功的分片数    "skipped" : 0, // 跳过查询的分片数    "failed" : 0  // 查询失败的分片数  },  "hits" : { // 命中结果    "total" : {      "value" : 1, // 数量      "relation" : "eq"  // 关系:等于    },    "max_score" : 2.8526313,  // 最高分数    "hits" : [      {        "_index" : "person", // 索引        "_type" : "_doc", // 类型        "_id" : "1",        "_score" : 2.8526313,        "_source" : {          "address" : "光明顶",          "modifyTime" : "2021-06-29 16:48:56",          "createTime" : "2021-05-14 16:50:33",          "sect" : "明教",          "sex" : "男",          "skill" : "九阳神功",          "name" : "张无忌",          "id" : 1,          "power" : 99,          "age" : 18        }      }    ]  }}

Java中构造ES请求的方式:(后续例子中只保留SearchSourceBuilder的构建语句)

/** * term精确查询 * * @throws IOException */@Autowiredprivate RestHighLevelClient client;@Testpublic void queryTerm() throws IOException {// 根据索引创建查询请求    SearchRequest searchRequest = new SearchRequest("person");    SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();    // 构建查询语句    searchSourceBuilder.query(QueryBuilders.termQuery("name.keyword", "张无忌"));    System.out.println("searchSourceBuilder=====================" + searchSourceBuilder);    searchRequest.source(searchSourceBuilder);    SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);    System.out.println(JSONObject.toJSON(response));}

仔细观察查询结果,会发现ES查询结果中会带有_score这一项,ES会根据结果匹配程度进行评分。打分是会耗费性能的,如果确认自己的查询不需要评分,就设置查询语句关闭评分:

GET /person/_search{"query": {"constant_score": {"filter": {"term": {"sect.keyword": {"value": "张无忌","boost": 1.0}}},"boost": 1.0}}}

Java构建查询语句:

SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();// 这样构造的查询条件,将不进行score计算,从而提高查询效率searchSourceBuilder.query(QueryBuilders.constantScoreQuery(QueryBuilders.termQuery("sect.keyword", "明教")));

1.2 多值查询-terms

多条件查询类似Mysql里的IN查询,例如:

select * from persons where sect in('明教','武当派');

ES查询语句:

GET /person/_search{"query": {"terms": {"sect.keyword": ["明教","武当派"],"boost": 1.0}}}

Java实现:

SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();// 构建查询语句searchSourceBuilder.query(QueryBuilders.termsQuery("sect.keyword", Arrays.asList("明教", "武当派")));}

1.3 范围查询-range

范围查询,即查询某字段在特定区间的记录。

SQL:

select * from pesons where age between 18 and 22;

ES查询语句:

GET /person/_search{"query": {"range": {"age": {"from": 10,"to": 20,"include_lower": true,"include_upper": true,"boost": 1.0}}}}

Java构建查询条件:

SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();// 构建查询语句searchSourceBuilder.query(QueryBuilders.rangeQuery("age").gte(10).lte(30));}

1.4 前缀查询-prefix

前缀查询类似于SQL中的模糊查询。

SQL:

select * from persons where sect like '武当%';

ES查询语句:

{"query": {"prefix": {"sect.keyword": {"value": "武当","boost": 1.0}}}}

Java构建查询条件:

SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();// 构建查询语句searchSourceBuilder.query(QueryBuilders.prefixQuery("sect.keyword","武当"));

1.5 通配符查询-wildcard

通配符查询,与前缀查询类似,都属于模糊查询的范畴,但通配符显然功能更强。

SQL:

select * from persons where name like '张%忌';

ES查询语句:

{"query": {"wildcard": {"sect.keyword": {"wildcard": "张*忌","boost": 1.0}}}}

Java构建查询条件:

SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();// 构建查询语句searchSourceBuilder.query(QueryBuilders.wildcardQuery("sect.keyword","张*忌"));

2 复合查询

前面的例子都是单个条件查询,在实际应用中,我们很有可能会过滤多个值或字段。先看一个简单的例子:

select * from persons where sex = '女' and sect = '明教';

这样的多条件等值查询,就要借用到组合过滤器了,其查询语句是:

{"query": {"bool": {"must": [{    "term": {"sex": {"value": "女","boost": 1.0}}},{"term": {"sect.keywords": {"value": "明教","boost": 1.0}}}],"adjust_pure_negative": true,"boost": 1.0}}}

Java构造查询语句:

SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();// 构建查询语句searchSourceBuilder.query(QueryBuilders.boolQuery()        .must(QueryBuilders.termQuery("sex", "女"))        .must(QueryBuilders.termQuery("sect.keyword", "明教")));

2.1 布尔查询

布尔过滤器(bool filter)属于复合过滤器(compound filter)的一种 ,可以接受多个其他过滤器作为参数,并将这些过滤器结合成各式各样的布尔(逻辑)组合。

bool 过滤器下可以有4种子条件,可以任选其中任意一个或多个。filter是比较特殊的,这里先不说。

{   "bool" : {      "must" :     [],      "should" :   [],      "must_not" : [],   }}
  • must:所有的语句都必须匹配,与 ‘=’ 等价。
  • must_not:所有的语句都不能匹配,与 ‘!=’ 或 not in 等价。
  • should:至少有n个语句要匹配,n由参数控制。

精度控制:

所有 must 语句必须匹配,所有 must_not 语句都必须不匹配,但有多少 should 语句应该匹配呢?默认情况下,没有 should 语句是必须匹配的,只有一个例外:那就是当没有 must 语句的时候,至少有一个 should 语句必须匹配。

我们可以通过 minimum_should_match 参数控制需要匹配的 should 语句的数量,它既可以是一个绝对的数字,又可以是个百分比:

GET /person/_search{"query": {"bool": {"must": [{"term": {"sex": {"value": "女","boost": 1.0}}}],"should": [{"term": {"address.keyword": {"value": "峨眉山","boost": 1.0}}},{"term": {"sect.keyword": {"value": "明教","boost": 1.0}}}],"adjust_pure_negative": true,"minimum_should_match": "1","boost": 1.0}}}

Java构建查询语句:

SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();// 构建查询语句searchSourceBuilder.query(QueryBuilders.boolQuery()        .must(QueryBuilders.termQuery("sex", "女"))        .should(QueryBuilders.termQuery("address.word", "峨眉山"))        .should(QueryBuilders.termQuery("sect.keyword", "明教"))        .minimumShouldMatch(1));

最后,看一个复杂些的例子,将bool的各子句联合使用:

select *frompersonswhere sex = '女'andage between 30 and 40and sect != '明教'and (address = '峨眉山' OR skill = '暗器')

Elasticsearch 来表示上面的 SQL 例子:

GET /person/_search{"query": {"bool": {"must": [{"term": {"sex": {"value": "女","boost": 1.0}}},{"range": {"age": {"from": 30,"to": 40,"include_lower": true,"include_upper": true,"boost": 1.0}}}],"must_not": [{"term": {"sect.keyword": {"value": "明教","boost": 1.0}}}],"should": [{"term": {"address.keyword": {"value": "峨眉山","boost": 1.0}}},{"term": {"skill.keyword": {"value": "暗器","boost": 1.0}}}],"adjust_pure_negative": true,"minimum_should_match": "1","boost": 1.0}}}

用Java构建这个查询条件:

SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();// 构建查询语句BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery()        .must(QueryBuilders.termQuery("sex", "女"))        .must(QueryBuilders.rangeQuery("age").gte(30).lte(40))        .mustNot(QueryBuilders.termQuery("sect.keyword", "明教"))        .should(QueryBuilders.termQuery("address.keyword", "峨眉山"))        .should(QueryBuilders.rangeQuery("power.keyword").gte(50).lte(80))        .minimumShouldMatch(1);  // 设置should至少需要满足几个条件// 将BoolQueryBuilder构建到SearchSourceBuilder中searchSourceBuilder.query(boolQueryBuilder);

2.2 Filter查询

query和filter的区别:query查询的时候,会先比较查询条件,然后计算分值,最后返回文档结果;而filter是先判断是否满足查询条件,如果不满足会缓存查询结果(记录该文档不满足结果),满足的话,就直接缓存结果,filter不会对结果进行评分,能够提高查询效率

filter的使用方式比较多样,下面用几个例子演示一下。

方式一,单独使用:

{"query": {"bool": {"filter": [{"term": {"sex": {"value": "男","boost": 1.0}}}],"adjust_pure_negative": true,"boost": 1.0}}}

单独使用时,filter与must基本一样,不同的是filter不计算评分,效率更高

Java构建查询语句:

SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();// 构建查询语句searchSourceBuilder.query(QueryBuilders.boolQuery()        .filter(QueryBuilders.termQuery("sex", "男")));

方式二,和must、must_not同级,相当于子查询:

select * from (select * from persons where sect = '明教')) a where sex = '女';

ES查询语句:

{"query": {"bool": {"must": [{"term": {"sect.keyword": {"value": "明教","boost": 1.0}}}],"filter": [{"term": {"sex": {"value": "女","boost": 1.0}}}],"adjust_pure_negative": true,"boost": 1.0}}}

Java:

SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();// 构建查询语句searchSourceBuilder.query(QueryBuilders.boolQuery()        .must(QueryBuilders.termQuery("sect.keyword", "明教"))        .filter(QueryBuilders.termQuery("sex", "女")));

方式三,将must、must_not置于filter下,这种方式是最常用的:

{"query": {"bool": {"filter": [{"bool": {"must": [{"term": {"sect.keyword": {"value": "明教","boost": 1.0}}},{"range": {"age": {"from": 20,"to": 35,"include_lower": true,"include_upper": true,"boost": 1.0}}}],"must_not": [{"term": {"sex.keyword": {"value": "女","boost": 1.0}}}],"adjust_pure_negative": true,"boost": 1.0}}],"adjust_pure_negative": true,"boost": 1.0}}}

Java:

SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();// 构建查询语句searchSourceBuilder.query(QueryBuilders.boolQuery()        .filter(QueryBuilders.boolQuery()                .must(QueryBuilders.termQuery("sect.keyword", "明教"))                .must(QueryBuilders.rangeQuery("age").gte(20).lte(35))                .mustNot(QueryBuilders.termQuery("sex.keyword", "女"))));

3 聚合查询

接下来,我们将用一些案例演示ES聚合查询。

3.1 最值、平均值、求和

案例:查询最大年龄、最小年龄、平均年龄。

SQL:

select max(age) from persons;

ES:

GET /person/_search{"aggregations": {"max_age": {"max": {"field": "age"}}}}

Java:

@Autowiredprivate RestHighLevelClient client;@Testpublic void maxQueryTest() throws IOException {// 聚合查询条件    AggregationBuilder aggBuilder = AggregationBuilders.max("max_age").field("age");    SearchRequest searchRequest = new SearchRequest("person");    SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();    // 将聚合查询条件构建到SearchSourceBuilder中    searchSourceBuilder.aggregation(aggBuilder);    System.out.println("searchSourceBuilder----->" + searchSourceBuilder);    searchRequest.source(searchSourceBuilder);    // 执行查询,获取SearchResponse    SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);    System.out.println(JSONObject.toJSON(response));}

使用聚合查询,结果中默认只会返回10条文档数据(当然我们关心的是聚合的结果,而非文档)。返回多少条数据可以自主控制:

GET /person/_search{"size": 20,"aggregations": {"max_age": {"max": {"field": "age"}}}}

而Java中只需增加下面一条语句即可:

searchSourceBuilder.size(20);

与max类似,其他统计查询也很简单:

AggregationBuilder minBuilder = AggregationBuilders.min("min_age").field("age");AggregationBuilder avgBuilder = AggregationBuilders.avg("min_age").field("age");AggregationBuilder sumBuilder = AggregationBuilders.sum("min_age").field("age");AggregationBuilder countBuilder = AggregationBuilders.count("min_age").field("age");

3.2 去重查询

案例:查询一共有多少个门派。

SQL:

select count(distinct sect) from persons;

ES:

{"aggregations": {"sect_count": {"cardinality": {"field": "sect.keyword"}}}}

Java:

@Testpublic void cardinalityQueryTest() throws IOException {// 创建某个索引的request    SearchRequest searchRequest = new SearchRequest("person");    // 查询条件    SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();    // 聚合查询    AggregationBuilder aggBuilder = AggregationBuilders.cardinality("sect_count").field("sect.keyword");    searchSourceBuilder.size(0);    // 将聚合查询构建到查询条件中    searchSourceBuilder.aggregation(aggBuilder);    System.out.println("searchSourceBuilder----->" + searchSourceBuilder);    searchRequest.source(searchSourceBuilder);    // 执行查询,获取结果    SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);    System.out.println(JSONObject.toJSON(response));}

3.3 分组聚合

3.3.1 单条件分组

案例:查询每个门派的人数

SQL:

select sect,count(id) from mytest.persons group by sect;

ES:

{"size": 0,"aggregations": {"sect_count": {"terms": {"field": "sect.keyword","size": 10,"min_doc_count": 1,"shard_min_doc_count": 0,"show_term_doc_count_error": false,"order": [{"_count": "desc"},{"_key": "asc"}]}}}}

Java:

SearchRequest searchRequest = new SearchRequest("person");SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();searchSourceBuilder.size(0);// 按sect分组AggregationBuilder aggBuilder = AggregationBuilders.terms("sect_count").field("sect.keyword");searchSourceBuilder.aggregation(aggBuilder);

3.3.2 多条件分组

案例:查询每个门派各有多少个男性和女性

SQL:

select sect,sex,count(id) from mytest.persons group by sect,sex;

ES:

{"aggregations": {"sect_count": {"terms": {"field": "sect.keyword","size": 10},"aggregations": {"sex_count": {"terms": {"field": "sex.keyword","size": 10}}}}}}

3.4 过滤聚合

前面所有聚合的例子请求都省略了 query ,整个请求只不过是一个聚合。这意味着我们对全部数据进行了聚合,但现实应用中,我们常常对特定范围的数据进行聚合,例如下例。

案例:查询明教中的最大年龄。 这涉及到聚合与条件查询一起使用。

SQL:

select max(age) from mytest.persons where sect = '明教';

ES:

GET /person/_search{"query": {"term": {"sect.keyword": {"value": "明教","boost": 1.0}}},"aggregations": {"max_age": {"max": {"field": "age"}}}}

Java:

SearchRequest searchRequest = new SearchRequest("person");SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();// 聚合查询条件AggregationBuilder maxBuilder = AggregationBuilders.max("max_age").field("age");// 等值查询searchSourceBuilder.query(QueryBuilders.termQuery("sect.keyword", "明教"));searchSourceBuilder.aggregation(maxBuilder);

另外还有一些更复杂的查询例子。

案例:查询0-20,21-40,41-60,61以上的各有多少人。

SQL:

select sum(case when age<=20 then 1 else 0 end agegroup1sumcase when age>20 and age <=40 then 1 else 0 end agegroup2sumcase when age>40 and age <=60 then 1 else 0 end agegroup3sumcase when age>60 and age <=200 then 1 else 0 end) ageGroup4from mytest.persons;

ES:

{"size": 0,"aggregations": {"age_avg": {"range": {"field": "age","ranges": [{"from": 0.0,"to": 20.0},{"from": 21.0,"to": 40.0},{"from": 41.0,"to": 60.0},{"from": 61.0,"to": 200.0}],"keyed": false}}}}

Java:

查询结果:

"aggregations" : {  "age_avg" : {    "buckets" : [      {        "key" : "0.0-20.0",        "from" : 0.0,        "to" : 20.0,        "doc_count" : 3      },      {        "key" : "21.0-40.0",        "from" : 21.0,        "to" : 40.0,        "doc_count" : 13      },      {        "key" : "41.0-60.0",        "from" : 41.0,        "to" : 60.0,        "doc_count" : 4      },      {        "key" : "61.0-200.0",        "from" : 61.0,        "to" : 200.0,        "doc_count" : 1      }    ]  }}

以上是ElasticSearch查询的全部内容,丰富详实,堪比操作手册,强烈建议收藏!ElasticSearch多种查询操作

ElasticSearch多种查询操作

  • 前言
  • 1 词条查询
  • 1.1 等值查询-term1.2 多值查询-terms1.3 范围查询-range1.4 前缀查询-prefix1.5 通配符查询-wildcard
  • 2 复合查询
  • 2.1 布尔查询2.2 Filter查询
  • 3 聚合查询
  • 3.1 最值、平均值、求和3.2 去重查询3.3 分组聚合3.3.1 单条件分组3.3.2 多条件分组3.4 过滤聚合

前言

  • ElasticSearch第一篇:ElasticSearch基础:从倒排索引说起,快速认知ES

这篇博文的主题是ES的查询,因此我整理了尽可能齐全的ES查询场景,形成下面的图:

本文基于elasticsearch 7.13.2版本,es从7.0以后,发生了很大的更新。7.3以后,已经不推荐使用TransportClient这个client,取而代之的是Java High Level REST Client

测试使用的数据示例

首先是,Mysql中的部分测试数据:

id

name

age

sex

address

sect

skill

power

create_time

modify_time

1

张无忌

18

光明顶

明教

九阳神功

99

2021-05-14 16:50:33

2021-06-29 16:48:56

2

周芷若

17

峨眉山

峨嵋派

九阴真经

88

2021-05-14 11:37:07

2021-06-29 16:56:40

3

赵敏

14

大都

朝廷

40

2021-05-14 11:37:07

2021-06-29 15:22:24

Mysql中的一行数据在ES中以一个文档形式存在:

{  "_index" : "person",  "_type" : "_doc",  "_id" : "4",  "_score" : 1.0,  "_source" : {    "address" : "峨眉山",    "modifyTime" : "2021-06-29 19:46:25",    "createTime" : "2021-05-14 11:37:07",    "sect" : "峨嵋派",    "sex" : "男",    "skill" : "降龙十八掌",    "name" : "宋青书",    "id" : 4,    "power" : 50,    "age" : 21  }}

简单梳理了一下ES JavaAPI的相关体系,感兴趣的可以自己研读一下源码。

接下来,我们用十几个实例,迅速上手ES的查询操作,每个示例将提供SQL语句、ES语句和Java代码。

1 词条查询

所谓词条查询,也就是ES不会对查询条件进行分词处理,只有当词条和查询字符串完全匹配时,才会被查询到。

1.1 等值查询-term

等值查询,即筛选出一个字段等于特定值的所有记录。

SQL:

select * from person where name = '张无忌';

而使用ES查询语句却很不一样(注意查询字段带上keyword):

GET /person/_search{"query": {"term": {"name.keyword": {"value": "张无忌","boost": 1.0}}}}

ElasticSearch 5.0以后,string类型有重大变更,移除了string类型,string字段被拆分成两种新的数据类型: text用于全文搜索的,而keyword用于关键词搜索的。

查询结果:

{  "took" : 0,  "timed_out" : false,  "_shards" : { // 分片信息    "total" : 1, // 总计分片数    "successful" : 1, // 查询成功的分片数    "skipped" : 0, // 跳过查询的分片数    "failed" : 0  // 查询失败的分片数  },  "hits" : { // 命中结果    "total" : {      "value" : 1, // 数量      "relation" : "eq"  // 关系:等于    },    "max_score" : 2.8526313,  // 最高分数    "hits" : [      {        "_index" : "person", // 索引        "_type" : "_doc", // 类型        "_id" : "1",        "_score" : 2.8526313,        "_source" : {          "address" : "光明顶",          "modifyTime" : "2021-06-29 16:48:56",          "createTime" : "2021-05-14 16:50:33",          "sect" : "明教",          "sex" : "男",          "skill" : "九阳神功",          "name" : "张无忌",          "id" : 1,          "power" : 99,          "age" : 18        }      }    ]  }}

Java中构造ES请求的方式:(后续例子中只保留SearchSourceBuilder的构建语句)

/** * term精确查询 * * @throws IOException */@Autowiredprivate RestHighLevelClient client;@Testpublic void queryTerm() throws IOException {// 根据索引创建查询请求    SearchRequest searchRequest = new SearchRequest("person");    SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();    // 构建查询语句    searchSourceBuilder.query(QueryBuilders.termQuery("name.keyword", "张无忌"));    System.out.println("searchSourceBuilder=====================" + searchSourceBuilder);    searchRequest.source(searchSourceBuilder);    SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);    System.out.println(JSONObject.toJSON(response));}

仔细观察查询结果,会发现ES查询结果中会带有_score这一项,ES会根据结果匹配程度进行评分。打分是会耗费性能的,如果确认自己的查询不需要评分,就设置查询语句关闭评分:

GET /person/_search{"query": {"constant_score": {"filter": {"term": {"sect.keyword": {"value": "张无忌","boost": 1.0}}},"boost": 1.0}}}

Java构建查询语句:

SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();// 这样构造的查询条件,将不进行score计算,从而提高查询效率searchSourceBuilder.query(QueryBuilders.constantScoreQuery(QueryBuilders.termQuery("sect.keyword", "明教")));

1.2 多值查询-terms

多条件查询类似Mysql里的IN查询,例如:

select * from persons where sect in('明教','武当派');

ES查询语句:

GET /person/_search{"query": {"terms": {"sect.keyword": ["明教","武当派"],"boost": 1.0}}}

Java实现:

SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();// 构建查询语句searchSourceBuilder.query(QueryBuilders.termsQuery("sect.keyword", Arrays.asList("明教", "武当派")));}

1.3 范围查询-range

范围查询,即查询某字段在特定区间的记录。

SQL:

select * from pesons where age between 18 and 22;

ES查询语句:

GET /person/_search{"query": {"range": {"age": {"from": 10,"to": 20,"include_lower": true,"include_upper": true,"boost": 1.0}}}}

Java构建查询条件:

SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();// 构建查询语句searchSourceBuilder.query(QueryBuilders.rangeQuery("age").gte(10).lte(30));}

1.4 前缀查询-prefix

前缀查询类似于SQL中的模糊查询。

SQL:

select * from persons where sect like '武当%';

ES查询语句:

{"query": {"prefix": {"sect.keyword": {"value": "武当","boost": 1.0}}}}

Java构建查询条件:

SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();// 构建查询语句searchSourceBuilder.query(QueryBuilders.prefixQuery("sect.keyword","武当"));

1.5 通配符查询-wildcard

通配符查询,与前缀查询类似,都属于模糊查询的范畴,但通配符显然功能更强。

SQL:

select * from persons where name like '张%忌';

ES查询语句:

{"query": {"wildcard": {"sect.keyword": {"wildcard": "张*忌","boost": 1.0}}}}

Java构建查询条件:

SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();// 构建查询语句searchSourceBuilder.query(QueryBuilders.wildcardQuery("sect.keyword","张*忌"));

2 复合查询

前面的例子都是单个条件查询,在实际应用中,我们很有可能会过滤多个值或字段。先看一个简单的例子:

select * from persons where sex = '女' and sect = '明教';

这样的多条件等值查询,就要借用到组合过滤器了,其查询语句是:

{"query": {"bool": {"must": [{    "term": {"sex": {"value": "女","boost": 1.0}}},{"term": {"sect.keywords": {"value": "明教","boost": 1.0}}}],"adjust_pure_negative": true,"boost": 1.0}}}

Java构造查询语句:

SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();// 构建查询语句searchSourceBuilder.query(QueryBuilders.boolQuery()        .must(QueryBuilders.termQuery("sex", "女"))        .must(QueryBuilders.termQuery("sect.keyword", "明教")));

2.1 布尔查询

布尔过滤器(bool filter)属于复合过滤器(compound filter)的一种 ,可以接受多个其他过滤器作为参数,并将这些过滤器结合成各式各样的布尔(逻辑)组合。

bool 过滤器下可以有4种子条件,可以任选其中任意一个或多个。filter是比较特殊的,这里先不说。

{   "bool" : {      "must" :     [],      "should" :   [],      "must_not" : [],   }}
  • must:所有的语句都必须匹配,与 ‘=’ 等价。
  • must_not:所有的语句都不能匹配,与 ‘!=’ 或 not in 等价。
  • should:至少有n个语句要匹配,n由参数控制。

精度控制:

所有 must 语句必须匹配,所有 must_not 语句都必须不匹配,但有多少 should 语句应该匹配呢?默认情况下,没有 should 语句是必须匹配的,只有一个例外:那就是当没有 must 语句的时候,至少有一个 should 语句必须匹配。

我们可以通过 minimum_should_match 参数控制需要匹配的 should 语句的数量,它既可以是一个绝对的数字,又可以是个百分比:

GET /person/_search{"query": {"bool": {"must": [{"term": {"sex": {"value": "女","boost": 1.0}}}],"should": [{"term": {"address.keyword": {"value": "峨眉山","boost": 1.0}}},{"term": {"sect.keyword": {"value": "明教","boost": 1.0}}}],"adjust_pure_negative": true,"minimum_should_match": "1","boost": 1.0}}}

Java构建查询语句:

SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();// 构建查询语句searchSourceBuilder.query(QueryBuilders.boolQuery()        .must(QueryBuilders.termQuery("sex", "女"))        .should(QueryBuilders.termQuery("address.word", "峨眉山"))        .should(QueryBuilders.termQuery("sect.keyword", "明教"))        .minimumShouldMatch(1));

最后,看一个复杂些的例子,将bool的各子句联合使用:

select *frompersonswhere sex = '女'andage between 30 and 40and sect != '明教'and (address = '峨眉山' OR skill = '暗器')

Elasticsearch 来表示上面的 SQL 例子:

GET /person/_search{"query": {"bool": {"must": [{"term": {"sex": {"value": "女","boost": 1.0}}},{"range": {"age": {"from": 30,"to": 40,"include_lower": true,"include_upper": true,"boost": 1.0}}}],"must_not": [{"term": {"sect.keyword": {"value": "明教","boost": 1.0}}}],"should": [{"term": {"address.keyword": {"value": "峨眉山","boost": 1.0}}},{"term": {"skill.keyword": {"value": "暗器","boost": 1.0}}}],"adjust_pure_negative": true,"minimum_should_match": "1","boost": 1.0}}}

用Java构建这个查询条件:

SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();// 构建查询语句BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery()        .must(QueryBuilders.termQuery("sex", "女"))        .must(QueryBuilders.rangeQuery("age").gte(30).lte(40))        .mustNot(QueryBuilders.termQuery("sect.keyword", "明教"))        .should(QueryBuilders.termQuery("address.keyword", "峨眉山"))        .should(QueryBuilders.rangeQuery("power.keyword").gte(50).lte(80))        .minimumShouldMatch(1);  // 设置should至少需要满足几个条件// 将BoolQueryBuilder构建到SearchSourceBuilder中searchSourceBuilder.query(boolQueryBuilder);

2.2 Filter查询

query和filter的区别:query查询的时候,会先比较查询条件,然后计算分值,最后返回文档结果;而filter是先判断是否满足查询条件,如果不满足会缓存查询结果(记录该文档不满足结果),满足的话,就直接缓存结果,filter不会对结果进行评分,能够提高查询效率

filter的使用方式比较多样,下面用几个例子演示一下。

方式一,单独使用:

{"query": {"bool": {"filter": [{"term": {"sex": {"value": "男","boost": 1.0}}}],"adjust_pure_negative": true,"boost": 1.0}}}

单独使用时,filter与must基本一样,不同的是filter不计算评分,效率更高

Java构建查询语句:

SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();// 构建查询语句searchSourceBuilder.query(QueryBuilders.boolQuery()        .filter(QueryBuilders.termQuery("sex", "男")));

方式二,和must、must_not同级,相当于子查询:

select * from (select * from persons where sect = '明教')) a where sex = '女';

ES查询语句:

{"query": {"bool": {"must": [{"term": {"sect.keyword": {"value": "明教","boost": 1.0}}}],"filter": [{"term": {"sex": {"value": "女","boost": 1.0}}}],"adjust_pure_negative": true,"boost": 1.0}}}

Java:

SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();// 构建查询语句searchSourceBuilder.query(QueryBuilders.boolQuery()        .must(QueryBuilders.termQuery("sect.keyword", "明教"))        .filter(QueryBuilders.termQuery("sex", "女")));

方式三,将must、must_not置于filter下,这种方式是最常用的:

{"query": {"bool": {"filter": [{"bool": {"must": [{"term": {"sect.keyword": {"value": "明教","boost": 1.0}}},{"range": {"age": {"from": 20,"to": 35,"include_lower": true,"include_upper": true,"boost": 1.0}}}],"must_not": [{"term": {"sex.keyword": {"value": "女","boost": 1.0}}}],"adjust_pure_negative": true,"boost": 1.0}}],"adjust_pure_negative": true,"boost": 1.0}}}

Java:

SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();// 构建查询语句searchSourceBuilder.query(QueryBuilders.boolQuery()        .filter(QueryBuilders.boolQuery()                .must(QueryBuilders.termQuery("sect.keyword", "明教"))                .must(QueryBuilders.rangeQuery("age").gte(20).lte(35))                .mustNot(QueryBuilders.termQuery("sex.keyword", "女"))));

3 聚合查询

接下来,我们将用一些案例演示ES聚合查询。

3.1 最值、平均值、求和

案例:查询最大年龄、最小年龄、平均年龄。

SQL:

select max(age) from persons;

ES:

GET /person/_search{"aggregations": {"max_age": {"max": {"field": "age"}}}}

Java:

@Autowiredprivate RestHighLevelClient client;@Testpublic void maxQueryTest() throws IOException {// 聚合查询条件    AggregationBuilder aggBuilder = AggregationBuilders.max("max_age").field("age");    SearchRequest searchRequest = new SearchRequest("person");    SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();    // 将聚合查询条件构建到SearchSourceBuilder中    searchSourceBuilder.aggregation(aggBuilder);    System.out.println("searchSourceBuilder----->" + searchSourceBuilder);    searchRequest.source(searchSourceBuilder);    // 执行查询,获取SearchResponse    SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);    System.out.println(JSONObject.toJSON(response));}

使用聚合查询,结果中默认只会返回10条文档数据(当然我们关心的是聚合的结果,而非文档)。返回多少条数据可以自主控制:

GET /person/_search{"size": 20,"aggregations": {"max_age": {"max": {"field": "age"}}}}

而Java中只需增加下面一条语句即可:

searchSourceBuilder.size(20);

与max类似,其他统计查询也很简单:

AggregationBuilder minBuilder = AggregationBuilders.min("min_age").field("age");AggregationBuilder avgBuilder = AggregationBuilders.avg("min_age").field("age");AggregationBuilder sumBuilder = AggregationBuilders.sum("min_age").field("age");AggregationBuilder countBuilder = AggregationBuilders.count("min_age").field("age");

3.2 去重查询

案例:查询一共有多少个门派。

SQL:

select count(distinct sect) from persons;

ES:

{"aggregations": {"sect_count": {"cardinality": {"field": "sect.keyword"}}}}

Java:

@Testpublic void cardinalityQueryTest() throws IOException {// 创建某个索引的request    SearchRequest searchRequest = new SearchRequest("person");    // 查询条件    SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();    // 聚合查询    AggregationBuilder aggBuilder = AggregationBuilders.cardinality("sect_count").field("sect.keyword");    searchSourceBuilder.size(0);    // 将聚合查询构建到查询条件中    searchSourceBuilder.aggregation(aggBuilder);    System.out.println("searchSourceBuilder----->" + searchSourceBuilder);    searchRequest.source(searchSourceBuilder);    // 执行查询,获取结果    SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);    System.out.println(JSONObject.toJSON(response));}

3.3 分组聚合

3.3.1 单条件分组

案例:查询每个门派的人数

SQL:

select sect,count(id) from mytest.persons group by sect;

ES:

{"size": 0,"aggregations": {"sect_count": {"terms": {"field": "sect.keyword","size": 10,"min_doc_count": 1,"shard_min_doc_count": 0,"show_term_doc_count_error": false,"order": [{"_count": "desc"},{"_key": "asc"}]}}}}

Java:

SearchRequest searchRequest = new SearchRequest("person");SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();searchSourceBuilder.size(0);// 按sect分组AggregationBuilder aggBuilder = AggregationBuilders.terms("sect_count").field("sect.keyword");searchSourceBuilder.aggregation(aggBuilder);

3.3.2 多条件分组

案例:查询每个门派各有多少个男性和女性

SQL:

select sect,sex,count(id) from mytest.persons group by sect,sex;

ES:

{"aggregations": {"sect_count": {"terms": {"field": "sect.keyword","size": 10},"aggregations": {"sex_count": {"terms": {"field": "sex.keyword","size": 10}}}}}}

3.4 过滤聚合

前面所有聚合的例子请求都省略了 query ,整个请求只不过是一个聚合。这意味着我们对全部数据进行了聚合,但现实应用中,我们常常对特定范围的数据进行聚合,例如下例。

案例:查询明教中的最大年龄。 这涉及到聚合与条件查询一起使用。

SQL:

select max(age) from mytest.persons where sect = '明教';

ES:

GET /person/_search{"query": {"term": {"sect.keyword": {"value": "明教","boost": 1.0}}},"aggregations": {"max_age": {"max": {"field": "age"}}}}

Java:

SearchRequest searchRequest = new SearchRequest("person");SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();// 聚合查询条件AggregationBuilder maxBuilder = AggregationBuilders.max("max_age").field("age");// 等值查询searchSourceBuilder.query(QueryBuilders.termQuery("sect.keyword", "明教"));searchSourceBuilder.aggregation(maxBuilder);

另外还有一些更复杂的查询例子。

案例:查询0-20,21-40,41-60,61以上的各有多少人。

SQL:

select sum(case when age<=20 then 1 else 0 end agegroup1sumcase when age>20 and age <=40 then 1 else 0 end agegroup2sumcase when age>40 and age <=60 then 1 else 0 end agegroup3sumcase when age>60 and age <=200 then 1 else 0 end) ageGroup4from mytest.persons;

ES:

{"size": 0,"aggregations": {"age_avg": {"range": {"field": "age","ranges": [{"from": 0.0,"to": 20.0},{"from": 21.0,"to": 40.0},{"from": 41.0,"to": 60.0},{"from": 61.0,"to": 200.0}],"keyed": false}}}}

Java:

查询结果:

"aggregations" : {  "age_avg" : {    "buckets" : [      {        "key" : "0.0-20.0",        "from" : 0.0,        "to" : 20.0,        "doc_count" : 3      },      {        "key" : "21.0-40.0",        "from" : 21.0,        "to" : 40.0,        "doc_count" : 13      },      {        "key" : "41.0-60.0",        "from" : 41.0,        "to" : 60.0,        "doc_count" : 4      },      {        "key" : "61.0-200.0",        "from" : 61.0,        "to" : 200.0,        "doc_count" : 1      }    ]  }}

以上是ElasticSearch查询的全部内容,丰富详实,堪比操作手册,强烈建议收藏!ElasticSearch多种查询操作

来源:

https://blog.csdn.net/mu_wind/article/details/118267537

相关推荐

探索Java项目中日志系统最佳实践:从入门到精通

探索Java项目中日志系统最佳实践:从入门到精通在现代软件开发中,日志系统如同一位默默无闻却至关重要的管家,它记录了程序运行中的各种事件,为我们排查问题、监控性能和优化系统提供了宝贵的依据。在Java...

用了这么多年的java日志框架,你真的弄懂了吗?

在项目开发过程中,有一个必不可少的环节就是记录日志,相信只要是个程序员都用过,可是咱们自问下,用了这么多年的日志框架,你确定自己真弄懂了日志框架的来龙去脉嘛?下面笔者就详细聊聊java中常用日志框架的...

物理老师教你学Java语言(中篇)(物理专业学编程)

第四章物质的基本结构——类与对象...

一文搞定!Spring Boot3 定时任务操作全攻略

各位互联网大厂的后端开发小伙伴们,在使用SpringBoot3开发项目时,你是否遇到过定时任务实现的难题呢?比如任务调度时间不准确,代码报错却找不到方向,是不是特别头疼?如今,随着互联网业务规模...

你还不懂java的日志系统吗 ?(java的日志类)

一、背景在java的开发中,使用最多也绕不过去的一个话题就是日志,在程序中除了业务代码外,使用最多的就是打印日志。经常听到的这样一句话就是“打个日志调试下”,没错在日常的开发、调试过程中打印日志是常干...

谈谈枚举的新用法--java(java枚举的作用与好处)

问题的由来前段时间改游戏buff功能,干了一件愚蠢的事情,那就是把枚举和运算集合在一起,然后运行一段时间后buff就出现各种问题,我当时懵逼了!事情是这样的,做过游戏的都知道,buff,需要分类型,且...

你还不懂java的日志系统吗(javaw 日志)

一、背景在java的开发中,使用最多也绕不过去的一个话题就是日志,在程序中除了业务代码外,使用最多的就是打印日志。经常听到的这样一句话就是“打个日志调试下”,没错在日常的开发、调试过程中打印日志是常干...

Java 8之后的那些新特性(三):Java System Logger

去年12月份log4j日志框架的一个漏洞,给Java整个行业造成了非常大的影响。这个事情也顺带把log4j这个日志框架推到了争议的最前线。在Java领域,log4j可能相对比较流行。而在log4j之外...

Java开发中的日志管理:让程序“开口说话”

Java开发中的日志管理:让程序“开口说话”日志是程序员的朋友,也是程序的“嘴巴”。它能让程序在运行过程中“开口说话”,告诉我们它的状态、行为以及遇到的问题。在Java开发中,良好的日志管理不仅能帮助...

吊打面试官(十二)--Java语言中ArrayList类一文全掌握

导读...

OS X 效率启动器 Alfred 详解与使用技巧

问:为什么要在Mac上使用效率启动器类应用?答:在非特殊专业用户的环境下,(每天)用户一般可以在系统中进行上百次操作,可以是点击,也可以是拖拽,但这些只是过程,而我们的真正目的是想获得结果,也就是...

Java中 高级的异常处理(java中异常处理的两种方式)

介绍异常处理是软件开发的一个关键方面,尤其是在Java中,这种语言以其稳健性和平台独立性而闻名。正确的异常处理不仅可以防止应用程序崩溃,还有助于调试并向用户提供有意义的反馈。...

【性能调优】全方位教你定位慢SQL,方法介绍下!

1.使用数据库自带工具...

全面了解mysql锁机制(InnoDB)与问题排查

MySQL/InnoDB的加锁,一直是一个常见的话题。例如,数据库如果有高并发请求,如何保证数据完整性?产生死锁问题如何排查并解决?下面是不同锁等级的区别表级锁:开销小,加锁快;不会出现死锁;锁定粒度...

看懂这篇文章,你就懂了数据库死锁产生的场景和解决方法

一、什么是死锁加锁(Locking)是数据库在并发访问时保证数据一致性和完整性的主要机制。任何事务都需要获得相应对象上的锁才能访问数据,读取数据的事务通常只需要获得读锁(共享锁),修改数据的事务需要获...