跳至主要內容
关于elasticsearch的access log

关于elasticsearch的access log

今天才知道原来elasticsearch其实并没有传统意义上的access log。我这里说的传统意义 上的access log是指类似于tomcat,nginx等的access log。 我们公司在访问elasticsearch的时候,java语言相关的使用的公司封装的(其实是我封装的) elasticsearch client,在这个封装的client里面,实现了传统意义上的类似于dubbo-consumer-access.log 一样的东西,记录了请求时间,traceId,请求的参数信息,请求是否成功的状态,响应时间,响应的内容等等 的信息。但是在elasticsearch的server端其实是并没有这些内容的。 Google了一下,发现elasticsearch其实并没有access log这个功能。但是elasticsearch缺提供了slow log的这个功能。 虽然说slow log和access log其实是两种不同的东西,但是在一定程度上slow log其实可以做到access log的一部分功能。


xkrivzooh大约 3 分钟postes
不规范的查询导致elasticsearch StackOverflowError

不规范的查询导致elasticsearch StackOverflowError

昨天我们公司的elasticsearch的集群有几个节点出现了StackOverflowError,然后es进程退出的问题。最终排查发现导致es节点出现下面的异常:

[2019-08-05T20:31:35,367][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [order6] fatal error in thread [elasticsearch[order6][search][T#38]], exiting
java.lang.StackOverflowError: null
	at org.apache.lucene.store.DataInput.readVLong(DataInput.java:184) ~[lucene-core-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 - shalin - 2016-11-02 19:47:11]
	at org.apache.lucene.store.DataInput.readVLong(DataInput.java:169) ~[lucene-core-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 - shalin - 2016-11-02 19:47:11]
	at org.apache.lucene.util.fst.FST.readUnpackedNodeTarget(FST.java:931) ~[lucene-core-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 - shalin - 2016-11-02 19:47:11]
	at org.apache.lucene.util.fst.FST.readNextRealArc(FST.java:1143) ~[lucene-core-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 - shalin - 2016-11-02 19:47:11]
	at org.apache.lucene.util.fst.FST.readFirstRealTargetArc(FST.java:992) ~[lucene-core-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 - shalin - 2016-11-02 19:47:11]
	at org.apache.lucene.util.fst.FST.findTargetArc(FST.java:1270) ~[lucene-core-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 - shalin - 2016-11-02 19:47:11]
	at org.apache.lucene.util.fst.FST.findTargetArc(FST.java:1186) ~[lucene-core-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 - shalin - 2016-11-02 19:47:11]
	at org.apache.lucene.codecs.blocktree.SegmentTermsEnum.seekExact(SegmentTermsEnum.java:483) ~[lucene-core-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 - shalin - 2016-11-02 19:47:11]
	at org.apache.lucene.search.TermQuery$TermWeight.getTermsEnum(TermQuery.java:132) ~[lucene-core-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 - shalin - 2016-11-02 19:47:11]
	at org.apache.lucene.search.TermQuery$TermWeight.scorer(TermQuery.java:100) ~[lucene-core-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 - shalin - 2016-11-02 19:47:11]
	at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.scorer(LRUQueryCache.java:746) ~[lucene-core-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 - shalin - 2016-11-02 19:47:11]
	at org.elasticsearch.indices.IndicesQueryCache$CachingWeightWrapper.scorer(IndicesQueryCache.java:155) ~[elasticsearch-5.1.1.jar:5.1.1]
	at org.apache.lucene.search.BooleanWeight.scorer(BooleanWeight.java:389) ~[lucene-core-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 - shalin - 2016-11-02 19:47:11]
	at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.scorer(LRUQueryCache.java:746) ~[lucene-core-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 - shalin - 2016-11-02 19:47:11]
	at org.elasticsearch.indices.IndicesQueryCache$CachingWeightWrapper.scorer(IndicesQueryCache.java:155) ~[elasticsearch-5.1.1.jar:5.1.1]
	at org.apache.lucene.search.BooleanWeight.scorer(BooleanWeight.java:389) ~[lucene-core-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 - shalin - 2016-11-02 19:47:11]
	at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.scorer(LRUQueryCache.java:746) ~[lucene-core-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 - shalin - 2016-11-02 19:47:11]
	at org.elasticsearch.indices.IndicesQueryCache$CachingWeightWrapper.scorer(IndicesQueryCache.java:155) ~[elasticsearch-5.1.1.jar:5.1.1]
	at org.apache.lucene.search.BooleanWeight.scorer(BooleanWeight.java:389) ~[lucene-core-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 - shalin - 2016-11-02 19:47:11]
	at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.scorer(LRUQueryCache.java:746) ~[lucene-core-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 - shalin - 2016-11-02 19:47:11]

xkrivzooh大约 3 分钟postes
优化Elasticsearch的写入

优化Elasticsearch的写入

我们这边因为需要对trace系统的数据做一些高级查询,所以会将Span的可能会被用作搜索条件的信息写入elasticsearch中。 由于trace系统的数据量比较大,虽然trace系统本身的设计会有采样率这个东西来降低trace采集的数据量,但是本身还是 比较大的数据量。所以需要对es的写入做一些优化。这篇文章记录一下我们的优化项


xkrivzooh大约 1 分钟postes
ES集群NoNodeAvailableException

ES集群NoNodeAvailableException

问题描述

今天在操作es的过程中出现了异常:

org.elasticsearch.client.transport.NoNodeAvailableException: None of the
  configured nodes are available

xkrivzooh大约 2 分钟postes
打赏
给作者赏一杯咖啡吧
您的支持将是我继续更新下去的动力
微信微信
支付宝支付宝