Enonic version: 6.4.0
OS: Ubuntu 15.10
Any suggestion on how to solve this error ?
19:09:32.088 INFO org.elasticsearch.node - [local-node] initialized
19:09:32.089 INFO org.elasticsearch.node - [local-node] starting ...
19:09:32.137 INFO org.elasticsearch.transport - [local-node] bound_address {inet[/127.0.0.1:9300]}, publish_address {inet[/127.0.0.1:9300]}
19:09:32.145 INFO org.elasticsearch.discovery - [local-node] mycluster/Iqwj9DYZRDKFMSqmIdYGtA
19:09:35.160 INFO org.elasticsearch.cluster.service - [local-node] new_master [local-node][Iqwj9DYZRDKFMSqmIdYGtA][localhost][inet[/127.0.0.1:9300]]{master=true}, reason: zen-disco-join (elected_as_master)
19:09:35.199 INFO org.elasticsearch.node - [local-node] started
19:09:35.969 WARN org.elasticsearch.indices.cluster - [local-node] [[storage-cms-repo][0]] marking and sending shard failed due to [failed recovery]
org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException: [storage-cms-repo][0] failed recovery
at org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:162) [elasticsearch-1.5.2_1.jar:na]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_45]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_45]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
Caused by: org.elasticsearch.index.engine.EngineCreationFailureException: [storage-cms-repo][0] failed to upgrade 3x segments
at org.elasticsearch.index.engine.InternalEngine.<init>(InternalEngine.java:121) ~[na:na]
at org.elasticsearch.index.engine.InternalEngineFactory.newReadWriteEngine(InternalEngineFactory.java:32) ~[na:na]
at org.elasticsearch.index.shard.IndexShard.newEngine(IndexShard.java:1262) ~[na:na]
at org.elasticsearch.index.shard.IndexShard.createNewEngine(IndexShard.java:1257) ~[na:na]
at org.elasticsearch.index.shard.IndexShard.prepareForTranslogRecovery(IndexShard.java:784) ~[na:na]
at org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:226) ~[na:na]
at org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:112) [elasticsearch-1.5.2_1.jar:na]
... 3 common frames omitted
Caused by: java.io.EOFException: read past EOF: NIOFSIndexInput(path="/var/www/dev1.vetinst.no/enonic-xp-6.4.0/home/repo/index/data/mycluster/nodes/0/indices/storage-cms-repo/0/index/segments_46")
at org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:336) ~[na:na]
at org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:54) ~[na:na]
at org.apache.lucene.store.DataInput.readInt(DataInput.java:98) ~[na:na]
at org.apache.lucene.store.BufferedIndexInput.readInt(BufferedIndexInput.java:183) ~[na:na]
at org.elasticsearch.common.lucene.Lucene.indexNeeds3xUpgrading(Lucene.java:767) ~[na:na]
at org.elasticsearch.common.lucene.Lucene.upgradeLucene3xSegmentsMetadata(Lucene.java:778) ~[na:na]
at org.elasticsearch.index.engine.InternalEngine.upgrade3xSegments(InternalEngine.java:1084) ~[na:na]
at org.elasticsearch.index.engine.InternalEngine.<init>(InternalEngine.java:119) ~[na:na]
... 9 common frames omitted
19:09:35.972 WARN o.elasticsearch.cluster.action.shard - [local-node] [storage-cms-repo][0] received shard failed for [storage-cms-repo][0], node[Iqwj9DYZRDKFMSqmIdYGtA], [P], s[INITIALIZING], indexUUID [40Rd3jAqRXqByU6y-djm8g], reason [shard failure [failed recovery][IndexShardGatewayRecoveryException[[storage-cms-repo][0] failed recovery]; nested: EngineCreationFailureException[[storage-cms-repo][0] failed to upgrade 3x segments]; nested: EOFException[read past EOF: NIOFSIndexInput(path="/var/www/dev1.vetinst.no/enonic-xp-6.4.0/home/repo/index/data/mycluster/nodes/0/indices/storage-cms-repo/0/index/segments_46")]; ]]
19:09:35.989 INFO org.elasticsearch.gateway - [local-node] recovered [4] indices into cluster_state
19:09:35.996 WARN o.elasticsearch.cluster.action.shard - [local-node] [storage-cms-repo][0] received shard failed for [storage-cms-repo][0], node[Iqwj9DYZRDKFMSqmIdYGtA], [P], s[INITIALIZING], indexUUID [40Rd3jAqRXqByU6y-djm8g], reason [master [local-node][Iqwj9DYZRDKFMSqmIdYGtA][localhost][inet[/127.0.0.1:9300]]{master=true} marked shard as initializing, but shard is marked as failed, resend shard failure]
19:09:41.010 ERROR c.e.x.e.impl.ClientActivator - Cluster health in state 'RED'
19:09:41.369 INFO org.eclipse.jetty.util.log - Logging initialized @12785ms
19:09:41.435 INFO org.eclipse.jetty.server.Server - jetty-9.3.5.v20151012
19:09:41.491 INFO o.e.j.server.handler.ContextHandler - Started o.e.j.s.ServletContextHandler@51be0347{/,null,AVAILABLE}
19:09:41.496 INFO o.e.jetty.server.ServerConnector - Started ServerConnector@dcaefe6{HTTP/1.1,[http/1.1]}{0.0.0.0:8080}
19:09:41.499 INFO org.eclipse.jetty.server.Server - Started @12915ms
19:09:41.514 INFO c.e.x.w.v.i.c.VirtualHostConfigImpl - Virtual host is enabled and mappings updated.
19:09:41.949 INFO c.e.x.l.i.framework.FrameworkService - Started Enonic XP in 12943 ms
19:09:45.187 WARN org.elasticsearch.indices.cluster - [local-node] [[storage-cms-repo][0]] marking and sending shard failed due to [failed recovery]
org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException: [storage-cms-repo][0] failed recovery
at org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:162) [elasticsearch-1.5.2_1.jar:na]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_45]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_45]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
Caused by: org.elasticsearch.index.engine.EngineCreationFailureException: [storage-cms-repo][0] failed to upgrade 3x segments
at org.elasticsearch.index.engine.InternalEngine.<init>(InternalEngine.java:121) ~[na:na]
at org.elasticsearch.index.engine.InternalEngineFactory.newReadWriteEngine(InternalEngineFactory.java:32) ~[na:na]
at org.elasticsearch.index.shard.IndexShard.newEngine(IndexShard.java:1262) ~[na:na]
at org.elasticsearch.index.shard.IndexShard.createNewEngine(IndexShard.java:1257) ~[na:na]
at org.elasticsearch.index.shard.IndexShard.prepareForTranslogRecovery(IndexShard.java:784) ~[na:na]
at org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:226) ~[na:na]
at org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:112) [elasticsearch-1.5.2_1.jar:na]
... 3 common frames omitted
Caused by: java.io.EOFException: read past EOF: NIOFSIndexInput(path="/var/www/dev1.vetinst.no/enonic-xp-6.4.0/home/repo/index/data/mycluster/nodes/0/indices/storage-cms-repo/0/index/segments_46")
at org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:336) ~[na:na]
at org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:54) ~[na:na]
at org.apache.lucene.store.DataInput.readInt(DataInput.java:98) ~[na:na]
at org.apache.lucene.store.BufferedIndexInput.readInt(BufferedIndexInput.java:183) ~[na:na]
at org.elasticsearch.common.lucene.Lucene.indexNeeds3xUpgrading(Lucene.java:767) ~[na:na]
at org.elasticsearch.common.lucene.Lucene.upgradeLucene3xSegmentsMetadata(Lucene.java:778) ~[na:na]
at org.elasticsearch.index.engine.InternalEngine.upgrade3xSegments(InternalEngine.java:1084) ~[na:na]
at org.elasticsearch.index.engine.InternalEngine.<init>(InternalEngine.java:119) ~[na:na]
... 9 common frames omitted
19:09:45.188 WARN o.elasticsearch.cluster.action.shard - [local-node] [storage-cms-repo][0] received shard failed for [storage-cms-repo][0], node[Iqwj9DYZRDKFMSqmIdYGtA], [P], s[INITIALIZING], indexUUID [40Rd3jAqRXqByU6y-djm8g], reason [shard failure [failed recovery][IndexShardGatewayRecoveryException[[storage-cms-repo][0] failed recovery]; nested: EngineCreationFailureException[[storage-cms-repo][0] failed to upgrade 3x segments]; nested: EOFException[read past EOF: NIOFSIndexInput(path="/var/www/dev1.vetinst.no/enonic-xp-6.4.0/home/repo/index/data/mycluster/nodes/0/indices/storage-cms-repo/0/index/segments_46")]; ]]
19:09:47.013 ERROR c.e.x.e.impl.ClientActivator - Cluster health in state 'RED'