Recovery blob storage


#1

Enonic version: Enonic-xp-6.15.5
OS: Linux (ubuntu 18.04)

Recently, the server where my Enonic app where running, have faced a space error, so I decided to restart the machine, and then start the application. While the application was starting, the command prompt showed this error.

020-06-03 18:28:36,194 ERROR com.enonic.xp.core.app - [com.enonic.xp.core.impl.app.ApplicationServiceImpl(26)] Failed creating the component instance; see log for reason
2020-06-03 18:28:36,195 ERROR com.enonic.xp.core.app - [com.enonic.xp.core.impl.app.event.ApplicationInvalidatorListener(32)] Cannot create component instance due to failure to bind reference ApplicationService
2020-06-03 18:28:36,195 ERROR com.enonic.xp.core.app - [com.enonic.xp.core.impl.app.event.ApplicationInvalidatorListener(32)] Failed creating the component instance; see log for reason
2020-06-03 18:28:45,600 WARN org.elasticsearch.index.engine - [7eafd0f3-2473-4f77-86ea-6f70dbbc3e26] [search-cms-repo][0] failed to sync translog
2020-06-03 18:28:45,607 WARN org.elasticsearch.indices.cluster - [7eafd0f3-2473-4f77-86ea-6f70dbbc3e26] [[search-cms-repo][0]] marking and sending shard failed due to [failed recovery]
org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException: [search-cms-repo][0] failed to recover shard
at org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:290)
at org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:112)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

The problem with this issue is that information that was in that application is highly important and it needs to be recovered.

So considering that, is that any way to recover that information after that error?

In the attachment is the full error showed in the command prompt.



I hope to get any comments about this. Thanks in advance.


#2

Hi soldier!

Just a little clarification first. The blob storage is probably fine. This is a problem with the indexes. The indexes are essentially there to tell XP where to find files in the blob storage. Now if the indexes are lost, you will have a hard time finding your data in the blob storage.

I hope you have been running the snapshotter app in the cluster and you have backups of both the blob store and the snapshots created by the snappshotter. I also hope you have replicas set to the appropriate value for your cluster size.

Is this a single node deployment or clustered?