Thanks for the information. I'll try to recreate backup storage. But as I thought the NFS 4.2 is also supported as destination for synthetic backups.
I have mounted share:
172.16.16.13:/mnt/seagate_4tb/storware01 on /mnt/srv04/storware01 type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=172.16.16.104,local_lock=none,addr=172.16.16.13)
May be you can tell something about XCP-ng "export strategy" error? Thanks.
I have tried to get incremental backup for ESXi host. Full backup finished successfully. But incremental failed with error:
Store task failed: SyntheticBackupException: Command execution failed: [cp, -v, --reflink=always, /mnt/srv04/storware01/build-agent01__421ba2e0/mnt/srv04/storware01/vprotect_data/build-agent01__421ba2e0/build-agent01__Hard_disk_1-flat.vmdk/1701634963404, /mnt/srv04/storware01/build-agent01__421ba2e0/mnt/srv04/storware01/vprotect_data/build-agent01__421ba2e0/build-agent01__Hard_disk_1-flat.vmdk/1701720032667] Return code: 1 output: ['/mnt/srv04/storware01/build-agent01__421ba2e0/mnt/srv04/storware01/vprotect_data/build-agent01__421ba2e0/build-agent01__Hard_disk_1-flat.vmdk/1701634963404' -> '/mnt/srv04/storware01/build-agent01__421ba2e0/mnt/srv04/storware01/vprotect_data/build-agent01__421ba2e0/build-agent01__Hard_disk_1-flat.vmdk/1701720032667' ] error: [cp: failed to clone '/mnt/srv04/storware01/build-agent01__421ba2e0/mnt/srv04/storware01/vprotect_data/build-agent01__421ba2e0/build-agent01__Hard_disk_1-flat.vmdk/1701720032667' from '/mnt/srv04/storware01/build-agent01__421ba2e0/mnt/srv04/storware01/vprotect_data/build-agent01__421ba2e0/build-agent01__Hard_disk_1-flat.vmdk/1701634963404': Operation not supported ]. Command execution failed: [cp, -v, --reflink=always, /mnt/srv04/storware01/build-agent01__421ba2e0/mnt/srv04/storware01/vprotect_data/build-agent01__421ba2e0/build-agent01__Hard_disk_1-flat.vmdk/1701634963404, /mnt/srv04/storware01/build-agent01__421ba2e0/mnt/srv04/storware01/vprotect_data/build-agent01__421ba2e0/build-agent01__Hard_disk_1-flat.vmdk/1701720032667] Return code: 1 output: ['/mnt/srv04/storware01/build-agent01__421ba2e0/mnt/srv04/storware01/vprotect_data/build-agent01__421ba2e0/build-agent01__Hard_disk_1-flat.vmdk/1701634963404' -> '/mnt/srv04/storware01/build-agent01__421ba2e0/mnt/srv04/storware01/vprotect_data/build-agent01__421ba2e0/build-agent01__Hard_disk_1-flat.vmdk/1701720032667' ] error: [cp: failed to clone '/mnt/srv04/storware01/build-agent01__421ba2e0/mnt/srv04/storware01/vprotect_data/build-agent01__421ba2e0/build-agent01__Hard_disk_1-flat.vmdk/1701720032667' from '/mnt/srv04/storware01/build-agent01__421ba2e0/mnt/srv04/storware01/vprotect_data/build-agent01__421ba2e0/build-agent01__Hard_disk_1-flat.vmdk/1701634963404': Operation not supported ]
P.S.: what "SyntheticBackupException: This export strategy doesn't support Synthetic backups" means for XCP-ng incremental backup? Full backup is working.
Today all hosts are gone from the inventory with the same error. Why can't I attach files in thread? This is a part of the log.
eu.storware.vprotect.engine.api.APIErrorException: Server has encountered the runtime error (RUNTIME_ERROR)
at eu.storware.vprotect.engine.api.controller.TaskController.sendInventory(TaskController.java:452) ~[engine.jar:?]
at eu.storware.vprotect.engine.task.InventorySyncTask.lambda$sendInventory$21(InventorySyncTask.java:623) ~[engine.jar:?]
at dev.failsafe.Functions.lambda$toCtxSupplier$9(Functions.java:228) ~[engine.jar:?]
at dev.failsafe.Functions.lambda$get$0(Functions.java:46) ~[engine.jar:?]
at dev.failsafe.internal.RetryPolicyExecutor.lambda$apply$0(RetryPolicyExecutor.java:74) ~[engine.jar:?]
at dev.failsafe.SyncExecutionImpl.executeSync(SyncExecutionImpl.java:187) ~[engine.jar:?]
at dev.failsafe.FailsafeExecutor.call(FailsafeExecutor.java:376) ~[engine.jar:?]
at dev.failsafe.FailsafeExecutor.run(FailsafeExecutor.java:220) ~[engine.jar:?]
at eu.storware.vprotect.engine.task.InventorySyncTask.sendInventory(InventorySyncTask.java:623) ~[engine.jar:?]
at eu.storware.vprotect.engine.task.InventorySyncTask.syncAndUpdateProgress(InventorySyncTask.java:266) ~[engine.jar:?]
at eu.storware.vprotect.engine.task.InventorySyncTask.lambda$updateProgress$1(InventorySyncTask.java:254) ~[engine.jar:?]
at java.util.stream.ReduceOps$1ReducingSink.accept(ReduceOps.java:80) [?:?]
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655) [?:?]
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) [?:?]
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) [?:?]
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) [?:?]
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) [?:?]
at java.util.stream.ReferencePipeline.reduce(ReferencePipeline.java:563) [?:?]
at eu.storware.vprotect.engine.task.InventorySyncTask.updateProgress(InventorySyncTask.java:254) [engine.jar:?]
at eu.storware.vprotect.engine.task.InventorySyncTask.syncHypervisor(InventorySyncTask.java:201) [engine.jar:?]
at eu.storware.vprotect.engine.task.InventorySyncTask.run(InventorySyncTask.java:108) [engine.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
Caused by: io.swagger.client.ApiException: {"errorCode":"RUNTIME_ERROR","errors":[],"message":"Server has encountered the runtime error"}
at io.swagger.client.ApiClient.invokeAPI(ApiClient.java:706) ~[engine.jar:?]
at io.swagger.client.api.TaskControllerApi.updateInventory(TaskControllerApi.java:1769) ~[engine.jar:?]
at eu.storware.vprotect.engine.api.controller.TaskController.sendInventory(TaskController.java:450) ~[engine.jar:?]
... 23 more
[2023-12-03 08:10:43.528] ERROR [pool-5-thread-5] InventorySyncTask.lambda$getRetryPolicy$24:699
[3655a122-fbfb-4a7f-ba52-595a0de48f5f] Attempt number 6 failed.
eu.storware.vprotect.engine.api.APIErrorException: Server has encountered the runtime error (RUNTIME_ERROR)
at eu.storware.vprotect.engine.api.controller.TaskController.sendInventory(TaskController.java:452) ~[engine.jar:?]
at eu.storware.vprotect.engine.task.InventorySyncTask.lambda$sendInventory$21(InventorySyncTask.java:623) ~[engine.jar:?]
at dev.failsafe.Functions.lambda$toCtxSupplier$9(Functions.java:228) ~[engine.jar:?]
at dev.failsafe.Functions.lambda$get$0(Functions.java:46) ~[engine.jar:?]
at dev.failsafe.internal.RetryPolicyExecutor.lambda$apply$0(RetryPolicyExecutor.java:74) ~[engine.jar:?]
at dev.failsafe.SyncExecutionImpl.executeSync(SyncExecutionImpl.java:187) ~[engine.jar:?]
at dev.failsafe.FailsafeExecutor.call(FailsafeExecutor.java:376) ~[engine.jar:?]
at dev.failsafe.FailsafeExecutor.run(FailsafeExecutor.java:220) ~[engine.jar:?]
at eu.storware.vprotect.engine.task.InventorySyncTask.sendInventory(InventorySyncTask.java:623) ~[engine.jar:?]
at eu.storware.vprotect.engine.task.InventorySyncTask.syncAndUpdateProgress(InventorySyncTask.java:266) ~[engine.jar:?]
at eu.storware.vprotect.engine.task.InventorySyncTask.lambda$updateProgress$1(InventorySyncTask.java:254) ~[engine.jar:?]
at java.util.stream.ReduceOps$1ReducingSink.accept(ReduceOps.java:80) [?:?]
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655) [?:?]
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) [?:?]
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) [?:?]
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) [?:?]
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) [?:?]
at java.util.stream.ReferencePipeline.reduce(ReferencePipeline.java:563) [?:?]
at eu.storware.vprotect.engine.task.InventorySyncTask.updateProgress(InventorySyncTask.java:254) [engine.jar:?]
at eu.storware.vprotect.engine.task.InventorySyncTask.syncHypervisor(InventorySyncTask.java:201) [engine.jar:?]
at eu.storware.vprotect.engine.task.InventorySyncTask.run(InventorySyncTask.java:108) [engine.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
Caused by: io.swagger.client.ApiException: {"errorCode":"RUNTIME_ERROR","errors":[],"message":"Server has encountered the runtime error"}
at io.swagger.client.ApiClient.invokeAPI(ApiClient.java:706) ~[engine.jar:?]
at io.swagger.client.api.TaskControllerApi.updateInventory(TaskControllerApi.java:1769) ~[engine.jar:?]
at eu.storware.vprotect.engine.api.controller.TaskController.sendInventory(TaskController.java:450) ~[engine.jar:?]
... 23 more
[2023-12-03 08:10:43.529] ERROR [pool-5-thread-5] InventorySyncTask.lambda$getRetryPolicy$27:702
[3655a122-fbfb-4a7f-ba52-595a0de48f5f] Failed to send VMS, max retries exceeded
[2023-12-03 08:10:43.530] ERROR [pool-5-thread-5] InventorySyncTask.lambda$run$0:121
[3655a122-fbfb-4a7f-ba52-595a0de48f5f] {"errorCode":"RUNTIME_ERROR","errors":[],"message":"Server has encountered the runtime error"}
eu.storware.vprotect.engine.api.APIErrorException: Server has encountered the runtime error (RUNTIME_ERROR)
at eu.storware.vprotect.engine.api.controller.TaskController.sendInventory(TaskController.java:452) ~[engine.jar:?]
at eu.storware.vprotect.engine.task.InventorySyncTask.lambda$sendInventory$21(InventorySyncTask.java:623) ~[engine.jar:?]
at dev.failsafe.Functions.lambda$toCtxSupplier$9(Functions.java:228) ~[engine.jar:?]
at dev.failsafe.Functions.lambda$get$0(Functions.java:46) ~[engine.jar:?]
at dev.failsafe.internal.RetryPolicyExecutor.lambda$apply$0(RetryPolicyExecutor.java:74) ~[engine.jar:?]
at dev.failsafe.SyncExecutionImpl.executeSync(SyncExecutionImpl.java:187) ~[engine.jar:?]
at dev.failsafe.FailsafeExecutor.call(FailsafeExecutor.java:376) ~[engine.jar:?]
at dev.failsafe.FailsafeExecutor.run(FailsafeExecutor.java:220) ~[engine.jar:?]
at eu.storware.vprotect.engine.task.InventorySyncTask.sendInventory(InventorySyncTask.java:623) ~[engine.jar:?]
at eu.storware.vprotect.engine.task.InventorySyncTask.syncAndUpdateProgress(InventorySyncTask.java:266) ~[engine.jar:?]
at eu.storware.vprotect.engine.task.InventorySyncTask.lambda$updateProgress$1(InventorySyncTask.java:254) ~[engine.jar:?]
at java.util.stream.ReduceOps$1ReducingSink.accept(ReduceOps.java:80) ~[?:?]
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655) ~[?:?]
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?]
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[?:?]
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
at java.util.stream.ReferencePipeline.reduce(ReferencePipeline.java:563) ~[?:?]
at eu.storware.vprotect.engine.task.InventorySyncTask.updateProgress(InventorySyncTask.java:254) ~[engine.jar:?]
at eu.storware.vprotect.engine.task.InventorySyncTask.syncHypervisor(InventorySyncTask.java:201) ~[engine.jar:?]
at eu.storware.vprotect.engine.task.InventorySyncTask.run(InventorySyncTask.java:108) [engine.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
Caused by: io.swagger.client.ApiException: {"errorCode":"RUNTIME_ERROR","errors":[],"message":"Server has encountered the runtime error"}
at io.swagger.client.ApiClient.invokeAPI(ApiClient.java:706) ~[engine.jar:?]
at io.swagger.client.api.TaskControllerApi.updateInventory(TaskControllerApi.java:1769) ~[engine.jar:?]
at eu.storware.vprotect.engine.api.controller.TaskController.sendInventory(TaskController.java:450) ~[engine.jar:?]
... 23 more
[2023-12-03 08:10:43.531] ERROR [pool-5-thread-5] ITask.finishTask:47
[3655a122-fbfb-4a7f-ba52-595a0de48f5f] Synchronization task failed:
eu.storware.vprotect.engine.api.APIErrorException: Server has encountered the runtime error (RUNTIME_ERROR)
at eu.storware.vprotect.engine.api.controller.TaskController.sendInventory(TaskController.java:452) ~[engine.jar:?]
at eu.storware.vprotect.engine.task.InventorySyncTask.lambda$sendInventory$21(InventorySyncTask.java:623) ~[engine.jar:?]
at dev.failsafe.Functions.lambda$toCtxSupplier$9(Functions.java:228) ~[engine.jar:?]
at dev.failsafe.Functions.lambda$get$0(Functions.java:46) ~[engine.jar:?]
at dev.failsafe.internal.RetryPolicyExecutor.lambda$apply$0(RetryPolicyExecutor.java:74) ~[engine.jar:?]
at dev.failsafe.SyncExecutionImpl.executeSync(SyncExecutionImpl.java:187) ~[engine.jar:?]
at dev.failsafe.FailsafeExecutor.call(FailsafeExecutor.java:376) ~[engine.jar:?]
at dev.failsafe.FailsafeExecutor.run(FailsafeExecutor.java:220) ~[engine.jar:?]
at eu.storware.vprotect.engine.task.InventorySyncTask.sendInventory(InventorySyncTask.java:623) ~[engine.jar:?]
at eu.storware.vprotect.engine.task.InventorySyncTask.syncAndUpdateProgress(InventorySyncTask.java:266) ~[engine.jar:?]
at eu.storware.vprotect.engine.task.InventorySyncTask.lambda$updateProgress$1(InventorySyncTask.java:254) ~[engine.jar:?]
at java.util.stream.ReduceOps$1ReducingSink.accept(ReduceOps.java:80) ~[?:?]
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655) ~[?:?]
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?]
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[?:?]
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
at java.util.stream.ReferencePipeline.reduce(ReferencePipeline.java:563) ~[?:?]
at eu.storware.vprotect.engine.task.InventorySyncTask.updateProgress(InventorySyncTask.java:254) ~[engine.jar:?]
at eu.storware.vprotect.engine.task.InventorySyncTask.syncHypervisor(InventorySyncTask.java:201) ~[engine.jar:?]
at eu.storware.vprotect.engine.task.InventorySyncTask.run(InventorySyncTask.java:108) [engine.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
Caused by: io.swagger.client.ApiException: {"errorCode":"RUNTIME_ERROR","errors":[],"message":"Server has encountered the runtime error"}
at io.swagger.client.ApiClient.invokeAPI(ApiClient.java:706) ~[engine.jar:?]
at io.swagger.client.api.TaskControllerApi.updateInventory(TaskControllerApi.java:1769) ~[engine.jar:?]
at eu.storware.vprotect.engine.api.controller.TaskController.sendInventory(TaskController.java:450) ~[engine.jar:?]
... 23 more
[2023-12-03 08:10:43.555] INFO [pool-5-thread-5] ITask.lambda$finishTask$2:71
[3655a122-fbfb-4a7f-ba52-595a0de48f5f] Finishing 3655a122-fbfb-4a7f-ba52-595a0de48f5f: [Inventory synchronization] from 2023-12-03T08:07:46+01:00 to 2023-12-03T09:07:46+01:00 Task with message "Synchronization task failed: Server has encountered the runtime error (RUNTIME_ERROR)"
[2023-12-03 08:10:43.606] INFO [pool-5-thread-5] InventorySyncTask.sendInventory:620
[3655a122-fbfb-4a7f-ba52-595a0de48f5f] Sending inventory with type FINALIZE to Server
[2023-12-03 08:10:43.624] INFO [pool-5-thread-5] InventorySyncTask.sendInventory:624
[3655a122-fbfb-4a7f-ba52-595a0de48f5f] Finished step: FINALIZE
Hi. Thanks for information.
Anyway the first mentioned error is not belonged to XCP-ng limitations. OK. I'll watching for it.
Hi.
I'm testing the deployed OVA storware instance.
I have 2 ESXi hosts and 1 XCP-ng host. Backup storage is mounted as NFS share. The staging folder is placed on that share too.
I've successfully backed up windows VM from vmware host. Then I tried to backup ubuntu VM from xcp-ng host. Because of long backup time I left backup task to proceed and came back to it in the morning. I saw the error "Store task failed: Server has encountered the runtime error (RUNTIME_ERROR)". All my hosts are disappeared from the inventory and I couldn't rescan them again. I see the XVA image of the ubuntu VM in the staging folder. After vprotect service reload my hosts returned to the inventory. But there are no any signs of the failed backup in the "Instances". I've tried to backup again and now everything looks fine except I can't mount backuped vm with error "Cannot mount backup made by hypervisor with VM image strategy enabled".