The issue of missing VMs in Hyper-V can typically be resolved by checking configuration settings, updating Hyper-V, or using management tools to refresh and register all VMs. https://techdoor.us/

If you're unable to take a backup in an Oracle KVM environment and the status is stuck in "Queued," it could be due to resource contention, network issues, or a problem with the backup software. Try restarting the backup process, checking system resources, ensuring there are no network disruptions, and verifying the backup configuration. If the issue persists, reviewing logs or contacting support for further diagnosis might be necessary.

Hello @jguilford81 Please go to the Virtual Environments -> Virtualization Providers -> click your vCenter -> Certificates tab. Then select "Trust all certificates" and save. Perform inventory synchronization again and check the backup.

Hello @branes What do your backup retention settings look like? With what parameters is NFS shared? Does the storware node have full access to this NFS resource? You are sure that nothing changed in the DataDomain settings in the meantime?

Hello @kirkscollision Sorry for the late response time, I hope you are still interested in our solution. I'm not sure what installation you're asking about. The error occurs while installing our software or XCP? Please provide more information and we will try to help.

@j-linda @krzysztof-sz attached is the log. [2024-09-23 08:50:08.718] INFO [pool-5-thread-89] InventorySyncTask.initHVSync:306 [d80c9435-2fc4-461b-9f90-e119bc4e7e8f] Started Inventory Sync Task of Hypervisor: 192.168.250.53 [2024-09-23 08:50:08.718] INFO [pool-5-thread-89] HypervApi.getAgentVersion:557 [d80c9435-2fc4-461b-9f90-e119bc4e7e8f] Getting Agent version [2024-09-23 08:50:08.724] INFO [pool-5-thread-89] HypervRequestSender.refreshAuthTokenIfExpired:70 [d80c9435-2fc4-461b-9f90-e119bc4e7e8f] Auth token expired, calling refresh(): 192.168.250.53 [2024-09-23 08:50:09.742] WARN [pool-5-thread-89] IVmSource.fetchApiVersion:138 [d80c9435-2fc4-461b-9f90-e119bc4e7e8f] Cannot fetch API version: Bad Request. Skipping. [2024-09-23 08:50:09.805] INFO [pool-5-thread-89] InventorySyncTask.sendInventory:603 [d80c9435-2fc4-461b-9f90-e119bc4e7e8f] Sending inventory with type INIT to Server [2024-09-23 08:50:09.826] INFO [pool-5-thread-89] InventorySyncTask.sendInventory:607 [d80c9435-2fc4-461b-9f90-e119bc4e7e8f] Finished step: INIT [2024-09-23 08:50:09.913] INFO [pool-5-thread-89] InventorySyncTask.sendInventory:603 [d80c9435-2fc4-461b-9f90-e119bc4e7e8f] Sending inventory with type DATACENTERS to Server [2024-09-23 08:50:09.953] INFO [pool-5-thread-89] InventorySyncTask.sendInventory:607 [d80c9435-2fc4-461b-9f90-e119bc4e7e8f] Finished step: DATACENTERS [2024-09-23 08:50:09.953] INFO [pool-5-thread-89] HypervApi.getCluster:303 [d80c9435-2fc4-461b-9f90-e119bc4e7e8f] Getting cluster information from host: https://192.168.250.53:50882 [2024-09-23 08:50:09.953] INFO [pool-5-thread-89] HypervRequestSender.refreshAuthTokenIfExpired:70 [d80c9435-2fc4-461b-9f90-e119bc4e7e8f] Auth token expired, calling refresh(): 192.168.250.53 [2024-09-23 08:50:10.855] ERROR [pool-5-thread-89] InventorySyncTask.lambda$run$0:118 [d80c9435-2fc4-461b-9f90-e119bc4e7e8f] Bad Request eu.storware.vprotect.engine.exception.ExternalAPIException: Bad Request at eu.storware.vprotect.engine.hypervisor.hyperv.HypervHypervisor.getHvCluster(HypervHypervisor.java:134) ~[engine.jar:?] at eu.storware.vprotect.engine.task.InventorySyncTask.getHypervisorCluster(InventorySyncTask.java:370) ~[engine.jar:?] at java.util.Optional.orElseGet(Optional.java:369) [?:?] at eu.storware.vprotect.engine.task.InventorySyncTask.getHypervisorClusters(InventorySyncTask.java:363) ~[engine.jar:?] at eu.storware.vprotect.engine.task.InventorySyncTask.syncAndUpdateProgress(InventorySyncTask.java:256) ~[engine.jar:?] at eu.storware.vprotect.engine.task.InventorySyncTask.lambda$updateProgress$1(InventorySyncTask.java:251) ~[engine.jar:?] at java.util.stream.ReduceOps$1ReducingSink.accept(ReduceOps.java:80) ~[?:?] at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655) ~[?:?] at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?] at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?] at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[?:?] at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?] at java.util.stream.ReferencePipeline.reduce(ReferencePipeline.java:563) ~[?:?] at eu.storware.vprotect.engine.task.InventorySyncTask.updateProgress(InventorySyncTask.java:251) ~[engine.jar:?] at eu.storware.vprotect.engine.task.InventorySyncTask.syncHypervisor(InventorySyncTask.java:198) ~[engine.jar:?] at eu.storware.vprotect.engine.task.InventorySyncTask.run(InventorySyncTask.java:105) [engine.jar:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] at java.lang.Thread.run(Thread.java:829) [?:?] Caused by: eu.storware.vprotect.engine.exception.ExternalAPIException: Bad Request at eu.storware.vprotect.engine.hypervisor.hyperv.HypervRequestSender.send(HypervRequestSender.java:50) ~[engine.jar:?] at eu.storware.vprotect.engine.hypervisor.hyperv.HypervApi.getCluster(HypervApi.java:304) ~[engine.jar:?] at eu.storware.vprotect.engine.hypervisor.hyperv.HypervHypervisor.getHvCluster(HypervHypervisor.java:124) ~[engine.jar:?] ... 18 more Caused by: eu.storware.vprotect.engine.exception.ExternalAPIException: Bad Request at eu.storware.vprotect.engine.hypervisor.hyperv.HypervRequestSender.getAuthenticationToken(HypervRequestSender.java:128) ~[engine.jar:?] at eu.storware.vprotect.engine.hypervisor.hyperv.HypervRequestSender.refreshAuthTokenIfExpired(HypervRequestSender.java:71) ~[engine.jar:?] at eu.storware.vprotect.engine.hypervisor.hyperv.HypervRequestSender.checkTokenAndCall(HypervRequestSender.java:64) ~[engine.jar:?] at eu.storware.vprotect.engine.hypervisor.hyperv.HypervRequestSender.send(HypervRequestSender.java:41) ~[engine.jar:?] at eu.storware.vprotect.engine.hypervisor.hyperv.HypervApi.getCluster(HypervApi.java:304) ~[engine.jar:?] at eu.storware.vprotect.engine.hypervisor.hyperv.HypervHypervisor.getHvCluster(HypervHypervisor.java:124) ~[engine.jar:?] ... 18 more Caused by: eu.vprotect.hypervclient.ApiException: Bad Request at eu.vprotect.hypervclient.ApiClient.handleResponse(ApiClient.java:926) ~[engine.jar:?] at eu.vprotect.hypervclient.ApiClient.execute(ApiClient.java:842) ~[engine.jar:?] at eu.storware.vprotect.engine.hypervisor.hyperv.HypervRequestSender.getAuthenticationToken(HypervRequestSender.java:122) ~[engine.jar:?] at eu.storware.vprotect.engine.hypervisor.hyperv.HypervRequestSender.refreshAuthTokenIfExpired(HypervRequestSender.java:71) ~[engine.jar:?] at eu.storware.vprotect.engine.hypervisor.hyperv.HypervRequestSender.checkTokenAndCall(HypervRequestSender.java:64) ~[engine.jar:?] at eu.storware.vprotect.engine.hypervisor.hyperv.HypervRequestSender.send(HypervRequestSender.java:41) ~[engine.jar:?] at eu.storware.vprotect.engine.hypervisor.hyperv.HypervApi.getCluster(HypervApi.java:304) ~[engine.jar:?] at eu.storware.vprotect.engine.hypervisor.hyperv.HypervHypervisor.getHvCluster(HypervHypervisor.java:124) ~[engine.jar:?] ... 18 more [2024-09-23 08:50:10.857] ERROR [pool-5-thread-89] ITask.finishTask:48 [d80c9435-2fc4-461b-9f90-e119bc4e7e8f] Synchronization task failed: eu.storware.vprotect.engine.exception.ExternalAPIException: Bad Request at eu.storware.vprotect.engine.hypervisor.hyperv.HypervHypervisor.getHvCluster(HypervHypervisor.java:134) ~[engine.jar:?] at eu.storware.vprotect.engine.task.InventorySyncTask.getHypervisorCluster(InventorySyncTask.java:370) ~[engine.jar:?] at java.util.Optional.orElseGet(Optional.java:369) ~[?:?] at eu.storware.vprotect.engine.task.InventorySyncTask.getHypervisorClusters(InventorySyncTask.java:363) ~[engine.jar:?] at eu.storware.vprotect.engine.task.InventorySyncTask.syncAndUpdateProgress(InventorySyncTask.java:256) ~[engine.jar:?] at eu.storware.vprotect.engine.task.InventorySyncTask.lambda$updateProgress$1(InventorySyncTask.java:251) ~[engine.jar:?] at java.util.stream.ReduceOps$1ReducingSink.accept(ReduceOps.java:80) ~[?:?] at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655) ~[?:?] at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?] at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?] at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[?:?] at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?] at java.util.stream.ReferencePipeline.reduce(ReferencePipeline.java:563) ~[?:?] at eu.storware.vprotect.engine.task.InventorySyncTask.updateProgress(InventorySyncTask.java:251) ~[engine.jar:?] at eu.storware.vprotect.engine.task.InventorySyncTask.syncHypervisor(InventorySyncTask.java:198) ~[engine.jar:?] at eu.storware.vprotect.engine.task.InventorySyncTask.run(InventorySyncTask.java:105) [engine.jar:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] at java.lang.Thread.run(Thread.java:829) [?:?] Caused by: eu.storware.vprotect.engine.exception.ExternalAPIException: Bad Request at eu.storware.vprotect.engine.hypervisor.hyperv.HypervRequestSender.send(HypervRequestSender.java:50) ~[engine.jar:?] at eu.storware.vprotect.engine.hypervisor.hyperv.HypervApi.getCluster(HypervApi.java:304) ~[engine.jar:?] at eu.storware.vprotect.engine.hypervisor.hyperv.HypervHypervisor.getHvCluster(HypervHypervisor.java:124) ~[engine.jar:?] ... 18 more Caused by: eu.storware.vprotect.engine.exception.ExternalAPIException: Bad Request at eu.storware.vprotect.engine.hypervisor.hyperv.HypervRequestSender.getAuthenticationToken(HypervRequestSender.java:128) ~[engine.jar:?] at eu.storware.vprotect.engine.hypervisor.hyperv.HypervRequestSender.refreshAuthTokenIfExpired(HypervRequestSender.java:71) ~[engine.jar:?] at eu.storware.vprotect.engine.hypervisor.hyperv.HypervRequestSender.checkTokenAndCall(HypervRequestSender.java:64) ~[engine.jar:?] at eu.storware.vprotect.engine.hypervisor.hyperv.HypervRequestSender.send(HypervRequestSender.java:41) ~[engine.jar:?] at eu.storware.vprotect.engine.hypervisor.hyperv.HypervApi.getCluster(HypervApi.java:304) ~[engine.jar:?] at eu.storware.vprotect.engine.hypervisor.hyperv.HypervHypervisor.getHvCluster(HypervHypervisor.java:124) ~[engine.jar:?] ... 18 more Caused by: eu.vprotect.hypervclient.ApiException: Bad Request at eu.vprotect.hypervclient.ApiClient.handleResponse(ApiClient.java:926) ~[engine.jar:?] at eu.vprotect.hypervclient.ApiClient.execute(ApiClient.java:842) ~[engine.jar:?] at eu.storware.vprotect.engine.hypervisor.hyperv.HypervRequestSender.getAuthenticationToken(HypervRequestSender.java:122) ~[engine.jar:?] at eu.storware.vprotect.engine.hypervisor.hyperv.HypervRequestSender.refreshAuthTokenIfExpired(HypervRequestSender.java:71) ~[engine.jar:?] at eu.storware.vprotect.engine.hypervisor.hyperv.HypervRequestSender.checkTokenAndCall(HypervRequestSender.java:64) ~[engine.jar:?] at eu.storware.vprotect.engine.hypervisor.hyperv.HypervRequestSender.send(HypervRequestSender.java:41) ~[engine.jar:?] at eu.storware.vprotect.engine.hypervisor.hyperv.HypervApi.getCluster(HypervApi.java:304) ~[engine.jar:?] at eu.storware.vprotect.engine.hypervisor.hyperv.HypervHypervisor.getHvCluster(HypervHypervisor.java:124) ~[engine.jar:?] ... 18 more [2024-09-23 08:50:10.891] INFO [pool-5-thread-89] ITask.lambda$finishTask$2:72 [d80c9435-2fc4-461b-9f90-e119bc4e7e8f] Finishing d80c9435-2fc4-461b-9f90-e119bc4e7e8f: [Inventory synchronization] from 2024-09-23T08:49:55+02:00 to 2024-09-23T09:49:55+02:00 Task with message "Synchronization task failed: ExternalAPIException: Bad Request" [2024-09-23 08:50:10.989] INFO [pool-5-thread-89] InventorySyncTask.sendInventory:603 [d80c9435-2fc4-461b-9f90-e119bc4e7e8f] Sending inventory with type FINALIZE to Server [2024-09-23 08:50:11.017] INFO [pool-5-thread-89] InventorySyncTask.sendInventory:607 [d80c9435-2fc4-461b-9f90-e119bc4e7e8f] Finished step: FINALIZE

@krzysztof-sz Yes, I tried that, didn't work. Could do 1 backup, the next one failed. I think I'll just reinstall it all.

@a-spinek Thank you, that works!

@djaw Yes it is a stand alone host

Hi @mario-krizan, Also, is it possible to migrate a virtual machine from OVM to OLVM with Stormware? Unfortunately, we do not support this type of migration. Failed to collect file system information: [Command execution failed: [/usr/bin/virt-filesystems, --filesystems, -l, --csv, -a, /vprotect_data/abc_fmo_pool_01/A22000A0__0004fb00000600007ee414f01ab0c9f5/0004fb00001200002bb2cfe407f43a53.img] Return code: 1 output: [] error: [libguestfs: error: force_kvm supplied but kvm not available. ]] Please provide a log with DEBUG mode enabled. Instructions how to do this: https://storware.gitbook.io/backup-and-recovery/troubleshooting/how-switch-vprotect-to-debug-mode

Hello! Sorry for late response. Please tell me at first, did you tried to update to version 6.2.0 and then restore backup? If problem still occurs we need to look into log f backup and restore. Before you generate it please follow this guide: DEBUG MODE Regards Krzysztof

@carl-ceriola Hi, what backup method do you use?

@vguillard Hi, no Storware is not capable of backuping/restoring user-data yet.

Scratch that, was able to modify some S3 settings and it now shows as a destination. Now my issue is that all backup jobs including internal db show queued and not doing anything.

Hello @victors, This new limitation was introduced in the 6.2 version of Storware Backup & Recovery. In this strategy, you can make only full backups of OpenStack instances because every time a snapshot is kept on the VM, the disk type changes to QCOW2. Incremental backups are only available for CEPH volumes in SSH/libvirt strategy, https://storware.gitbook.io/backup-and-recovery/changelog

Hi! I am sorry for delay, but could you tell me more about your environment? Where is server and node (inside/outside VMWare)? Regards