Hello
I tried to test incremental backup on openstack with libvirt strategy . However I receive following error
[b7277ddb-a4d1-4284-8783-30ba407d38bd] Unable to create incremental backup for a non-excluded QCOW2 driver [UUID: 4afff436-71cc-4a30-8ca5-a9406e6350aa]
This is the log for first full backup
Task Assignment Utility: EXPORT task 162d6b39-42fa-418b-af16-41777176a8c0 has been assigned to the node node1 (206e4177-b07e-413d-ac71-2287ac5df8f5) [305/1825]
==> node1/vprotect_daemon.log <==
[2024-03-22 15:07:03.747] INFO [pool-5-thread-56] ITask.lambda$startTask$1:39
[] Starting Task 162d6b39-42fa-418b-af16-41777176a8c0: [Export] from 2024-03-22T15:06+01:00 to 2024-03-22T20:06+01:00
[2024-03-22 15:07:03.781] INFO [pool-5-thread-56] ExportTask.run:115
[162d6b39-42fa-418b-af16-41777176a8c0] Starting export of: VM cirros15 (551df74b-dd5a-4b64-b5e2-0fe523d47da3) from 10.150.6.73 (OpenStack)...
[2024-03-22 15:07:03.781] INFO [pool-5-thread-56] BackupProviderUtil.testConnection:238
[162d6b39-42fa-418b-af16-41777176a8c0] Testing connection with backup provider: backupdest1
[2024-03-22 15:07:03.782] INFO [pool-5-thread-56] FileUtil.removeLocalTestFiles:645
[162d6b39-42fa-418b-af16-41777176a8c0] Cleaning up after Connectivity Test
[2024-03-22 15:07:03.800] INFO [pool-5-thread-56] ExportImportUtils.getVmExportImportModeByHvManager:43
[162d6b39-42fa-418b-af16-41777176a8c0] OpenStack export import strategy by hypervisor manager: SSH_TRANSFER
[2024-03-22 15:07:03.812] INFO [pool-5-thread-56] OpenStackClientFactory.authenticateScopedToProjectName:140
[162d6b39-42fa-418b-af16-41777176a8c0] Authentication scoped by project name - url: 'http://10.150.6.73:5000/v3' | region: 'RegionOne' | domain: 'default' | project: 'admin' | user: 'admin' | execution source: 'SshTransferOpenStackImportExportStrategy.java#<init>:81'
[2024-03-22 15:07:04.278] INFO [pool-5-thread-56] SshFingerprintCache.getOrCreateTrustedCertificateWithDecodedChain:55
[162d6b39-42fa-418b-af16-41777176a8c0] Getting ssh fingerprint for TrustedCertificateChainId(sourceId=4d54f698-c559-4261-8e5c-46b23ea99bf7, sourceType=HYPERVISOR, chainType=SSH_HOST_FINGERPRINT) and task: 162d6b39-42fa-418b-af16-41777176a8c0: [Export] from 2024-03-22T15:
06+01:00 to 2024-03-22T20:06+01:00
[2024-03-22 15:07:04.283] INFO [pool-5-thread-56] SshUtil.getConfig:17
[162d6b39-42fa-418b-af16-41777176a8c0] StrictHostKeyChecking = yes
[2024-03-22 15:07:04.283] INFO [pool-5-thread-56] SshUtil.getConfig:17
[162d6b39-42fa-418b-af16-41777176a8c0] StrictHostKeyChecking = yes
[2024-03-22 15:07:05.051] INFO [pool-5-thread-56] OpenStackClientFactory.authenticateScopedToProjectName:140
[162d6b39-42fa-418b-af16-41777176a8c0] Authentication scoped by project name - url: 'http://10.150.6.73:5000/v3' | region: 'RegionOne' | domain: 'default' | project: 'admin' | user: 'admin' | execution source: 'OpenStackHypervisorManager.java#listVmDisks:204'
[2024-03-22 15:07:05.586] INFO [pool-5-thread-56] ExportFull.export:30
[162d6b39-42fa-418b-af16-41777176a8c0] Performing full backup
[2024-03-22 15:07:06.280] WARN [pool-5-thread-56] SshConnector.chownRemote:424
[162d6b39-42fa-418b-af16-41777176a8c0] Could not chown file /var/lib/nova/mnt/404d50d99406bdb14eefa7d751782ec1/volume-4afff436-71cc-4a30-8ca5-a9406e6350aa
[2024-03-22 15:07:07.372] INFO [pool-5-thread-56] SshConnector.chownSnapshots:398
[162d6b39-42fa-418b-af16-41777176a8c0] Trying chown snapshots
[2024-03-22 15:07:07.661] WARN [pool-5-thread-56] SshConnector.chownRemote:424
[162d6b39-42fa-418b-af16-41777176a8c0] Could not chown file /var/lib/nova/mnt/404d50d99406bdb14eefa7d751782ec1/volume-4afff436-71cc-4a30-8ca5-a9406e6350aa-vProtect-c08f07b0-2373-48c7-b212-8d57a755b15d
[2024-03-22 15:07:08.240] INFO [pool-5-thread-56] OpenStackHypervisorManager.exportImage:291
[162d6b39-42fa-418b-af16-41777176a8c0] Start exporting image
[2024-03-22 15:07:08.240] INFO [pool-5-thread-56] OpenStackImageApi.fetchAsVmDisks:45
[162d6b39-42fa-418b-af16-41777176a8c0] Fetching images as vm disk in server id: 551df74b-dd5a-4b64-b5e2-0fe523d47da3
[2024-03-22 15:07:08.248] INFO [pool-5-thread-56] ImageExporter.export:32
[162d6b39-42fa-418b-af16-41777176a8c0] isDownloadImageFromGlance : false
[2024-03-22 15:07:08.249] INFO [pool-5-thread-56] OpenStackExcludedImageExporter.export:23
[162d6b39-42fa-418b-af16-41777176a8c0] Image excluded from backup - skipping export
[2024-03-22 15:07:08.632] INFO [pool-5-thread-56] LibvirtHypervisor.exportDisks:576
[162d6b39-42fa-418b-af16-41777176a8c0] Disks to export: [file/disk: /var/lib/nova/mnt/404d50d99406bdb14eefa7d751782ec1/volume-4afff436-71cc-4a30-8ca5-a9406e6350aa]
[2024-03-22 15:07:09.364] INFO [pool-5-thread-56] LibvirtHypervisor.prepareSnapDownload:1256
[162d6b39-42fa-418b-af16-41777176a8c0] Adding snapshot file to the download list: /var/lib/nova/mnt/404d50d99406bdb14eefa7d751782ec1/volume-4afff436-71cc-4a30-8ca5-a9406e6350aa
[2024-03-22 15:07:09.711] INFO [pool-5-thread-56] LibvirtHypervisor.downloadFromTransferList:666
[162d6b39-42fa-418b-af16-41777176a8c0] Downloading VM data: cirros15
[2024-03-22 15:07:10.017] INFO [pool-5-thread-56] SshConnector.transfer:429
[162d6b39-42fa-418b-af16-41777176a8c0] Starting transfer
[2024-03-22 15:07:10.295] INFO [pool-5-thread-56] SshConnector.transfer:453
[162d6b39-42fa-418b-af16-41777176a8c0] Downloading: /var/lib/nova/mnt/404d50d99406bdb14eefa7d751782ec1/volume-4afff436-71cc-4a30-8ca5-a9406e6350aa -> /vprotect_data/cirros15__551df74b/volume-4afff436-71cc-4a30-8ca5-a9406e6350aa
[2024-03-22 15:07:17.472] INFO [pool-5-thread-56] ProtectedEntityUtil.collectFileSystemInfo:198
[162d6b39-42fa-418b-af16-41777176a8c0] Collecting file system information...
[2024-03-22 15:07:35.538] INFO [pool-5-thread-56] ITask.lambda$finishTask$2:71
[162d6b39-42fa-418b-af16-41777176a8c0] Finishing 162d6b39-42fa-418b-af16-41777176a8c0: [Export] from 2024-03-22T15:06+01:00 to 2024-03-22T20:06+01:00 Task with message "Backup of Virtual Machine: cirros15 (size: 1 GiB) exported in: 31s."
==> api/api.log <==
[2024-03-22 15:07:40] INFO [__ejb-thread-pool14] {} TaskLoadBalanceTimer.assignTasks:263
Task Assignment Utility: STORE task fa94c0fe-5e62-45bf-b4b3-ecb22cfc7d8c has been assigned to the node node1 (206e4177-b07e-413d-ac71-2287ac5df8f5)
==> node1/vprotect_daemon.log <==
[2024-03-22 15:07:43.784] INFO [pool-5-thread-56] ITask.lambda$startTask$0:38
[162d6b39-42fa-418b-af16-41777176a8c0] Starting Task fa94c0fe-5e62-45bf-b4b3-ecb22cfc7d8c: [Store] from 2024-03-22T15:07:35+01:00 to 2024-03-22T20:07:35+01:00 as a result of 162d6b39-42fa-418b-af16-41777176a8c0: [Export] from 2024-03-22T15:06+01:00 to 2024-03-22T20:06+01
:00
[2024-03-22 15:07:43.834] INFO [pool-5-thread-56] StoreTask.run:80
[fa94c0fe-5e62-45bf-b4b3-ecb22cfc7d8c] Files to store: [3a1b0f66-c630-4970-8cc7-a80a3c8f6282: /vprotect_data/cirros15__551df74b/18ea56d6-f5d0-4132-ad6b-e35aeed97890.img - null (modified: null | format: EXCLUDED_DISK | vmDiskGuid: 5175b419-687f-455e-b462-27c17c5ccfa3), 02
fe3a0a-e94f-4aa7-8be9-d31a52fd0a97: /vprotect_data/cirros15__551df74b/4afff436-71cc-4a30-8ca5-a9406e6350aa.json - null (modified: null | format: DISK_META_JSON | vmDiskGuid: 694e11e6-6044-43ce-a5bf-a85d3b55af12), a4d14c95-53ab-4b09-b836-eabc16331882: /vprotect_data/cirro
s15__551df74b/551df74b-dd5a-4b64-b5e2-0fe523d47da3.json - null (modified: null | format: VM_META_OPENSTACK | vmDiskGuid: null), 849e3f39-57c9-4d04-bb62-a9041915c583: /vprotect_data/cirros15__551df74b/volume-4afff436-71cc-4a30-8ca5-a9406e6350aa - null (modified: null | fo
rmat: RAW | vmDiskGuid: 694e11e6-6044-43ce-a5bf-a85d3b55af12)]
[2024-03-22 15:07:43.834] INFO [pool-5-thread-56] StoreTask.run:84
[fa94c0fe-5e62-45bf-b4b3-ecb22cfc7d8c] Storing backup files (Virtual Machine: cirros15) in File System...
[2024-03-22 15:07:43.927] INFO [pool-5-thread-56] BackupProvider.checkIfFileCanBeCopiedOrMoved:271
[fa94c0fe-5e62-45bf-b4b3-ecb22cfc7d8c] Files will be copied or moved - they will be stored on the same file system
[2024-03-22 15:07:44.177] INFO [pool-5-thread-56] ExportImportUtils.getVmExportImportMode:29
[fa94c0fe-5e62-45bf-b4b3-ecb22cfc7d8c] Export import strategy by backup: SSH_TRANSFER
[2024-03-22 15:07:44.190] INFO [pool-5-thread-56] OpenStackClientFactory.authenticateScopedToProjectName:140
[fa94c0fe-5e62-45bf-b4b3-ecb22cfc7d8c] Authentication scoped by project name - url: 'http://10.150.6.73:5000/v3' | region: 'RegionOne' | domain: 'default' | project: 'admin' | user: 'admin' | execution source: 'SshTransferOpenStackImportExportStrategy.java#<init>:81'
[2024-03-22 15:07:44.736] INFO [pool-5-thread-56] StoreTask.generateTimeStatsForStoreTask:228
[fa94c0fe-5e62-45bf-b4b3-ecb22cfc7d8c] Backup of Virtual Machine: cirros15 (size: 1 GiB) finished. Stored in: 0s, total time: 41s.
[2024-03-22 15:07:44.778] INFO [pool-5-thread-56] ITask.lambda$finishTask$2:71
[fa94c0fe-5e62-45bf-b4b3-ecb22cfc7d8c] Finishing fa94c0fe-5e62-45bf-b4b3-ecb22cfc7d8c: [Store] from 2024-03-22T15:07:35+01:00 to 2024-03-22T20:07:35+01:00 Task with message "Stored in backupdest1 (total time: 41s)"
This is log for incremental backup
==> api/api.log <==
[2024-03-22 15:08:02] INFO [__ejb-thread-pool12] {rg=1, rt=USER} AuditLogService.log:87
User has submitted 1 tasks of types EXPORT
[2024-03-22 15:08:10] INFO [__ejb-thread-pool5] {} TaskLoadBalanceTimer.assignTasks:263
Task Assignment Utility: EXPORT task b7277ddb-a4d1-4284-8783-30ba407d38bd has been assigned to the node node1 (206e4177-b07e-413d-ac71-2287ac5df8f5)
==> node1/vprotect_daemon.log <==
[2024-03-22 15:08:13.752] INFO [pool-5-thread-57] ITask.lambda$startTask$1:39
[699ccbec-a419-4745-a5d8-0314f69c8576] Starting Task b7277ddb-a4d1-4284-8783-30ba407d38bd: [Export] from 2024-03-22T15:07+01:00 to 2024-03-22T20:07+01:00
[2024-03-22 15:08:13.789] INFO [pool-5-thread-57] ExportTask.run:115
[b7277ddb-a4d1-4284-8783-30ba407d38bd] Starting export of: VM cirros15 (551df74b-dd5a-4b64-b5e2-0fe523d47da3) from 10.150.6.73 (OpenStack)...
[2024-03-22 15:08:13.790] INFO [pool-5-thread-57] BackupProviderUtil.testConnection:238
[b7277ddb-a4d1-4284-8783-30ba407d38bd] Testing connection with backup provider: backupdest1
[2024-03-22 15:08:13.791] INFO [pool-5-thread-57] FileUtil.removeLocalTestFiles:645
[b7277ddb-a4d1-4284-8783-30ba407d38bd] Cleaning up after Connectivity Test
[2024-03-22 15:08:13.809] INFO [pool-5-thread-57] ExportImportUtils.getVmExportImportModeByHvManager:43
[b7277ddb-a4d1-4284-8783-30ba407d38bd] OpenStack export import strategy by hypervisor manager: SSH_TRANSFER
[2024-03-22 15:08:13.820] INFO [pool-5-thread-57] OpenStackClientFactory.authenticateScopedToProjectName:140
[b7277ddb-a4d1-4284-8783-30ba407d38bd] Authentication scoped by project name - url: 'http://10.150.6.73:5000/v3' | region: 'RegionOne' | domain: 'default' | project: 'admin' | user: 'admin' | execution source: 'SshTransferOpenStackImportExportStrategy.java#<init>:81'
[2024-03-22 15:08:14.324] INFO [pool-5-thread-57] SshFingerprintCache.getOrCreateTrustedCertificateWithDecodedChain:55
[b7277ddb-a4d1-4284-8783-30ba407d38bd] Getting ssh fingerprint for TrustedCertificateChainId(sourceId=4d54f698-c559-4261-8e5c-46b23ea99bf7, sourceType=HYPERVISOR, chainType=SSH_HOST_FINGERPRINT) and task: b7277ddb-a4d1-4284-8783-30ba407d38bd: [Export] from 2024-03-22T15:
07+01:00 to 2024-03-22T20:07+01:00
[2024-03-22 15:08:14.330] INFO [pool-5-thread-57] SshUtil.getConfig:17
[b7277ddb-a4d1-4284-8783-30ba407d38bd] StrictHostKeyChecking = yes
[2024-03-22 15:08:14.330] INFO [pool-5-thread-57] SshUtil.getConfig:17
[b7277ddb-a4d1-4284-8783-30ba407d38bd] StrictHostKeyChecking = yes
[2024-03-22 15:08:15.827] INFO [pool-5-thread-57] OpenStackClientFactory.authenticateScopedToProjectName:140
[b7277ddb-a4d1-4284-8783-30ba407d38bd] Authentication scoped by project name - url: 'http://10.150.6.73:5000/v3' | region: 'RegionOne' | domain: 'default' | project: 'admin' | user: 'admin' | execution source: 'OpenStackHypervisorManager.java#listVmDisks:204'
[2024-03-22 15:08:17.468] WARN [pool-5-thread-57] LibvirtHypervisor.lambda$logUnsupportedQcow2Warning$45:1384
[b7277ddb-a4d1-4284-8783-30ba407d38bd] Unable to create incremental backup for a non-excluded QCOW2 driver [UUID: 4afff436-71cc-4a30-8ca5-a9406e6350aa]
==> api/api.log <==
[2024-03-22 15:08:17] INFO [http-thread-pool::http-listener(9)] {r=NODE, s=c20ddf27a4761e7799394d78ab62} SystemNotificationService.createNotification:119
System Notification of code: NODE_BACKUP_PE_NOT_READY_FOR_INCREMENTAL_BACKUP
==> node1/vprotect_daemon.log <==
[2024-03-22 15:08:17.546] INFO [pool-5-thread-57] LibvirtHypervisor.snapshotExists:1489
[b7277ddb-a4d1-4284-8783-30ba407d38bd] Checking if snapshot VM cirros15 (bd5d0690-1098-41f1-aab5-473d04351a35) - snapshot: c08f07b0-2373-48c7-b212-8d57a755b15d (8eac6082-aa6e-4b1f-8a35-618b946b73ce) @ 2024-03-22T15:07:07+01:00 exists
[2024-03-22 15:08:19.030] WARN [pool-5-thread-57] ExportIncremental.isIncrementalExportPossible:172
[b7277ddb-a4d1-4284-8783-30ba407d38bd] Reasons why incremental is not possible of the VM (551df74b-dd5a-4b64-b5e2-0fe523d47da3 - bd5d0690-1098-41f1-aab5-473d04351a35): Warning: Virtual Machine is not ready for incremental
[2024-03-22 15:08:19.031] WARN [pool-5-thread-57] ExportTask.pickApplicableBackupType:248
[b7277ddb-a4d1-4284-8783-30ba407d38bd] Incremental backup for OpenStack is not available at this time.
[2024-03-22 15:08:19.064] INFO [pool-5-thread-57] ExportFull.export:30
[b7277ddb-a4d1-4284-8783-30ba407d38bd] Performing full backup
When a new machine is created it has following virsh configuration
<devices>
<emulator>/usr/libexec/qemu-kvm</emulator>
<disk type='file' device='disk'>
<driver name='qemu' type='raw' cache='none' io='native'/>
<source file='/var/lib/nova/mnt/404d50d99406bdb14eefa7d751782ec1/volume-68e5a527-05e4-4451-aa96-983f404fa3f1' index='1'/>
<backingStore/>
<target dev='vda' bus='virtio'/>
<serial>68e5a527-05e4-4451-aa96-983f404fa3f1</serial>
<alias name='virtio-disk0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
</disk>
[root@openstackdev rocky]# qemu-img info /var/lib/nova/mnt/404d50d99406bdb14eefa7d751782ec1/volume-68e5a527-05e4-4451-aa96-983f404fa3f1
image: /var/lib/nova/mnt/404d50d99406bdb14eefa7d751782ec1/volume-68e5a527-05e4-4451-aa96-983f404fa3f1
file format: raw
virtual size: 1 GiB (1073741824 bytes)
disk size: 75 MiB
Child node '/file':
filename: /var/lib/nova/mnt/404d50d99406bdb14eefa7d751782ec1/volume-68e5a527-05e4-4451-aa96-983f404fa3f1
protocol type: file
file length: 1 GiB (1073741824 bytes)
disk size: 75 MiB
After first full snapshot:
<devices>
<emulator>/usr/libexec/qemu-kvm</emulator>
<disk type='file' device='disk'>
<driver name='qemu' type='qcow2' cache='none' io='native'/>
<source file='/var/lib/nova/mnt/404d50d99406bdb14eefa7d751782ec1/volume-68e5a527-05e4-4451-aa96-983f404fa3f1-vProtect-58e8e8b3-9af9-434e-9cbd-7a991ede8328' index='2'/>
<backingStore type='file' index='1'>
<format type='raw'/>
<source file='/var/lib/nova/mnt/404d50d99406bdb14eefa7d751782ec1/volume-68e5a527-05e4-4451-aa96-983f404fa3f1'/>
<backingStore/>
</backingStore>
<target dev='vda' bus='virtio'/>
<serial>68e5a527-05e4-4451-aa96-983f404fa3f1</serial>
<alias name='virtio-disk0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
</disk>
I tried also to start instance with a nova boot device (no cinder), initial disk is a qcow2, but I receive similar error when try to do incremental backup