vbr backups deletion issue - vertica 9.3
p_kovalenko
Vertica Customer
Hello.
I am doing daily backups via vbr tool, with restorePointLimit=7. However, sometimes old backups can not be deleted (but sometimes it can delete it). For example, now I have 13th and 15th October backups remaining, but 14th was deleted.
When I try to remove backup manually via vbr, it shows me error:
vbr -t remove -c /u01/vertica_backup/vbr_full_backup.ini --archive 20201030_230106 --debug 3 Removing restore points: 20201030_230106 Error: Trying to delete untracked object: 029645f15d46e69ab9a75d6b80db404d00a00000066fa8e9. Run backup validation. Remove FAILED.
And when I try to do garbadge collection, it also ends up with error:
vbr -t collect-garbage -c /u01/vertica_backup/vbr_full_backup.ini --debug 3 Deleting unreferenced objects Regenerating backup manifest for location [10.178.33.193]:/u01/vertica_backup/full_backup Error: Error accessing remote storage: failed accessing remote storage on <IP_ADDR>: ssh_exchange_identification: read: Connection reset by peer rsync: connection unexpectedly closed (0 bytes received so far) [Receiver] rsync error: unexplained error (code 255) at io.c(601) [Receiver=3.0.7] : returncode=None Collect-garbage FAILED.
And it is interesting, that in log file I can see rsync operations performing successfully, and after some time - rsync error:
File "/opt/vertica/bin/vbr", line 3788, in _run_rsync "failed accessing remote storage on %s: %s" % (remote_addr, err), proc.returncode) VbrRsyncError: Error accessing remote storage: failed accessing remote storage on <IP_ADDR>: ssh_exchange_identification: read: Connection reset by peer^M rsync: connection unexpectedly closed (0 bytes received so far) [Receiver] rsync error: unexplained error (code 255) at io.c(601) [Receiver=3.0.7] : returncode=None
So I have no idea what to do to fix this issue, or even how to remove it manually/using bash script. Maybe you can help me.
Best Answers
-
Nimmi_gupta
- Select Field - Employee
@p_kovalenko
What's the vertica version?
As per the first error message you have shared "Error: Trying to delete untracked object" looks like a known bug and fix is available in 10.0SP1
vbr -t remove -c /u01/vertica_backup/vbr_full_backup.ini --archive 20201030_230106 --debug 3
Removing restore points: 20201030_230106
Error: Trying to delete untracked object: 029645f15d46e69ab9a75d6b80db404d00a00000066fa8e9. Run backup validation.
Remove FAILED.
Check the release notes:
https://www.vertica.com/docs/ReleaseNotes/10.0.x/Vertica_10.0.x_Release_Notes.htm
VER-71997 : Backup/DR Vertica backups were failing to delete old restore points when the number of backup objects exceeded 10,000 objects. This issue has been fixed.1 -
p_kovalenko
Vertica Customer
@Nimmi_gupta
Thank you for the reply.
I am counting restore point objects by checking corresponding snapshot manifest file and counting lines of this file. Am I conting right?
If i am understand counting objects right, yes, I usually have more than 10 000 objects in one restore point.0
Answers
Is SSH working fine? Can you share me the contents of /etc/ssh/sshd_config?
VbrRsyncError: Error accessing remote storage: failed accessing remote storage on : ssh_exchange_identification: read: Connection reset by peer^M