vertica backups failing with assertion error

sarafumes

A couple of months back i had the vertica backups to s3 set up, but they are now suddenly failing with the below error please help me to solve this, Thanks!

2018-05-14 14:04:35 localhost vbr Old restore points to be removed: 20171106_191224
2018-05-14 14:04:39 localhost vbr Approximate bytes to copy: 350062619 of 67528612898 total.
2018-05-14 14:05:30 localhost vbr Copying backup metadata.
2018-05-14 14:05:33 localhost vbr Finalizing backup.
2018-05-14 14:05:33 localhost vbr Location s3://****************/: putting updated backup manifest.
2018-05-14 14:05:33 localhost vbr Deleting old restore points.
2018-05-14 14:05:35 localhost vbr Deleting restore point metadata.
2018-05-14 14:05:35 localhost vbr Traceback (most recent call last):
File "/opt/vertica/bin/vbr", line 8971, in main
if vbr_task.run():
File "/opt/vertica/bin/vbr", line 4325, in run
return self._run()
File "/opt/vertica/bin/vbr", line 5485, in _run
Parallel.foreach(delete_snap_metadata, old_backup_node_timestamps.items())
File "/opt/vertica/bin/vbr", line 8656, in foreach
cls.map(func, iterable, threads_num=threads_num)
File "/opt/vertica/bin/vbr", line 8635, in map
if not thr.join(Parallel.WAIT_QUANTUM):
File "/opt/vertica/bin/vbr", line 8611, in join
return self._thr.join(*args, **kwargs)
File "/opt/vertica/bin/vbr", line 8565, in run
super(Parallel.CancellableThread, self).run()
File "/opt/vertica/oss/python/lib/python2.7/threading.py", line 763, in run
self.__target(*self.__args, **self.__kwargs)
File "/opt/vertica/bin/vbr", line 8599, in do_run
self._result = func(val)
File "/opt/vertica/bin/vbr", line 5478, in delete_snap_metadata
backup_access.delete_all(metadata_dir)
File "/opt/vertica/bin/vbr", line 3009, in delete_all
self._delete_all(remote_prefix)
File "/opt/vertica/bin/vbr", line 3185, in _delete_all
self._delete_many(remote_prefix, self._list_prefix_recursive(remote_prefix))
File "/opt/vertica/bin/vbr", line 3117, in _list_prefix_recursive
assert basepath.endswith(prefix)
AssertionError

SruthiA

What is your vertica version? This is a known bug , VER-61190 , fixed in 9.0.1-6

https://www.vertica.com/docs/ReleaseNotes/9.0.x/Vertica_9.0.x_Release_Notes.htm#9.0.1-6

sarafumes

Aah thanks Sruthi, we are currently using Vertica Analytic Database v8.1.1-3

SruthiA

In 8.1, this issue has been fixed in 8.1.1-17