How does Vertica backup work?

Old_Community

Vertica provides a utility which fully automates the backup process. The standard backup script is designed to take hot, incremental backups. Vertica’s backup is entirely file-based. Because we never change data files in place (only add new files, and remove old ones), our backup process concentrates on getting a version of the Vertica configuration, catalog, and data files that comprise a consistent snapshot of the system. The Vertica backup script does exactly that – it stores catalog information to a file, determines which corresponding data files need to be backed up, and takes reference counts to make sure that none of the files that need to back up will be deleted before the backup completes. Once the files have been identified, the script uses rsync to transfer the files to the backup location. The first time you run the backup utility it creates a complete backup. Subsequent runs are incremental, and the utility copies only files that are new since the previous backup. As a hot backup implementation, the procedure is designed to have minimal impact on a running database. Taking the snapshot itself is extremely lightweight; the more expensive part is simply the transfer of the backed up data to the backup location.

Old_Community

I think it would be helpful if people talk about their solutions to storing backups. Personally, I’m struggling because I can’t get the budget to attach a NAS or SAN to my machines and backup, and my database is nearing 3tb uncompressed. Right now, I’m choking trying to use a cloud drive backup solution so what are others doing?

Old_Community

S3 works great for me. I only do full backups once a month, but even if you are trying to do incremental backups, you can use any kind of sync tool to do so. I wouldn’t even imagine using anything else than S3 for my backups.

Old_Community

In our environment vertica full backup is taking lot of time , near to 10 hrs to backup 1.6TB data which is distributed on 6 nodes with 32GB RAM each. And any plan for table level backup or schema level backup?

Old_Community

Schema and table level backups are now available with Vertica 6.0 and later. You can specify "objects" which you want to back up. These objects can be whole shemas or individual tables. You can find more info in the docs here: http://my.vertica.com/docs/6.1.0/HTML/index.htm#16054.htm

[Deleted User]

testing new user

Ray_1

Can you please post the document for setting up Vertica to backup to S3? A quick summary would help too. Thank you!

[Deleted User]

Hello Ray: Unfortunately Vbr.py does not work with S3 right now. There is a feature request pending to replace the use of hardlinks in the backup utility. I will add your input to the interested parties for the existing request, to offer other utilities that can access S3. Thanks for your input. Kanti

abinas

Hi

 

I have DB size of 20 TB and it takes more than 20 days to take backup.

whats the ideal way to take backup for 20 tb backup.

Also suggest if any third party backup utlity for backup vertica.

 

 

Thanks

Abinas

GaneshRaja

Is it on-prem or AWS environment? Depend upon the env you have various solution that you can explore.