Manage CDP Backups¶
CDP Backups can help you manage a node’s backup policy to save files and folders of your E2E Node and also, rolling back to a previous backup recovery point anytime. We recommend you to subscribe for CDP backups service for all your nodes especially the servers for a production environment. CDP backups secure your data from accidental data loss or modifications along with the option of retaining data for a longer period without consuming the server’s disk space. They are helpful in projects or milestones to back up the complete data or a set of files containing important information.
How does CDP Backup work?¶
Backups are created as a replica of your file system which is taken automatically at periodic intervals. Each backup recovery point contains all the data which was backed up to restore from the provisioning till the most recent data block update. This allows point-in-time to restore from any of the recovery points present in the archive. Before you enable CDP backup service, we recommend familiarizing yourself with concepts and terminologies.
The backups are created and stored in our backup servers. The connections to these are encrypted using SSL to ensure the security of the data being backed up.
Incremental in nature¶
The backup server takes a complete snapshot of the entire filesystem for the first time and from the next run, it only copies over the differential Deltas (data that has changed since the last backup run) thus speeding up the backup process.
It is always important to backup all your databases stored in your server.
- The CDP for MySQL Add-On integrates Continuous Data Protection with an online hot backup (snapshot) of MySQL databases to provide fast, efficient, and safe MySQL backups.
- MySQL add-on ensures that tables are locked and flushed before a backup operation can take place. MySQL Add-On coordinates with the point-in-time file system snapshot.
- The only requirement for MySQL Add-On is a MySQL account with administrative privileges that is used by the Agent to flush tables with the read lock before the file system snapshot is taken and the lock must be held until the snapshot completes. Click here to check more details.
Bacup frequency is customizable which can be subject to your requirement.
Backup Recovery Points¶
A recovery point is created as a result of data replication. The first recovery point is called an initial replica because it is performed only one time. In simple words, a Recovery Point appears like a Full backup as each Recovery Point only contains block-level Deltas or changes since the last Synchronization (recovery point creation). In other words, a Recovery Point is a point-in-time backup of the node (or selected files/folders on devices). Each recovery point is all-inclusive i.e it contains all the data and can be used for a restore independently.
Recovery Points Retention Policy¶
The Recovery Point Retention policy is default defined and it is applicable for each server for which backup service is enabled. The retention policy automatically decides on how many numbers of Recovery Points needed to retain and when to merge old recovery points. The deltas are merged into existing recovery points to create recovery/archive points. According to this, the recovery/archive point creation is done by merging the Old Recovery Points automatically as follows: - The last backup recovery points of the last 14 days (last backup to complete before midnight) - The last backup recovery points of the last day of the week(Saturday) for the previous 4 weeks. - The last backup recovery points of the last day of the month for the previous 2 months.
After exceeding the defined recovery/archive point number, the old Archive Points will subsequently be replaced by the new ones.
The backup restore rates are inversely proportional to the load on your system. In an ideal environment, Our backup systems can restore at an avg rate of 20 MB/s which translates to 65-70 GB per hour. However, the actual rates may vary and we have seen better speeds than this. The way to request a rollback/restore is to select the files/folders from the recovery points and initiate the restore process.
How to activate CDP Backup service for nodes?¶
- Logging into E2E Networks` MyAccount Portal <https://myaccount.e2enetworks.com/>`_
- Navigate to CDP Backups page either from dashboard or from Side menu
- The number of E2E Nodes currently created by you will be listed on the ‘Manage Backups’ page.
- If you have not enabled the backups for your E2E Node while creation then the backup status will be ‘Backup Not Activated’. To enable CDP backup service for your E2E Node you will need to click on ‘Activate Backup’ action
Backup Schedule Configuration pop-up window will appear on the screen.
Backup Schedule Configuration¶
Choose Schedule Type- Please select the type either “Hourly” or “Daily” based on your use case. Hourly: If you selected the hourly backup schedule type then you need to select the starting Minute and Hour of Day so that the hourly backups are scheduled.
Daily: If you selected the ‘daily’ backup schedule type then you need to specify an hour of the day and select when to run (either at ‘Selected days of the week’ OR ‘At Everyday’ ) so that the daily backups are scheduled.
Exclude File/Folder- By default, all files and folders located on your node are replicated. However, if you want to ignore specific files/folders during backup, you can exclude them by specifying the path of file/folder.
MySQL Backup- If MySQL server running on this E2E Node, then you may provide the Admin credentials of MySQL server for activating backup to protect the following MySQL objects: - Databases - Tables - Indexes - Constraints - Triggers - Events - Views - Stored procedures
You will receive notification emails for backup service activation and after first backup recovery point creation.
The Backups are not available for E2E Nodes using ‘CentOS-8’ or ‘Opensuse’ operating system.
- Backup Unsupported
‘Backup Unsupported’ status represents that CDP backup service is not available for the respective Node type.
- Backup Not Activated
‘Backup Not Activated’ status represents that CDP backup service is disabled for the respective Node type
- Backup Provisioning Failed
‘Backup Provisioning Failed’ status represents that activation of backup service is failed for the respective node.
- First Run Pending
‘First Run Pending’ status represents that Backup service has activated for the respective node and first backup recovery point creation is pending as per your scheduled time.
- Backup Available
‘Backup Available’ status represents that backup service is active and backup recovery points are available.
- Powered Off
‘Powered Off’ status represents that backup service is turned off due to the respective node being powered off.
- Backup Failed
‘Backup Failed’ status represents that the last backup recovery point creation is failed for the respective node.
Manage CDP Backup Service¶
Once the CDP backup service is enabled and status changes from ‘First Run Pending’ to ‘Backup Available’ after initial recovery point creation for the respective E2E Node then you will be able to manage and check backup details & actions as described.
CDP Backup Details¶
The CDP Backup Details tab contains information about the Backup status, Backup Size, Backup Schedule & frequency, Last Recovery Points Created On, Next Backup Scheduled Run, and MySQL Database for the respective E2E Node.
Backup Recovery Points¶
The Backup Recovery Points tab having information about backup recovery points created for the respective E2E Node in chronological order. You can select any of backup recovery points to restore file/folders to the respective node.
You can perform the following actions for the backup recovery points.
You can use browse action to search & select files/folders from the Recovery Point for restoration.
You can use lock action to prevent backup recovery point from being merged automatically as per the recovery point protection policy.
You can use unlock action to unlock the locked backup recovery point to allow it to be merged automatically as per the recovery point protection policy.
Backup Task History¶
The Backup Task details tab has information about start and ends (with the date and time stamp) backup scheduled Tasks in a grid format.
‘Finished’ status represents that the backup recovery point is created.
‘Failed’ status represents that backup recovery point creation is failed for the respective node.
Restore Task History¶
The Restore Task details tab has information about start and ends (with the date and time stamp) file/folder restore scheduled Tasks in a grid format.
‘Finished’ status represents that file/folder restored successfully.
‘Failed’ status represents that the file/folder restore is failed.
‘Warning’ status represents that file/folder restored but the respective node’s disk space is running low.
You can perform the following actions for the backup service.
Click on the “Change Schedule” button to modify the current backup schedule type & frequency, configuring MYSQL and to change/add exclude files/folders paths.
To deactivate CDP backup service, you have to click on the ‘Deactivate Backup’ switch to disable the backup service for your E2E Networks E2E Node. A confirmation popup will appear, click on the ‘Deactivate’ button to stop your E2E Node backups.
How to restore Files/ Folders?¶
The ‘Restore backups’ feature provides you with the facility to restore the entire file system or selected files and folders into your E2E Node from any available Backup Recovery Point. To know the process of restoring a database in MySQL click here
- Go to the ‘Backup Recovery Point’ tab and select the backup recovery point.
- Now, you have to click on the browse action to browse the file system.
- Browse Recovery Point pop up window will appear on the screen.
- Select the required files/folders.
- After selecting the files/folder you have to click the Restore selected button.
- Now, you need to select the restore location radio button.
Default Location: If you selected the default location radio button the selected files/folders will be restored at a different path which is /e2e-backup-restore/ instead of the original path. You need to manually move folders/files from the restore path to the original path. The selected folder/files.
Alternate Location: If you selected the alternate location radio button the selected files/folders will be restored at your specified location of a disk in your Node.
Please ensure that enough disk space is available on your node to restore files. Otherwise, the restoration will fail and due to restore failure might be node itself face downtime and you will be solely responsible for this.
- An Information message will appear when the restoration process is initiated.
- The restoration process will take a few minutes for completion. It will depend on the size of files/folders you have selected for restoration.
- Once, Backup Restoration completed successfully for your E2E Node, you will be notified via email and the selected files/folders will be available in your E2E Node.
How to take a complete cPanel backup¶
Introduction to cPanel¶
cPanel is a Linux-based web hosting control panel. It provides a graphical interface and automation tools designed to simplify the process of hosting a web site. cPanel provides capabilities for administrators, resellers, and end-user website owners to control the various aspects of website and server administration through a standard web browser. To take a complete backup of the website using cPanel, follow the steps below:
Step 1: Log in to your cPanel.
Step 2: Select ‘Backups’ in the Files section of cPanel.
Step 3: In the Full Backup section, click the Download a Full Website Backup button.
Step 4: On the next screen make sure the Home Directory option is selected.
Step 5: If you want an email notification when the backup completes, enter your email address. If you do not want an email, check the box next to Do not send an email notification of backup completion.
Step 6: Click the ‘Generate Backup’ button to begin the Full cPanel Backups.
You will then see a message stating “Full Backup in Progress…” Allow some time for the backup to complete, you will receive an email notification upon completion (if you entered your email address in the last step).
Step 7: Download the file from your cPanel. It will be located in your /home directory, begin with backup, and include the date. For example backup-10.9.2015_15-52-08_exampl99.tar.gz You will now have a Full cPanel backup!
How to take backup and restore containers in Nvidia Docker?¶
Please note that this section mainly discusses committing a container as an image. This works on the container that does not use data volume. For containers with data volume, backup of the data volume must be taken separately.
Now a new image jupyter-backup will be created. Kindly note that this will not cover the data volume. You need to take the backup of data-volume (if any) separately.
To know this data-directory (data volume location) of a container, use the command ‘docker inspect container-name‘. You will get a section called “Mounts”. Location mentioned in “Source” is the data volume. You may directly backup this folder(here /var/lib/docker/volumes/my-volume/_data) to get backup of data volume.
Backing up the Containers¶
First of all, to backup the containers in docker, we’ll wanna see the list of containers that we wanna backup. To do so, we’ll need to run docker ps in our linux machine running docker engine with containers already created.
# nvidia-docker ps
After that, we’ll choose the containers we wanna backup and then we’ll go for creating the snapshot of the container. We can use nvidia-docker commit command to create the snapshot.
# nvidia-docker commit -p 1568555b36ef jupyter-backup
This will generate a snapshot of the container as the docker image. We can see the docker image by running the command nvidia-docker images as shown below.
# nvidia-docker images
As we can see the snapshot that was taken above has been preserved as a docker image. Now, to back up that snapshot, we have two options, one is that we can log in into the docker registry hub and push the image and the next is that we can back up the docker image as tarballs for further use as it can be downloaded locally in your laptop or to the object storage like minio,s3.
If we wanna upload or backup the image in the docker registry hub, we can simply run docker login command to login into the docker registry hub and then push the required image.
# nvidia-docker login
Executing the above command will prompt for user and password for docker hub registry.
# nvidia-docker tag e469e7acb748 hrrahman/jupyter-backup:test
# nvidia-docker push hrrahman/jupyter-backup
If we don’t want a backup to the docker registry hub and want to save the image for future use in the machine locally or to the object storage like minio,s3. then we can backup the image as tarballs. To do so, we’ll need to run the following nvidia-docker save command.
# nvidia-docker save -o ~/jupyter-backup.tar jupyter-backup
To verify if the tarball has been generated or not, we can simply run ls inside the directory where we saved the tarball.
Restoring the Containers¶
Next, after we have successfully backed up our docker containers, we will now go for restoring those containers which are snapshotted as docker images. If we have pushed those docker images in the registry hub, then we can simply pull that docker image and run it out of the box.
# nvidia-docker pull hrrahman/jupyter-backup:test
But if we have backed up those docker images locally or any other cloud storage like minio or s3 as tarball file, then we can easily sync from the respective source and load that docker image using nvidia-docker load command followed by the backed up tarball.
# docker load -i ~/jupyter-backup.tar
Now, to ensure that those docker images have been loaded successfully, we’ll run nvidia-docker images command.
# nvidia-docker images
After the images have been loaded, we will be going to run the docker container from the loaded image.
# nvidia-docker run –rm -p 8888:8888 -p 4040:4040 jupyter-backup:latest
We have documented how we can backup, restore docker containers using nvidia-docker out of the box. This tutorial is the same for every platform of an operating system where nvidia-docker runs successfully.
The above methods make us comfortable to backup our containers so that we can restore them when needed in future. This can help us recover our containers and images even if our host system crashes or gets wiped out accidentally.
Disasters cannot be predicted or prevented, but you can always quickly bounce back from them if you maintain adequate backup plans for your business.