Restoring Data from a Remote Storage System Backup
The following script restores data from a remote storage system backup (Destination Volume of an Asynchronous Stack Mode REC Session and its Clone). This script is used in the operation of periodically backing up the business volume on the remote storage system using REC in asynchronous stack mode and QuickOPC. For more information, see the description of Creating Backups to Remote Storage Systems Using REC and QuickOPC.
The script checks the Status of the REC Session to automatically determine if the data should be restored from the remote storage system backup volume (Copy Destination Volume of an Asynchronous Stack Mode REC Session) or from the Clone of the remote storage system backup volume.
This script can only be used in an environment where a Backup Volume is created in the remote storage system described in Creating Backups to Remote Storage Systems Using REC and QuickOPC. It is not available for restoring data in any other environment.
This script can be used when the REC Session state is normal. If the storage system is physically damaged by a disaster and the REC path is "Halt", restore the path and then recreate the REC Session from the Backup Volume or Clone on the remote storage system to recover the data.
The sequence of script processing is as follows.
Use GET /copysession/{copysession_id} API to obtain information about the REC Session and determine if the REC Session can be reversed (If Status is Error or the REC Session has already been reversed, processing ends).
Using GET /copysession/{copysession_id} API to capture REC Session information, determine whether to restore from a backup volume or a Clone of the backup volume (If Suspend is in progress, restore from the backup volume. If Suspend is not in progress, restore from the Clone of the backup volume.).
When restoring from the backup volume of the remote storage system, perform the following processing.
Use POST /copysession/{copysession_id}/reverse API to reverse the REC Session.
Restart the REC Session using POST /copysession/{copysession_id}/resume API.
Use the GET /copysession/{copysession_id} API to periodically retrieve the REC Session information and wait for the data transfer from the remote storage system to the local storage system to complete.
Use POST /copysession/{copysession_id}/suspend API to Suspend the REC Session.
Use POST /copysession/{copysession_id}/reverse API to reverse the REC Session (return to the original orientation).
When restoring from the Clone of the backup volume of the remote storage system, perform the following processing.
Use POST /copysession/{copysession_id}/suspend API to Suspend the REC Session.
Use POST /copysession/{copysession_id}/reverse API to reverse the REC Session.
Using an POST /volume/{volume_id}/restore/{backup_volume_id} API, copy the data from the Clone of the remote storage system backup volume to the remote storage system backup volume.
Use POST /copysession/{copysession_id}/resume API to restart the REC Session and start transferring data from the remote storage system to the local storage system.
Use the GET /copysession/{copysession_id} API to periodically retrieve the REC Session information and wait for the data transfer from the remote storage system to the local storage system to complete.
Use POST /copysession/{copysession_id}/suspend API to Suspend the REC Session.
Use the GET /volume/{volume_id}/copysession API to determine if the data has been copied from the Clone of the remote storage system backup volume to the remote storage system backup volume.
Use POST /copysession/{copysession_id}/reverse API to restore the orientation of the REC Session to the state it was in prior to the restore.
Restart the REC Session using POST /copysession/{copysession_id}/resume API.
rec_stack_recover.py
import sys
import time
from eternus_rest import EtdxBaseApi
# Change the following values for your environment.
# Source storage (storage that has the source volume)
storage1_url = "https://192.168.0.1:5665"
storage1_user_name = "username"
storage1_password = "password"
# Target storage (storage that has the destination volume)
storage2_url = "https://192.168.0.2:5665"
storage2_user_name = "username"
storage2_password = "password"
# This case is for a REC(Stack) session from the volume on storage1
# to storage2.
# The destination volume of REC has been cloned using QuickOPC on storage2.
# You can paste the outputs of rec_stack_create.
# REC session ID on storage1 created by the script rec_stack_create.
storage1_copysession_id = 1
# Backup volume ID created by the script rec_stack_create as the destination
# volume of REC.
storage2_volume_id = 200002
# Clone volume ID created by the script rec_stack_create as the clone for the
# REC destination volume.
storage2_clone_volume_id = 300003
class EtdxApi(EtdxBaseApi):
def __init__(self, base_url, boxid=None, proxies=None):
"""Constructor of EtdxApi
If storage is used as target storage of REC without calling login
method in this script, need to specify the boxid of this storage
as follows.
storage2 = EtdxApi(storage2_url,
boxid="00ETERNUSDXHS3ET00000A####EI000001######")
Args:
base_url ([type]): [description]
boxid ([type], optional): [description]. Defaults to None.
proxies ([type], optional): [description]. Defaults to None.
"""
super().__init__(base_url, boxid=boxid, proxies=proxies)
def login(self, user_name="", password=""):
"""Create a session of RESTful API and get boxid.
Args:
user_name (str, optional): User name to login. Defaults to "".
password (str, optional): Password for the user. Defaults to "".
Returns:
bool: True if successful, False otherwise.
"""
rtn = super().login(user_name, password)
if rtn and self.boxid is None:
r = super().get("/api/v1/storagesystem")
if r.status_code != 200:
print(r)
print(r.json())
print("Failed to get boxid.")
return False
self.boxid = r.json()["boxid"]
return rtn
def rec_can_recover(self, copysession_id):
"""Check if the REC session can start the recovery.
Args:
copysession_id (int): Copy Session ID to check.
Returns:
bool: True if the specified REC session can start the recovery.
False otherwise. Returns false if retrieving copy session
information failed.
"""
r = super().get("/api/v1/copysession/{0!s}?fields=status,"
"is_local_source_volume"
.format(copysession_id))
data = r.json()
if r.status_code != 200:
print("Failed to get information about the copy session.")
print(r)
print(data)
return False
if data["status"] not in ["Active", "Suspended"]:
print("The REC session is not in normal status.")
print(data)
return False
if not data["is_local_source_volume"]:
print("The REC session might be in recovering.")
print(data)
return False
return True
def rec_is_suspended(self, copysession_id):
"""Check if the REC session is in "Suspended" status.
Args:
copysession_id (int): Copy session ID to check.
Returns:
bool: True if the REC session is in "Suspended". False otherwise.
Returns false if retrieving copy session information failed.
"""
r = super().get("/api/v1/copysession/{0!s}?fields=status"
.format(copysession_id))
if r.status_code != 200:
print(r)
print(r.json())
raise Exception("Failed to get information about the "
"copy session.")
if r.json()["status"] == "Suspended":
return True
return False
def rec_wait_for_equivalent(self, copysession_id):
"""Wait for the REC session phase to be equivalent.
Args:
copysession_id (int): Copy session ID to wait.
Returns:
bool: True if the copy session phase is equivalent.
False if error occurs.
"""
while True:
r = super().get("/api/v1/copysession/{0!s}".format(copysession_id))
data = r.json()
if r.status_code != 200 or data["status"] == "Error":
print("Failed to get information of the copy session or some "
"error occurred on the copy session.")
print(r)
print(data)
return False
if data["phase"] == "Equivalent":
break
print("The REC session is not yet equivalent. Check again in 60 "
"seconds.")
print("Total Data Size: ", data["total_data_size"])
print("Copied Data Size:", data["copied_data_size"])
print("Progress: ", data["progress"])
print("Elapsed Time: ", data["elapsed_time"])
time.sleep(60)
return True
def rec_suspend(self, copysession_id, force=False):
"""Suspend the REC session.
Args:
copysession_id (int): Copy session ID to suspend.
force (bool, optional): True to force suspend. Defaults to False.
Returns:
bool: True if successful, False otherwise.
"""
if force:
r = super().post("/api/v1/copysession/{0!s}/suspend?force=true"
.format(copysession_id))
else:
r = super().post("/api/v1/copysession/{0!s}/suspend"
.format(copysession_id))
if r.status_code != 202:
print("Failed to request the REC session suspend.")
print(r)
print(r.json())
return False
# Wait for the job completion
job_r = super().wait_job(r.json()["job_id"])
if job_r.json()["status"] != "Success":
print("Failed to suspend the REC session.")
print(job_r)
print(job_r.json())
return False
return True
def rec_resume(self, copysession_id):
"""Resume the REC session
Args:
copysession_id (int): Copy session ID to resume.
Returns:
bool: True if successful, False otherwise.
"""
r = super().post("/api/v1/copysession/{0!s}/resume"
.format(copysession_id))
if r.status_code != 202:
print("Failed to request the REC session resume.")
print(r)
print(r.json())
return False
# Wait for the job completion
job_r = super().wait_job(r.json()["job_id"])
if job_r.json()["status"] != "Success":
print("Failed to resume the REC session.")
print(job_r)
print(job_r.json())
return False
return True
def rec_reverse(self, copysession_id):
"""Reverse the REC session
Args:
copysession_id (int): Copy session ID to reverse.
Returns:
bool: True if successful, False otherwise.
"""
r = super().post("/api/v1/copysession/{0!s}/reverse"
.format(copysession_id))
if r.status_code != 202:
print("Failed to request the REC session reverse.")
print(r)
print(r.json())
return False
# Wait for the job completion
job_r = super().wait_job(r.json()["job_id"])
if job_r.json()["status"] != "Success":
print("Failed to reverse the REC session.")
print(job_r)
print(job_r.json())
return False
return True
def quickopc_wait_for_restore(self, volume_id):
"""Wait for the restore copy session from the clone to complete.
Args:
volume_id (int): Volume ID of the original of the clone.
Returns:
bool: True if the restore copy session completed. False if
retrieving copy session information failed or some error
occurred on the copy session.
"""
while True:
r = super().get("/api/v1/volume/{0!s}/copysession"
.format(volume_id))
data = r.json()
if r.status_code != 200:
print("Failed to get information of the copy session of volume"
" (ID: {0!s})."
.format(volume_id))
print(r)
print(data)
return False
# If the original volume of clone is not a copy destination, the
# restore copy session is considered complete.
if not data["is_copy_destination_volume"]:
break
# If the copy session is not a restore, the restore copy session
# is considered complete.
if data["backup_type"] != "Restored":
break
# If the restore copy session is in "Suspend" or "Error" status.
if data["status"] in ["Suspended", "Error"]:
print("Restore Copy session is not in expected status ({0!s})"
.format(data["status"]))
print(data)
return False
time.sleep(10)
return True
def quickopc_restore(self, volume_id, backup_volume_id):
"""Restore the QuickOPC.
Args:
volume_id (int): Volume ID of the original of the clone.
backup_volume_id (int): Volume ID of the clone.
Returns:
bool: True if successful, False otherwise.
"""
r = super().post("/api/v1/volume/{0!s}/restore/{1!s}"
.format(volume_id, backup_volume_id))
if r.status_code != 202:
print("Failed to request the QuickOPC restore.")
print(r)
print(r.json())
return False
# Wait for the job completion
job_r = super().wait_job(r.json()["job_id"])
if job_r.json()["status"] != "Success":
print("Failed to restore the QuickOPC.")
print(job_r)
print(job_r.json())
return False
return True
def rec_stack_recover_under_operating(self, copysession_id):
"""Recovery process under operating.
Args:
copysession_id (int): Copy session ID of the REC.
Returns:
bool: True if successful, False otherwise.
"""
# Reverse the REC session
print("Reversing REC session.")
if not self.rec_reverse(copysession_id):
return False
# Restore the data from the destination volume to the source volume
# of the REC.
print("Resuming the REC session.")
if not self.rec_resume(copysession_id):
return False
print("Waiting for the REC session to be equivalent.")
if not self.rec_wait_for_equivalent(copysession_id):
return False
print("The data restore Completed.")
# Restore the REC session to its former state.
print("Suspending the REC session.")
if not self.rec_suspend(copysession_id):
return False
print("Reversing REC session.")
if not self.rec_reverse(copysession_id):
return False
return True
def rec_stack_recover_not_operating(self, copysession_id, remote_storage,
volume_id, clone_volume_id):
"""Recovery process while not operating.
Args:
copysession_id (int): Copy session ID of the REC.
remote_storage (obj): Object of the destination storage of the REC.
volume_id (int): Volume ID of the destination volume of the REC.
clone_volume_id (int): Volume ID of the clone of the REC
destination volume.
Returns:
bool: True if successful, False otherwise.
"""
# Reverse the REC session
print("Suspending the REC session.")
if not self.rec_suspend(copysession_id, force=True):
return False
print("Reversing the REC session.")
if not self.rec_reverse(copysession_id):
return False
# Restore the data from clone to the destination volume of the REC.
print("Restoring data from the clone (volume ID: {0!s}) to the REC "
"destination volume (ID: {1!s})."
.format(clone_volume_id, volume_id))
if not remote_storage.quickopc_restore(volume_id, clone_volume_id):
return False
# Restore the data from the destination volume to the source volume
# of the REC.
print("Resuming the REC session.")
if not self.rec_resume(copysession_id):
return False
print("Waiting for the REC session to be equivalent.")
if not self.rec_wait_for_equivalent(copysession_id):
return False
print("The data restore Completed.")
# Restore the REC session to its former state.
print("Suspending the REC session.")
if not self.rec_suspend(copysession_id):
return False
# Wait for the restore copy session from the clone to complete.
print("Waiting for the restore copy session from the clone to "
"complete.")
if not remote_storage.quickopc_wait_for_restore(volume_id):
return False
print("Reversing REC session.")
if not self.rec_reverse(copysession_id):
return False
print("Resuming REC session.")
if not self.rec_resume(copysession_id):
return False
return True
def main():
storage1 = EtdxApi(storage1_url)
storage2 = EtdxApi(storage2_url)
# Login
if not storage1.login(storage1_user_name, storage1_password):
return False
if not storage2.login(storage2_user_name, storage2_password):
return False
print("Start recovery of the source volume data about the REC session"
" (ID: {0!s}).".format(storage1_copysession_id))
# Change recovery process according to the operation condition.
try:
print("Check the copy session whether the recovery process can work.")
if not storage1.rec_can_recover(storage1_copysession_id):
print("Could not start recovery.")
return False
if storage1.rec_is_suspended(storage1_copysession_id):
print("Starting Recovery during operation.")
if not storage1.rec_stack_recover_under_operating(
storage1_copysession_id):
print("Failed to recover.")
return False
else:
print("Starting Recovery during shutdown.")
if not storage1.rec_stack_recover_not_operating(
storage1_copysession_id,
storage2,
storage2_volume_id,
storage2_clone_volume_id):
print("Failed to recover.")
return False
except Exception as e:
print(e)
return False
print("Recovery completed.")
# Logout
storage1.logout()
storage2.logout()
return True
if __name__ == '__main__':
if not main():
sys.exit(1)