Deduplication/Compression

This function removes duplicate data blocks, compresses the remaining data blocks, and then stores the data.

Figure: Deduplication/Compression Overview

This function removes duplicate data blocks and stores the data.

Figure: Deduplication Overview

This function compresses each data block and stores the data.

Figure: Compression Overview

The ETERNUS DX performs data deduplication/compression in synchronization with the I/O from the server.

• Using this function in environments where random access occurs is recommended. Data is intermittently stored in Deduplication/Compression Volumes because data is appended.

• I/O response may significantly degrade when compared to systems that do not use the Deduplication/Compression function.

• Using this function in environments where the I/O size is 32KB or smaller is recommended. The performance is affected in environments where the I/O size is large because data is deduplicated and compressed every 8KB.

• If the I/O size or the boundaries of the I/O address are not 8KB, the performance is affected because the parts less than 8KB are read in the ETERNUS DX.

• If I/Os are issued to the Deduplication/Compression Volumes, the CPU usage rate increases. The performance of non-Deduplication/Compression Volumes may also be affected.

• The controller that controls the data reduction process of the Deduplication/Compression function is called the data reduction CM. By replacing the data reduction CM assigned to the Deduplication/Compression Volume, the memory and load of the CMs can be balanced during the data reduction process.

• Enabling the Deduplication/Compression function

  From ETERNUS Web GUI or ETERNUS CLI, enable the Deduplication/Compression function for the TPP. Not only can the Deduplication/Compression function be enabled, but the Deduplication function or the Compression function can be individually enabled.

  The Deduplication/Compression function can be enabled by performing one of the methods below.

  Table: Method for Enabling the Deduplication/Compression Function

  Condition of the TPP	Chunk size (*1)	Creation method
  New TPP (recommended)	21MB	During the TPP creation, select the Automatic mode and specify the Deduplication/Compression function option to enable it
  	21MB (default) – 1,344MB (*2)	During the TPP creation, select the Manual mode and specify the Deduplication/Compression function option to enable it
  Existing TPP	21MB – 1,344MB (*3)	Select a target TPP to enable the Deduplication/Compression function (*4)

  *1	:	The chunk size can be checked in the list display of the TPP.
  *2	:	The chunk size can be changed only if the maximum pool capacity is selected. Select the chunk size (21MB – 1,344MB) from Advanced Settings. For normal operations, the default value (21MB) does not need to be changed.
  *3	:	The chunk size (21MB – 1,344MB) during a TPP creation is maintained. The Deduplication/Compression function can be enabled for TPPs whose chunk sizes are not 21MB.
  *4	:	If I/O load exists in the ETERNUS DX, enabling or disabling the Deduplication/Compression function in the TPP may take time. If I/O load exists, changing the setting of the Deduplication/Compression function for each TPP is recommended.

  Caution

  If the Deduplication/Compression function is enabled for TPPs, the efficiency of the physical capacity decreases as the chunk size increases. Therefore, when the capacity of Thin Provisioning is sufficient, not changing the maximum Thin Provisioning capacity is recommended if the change increases the chunk size.

• Configuration method for the Deduplication/Compression function

  Select the TPP where the Deduplication/Compression function is enabled, and create Deduplication/Compression Volumes (TPVs) in the selected TPP.

  Specify whether to enable or disable the Deduplication/Compression function for each TPV. TPVs (or Deduplication/Compression Volumes) where the Deduplication/Compression function is enabled and disabled can exist together within the same TPP. However, these two types of TPVs should be located in separate TPPs.

  Deduplication is performed for Deduplication/Compression Volumes within the same TPP. Deduplication is not performed for data in different TPPs. In some cases, deduplication might not be performed even within the same TPP.

  To enable the Deduplication/Compression function for existing volumes, use the RAID Migration function.

Volumes that are to be created and the Deduplication/Compression setting for TPPs where the target volumes can be created vary depending on the selection of "Deduplication" and "Compression".

• Volumes that are to be created

  Table: Volumes That Are to Be Created depending on the Selection of "Deduplication" and "Compression"

  Condition		Volumes that are to be created
  Deduplication	Compression
  Enable	Enable	Deduplication/Compression Volumes where both Deduplication and Compression are enabled
  Enable	Disable	Deduplication/Compression Volumes where only Deduplication is enabled
  Disable	Enable	Deduplication/Compression Volumes where only Compression is enabled
  Disable	Disable	TPVs for SAN where both Deduplication and Compression are disabled

• Deduplication/Compression setting for TPPs where the volumes can be created

  Table: Deduplication/Compression Setting for TPPs Where the Target Volumes Can Be Created

  Condition		Deduplication/Compression setting for the destination TPP
  Deduplication	Compression	Only Deduplication is enabled	Only Compression is enabled	Both Deduplication and Compression are enabled	Both Deduplication and Compression are disabled
  Enable	Enable	×	×	¡	×
  Enable	Disable	¡	×	×	×
  Disable	Enable	×	¡	×	×
  Disable	Disable	¡	¡	¡	¡

  ¡: Volumes can be created, ×: Volumes cannot be created

One DATA_CNTNR Volume is created for each TPP where the Deduplication/Compression function is enabled.

Before enabling the Deduplication/Compression function for a TPP, check whether the remaining capacity is sufficient because the DATA_CNTNR volume is created just within the maximum pool capacity.

Because the data after deduplication/compression is stored in the DATA_CNTNR Volume, add RAID groups to the TPP or expand the DATA_CNTNR Volume before the usage rate of the TPP or the usage rate of the DATA_CNTNR Volume reaches 100%.

The DATA_CNTNR Volume cannot be expanded to a capacity larger than the maximum pool capacity. If the capacity of the DATA_CNTNR Volume exceeds the maximum pool capacity, use the RAID Migration function to migrate the Deduplication/Compression Volumes of the TPP to non-Deduplication/Compression Volumes (TPVs) or another TPP. For details on the maximum logical capacity of the DATA_CNTNR Volume, refer to Table: Deduplication/Compression Function Specifications (ETERNUS DX600 S6) or Table: Compression Function Specifications (ETERNUS DX900 S6 and ETERNUS DX8900 S6).

Apart from the data after deduplication/compression, the control information is written to the DATA_CNTNR Volume. The physical capacity that is used for the control information is the total of the fixed capacity of 4GB maximum and the variable capacity (1 - 15%) according to the written size from the server.

The physical capacity may temporarily be larger than the logical capacity that is written because data is appended in Deduplication/Compression Volumes. If the I/O load is high, the physical capacity may run out. Monitoring the physical capacity on a regular basis and enabling SNMP notifications are recommended.

As with normal TPVs, the size of the used areas that are recognized by the server may differ from the areas that are actually allocated if physical capacity is allocated to areas not used by the server.

In this case, perform one of the following area release operations.

• Release unused areas by issuing the space reclamation (UNMAP) command from the server.

  For the procedure on issuing the UNMAP command, refer to the manuals of the relevant server or OS.

• Zero out the unused areas and then execute "TPV/FTV capacity optimization" for the DATA_CNTNR Volume.

  For details, refer to TPV/FTV Capacity Optimization.

The following table provides management functions for volumes related to Deduplication/Compression.