基于数据存储的重复删除技术的研究

重复数据删除也称为单一实例存储或智能压缩.首先对文件分段,再使用SHA-1算法进行重复数据段的检测,最后建立数据字典存储单一实例的数据段并进行压缩,从而节省大量的存储空间,实现有效的数据存储和数据备份.本文通过对数据重复删除的关键技术进行分析,找出现有技术的不足,然后加以改进.重复数据删除技术在数据存储领域的应用相当广泛,有效地节省了数据存储的硬盘成本.

Research on duplication deletion based on data storage

Data duplication deletion was called single instance storage or intelligent compression. In order to reduce the consumption of space, first to segment files, then use SHA-1 algorithm for the detection of duplicate data segments, finally, establish data dictionary for the storage of single instance of the data segment and compress it. Thus, a lot of storage space was saved, effective data storage and data backup were implemented. Through the analysis of the key technology of data duplication, it was found the shortage of existing technology, and then improved. This technology was widely used in the area of data storage, brought the revolutionary breakthrough, and it could effectively save the cost of data storage hard disk.