Network Packet De-duplication for your Data Optimization via Nework Packet Broker

Data De-duplication is a popular and popular storage technology that optimizes storage capacity.It eliminates redundant data by removing duplicate data from the dataset, leaving only one copy.As shown in the figure below.This technology can greatly reduce the need for physical storage space to meet the growing demand for data storage.Dedupe technology can bring many practical benefits, mainly including the following aspects:

(1)  Meet ROI(Return On Investment)/TCO(Total Cost of Ownership) requirements;
(2)  The rapid growth of data can be effectively controlled;
(3)  Increase effective storage space and improve storage efficiency;
(4)  Save the total storage cost and management cost;
(5)  Save the network bandwidth of data transmission;
(6)  Save operation and maintenance costs such as space, power supply and cooling.

Mylinking™ Packet Deduplication

Dedupe technology is widely used in data backup and archiving systems, because there are a lot of duplicate data after multiple backups of data, which is very suitable for this technology.In fact, dedupe technology can be used in many situations, including online data, near-line data, and offline data storage systems. It can be implemented in file systems, volume managers, NAS, and sans.Dedupe can also be used for data disaster recovery, data transmission and synchronization, as a data compression technology can be used for data packaging.Dedupe technology can help many applications reduce data storage, save network bandwidth, improve storage efficiency, reduce the backup window, and save costs.

Dedupe has two main dimensions: deduplocation ratios and performance.Dedupe performance depends on the specific implementation technology, while Dedupe rate is determined by the characteristics of the data itself and application patterns, as shown in the table below.Storage vendors currently report deduplication rates ranging from 20:1 to 500:1.

High deduplication rate Low deduplication rate
Data created by the user Data from the natural world
Data low rate of change Data high rate of change
Reference data, inactive data Active data
Low data change rate application High data change rate application
Full data backup Incremental data backup
Data long-term storage Data short-term storage
Wide range of data applications Small range of data applications
Continuous data business processing General data business processing
Small data segmentation Big data segmentation
Elongate data segmentation Fixed length data segmentation
Data content perceived Data content unknown
Time data deduplication Spatial data deduplication

ML-NPB-5660 Packet De-duplication

Dedupe Implementation Points

Various factors should be considered when developing or applying Dedupe technology, as these factors directly affect its performance and effectiveness.

(1)  What  What data are de-weighted?
(2)  When  When will the weight be eliminated?
(3)  Where  Where is the weight elimination?
(4)  How  How to reduce the weight?

Dedupe Key Technology

Deduplication process of storage system in general is this: first of all the data file is divided into a set of data, for each block of data to calculate the fingerprint, and then based on fingerprint Hash search keywords, matching indicates the data for the duplicate data blocks, only stores data block index number, otherwise it means the data block is the only piece of a new, storage of data block and create relevant meta information.Thus, a physical file in the storage system corresponds to a logical representation of a set of FP metadata.When reading the file, first read the logical file, then according to the FP sequence, take out the corresponding data block from the storage system, restore the copy of the physical file.It can be seen from the above process that the key technologies of Dedupe mainly include file data block segmentation, data block fingerprint calculation and data block retrieval.

(1) File data block segmentation

(2) Data block fingerprint calculation

(3) Data block retrieval

To find these recommend models to start your Network Packet Deduplication:

Mylinking™ Network Packet Broker(NPB) ML-NPB-6400  48*10GE SFP+ plus 4*40GE/100GE QSFP28, Max 880Gbps

Mylinking™ Network Packet Broker(NPB) ML-NPB-5660  6*40GE/100GE QSFP28 plus 48*10GE/25GE SFP28, Max 1.8Tbps

Mylinking™ Network Packet Broker(NPB) ML-NPB-5060  48*10GE SFP+ plus 2*40GE QSFP, Max 560Gbps

Mylinking™ Network Packet Broker(NPB) ML-NPB-4860  48*10GE SFP+, Max 480Gbps, Function Plus

Mylinking™ Network Packet Broker(NPB) ML-NPB-4810  48*10GE SFP+, Max 480Gbps

Mylinking™ Network Packet Broker(NPB) ML-NPB-2410P  24*10GE SFP+, Max 240Gbps, DPI Function




Mylinking™ Network Packet Broker(NPB) ML-NPB-6400

48*10GE SFP+ plus 4*40GE/100GE QSFP28, Max 880Gbps

Post time: Oct-18-2022