Technical Note: XTN1901
Overview
End to End Logical Block Protection (LBP) is a feature that validates data and identifies corrupted data transfers. LBP is supported by XenData products running Archive Series software that manage LTO hardware (from LTO6 onwards).
When using Logical Block Protection with the XenData FS Mirror or Workflow API programs to write to XenData managed storage, all data is verified by comparing checksums for a file in its original location and the new XenData managed storage location. If a data transfer utility that does not support LBP, such as Windows File Explorer, is used to copy a file to XenData, Logical Block Protection may still be enabled. However, in this case, LBP will only apply to the transfer between the XenData cache disk and the managed storage i.e. LTO.
Enabling Logical Block Protection
It can be turned on via the XenData Management Console, see image below.
Once it is turned on within the Management Console, it can be used for end to end protection by FS Mirror and XenData Workflow API. This is done by selecting the ‘Use end to end checksum verification’ in FS Mirror (see image below), or by adding a file’s checksum to the archive XML command in Workflow API.
How Does It Work?
When a utility that supports Logical Block Protection copies a file to the XenData managed storage, a 4-byte checksum is generated for each block of data that will be transferred to the managed storage. That is 4 bytes per 512K block of data for LTO tapes written in LTFS, typically 4 bytes per 64KB block of data for LTO tapes written with XenData TAR.
This 4-byte checksum is stored in a file on the cache drive alongside the file’s metadata (see image below). The file has an extension -X-AA-BB, where AA is the file’s generation and BB is the file’s version. The checksums must be kept in a file as the write to tape is not always immediate (pending writes, deferred writes, etc.).
The image below shows the contents of the above file, the highlighted section shows the first 4-byte checksum.
For every block of data transferred to tape, the 4 bytes is also transferred. When the data arrives on the final storage medium, a checksum is created and checked against the extra 4 bytes transferred with the data. Below is a SCSI trace showing the extra 4 bytes on an LTFS 524288 block of data.
It is a very rare event that the checksums do not match. If this ever occurs, a write error will be reported, the file will not be written to LTO and therefore will not be flushed from XenData disk cache. The LTO volume will go into an Alert state, preventing further files being written. In addition, error messages will be put in the Windows Event Log and emailed, if you have the XenData Alert Module configured.
Applicable Operating Systems
- Windows Server 2012 R2
- Windows Server 2016
- Windows Server 2019
Applicable XenData Software
This technical note is applicable to:
- Version 6 – Server Editions of Archive Series software
- Version 7 – Server Editions of Archive Series software including the Cloud File Gateway
PBS
XenData’s fully supported clustered server meets all the requirements for our disaster recovery site. We have been very pleased with the SX-525 product and XenData’s technical team who have provided system design and integration services above and beyond expectations.
James Cutright, Director, Project Management
PBS
Crew Cuts
I did my research, and the best archive solution was from XenData which manages a robotic LTO library behind the scenes. This helps to minimize our total costs and allows us to focus on producing great content for our customers.
Chris Keenan, CTO
Crew Cuts
Ideal Systems Singapore
We partnered with XenData for the GlobeCast Broadcast facility rollout, because it offers one of the most cost effective, robust digital video archives on the market and a great migration alternative to some of the more expensive digital video archive vendor offerings.
Fintan Mc Kiernan, CEO
Ideal Systems Singapore
North Shore Automation
Having completed over 25 joint installations of XenData archive systems, we have found everyone on the XenData team to be both highly professional and super responsive and their technology to be rock solid. They are truly a pleasure to work with.
Bryson Jones, Founder
North Shore Automation
This is Just a Test
With over 558 TB of content and growing, conventional backup solutions just don’t work for us. Our XenData system allows us to maintain an ongoing mirrored copy of our shared storage while creating replica LTO copies for offsite storage.
Nathan Flom
This is Just a Test
Excelencia
We have now installed over 170 XenData archive systems across Mexico and they have proven to be very reliable. In addition, the technical support from XenData has been outstanding.
Jorge Castaneda, President
Excelencia en Comunicaciones y Tecnología