any LTO Ultrium users? (& what they do if HD cant write that fast)

[Twice_Shy]

I think I posted once in the past six months ago about other LTO users and got scarce responses... if anyone can suggest forums/places to go to better discuss this i'm all ears. I don't know where else to ask or to join to ask.

I'm planning on building a server dedicated to writing LTO6 tapes which are designed for 160MB/second. My understanding is that such tape drives are capable of adjusting themselves slower to avoid a failed backup situation in the same way that CDR and DVD writers have for quite some time.

However I don't think they are designed to stay at a low rate the whole time. There isn't that much publically available data on this topic, only offhand comment I can find: LTO tape vendors announce LTO-9 and LTO-10
Opinions vary on new LTO generations

When LTO-6 was announced in 2010, W. Curtis Preston noted that he liked that transfer rates were not increasing at such a rapid rate, as they had with previous generations. "My only problem with tape is that it's too fast. When you slow down, you get shoeshining; you wear out your tapes, you wear out your drives, you get backup failures, etc.," he said."

I dont even know what that means. I'd like to find somebody who does. There may be workarounds (like simply copying data to a fast dedicated drive or RAID0 pair) but i'd rather know about them before kitting out my intended server as it may require me to alter things slightly to accomodate it.

 

[Evan]

LTO will throttle to a point but even then it’s still often too fast for the data being fed, usually the backup server has a staging pool in the form of fast disks that fill first then is used to de-stage to tape.

If it’s local backup... faster disk is the only option or change the type of backup from files to image style backup where the system can stream faster.

 

[Twice_Shy]

Any idea how much is to be of concern? Will it actually wear out the tape faster?

For some reason on all the tape selling sites I always see comments that "half height drives may not back up as fast as full height drives" though I dont normally see any handy comments on how fast any of them DO back up.

My originally planned drive was going to be some Seagate 3tbs externals originally for USB 2.0 that I decase and use as work drives - i'm assuming the initial feed speed might be 120MB/sec and probably slows at the end of the drive to like 80MB/sec. I haven't pulled one and tested it that's just a random guess. Even if it were a bit faster or didn't drop off as much if it's a situation where drive wear actually INCREASES I might want to design around that. (like a RAID0 pair or a faster newer drive) I'm curious whether the tapes would still write as they are though or if large SSD caches could be set up or what. I'm wondering how much 'speed overage' I should design for since things like smaller files don't transfer as fast as large ones (assuming I want to use LTFS which I do) or if i'm just overworrying the speed issue and it should work fine just taking twice as long to finish some tapes.

EDIT: Like here I guess I found another comment on the issue, How fast should my LTO tape access be? : Archiving and Back-Up "For LTO-5, your source disk must be able to maintain a sustained read rate of 150MB/sec. For LTO-6, your source disk must be able to maintain a sustained read rate of 190MB/sec to 200MB/sec. Anything less and your tape drive will "shoe shine" or suffer from buffer underruns while writing. This shoe shining will slow the process further since the tape drive must reposition the tape to start writing again after the buffer catches up." Yet another user afterwards posts the useful tidbit making me think it may be okay:
"I don't know about IBM drives, but from what we'd been advised by HP, their drives and Quantum drives support variable write speeds and do not start shoeshining until data rates fall off severely. Quoted values of data rate matching speeds are 46.7–140MB/s for LTO-5 and 54-160 MB/s for LTO-6."

The above makes more sense to me, in the sense that I couldnt understand why a drive slowing down should be a problem inherently/running tape slower is not MORE wearing... unless it causes some big compensatory whatever it is up there that I guess is called shoeshining. Yet I see no problem maintaining 54MB/sec with even my lowly 3tb drives, just expecting a longer write time obviously. I'm okay with that. (at the start) It may be highly drive specific/not all tape drives like variable rates as much too.

I guess i'm rambling a little publically just not sure where else to do it. I'm a tape noob... just trying to figure out what specific terms and things I need to research possibly in addition to the above minimums... such as how to buffer multiple small files going to LTFS and how many it involves before it turns into a problem.

 

[Evan]

I had in mind 60-70 MB/s they will do on, of course faster is better.
A single disk will work but only if it’s an image backup or very large files, small files and your not going anywhere.

 

[Twice_Shy]

Can an image backup be made that makes directly readable files under LTFS?

Sort of like most of us used to burn CD's and DVD's - where first you make an image, then it just has to keep up with the image. Could I create a 'tape image' that writes to tape nonstop, but is already in the LTFS format? If so what software would do this? I'm sure I can create a monolithic tarred type file but my goal (just like with a CDR) is not a 'backup file' but directly readable files at least much of the time.

Most of my large files will be video files when they are. But raw workdata can have various small metadata files that are a part of it.

 

[Evan]

Hmm,
Probably just have to try and see if the small files slow it down too much.
If you do an image backup the it’s not much point using LTFS. I have only every used LTFS as a way to move bulk data around, never tried it for a pure backup solution.

 

[Twice_Shy]

Hmm,
Probably just have to try and see if the small files slow it down too much.
If you do an image backup the it’s not much point using LTFS. I have only every used LTFS as a way to move bulk data around, never tried it for a pure backup solution.

What I mean is, is there any way to write something as a single monolithic image, that could be read by LTFS?

DVD/CDroms have worked this way forever - you put all the tiny little files (with their slow accesses) into an ISO - and write that ISO somewhere. Then it reads as a monolithic file and writes to disc. But when you load the disc back you can pick and choose any files off the drive.

I'd like to do the same thing with Ultrium if I could - that would seem one way around filling a tape with at least some smaller files, to prevent hiccups in speed.


Also does anyone know what happens if an LTFS system has a single bit nonrecoverable error in the middle of say some ginormous 2terabyte file? Do I lose the whole 2tera file or would it write me a file that just has one corrupt bit? (which for instance could be repaired using usenet tools like PAR - but only if I have a file, on hard disk, for it to inspect and repair)