Discussion:
Achival storage
(too old to reply)
Roy
2015-03-20 21:18:11 UTC
Permalink
I was in a discussion today on archival storage.

Someone has a large amount of data (several TB) and loads it onto an
external storage device. The device sits on a shelf in a closet for X
years. Is SSD or a regular disk better for this?
David Kaye
2015-03-21 01:53:34 UTC
Permalink
Post by Roy
I was in a discussion today on archival storage.
Someone has a large amount of data (several TB) and loads it onto an
external storage device. The device sits on a shelf in a closet for X
years. Is SSD or a regular disk better for this?
From what I know so far, SSD is far superior to HD because HD is mechanical
and thus can fail more easily. I've been able to recover frozen HDs,
however, by warming them in the afternoon sun or in an oven that has a pilot
light -- enough to make the lubricating oil softer.

But SSD has it over HD far and away because the bits stay flipped, as far as
I know. SSD's main failure as far as I know is with thousands of erasures
and rewrites. Eventually it loses its ability to take on a new write. But
if you just record onto SSD once and leave it on a shelf, as far as I know,
it will outlast any other recording medium.

Of course, only the test of time will bear this out, but I have some very
early 512kb USB "thumb drives" that haven't lost anything. And that was
back in 2001, shortly after they were introduced. that's 14 years.
Meanwhile, I've had R-W CDs fail within a year or two.




---
This email has been checked for viruses by Avast antivirus software.
http://www.avast.com
John Slade
2015-03-21 18:20:43 UTC
Permalink
Post by Roy
I was in a discussion today on archival storage.
Someone has a large amount of data (several TB) and loads it
onto an external storage device. The device sits on a shelf in
a closet for X years. Is SSD or a regular disk better for this?
I agree with David that SSDs are now the best way to go
if you can afford it. Even though SSDs are coming down in price,
they still cost about three times the price of an enterprise HD.
If you must use HDs you can archive data on them for years but
you have to power them up every so often to keep them working.
You should also transfer to new drives every three years. I
would recommend buying enterprise drives only they usually come
with a five-year warranty.

The way prices are coming down for SSDs you probably have
from 3-5 years before they are close to current mechanical HD
prices. They may get to that price point even sooner.

John


---
This email has been checked for viruses by Avast antivirus software.
http://www.avast.com
Roy
2015-03-21 18:44:21 UTC
Permalink
Post by Roy
I was in a discussion today on archival storage.
Someone has a large amount of data (several TB) and loads it
onto an external storage device. The device sits on a shelf in
a closet for X years. Is SSD or a regular disk better for this?
I agree with David that SSDs are now the best way to go if you
can afford it. Even though SSDs are coming down in price, they still
cost about three times the price of an enterprise HD. If you must use
HDs you can archive data on them for years but you have to power them up
every so often to keep them working. You should also transfer to new
drives every three years. I would recommend buying enterprise drives
only they usually come with a five-year warranty.
The way prices are coming down for SSDs you probably have from 3-5
years before they are close to current mechanical HD prices. They may
get to that price point even sooner.
John
I used SSD for any solid state drive. In this case, we will probably
use USB Flash sticks. There is some difference in the archival of a
Flash drive and the electronics of an SSD Drive. I should have made the
question more definite.
Julian Macassey
2015-03-21 20:19:38 UTC
Permalink
Post by John Slade
Post by Roy
I was in a discussion today on archival storage.
Someone has a large amount of data (several TB) and loads it
onto an external storage device. The device sits on a shelf in
a closet for X years. Is SSD or a regular disk better for this?
I agree with David that SSDs are now the best way to go
if you can afford it. Even though SSDs are coming down in price,
they still cost about three times the price of an enterprise HD.
If you must use HDs you can archive data on them for years but
you have to power them up every so often to keep them working.
You should also transfer to new drives every three years. I
would recommend buying enterprise drives only they usually come
with a five-year warranty.
The way prices are coming down for SSDs you probably have
from 3-5 years before they are close to current mechanical HD
prices. They may get to that price point even sooner.
Also bear in mind that file formats change over time, you may
end up with archived data and no way to read it.

There are businesses with rooms full of equip to read and
transfer all that data your recorded on things like optical disks,
punched tape etc.
--
Hipsters have ruined everything. - Tim May April 5 2014
(null)
2015-03-22 06:34:51 UTC
Permalink
Post by Julian Macassey
Also bear in mind that file formats change over time, you may
end up with archived data and no way to read it.
Virtual machines alleviate that burden somewhat. TRS-80 emulator, anyone?
John Slade
2015-03-25 00:00:46 UTC
Permalink
Post by Julian Macassey
Post by John Slade
Post by Roy
I was in a discussion today on archival storage.
Someone has a large amount of data (several TB) and loads it
onto an external storage device. The device sits on a shelf in
a closet for X years. Is SSD or a regular disk better for this?
I agree with David that SSDs are now the best way to go
if you can afford it. Even though SSDs are coming down in price,
they still cost about three times the price of an enterprise HD.
If you must use HDs you can archive data on them for years but
you have to power them up every so often to keep them working.
You should also transfer to new drives every three years. I
would recommend buying enterprise drives only they usually come
with a five-year warranty.
The way prices are coming down for SSDs you probably have
from 3-5 years before they are close to current mechanical HD
prices. They may get to that price point even sooner.
Also bear in mind that file formats change over time, you may
end up with archived data and no way to read it.
I had a brush with this type of thing. I used and ATI TV
card to record shows I wanted to archive in .vcr format. It's
now hard to find a player that will play those files. They do
have a conversion program in the ATI software suite but you need
an actual card to install it. I found a workaround but for those
who don't know it could be hard.

John



---
This email has been checked for viruses by Avast antivirus software.
http://www.avast.com
David Kaye
2015-03-25 08:45:20 UTC
Permalink
I had a brush with this type of thing. I used and ATI TV card to
record shows I wanted to archive in .vcr format. It's now hard to find a
player that will play those files. They do have a conversion program in
the ATI software suite but you need an actual card to install it. I found
a workaround but for those who don't know it could be hard.
This is the main reason I like flash drives. I think the USB standard will
be around for a long time, and given that various versions of Windows are
able to set up a quick driver to read them I'm assuming that the driver is
fairly simple. Unlike hardware with moving parts, USB sticks are mostly
software instead of hardware.




---
This email has been checked for viruses by Avast antivirus software.
http://www.avast.com
John Slade
2015-03-25 20:57:26 UTC
Permalink
Post by David Kaye
I had a brush with this type of thing. I used and ATI TV card to
record shows I wanted to archive in .vcr format. It's now hard to find a
player that will play those files. They do have a conversion program in
the ATI software suite but you need an actual card to install it. I found
a workaround but for those who don't know it could be hard.
This is the main reason I like flash drives. I think the USB standard will
be around for a long time, and given that various versions of Windows are
able to set up a quick driver to read them I'm assuming that the driver is
fairly simple. Unlike hardware with moving parts, USB sticks are mostly
software instead of hardware.
I like USB 3.0. It's pretty fast and backwards compatible
with previous USB versions. I use it for my HDs because the
speed is pretty close to the SATA speeds I get.

John



---
This email has been checked for viruses by Avast antivirus software.
http://www.avast.com
(null)
2015-03-25 21:41:21 UTC
Permalink
Post by John Slade
I like USB 3.0. It's pretty fast and backwards compatible
with previous USB versions. I use it for my HDs because the
speed is pretty close to the SATA speeds I get.
For now. The underlaying technology in 3.0 is different than 1.1/2.0
(NRZI half-duplex diff pair vs. 8b10b serdes) and hopefully the USB-IF won't
change their minds about backwards compatibility some time down the line.
At the very least, you'll likely need a USB-C to USB-A/B adapter in the future.
John Slade
2015-03-26 18:32:17 UTC
Permalink
Post by (null)
Post by John Slade
I like USB 3.0. It's pretty fast and backwards compatible
with previous USB versions. I use it for my HDs because the
speed is pretty close to the SATA speeds I get.
For now. The underlaying technology in 3.0 is different than 1.1/2.0
(NRZI half-duplex diff pair vs. 8b10b serdes) and hopefully the USB-IF won't
change their minds about backwards compatibility some time down the line.
At the very least, you'll likely need a USB-C to USB-A/B adapter in the future.
I'm not even concerned with USB-C now and it will
probably go the way of Thunderbolt. The regular, mini and micro
USB connectors seem to work well and we really don't need USB-C
for anything.

John

---
This email has been checked for viruses by Avast antivirus software.
http://www.avast.com
David Kaye
2015-03-26 20:27:41 UTC
Permalink
I'm not even concerned with USB-C now and it will probably go the
way of Thunderbolt. The regular, mini and micro USB connectors seem to
work well and we really don't need USB-C for anything.
I don't think they're going away because (1) they're universally
recognizable (even the least tech literate person knows what a USB looks
like even if they don't know VGA or mini phone plugs), and (2) they're easy
to fit in tight spaces and can provide more power than mini and micros.




---
This email has been checked for viruses by Avast antivirus software.
http://www.avast.com
John Slade
2015-03-31 16:59:09 UTC
Permalink
Post by David Kaye
I'm not even concerned with USB-C now and it will probably go the
way of Thunderbolt. The regular, mini and micro USB connectors seem to
work well and we really don't need USB-C for anything.
I don't think they're going away because (1) they're universally
recognizable (even the least tech literate person knows what a USB looks
like even if they don't know VGA or mini phone plugs), and (2) they're easy
to fit in tight spaces and can provide more power than mini and micros.
I don't think USB-C will go away totally, I think it will
be like Thunderbolt. Thunderbolt will probably find a niche in
the enterprise hardware world. Only way it will really be needed
for the consumer hardware market is if storage device speed
increases. As it is now USB 3.0 is about the limit of what
consumer hardware devices can do. The ultimate judge will be the
consumers, if they find a reason to drop all their USB 3.0 and
below and decide they want to move to USB-C, then it will flourish.

John


---
This email has been checked for viruses by Avast antivirus software.
http://www.avast.com
d***@81.usenet.us.com
2015-03-24 01:37:04 UTC
Permalink
Post by Roy
Someone has a large amount of data (several TB) and loads it onto an
external storage device. The device sits on a shelf in a closet for X
years. Is SSD or a regular disk better for this?
Meager data points: I have a pair of Maxtor 190 ST506 disk drives that
haven't been powered on in 20 years. I could give them a test spin, just
for you ;-) My wife would like for me to throw out the ~1985 computer they
are in, but I was part of the build team, and I just can't let it go.

I had a spinning disk drive that had some weak spots. I believe, after
several rounds of reformatting and failure, that there were spots that
would not hold their bits. I could write test for days, no errors. But
after a few months of use, there were read errors that could not be
recovered in a read/swap-to-spare-block scenario, as if the bits that used
to be there were just bad, now.

I read an article recently that cheap SSD drives are prone to startlingly
complete and unexpected failure.

I had two different Corsair 16 GB sticks that just failed... worked for a
while, not used much, appeared as USB devices, but no mount, and then
completely dead minutes later.


Multiple copies are the only way to help ensure long term safety.
I used to back up three months worth of archive each month to IOMega 100
MB disks. Now, if I could just find some place to plug in my Parallel
port to see if they are still readable ... ;-)
--
Clarence A Dold - Santa Rosa, CA, USA GPS: 38.47,-122.65
David Kaye
2015-03-24 01:42:58 UTC
Permalink
Post by d***@81.usenet.us.com
I read an article recently that cheap SSD drives are prone to startlingly
complete and unexpected failure.
Well, there is the tendency for USB flash drives to fail to write after
thousands of write/reads, but this question was about archiving, which would
basically be a write-once or twice situation. I'm assuming that SSD drives
are similar technology to USB flash drives and thus would be prone to a
similar failure after thousands of writes.

But less than that, well, it gives one pause to think about going back to
the old ink and paper model...




---
This email has been checked for viruses by Avast antivirus software.
http://www.avast.com
(null)
2015-03-24 13:48:07 UTC
Permalink
Post by David Kaye
I'm assuming that SSD drives
are similar technology to USB flash drives and thus would be prone to a
similar failure after thousands of writes.
Well, there's the whole MLC vs. SLC thing that differentiates them.
b***@MIX.COM
2015-03-24 15:49:35 UTC
Permalink
Post by (null)
Well, there's the whole MLC vs. SLC thing that differentiates them.
Yes, 4x the cost buys about 2x the static data retention time.
See, for example -

[Swissbit]
http://preview.tinyurl.com/nxw9xmx

Billy Y..
--
sub #'9+1 ,r0 ; convert ascii byte
add #9.+1 ,r0 ; to an integer
bcc 20$ ; not a number
b***@MIX.COM
2015-03-24 14:49:23 UTC
Permalink
Post by d***@81.usenet.us.com
Multiple copies are the only way to help ensure long term safety.
You can never have too many backups.

Heh.

Billy Y..
--
sub #'9+1 ,r0 ; convert ascii byte
add #9.+1 ,r0 ; to an integer
bcc 20$ ; not a number
Jeff Liebermann
2015-03-24 16:28:32 UTC
Permalink
Post by b***@MIX.COM
Post by d***@81.usenet.us.com
Multiple copies are the only way to help ensure long term safety.
You can never have too many backups.
Quantity is at best a poor substitute for quality.

I once had a customer that ran two sequential backups and two verify
passes each night. Don't ask why, I don't know. That wore out their
tape heads 4 times as fast. When it was necessary to restore some of
his data, I found that it could only be read on the drive with the
worn out heads. Also, none of his older backup tapes, which were made
when the drive had a different wear pattern, were readable. They were
not very thrilled when I told them that they didn't have any backups.

Somene's solution was a 2nd server, running as a mirror in another
part of the building. Backups were delayed by one day to prevent
backing up corrupted data on the assumption that it would be caught
within one day. That didn't work over a weekend, and they ended up
with two identical servers, both full of useless corrupted data.

Backing up petabytes is a topic often discussed under "big data" and
Hadoop. I don't have any such customers, so I'm very much out of date
on what's fashionable in todays backups and archival storage.
--
Jeff Liebermann ***@cruzio.com
150 Felker St #D http://www.LearnByDestroying.com
Santa Cruz CA 95060 http://802.11junk.com
Skype: JeffLiebermann AE6KS 831-336-2558
poldy
2015-03-24 20:10:14 UTC
Permalink
Post by Jeff Liebermann
Post by b***@MIX.COM
Post by d***@81.usenet.us.com
Multiple copies are the only way to help ensure long term safety.
You can never have too many backups.
Quantity is at best a poor substitute for quality.
I once had a customer that ran two sequential backups and two verify
passes each night. Don't ask why, I don't know. That wore out their
tape heads 4 times as fast. When it was necessary to restore some of
his data, I found that it could only be read on the drive with the
worn out heads. Also, none of his older backup tapes, which were made
when the drive had a different wear pattern, were readable. They were
not very thrilled when I told them that they didn't have any backups.
Somene's solution was a 2nd server, running as a mirror in another
part of the building. Backups were delayed by one day to prevent
backing up corrupted data on the assumption that it would be caught
within one day. That didn't work over a weekend, and they ended up
with two identical servers, both full of useless corrupted data.
Backing up petabytes is a topic often discussed under "big data" and
Hadoop. I don't have any such customers, so I'm very much out of date
on what's fashionable in todays backups and archival storage.
Just saw an episode of The Americans, set in the '80s, where a girl goes
to the public library to read old newspapers and magazines in microfiche
form. Not sure if public libraries still offer that.

But people are scanning old film negatives, though no longer using film
for photography, while movie studios convert old film masters to digital.

But are those bits going to last or are they as vulnerable as film or
old books which have to be kept in climate-controlled vaults?
Roy
2015-03-24 21:56:36 UTC
Permalink
Post by poldy
...
Just saw an episode of The Americans, set in the '80s, where a girl goes
to the public library to read old newspapers and magazines in microfiche
form. Not sure if public libraries still offer that.
But people are scanning old film negatives, though no longer using film
for photography, while movie studios convert old film masters to digital.
But are those bits going to last or are they as vulnerable as film or
old books which have to be kept in climate-controlled vaults?
They are as vulnerable BUT the cost of reproduction is very low. If you
take an old film master and digitize it, you can then have a thousand
copies in a jiffy. Each copy can spawn another thousand.

New ba.internet motto:

You can never have too many backups.

Just remember to geographically separate them
David Kaye
2015-03-25 08:36:41 UTC
Permalink
Post by Roy
Just remember to geographically separate them
Archive.org has servers in SF, in I think Arizona or Palo Alto (I forget)
and in Alexandria Egypt of all places. Sort of a nod to the great library
of Alexandria, I guess.




---
This email has been checked for viruses by Avast antivirus software.
http://www.avast.com
Jeff Liebermann
2015-03-24 23:49:55 UTC
Permalink
Post by poldy
But are those bits going to last or are they as vulnerable as film or
old books which have to be kept in climate-controlled vaults?
Yes, because they are so easy to transfer from old digital media to
new digital media. In the future, we'll probably be using 3D data
cubes that store over-kill-o-Bytes of data at FTL (faster than light)
access times. Well, maybe just the 3D data cubes. Copying old
digital data from WORM, tape, disk, platter, etc only needs to be done
once to preserve it for the next generation of data storage gluttons.
I suspect the problem will not be storage or storage life, but rather
finding the bloody data among yotabytes of cat photos, porn, selfies,
blogs, cached advertising, and chain letters that seem to monopolize
most users storage.
--
Jeff Liebermann ***@cruzio.com
150 Felker St #D http://www.LearnByDestroying.com
Santa Cruz CA 95060 http://802.11junk.com
Skype: JeffLiebermann AE6KS 831-336-2558
(null)
2015-03-25 06:40:34 UTC
Permalink
Post by Jeff Liebermann
I suspect the problem will not be storage or storage life, but rather
finding the bloody data among yotabytes of cat photos, porn, selfies,
blogs, cached advertising, and chain letters that seem to monopolize
most users storage.
You know, I'll bet you could come up with a solid business model around
a good search algorithm that helps in "finding the bloody data among yotabytes".
You might even make enough money to fund weird stuff like smartphone
operating systems and self-driving cars :-)
Jeff Liebermann
2015-03-25 15:58:01 UTC
Permalink
Post by (null)
Post by Jeff Liebermann
I suspect the problem will not be storage or storage life, but rather
finding the bloody data among yotabytes of cat photos, porn, selfies,
blogs, cached advertising, and chain letters that seem to monopolize
most users storage.
You know, I'll bet you could come up with a solid business model around
a good search algorithm that helps in "finding the bloody data among yotabytes".
I already have such a system. Not only will it find things among
yotabytes of junk, but is also infinitely expandable. I use it for
both my computer organization and my paper filing system. I sort
everything by date. First in, First out. If I know when something
happened, I can find it. For example, I couldn't recall the name of
the worthless backup software vendor. However, I could recall when it
happened. Digging through my chronological email tree, I found the
date (mid 2007), which soon found the vendors name, product name,
versions, etc. This doesn't work for every type of search, but is
good enough for most of what I need.
Post by (null)
You might even make enough money to fund weird stuff like smartphone
operating systems and self-driving cars :-)
My smartphone is old, dumb, and prone to crashing Rotomola Droid-X2.
I'm debating if I should get a newer device, or just do without. For
talking, I use an ancient LG VX-8300 handset. If I want a self
driving vehicle, I'll take the bus.

High technology is for my customers. I prefer things that actually
work.
--
Jeff Liebermann ***@cruzio.com
150 Felker St #D http://www.LearnByDestroying.com
Santa Cruz CA 95060 http://802.11junk.com
Skype: JeffLiebermann AE6KS 831-336-2558
David Kaye
2015-03-25 20:00:03 UTC
Permalink
Post by Jeff Liebermann
High technology is for my customers. I prefer things that actually
work.
Still got my dumbphone. While my friends are bitching about getting just 2
or 3 hours on a charge, I mentioned that I get 3 days on my charges. Well,
I used to tell them that; they don't want to hear it anymore...



---
This email has been checked for viruses by Avast antivirus software.
http://www.avast.com
Peter Lawrence
2015-03-25 22:30:31 UTC
Permalink
Post by David Kaye
Post by Jeff Liebermann
High technology is for my customers. I prefer things that actually
work.
Still got my dumbphone. While my friends are bitching about getting just 2
or 3 hours on a charge, I mentioned that I get 3 days on my charges. Well,
I used to tell them that; they don't want to hear it anymore...
Then they screwed up and damaged their smartphone batteries. At a minimum,
even with heavy use, a smartphone should be able to run on its battery for
at least 10 hours. And those are the worst ones.

A well designed smartphone with a large battery should easily last 18 hours
if not 24 hours on a single charge *under heavy use*.

Under moderate use, some of my smartphones will last at least 36 hours on a
single charge. That's almost three days. They might last longer, but I
don't want to drain their batteries under 15% of capacity.


- Peter
Travis James
2015-03-30 03:22:17 UTC
Permalink
Post by Peter Lawrence
Under moderate use, some of my smartphones will last at least 36 hours
on a single charge. That's almost three days. They might last
longer, but I don't want to drain their batteries under 15% of capacity.
- Peter
Yes, 36 hours, almost 3 days.
Peter Lawrence
2015-03-30 06:24:22 UTC
Permalink
Post by Travis James
Post by Peter Lawrence
Under moderate use, some of my smartphones will last at least 36 hours on
a single charge. That's almost three days. They might last longer, but
I don't want to drain their batteries under 15% of capacity.
- Peter
Yes, 36 hours, almost 3 days.
Sorry about the error, I meant to say at least 60 hours. Don't know why I
typed in 36. Basically about 2 1/2 days with moderate use.


- Peter
Travis James
2015-04-01 04:09:01 UTC
Permalink
Post by Peter Lawrence
Post by Travis James
Post by Peter Lawrence
Under moderate use, some of my smartphones will last at least 36 hours on
a single charge. That's almost three days. They might last longer, but
I don't want to drain their batteries under 15% of capacity.
- Peter
Yes, 36 hours, almost 3 days.
Sorry about the error, I meant to say at least 60 hours. Don't know
why I typed in 36. Basically about 2 1/2 days with moderate use.
- Peter
The only smarthone I could stretch about that far was a Palm Pixi. My
4S will go all day with well over 50% when I'm working at the company
office -- and not playing with my phone. :-)
Peter Lawrence
2015-03-25 22:58:48 UTC
Permalink
Post by Jeff Liebermann
My smartphone is old, dumb, and prone to crashing Rotomola Droid-X2.
I'm debating if I should get a newer device, or just do without. For
talking, I use an ancient LG VX-8300 handset. If I want a self
driving vehicle, I'll take the bus.
A lot of good deals on unlocked smartphones are available nowadays. Now
would be a good time to upgrade your smartphone to an unlocked variety which
would enable you to use your smartphone with some very low cost cellular
service providers.

Amazon often puts their unlocked 32 GB Fire Phone on sell for $200 (though
right now they are out-of-stocked of them so the unlocked Fire Phone is
being sold at its regular $449 price with a three to four week wait).

Motorola has just released unlocked versions of their 2nd generation Moto E
Android smartphone with the 8GB 4G LTE capable one selling for $149 and the
8GB 3G versions selling for $119. They run on the latest Andorid O.S.,
Version 5.0 (Lollipop).

And one can easily find unlocked versions of Microsoft's (formerly Nokia's)
Lumia Window Phones for sale for under $100.

The Amazon, Motorola Moto series, and Microsoft Lumia smartphones are all
quality phones with respectable battery life. And the Fire O.S. that the
Amazon Fire Phone runs on and the Windows Phone 8.1 (or 8.0) O.S. that the
Microsoft Lumia smartphones run on are both very stable and not prone to
crashing.


- Peter
Jeff Liebermann
2015-03-26 02:44:42 UTC
Permalink
Post by Peter Lawrence
Post by Jeff Liebermann
My smartphone is old, dumb, and prone to crashing Rotomola Droid-X2.
I'm debating if I should get a newer device, or just do without. For
talking, I use an ancient LG VX-8300 handset. If I want a self
driving vehicle, I'll take the bus.
A lot of good deals on unlocked smartphones are available nowadays. Now
would be a good time to upgrade your smartphone to an unlocked variety which
would enable you to use your smartphone with some very low cost cellular
service providers.
I guess I should mention that I don't need a smartphone. I have a
Google Nexus 7 tablet (wi-fi only) that does the smart stuff. I have
a cheap junk phone for the talking. I don't want to pay for cellular
data minutes (because I'm cheap). My cell phone bill averages about
$20/month (PagePlus).

However, I may now be forced to get a new phone. Verizon was nice
enough to plant a new cell site at the Ben Lomond fire department.
However, it seems that there's no 800/1900 MHz CDMA service at the new
site. Only VoLTE (voice of LTE), also known as Advanced Calling 1.0
or HD Voice. None of my phones will do LTE or 700 MHz, do I don't get
to use the new site. AT&T did exactly the same thing at the nearby CO
(central office) so I'm doubly screwed. Verizon also announced that
they're moving 1900 MHz systems to LTE.

I'm not 100.0% sure that this is actually true because it came 3rd
hand from one of the volunteers at the BLFD, who doesn't really know
anything about the cell site. It might be that VZW just forgot to
turn on CDMA. I'm checking, but can't seem to get anyone at Verizon
to provide accurate info. Stay tuned.
Post by Peter Lawrence
Amazon often puts their unlocked 32 GB Fire Phone on sell for $200 (though
right now they are out-of-stocked of them so the unlocked Fire Phone is
being sold at its regular $449 price with a three to four week wait).
(...)
Post by Peter Lawrence
Microsoft Lumia smartphones run on are both very stable and not prone to
crashing.
I have a friend that just dropped his Lumia something and clobbered
the screen. I could probably get it cheap. I paid about $50 for my
Droid-X2. Somehow, I don't see myself spending 10 times that for the
latest greatest. Also, provisioning such a phone requires that I
subscribe to VZW cellular data service, which I neither need or want
to pay for. When VoLTE is more ubiquitous on "feature phones", I'll
probably get one of those instead of a smartphone. At this time, the
only VoLTE phone I've tried is an iPhone6. The new 13Kbits/sec CODEC
audio quality is MUCH better than the old 8Kbits/sec CODEC. Place
your bets on how long that will last.
--
Jeff Liebermann ***@cruzio.com
150 Felker St #D http://www.LearnByDestroying.com
Santa Cruz CA 95060 http://802.11junk.com
Skype: JeffLiebermann AE6KS 831-336-2558
Peter Lawrence
2015-03-26 03:00:11 UTC
Permalink
Post by Jeff Liebermann
I have a friend that just dropped his Lumia something and clobbered
the screen. I could probably get it cheap. I paid about $50 for my
Droid-X2. Somehow, I don't see myself spending 10 times that for the
latest greatest. Also, provisioning such a phone requires that I
subscribe to VZW cellular data service, which I neither need or want
to pay for. When VoLTE is more ubiquitous on "feature phones", I'll
probably get one of those instead of a smartphone. At this time, the
only VoLTE phone I've tried is an iPhone6. The new 13Kbits/sec CODEC
audio quality is MUCH better than the old 8Kbits/sec CODEC. Place
your bets on how long that will last.
How is T-Mobile or Sprint cellular service where you live and work? If
either is adequate for your needs, Ting Mobile could provide service for
your smartphone for about the amount your pay for your "dumb phone" service,
maybe even for less money than what you now spend with PagePlus.

https://ting.com/rates


- Peter
Jeff Liebermann
2015-03-26 03:37:11 UTC
Permalink
Post by Peter Lawrence
Post by Jeff Liebermann
I have a friend that just dropped his Lumia something and clobbered
the screen. I could probably get it cheap. I paid about $50 for my
Droid-X2. Somehow, I don't see myself spending 10 times that for the
latest greatest. Also, provisioning such a phone requires that I
subscribe to VZW cellular data service, which I neither need or want
to pay for. When VoLTE is more ubiquitous on "feature phones", I'll
probably get one of those instead of a smartphone. At this time, the
only VoLTE phone I've tried is an iPhone6. The new 13Kbits/sec CODEC
audio quality is MUCH better than the old 8Kbits/sec CODEC. Place
your bets on how long that will last.
How is T-Mobile or Sprint cellular service where you live and work?
T-Mobile is spotty while Sprint is a bad joke. One of the reasons I
picked VZW (PagePlus) is that they have much better coverage in the
San Lorenzo Valley than the other providers. The good news is that
VZW CDMA coverage is good at my house and most of the places I tend to
visit. Only in metropolitan Ben Lomond is there a problem.
Post by Peter Lawrence
If
either is adequate for your needs, Ting Mobile could provide service for
your smartphone for about the amount your pay for your "dumb phone" service,
maybe even for less money than what you now spend with PagePlus.
https://ting.com/rates
Thanks. It looks close. $6/month for the phone. $9/month for
100-500 minutes. I typically run about 400 minutes/month on PagePlus
for 5 cents per minute oor $20/month. With Ting, that would $15/month
or about 3.75 cents/minute. If I can't get VZW to disclose what
they're doing (or not doing), I'll think about switching.
--
Jeff Liebermann ***@cruzio.com
150 Felker St #D http://www.LearnByDestroying.com
Santa Cruz CA 95060 http://802.11junk.com
Skype: JeffLiebermann AE6KS 831-336-2558
David Kaye
2015-03-26 09:28:08 UTC
Permalink
Post by Jeff Liebermann
I guess I should mention that I don't need a smartphone. I have a
Google Nexus 7 tablet (wi-fi only) that does the smart stuff. I have
a cheap junk phone for the talking. I don't want to pay for cellular
data minutes (because I'm cheap). My cell phone bill averages about
$20/month (PagePlus).
I'm in exactly the same boat as you. I have a 7-inch wi-fi Android tablet
which I bought for $50 at Office Max. I have a cheap Verizon Wireless plan
for $35 a month. Though not as cheap as yours I've always had excellent
service with VZW, especially in the nooks and crannies of northern
California, so even though I could save a few dollars, I'd prefer to go with
a carrier that has always treated me well, including accurate bills and live
customer support when I've needed it.






---
This email has been checked for viruses by Avast antivirus software.
http://www.avast.com
(null)
2015-03-26 14:16:31 UTC
Permalink
Post by Jeff Liebermann
I don't want to pay for cellular
data minutes (because I'm cheap).
Have you looked into FreedomPop?
Jeff Liebermann
2015-03-26 16:06:08 UTC
Permalink
Post by (null)
Post by Jeff Liebermann
I don't want to pay for cellular
data minutes (because I'm cheap).
Have you looked into FreedomPop?
Nope. I already have and use Skype, SIP phones, and softphone apps on
my tablet via WiFi or wired ethernet. Nothing new there. Currently,
I have numbers from several private Asterisk servers and subscribe to
Future-Nine.com at $75/year for my office phone. FreedomPop doesn't
work when I can't find a wi-fi hotspot, when I'm on the road, or where
UDP is blocked by the company firewall, such as in some hospitals. No
thanks.
--
Jeff Liebermann ***@cruzio.com
150 Felker St #D http://www.LearnByDestroying.com
Santa Cruz CA 95060 http://802.11junk.com
Skype: JeffLiebermann AE6KS 831-336-2558
(null)
2015-03-26 22:20:31 UTC
Permalink
Post by Jeff Liebermann
FreedomPop doesn't
work when I can't find a wi-fi hotspot, when I'm on the road, or where
UDP is blocked by the company firewall, such as in some hospitals.
I'm a bit confused by this statement. FreedomPop *is* a wi-fi hotspot.
More specifically, a wi-fi hotspot that connects to EVDO or LTE and is
meant for use on the road or where an existing wi-fi hotspot is unacceptable.
Jeff Liebermann
2015-03-27 02:14:36 UTC
Permalink
Post by (null)
Post by Jeff Liebermann
FreedomPop doesn't
work when I can't find a wi-fi hotspot, when I'm on the road, or where
UDP is blocked by the company firewall, such as in some hospitals.
I'm a bit confused by this statement. FreedomPop *is* a wi-fi hotspot.
More specifically, a wi-fi hotspot that connects to EVDO or LTE and is
meant for use on the road or where an existing wi-fi hotspot is unacceptable.
I'll admit that I only did a quick glance at the web pile:
<https://www.freedompop.com/home.htm>
I don't see such a device there. Under "Products", they have various
smartphones. Oh, I see what happened. I didn't scroll down past the
smartphones to find the "hotspot" devices. It's not clear what I'm
paying for.

Hmmm... I can't check their coverage area without also giving them my
email address which they share with literally everyone possible:
<https://www.freedompop.com/privacy.htm#sharingInfo>
Oddly, I don't see any pricing.
Are you sure these guys are reputable?

Digging, I find that they're a Sprint reseller and that they're
thinking of partnering with AT&T and/or T-Mobile:
<http://www.fiercewireless.com/story/sprint-mvno-freedompop-may-partner-att-or-t-mobile/2014-05-07>
If it's only on Sprint, it's useless for my area because of the
general lack of usable coverage in the hills. From my experience,
Sprint users end up roaming onto Verizon and will eventually be asked
to find another cellular service provider if it happens to often. The
Santa Cruz coverage map also looks like fiction:
<http://coverage.sprint.com/IMPACT.jsp?covType=4glte&serviceType=data&mapcity=Santa%20Cruz&mapstate=CA>
Oh, I see:
"Our coverage maps are high-level estimates when using your
device outdoors under optimal conditions."
Right. Works great if I stand on the roof of my house.

I'll take a closer look this weekend. Kinda overloaded right now.
--
Jeff Liebermann ***@cruzio.com
150 Felker St #D http://www.LearnByDestroying.com
Santa Cruz CA 95060 http://802.11junk.com
Skype: JeffLiebermann AE6KS 831-336-2558
Peter Lawrence
2015-03-27 03:10:36 UTC
Permalink
Post by Jeff Liebermann
Post by (null)
Post by Jeff Liebermann
FreedomPop doesn't
work when I can't find a wi-fi hotspot, when I'm on the road, or where
UDP is blocked by the company firewall, such as in some hospitals.
I'm a bit confused by this statement. FreedomPop *is* a wi-fi hotspot.
More specifically, a wi-fi hotspot that connects to EVDO or LTE and is
meant for use on the road or where an existing wi-fi hotspot is unacceptable.
<https://www.freedompop.com/home.htm>
I don't see such a device there. Under "Products", they have various
smartphones. Oh, I see what happened. I didn't scroll down past the
smartphones to find the "hotspot" devices. It's not clear what I'm
paying for.
Hmmm... I can't check their coverage area without also giving them my
<https://www.freedompop.com/privacy.htm#sharingInfo>
Oddly, I don't see any pricing.
Are you sure these guys are reputable?
Digging, I find that they're a Sprint reseller and that they're
<http://www.fiercewireless.com/story/sprint-mvno-freedompop-may-partner-att-or-t-mobile/2014-05-07>
If it's only on Sprint, it's useless for my area because of the
general lack of usable coverage in the hills. From my experience,
Sprint users end up roaming onto Verizon and will eventually be asked
to find another cellular service provider if it happens to often. The
<http://coverage.sprint.com/IMPACT.jsp?covType=4glte&serviceType=data&mapcity=Santa%20Cruz&mapstate=CA>
"Our coverage maps are high-level estimates when using your
device outdoors under optimal conditions."
Right. Works great if I stand on the roof of my house.
I'll take a closer look this weekend. Kinda overloaded right now.
For portable WiFi hotspots the other one you might want to consider is Karma Go:

If you pre-order the device in the next six day, it will cost $99 (otherwise
after it's launched the cost goes up to $149)

The prepaid price for the 4G LTE data $14/GB (with no expiration date of the
prepaid data you purchase) and there are no monthly fees associated with the
service.

https://yourkarma.com/store


- Peter
Jeff Liebermann
2015-03-27 04:06:30 UTC
Permalink
Post by Peter Lawrence
If you pre-order the device in the next six day, it will cost $99 (otherwise
after it's launched the cost goes up to $149)
The 2013 version of the mobile hotspot was $79. See FierceWireless
URL below.
Post by Peter Lawrence
The prepaid price for the 4G LTE data $14/GB (with no expiration date of the
prepaid data you purchase) and there are no monthly fees associated with the
service.
https://yourkarma.com/store
Hmmm.... Wikipedia has them listed as a Sprint WiMax/LTE MVNO.
Probably out of date:
<http://en.wikipedia.org/wiki/List_of_United_States_mobile_virtual_network_operators>

More, also a bit dated:
<http://www.fiercewireless.com/story/social-bandwidth-mvno-karma-crosses-50k-subs-jumps-sprints-lte-network/2013-11-06>
The coverage map on the Karma site looks suspiciously like a combined
Sprint and Verizon 4G coverage map, not just Sprint. Do they have
data roaming? I doubt it. Looks like a direct competitor to
FreedomPop.

I don't really need or want cellular data, but I do have a problem
with voice thanks to Verizon VoLTE. So, if I use a SIP softphone
program on an Android cell phone, I should be able to do some kind of
voice over data. I like to understand people over the phone, so I
prefer G.711 at 87.2 Kbit/sec (one way, including IP headers).
<http://www.voip-info.org/wiki/view/Bandwidth+consumption>
<http://www.asteriskguru.com/tools/bandwidth_calculator.php>
Looks like 160 Kbits/sec total for both directions.

Assuming 1GB means 1GigaByte:
160 Kbits/sec = 0.00001907 GBytes/sec = 52428.8 sec/GByte
= 874 min/GByte.
$14/GByte / 874 min = 1.6 cents/min.
Cheap if I have a free Asterisk server available, but since I pay
someone to terminate the SIP call, my tital costs will be about:
0.5 cents/min call termination
1.6 cents/min Karma service
======
2.1 cents/min
which is cheaper than the 5 cents per minute I'm paying for PagePlus.
Tempting... very tempting, assuming I didn't screw up the math
somewhere.
--
Jeff Liebermann ***@cruzio.com
150 Felker St #D http://www.LearnByDestroying.com
Santa Cruz CA 95060 http://802.11junk.com
Skype: JeffLiebermann AE6KS 831-336-2558
(null)
2015-03-27 04:41:41 UTC
Permalink
FreedomPop is 500MB/month for free + 50MB/month for every "friend" you
connect with (there are wikis and forums with lists of potential "friends").
Just pay for the device, which at one point was $45 from Newegg but
seems to be out of stock now.
http://www.fatwallet.com/forums/hot-deals/1402940/

Oddly enough, I don't pay for 3G coverage yet I seem to be getting it
anyway and I'm not charged for it.
David Kaye
2015-03-25 08:33:48 UTC
Permalink
But are those bits going to last or are they as vulnerable as film or old
books which have to be kept in climate-controlled vaults?
You've literally hit up on the question of the century. While history has
lost movies shot on nitrate film stock (largely to fires and to
disintegation), since the advent of acetate and polyester film stock, there
hasn't been a problem with film being lost. Printed materials that use
carbon black inks tend to last centuries.

But, well, maybe the best way to preserve data today is via cloud backups.
As long as the backup company stays in business (or transfers their files to
another company if they go into bankruptcy), this will probably be the most
viable way to archive of all. Companies that do backups have to keep
changing recording media and they have to keep backing up existing media, so
maybe this is the way.

I have messages in my Yahoo Groups that goes back to when the service was
called eGroups, and that goes back about 20 years by now. So, we know that
Yahoo's cloud backup for their Groups service is reliable. I'd think that
Carbonite, Iron Mountain, and other cloud backup services will be as
reliable as Yahoo.




---
This email has been checked for viruses by Avast antivirus software.
http://www.avast.com
b***@MIX.COM
2015-03-24 20:53:40 UTC
Permalink
Post by Jeff Liebermann
Post by b***@MIX.COM
You can never have too many backups.
Quantity is at best a poor substitute for quality.
Even with properly maintained drives, I'm more comfortable with backup
in depth.
Post by Jeff Liebermann
Backing up petabytes is a topic often discussed under "big data" and
Hadoop. I don't have any such customers, so I'm very much out of date
on what's fashionable in todays backups and archival storage.
In my work, where it's in smaller chunks (movies and tv shows), HDCAM-SR
and LTO (tape) are probably the most widely used.

http://www.tvtechnology.com/insight/0083/examining-the-evolution-of-archives/222715

Google is using Oracle's StreamLine 8500 tape (LTO) systems.

Billy Y..
--
sub #'9+1 ,r0 ; convert ascii byte
add #9.+1 ,r0 ; to an integer
bcc 20$ ; not a number
Jeff Liebermann
2015-03-24 22:00:15 UTC
Permalink
Post by b***@MIX.COM
Post by Jeff Liebermann
Post by b***@MIX.COM
You can never have too many backups.
Quantity is at best a poor substitute for quality.
Even with properly maintained drives, I'm more comfortable with backup
in depth.
Do you verify you backups to see if they're usable? I don't mean a
verify pass immediately after a backup on the same machine that
created the backup, but rather removing the backup media, moving it to
another machine, and trying the recovery a few days or weeks later. I
once gave a client a rather rude wake-up call when I found that NONE
of their backups were usable except on the machine that created them.
Post by b***@MIX.COM
Post by Jeff Liebermann
Backing up petabytes is a topic often discussed under "big data" and
Hadoop. I don't have any such customers, so I'm very much out of date
on what's fashionable in todays backups and archival storage.
In my work, where it's in smaller chunks (movies and tv shows), HDCAM-SR
and LTO (tape) are probably the most widely used.
http://www.tvtechnology.com/insight/0083/examining-the-evolution-of-archives/222715
LTO (linear tape open) is the best tape system for today. LTO-6 holds
2.5TB per tape, with bigger tapes on the way.
<http://en.wikipedia.org/wiki/Linear_Tape-Open#Generations>

Such archives have one advantage over other backups. The data is not
live while it is being backed up. This is in contrast to a large
database that is being read from and possibly written to while the
backup is being performed. Claimed archival lifetimes are allegedly
15-30 years:
<http://en.wikipedia.org/wiki/Linear_Tape-Open#Tape_durability>
"There is a large amount of lifespan variability in actual use"
Yep. Please forgive my cynical attitude, but I've seen far too many
tape systems announced, flounder, and then discarded when the next big
backup thing arrives. At least the problem was recognized so they
made it easy to move DLT libraries to LTO.
Post by b***@MIX.COM
Google is using Oracle's StreamLine 8500 tape (LTO) systems.
Nice.
<http://www.oracle.com/us/products/servers-storage/storage/tape-storage/sl8500-modular-library-system/overview/index.html>
However, it's not something that my customers or I could afford. My
current methodology is an image backup program (Acronis True Image
2014) and various USB 3.0 hard disk drives:
<Loading Image...>
I number the drives sequentially and am now up to drive number 26. The
older drives are 500GB while the current variety are 1 and 1.5TB.
Keeping them organized is a major problem which I haven't solved yet.
--
Jeff Liebermann ***@cruzio.com
150 Felker St #D http://www.LearnByDestroying.com
Santa Cruz CA 95060 http://802.11junk.com
Skype: JeffLiebermann AE6KS 831-336-2558
Roy
2015-03-24 22:27:42 UTC
Permalink
Post by Jeff Liebermann
Post by b***@MIX.COM
Post by Jeff Liebermann
Post by b***@MIX.COM
You can never have too many backups.
Quantity is at best a poor substitute for quality.
Even with properly maintained drives, I'm more comfortable with backup
in depth.
Do you verify you backups to see if they're usable? I don't mean a
verify pass immediately after a backup on the same machine that
created the backup, but rather removing the backup media, moving it to
another machine, and trying the recovery a few days or weeks later. I
once gave a client a rather rude wake-up call when I found that NONE
of their backups were usable except on the machine that created them.
Post by b***@MIX.COM
Post by Jeff Liebermann
Backing up petabytes is a topic often discussed under "big data" and
Hadoop. I don't have any such customers, so I'm very much out of date
on what's fashionable in todays backups and archival storage.
In my work, where it's in smaller chunks (movies and tv shows), HDCAM-SR
and LTO (tape) are probably the most widely used.
http://www.tvtechnology.com/insight/0083/examining-the-evolution-of-archives/222715
LTO (linear tape open) is the best tape system for today. LTO-6 holds
2.5TB per tape, with bigger tapes on the way.
<http://en.wikipedia.org/wiki/Linear_Tape-Open#Generations>
Such archives have one advantage over other backups. The data is not
live while it is being backed up. This is in contrast to a large
database that is being read from and possibly written to while the
backup is being performed. Claimed archival lifetimes are allegedly
<http://en.wikipedia.org/wiki/Linear_Tape-Open#Tape_durability>
"There is a large amount of lifespan variability in actual use"
Yep. Please forgive my cynical attitude, but I've seen far too many
tape systems announced, flounder, and then discarded when the next big
backup thing arrives. At least the problem was recognized so they
made it easy to move DLT libraries to LTO.
Post by b***@MIX.COM
Google is using Oracle's StreamLine 8500 tape (LTO) systems.
Nice.
<http://www.oracle.com/us/products/servers-storage/storage/tape-storage/sl8500-modular-library-system/overview/index.html>
However, it's not something that my customers or I could afford. My
current methodology is an image backup program (Acronis True Image
<http://802.11junk.com/jeffl/crud/backup-drives.jpg>
I number the drives sequentially and am now up to drive number 26. The
older drives are 500GB while the current variety are 1 and 1.5TB.
Keeping them organized is a major problem which I haven't solved yet.
How about Blu-Ray. No worries about head tracking like tape. A
dual-layer disk will hold 50GB.

And more on the way

Goodbye to Blu-Ray, Sony And Panasonic Release 1TB Optical Discs

http://wonderfulengineering.com/goodbye-to-blu-ray-sony-and-panasonic-release-1tb-optical-discs/

http://en.wikipedia.org/wiki/Archival_Disc
Jeff Liebermann
2015-03-24 23:42:29 UTC
Permalink
Post by Roy
How about Blu-Ray. No worries about head tracking like tape. A
dual-layer disk will hold 50GB.
The problem with Blu Ray is write speed and lack of hardware
compression. 1x is 36 Mbits/sec. The fastest claimed is 8x or 288
Mbits/sec. With 2:1 hardware compression, you could get 576 Mbits/sec
= 72 MBytes/sec = 1.2 MBytes/minute. However, unlike tape drives,
which have built in hardware compression, there's no such feature in
Blu Ray drives (unless I missed something) because the Blu Ray platter
is expected to hold pre-compressed digitized video, which would not
beneifit from additional compression. That means the CPU has to work
hard dealing with compression and decompression (for write verify) or
you waste lots of speed and capacity writing uncompressed data.

More detail on speeds:
<http://www.blu-ray.com/faq/#bluray_speed> [1]

By comparison, I'm getting 3-5 MBytes/sec backing up to a portable USB
3.0 hard disk. I suspect that it can go faster when decent USB 3.0 or
the new USB 3.1 chips and devices arrive. Cost per byte is also less
for hard disks than Blu Ray.
Post by Roy
And more on the way
Big data is big business. Whether it actually works will be difficult
to determine until they are fully deployed and the problems emerge.

I'm still waiting for 3D storage.
<http://en.wikipedia.org/wiki/3D_optical_data_storage>
Too bad the storage cube is still science fiction.
Post by Roy
Goodbye to Blu-Ray, Sony And Panasonic Release 1TB Optical Discs
http://wonderfulengineering.com/goodbye-to-blu-ray-sony-and-panasonic-release-1tb-optical-discs/
http://en.wikipedia.org/wiki/Archival_Disc
Thanks. I missed those. I guess big data is now big business.


[1] A violent death to those that randomly abreviate bits and Bytes
without bothering to mumble something about which letter means what.
--
Jeff Liebermann ***@cruzio.com
150 Felker St #D http://www.LearnByDestroying.com
Santa Cruz CA 95060 http://802.11junk.com
Skype: JeffLiebermann AE6KS 831-336-2558
David Kaye
2015-03-25 08:42:45 UTC
Permalink
Post by Jeff Liebermann
Big data is big business. Whether it actually works will be difficult
to determine until they are fully deployed and the problems emerge.
Makes sense. People are packrats, not only with their personal possessions,
but with their data. A customer who had 3 3.5 inch Western Digital backup
drives (1 which failed and 1 which was flaky), asked me what I'd recommend.
I suggested those cute 2.5 inch Seagate USB drives. I use two to back up my
music, video, and documents. Well, over the weekend I had to try to read
the WD drives onto the 3 new Seagates. S-L-O-W. First I had to chkdsk the
drives to make sure I had viable media to read from. Then I started the
backups.

She's a personal chef who publishes, puts out brochures, etc. But MAN, she
is a packrat. She has photo files of food she's prepared going back 15
years. I'm not sure that she'll ever reference them, but at least she has 3
copies in case she wants to.




---
This email has been checked for viruses by Avast antivirus software.
http://www.avast.com
Keith Keller
2015-03-24 23:55:37 UTC
Permalink
Post by Jeff Liebermann
LTO (linear tape open) is the best tape system for today. LTO-6 holds
2.5TB per tape, with bigger tapes on the way.
<http://en.wikipedia.org/wiki/Linear_Tape-Open#Generations>
Such archives have one advantage over other backups. The data is not
live while it is being backed up. This is in contrast to a large
database that is being read from and possibly written to while the
backup is being performed.
I don't understand this statement. The source filesystem is either
live or offline, no matter what the target media is. And you can't
reliably back up a live database from the filesystem (generally) no
matter what the target media is, which is why you make a database dump
first using the database backup tools, and then back up those files
those tools write to the filesystem (which is still "live" but less
likely to be modified during a filesystem backup).

--keith
--
kkeller-***@wombat.san-francisco.ca.us
(try just my userid to email me)
AOLSFAQ=http://www.therockgarden.ca/aolsfaq.txt
see X- headers for PGP signature information
Jeff Liebermann
2015-03-25 00:21:42 UTC
Permalink
On Tue, 24 Mar 2015 16:55:37 -0700, Keith Keller
Post by Keith Keller
Post by Jeff Liebermann
LTO (linear tape open) is the best tape system for today. LTO-6 holds
2.5TB per tape, with bigger tapes on the way.
<http://en.wikipedia.org/wiki/Linear_Tape-Open#Generations>
Such archives have one advantage over other backups. The data is not
live while it is being backed up. This is in contrast to a large
database that is being read from and possibly written to while the
backup is being performed.
I don't understand this statement. The source filesystem is either
live or offline, no matter what the target media is. And you can't
reliably back up a live database from the filesystem (generally) no
matter what the target media is, which is why you make a database dump
first using the database backup tools, and then back up those files
those tools write to the filesystem (which is still "live" but less
likely to be modified during a filesystem backup).
--keith
Sorry, I wasn't too clear. By "live" I meant that it was in use by
some program. For databases, that means that the database file is
open for reading and possibly for writing. I suspect "busy" might be
a better choice of buzzword.

I agree that you can't backup a live/busy database from the same
filesystem. Still, that doesn't stop backup vendors from advertising
products that claim to do an image backup of a Windoze partition while
running Windoze from the same partition. I had two rather nasty
disasters when I actually tried to use such products. I'm trying to
remember the name of the company or product but am failing.

Microsoft has had VSS (Volume Shadow copy Service) for quite some
time:
<https://msdn.microsoft.com/en-us/library/aa384649%28v=vs.85%29.aspx>
It allegedly allows backing up a busy filesystem. It seems to work
well for defrag products, but my luck hasn't been so good with image
backups. The scary part was that one of the failed image backups
passed a verify pass. However, when I checked the restored file list,
every file that was open at the time of the backup was gone on the
restore. Although most backup programs use VSS and it mostly works, I
refuse to deal with image backups that run from Windoze and prefer to
use a Linux boot CD or USB flash thing.
--
Jeff Liebermann ***@cruzio.com
150 Felker St #D http://www.LearnByDestroying.com
Santa Cruz CA 95060 http://802.11junk.com
Skype: JeffLiebermann AE6KS 831-336-2558
David Arnstein
2015-03-25 01:33:55 UTC
Permalink
Post by Jeff Liebermann
Microsoft has had VSS (Volume Shadow copy Service) for quite some
<https://msdn.microsoft.com/en-us/library/aa384649%28v=vs.85%29.aspx>
It allegedly allows backing up a busy filesystem. It seems to work
well for defrag products, but my luck hasn't been so good with image
backups. The scary part was that one of the failed image backups
passed a verify pass. However, when I checked the restored file list,
every file that was open at the time of the backup was gone on the
restore. Although most backup programs use VSS and it mostly works, I
refuse to deal with image backups that run from Windoze and prefer to
use a Linux boot CD or USB flash thing.
Was the failed backup/restore program CrashPlan? I had a very bad
experience with CrashPlan's handling of open files.

I think VSS is Microsoft's best idea ever. The implementation of the
idea may have some issues.
--
David Arnstein (00)
arnstein+***@pobox.com {{ }}
^^
Jeff Liebermann
2015-03-25 15:17:14 UTC
Permalink
On Wed, 25 Mar 2015 01:33:55 +0000 (UTC), David Arnstein
Post by David Arnstein
Post by Jeff Liebermann
Microsoft has had VSS (Volume Shadow copy Service) for quite some
<https://msdn.microsoft.com/en-us/library/aa384649%28v=vs.85%29.aspx>
It allegedly allows backing up a busy filesystem. It seems to work
well for defrag products, but my luck hasn't been so good with image
backups. The scary part was that one of the failed image backups
passed a verify pass. However, when I checked the restored file list,
every file that was open at the time of the backup was gone on the
restore. Although most backup programs use VSS and it mostly works, I
refuse to deal with image backups that run from Windoze and prefer to
use a Linux boot CD or USB flash thing.
Was the failed backup/restore program CrashPlan? I had a very bad
experience with CrashPlan's handling of open files.
No. It was earlier versions of Farstone "Drive Clone" that failed:
<http://www.farstone.com>
I had erratic failures on DriveClone Pro 5.1 and 6.0. The disaster
was on DriveClone Express 7.02. I tried to work with their support
personality, which largely consisted of excuses, trivial tweaks, and
time burners via email. In the end, they sent me my $40 back and
proclaimed that they were unable to support me. They simultaneously
discontinued all the older defective versions and renamed the then
current version something else.
Post by David Arnstein
I think VSS is Microsoft's best idea ever. The implementation of the
idea may have some issues.
Agreed. I like everything about VSS except that in the hands of the
programmers of backup software, the promised reliaibility seems to be
somewhat problematic.
--
Jeff Liebermann ***@cruzio.com
150 Felker St #D http://www.LearnByDestroying.com
Santa Cruz CA 95060 http://802.11junk.com
Skype: JeffLiebermann AE6KS 831-336-2558
Travis James
2015-03-30 03:29:04 UTC
Permalink
Post by David Arnstein
Post by Jeff Liebermann
Microsoft has had VSS (Volume Shadow copy Service) for quite some
Was the failed backup/restore program CrashPlan? I had a very bad
experience with CrashPlan's handling of open files.
I think VSS is Microsoft's best idea ever. The implementation of the
idea may have some issues.
Did MS run out of naming ideas and letters? I about flipped when you
said VSS is MS's best ever, but I was thinking of Visual SourceSafe
always referred to as VSS. Worst SCM ever.
Keith Keller
2015-03-25 03:56:57 UTC
Permalink
Post by Jeff Liebermann
Post by Jeff Liebermann
LTO (linear tape open) is the best tape system for today. LTO-6 holds
2.5TB per tape, with bigger tapes on the way.
<http://en.wikipedia.org/wiki/Linear_Tape-Open#Generations>
Such archives have one advantage over other backups. The data is not
live while it is being backed up. This is in contrast to a large
database that is being read from and possibly written to while the
backup is being performed.
Sorry, I wasn't too clear. By "live" I meant that it was in use by
some program. For databases, that means that the database file is
open for reading and possibly for writing. I suspect "busy" might be
a better choice of buzzword.
I am sorry, maybe I wasn't clear. In your first post, you said that LTO
tapes have an advantage that the data isn't busy. But I don't see how
the target media makes any difference as to whether a file is open or
not. If a file is open for writing (by any process, not just a
database), writing it to LTO will be just as bad as writing it to a
different filesystem. Is there something I'm not getting in your
original paragraph?

--keith
--
kkeller-***@wombat.san-francisco.ca.us
(try just my userid to email me)
AOLSFAQ=http://www.therockgarden.ca/aolsfaq.txt
see X- headers for PGP signature information
Jeff Liebermann
2015-03-25 15:41:49 UTC
Permalink
On Tue, 24 Mar 2015 20:56:57 -0700, Keith Keller
Post by Keith Keller
Post by Jeff Liebermann
Post by Jeff Liebermann
LTO (linear tape open) is the best tape system for today. LTO-6 holds
2.5TB per tape, with bigger tapes on the way.
<http://en.wikipedia.org/wiki/Linear_Tape-Open#Generations>
Such archives have one advantage over other backups. The data is not
live while it is being backed up. This is in contrast to a large
database that is being read from and possibly written to while the
backup is being performed.
Sorry, I wasn't too clear. By "live" I meant that it was in use by
some program. For databases, that means that the database file is
open for reading and possibly for writing. I suspect "busy" might be
a better choice of buzzword.
I am sorry, maybe I wasn't clear. In your first post, you said that LTO
tapes have an advantage that the data isn't busy. But I don't see how
the target media makes any difference as to whether a file is open or
not. If a file is open for writing (by any process, not just a
database), writing it to LTO will be just as bad as writing it to a
different filesystem. Is there something I'm not getting in your
original paragraph?
--keith
Y're right. I screwed up. I somehow left out a paragraph on the
different ways of creating an image backup which should have been
between my comments on LTO tape and my mention of not backing up a
"live" filesystem. I'll try to fill in the blanks.

An image backup of a live filesystem (files open for writing) is going
to be a problem. Chances are good that the database will change
during the backup and that a verify pass will show that it has
changed. Just log files will do that. Whether the data is usable
after a restore is largely dependent on whatever utility is used to
repair the mangled pointers in the database. Since the backup
software is doing a block by block backup, there's no chance of using
record locking for protection. Files open for reading are not a
problem because nothing changes.

In my never humble opinion, the only useful option is to shut down the
server, boot from a CD/DVD/USB-Flash, and run the image backup as fast
as possible. No files are open, therefore not problems. I'm getting
about 3GB/min to USB 3.0 hard disks which is quite adequate. In the
past, when I was still running image backups while Windoze was active,
I noticed that the backup speed was dramatically slower, as the hard
disk had to service Windoze and user programs, while simultaneously
trying to also do the backup. As I recall, it was something like
three times slower. Better off stopping Windoze and getting it over
with.
--
Jeff Liebermann ***@cruzio.com
150 Felker St #D http://www.LearnByDestroying.com
Santa Cruz CA 95060 http://802.11junk.com
Skype: JeffLiebermann AE6KS 831-336-2558
Keith Keller
2015-03-26 05:03:19 UTC
Permalink
Post by Jeff Liebermann
Y're right. I screwed up. I somehow left out a paragraph on the
different ways of creating an image backup which should have been
between my comments on LTO tape and my mention of not backing up a
"live" filesystem. I'll try to fill in the blanks.
Whew! I thought you'd really lost your mind. :)
Post by Jeff Liebermann
An image backup of a live filesystem (files open for writing) is going
to be a problem. Chances are good that the database will change
during the backup and that a verify pass will show that it has
changed. Just log files will do that. Whether the data is usable
after a restore is largely dependent on whatever utility is used to
repair the mangled pointers in the database. Since the backup
software is doing a block by block backup, there's no chance of using
record locking for protection. Files open for reading are not a
problem because nothing changes.
I just want to reiterate that *any* files open for writing are subject
to this issue (and I know you know this already but others may not).
Generally databases get the most attention, because they often have
files open for writing indefinitely, cache writes in their processes (in
addition to the normal caching the OS may do), and non-database
processes generally write their files quickly enough that it's hard to
catch the file in an unbackupable state. But applications that might be
writing a large file could be writing during a backup, and the backup
copies may be in an inconsistent state as a result (and it's highly
unlikely the application has a tool for recovering that file).
Post by Jeff Liebermann
In my never humble opinion, the only useful option is to shut down the
server, boot from a CD/DVD/USB-Flash, and run the image backup as fast
as possible. No files are open, therefore not problems.
Sure, that's always the most reliable option, but doesn't help you if
you want your application up 24/7 (or most of it), or don't have
reasonable access to a remote console, or don't have DC monkeys to
insert media, or probably a dozen other scenarios. That's why things
like database dumps and filesystem and/or LVM snapshots exist. (FWIW I
still would not rely on a filesystem snapshot for a database backup; use
the db backup tools.) Replication can also help here; if you replicate
your database to multiple slaves, you can remove a slave from
replication, take your db dump, back that up elsewhere, restore the
slave and have it catch up, without disrupting production.

--keith
--
kkeller-***@wombat.san-francisco.ca.us
(try just my userid to email me)
AOLSFAQ=http://www.therockgarden.ca/aolsfaq.txt
see X- headers for PGP signature information
Jeff Liebermann
2015-03-26 16:32:27 UTC
Permalink
On Wed, 25 Mar 2015 22:03:19 -0700, Keith Keller
Post by Keith Keller
Whew! I thought you'd really lost your mind. :)
Actually, I had. I was dealing with some medical issues that
fortunately are turning out to be non-serious. I was sufficiently
brain dead earlier this week to cancel most of my appointments and
occupy myself with cleaning up the office mess and posting to Usenet.
Post by Keith Keller
I just want to reiterate that *any* files open for writing are subject
to this issue (and I know you know this already but others may not).
Yep. Just to make it difficult, if the database had record locking
features, it might be possible to lock a record, do a
read-before-write cycle, backup the record, and release the record
lock. That would insure that the database pointers are up to date and
that it's not doing simultaneous record update and backup. I don't
know if modern databases still do this, but when I was fighting such
band aids in the 1990's, it barely worked, was incredibly slow, and
still didn't guarantee a usable backup. However, there was no choice
in some situations. For example, I could not shutdown the hospital ER
admissions computer for very long in order to do a backup. They
needed the computah full time, even if it was running slow. So, the
backups were file by file and timed to coincide with slow times of the
day (2-4am).
Post by Keith Keller
Generally databases get the most attention, because they often have
files open for writing indefinitely, cache writes in their processes (in
addition to the normal caching the OS may do), and non-database
processes generally write their files quickly enough that it's hard to
catch the file in an unbackupable state. But applications that might be
writing a large file could be writing during a backup, and the backup
copies may be in an inconsistent state as a result (and it's highly
unlikely the application has a tool for recovering that file).
I haven't seen any that open the entire database for writing full
time. They might do that for short periods in order to do a cache
flush. What I've seen are transaction logs, where the database
application writes small transaction files, but does not update the
main database. When the backup is done, the transaction files are
used to update the main database. The advantage of this scheme is
that the database can be open for reading only, while not allowing any
writes, thus eliminating the record locking slow down mess.

Note: I are not a programmist and am far behind the state of the art
in databases.
Post by Keith Keller
Sure, that's always the most reliable option, but doesn't help you if
you want your application up 24/7 (or most of it), or don't have
reasonable access to a remote console, or don't have DC monkeys to
insert media, or probably a dozen other scenarios. That's why things
like database dumps and filesystem and/or LVM snapshots exist. (FWIW I
still would not rely on a filesystem snapshot for a database backup; use
the db backup tools.) Replication can also help here; if you replicate
your database to multiple slaves, you can remove a slave from
replication, take your db dump, back that up elsewhere, restore the
slave and have it catch up, without disrupting production.
Replication and transaction files is the way larger systems do it. I
don't know much about these as I deal with much smaller systems where
such backups are not practical. I also got involved in a stupid
problem where the databases were being properly backed up (using an
image backup) but nobody was backing up the OS and program files
because it couldn't be taken offline for very long. When it crashed,
I got to put it back together from a 2 year old image file that I
created to cover my posterior, and the rest from bits and pieces over
a 2 day marathon rebuild. There's a reason I keep a sleeping bag and
camping kit in my office.
--
Jeff Liebermann ***@cruzio.com
150 Felker St #D http://www.LearnByDestroying.com
Santa Cruz CA 95060 http://802.11junk.com
Skype: JeffLiebermann AE6KS 831-336-2558
Keith Keller
2015-03-26 23:18:41 UTC
Permalink
Post by Jeff Liebermann
I haven't seen any that open the entire database for writing full
time.
Not sure; small dbs like SQLite might lock the entire "database", since
it may just be a handful of files. MyISAM tables are three files per
table, and when the table is being written these files are open rw.
Other databases and table formats use many more files for their data
store.

--keith
--
kkeller-***@wombat.san-francisco.ca.us
(try just my userid to email me)
AOLSFAQ=http://www.therockgarden.ca/aolsfaq.txt
see X- headers for PGP signature information
Jeff Liebermann
2015-03-27 02:21:58 UTC
Permalink
On Thu, 26 Mar 2015 16:18:41 -0700, Keith Keller
Post by Keith Keller
Post by Jeff Liebermann
I haven't seen any that open the entire database for writing full
time.
Not sure; small dbs like SQLite might lock the entire "database", since
it may just be a handful of files. MyISAM tables are three files per
table, and when the table is being written these files are open rw.
Other databases and table formats use many more files for their data
store.
--keith
I'm not sure either, but I vagely recall that some sparse file
databases are permanently open for writing because they don't use the
filesystem for file or record locking.
<http://en.wikipedia.org/wiki/Sparse_file>
I used to deal with one of these in some obscure medical office
management software and recall that backing up required everyone to
logout and disconnect. About the only way to backup a sparse file
database was to either use their utility, which converted it to a
monsterous file, or do an image backup because it was on a raw
filesystem. The only good part is that access was very fast (when not
fragmented).

I don't want to think about this any more. It brings back too many
nightmares of 24 hr index (pointer) rebuilds, and overnight crash
recoveries.
--
Jeff Liebermann ***@cruzio.com
150 Felker St #D http://www.LearnByDestroying.com
Santa Cruz CA 95060 http://802.11junk.com
Skype: JeffLiebermann AE6KS 831-336-2558
David Kaye
2015-03-26 20:25:20 UTC
Permalink
Post by Keith Keller
Generally databases get the most attention, because they often have
files open for writing indefinitely, cache writes in their processes (in
addition to the normal caching the OS may do), and non-database
processes generally write their files quickly enough that it's hard to
catch the file in an unbackupable state.
When I worked at Bank of America and later at TeleResults I made sure that
db records were opened for write ONLY in that moment before the record was
to be written. I think it's very sloppy programming practice to simply
leave records open. In the years I was with TeleResults we didn't have any
database crashes or unreconcilable transactions.

It amazes me even now as I look over other people's code how sloppy they can
be and how they don't seem to care the least about data safety except for
encryption. But it does no good to encrypt dirty records, so what's the
point?




---
This email has been checked for viruses by Avast antivirus software.
http://www.avast.com
David Kaye
2015-03-25 08:48:13 UTC
Permalink
Post by Jeff Liebermann
Sorry, I wasn't too clear. By "live" I meant that it was in use by
some program. For databases, that means that the database file is
open for reading and possibly for writing. I suspect "busy" might be
a better choice of buzzword.
When I was writing medical software for human organ transplants, I had the
option of backing up live databases. Instead I chose to shut down the
databases while doing the backup, because who knows what problems can happen
when you're trying to back up something at 3:00am and some lab is trying to
send you test results at that time. I'd rather take it offline and let the
sending software keep trying to connect than to run the risk of corrupt
data. In the 15 years since I wrote the original software it's still
working fine.




---
This email has been checked for viruses by Avast antivirus software.
http://www.avast.com
b***@MIX.COM
2015-03-25 19:12:21 UTC
Permalink
Post by Jeff Liebermann
Do you verify you backups to see if they're usable? I don't mean a
verify pass immediately after a backup on the same machine that
created the backup, but rather removing the backup media, moving it to
another machine, and trying the recovery a few days or weeks later.
Yes, definitely. But, speaking of verify passes...

One of my favorite small media is DVD-RAM. It does reads after writes,
and has an excellent bad block replacement (defect management) scheme,
but Panasonic stopped making the (best) drives some years ago, and now
seems to have stopped making the discs. Probably another tsunami
victim.
Post by Jeff Liebermann
Such archives have one advantage over other backups. The data is not
live while it is being backed up.
True. Everything I'm doing is either quiescent, or being taken live
(just streaming data).
Post by Jeff Liebermann
<http://en.wikipedia.org/wiki/Linear_Tape-Open#Tape_durability>
"There is a large amount of lifespan variability in actual use"
Yep. Please forgive my cynical attitude, but I've seen far too many
tape systems announced, flounder, and then discarded when the next big
backup thing arrives. At least the problem was recognized so they
made it easy to move DLT libraries to LTO.
This is a real problem.. Luckily Quantum bought DEC's stuff, but
regardless, hardware has to be archived, too. It's a very fickle
business, heh.

Billy Y..
--
sub #'9+1 ,r0 ; convert ascii byte
add #9.+1 ,r0 ; to an integer
bcc 20$ ; not a number
d***@81.usenet.us.com
2015-04-02 23:35:08 UTC
Permalink
Post by Jeff Liebermann
tape heads 4 times as fast. When it was necessary to restore some of
his data, I found that it could only be read on the drive with the
worn out heads. Also, none of his older backup tapes, which were made
In the days when I had 4mm DAT, I thought it was head movement from bad
adhesive, not wear, per se, that caused the problem.

Tapes would read fine for a while, but eventually, you couldn't read tapes
that had been written less than a year ago, and interchangeability between
our two drives almost never worked.

Wear seems like an odd thought, although the consumer drives were obviously
cheap build quality. We had commercial drives that moved 1/2" open reel tape
all day long, and I don't recall ever changing a head or having any
failures.
--
Clarence A Dold - Santa Rosa, CA, USA GPS: 38.47,-122.65
Jeff Liebermann
2015-04-03 05:12:21 UTC
Permalink
Post by d***@81.usenet.us.com
Post by Jeff Liebermann
tape heads 4 times as fast. When it was necessary to restore some of
his data, I found that it could only be read on the drive with the
worn out heads. Also, none of his older backup tapes, which were made
In the days when I had 4mm DAT, I thought it was head movement from bad
adhesive, not wear, per se, that caused the problem.
I'm not sure what you mean by "head movement". DAT uses helical scan
heads. The heads move all the time. There is no adhesive.

Both the tapes and the heads wear out. The tapes are allegedly good
for 2000 passes, but even the most optimistic vendors suggest retiring
tapes after about 75 backups. I tended to use cartridge loaders for
DAT, which resulting in a shorter head life. As I recall, the nightly
backups usually killed the drive after about 6 months.

The least reliable backups I had to deal with were DAT and DAT2. I
never could reliably read a tape created on a different drive.

However, the section you quoted from my previous rant was not about
DAT. It was 1/4" tape, probably DC6150 or maybe higher density. I
don't recall. No helical scan. Just back and forth scrubbing of the
heads. Tape does stetch unevenly, resulting in uneven head pressure,
and uneven wear. Eventually, a side view of the head looks rather
rough. After a while, the wear gets to the point where the head gap
is widened, which reduces signal pickup and produces errors. Tapes
made on drives with such uneven head wear patterns often cannot be
read on a different drive, even if it's an identical model.
Post by d***@81.usenet.us.com
Tapes would read fine for a while, but eventually, you couldn't read tapes
that had been written less than a year ago, and interchangeability between
our two drives almost never worked.
Yep. DAT's the way it worked or didn't work. About 1/3 of the data
written on a DAT tape is error correction and it still can't get it
right. The moral is that after some point, additional error
correction doesn't magically buy more reliability.
Post by d***@81.usenet.us.com
Wear seems like an odd thought, although the consumer drives were obviously
cheap build quality. We had commercial drives that moved 1/2" open reel tape
all day long, and I don't recall ever changing a head or having any
failures.
Yep. 9 track 6250 bpi tape was very reliable. I found a few creative
ways to introduce errors, but in general, they worked well. The
problem was that they were just too small. A 2400ft 6250 bpi tape
would only hold about 160 MBytes uncompressed. What was needed was
gigabytes, which wasn't going to happen with open reel magnetic tapes.
--
Jeff Liebermann ***@cruzio.com
150 Felker St #D http://www.LearnByDestroying.com
Santa Cruz CA 95060 http://802.11junk.com
Skype: JeffLiebermann AE6KS 831-336-2558
d***@81.usenet.us.com
2015-04-03 19:40:33 UTC
Permalink
Post by Jeff Liebermann
I'm not sure what you mean by "head movement". DAT uses helical scan
heads. The heads move all the time. There is no adhesive.
Little tiny heads, glued into a notch at the edge of that big spinning
flywheel that was supposed to present enough surface that the tape would
actually fly like a disk head.

We chose 4mm DAT for that reason, because it was less tape wear than the
8MM cartridge. Sigh.
Post by Jeff Liebermann
The least reliable backups I had to deal with were DAT and DAT2. I
never could reliably read a tape created on a different drive.
Junk. Just junk.
Post by Jeff Liebermann
However, the section you quoted from my previous rant was not about
DAT. It was 1/4" tape, probably DC6150 or maybe higher density. I
don't recall. No helical scan. Just back and forth scrubbing of the
The AT&T Unix PC used a floppy-on-tape with some tiny buffers. I don't
remember the detail, but they would shoe-shine like crazy.
Post by Jeff Liebermann
rough. After a while, the wear gets to the point where the head gap
is widened, which reduces signal pickup and produces errors. Tapes
The SyQuest 5 MB cartridges, ~1984, had that problem. Hard disk, flying
head, but they heads didn't fly quite right. The gap would widen, or
narrow... I forget, now. Only the drive that wrote it could read it, and
sometimes not for very long.

At one point, we had an array of drives of different wear, and in a
critical situation, the signal could be examined with an oscilloscope, and
you would then pick the drive most likely to read the disk, based on the
observed signal.
--
Clarence A Dold - Santa Rosa, CA, USA GPS: 38.47,-122.65
Steve Pope
2015-03-26 17:36:11 UTC
Permalink
Post by d***@81.usenet.us.com
Multiple copies are the only way to help ensure long term safety.
Mutliple copies AND periodic migration to less-obsolete
storage/media units.

Steve
Roy
2015-03-24 05:18:49 UTC
Permalink
Post by Roy
I was in a discussion today on archival storage.
Someone has a large amount of data (several TB) and loads it onto an
external storage device. The device sits on a shelf in a closet for X
years. Is SSD or a regular disk better for this?
After reading all the comments, it sounds like a potential product for
archival storage.

You have a master disk and a program produces "X" copies. Each copy
contains data, checksums, and ECC data of the master.

In the distant future, you cycle the "X" copies through the program and
the master disk is reconstructed.

Some sort of over redundant RAID array

And some people are thinking that way

Highly reliable two-dimensional RAID arrays for archival storage

"The new organization tolerates all triple disk failures and nearly
all quadruple and quintuple disk failures."

http://www.ssrc.ucsc.edu/pub/paris12-ipccc.html
David Kaye
2015-03-24 08:24:57 UTC
Permalink
Post by Roy
Some sort of over redundant RAID array
And some people are thinking that way
The thing about RAID is that it's for redundancy in active use, not
archiving. And RAID 1 is useless as I discovered a week ago when a customer
lots his point of sale system's server. It uses RAID 1 and the computer
failed to boot. Trouble was that whatever the bug was that caused the
problem spread to both drives and neither would boot the system. I don't
know what the outcome was because before I arrived they called in their POS
vendor and I left it to him to fix.

As I understand it from people who use RAID 5 is that it's really only
useful for online serving where when one drive fails you can immediately
swap it out and no data will be lost. But file corruption, malware, and
accidental deletion will corrupt or delete the files across all components
of the system. So, it's not at all like having a spare; it's more like
having just 1 hard drive.

I think it's time to call in Brewster Kahle over at archive.org and see what
he has to say, given that he ihas dealt with files going back to the
earliest part of Usenet and even some BBS files before that.




---
This email has been checked for viruses by Avast antivirus software.
http://www.avast.com
Roy
2015-03-24 14:42:40 UTC
Permalink
Post by David Kaye
Post by Roy
Some sort of over redundant RAID array
And some people are thinking that way
The thing about RAID is that it's for redundancy in active use, not
archiving. And RAID 1 is useless as I discovered a week ago when a customer
lots his point of sale system's server. It uses RAID 1 and the computer
failed to boot. Trouble was that whatever the bug was that caused the
problem spread to both drives and neither would boot the system. I don't
know what the outcome was because before I arrived they called in their POS
vendor and I left it to him to fix.
...
Think of it this way. You put your data on a Raid Array and turn it
off. Remove all the drives and store them. In the future reassemble
the raid array and turn it on. Let it recovoer and you should have your
data,

The article I mentions had 9 data and 7 backup disks. Like any RAID
array, it protects against drive failure and not software.
Jeff Liebermann
2015-03-24 15:14:46 UTC
Permalink
Post by Roy
Think of it this way. You put your data on a Raid Array and turn it
off. Remove all the drives and store them. In the future reassemble
the raid array and turn it on. Let it recovoer and you should have your
data,
I've had the drives in a RAID 5 array die sequentially over a fairly
short period of time. The problem is that identical drives, operating
in an identical manner, for an identical amount of time, tend to fail
identically.

I've also seen a few drives fail without actually running, which I
attribute to IC packaging leaks and defects. The plan was to keep a
spare drive handy in case the RAID array went down. When the RAID 5
started showing errors on one drive, I swapped in the unused drive and
found that it was also failing.
Post by Roy
The article I mentions had 9 data and 7 backup disks. Like any RAID
array, it protects against drive failure and not software.
I suspect that will end up like DAT tape drives, where in order to
deal with a fundamentally unreliable media, about half the data stored
on the tape was error correction and error detection data. Beyond a
certain point additional error correction doesn't add much.

I don't have an easy answer to the problem. I suggest you look at how
the data is stored on the various devices:
- Magnetic
hard disk
tape
floppy
- Color dye phase change
CD-R
DVD
Blu Ray dye (LTH) [1]
- Color pigment phase change
Blu Ray pigment (HTL)
- Electric Charge
SSD, NAND Flash, USB dongle

So, which would you trust your data? Magnetic domains tend to move by
themselves or by external influence. Dye phase changes tend to
decompose, bleed, fade, and are affected by the environment, such as
temperature and humidity[2]. I don't know much about pigment BD-R
technology. Solid state devices that store charge can be discharged
by energetic atomic particles, manufacturing defects, packaging
problems, and possibly magnetic fields (not sure). None are perfect
or forever. I would tend to stay away from magnetic and electric
charge mechanisms because they are too fragile and tend to deteriorate
faster than chemical storage mechanisms. Color dye isn't too
horrible, but I think the new pigment based BD-R disks offer the best
chance of not screwing up. Like the WORM and DAT drives of the past,
you'll probably need to archive a reader along with the data.
Unfortunately, 25GB to 100GB per disk is going to require a fairly
large number of disks. That can probably be tolerated because
methinks it would better to have usable bloated archival data than to
have high density unusable garbage.

Note:
[1]
<http://blog.digistor.com/not-all-blu-ray-discs-are-created-equal-but-does-bd-r-quality-matter/>

[2] <http://www.cd-info.com/archiving/degradation/>
--
Jeff Liebermann ***@cruzio.com
150 Felker St #D http://www.LearnByDestroying.com
Santa Cruz CA 95060 http://802.11junk.com
Skype: JeffLiebermann AE6KS 831-336-2558
Loading...