Board index » cppbuilder » Data recovery tools (was .nativeapi "Re: Need help with where to start")

2005-06-23 06:17:12 PM
cppbuilder21
Hmmm. There is a danger of this post reading like a rant and/or
criticism of Mark. It isn't intended to be either but I've been
recovering data and writing tools to do that for over 13 years now and
I have seen the negative consequences of DiY recovery all too often.
When it works it's good. When it doesn't work it mostly screws things
up good and proper. I probably have some emotional attachment to this
subject despite my best endeavours :)
Mark Jacobs wrote:
be in complete control of (and have complete knowledge of) what a given
tool does and be able to make adjustments and add features as and when
I need to.
Data recovery is not like fixing a car. Data recovery is more like bug
fixing. All you can do is work the problem until you locate the fault
then determine a solution. The concept of an ideal data recovery tool
is like proposing an ideal bug fixing tool. Would you just let someone
else' software loose on your source trusting it to find all the bugs
and letting it modify the source to fix them?
Before going into a more lengthy explanation I want to reiterate rule
number one where data recovery is concerned:
*Never*, ever, work on the affected drive. A tool that allows (or God
forbid, encourages you) to dig around a failed drive or corrupt file
system should be kept out of the hands of amateurs and has only limited
use to professionals (see later). If you don't have a backup then that
failed drive/file system is last remaining place in the known universe
that contains your data. Do you really think Mr. Joe Average should
start piddling around with it?
Going back to the mythical bug fixing tool:Would you run it without
backing up your source code first?
Lengthy explanation:
Most publicly available tools are intended to allow you to fix a
filesystem. Some (Spinrite for one) even claim to allow you to fix a
physical fault.
In order to fix a filesystem you need to:
* Fully understand the nature of the problem.
* Fully understand all the changes that need to be made to rectify
the problem.
* Be able to make all the required changes with a guarantee that the
result will be exactly what the operating system expects to find.
None of these stages are easy and if you get the last two wrong you are
/by definition/ making things worse. Not only could the act of changing
file system meta data destroy useful evidence but operating systems can
be crashed by corrupt filesystems, or led into running rampant over the
disk.
Who do you trust to recognise misleading information - a human or a
computer program?
Even assuming you are in full control of the tool I still question the
wisdom of attempting to repair the damage. Here's an analogy. Your
house becomes unfit to live in. What do you do about it?
Solution 1:
Remove all your possessions (those that you can safely get to anyway).
Remove whatever construction materials are unsafe (shoring up as
needed).
Install new construction materials making sure that the building is up
to code.
Reinstall all your possessions.
Solution 2:
Remove all your possessions (those that you can safely get to anyway).
Buy a new house.
Reinstall all your possessions.
Which would you do?
Of course some people might be keen and skilled builders so might enjoy
the challenge of solution 1. But I think that most people (particularly
if you consider that buying a new hard disk costs next to nothing)
would opt for solution 2.
As for fixing the hard disk:That depends on the nature of the fault.
Certainl electronic faults (EEPROMs getting corrupt etc) can certainly
be done through software. To this I would say that as long the recovery
tool presents the fix as being a temporary reprieve and encourages you
to take the opportunity to get your files off while you still can then
I have /slightly/ fewer objections. Nonetheless rule number one is of
such overriding importance that I don't think the general public should
have access to such tools.
If the drive has a physical fault then it probably has a very limited
remaining lifetime. The time wasted by the recovery tool trying to
figure out what's going on or trying half a dozen different tricks
could be using up time better spent salvaging data.
And there you have it:Andrue's rant on commercial data recovery tools.
next week:World peace - is it the lifestyle choice of the 21st century?
:)
--
Andrue Cope [TeamB]
[Bicester, Uk]
info.borland.com/newsgroups/guide.html
 

Good post.
Would you mention tools that can be used to do these things?
For example, you suggest that the work be done on a copy of the offending
drive. How does one make such a copy with a commercially available tool?
Will Ghost copy a failed drive?
While I'm at it, since you are into low level disk handling, do you know of
anything that will wipe the free areas of a disk? I've been using
sdelete.exe from SysInternals but now notice that it is listed as supporting
Win95/98/NT/2K but not listed as supporting XP. That tells me that
something in XP's NTFS doesn't get handled OR that there is a problem with
it under XP's NTFS (I know from experience that it handles XP's FAT32).
What I suspect is that XP's auto-recovery system will have copies of deleted
files whose data you are hoping will be wiped.
. Ed

Ed Mulroy wrote:
failing drive is not as simple as detecting an error code and
continuing. Failing drives do some downright weird things including our
least favourite which can be summed up as:
If you keep being pestered just find another sector you /can/ read
and give them that instead.
I have written four disk imagers now. They all start off as a simple
loop:
1.If not done
2.Read from source
3.Write to target
4.Goto (1).
Then the tweaks, recovery features and black magic start to creep in.
Switches to make the disk go back to track 0 on error. Or to /not/ go
back to track 0 on error. Or to give up on an error and just take a
large leap past as part of an initial data grab. Or to read backwards
because that puts less stress on the drive. or..whatever. Although I
like to think I'm a good programmer I have to conceded that after a few
years it's best to just start over to clean the code up. :)
From when I last looked at it Ghost now has a proper recovery mode and
although I personally would still rather use my own tool I think Ghost
might be an acceptable alternative as a fallback. The product feels
like someone has actually expended development effort in testing it on
failed media. Hmmm. One thing though:Don't use a Windows based imager.
Windows has its own ideas about how to talk to drives and its reactions
to failing drives are impossible to control and hard to live with.
Plug and play device discovery is really not a good thing to throw into
the mix :(
There are many disk imagers available but unless you are sure that they
are actually designed with recovery in mind they may have various
problems.
I would also want to test such a tool myself because one of our
competitor's tool's reaction to bad sectors was to give up and return
without any indication of error. The result was 32kB of data of which
an unknown amount was left over from the prior successful read.
In forensics a strong recommendation is to always image twice and then
compare. This is because even an apparently functional drive doesn't
always read all its sectors the same.
space, making sure to actually write stuff into it rather than just
performing a seek(). Also determining the size for that file is best
done iteratively. Just keep extending the file in reducing increments
until the OS won't let you add even one more byte.
with a 'recycle bin' it becomes complicated. I can write a utility to
clean out the clusters listed in the volume bitmap as unused but that
won't affect files in the recycle bin since as far as the file system
is concerned they are active files.
It's wise to suspect XP of many things. Not that I complain - I think
XP is the best Windows MS has produced so far and think it's fantastic
for home use. I'm a bit unsure about development though, preferring
Win2k for that.
--
Andrue Cope [TeamB]
[Bicester, Uk]
info.borland.com/newsgroups/guide.html
 

{smallsort}

Ed Mulroy wrote:
rollback facilities). It's a lot easier to find network space for a
given drive image than it is to ensure you have a big enough drive in
stock when you need a doner.
In my case (checks IT department isn't looking) I have two RAID arrays
with a nearly 200TB volume spanned across them attached to my machine
(NTFS compression enabled, naturally). I'm not greedy but I do like a
lot :)
--
Andrue Cope [TeamB]
[Bicester, Uk]
info.borland.com/newsgroups/guide.html
 

Andrue Cope [TeamB] wrote:
I appreciate the posting, the sharing of experience. I just wish I had
such interesting stuff to talk about!
-Brion
 

Brion L. Webster wrote:
'interesting' for everyone but it's relevant.
Saddest 'phone call I had to deal with.
Someone from a well known UK telecoms company had a drive failure.
Their service company sent an engineer. He opened the cabinet and was
pleased to see a spare disk in the cabinet so he reconfigured the
server and recreated the NetWare volume onto it.
Unfortunately they had no backup and it turned out that the cabinet was
actually a hardware mirror that had failed to switch over to the
surviving disk.
We did what we could but it was NetWare 4 with suballocation and that
pretty much stuffed the last cluster of most of the files.
--
Andrue Cope [TeamB]
[Bicester, Uk]
info.borland.com/newsgroups/guide.html
 

Andrue Cope [TeamB] wrote:
batch files onto that! ;-)
--
Mark Jacobs
www.dkcomputing.co.uk
 

Don't forget, size isn't everything - despite what all the spam says.
What do they say about people with a large hard disk?
Rgds Pete
"Mark Jacobs" <www.jacobsm.com/mjmsg.htm?mj@critical>wrote in
message news: XXXX@XXXXX.COM ...

Sorry, this should really have a <g>or a :) at the end of it.
Pete
"Pete Fraser" < XXXX@XXXXX.COM >wrote in message