This file is aimed at all the people who want to understand more about HD maps without going deeply into technical speech. If you have just some questions you can try to read the "Possible Questions" section and ignore the rest.
This file will only deal with new format maps (E-Format) since the old ones (D-Format) are completely out-of-date.
Before starting with the map itself, we have to speak about files and disc structure.
Each file on a disc (including directories) can be identified from a SIN (System Internal Number) that is stored together with all the other file informations (name, filetype, size, etc.) in the directory that contains the file. The SIN is made up by two parts, the file number and the sector offset (more on this next).
The disc (HD or floppy) is physically made up by a number of heads (2 for floppies), tracks and sectors per track. RISC OS will generally ignore this and will consider the disc as a consecutive sequence of sectors.
The map of an E-Format disc is divided into zones: each zone represents a number of consecutive sectors and can be considered as a stand-alone entity. So, unless specified, the map format can be applied in the same way to both single-zone discs (like 800K floppies) and multi-zone ones. Note that there are no relations between the physical disc structure (ie. number of heads, tracks, etc.) and the zones subdivision.
On each disc there are 2 copies of the map (one is taken as a backup): quite ironically the 2 maps are (obviously!) always identical, so if the first is corrupted the second is of no use. The second map can be useful only if the first one has been physically damaged but (apparently) RISC OS always read the first and ignores the other...
Each zone consists of a list of blocks of variable length, each one representing an occupied or a free part of the disc; the formers contain the file numbers (ie. the first part of a SIN) and the latters contain a link to the next free-space block. if you know the SIN of a file, you can easily find out where it is located in the zone and so you can know the exact position of the file on the disc (ie. the disc address). The same file number can appear in more than one block, in the same or in different zones, if a file is fragmented in small chunks; this way you can also have files that straddle across two or more zones.
Sometimes two or more files have the same file number but different sector offset (ie. the second part of a SIN): this means that in the map they appear to be a single file (occupying only one block) but, knowing the SIN, it is possible to find out the right address of each one. This trick is useful to avoid a lot of disc space wasting: because of the organization of the map, each block must occupy a minimum fixed length (that generally depends on the size of the HD) and so, even if the file is really short, it has to use up some extra disc space. Thanks to the sector offsets, a lot of small files can share the same block (thus the same file number) but each one could only use the size of a physical sector.
I tried to explain the map structure in a simplified way, without going deeply into the real data structure (bit streams, checksums, etc.), but I think I said enough to allow you to understand:
Obviously the above structure is really rigid and there is no place for errors! Unfortunately errors always occur because of unexpected power down, system crashes, physical damages, (errors in FileCore module itself?), etc; this is a list of possible problems:
Because of the disc structure and the map organization, some space can be 'naturally' wasted.
First of all, the sector bounds are a physical limit and so all the files must be 'sector aligned'. So, on a 512 bytes sector sized HD, a 1 byte file always uses (at least) 512 bytes, while a 513 bytes one uses 1024 bytes.
But the real waste of space is caused by the map organization: as I already said, there is a minimun length for each map block (from now I will consider it to be 4K but note that this is only an example).
This is not so bad because more than one file can share it, hence 4*1K files can fill up the same block wasting no space. The bad point is that only files in the same directory can share blocks, so 4*1K files in 4 different directories will use 16K.
A directory always needs 2K of disc space to keep its data. Since a directory from the map point of view is just a file, it will still use 4K: fortunately it can share the remaining 2K with files contained in it (not with files from its parent). If a directory contains a 2K file or 2*1K files all the space is used but if it contains only files longer than 2K the space will be wasted.
Wasted space isn't lost space: if you have an empty directory you will waste 2K. But when you delete it you will regain the whole 4K, no loss. If you save a 1K file in it, the free space on your disc will remain the same and the wasted space will be just 1K.
During normal operations, there is no way to avoid wasting space: RISC OS does all the work and you don't have to worry about the file allocation strategy. Anyway, sometimes you can recover some wasted space by simply copying a file, deleting the original one and renaming the new one as the old one (usually it is better to copy the whole content of a directory to another and then rename the whole directory back) but don't expect too much gain...
CheckMap performs quite a simple task: it just scans the directory tree and checks that all the file numbers in the map are actually contained in a directory. Considering that CheckMap should be used when something went wrong, it seems to be a very poorly written piece of code: sometimes it hangs the whole computer, sometimes it just prints the cryptic message "Map inconsistent with directory tree" and the only action that it should do to fix the situation is the swapping between the 2 copies of the map (but see above to understand why nobody has never been able to use this facility...).
Compact has been a very useful command when the only way to format discs was
D-Format: in fact with E-Format discs it is almost useless. Its function is to
collect together fragmented files or fragmented free space inside zones (but
not from one zone to another). With E-Format discs this operation is performed
automatically when a new file is saved in that zone, but you can anyway use it
'manually' to speed up future savings (actually the speed gain is almost
irrelevant). With D-Format discs the 'auto-compaction' doesn't exist and files
cannot be fragmented: if there is enough space on the disc for the file that
has to be saved but this free space is fragmented in small chunks, the error
"Compaction required" is issued.
On E-Format discs *Compact isn't really useful but it can be used safely if the map is good. If the map is inconsistent I can't assure you of its behaviour (ie. I can't exclude that it can cause worse corruptions). Note that RISC OS can automatically run a 'subset' of Compact when it has to save a file (especially if the file is large and the free space is limited and fragmented).
I have to admit that CheckMap isn't a very well written command but if it says that the map is good, the map IS good and you have no lost space on your disc. If it crashes the computer or if it says "Map inconsistent with directory tree", your map or directory structure IS damaged and the damage can worsen if not cured soon.
A broken directory cannot be deleted using standard OS commands and it can just be considered as a 'normal' undeletable file. To remove it you must use some directory manipulation program (like Look System's Disc Doctor?) or the shareware program 'eliminate' (distributed with 'fsck').
Apparently the only thing you can do is to try the shareware program 'fsck' that performs the same task as *CheckMap but also tries to fix the damage.
I am not the only person who thinks that having a nearly full HD (say, less than 1Mb free) can be dangerous, especially if you handle very large files. Using bugged programs that sometimes can crash the computer while it is writing to the HD is REALLY dangerous but fortunately there are very few programs of this kind around...
Generally no. For example if the corruption was caused by a delete operation that was interrupted after it removed the file from the directory but before the removal from the map, the map will be inconsistent but no data has been lost/overwritten. On the other hand, a 'Broken Directory' can be the evident sign that something actually has been corrupted on the disc, so you can find that some other files are damaged. In fact, RISC OS can easily notice the wrong directory header/checksum but it cannot really understand if a text file has been corrupted!
I can be contacted at the following address:
pel0015@cdc8g5.cdc.polimi.it
If you don't receive a reply (or if the message is bounced) I can also be
contacted at:
sergio@phantom.com
or:
sergio@g386bsd.first.gmd.de