[Coco] Large file deletion fails on NitrOS9/6309

[Gene Heskett]

On Thursday 28 May 2020 19:52:48 Jeff Teunissen wrote:

> The file *was* put on the disk using Toolshed, and as it turns out,
> that was the problem.
>
> After some investigation with Curtis, we have figured out that
> Toolshed creates files that can't currently be deleted (safely) by
> (Nitr)OS-9/6x09.
>
> The on-disk structure of an RBF file provides for a file to contain up
> to 48 segments, where a segment is a contiguous group of blocks,
> starting at a 24-bit cluster number and containing a 16-bit cluster
> count. This allows a file to use a maximum of 3145680 clusters, or
> 805294080 bytes if a cluster is a single sector and those sectors are
> 256 bytes in length. This , of course, means that the limitation on
> file size is technically higher than that of the filesystem as a
> whole.
>
> HOWEVER...
>
> 6x09 RBF can only deal with one disk allocation map (DAM) sector at a
> time. As I have discovered, currently a file segment must never be
> allowed to span a single sector, or Bad Things(tm) happen when you try
> to delete it.
>
> RBF on 6x09 CPUs never creates file segments that span a single DAM
> sector. If a file segment starts in the middle of a DAM sector, that
> first segment will end at a $800 cluster boundary, and another one
> will be created at the start of the next. As well, if a file segment
> starts at a DAM sector boundary, then the maximum segment size is $800
> clusters ($800 bits == $100 bytes == 1 allocation map sector).
>
> Because of these two limitations (48 segments * 2048 clusters), the
> maximum possible size of a file created by 6x09 RBF on a drive with 1
> sector per cluster is exactly 24 megabytes. Files larger than this are
> possible, and most things can deal with them, but RBF can't make them
> or delete them without something crazy happening.
>
> Now, presumably, other versions of OS-9 do not share this limitation,
> which may be why Toolshed doesn't take it into account. But it can
> result in certified bad juju if you try to delete a big file dropped
> onto a disk by Toolshed definitely, and possibly other tools.
>
>
>
> On Thu, May 28, 2020 at 3:48 PM L. Curtis Boyle
> <curtisboyle at sasktel.net>
>
> wrote:
> > I am wondering if the these files were created ToolShed or something
> > similar. I think I recall the 1 sector of the BAM at a time too, and
> > if RBF internally only allows allocating one sector of BAM at a time
> > (so 2048 sectors), and maxes the size of each segment to match, it
> > makes sense that having an external tool create segments too large
> > would overflow its internal buffer. Jeff - if you copy that file to
> > a duplicate from within NitrOS9 itself, see if it makes the new copy
> > with more than one segment. If it does, see if it will delete that
> > version ok. (I am still out for a bit, so I can’t try it here).
> >
> > Sent from my iPhone
> >
> > > On May 28, 2020, at 1:25 PM, Gene Heskett <gheskett at shentel.net>
> > > wrote:
> > >
> > > On Thursday 28 May 2020 13:43:52 Jeff Teunissen wrote:
> > >> I'm aware of the 18 bits in the DAM for a bootable floppy; this
> > >> is more like thousands of them.
> > >
> > > Wow! This is a place for something that works like git's bisect,
> > > but I have no clue if hg can do that. I can say that at the time I
> > > last walked around in that code 25 years ago, something like this
> > > never happened.
> > >
> > > But I also recall that rbf keeps only 1 sector of the FAM in
> > > memory at one time.  So I'd see if I could locate the code that
> > > handles the search for clean bits and see what it does when there
> > > is not enough in the current sector to satisfy the request. How
> > > does it discover that, and go on to load the next sector and
> > > continue the search?  It all worked back then but people have
> > > since merged and conditionalized that since since I was building
> > > separate versions for 6809 and 6309.  That was back in 1.16 days.
> > > Now, theoretically, it runs optimally on either cpu, but I didn't
> > > do the merging.
> > >
> > > Except now it is not, and its not likely I could ever get my head
> >
> > wrapped
> >
> > > around todays rbf.mn code again.
> > >
One other thing I should point out as this conversation is refreshing my 
memory a bit, is the segment allocation size (DD.SAS) in the disk 
descriptor, which may, for old installs, still be set at 8 which does 
allow all 48 segment 5 byte entries, 24 bits for offset start, and 16 
bits for length, to be used.  There is no recovery for that, never has 
been, probably never will be.

I submitted a patch that expands that DD.SAS to $20 (32 decimal) sectors 
as insurance of never hitting the end of that 48 segments because if you 
do, its blowup time. Use dmode to check that, and or fix it if yours 
hasn't been corrected to $20. Do it NOW. The only side effect will be 
the inability to use the last 31 sectors of a floppy.

But what size is that set to in the toolshed disk code?  If it is still 
using 8, there quite likely is your problem. When set at $20, but the 
file does not use that $20, rbf gives back at the end of the 
transaction, all sectors that are not actually used. But you cannot play 
mix-n-match with the DD.SAS, it must be consistent across all software 
that can access that storage media.  And of course if toolshed is 
cluster size sensitive, that must also match what the disk (or file) is 
formatted to. I don't know that toolshed is cluster size sensitive.  Its 
possible it is not. I've never kicked its tires as its newer than when I 
worked on rbf, and after a pulmonary embolism several years ago at 79 
yo, I am not capable of doing it today at 85 yo. Those things, with a 
98+% fatal rate, are hard on the IQ.  There are folks here who could, 
but I'm sadly no longer in that group.  Now its up to the newer 
generaion of coco/nitros9 fans here to find this one and fix it.

Obviously toolshed's version of dd.sas should be checked, and if not 
fixable, disabled from ever working with os9 formatted media. Or 
disabled if cluster sizes don't match.  Thats 2 conditions it should 
check for.
[...]

Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
If we desire respect for the law, we must first make the law respectable.
 - Louis D. Brandeis
Genes Web page <http://geneslinuxbox.net:6309/gene>