[Coco] Large file deletion fails on NitrOS9/6309

Thu May 28 19:52:48 EDT 2020

The file *was* put on the disk using Toolshed, and as it turns out, that
was the problem.

After some investigation with Curtis, we have figured out that Toolshed
creates files that can't currently be deleted (safely) by (Nitr)OS-9/6x09.

The on-disk structure of an RBF file provides for a file to contain up to
48 segments, where a segment is a contiguous group of blocks, starting at a
24-bit cluster number and containing a 16-bit cluster count. This allows a
file to use a maximum of 3145680 clusters, or 805294080 bytes if a cluster
is a single sector and those sectors are 256 bytes in length. This , of
course, means that the limitation on file size is technically higher than
that of the filesystem as a whole.

HOWEVER...

6x09 RBF can only deal with one disk allocation map (DAM) sector at a time.
As I have discovered, currently a file segment must never be allowed to
span a single sector, or Bad Things(tm) happen when you try to delete it.

RBF on 6x09 CPUs never creates file segments that span a single DAM sector.
If a file segment starts in the middle of a DAM sector, that first segment
will end at a $800 cluster boundary, and another one will be created at the
start of the next. As well, if a file segment starts at a DAM sector
boundary, then the maximum segment size is $800 clusters ($800 bits == $100
bytes == 1 allocation map sector).

Because of these two limitations (48 segments * 2048 clusters), the maximum
possible size of a file created by 6x09 RBF on a drive with 1 sector per
cluster is exactly 24 megabytes. Files larger than this are possible, and
most things can deal with them, but RBF can't make them or delete them
without something crazy happening.

Now, presumably, other versions of OS-9 do not share this limitation, which
may be why Toolshed doesn't take it into account. But it can result in
certified bad juju if you try to delete a big file dropped onto a disk by
Toolshed definitely, and possibly other tools.

On Thu, May 28, 2020 at 3:48 PM L. Curtis Boyle <curtisboyle at sasktel.net>
wrote:

> I am wondering if the these files were created ToolShed or something
> similar. I think I recall the 1 sector of the BAM at a time too, and if RBF
> internally only allows allocating one sector of BAM at a time (so 2048
> sectors), and maxes the size of each segment to match, it makes sense that
> having an external tool create segments too large would overflow its
> internal buffer. Jeff - if you copy that file to a duplicate from within
> NitrOS9 itself, see if it makes the new copy with more than one segment. If
> it does, see if it will delete that version ok. (I am still out for a bit,
> so I can’t try it here).
>
> Sent from my iPhone
>
> > On May 28, 2020, at 1:25 PM, Gene Heskett <gheskett at shentel.net> wrote:
> >
> > On Thursday 28 May 2020 13:43:52 Jeff Teunissen wrote:
> >
> >> I'm aware of the 18 bits in the DAM for a bootable floppy; this is
> >> more like thousands of them.
> >>
> > Wow! This is a place for something that works like git's bisect, but I
> > have no clue if hg can do that. I can say that at the time I last walked
> > around in that code 25 years ago, something like this never happened.
> >
> > But I also recall that rbf keeps only 1 sector of the FAM in memory at
> > one time.  So I'd see if I could locate the code that handles the search
> > for clean bits and see what it does when there is not enough in the
> > current sector to satisfy the request. How does it discover that, and go
> > on to load the next sector and continue the search?  It all worked back
> > then but people have since merged and conditionalized that since since I
> > was building separate versions for 6809 and 6309.  That was back in 1.16
> > days. Now, theoretically, it runs optimally on either cpu, but I didn't
> > do the merging.
> >
> > Except now it is not, and its not likely I could ever get my head
> wrapped
> > around todays rbf.mn code again.
> >
> >> On Thu, May 28, 2020 at 12:28 PM Gene Heskett <gheskett at shentel.net>
> > wrote:
> >>> On Thursday 28 May 2020 09:06:28 Robert Gault wrote:
> >>>> Jeff,
> >>>>
> >>>> Your problem may not be as straight forward as you think. I ran a
> >>>> test with MESS 0.158 and VCC on a file with 5,958,126 bytes and
> >>>> was able to delete it without any problems. The disk used was a
> >>>> .vhd with $5,A000 OS-9 sectors. However, there was one small
> >>>> catch. :)
> >>>>
> >>>> The first time I tried to delete the file using MESS, the system
> >>>> froze. After rebooting, I was able to delete the file. I then did
> >>>> a DCHECK of the .vhd and found MANY clusters were in the map but
> >>>> not in the file structure.
> >>>> After correcting the allocation map, I was able to repeatably put
> >>>> the file back on the .vhd and then delete it without any problem.
> >>>>
> >>>> So it may be there is a problem with a system file like ioman or
> >>>> it may be that your disks had problems unrelated to file size that
> >>>> caused DEL to fail for you. Why don't you try doing a dcheck on a
> >>>> disk, copy your cococ.ar file to that disk, and then try to delete
> >>>> it. See if you still get the lock-up.
> >>>>
> >>>> Robert
> >>>>
> >>>> Jeff Teunissen wrote:
> >>>>> Deleting files that cross a $x000 cluster boundary seems to be
> >>>>> broken in the latest NitrOS9, and probably has been for a very
> >>>>> long time.
> >>>>>
> >>>>> I have an uncompressed .ar file, 'cococ.ar', that's about 3
> >>>>> megabytes in size. I use it to move up-to-date C source code
> >>>>> from my Linux system to my virtual CoCo hard disks. When I
> >>>>> delete the archive from the disk, the system mostly locks up --
> >>>>> that is, the "delete" command never finishes and no programs can
> >>>>> be run, though things like CLEAR to change windows still
> >>>>> function normally.
> >>>>>
> >>>>> When I issue the command "del cococ.ar", the directory entry is
> >>>>> marked deleted as expected, and the allocation map is
> >>>>> updated...but it seems to stop updating the disk allocation map
> >>>>> at the next $x000 cluster boundary.
> >>>>>
> >>>>> On the disk, the file starts at cluster $189C and goes all the
> >>>>> way to cluster $5A80, and is referenced in LSNs 4-12. The
> >>>>> deletion succeeds up to the end of LSN 4, which describes
> >>>>> clusters $1800-$1FFF, and then nothing further works.
> >>>
> >>> Jeff should also be aware that dcheck has never learned to ignore
> >>> boot tracks and there will be 18 sectors in the allocation map that
> >>> are not in the file structure, unless he has also previously
> >>> executed my krnl-to-dir.b09 utility.  It generates a dir entry and a
> >>> FD sector that accounts for the boot track on track 34. In that
> >>> event, you will have a directory entry "KERNAL", miss-spelling
> >>> intentional in the root dir of that disk or disk image.
> >>>
> >>>
> >>> Cheers, Gene Heskett
> >>> --
> >>> "There are four boxes to be used in defense of liberty:
> >>> soap, ballot, jury, and ammo. Please use in that order."
> >>> -Ed Howdershelt (Author)
> >>> If we desire respect for the law, we must first make the law
> >>> respectable. - Louis D. Brandeis
> >>> Genes Web page <http://geneslinuxbox.net:6309/gene>
> >>>
> >>> --
> >>> Coco mailing list
> >>> Coco at maltedmedia.com
> >>> https://pairlist5.pair.net/mailman/listinfo/coco
> >
> >
> > Cheers, Gene Heskett
> > --
> > "There are four boxes to be used in defense of liberty:
> > soap, ballot, jury, and ammo. Please use in that order."
> > -Ed Howdershelt (Author)
> > If we desire respect for the law, we must first make the law respectable.
> > - Louis D. Brandeis
> > Genes Web page <http://geneslinuxbox.net:6309/gene>
> >
> > --
> > Coco mailing list
> > Coco at maltedmedia.com
> > https://pairlist5.pair.net/mailman/listinfo/coco
> >
>
>
> --
> Coco mailing list
> Coco at maltedmedia.com
> https://pairlist5.pair.net/mailman/listinfo/coco
>