BetaArchive Logo
Navigation Home Database Screenshots Gallery Image Uploader Server Info FTP Servers Wiki Forum RSS Feed Rules Please Donate
UP: 59d, 15h, 37m | CPU: 21% | MEM: 5054MB of 12235MB used
{The community for beta collectors}

Post new topic Reply to topic  [ 11 posts ] 
Author Message
 PostPost subject: Converting KF to DC42?        Posted: Thu Mar 09, 2017 11:35 pm 
Reply with quote
FTP Access
Offline

Joined
Sat Feb 04, 2017 12:59 am

Posts
28

Favourite OS
Debian Linux Stable
I'm considering the purchase of a KF to contribute better images of the large Mac software collection I have. What I'm curious about, and something I have been unable to find clear instructions for and ascertain is if it is possible/how to convert KF raw stream to Apple Disk Copy 4.2 format images. I would strongly prefer not to have to image disks twice as I have many hundreds of them and even imaging and scanning them all once is likely going to be a multi-year endeavor. It's also important to me that I can access the disk images from my vintage Mac hardware (without writing back out to a floppy).

So has anyone tried converting a KF raw stream into a DC42 format file? Have you tried opening the resulting image on actual Mac hardware? It looks like this might be feasible with the PSI utility from the PCE Tools package, but I haven't found any reference to people trying or successfully doing it (only going the other way, from DC42 to a KF raw stream and then writing to disk).

In the absence of someone confirming if/how this can be done I'll try it myself, but not having a KF I'd be looking for a sample 400k, 800k, and 1.44MB Macintosh KF raw stream files that I can test with.


Top  Profile
 PostPost subject: Re: Converting KF to DC42?        Posted: Fri Mar 10, 2017 1:45 am 
Reply with quote
Offline

Joined
Fri Sep 07, 2012 6:45 pm

Posts
164
You can ask to BMOW to create a working converter if it doesn't exist. He's the creator of the only existent GCR floppy emulator in the world, and he knows everything about the DC42 format, because that's the format he uses for his own project. I'm sure he would be glad to make the converter if that makes more old software in its original format available for his emulator platform.


Top  Profile
 PostPost subject: Re: Converting KF to DC42?        Posted: Fri Mar 10, 2017 3:01 am 
Reply with quote
FTP Access
Offline

Joined
Sat Feb 04, 2017 12:59 am

Posts
28

Favourite OS
Debian Linux Stable
My understanding of the DC42 format is that it is essentially a raw image with an extra header tacked on the front so I agree, it shouldn't be too hard to create a converter. I kind of suspect there's a way to do this already, I just want to be certain before I commit to the KF. One of the nice things about DC42 also is that it doesn't store critical information in the resource fork so the files can be created/stored on non-HFS platforms and still usable from Macs.

I believe that mrpijey has a KF as well as some Mac software so I'm hoping that he can either confirm a procedure or point me towards a 400k, 800k, and 1.44MB KF stream so that I can check it myself. There's just not an easy way to dig through the FTP by dump type so I figure someone who uploaded something is probably the most likely to know where to look.


Top  Profile
 PostPost subject: Re: Converting KF to DC42?        Posted: Fri Mar 10, 2017 9:37 am 
Reply with quote
Donator
User avatar
Offline

Joined
Fri May 14, 2010 1:29 pm

Posts
873

Location
Southern Germany

Favourite OS
IRIX 5.3
Hyoenmadan86 wrote:
...the only existent GCR floppy emulator in the world

the only Apple GCR floppy emulator in the world you mean ;-)

OldMac wrote:
...point me towards a 400k, 800k, and 1.44MB KF stream so that I can check it myself

For testing you can try some of my Apple 800k KF images that I uploaded to archive.org, this one for example.

_________________
I upload stuff to archive.org from time to time. See here for everything that doesn't fit BA


Top  Profile
 PostPost subject: Re: Converting KF to DC42?        Posted: Fri Mar 10, 2017 9:35 pm 
Reply with quote
FTP Access
Offline

Joined
Sat Feb 04, 2017 12:59 am

Posts
28

Favourite OS
Debian Linux Stable
So I've been trying to make this work for a while now using the images Darkstar linked to, but I'm not having much success.

What I think should be possible is to use the PCE Tools to go from KF Stream -> PFI -> PRI -> PSI -> DC42 (though I also think that it may be possible to skip at least the PFI step and go straight from KF Stream to PRI).

I seem to be able to convert the KF Stream to PRI using this command (this skips the PFI step):
Code:
pfi.exe track00.0.raw -p decode gcr-raw disk1.pri

but the PRI file doesn't seem to be usable. I think the next command in the chain should be:
Code:
pri.exe disk1.pri -p decode gcr-raw disk1.psi

but this gives a "loading failed (disk1.pri)" error. Even a simple query on the file:
Code:
pri.exe disk1.pri -p info

fails to load.

I think this should be feasible and I strongly suspect I just don't have the command figured out but the documentation (official and unofficial) on the PCE Tools is...lacking, doubly so when it comes to working with Macintosh GCR data. There are only a few references to it that I've found at all such as this one which is trying to go the opposite way (DC42 to KF).


Top  Profile
 PostPost subject: Re: Converting KF to DC42?        Posted: Sat Mar 11, 2017 8:30 am 
Reply with quote
FTP Access
Offline

Joined
Wed Dec 11, 2013 3:24 am

Posts
219

Favourite OS
Pencil and paper
The kryoflux natively supports decoding 400k and 800k disks in to sector image files (usually named with .IMG extension). These can be imported in to a Mac file system, and opened in Apple DiskCopy. (setting a type of "rohd" with a creator of "ddsk" works for me) Unlike DC42, a plain sector image file does not store Apple GCR sector tags. However, sector tags aren't actually used on Mac so this is only an issue for archiving Lisa disks.

BTW, you don't need a Kryoflux to use the Kryoflux software. You can load up their software, select the stream file as the source instead of a floppy, select the format you want to convert to, and let it rip.

The PCE stuff is a little goofy about how it works with Mac images. PCE expects images to appear as if they were read on an Apple variable bit rate floppy drive - all tracks are read and written at the same bit rate but with different track lengths and RPM values. However, that is not how PC 1.44mb floppy drives work - the KF sees different bitrates and always 300RPM. (The reason for this is that PCE has a Mac emulation mode that closely emulates the real hardware)

The utilities with PCE can create a Kroflux image from a Mac sector image or DC42, but the RPMs will be "wrong". Oddly, the Kryoflux can adjust for this, but for some reason only after the image is fed through HxC.

Going the other way with PCE is a bit of a hack as you have to manually specify the bitrate (as seen by the 300RPM 1.44mb drive), and this method will still only create a sector image, not a DC42. So you gain little over the Kryoflux's built in decoding.

Kryoflux does not natively support ENcoding either format (IMG or DC42) to a stream file. Only DEcoding to IMG.

Mac 1.44mb disks use exactly the same low-level format as IBM 1.44mb disks.

the following PCE commands will convert a Kryoflux (or SuperCard Pro image, if you specify one instead) image to an IMG sector file. (Note: for best results on non-index aligned disks, sample at least three revolutions and decode the second with -R 2)

Code:
pfi "track00.0.raw" disk1.pfi

pfi disk1.pfi -c 00-15 -R 2 -r 381310 -p decode pri diska.pri
pri diska.pri -p auto-align-gcr -p decode gcr diska.psi -f -v

pfi disk1.pfi -c 16-31 -R 2 -r 349510 -p decode pri diskb.pri
pri diskb.pri -p auto-align-gcr -p decode gcr diskb.psi -f -v

pfi disk1.pfi -c 32-47 -R 2 -r 317700 -p decode pri diskc.pri
pri diskc.pri -p auto-align-gcr -p decode gcr diskc.psi -f -v

pfi disk1.pfi -c 48-63 -R 2 -r 285950 -p decode pri diskd.pri
pri diskd.pri -p auto-align-gcr -p decode gcr diskd.psi -f -v

pfi disk1.pfi -c 64-79 -R 2 -r 254190 -p decode pri diske.pri
pri diske.pri -p auto-align-gcr -p decode gcr diske.psi -f -v

psi diska.psi -m diskb.psi -m diskc.psi -m diskd.psi -m diske.psi disk1.psi
psi disk1.psi disk1.img
psi disk1.psi -L > disk1.txt

erase disk1.pfi
erase disk1.pri
erase diska.pri
erase diskb.pri
erase diskc.pri
erase diskd.pri
erase diske.pri
erase diska.psi
erase diskb.psi
erase diskc.psi
erase diskd.psi
erase diske.psi

As a final step, you actually can convert the IMG to DC42 using the command:

psi disk1.img disk1.dc42 -I mac -O dc42

(It just occurred to me you might be able to go directly to DC42 in the above commands using: psi disk1.psi disk1.dc42 -O dc42, but haven't tried that)

The following will encode an IMG sector file to a Kryoflux stream file - but for some reason the resulting steam file must be opened and resaved with HxC first. Does not seem to work with SuperCard Pro. Resulting image has RPM values that expect a Mac VBR drive, but Kryoflux adjusts for this.

Code:
psi disk1.img disk1.psi -I mac
rem or use psi disk1.dc42 disk1.psi
psi disk1.psi -L > disk1.txt

pri -p encode gcr disk1.psi disk1.pri -f -v
pri disk1.pri -l -L -p info > disk1pri.txt

pfi -p encode pri disk1.pri disk1.pfi
pfi disk1.pfi track00.0.raw


Top  Profile
 PostPost subject: Re: Converting KF to DC42?        Posted: Sun Mar 12, 2017 12:30 am 
Reply with quote
FTP Access
Offline

Joined
Sat Feb 04, 2017 12:59 am

Posts
28

Favourite OS
Debian Linux Stable
Thanks for the tips, I think we're making some progress but I still haven't been able to actually make it work...

Regarding the PCE method, I tried that but at least my build of PFI crashes when attempting to execute the
Code:
pfi disk1.pfi -c 00-15 -R 2 -r 381310 -p decode pri diska.pri

command.

As it happened I thought that DTC might be able to do this so I had downloaded a copy of that. Of course I'm able to seemingly successfully output raw sector IMG files from the stream files. For good measure I used a psi command to make a dc42 version of the sector images as well.
Code:
psi.exe disk1.img disk1.dc42 -I mac -O dc42


I then copied the images to my AppleShare server and copied them back down to both the Basilisk II emulator and real Mac hardware. Using resedit I added the "rohd" type and "ddsk" creator to both the IMG and DC42 files and tried accessing them with DiskCopy 6.3.3. DiskCopy returned a "The Mount Image operation did not complete. -39 An unexpected end-of-file was encountered in "disk1.img" which may indicate that it is corrupted or damaged." error message. The same message happened on both the emulator and real hardware and on both the IMG and DC42 files.

As I've also read that the type/creator should be dImg/dCpy (see here for example) I tried that as well. On Basilisk opening those caused a crash of the Basilisk application. On the real hardware it generates an error message "The Mount Image operation did not complete. (-8816) "disk1.img" is damages and cannot be used.

Thinking there may actually be a problem with the Photoshop image you linked to I also downloaded a couple more of yours including Beyond Dark Castle, MacDraw, and for good measure the LaserWriter disk set which is 1.44MB to see if that might be an issue but had the same problems with those.

Since it sounds like you have done this successfully any ideas on what I'm doing wrong?


Top  Profile
 PostPost subject: Re: Converting KF to DC42?        Posted: Sun Mar 12, 2017 5:37 pm 
Reply with quote
FTP Access
Offline

Joined
Wed Dec 11, 2013 3:24 am

Posts
219

Favourite OS
Pencil and paper
Thats odd, I've never had PFI crash on me before. The latest version is here: http://www.hampa.ch/pub/pce/pre/pce-201 ... -win32.zip

I noticed that disk1 of the photoshop set linked above would not decode quite right unless the -p auto-align-gcr command was removed, but that command can help with other images. Also noticed PSI won't create a DC42 unless all sectors are 100% accounted for.

I also got that length error when trying to import an IMG on to a Power Mac, but it works fine in Basillisk II under MacOS 7.5.5 with DiskCopy 6.1.2 (Same DiskCopy 6.1.2 on the Power Mac). So, not sure what the deal is there.

Yes, the creator and type should be dImg/dCpy for DC42.

Edit: For the sector image files, try dimg/ddsk - note this garbage appears to be case sensitive.


Top  Profile
 PostPost subject: Re: Converting KF to DC42?        Posted: Sun Mar 12, 2017 8:18 pm 
Reply with quote
FTP Access
Offline

Joined
Sat Feb 04, 2017 12:59 am

Posts
28

Favourite OS
Debian Linux Stable
DTC actually showed a problem with the decoding on the first sector of the Photoshop disk as well, that's part of why I downloaded some of the other images to try.

Interestingly the length error seems to only happen for me with the type/creator set to rohd/ddsk and the alternative error (on physical hardware) and crash (on Basilisk) come when set to dImg/dCpy. The finder icons for the files change when these are set, even though both are recognized by DiskCopy so there must have been a change at some point. I did check a disk image that I actually created with DiskCopy 6.3.3 and it was set to rohd/ddsk I believe.

Next step I guess is to see about tracking down a copy of DiskCopy 6.1.2 and if that makes a difference. The physical mac I've been trying on is also an OS 9 PPC based system, another variable to try is for me to pull out one of my 68k Macs with OS 7 on it and see if they can mount the images.

I was also using the latest copy of PFI, though in this case I was using it on Windows. I may try giving that another go on Linux though. In the end I would want to automate the processing of the stream files into DC42 files by scripting it in Linux anyway (though it seems likely that I'd need to write some Applescript to run on a Mac to add the resource forks/type/creator and binhex the files as I haven't found a way to do that in Linux either yet).


Top  Profile
 PostPost subject: Re: Converting KF to DC42?        Posted: Sun Mar 12, 2017 9:41 pm 
Reply with quote
FTP Access
Offline

Joined
Sat Feb 04, 2017 12:59 am

Posts
28

Favourite OS
Debian Linux Stable
Slight detour... I made some progress on the automation part of this. There exists both a Python and Perl library for creating binhex files, though it looks like there were possibly some changes to Python 3 which may not support the Macintosh (at least the Python library manual has a cryptic reference to that -- my guess is they may have removed the code that dealt with resource forks) and there are also a series of programs in the *NIX "macutils" package which can output either MacBinary or BinHex files and set the file type/creator data for the encoded file. At least one of these three should be able to work.

So... if we can figure out how to convert the KF streams reliably to DC42 images that can be read on both emulators and physical systems I should be able to automate the process. I already have scripts that I wrote to help process the CD images (mass 7zipping of data files and photo files, file naming, etc.) I'm doing while we sort out the floppy image conversion so I should be able to adapt that and combine with the floppy imaging process. My automation goal for the floppies if we can get the KF conversion to work would be:
1) Scan a batch of floppies (and auxiliary material if applicable) and save PNG files in a directory for each software program
2) Image the batch of floppies with KF/DTC to stream files, each disk as a subdirectory of the software program it belongs to
3) Run script which will create derivative DC images from stream files, BinHex those files with correct type/creator data, create 7z files of everything and prepare for upload

I've already processed about 50 CDs using a similar process (minus the KF conversion obviously) and I've got another 125 or so of those to go. I did an additional 10 or so manually and decided that was going to be a non-starter, the batch processing (scan a bunch at once, image a bunch at once, use a script to 7zip them overnight) is key to making this not be excruciatingly slow. Hopefully we can figure out the process to do the KF stream to consistently working DC image because I don't relish having to add the extra step of imaging all the disks a second time on a mac to create the DC images and it's important to me the software be provided in working DC images too because honestly that's how most people will use it. The number of floppies to image dwarfs the CDs...


Top  Profile
 PostPost subject: Re: Converting KF to DC42?        Posted: Thu Apr 13, 2017 4:57 am 
Reply with quote
FTP Access
Offline

Joined
Sat Feb 04, 2017 12:59 am

Posts
28

Favourite OS
Debian Linux Stable
Yeah! Tonight I was able to successfully test a suggested method for converting the KF raw flux data into working DC42 files. I have tested opening the files both with an emulator and on a physical vintage mac and the files are accessible. The method is:
1) Use DCT to convert the raw flux data to a Mac400k/800k IMG file
2) Use psi.exe as "psi.exe disk1.img disk1.dc42 -I mac -O dc42" to convert the sector image into a DiskCopy 4.2 image
3) Use ResEdit on a Mac to create the resource fork in the DC42 file and set the type/creator to dImg/dCpy
4) Open/Mount the DC42 file with DiskCopy on the Mac

To give credit where it is due I think that JustZisGuy pretty much suggested this above but it was my lack of understanding about DCT which was the issue. I thought that when I chose Mac 400k/800k as the output image type in DCT it was going to create a DC42 format file so I just took what it created into the Mac and created the resource fork with the type/creator data. As it turns out the Mac 400k/800k output from DCT is NOT DC42, it's still an IMG type disk sector image. In order to be usable by DiskCopy (at least as far as I've found, though some of what I've read indicates this IMG format should work with the rohd/ddsk type/creator in some cases) it still must be converted to DC42 format with psi.exe in order to be usable as dImg/dCpy.

Time being as limited as it is I haven't gotten back to the rest of the CDs which I still need to work through before I tackle the floppies but I'm satisfied this will work. I do still plan to automate the DC42 creation as part of my process as I indicated above and I'll probably share the script here when I get there. I'm still undecided about preserving the resource fork type/creator data using MacBinary vs. BinHex. Input on that is welcome. Although StuffIt is also commonly used for this I'm ruling that out because AFAIK there is no Linux library core creating the SIT files and setting the type/creator data on the file(s) inside them so I can't automate it in a single step. Also, MacBinary and BinHex are not super-obscure or anything, they are also pretty standard ways of distributing forked files over the Internet.


Top  Profile
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 11 posts ] 




Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to:  

All views expressed in these forums are those of the author and do not necessarily represent the views of the BetaArchive site owner.

Powered by phpBB® Forum Software © phpBB Group

Copyright © 2006-2020

 

Sitemap | XML | RSS