Community Discussion: Preserving Obscure Software Distributi

Discussion of beta and abandonware topics not fit for the other forums goes here.
Post Reply
jackhenry
Posts: 4
Joined: Fri Feb 07, 2020 6:00 pm

Community Discussion: Preserving Obscure Software Distributi

Post by jackhenry »

Hello to all the members of BetaArchive. Today I would like to start a discussion to gather your thoughts, opinions, and viewpoints on the subject of preserving obscure software formats. As you are doubtless aware, the BetaArchive contribution rules clearly state that if you wish to make a contribution to BetaArchive, you must make a perfect 1:1 copy of the physical medium the software was distributed on via either MDF for optical media (CDs, DVDs, Blu-Rays, etc.) or via a KryoFlux dump. We know that any other duplication method is prohibited for preservation on BetaArchive (and BetaArchive’s methods and standards should be the standard for all software preservation) as any other method makes inferior copies of physical media unsuitable for any archival application. Additionally, the BetaArchive contribution rules clearly state that a high-resolution scan must be provided so that all media artwork, labels, information, etc. can be perfectly preserved for future generations and, if the need arises, duplicated. Boxes and manuals must also be preserved in the same way but that is a trivial task.

Of course, these rules are all well and good for common formats such as the venerable floppy disk or compact disc which have been around for decades, perhaps longer than some people on this Internet forum have been alive – please understand I am not intending for this to be taken as a slight against any member of BetaArchive; this is merely a statement of fact. However, not all computer software was distributed on floppy disk or compact disc; indeed, there were many alternate distribution methods out there some of which predate floppy disks and compact discs, and others which were designed to overcome other limitations of these formats. In this post I would like to generate a discussion on the different obscure distribution formats that I have seen as well as a community discussion on the best way to properly archive the data and physical qualities of each medium.

Punch Cards and Paper Tape
Let us start with perhaps the oldest software distribution format known to mankind, the punched card. Wikipedia claims that punched cards were introduced in the late 1700s as a way to mass-produce weaved garments where you could punch holes in a series of cards to make a garment a certain way and from that moment on if you wanted to make a new copy of that garment all that would have to be done would be to load the punched cards again and the machine would make an exact duplicate of the garment. By the 1920s, International Business Machines (IBM) had adapted the punched card as a computer storage medium until such a time as magnetic tape and floppy disks were both invented and came down in price to be utilized as a data storage method.

As punched cards are simply a sheet of thick paper, it is a trivial matter to scan in each individual card on both sides at a high resolution. The scans could then be used to duplicate the punched card software at a later date, but is of little value if you were in need of a binary copy that you could quickly load into a mainframe. The question is, how do you preserve punched card software in a binary format? Are there tools available to read commercial punched card software releases on modern computers? Can you even connect a punched card reader to a modern computer to preserve the software in a binary format?

The next iteration of punched cards was paper tape, perhaps most famously known due to Microsoft Corporation’s Altair BASIC (Beginner’s All-purpose Symbolic Instruction Code) or more accurately, an incident which happened with Altair BASIC. For those who are unaware, the incident in question happened in 1976 when certain computer hobbyists came together and illegally duplicated copies of Microsoft’s Altair BASIC (which at the time was distributed on paper tape and therefore could easily be copied by anyone who had access to a paper tape punch) and resulted in the infamous “Open Letter to Hobbyists” in which none other than Bill Gates himself accused the entire budding computer hobbyist scene of being thieves.

Like the punched card, paper tape is simply paper with a pattern of holes punched into it to represent data. Unliked a punched card, paper tape is quite easy to preserve as many mini- and microcomputers of the time featured RS-232 interfaces which would have also had an optional paper tape reader that would communicate over that method. This brings us to the question of how to make scans of paper tape. As the name suggests, paper tape is simply a long, continuous spool of paper which makes it quite difficult to scan. How would you suggest performing a high resolution scan of paper tape? I am aware that there are consumer-grade products meant for scanning identification cards such as a driver’s license which can feed documents in with rollers, but I am not certain of their scanning capabilities or if such a scanner would be adequate to comply with BetaArchive’s contribution rules.

Magnetic Tape – Industrial
When the Allied forces won World War II, we discovered one of Germany’s inventions – the audio tape recorder. This device stored spoken word on tape containing metallic particles that could be magnetized to store information and later played back. Of course once the technology found its way back to the United Kingdom and the United States of America it would only be a matter of time before magnetic tape found its way into the world of computer technology, where even to this day magnetic tape is still used and relied upon to store data.

The two main formats for distributing software on tape (to the best of my knowledge, at least) were open-reel tape, in which you could easily unspool the tape from its reel if you were not careful, and data cartridges, which contained both the supply reel and the takeup reel in a single, rugged cartridge that could be handled easily without too much fear of mishandling damaging the tape. Preservation of the label is straightforward enough; if the software had a label it was either typewritten or handwritten and stuck somewhere on the tape or carrier. My question is, what is the best method of preserving the data on the tape? Open-reel tape drives and data cartridge drives are still plentiful on second-hand sites such as eBay, but are there any tools available to us that can make a 1:1 copy of a magnetic tape akin to that of the KryoFlux? Is using the venerable old UNIX tool dd sufficient for making copies of magnetic tapes for release? There are also things such as LINCtape and DECtape used in the Digital Equipment Corporation PDP series of minicomputers which use a tape size unique to those computers, meaning if one were to attempt to archive software for a PDP they would have to have a fully restored and working PDP and tape drive.

Magnetic Tape – Consumer

Of course, time makes all technology cheaper and it wouldn’t be long before magnetic tape left the world of professional and industrial markets and entered the home. Computer software meant for home computers was often distributed on cassette tape (first introduced by Philips in 1962 as the Compact Cassette) as standard audiotape mechanisms could be utilized to store data. Notable examples of computers that used magnetic tape were the Coleco ADAM (which was also infamous for erasing tapes left in or on the computer during startup), the Commodore VIC-20 (or VC-20 in Germany), and the Sinclair ZX Spectrum in the United Kingdom.

Like the industrial-grade data tapes above, archiving the label and packaging for such software is trivial with a typical flatbed scanner, it is the preservation of the digital data on the tape that is the problem – and it is perhaps even more of a problem as consumer-grade tape formulations and tape players could vary wildly in quality, especially once cassette tape started to go out of fashion in the late 90s and early 2000s. How then should consumer tape formats be archived? Is it acceptable to play the tape back in an ordinary audio tape deck? If so, is any kind of tape deck acceptable or is only the highest-end machine (for example a Nakamichi Dragon) acceptable? What about recording the audio to a computer? Can you use a low-end analog-to-digital converter such as the Realtek High Definition Audio codec, or should you take your tapes to a professional recording studio and have them use their industrial-quality gear to copy the tape? And of course, is there a way to make a perfect 1:1 copy of a cassette tape like a KryoFlux can for a floppy disk?

Floppy Disks – 8 Inch
The floppy disk was invented by International Business Machines in the late 60s, and upon introduction became the de facto standard for software distribution for not only IBM mainframes but hundreds of mini- and microcomputers until the introduction of the 5.25” floppy disk. A notable example of the 8-inch floppy was perhaps with Seattle Computer Products, who originally wrote QDOS (later 86-DOS) which would later be bought by Microsoft to become MS-DOS for IBM’s original PC 5150. This original version of DOS relied solely on 8-inch floppies; support for 5.25” disks was not in 86-DOS and was only added by Microsoft.

Again, scanning labels is a trivial task and can be accomplished with any flatbed scanner. Archiving the digital data is a question that perhaps can be answered a little easier than any other question that has been posed thus far – I have read reports of some people successfully connecting 8-inch floppy drives to a modern PC, being able to read and write to the drive as if it were nothing more than a comically large 5.25” floppy drive. If this is the case, then is it possible to connect an 8-inch floppy drive to a KryoFlux and perform a 1:1 copy of the data that way? If not, is it acceptable to perform a disk image dump using some other tool like UNIX’s dd?

ROM Software
Finally, the last method of uncommon software distribution is software that was distributed in read-only memory. Please note that in this section I will be talking about classic ROM chips and modern-day flash storage together, as while they have fundamental differences between them they also have similarities – namely, that they are both semiconductor-based storage mediums that are fabbed by a chip production company.

The most well-known use of ROM for distributing software is the video game industry. From the first microprocessor-based arcade games to the modern-day Nintendo Switch, ROM software has been a staple of the video game industry and for good reason – it’s reliable, doesn’t behave suboptimally when temperatures swing high or low within reason, is randomly-accessible and is easy to duplicate. Of course, BetaArchive is not interested in becoming a ROM site, even less so with Nintendo eager to send a cease and desist notice to anyone who dares even think about downloading a ROM instead of using their Virtual Console services. But BetaArchive is interested in preserving prototype games I would assume, and thus there is a need to properly preserve ROMs. Additionally, more modern uses of ROM include things like personal digital assistants (PDAs) and of course, iPhones running SwitchBoard instead of the usual iOS firmware.

Let’s start with prototype games first. I have seen some prototype games come in cartridge shells with either handwritten or some typed label on the cartridge. That is easy to scan and preserve. However, I have also seen a fair number of prototype games that simply come on bare cartridges with no labels. More recently, the latest Sonic the Hedgehog releases from Hidden Palace don’t even come with a printed circuit board and instead only exist as bare EPROMs! How would such software be scanned and preserved then? Should we scan the bare printed circuit board or the bare EPROMs? Should we even bother?

In more modern terms, what about embedded devices such as PDAs that might be running prototype software? In this instance there is generally no removable ROM to speak of as such ROMs are usually soldered to the device’s main board and is not easily removable. What needs to be scanned in this situation? In like manner, what should we do in cases of an iPhone running SwitchBoard? They may be more up-to-date than a PDA but the same issue still arises – in any Apple device, the NAND flash is soldered to the device’s logic board and is not easy to remove. What do you scan and preserve?

The other important question to ask is, how do you copy a ROM chip? For cases like prototype games, a classic EPROM burner or modern solutions such as the Willem Programmer can easily dump ROM chips from prototype games, but are they providing a 1:1 copy of the data on the ROM chip? How do you read data from a PDA without specialized tools to read those ROM chips? In the case of iPhones, it is a little easier as the Chinese have invented devices to dump an iPhone’s NAND flash for the purpose of reflashing it to a larger capacity NAND flash chip. Can such a device be used to preserve the software on prototype Apple devices?

Ultimately for ROM software there is one way that is guaranteed to provide a 1:1 copy of the ROM’s contents, and that is IC decapsulation. In layman’s terms, the silicon wafer that makes up the ROM storage on a ROM chip can be drilled or chemically etched away to expose the bare wafer, at which point the ROM chip can be placed under a scanning electron microscope and read out, bit-by-bit. It is a laborious and time-consuming process (not to mention dangerous), but was successfully used to copy the original Nintendo Game Boy ROM. Should we go to such lengths to ensure that when dumping any ROM software, we have a perfect 1:1 copy of the software?

Conclusion
As you can see, there are many different ways in which software was (and sometimes still is) distributed to the general public. Much care must be taken to ensure that when preserving software, research is done to ensure that a perfect, 1:1 copy of the software can be obtained from its original medium. Admittedly, this is only a brief summary of the many ways in which software was distributed and questions that we must ask before attempting to copy any kind of software distribution medium as it is of utmost importance that software is preserved exactly how it was originally distributed so it can be enjoyed exactly as the author(s) intended upon its initial release. Please, share your thoughts and opinions below so that we can become a better repository for all software preservation for decades to come.
Jack Henry, Ph.D.
Member of Mensa International, the high IQ society.

mrpijey
User avatar
Administrator
Posts: 9188
Joined: Tue Feb 12, 2008 5:28 pm
Contact:

Re: Community Discussion: Preserving Obscure Software Distri

Post by mrpijey »

Thank you for the summary, very well written out.

Some additions and comments I want to make:

8" floppies can be read using the proper drive, a kryoflux and minor alterations to a cable. I have such setup with my "dump station" (A PC that currently has a 5.25"/3.5" combo drive (modified for flip disks) , a Plextor DVD-ROM drive (compatible with DIC), a NEC DVD-ROM drive (for golden discs the Plextor can't read), a QIC-80 tape drive, an external 8" floppy drive, a combo memory card reader, two modular drive bays for 3.5" and 2.5" drives), a USB adapter for IDE drives, SCSI card for two external tape readers, a proper audio card for recordings from a Walkman (when dumping C64 and Atari tape games) and a modified Amiga drive) and it works fine for dumping 8" floppies. I only have three 8" titles so I have not tested it a lot.

ROMs are something we will in time add to the FTP, I have considered distributing the TOSEC ROM set with BA but I have not yet decided to do so. TOSEC is a preservation group focusing on preserving software from known and unknown hardware, consoles and computers, as well as more modern ones. There are also other groups managing various system ROMs and their dumping (MAME etc), I may add these to the archive in the future as I try to gather these as well.

As you have indicated there's a lot of software still that can be gathered and preserved if only we had the people and interest of doing so. I've gathered a sizable collection myself but the task too daunting to do alone, so I fear will lose tons to history as people in general don't seem interested to collect and preserve these above their own selfish needs. But the few of us here that try to collect these we do the best we can, one more title properly preserved is one more title we don't need to worry about in the future.
Image
Official guidelines: Contribution Guidelines
Channels: Discord :: Twitter :: YouTube

Post Reply