FTP requirements (on media formats and other things)

Problem with the site? Got a suggestion? Got feedback? Post here and the staff will discuss it with you.
Battler
User avatar
Donator
Posts: 2117
Joined: Sat Aug 19, 2006 8:13 am
Location: Slovenia, Central Europe.
Contact:

FTP requirements (on media formats and other things)

Post by Battler »

I have had a bit of a thought and a conversation with other members of this forum, and we have come to the conclusion that the FTP rules on media formats are too convoluted and often too strict, to the point of being detrimental to BetaArchive - there's more and more people who categorically refuse to upload their collections to our FTP because they don't like the requirements.

First, Overdoze posted something relevant in this thread, namely:
The more likely reason as to why these builds are rare nowadays is that they're, well... old. And the older something gets, the more likely it is someone will throw it away or it'll eventually deteriorate beyond recovery. Most software from the 80s is rare now because not many people still keep working 30+ year old installation disks (often in long obsolete formats too) around.
So why are original installer media an absolute requirement for software from the 70's, 80's and early 90's? Considering that we already forbid shareware and freeware, this already reduces the amount of software from the time period we'd accept considerably, so I see no need for additional constraints. Consider that a pre-installed copy might well be the only thing that's left of such an old commercial product. We already allow pre-installed copies of beta's, so why not extend this to abandonware up to, say, 1995?
Though of course, there is the risk of that causing an onslaught of bad uploads (fakes, hacks, and so on). Maybe splitting the queue into fast lane (for proper copies of stuff) and slow lane (for less proper copies where even identification might require more work) would be a good idea.

Second, why do CD (and DVD) images have to be MDF+MDS, even for unprotected media? We consider IMG good enough for floppies unless the floppies are such that storing them in such a format would cause loss of information, so why don't we apply the same standard to CD's? It would make it easier for people to upload their collections, and reduce the amount of disk space used by the FTP - not only because ISO's are smaller per se (though CUE+BIN are not) but also because uncompressed formats deduplicate better. Of course, copy-protected discs would still require MDF+MDS, but otherwise, the acceptable formats would depend on the type of disc, which by the way is a position supported by professional archivists.

My proposal would be to allow ISO for non-copy-protected data-only discs, and require MDF+MDS for everything else (maybe optionally allowing something like CUE+BIN for non-copy+protected discs with audio, but requiring MDF+MDS for those would not be that bad).

I understand the argument that there is already a lot of stuff to process and that this might increase the workload further, however, the current requirements actually get in the way of processing stuff. I have been downloading a considerable amount of stuff from the processing queue and for most of it, I'm not even sure what to do with it, since I'll honestly admit I'm not quite sure sure anymore what kind of stuff is accepted (other than the basic requirements) and what exceptions exist for what.
I believe that if we had simpler, easier to follow requirements, that also uniformly applied to everything, it would make processing things easier, and make more people willing to help.

I do realize that the current FTP requirements have arised out of a need to sort out the mess that the FTP once was and that they have been set up with the best intentions, but they have unfortunately grown to become detrimental, rather than beneficial, to BetaArchive, so I feel it's time a discussion is held in order to bring about FTP requirements that are beneficial.

Edit: To make it clear, the main purpose of this thread is to expose some of what I think are issues with the FTP requirements and, most importantly, to see what the rest of the members think.
Main developer of the 86Box emulator.
Join the 86Box Discord server, a nice community for true enthusiasts and 86Box supports!

The anime channel is on the Ring of Lightning Discord server.

Check out our SoftHistory Forum for quality discussion about older software.

claunia
Posts: 49
Joined: Tue Aug 07, 2012 3:08 pm

Re: FTP requirements (on media formats and other things)

Post by claunia »

There are some problems with Alcohol (not the format by itself but the software:
  • Whatever DVD-something you dump, Alcohol creates a fake 120mm DVD-ROM PFI. For example, some IBM POWER and System Z diagnostics and firmware updates come in DVD-RAM, where the LBA 0 corresponds to a different PSN than in -ROM and at the same time the PFI says it's a DVD-RAM version 2. Alcohol 120% created image indicates the disc is a DVD-ROM (it is not).
  • If you have a drive that only reads P and Q subchannels, Alcohol 120% image fills the other subchannels with zeroes. CD-G used those subchannels and you think you are getting a correct dump, when you are not.
  • This is a problem of the format: Interleaving the subchannel with the data not only makes it less compressible as Battler said but also means two dumps will NEVER be the same. That's because the subchannel is not error protected, so just reading the same disc twice in the same drive will give you a different image. Even if a copy protection uses subchannel (as psxcrypt does) it's just better to store it separate (like CloneCD does), something Alcohol does not support.

Battler
User avatar
Donator
Posts: 2117
Joined: Sat Aug 19, 2006 8:13 am
Location: Slovenia, Central Europe.
Contact:

Re: FTP requirements (on media formats and other things)

Post by Battler »

claunia wrote:This is a problem of the format: Interleaving the subchannel with the data not only makes it less compressible as Battler said but also means two dumps will NEVER be the same. That's because the subchannel is not error protected, so just reading the same disc twice in the same drive will give you a different image. Even if a copy protection uses subchannel (as psxcrypt does) it's just better to store it separate (like CloneCD does), something Alcohol does not support.
Thing is though, on a real disc, subchannel data comes after the data for the sector, so interleaving stores the closest it can to the data on the disc. Also storing subchannel data separately might render storing of data position measurement difficult or even impossible.
Though, you raised a valid concern with regards to Alcohol 120% just filling the subchannel data with zeroes if it can't read it from the drive, which raises the question how can we be sure of how proper the MDF+MDS dumps people upload even are?
Main developer of the 86Box emulator.
Join the 86Box Discord server, a nice community for true enthusiasts and 86Box supports!

The anime channel is on the Ring of Lightning Discord server.

Check out our SoftHistory Forum for quality discussion about older software.

TheCollector1988
User avatar
Donator
Posts: 3604
Joined: Wed Feb 23, 2011 12:11 am
Location: Italy
Contact:

Re: FTP requirements (on media formats and other things)

Post by TheCollector1988 »

I admit that using Alcohol 120% or Daemon Tools to dump, for example, an original CD with no protection and with barely any data to MDF/MDS is really overkill.

claunia
Posts: 49
Joined: Tue Aug 07, 2012 3:08 pm

Re: FTP requirements (on media formats and other things)

Post by claunia »

Battler wrote:
claunia wrote:This is a problem of the format: Interleaving the subchannel with the data not only makes it less compressible as Battler said but also means two dumps will NEVER be the same. That's because the subchannel is not error protected, so just reading the same disc twice in the same drive will give you a different image. Even if a copy protection uses subchannel (as psxcrypt does) it's just better to store it separate (like CloneCD does), something Alcohol does not support.
Thing is though, on a real disc, subchannel data comes after the data for the sector, so interleaving stores the closest it can to the data on the disc. Also storing subchannel data separately might render storing of data position measurement difficult or even impossible.
Though, you raised a valid concern with regards to Alcohol 120% just filling the subchannel data with zeroes if it can't read it from the drive, which raises the question how can we be sure of how proper the MDF+MDS dumps people upload even are?
DPM is stored in the MDS file separate and independent of the subchannel. Also when you make a "virtual drive" it doesnt matter that the subchannel is in another file, your underlying storage is several orders of magnitude faster than what you will emulate with the DPM data.

SoftPCMuseum
User avatar
Posts: 140
Joined: Sun Feb 01, 2015 2:15 am
Contact:

Re: FTP requirements (on media formats and other things)

Post by SoftPCMuseum »

There certainly are uses for sub-channel data though, even for non-copy-protected CD-ROMs and DVD-ROMs. Even if a CD-ROM or DVD-ROM did not appear to be copy-protected, there are cases where the developers still used the sub-channel data for other purposes (for example, Microsoft embedding a hidden copy of Microsoft BOB into the sub-channel data of Microsoft Windows XP CD-ROMs according to Raymond Chen, with the intention of making it easier for skilled users to tell genuine CD-ROMs apart from counterfeit CD-ROMs).

Also, the sub-channel data might still theoretically differ based upon the type of CD mastering used by the various developers for each product.
Image
Latest release of Virtual Computer emulator available here:
https://www.betaarchive.com/forum/viewt ... 72&t=39197

TheCollector1988
User avatar
Donator
Posts: 3604
Joined: Wed Feb 23, 2011 12:11 am
Location: Italy
Contact:

Re: FTP requirements (on media formats and other things)

Post by TheCollector1988 »

Also, for floppy disks with protection, well, not everyone has a SuperCard Pro or a Kryoflux or not all people wants to send their floppies to another person outside their basement or home, as a result of not owning a Kryoflux unit, because they might not trust that way,

Overdoze
User avatar
Posts: 1762
Joined: Mon Feb 24, 2014 10:28 am
Location: Slovenia

Re: FTP requirements (on media formats and other things)

Post by Overdoze »

I generally agree, especially about original media for old abandonware. There's several rare pieces of history that are only available in pre-installed formed and thus have to be kept off-site, by members. Battler's idea about preventing a flood of low quality/garbage uploads seems reasonable, perhaps limit this to 1980s only for now?
All roads lead to Neptune™

KRNL386 - my site about retro computing | My site about Windows 1.0 | My blog | 86Box Manager | LeakDB - list of PC OS warez leaks

Battler
User avatar
Donator
Posts: 2117
Joined: Sat Aug 19, 2006 8:13 am
Location: Slovenia, Central Europe.
Contact:

Re: FTP requirements (on media formats and other things)

Post by Battler »

SoftPCMusuem wrote:There certainly are uses for sub-channel data though, even for non-copy-protected CD-ROMs and DVD-ROMs. Even if a CD-ROM or DVD-ROM did not appear to be copy-protected, there are cases where the developers still used the sub-channel data for other purposes (for example, Microsoft embedding a hidden copy of Microsoft BOB into the sub-channel data of Microsoft Windows XP CD-ROMs according to Raymond Chen, with the intention of making it easier for skilled users to tell genuine CD-ROMs apart from counterfeit CD-ROMs).
But according to claunia, there's no drive out there that returns all the correct sub-channel data, so we're still preserving incomplete data. Though, I do acknowledge your point, it does mean that subchannel data might be needed for some non-copy-protected discs.
Overdoze wrote:I generally agree, especially about original media for old abandonware. There's several rare pieces of history that are only available in pre-installed formed and thus have to be kept off-site, by members. Battler's idea about preventing a flood of low quality/garbage uploads seems reasonable, perhaps limit this to 1980s only for now?
That would do for now, better than just letting the software disappear.
Main developer of the 86Box emulator.
Join the 86Box Discord server, a nice community for true enthusiasts and 86Box supports!

The anime channel is on the Ring of Lightning Discord server.

Check out our SoftHistory Forum for quality discussion about older software.

TheCollector1988
User avatar
Donator
Posts: 3604
Joined: Wed Feb 23, 2011 12:11 am
Location: Italy
Contact:

Re: FTP requirements (on media formats and other things)

Post by TheCollector1988 »

Overdoze wrote:I generally agree, especially about original media for old abandonware. There's several rare pieces of history that are only available in pre-installed formed and thus have to be kept off-site, by members. Battler's idea about preventing a flood of low quality/garbage uploads seems reasonable, perhaps limit this to 1980s only for now?
Microsoft Windows 1.03 for Apricot XEN and NEC PC-98x1 come to mind. 1.02 English for PC-D too (not the German one).

Overdoze
User avatar
Posts: 1762
Joined: Mon Feb 24, 2014 10:28 am
Location: Slovenia

Re: FTP requirements (on media formats and other things)

Post by Overdoze »

Yeah, those were the ones I had in mind, though there are probably more examples too.
All roads lead to Neptune™

KRNL386 - my site about retro computing | My site about Windows 1.0 | My blog | 86Box Manager | LeakDB - list of PC OS warez leaks

claunia
Posts: 49
Joined: Tue Aug 07, 2012 3:08 pm

Re: FTP requirements (on media formats and other things)

Post by claunia »

SoftPCMuseum wrote:There certainly are uses for sub-channel data though, even for non-copy-protected CD-ROMs and DVD-ROMs. Even if a CD-ROM or DVD-ROM did not appear to be copy-protected, there are cases where the developers still used the sub-channel data for other purposes (for example, Microsoft embedding a hidden copy of Microsoft BOB into the sub-channel data of Microsoft Windows XP CD-ROMs according to Raymond Chen, with the intention of making it easier for skilled users to tell genuine CD-ROMs apart from counterfeit CD-ROMs).

Also, the sub-channel data might still theoretically differ based upon the type of CD mastering used by the various developers for each product.
So you're referring to this post:
https://technet.microsoft.com/en-us/lib ... ntial.aspx

Read it correctly:
there was still about 30 megabytes of storage capacity remaining.
There is no Windows XP disc that filled a whole CD (74 or 80min) and even then there're not 30 megabytes of subchannel space in a full 80min disc.
Somebody decided to fill that extra capacity on the CD with dummy data and to have the Windows Setup program verify that the dummy data was still there.
Meaning, if it were true the data was stored in the subchannel not a single .iso of Windows XP would work. They do.

Also, you can repackage Windows XP with drivers and unattended installation information, without modifying SETUP.EXE. It wouldn't work either. Neither would just copying I386 folder, and this as demonstrated by hundreds of hacked images, work, with unmodified SETUP.EXE files.

And finally, just dumped the subchannel of a Windows XP original disc, and besides the usual reading error, the R-W subchannels are fully empty.
Also, end of ISO, after all files are stored, is empty, there's no encrypted data there.

P.S.: I may be wrong here, but afaik, Microsoft CDIMAGE doesn't allow to add dummy data to fill an ISO.
P.S.2: With reading error I mean spurious bits in the subchannels. Subchannel is not error corrected at all or contains any kind of error detection except for CRC16 on the Q subchannel and ReedSolomon ONLY for CD-G/EG/MIDI discs, so whatever you read, you don't know if the bit is correctly set or not.

mrpijey
User avatar
Administrator
Posts: 9193
Joined: Tue Feb 12, 2008 5:28 pm
Contact:

Re: FTP requirements (on media formats and other things)

Post by mrpijey »

I understand all your concerns, but there's two things I can come up with that encompasses everything...

1: KISS
2: Save as much as possible of the original media

Keeping multiple formats (as we already do and try to get away from) is a major hassle, especially for the absolute majority of users that couldn't care less about subchannel data, multisession tracks, file systems etc. and all they know is "ooh, a shiny disc and a tray to put it in = profit". These people would by no means be qualified to judge which format is the best. So instead of having a very lengthy explanation of what every format does and what apps saves it best we simply say "use A120%. Download, unzip, run the exe and hit it. It saves three files, pack it up and upload it."

Am I saying A120% is perfect? No, not by a long shot. But it's the best for what we need it for as it handles everything except for the custom console stuff, but for that there is no other solutions either.

And as for the "some original stuff comes preinstalled"... well, if it's original and preinstalled then it's no issue as we preserve it like that. But we won't de-evolve to any common abandonware site where we have unverified and unsourced material, quite the opposite. The ideal archive would be a 100% archive where everything has a proper verified source and in a proper format that saves all data possible.

In the end, this is a proper PRESERVATION site and we try to preserve it in the best (technically, not conveniently) format possible. The end user can then always downgrade the format to his or hers personal preferences, it being ISO, zip, tar or whatever. But we keep it as unified as possible, as original as possible with as much preserved data as possible. That may be inconvenient from the daily use, but it saves the most data. We've tried to make it easy by providing a standalone non-installable version of A120 that runs immediately and still dumps everything properly, just to save time without the need to install the software by installers. Easy unzip, easy delete when you're done. So yes, MS discs may not have metadata. But some might. Some even do have audio tracks where they don't need to be any (some MSDN discs come to mind). To make sure everything is saved regardless we use a format that works with everything with minimal user interaction.
Image
Official guidelines: Contribution Guidelines
Channels: Discord :: Twitter :: YouTube
Misc: Archived UUP

claunia
Posts: 49
Joined: Tue Aug 07, 2012 3:08 pm

Re: FTP requirements (on media formats and other things)

Post by claunia »

mrpijey wrote:saves it best we simply say "use A120%. Download, unzip, run the exe and hit it. It saves three files, pack it up and upload it."
Use DiscImageChef, it creates Alcohol format, it is free, works on non-Windows, and doesn't point you to a chinese clickbait domain holder (like Alcohol download does).

Need to push a new beta binary so people can test the feature, but just so you know, it's already there (and stores things that Alcohol doesn't or blalantly modifies).

mrpijey
User avatar
Administrator
Posts: 9193
Joined: Tue Feb 12, 2008 5:28 pm
Contact:

Re: FTP requirements (on media formats and other things)

Post by mrpijey »

Again, we have a portable version that doesn't send you anywhere.

I have to test DiscImageChef first and see if it handles all kinds of discs properly.
Image
Official guidelines: Contribution Guidelines
Channels: Discord :: Twitter :: YouTube
Misc: Archived UUP

3155ffGd
User avatar
Posts: 391
Joined: Wed May 02, 2012 12:57 am

Re: FTP requirements (on media formats and other things)

Post by 3155ffGd »

Battler wrote:Second, why do CD (and DVD) images have to be MDF+MDS, even for unprotected media? We consider IMG good enough for floppies unless the floppies are such that storing them in such a format would cause loss of information, so why don't we apply the same standard to CD's?
Because ISO can't store mode 2 (mixed data/audio) CDs, which were immensely popular in the 90's especially for games, but also for some applications. At some point we had a release on the FTP that was broken precisely because someone dumped it as ISO causing the audio tracks to get lost.

edit: BIN/CUE, which you suggested, can't do it either, at least the format for that is not standardized in any way and causes massive interoperatibility problems in my experience.

Overdoze
User avatar
Posts: 1762
Joined: Mon Feb 24, 2014 10:28 am
Location: Slovenia

Re: FTP requirements (on media formats and other things)

Post by Overdoze »

I don't think you gain anything with MDF+MDS over ISO for regular discs though. Or for the stuff which was already provided in ISO form by the developer/publisher/manufacturer.
All roads lead to Neptune™

KRNL386 - my site about retro computing | My site about Windows 1.0 | My blog | 86Box Manager | LeakDB - list of PC OS warez leaks

mrpijey
User avatar
Administrator
Posts: 9193
Joined: Tue Feb 12, 2008 5:28 pm
Contact:

Re: FTP requirements (on media formats and other things)

Post by mrpijey »

You lose nothing by using MDF format either, and it makes it a lot easier for everyone. Most members wouldn't know what tool to use and what format would be best for their particular release, so we make it easier for them by choosing one that works with all discs.

KISS.

As I said, our goal isn't to make it convenient for everyone, it's to provide the best copy for you.

We've had these arguments already in other threads on this forum, and nothing has changed. We don't want a multitude of different formats, and we don't need to write a book for the members explaining exactly which tool to use for which title and why. We give them one tool that does it all.

And yes yes, linux and Mac users blah blah. We can't cater to all needs, and if Linux can't handle anything beyond ISO then make a tool that can. MacOS users can at least mount MDFs using Daemon Tools. Or you can just virtualise Windows for those needs where you need to dump or convert something. Either way we accept DMGs from Macs where absolutely needed (at least DMG provides multisession support and built in hashes, something ISO doesn't), and both Mac and Linux users are an absolute minority of all BA members.

Find me a better universal format that handles everything on all platforms and I'll seriously consider it. ISO is not it, and I am not going to degrade the copies we have because using MDF images is not very hard. Nor am I going to confuse the members by allowing different formats because then everyone that contributes will need to understand multisession, copy protections, redbook and all those special quirks that comes with discs from various publishers, platforms and whatnot. One ring format to rule them all. There's a reason why I chose MDF above CCD, BIN/CUE etc. and no one has provided me any evidence that there is a more superior universal format that is better. Heck, if we had a software "kryoflux" for optical media we would have required that too.

Why don't you people understand it? We preserve software here in the best quality possible, and let you do whatever you want with it afterwards - convert it to ISO, print the ISO to trayfed paper and build a house of it, convert the 600/1200DPI artwork as a giant wallpaper and plaster your local city hall with it, I don't care. But we provide the data the best way we can.

If ISOs are so important to you then setup your own script to convert it. PowerISO, Daemon Tools (both Win and OSX) and A120%/A52% can handle MDFs just fine. Possibly more tools too. DiscImageChef might do it too, but I need more testing to see that it handles everything A120 does.

KISS and quality. Keep it simple and don't confuse members with 19 different formats when they don't even know the technical difference between a CD-ROM and DVD-ROM...

Try to understand....

Addition: Overdoze, can you provide us with a complete list of all software on all platforms with notes on which tool and format to use for each title? Makes it easier for members then to know what tool and image format to use to make a proper and complete copy :).
Image
Official guidelines: Contribution Guidelines
Channels: Discord :: Twitter :: YouTube
Misc: Archived UUP

Darkstar
User avatar
Donator
Posts: 1212
Joined: Fri May 14, 2010 1:29 pm
Location: Southern Germany

Re: FTP requirements (on media formats and other things)

Post by Darkstar »

Just my 2 cents:
  • If there is a tool that does a better job at generating MDF+MDS, in all use cases (audio tracks, multisession, mode2/xa, etc.) then by all means I'm all for using it
  • Accepting non-working IMG files for floppy disks with protection just because people are too lazy to get them properly dumped is not a good idea.
    There are probably enough cracked dumps of your super rare DOS game or whatever out there, so it adds nothing of value
  • And as for Linux / MacOS X users: If their OS of choice doesn't support a file format, well then, bad for them, this is not our problem. This is basically the same argument that the Linux community has been using against Windows for the last 20 years or so... ;-)
Dumping CDs/DVDs/Floppies is tricky. Even more tricky is being able to tell if a dump is proper or not, i.e. if it works (because all copy protection data was included) or not. Who is supposed to do all the testing of that dump? mrpijey? probably not happening. Suppose you dump a game that is copy-protected, how would you tell if it works from the dump you created or not? Install it? Play it? For how long? Maybe the missing copy protection only manifests itself after X hours of gameplay? How on earth should anyone be able to judge if the dump is good or not. Alcohol120% was at least made to dump copy-protected discs so there is a reasonably high chance that a dump created by it will work. Switch to another program, for example one that generates ISO from unprotected discs and MDF+MDS from protected discs automatically, how will anyone ever find out if the program judged correctly?

So better be on the safe side and always require MDF+MDS, because it can be converted to ISO without any loss of information for those who need/want an ISO (the reverse is not true). Yes, the exotic media might be a problem but there was no alternative to it when the rule for FTP contributions was written. If there's a good alternative now (DiscImageChef for example), sure, why not use that to generate MDF+MDS...
I upload stuff to archive.org from time to time. See here for everything that doesn't fit BA

Overdoze
User avatar
Posts: 1762
Joined: Mon Feb 24, 2014 10:28 am
Location: Slovenia

Re: FTP requirements (on media formats and other things)

Post by Overdoze »

mrpijey wrote:You lose nothing by using MDF format either, and it makes it a lot easier for everyone. Most members wouldn't know what tool to use and what format would be best for their particular release, so we make it easier for them by choosing one that works with all discs.

As I said, our goal isn't to make it convenient for everyone, it's to provide the best copy for you.

We've had these arguments already in other threads on this forum, and nothing has changed. We don't want a multitude of different formats, and we don't need to write a book for the members explaining exactly which tool to use for which title and why. We give them one tool that does it all.
We were only talking ISO and MDF, not "a multitude of different formats", so you're building an argument on something that isn't even part of the discussion in the first place. We're talking two formats, not 10. Furthermore, if a tool can handle MDF, there's a 99.9% chance it can also handle ISO (considering how widespread the latter is). It doesn't work vice-versa, though. You don't even need a separate tool for ISOs, so as you said yourself, "We give them one tool that does it all.".
mrpijey wrote:And yes yes, linux and Mac users blah blah. We can't cater to all needs, and if Linux can't handle anything beyond ISO then make a tool that can. MacOS users can at least mount MDFs using Daemon Tools. Or you can just virtualise Windows for those needs where you need to dump or convert something. Either way we accept DMGs from Macs where absolutely needed (at least DMG provides multisession support and built in hashes, something ISO doesn't), and both Mac and Linux users are an absolute minority of all BA members.
Well, fair point.
mrpijey wrote:Find me a better universal format that handles everything on all platforms and I'll seriously consider it. ISO is not it, and I am not going to degrade the copies we have because using MDF images is not very hard. Nor am I going to confuse the members by allowing different formats because then everyone that contributes will need to understand multisession, copy protections, redbook and all those special quirks that comes with discs from various publishers, platforms and whatnot. One ring format to rule them all. There's a reason why I chose MDF above CCD, BIN/CUE etc. and no one has provided me any evidence that there is a more superior universal format that is better. Heck, if we had a software "kryoflux" for optical media we would have required that too.
Same as above, no one said ISO is superior to MDF when it comes to preservation - it is, however, superior when it comes to the number of tools that support it and it's also an open, well-documented format, which MDF isn't. We also suggested using ISO only where MDF wouldn't make a difference, for reasons I just mentioned. No one advocated for immediate conversion of all MDF dumps to ISO.

I do agree on the part about most users not knowing all the technical details of optical media and thus being unable to chose the proper format of the two proposed without seeking advice beforehand.
mrpijey wrote:Why don't you people understand it? We preserve software here in the best quality possible, and let you do whatever you want with it afterwards - convert it to ISO, print the ISO to trayfed paper and build a house of it, convert the 600/1200DPI artwork as a giant wallpaper and plaster your local city hall with it, I don't care. But we provide the data the best way we can.

If ISOs are so important to you then setup your own script to convert it. PowerISO, Daemon Tools (both Win and OSX) and A120%/A52% can handle MDFs just fine. Possibly more tools too. DiscImageChef might do it too, but I need more testing to see that it handles everything A120 does.
Fair enough.
mrpijey wrote:KISS and quality. Keep it simple and don't confuse members with 19 different formats when they don't even know the technical difference between a CD-ROM and DVD-ROM...

Try to understand....

Addition: Overdoze, can you provide us with a complete list of all software on all platforms with notes on which tool and format to use for each title? Makes it easier for members then to know what tool and image format to use to make a proper and complete copy :).
See my first point.

With that said, as you can see I pretty much yield in this discussion since you did bring up some valid points, but I do feel the need to address the parts where you made an issue out of nothing.
All roads lead to Neptune™

KRNL386 - my site about retro computing | My site about Windows 1.0 | My blog | 86Box Manager | LeakDB - list of PC OS warez leaks

claunia
Posts: 49
Joined: Tue Aug 07, 2012 3:08 pm

Re: FTP requirements (on media formats and other things)

Post by claunia »

mrpijey wrote:Again, we have a portable version that doesn't send you anywhere.

I have to test DiscImageChef first and see if it handles all kinds of discs properly.
And if it doesn't, just tell me and I'll correct the situation. Thing is, I can only test myself what I have myself :p

mrpijey
User avatar
Administrator
Posts: 9193
Joined: Tue Feb 12, 2008 5:28 pm
Contact:

Re: FTP requirements (on media formats and other things)

Post by mrpijey »

@Overdoze: Understood, but the fact still remains that we would then need to keep track of two formats and make sure the right format is used every time. That's just too much of a hassle and risk, and we are trying to reduce that. And in the end, converting an MDF to ISO is a piece of cake and can be done in a multitude of ways.

And with the "We give them one tool that does it all" meaning the dumping was implied. How you use the dump after it was downloaded from us doesn't matter to us. There's a much larger risk that we provide you with an incomplete or broken ISO than an incomplete or broken MDF because the MDF format will have everything, and recently even a log, whereas the ISO may lack a lot of stuff due to an unknowing user that only sees a file extension and nothing else. I am not taking that risk. It takes the same time for the dumper to make an MDF as it takes to make an ISO, but with a much higher success it being a proper dump, with logs and without accidently missing out on any extra tracks or hidden info. And it saves the downloader the time too knowing the dump is proper and complete without realising that the ISO was incomplete (it has happened).

So in the end: We provide you with the best copy possible, and you do what you want with it. And not "sorry that it doesn't work/has tracks missing/copy protection is broken/is incomplete because the dumper didn't know it had secondary tracks/had copy protection/whatever". Now we just provide the MDF, says it's all there (preferrably with a log showing any possible bad sectors or other info) and you can down convert it in any way you want. And in the end I want both you (the users and the dumpers) to spend as little time needed to dump the media correctly, and me as little time as needed chasing you down for doing an improper dump :). That's why you will get a PM from me saying "read the dumping guide, we need MDF/MDS as ISOs are not accepted" :).

Until there's a better alternative A120% stays, and I am not going to mess around with other formats unless absolutely necessary. We already have like 4 different CD formats on the FTP (MDF, CCD, ISO, DMG) and I want to reduce that as much as possible. MDF can do everything the other formats can do, but none of the other formats can do everything MDF can.

Prove me wrong, please :).

If we are going to use multiple formats (ISO, MDF or any other advanced format) then we need a single piece of software that can without a doubt handle both, provide extensive logs of the disc layout and prove to any FTP member that the lesser format is absolutely 100% as good with that particular release. That piece of software doesn't exist so we go with the safe bet, which is Alcohol 120%. It's fast, exists in a free version, handles copy protection very well, has logging options, can be run as a portable version and has tons of options and features.

I didn't choose A120% randomly and if we are going to settle for an official second format then I need to be 200% sure it will preserve everything (i.e more than A120% does now), because in most cases we don't get second chances with the dumps. And we still got loads of junk from BA FTP v1 in ISO format that we have no chance of even verifying or possibly not even get redumps of because no one cared about a proper format or logging the dump. And I would be the happiest person on BetaArchive if we found/made a truly open and universal format for optical discs that also stores all the metadata and everything else.

This is like the 10th time i say all this and the third time we have a thread about MDF vs and I am tired of repeating it. Please read the other topics first before making any more comments please as I think everything has been said already.

@claunia: Well, look at A120% in terms of features. Also I understand your software doesn't do DPM yet? That's an absolute must for copy protected titles.
Image
Official guidelines: Contribution Guidelines
Channels: Discord :: Twitter :: YouTube
Misc: Archived UUP

claunia
Posts: 49
Joined: Tue Aug 07, 2012 3:08 pm

Re: FTP requirements (on media formats and other things)

Post by claunia »

mrpijey wrote:@claunia: Well, look at A120% in terms of features. Also I understand your software doesn't do DPM yet? That's an absolute must for copy protected titles.
It doesn't do, yet, but I still need tests for non copy protected titles before I dig deeper in their format.

claunia
Posts: 49
Joined: Tue Aug 07, 2012 3:08 pm

Re: FTP requirements (on media formats and other things)

Post by claunia »

claunia wrote:
mrpijey wrote:@claunia: Well, look at A120% in terms of features. Also I understand your software doesn't do DPM yet? That's an absolute must for copy protected titles.
It doesn't do, yet, but I still need tests for non copy protected titles before I dig deeper in their format.
mrpijey wrote: If we are going to use multiple formats (ISO, MDF or any other advanced format) then we need a single piece of software that can without a doubt handle both, provide extensive logs of the disc layout and prove to any FTP member that the lesser format is absolutely 100% as good with that particular release.
In a nutshell, MDF/MDS is like BIN/CUE, in the sense that you can have a ISO/MDS or a BIN/MDS, Alcohol format is only the MDS the data file is standard ISO for DVDs onward and BIN for CDs. (except it's like cdrdao's bin than cdrwin's, because of subchannel).

soulman
User avatar
Donator
Posts: 2239
Joined: Tue Dec 15, 2009 8:56 pm
Contact:

Re: FTP requirements (on media formats and other things)

Post by soulman »

It's an unnecessary hassle for people who want to straight up just mount an image into an emulator I imagine, which is probably what most of your audience do.
Image

Post Reply