BetaArchive Logo
Navigation Home Screenshots Image Uploader Server Info FTP Servers Wiki Forum RSS Feed Rules Please Donate
UP: 6d, 3h, 0m | CPU: 2% | MEM: 1875MB of 5099MB used
{The community for beta collectors}

Post new topic Reply to topic  [ 4 posts ] 
Author Message
 PostPost subject: Removing trailing zeroes from ISOs        Posted: Sun Jan 28, 2018 11:27 am 
Reply with quote
Administrator
User avatar
Offline

Joined
Tue Feb 12, 2008 5:28 pm

Posts
6942
This topic was created from this topic as it merits its own discussion and not part of the errata topic.

DOS wrote:
Something I've been thinking of for a while, now seems like an appropriate time to mention it: If at some point you create a list or database of all the files on the FTP site (not the compressed versions), I think it would be nice to not only provide the hash of the file, but also the hash of the file after removing all the trailing zero bytes!


mrpijey wrote:
To do that database thing I would need to find a way to trim the end zeroes from the files, and I just wonder if it's worth it. After all, the contents of the disc will be indexed as well...


DOS wrote:
I think it's useful because it's one hash to compare to verify that the discs are essentially identical, vs. a whole bunch of hashes to compare to verify that they have the same set of files which doesn't necessarily prove that it's the same disc, and comparing the single hash is a lot easier too.

I imagine it would be trivial to write a tool to do the trimming. I imagined writing something that I could use in a Linux pipeline:

trim_trailing_zeros < file.iso | sha1sum

so I'm not actually storing a second copy of the file on disk. I'd be happy to write something like this if it'd be useful.


It's a good idea, but are the trailing zeroes the ISOs original? Ie, was it added by Microsoft in some of their ISOs or did some disc dumpers just add them? The reason I ask is because if they are original (by Microsoft) they should be kept as-is, and hashed as-is. If they are not original then all discovered ISOs with trailing zeroes should be trimmed and repacked.

_________________
Image
Official guidelines: The Definitive Guide to BetaArchive :: Abandonware
Tools: Alcohol120% (Portable)
Listings: Online :: Offline


Top  Profile  WWW
 PostPost subject: Re: Removing trailing zeroes from ISOs        Posted: Sun Jan 28, 2018 2:52 pm 
Reply with quote
FTP Access
Offline

Joined
Tue Aug 07, 2012 3:08 pm

Posts
45

Favourite OS
OS/2 Warp 4
Most tools add padding zeros to a 2048 bytes multiple, some tools add padding zeros to a 2 seconds multiple, and when you make an iso from a recordable disc you usually get 2 seconds of zeros (aka 150 sectors).


Top  Profile
 PostPost subject: Re: Removing trailing zeroes from ISOs        Posted: Mon Jan 29, 2018 11:01 am 
Reply with quote
FTP Access
User avatar
Offline

Joined
Sun Mar 16, 2014 6:56 am

Posts
116

Favourite OS
DOS
mrpijey wrote:
This topic was created from this topic as it merits its own discussion and not part of the errata topic.

Thanks!
Quote:
It's a good idea, but are the trailing zeroes the ISOs original? Ie, was it added by Microsoft in some of their ISOs or did some disc dumpers just add them? The reason I ask is because if they are original (by Microsoft) they should be kept as-is, and hashed as-is. If they are not original then all discovered ISOs with trailing zeroes should be trimmed and repacked.


I had a look at .isos I've downloaded directly from Microsoft and they end in zeros. So I wouldn't say that the hash of the file with the zeros removed is the "correct hash" of the "actual image" or anything, it's just a tool for helping to detect images which aren't actually identical but are effectively identical.

Certainly if you gave me two .iso files, this technique would tell me that they're both effectively identical, but it wouldn't tell me which one was the original one from Microsoft. I don't know that there's any way to figure that out.

So I definitely wouldn't suggest that the "hash of the file with trailing zeros removed" be the only hash you record, just an extra one that is sometimes useful.

Thanks for the interesting information, claunia! I can certainly see that the size of the .iso image from MS I'm looking at is a multiple of 2KiB.

I wonder if various tools would complain if you were to trim all the zeros off the end of an .iso file? I certainly wouldn't try trimming the zeros off myself, it doesn't seem like a useful thing to do, particularly if it's only going to save me up to 2KB per file :)


Top  Profile
 PostPost subject: Re: Removing trailing zeroes from ISOs        Posted: Mon Jan 29, 2018 1:43 pm 
Reply with quote
Administrator
User avatar
Offline

Joined
Tue Feb 12, 2008 5:28 pm

Posts
6942
Well, in this case I am not sure what would be best. But it sure adds to my opposition of ISO as it doesn't seem to be any good way to make sure an ISO is complete or not.

If ISO managers in general works with trimmed zeroes then everything should be trimmed. But MS originals come with the zeroes, which means they would become less original if we trim it, thus going against what we are trying to accomplish....

In the end it seems better to just hash the ISO as it is, and also keep track of the contents. What we could do however is to has the header instead, if the header is identical between a trimmed and non-trimmed ISO then we can safely assume the contents should be the same as well...

_________________
Image
Official guidelines: The Definitive Guide to BetaArchive :: Abandonware
Tools: Alcohol120% (Portable)
Listings: Online :: Offline


Top  Profile  WWW
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 4 posts ] 




Who is online

Users browsing this forum: No registered users and 7 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to:  

All views expressed in these forums are those of the author and do not necessarily represent the views of the BetaArchive site owner.

Powered by phpBB® Forum Software © phpBB Group

Copyright © 2006-2018

 

Sitemap | XML | RSS