BetaArchive Logo
Navigation Home Database Screenshots Gallery Image Uploader Server Info FTP Servers Wiki Forum RSS Feed Rules Please Donate
UP: 65d, 9h, 29m | CPU: 21% | MEM: 5208MB of 12231MB used
{The community for beta collectors}

Post new topic This topic is locked, you cannot edit posts or make further replies.  [ 17 posts ] 
Author Message
 PostPost subject: Delta patches on the FTP        Posted: Thu Jun 27, 2013 11:00 am 
Administrator
User avatar
Offline

Joined
Tue Feb 12, 2008 5:28 pm

Posts
7993
Since space is always at a premium I've decided to save some space by using delta compression on the files on the FTP (see this thread for a lengthy discussion about the matter). I've gone through several ideas on how to implement this in the most efficient way and there's been several good ideas but since we're limited by a fixed folder system (on the FTP) I've decided the following (examples will follow):

  • Any release that comes in multiple languages will be delta compressed (when possible)
  • The US English release will always be the source file, to restore any other you will need the US English ISO file. The reason for this is that the US English file is the most common for members to download so it should always be unmodified.
  • Each delta patch release will be released as an individual RAR file on the FTP and will look like a regular unmodified release, except that it will contain a .iso.xd3 file instead of .iso. The archive filename will always be the same as the US English source (except for the additional "[Language]" suffix. This way you will also be able to quickly see the whole "set" of files, source and patches.
  • Each delta patch release will include a tool and batch file to simplify the restoration of the original file.

All delta patch files end with the extension ".iso.xd3". Put these (along with the tool and batch file) with your source file (the US English .ISO) and run the batch file and it will restore the patch file to its original size. The restored file will always have the same checksum etc as the original untouched one, so no modifications are made to the ISO when delta compressed.

Here are some examples (with the differences underlined):

Original BA archive: Microsoft Windows 7 (''Windows 7'' 6.1.7100.0) (Ultimate rc).rar
Delta patched archive: Microsoft Windows 7 (''Windows 7'' 6.1.7100.0) (Ultimate rc) [Danish].rar

Original ISO: 7100.0.090421-1700_x86fre_client_en-us_Retail_Ultimate-GRC1CULFRER_EN_DVD.iso
Delta patch: 7100.0.090421-1700_x86fre_client_da-da_Retail_Ultimate-GRC1CULFRER_DA_DVD.iso.xd3

In my example files above I managed to shrink a 2.6GB file down to ~350-400MB. If I apply this on any larger international abandonware/beta set we can make some significant savings on the FTP. Of course the results may vary, with some Windows 2000 releases I managed to reduce the file to a mere 90% of the original source file, and with some Windows XP (future release) I managed to reduce the file to a mere ~1-2% of the original source file.

With the delta patch archives I will include two files, xdelta3.exe and unpack.bat. xdelta3.exe is the packer/unpacker tool, unpack.bat is the batch file that will restore the file automatically for you. You can delete these two once you've restored your files. You can put several .xd3 files in the same folder and the batch file will restore all of them for you.

I will start implementing these changes soon and it will be done over an extended period of time (need to determine best files to patch and then download/upload/backup has to be done of the new archives). Not all languages can be successfully patched, for example Simpl. Chinese, Trad. Chinese, Korean and Japanese ISOs doesn't always delta compress very well with US English ISOs due to the rather different language structure coded into the software. So these will most likely be untouched on the server.

_________________
Image
Official guidelines: The Definitive Guide to BetaArchive :: Abandonware
Tools: Alcohol120% (Portable) :: DiscImageCreator
Listings: BetaArchive Database (beta)
Channels: Discord :: Twitter


Top  Profile  WWW
 PostPost subject: Re: Delta patches on the FTP        Posted: Thu Jun 27, 2013 5:50 pm 
FTP Access
Offline

Joined
Sun Sep 23, 2012 1:09 pm

Posts
28

Favourite OS
System 7
What about other Operating Systems? The appropriate tool should be included for Mac and Linux too. A shell script will be compatible with both OS.

It's a very good idea to save space but more bandwidth will be consumed, but if bandwidth it's not a problem then it's nice.


Top  Profile
 PostPost subject: Re: Delta patches on the FTP        Posted: Thu Jun 27, 2013 6:46 pm 
Administrator
User avatar
Offline

Joined
Tue Feb 12, 2008 5:28 pm

Posts
7993
I didn't consider Linux and Mac users since the majority of members sits on Windows. But I will see if I can compile a tool set and put it among the tools we got on BA.

_________________
Image
Official guidelines: The Definitive Guide to BetaArchive :: Abandonware
Tools: Alcohol120% (Portable) :: DiscImageCreator
Listings: BetaArchive Database (beta)
Channels: Discord :: Twitter


Top  Profile  WWW
 PostPost subject: Re: Delta patches on the FTP        Posted: Thu Jun 27, 2013 7:54 pm 
FTP Access
Offline

Joined
Sun Sep 23, 2012 1:09 pm

Posts
28

Favourite OS
System 7
I just build an Universal Binary (i386, x86_64 & ppc) of xdelta3 3.0.6 for Mac OS X 10.5 or greater.
For your convenience I uploaded it to the FTP.


Top  Profile
 PostPost subject: Re: Delta patches on the FTP        Posted: Sat Jun 29, 2013 6:28 am 
FTP Access
User avatar
Offline

Joined
Fri Apr 06, 2012 2:04 am

Posts
468

Favourite OS
Windows 8 Build 7898
SHOULD I download US English ISO to be able to extract any files?


Top  Profile
 PostPost subject: Re: Delta patches on the FTP        Posted: Sat Jun 29, 2013 3:05 pm 
Administrator
User avatar
Offline

Joined
Tue Feb 12, 2008 5:28 pm

Posts
7993
Yes, the US English is always the source file for any delta patches in the same series. If all you need is just the english version then you need only that, but if you want, say the german version, then you need to download both the English and German archives. It's a bit more to download for those that prefer international releases, but most prefer the US english anyway so I don't think it will be much of a problem anyway.

_________________
Image
Official guidelines: The Definitive Guide to BetaArchive :: Abandonware
Tools: Alcohol120% (Portable) :: DiscImageCreator
Listings: BetaArchive Database (beta)
Channels: Discord :: Twitter


Top  Profile  WWW
 PostPost subject: Re: Delta patches on the FTP        Posted: Sat Jun 29, 2013 8:13 pm 
Donator
Offline

Joined
Tue Jun 16, 2009 11:40 pm

Posts
80

Favourite OS
whistler
what about refs and its dedup feature? wouldnt that be more user friendly ?


Top  Profile
 PostPost subject: Re: Delta patches on the FTP        Posted: Sun Jun 30, 2013 1:39 am 
Administrator
User avatar
Offline

Joined
Tue Feb 12, 2008 5:28 pm

Posts
7993
With dedup you will need all the files unpacked which would not only increase the size of the archive but also increase the download time for all members and eliminate all the safeguards we have against data corruption. Doesn't matter which "dedup" technology you use, they would all actually decrease the space available. Dedup is good for many things, but not RAR files.

_________________
Image
Official guidelines: The Definitive Guide to BetaArchive :: Abandonware
Tools: Alcohol120% (Portable) :: DiscImageCreator
Listings: BetaArchive Database (beta)
Channels: Discord :: Twitter


Top  Profile  WWW
 PostPost subject: Re: Delta patches on the FTP        Posted: Sat Jul 06, 2013 10:12 am 
Administrator
User avatar
Offline

Joined
Tue Feb 12, 2008 5:28 pm

Posts
7993
The initial delta compression optimization has been completed. I did check the FTP size before I started, and by then we had just passed the 5TB marker with 5025GB, and that was before I added Windows 8.1 Preview, Server 2012 R2, Hyper-V 2012 R2 and SQL Server 2014. Now we've dipped slighly below 5TB with 4913GB so I would approximate that we've saved about 150-200GB so far. It may not sound a lot but every GB counts and it's 200GB we can better use for new releases. Where originally a new Windows 8.x or Server 2012 R2 release would take 100-150GB we can now reduce it to about 20GB or so. It's a lot easier to delta compress the newer beta releases than the older ones like Windows Vista, XP, Server 2003 etc so the savings are not extreme yet, but with the upcoming Windows XP abandonwares etc the savings will be more prominent. I am looking into some way to calculate the uncompressed archive as well as compressed so we can make an easy compare, but the feature isn't vital nor necessary for the time being, it would only be a fun fact for the site.

But my work with delta compressing releases has not yet finished and I will try to squeeze out every GB here and there when I can, but with newer Microsoft releases it will be used quite a lot since they pack up very well, one good example is the recent Windows Server 2012 R2 Essentials where a 3.8GB ISO is reduced to between ~140MB-300MB depending on language. That's basically a 3.5GB+ reduction per ISO!

But work continues and once a working system for comparing delta/non-delta releases is in place the real result of the delta compression will be displayed for everyone to see.

_________________
Image
Official guidelines: The Definitive Guide to BetaArchive :: Abandonware
Tools: Alcohol120% (Portable) :: DiscImageCreator
Listings: BetaArchive Database (beta)
Channels: Discord :: Twitter


Top  Profile  WWW
 PostPost subject: Re: Delta patches on the FTP        Posted: Mon Jul 15, 2013 11:33 pm 
Administrator
User avatar
Offline

Joined
Tue Feb 12, 2008 5:28 pm

Posts
7993
Well, to satisfy those of you that mentioned deduplication etc, with the help from dw5304 I've run a dedup bench test of the (Abandonware) Operating Systems folder on the BA FTP to see if we can save anything by running deduplication of our rar-files. Knowing how deduplication work (which is essentially the same as what I've done with xdelta but on a bit level) I knew it wouldn't yield good results, the files are already compressed and wouldn't generate many similiar bit patterns to optimize. As mentioned before deduplication works by finding similiar bit patterns among all the files (not just a single one like a regular compressor) and substitute them with a shorter bit string to cut down on the size the file occupy on the harddrive. I used the tool called ddpeval.exe which is included in Windows Server 2012 and allows you to run a test deduplication of a selected drive or folder on a Windows Server or Windows 7/8 system.

Here are the results when running ddpeval.exe on the folder with all the RAR-files intact, just as they are on the BA FTP:

Code:
Data Deduplication Savings Evaluation Tool
Copyright (c) 2012 Microsoft Corporation.  All Rights Reserved.

Evaluated folder: d:\ddpeval_test
Evaluated folder size: 120,05 GB
Files in evaluated folder: 931

Processed files: 930
Processed files size: 120,05 GB
Optimized files size: 117,21 GB
Space savings: 2,84 GB
Space savings percent: 2


As you see, the initial folder size is 120,05GB, and running it saved us a whopping 2,84GB. Hardly worth running it on RAR-files since the process takes a lot of time and CPU power with very little beneficial results.

For fun (since I am so easily amused :) ), I unpacked all the RAR-files and kept all the files unpacked, deleting the RAR-files. Mind you that RAR archive redundancy was lost, as well as any beneficial compression, but I wanted to see how much more efficient dedup would be on the entire folder in its uncompressed state. The results:

Code:
Data Deduplication Savings Evaluation Tool
Copyright (c) 2012 Microsoft Corporation.  All Rights Reserved.

Evaluated folder: d:\ddpeval_test
Evaluated folder size: 168,20 GB
Files in evaluated folder: 39048

Processed files: 12484
Processed files size: 168,00 GB
Optimized files size: 59,58 GB
Space savings: 108,42 GB
Space savings percent: 64


So, this time the entire folder with its unpacked contents take up 168GB. After running dedup on it we've concluded that we can save 108GB and shrink the entire thing down to 59GB on the harddrive. Impressive! This means that by not archiving the files we can shrink down the entire folder from its inefficient 120GB down to 59GB!

So, to summarize:

Code:
RAR, untouched:  120 GB
RAR, dedup:      117 GB

Unpacked:        168 GB
Unpacked, dedup:  59 GB

Difference:      117 GB > 59 GB


We basically cut the archive in two by first unpacking it all and then running dedup on it. Unfortunately there are some serious drawbacks:

  • No redundancy. Since everything is unpacked we won't be able to properly detect if an ISO becomes broken due to failing sector, hardware or bit rot. With RAR we can detect this and repair it with the built in 5% archive redundancy. We've lost a few releases in the past due to not having this reduncancy.
  • Larger files. Since a lot of the ISOs pack up quite neatly having them unpacked would create larger files for you to download. If you're on a slow link it can means several hours of longer download. This would also affect our upload times when making new releases or restoring the FTP after a failure or corruption.
  • Longer folder structure. Windows has its limits, and if we are to keep everything unpacked we will need to store each release in its own folder. Several releases contains multiple disk images, scans, readme files, extra documentation etc. This could create problems when extremely long filenames and folders are used.
  • Backup will grow accordingly. Even if we used dedup on our backup drives the backup would take considerably longer* and take up more space on our backup drives. Ponder that we could essentially cut the entire BA archive by half by extracting everything and running dedup on it, but the files would still take up its full space, so backup size could in theory double. Especially with all the delta compressed releases.
  • Cumbersome multiple downloads. Now, for each release you download you wouldn't only download a single file per release, but multiple files since the releases contains everything from 2 files to several thousands of files. This could drastically increase your download time due to the inefficiency of the FTP protocol.

However I have to admit that using dedup is quite tempting. But since we don't have a RAID system in place we can't trust that the files will be 100% at all times. Yes there are workarounds like having an active sync tool registering changes (such as rsync) and let it alert us when unexpected changes are found, but it would also create more issues for us and a lot of work to setup a reliable system. And we can't afford a local RAID solution either. Technically I could run dedup on the FTP server not part of the central BA server since I don't have the cost restraints as BA has (I could easily duplicate everything across several drives using RAID etc.) but it would still be impractical, especially now when we run a split server solution.

So, until some problems and concerns can be solved I can only say things about deduplication (or more exact, Microsofts implementation of it):

It works, it yields great results, but for our needs it's impractical and unwise to use it due to the high risk of undetected damage. And the last thing we need is damage to our files, intentional or not.

I hope you all got your answers, and thanks to dw5304 for the tips about testing it out and help with ddpeval.exe.

--------
*When Windows Servers and clients supporting dedup are on a local network the systems can actually work together and only send the deduplicated data, thus actually saving a lot of time when copying within the network. Unfortunately the main BA server and its auxiliary FTP server isn't connected in a local network, so no savings are done here, in reality we will lose a lot of time and bandwidth due to the growth of the files.

_________________
Image
Official guidelines: The Definitive Guide to BetaArchive :: Abandonware
Tools: Alcohol120% (Portable) :: DiscImageCreator
Listings: BetaArchive Database (beta)
Channels: Discord :: Twitter


Top  Profile  WWW
 PostPost subject: Re: Delta patches on the FTP        Posted: Tue Jul 16, 2013 9:11 am 
Donator
User avatar
Offline

Joined
Sat Jun 14, 2008 11:30 am

Posts
1206

Location
In the Floating island above the sky

Favourite OS
Windows NT 3.1 Build 297
Nice job, mrpijey. I think you can also apply this to MSDN compilations (since some discs are same).

_________________
Online periodically.
---
M$ Software Labels | OS Tester | Software Geek | Creative Agent!

Last edited by trustBA on Mon, 3rd Nov, 2013 6:99 am, edited 100 times in total.


Top  Profile  WWW
 PostPost subject: Re: Delta patches on the FTP        Posted: Tue Jul 16, 2013 10:12 am 
Administrator
User avatar
Offline

Joined
Tue Feb 12, 2008 5:28 pm

Posts
7993
Problem with MSDN discs is that they are not easily sorted into groups, so it would be very hard to know which ISO is the source. And I really don't want to start sorting the MSDN stuff into custom categories...

_________________
Image
Official guidelines: The Definitive Guide to BetaArchive :: Abandonware
Tools: Alcohol120% (Portable) :: DiscImageCreator
Listings: BetaArchive Database (beta)
Channels: Discord :: Twitter


Top  Profile  WWW
 PostPost subject: Re: Delta patches on the FTP        Posted: Fri Jul 19, 2013 4:01 pm 
Donator
Offline

Joined
Thu Dec 20, 2012 1:13 pm

Posts
437

Location
People's Republic of China

Favourite OS
Windows "Longhorn"
And wuat about beta Windows? For example, which longhorn build should I have first them I can use patch?

Willing to your answer.


Top  Profile  WWW
 PostPost subject: Re: Delta patches on the FTP        Posted: Sat Jul 20, 2013 2:30 am 
Administrator
User avatar
Offline

Joined
Tue Feb 12, 2008 5:28 pm

Posts
7993
The only delta patched releases are the one that come in multiple languages, and in those cases the US English one is always the source file. Example:

Quote:
Microsoft Windows 8 (''Windows 8'' 6.2.8250.0) (x64 beta).rar
Microsoft Windows 8 (''Windows 8'' 6.2.8250.0) (x64 beta) [French].rar
Microsoft Windows 8 (''Windows 8'' 6.2.8250.0) (x64 beta) [German].rar

Microsoft Windows 8 (''Windows 8'' 6.2.8400.0) (x64 rc).rar
Microsoft Windows 8 (''Windows 8'' 6.2.8400.0) (x64 rc) [Arabic].rar
Microsoft Windows 8 (''Windows 8'' 6.2.8400.0) (x64 rc) [French].rar
Microsoft Windows 8 (''Windows 8'' 6.2.8400.0) (x64 rc) [German].rar

The ones listed in bold are the source files, the others are delta patches for the source files.
If there are no multiple languages then there are no delta patches. I don't make patches between different builds.

In some cases I did pack together multiple SKUs of a particular software into one single archive, those archives are tagged with [+x] where x is the number of extra SKUs in the archive. For example:

Quote:
Microsoft Visual Studio 2013 (''Dev12'' 12.0.20617.1) (beta) [+4].rar


This archive contains the following releases:

Quote:
en_visual_studio_2013_preview_shell_isolated_x86_2359905.exe
en_visual_studio_premium_2013_preview_x86_dvd_2360138.iso
en_visual_studio_professional_2013_preview_x86_dvd_2360061.iso.xd3
en_visual_studio_test_professional_2013_preview_x86_dvd_2360657.iso
en_visual_studio_ultimate_2013_preview_x86_dvd_2359750.iso.xd3


So it essentially means that I grouped together all SKUs of that particular build of Visual Studio 2013 together, delta compressed it as much as possible and packed it up as much as possible.

But the general rule is that I never delta compress between different builds. It's always the same build, so no, there won't be any Longhorn deltas unless there are several SKUs or languages of the same build. Only then will I attempt at delta compressing it, if there are too little to be saved I just skip it and leave it as it is.

I hope I could answer your question.

_________________
Image
Official guidelines: The Definitive Guide to BetaArchive :: Abandonware
Tools: Alcohol120% (Portable) :: DiscImageCreator
Listings: BetaArchive Database (beta)
Channels: Discord :: Twitter


Top  Profile  WWW
 PostPost subject: Re: Delta patches on the FTP        Posted: Sat Jul 20, 2013 5:24 pm 
Administrator
User avatar
Offline

Joined
Tue Feb 12, 2008 5:28 pm

Posts
7993
Well, here's an update to my deduplication post a few posts up...

This time I tried with something new to see if we could keep the cake and eat it too. I mentioned before that by using dedup on unpacked file we would gain a lot of space, but at the same time everything would be unprotected from data corruption - no recovery data whatsoever. We would never know if something is broken and you users wouldn't be able to recover damaged downloads etc.

So, I though, why not keep the rars and still get good results? As I mentioned above, my attempts at gaining any favorable gains with the current BA archive yielded a measly 2%:

Code:
Data Deduplication Savings Evaluation Tool
Copyright (c) 2012 Microsoft Corporation.  All Rights Reserved.

Evaluated folder: d:\ddpeval_test
Evaluated folder size: 120,05 GB
Files in evaluated folder: 931

Processed files: 930
Processed files size: 120,05 GB
Optimized files size: 117,21 GB
Space savings: 2,84 GB
Space savings percent: 2


Clearly that was not an option for us.

So, I thought, why not pack up our archives with RAR, keep the 5% recovery data in the rars, but with the difference that instead of using maximum compression on the rar-files we use NO compression? That should basically dump the contents of the rar mainly unaltered and thus benefit from the deduplication and at the same time have that extra 5% data corruption insurance.

So, once again I had to unpack my test folder (a full copy of (Abandonware) Operating Systems\PC), I unpacked it all, then repacked it as-is but with 0 compression but with 5% recovery data. I.e same as current archive but without any compression.

First, the total archive grew from approximately 120GB to 185GB. So that alone was a size increase of 65GB. Then I ran the deduplication evaluation tool, this is the result:

Code:
Data Deduplication Savings Evaluation Tool
Copyright (c) 2012 Microsoft Corporation.  All Rights Reserved.

Evaluated folder: d:\0\0
Evaluated folder size: 184,91 GB
Files in evaluated folder: 931

Processed files: 931
Processed files size: 184,91 GB
Optimized files size: 91,80 GB
Space savings: 93,11 GB
Space savings percent: 50


Pretty good! We managed to actually cut the archive size by half! Granted, the archive grew after the repack, but still, we went from approximately 185GB to 92GB. So what were the gains? Let's do a quick summary again:

Code:
RAR, untouched:        120 GB
RAR, untouched, dedup: 117 GB

RAR, 0%:               185 GB
RAR, 0%, dedup:         92 GB

Difference:            117 GB > 92 GB


So by repacking the archives we managed to shave off 25GB, which is still a 27% difference. And a 30% difference compared to the current unaltered and undeduplicated archive size. Mind you, this test was only made on a single folder, but if we get similiar gains across the entire archive we could actually cut the storage needs down to 3.5TB, instead of the current 5TB. Unfortunately the whole archive would grow to approximately 7.7TB, but this time we would actually fit ~30% more on the same harddrives.

Benefits:

  • Uses less storage space, more space for more releases!
  • Recovery data intact, protection against occasional archive corruption.

Drawbacks:

  • Larger archives*, more to download for end user.

*Not necessarily larger archives in every situation. Some of the ISO files and releases we have packs up very poorly, so the end archive size wouldn't differ much between the current one and the one with 0% compression.

Remember, the main comparison here has to be done between the current archive size and the deduplicated 0% RAR archive size. That means a size reduction from 120GB > 92GB (~30%).

Just for fun I did the same deduplication test on a smaller beta folder and came up with with the following summary data:

(Beta) Operating Systems\Misc

Code:
RAR, untouched:        24,20 GB
RAR, untouched, dedup: 24,19 GB

RAR, 0%:               29,59 GB
RAR, 0%, dedup:        22,70 GB

Difference:            24 GB > 23 GB


Well, the difference here wasn't much at all, a little bit more than 1GB. This was a very small archive and the true savings seems to be with large archives and lots of files. And if we would apply deduplication on the entire BA archive then we would have the benefit of the deduplication process using all files in the archive, not just single folders as I have used in these examples.

However I still see some potential in this, and it may very well be a necessary process to take if we want to maximize our harddrive usage. Buying a harddrive at the store is cheap, buying a harddrive for a website is very expensive since you don't only rent it, but you also pay lots and lots as soon as you want to upgrade. So we need to use every bit we can squeeze out from the storage.

So, what do you members think? I was thinking about setting up a test server for myself and add the entire BA collection onto it and see what can be gained. This will naturally take some time to prepare, and also, if we choose to go this route with BA it would also be implemented over many months, because not only does all the releases need a full repack, but we also need to switch the server OS on BA (running 2008 R2 now, we need 2012) and that adds to the cost, both for license and time to install, optimize etc etc.

Well, this concludes my deduplication tests and I will leave it for now until we can test and implement it on a larger scale.

_________________
Image
Official guidelines: The Definitive Guide to BetaArchive :: Abandonware
Tools: Alcohol120% (Portable) :: DiscImageCreator
Listings: BetaArchive Database (beta)
Channels: Discord :: Twitter


Top  Profile  WWW
 PostPost subject: Re: Delta patches on the FTP        Posted: Wed Oct 30, 2013 4:06 pm 
Administrator
User avatar
Offline

Joined
Tue Feb 12, 2008 5:28 pm

Posts
7993
This topic is closed and locked as it will be followed up by a new one concerning delta compression, deduplication etc.

Delta compression is a no-go for BA, after considerable evaluations and tests with alternative technologies I've decided that delta compression - even when it's great - is not good enough for our needs. I will follow up shortly with a new thread covering new discoveries, results and discussions.

Locked

_________________
Image
Official guidelines: The Definitive Guide to BetaArchive :: Abandonware
Tools: Alcohol120% (Portable) :: DiscImageCreator
Listings: BetaArchive Database (beta)
Channels: Discord :: Twitter


Top  Profile  WWW
 PostPost subject: Re: Delta patches on the FTP        Posted: Wed Oct 30, 2013 10:48 pm 
Administrator
User avatar
Offline

Joined
Fri Aug 18, 2006 11:47 am

Posts
12565

Location
United Kingdom

Favourite OS
Windows 10
The new topic is located here: viewtopic.php?f=2&t=29865

_________________
Image

BetaArchive Discord: https://discord.gg/epK3r6A


Top  Profile  WWW
Display posts from previous:  Sort by  
Post new topic This topic is locked, you cannot edit posts or make further replies.  [ 17 posts ] 




Who is online

Users browsing this forum: No registered users and 2 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to:  

All views expressed in these forums are those of the author and do not necessarily represent the views of the BetaArchive site owner.

Powered by phpBB® Forum Software © phpBB Group

Copyright © 2006-2020

 

Sitemap | XML | RSS