Milestone: 40TB archive!

Post by **mrpijey** » Wed Jan 18, 2017 10:09 pm

It took only 7 months, but we've finally hit the big 40!

40.64TB!

The latest Visual Studio pack along with Windows 10 finally broke the barrier!

And BetaArchive keeps going strong!

Thanks to all the members bringing more and more unique content to this community! This wouldn't have been possible without all of you!

Post by **awesomeguy** » Wed Jan 18, 2017 11:08 pm

I've been on BA for 1.5 years now, and I think I have been here to witness the biggest increase in archive size BA has had yet: 15TB! Question: How do you have the space for this? Deduplication cannot be that effective.

Post by **johnleakedfan** » Wed Jan 18, 2017 11:35 pm

When I Joined, I think it was around 10 TB or so (can't really remember).
BA has sure grown to be the biggest repository like how it was meant to be and there's no need to stop.

Website · Post by **Windows OS** » Thu Jan 19, 2017 12:11 am

Congratulations. It's insane big BA has became over the years. WOW.

Post by **longhornwhistler** » Thu Jan 19, 2017 12:32 am

Woohoo!!! Great job everyone!! 50TB here we come!

Post by **WindowsBetaz01** » Thu Jan 19, 2017 12:35 am

Wow. Just thinking back to when I joined, it was at 7.5TB. It's just mind blowing how much everyone has contributed. Here's to another 40TB! (we can only hope)

Post by **BilinSun** » Thu Jan 19, 2017 1:33 am

Congratulations!40TB is really much.
But I bet 20TB of all the betas are boring Windows 10 betas……

Post by **AyamiOoruri29** » Thu Jan 19, 2017 3:40 am

Congratulations for 40TB!

EDIT: I'll be waiting for Reach 50TB.

Post by **Goldfish64** » Thu Jan 19, 2017 5:32 am

Congratulations! Wait till we reach 50 TB.

Post by **Andy** » Thu Jan 19, 2017 1:03 pm

awesomeguy wrote:I've been on BA for 1.5 years now, and I think I have been here to witness the biggest increase in archive size BA has had yet: 15TB! Question: How do you have the space for this? Deduplication cannot be that effective.

Due to massive similarities in data deduplication allows more than a 50% saving in disk space across the archive. If we used a RAID system we would get a higher deduplication rate, but we prefer to use separate drives for a number of logistical reasons including backups. This may change in the future but for now we still get an incredible saving with this system. Without it, the archive would never be cost viable for us.

Post by **ShadixAced** » Thu Jan 19, 2017 1:27 pm

Well done !
When i joined it was at 30 TB or 35TB i do not know really.
Now we have 10TB (or 5TB) more so it's freaking insane !

Now let's get to the 30k releases done

Post by **Resident007** » Thu Jan 19, 2017 2:05 pm

Congratulations, BA! Keep going

Post by **qta3426** » Thu Jan 19, 2017 3:17 pm

4 years ago : 4TB
now : 40TB
wow....

Post by **MSUser2013** » Thu Jan 19, 2017 10:38 pm

MunKeongSu wrote:4 years ago : 4TB
now : 40TB
wow....

I know how you feel. I remember 4.7TB when I first joined this site, and I also saw the 30TB announcement half a year ago, Here's to reaching 50TB in the near future

.

Post by **mrpijey** » Thu Jan 19, 2017 10:45 pm

awesomeguy wrote:I've been on BA for 1.5 years now, and I think I have been here to witness the biggest increase in archive size BA has had yet: 15TB! Question: How do you have the space for this? Deduplication cannot be that effective.

It is actually that effective. It works best with releases that are much alike, such as multiple languages of Windows ISOs etc.

Once the newest dedup session finishes by tomorrow I will post some stats to show the deduplication rate.

Post by **DiskingRound** » Fri Jan 20, 2017 1:33 am

Congratulations for forty grand tetrabytes of storage. Thanks for helping to establish ourselves as the true #1 beta site, and thanks to countless people who helped to contribute this releases. Andy, mrpijey, thanks for bringing this incredible place to the attention of many. It even made it to the news sometimes (it made it to newspaper when marcoguy did that thing with Apple I forget a few months ago). But I'm sure this site will continue for years to come.

Post by **dw5304** » Fri Jan 20, 2017 5:58 am

mrpijey wrote:
awesomeguy wrote:I've been on BA for 1.5 years now, and I think I have been here to witness the biggest increase in archive size BA has had yet: 15TB! Question: How do you have the space for this? Deduplication cannot be that effective.
It is actually that effective. It works best with releases that are much alike, such as multiple languages of Windows ISOs etc.

Once the newest dedup session finishes by tomorrow I will post some stats to show the deduplication rate.

i've been around for years. Friendships have grown and suggestions have been made.

hears to the next 40tb and the next version of dedup / file system / ect pretty soon ba will need to look into spanning boxes to keep up with the data demand

...

Post by **Hackerpcs** » Fri Jan 20, 2017 10:08 am

Congratulations to everyone managing the archive!

mrpijey wrote:Once the newest dedup session finishes by tomorrow I will post some stats to show the deduplication rate.

Waiting to see those!

Post by **mrpijey** » Fri Jan 20, 2017 12:24 pm

Here is the dedup stats. Worth to note is that the four volumes are four 4TB drives which gives you an idea of how effective the savings are. The first volume is the one that holds all the PC beta operating systems (i.e all the Windows betas). As Andy hited at, the efficiency would be higher with fewer volumes, my backup drives are two 8TB drives and I get about 1.5-1.8TB more space on those due to more efficient deduplication. In the future I will replace the main drives with new 10TB or 12TB drives but that's pretty costly at the moment and not a critical need at the moment.

Code: Select all

SavedSpace           SavingsRate          Volume
----------           -----------          ------
13.62 TB             82 %                 E:
 4.67 TB             59 %                 F:
 4.49 TB             60 %                 G:
 1.87 TB             33 %                 H:

Post by **Ludacris** » Fri Jan 20, 2017 2:45 pm

Well back when I joined there was no central archive like that - there was the abyssunderground FTP and several other Home-hosted FTPs. Things have surely changed over the last ~ 10 years.

Offtopic Comment

Edit: In fact it's really almost 10 years since I joined (27.01.2007). Time really flys by...

Post by **dj_cityboy** » Fri Jan 20, 2017 3:59 pm

logged in just to say congrats on reaching the 40TB mark, hard to believe its that size.

peas
dj_cityboy

Post by **Andy** » Fri Jan 20, 2017 4:10 pm

dj_cityboy wrote:logged in just to say congrats on reaching the 40TB mark, hard to believe its that size.

peas
dj_cityboy

We can't quite believe it ourselves sometimes

Post by **DarkerJ** » Fri Jan 20, 2017 6:19 pm

I hope you're ready for Vista Vista Vista!

Sent from my XT1254 using Tapatalk

Post by **Someguy2** » Sat Jan 21, 2017 1:16 pm

I remember the time when BetaArchive had only 1 TB of files. BetaArchive was the biggest source for beta software and abandonware for me when I joined the site (1 TB was a big deal back then) and I was really happy when I was granted FTP server access. BetaArchive offered more beta software compared to the other sites I've visited.

Today, BetaArchive is still the largest source for beta software and abandonware (I mean, 40 TB of files is pretty big). I hope we reach 50 TB (and maybe even 100 TB) someday.

Post by **awesomeguy** » Sat Jan 21, 2017 6:03 pm

How much space is actually being used on the FTP server?