Outage explained

Post by **Andy** » Wed Feb 27, 2008 1:43 pm

Hello all,

What a bloody night I've had...

9pm last night, C:\ ran out of space. I didn't understand why so I went to the temp folder and discovered a ton of files all around 170KB each, so I deleted them, but they kept coming back 1 or 2 a second.

I also found some odd looking files in the betaarchive folder. They weren't normal characters either, so I deleted them.

Come 10pm, the Temp folder was filling up again, so I did some investigation work with the help of DanielC and Mrpijey and we discovered it was a virus that had infected the system and was creating a load of temp files. It also infected all non-running EXE files, in both the OS and the website files, so anything that was an EXE has been deleted and will have to be restored from a non-infected backup.

It was nearly 1am before I even managed to backup what I could and do an OS reinstall. By this time I couldn't be bothered staying up to fix it, so I went to bed.

I came into work this morning an immediately started work on getting the server up. I've been on the phone to the DC for nearly 30 minutes, trying to get the NIC drivers to update etc, which was causing the main problem. It was only an hour or so ago that I managed to gain full control of the server with updated drivers and managed to get what I could back online.

No database data was lost, only exe files, which if running, were not affected (eg mysql, http, ftp, mail).

I sure hope this never happens again because this took the [censored]... seriously it did.

And thanks to the person who uploaded the file with the virus in. Yes it was a BETA, and I have a fair idea I know who it was. It could have been accidental or intentional but I can't prove either, so I'm going to forgive and forget this time.

Because of this I am now introducing the rule that ALL files MUST be RAR'ed or ZIPPED before being uploaded. No exe's or other extensions. This rule is final. Any exe's will be deleted with no questions asked. You HAVE been warned.

Problems aside, I hope everyone is glad the forum is back and that I never have to go through this again...

Enjoy the rest of your day

Post by **happy dude** » Wed Feb 27, 2008 2:00 pm

Ah, the joys of being a site admin.
Hopefully this doesnt happen again... but I guess some things are unavoidable.

Post by **Luckie** » Wed Feb 27, 2008 2:16 pm

great, that BA is back online

Post by **viper** » Wed Feb 27, 2008 3:56 pm

Good work getting BA back online.

I am very glad its back up again

Post by **WeirdEars** » Wed Feb 27, 2008 4:39 pm

Were the files .TMP files and did they begin with 'POS' by any chance?

Ex. POSXXXX.TMP

Because my computer's been having exactly the same problem...The files came in quantities of 4,500 or so in both the C: drive and the 'My Documents' folder...

Post by **Andy** » Wed Feb 27, 2008 4:40 pm

WeirdEars wrote:Were the files .TMP files and did they begin with 'POS' by any chance?

Ex. POSXXXX.TMP

Because my computer's been having exactly the same problem...The files came in quantities of 4,500 or so in both the C: drive and the 'My Documents' folder...

No, they were all random numbers and letters.

Post by **Gnome** » Wed Feb 27, 2008 4:50 pm

Yey,
ba back!
Well done Andy!!

Post by **RichardG867** » Wed Feb 27, 2008 6:48 pm

Didn't noticed this outage (I study in the morning and I disconnected ~5pm GMT-3). (Edited 26/Nov/2009)

(Pointless part removed 26/Nov/2009)

Post by **Vista Ultimate R2** » Wed Feb 27, 2008 9:36 pm

How did the virus in the uploaded file infect the server just out of interest, as it would surely have had to be run rather than just put on there?

Post by **Gnome** » Wed Feb 27, 2008 9:55 pm

Any idea what it was called?

Post by **Pureelite** » Thu Feb 28, 2008 12:40 am

Just wanted to say thanks andy, i know you have been working hard at it!
Good work on getting it all back.

Post by **DanielC** » Thu Feb 28, 2008 12:44 am

Toshua123 wrote:Any idea what it was called?

I found two when I scanned Andys backup ...

- Win32:Parite
- Win32:Parite-B@dll

Post by **Andy** » Thu Feb 28, 2008 12:49 am

DanielC wrote:
Toshua123 wrote:Any idea what it was called?
I found two when I scanned Andys backup ...

- Win32:Parite
- Win32:Parite-B@dll

What Dan said

I must have been infected by it when I "checked" one of the exe files was working, and it was infected. I was stupid enough not to have anti-virus because its so difficult to find a good one for server versions, and I had never had a problem in 3 years of running without one. Times change however, and when I get round to fixing the server back to 100% I will get round to installing an anti-virus package in the hope this never happens again.

Post by **stitch** » Thu Feb 28, 2008 2:40 am

I can donate a Symantec 9 license....

Post by **Bender** » Thu Feb 28, 2008 3:39 am

lol

Code: Select all

- Win32:Parite 
- Win32:Parite-B@dll

is really old. I was infected with that in ~2002

Post by **DanielC** » Thu Feb 28, 2008 4:00 am

SP is now running antivirus, Andy, you have a PM so you can do the same.

Post by **SaT** » Thu Feb 28, 2008 4:16 am

why you dont install NOD32?
it will stop any type of virus

and you can use the trial version

Post by **happy dude** » Thu Feb 28, 2008 7:08 am

what emperium said...

Post by **empireum** » Thu Feb 28, 2008 3:00 pm

happy dude wrote:*NO* anti-virus will stop ANY and every type of anti-virus.
Plus personally Ive never heard of Nod32 so I think theyll go wit hsomething more well known
That also depends if the site is on a Windows Server....

I think you mean "No anti-virus will stop any ... type of virus" Besides, NOD32 is a very good and thorough, yet fast scanner. It would also be my scanner of choice if I were running Windows boxen and were willing to pay for an anti-virus (as NOD32) is not free.

Post by **Vista Ultimate R2** » Fri Feb 29, 2008 11:54 pm

What I do is just have Kaspersky installed (got it legit now too, there was a promotion recently to get a free 1 year licence key) and scan anything I download from sources that can't be 100% trusted, I don't actually have it running in the background so I don't lose any performance to it - that would probably be the best solution on a server too.