BetaArchive Logo
Navigation Home Screenshots Image Uploader Server Info FTP Servers Wiki Forum RSS Feed Rules Please Donate
UP: 27d, 9h, 6m | CPU: 47% | MEM: 5595MB of 10761MB used
{The community for beta collectors}

Post new topic Reply to topic  [ 17 posts ] 
Author Message
 PostPost subject: The server is held together with cron jobs!        Posted: Fri Apr 03, 2009 2:39 pm 
Reply with quote
Administrator
User avatar
Offline

Joined
Fri Aug 18, 2006 11:47 am

Posts
12473

Location
Merseyside, United Kingdom

Favourite OS
Microsoft Windows 7 Ultimate x64
Well I said I'd get round to giving you guys the lowdown on this one, so here it is!

As you know the server does a fair amount of work on its own. It went 2 weeks without me even touching it while I was in hospital, something not many servers can do. The reason behind this is the cron jobs running behind the scenes keeping things running and fixing potential problems before they occur.

I have a number of cron jobs running at various times.

Quote:

1 minute cron job
10 minute cron job
1 hour cron job
4 hour cron job
1 week cron job


So, what runs on each one? Most are self explanatory but I've put some comments in green for those that aren't so obvious.

Quote:
1 minute cron job
The FTP statistics page update script
The mini-status update script
The memory usage update script
The CPU usage update script
The UPTIME update script
MRTG Traffic Graphs update script

Quote:
10 minute cron job
Setting the CPU usage priority of the following processes to LOW:
Quote:
PRTG Traffic Grapher.exe
DFRGNTFS.exe //Defrag, which doesn't get run often
WinSCP.exe
IRCXPro.exe
FileZilla.exe
uTorrent.exe
FileZilla Server.exe
php.exe // PHP4
php-cgi.exe //PHP5
sven1.exe // Game server process
sven2.exe // Game server process
gmod1.exe // Game server process
css1.exe // Game server process

Delete all files in d:\*.tmp // temporary files build up due to a php bug, so this deletes them
Delete all files in d:\public_html\betaarchive\*.tmp // temporary files build up due to a php bug, so this deletes them
Run the awshost status update script
Run the sessionfix script for the forum // without this the forum kept getting session errors, so this runs every 10 minutes regardless of any errors
Kill all cmd.exe processes running

Quote:
1 hour cron job
Kill all php.exe processes // php sometimes crashes, so this kills all the processes once an hour in case this happens, so the max downtime is 1 hour

Quote:
4 hour cron job
Kill all DeltaC.exe processes // this is the backups, which have a habit of crashing, so they get killed every 4 hours and reopened by the next cron job
Kill all rsync.exe processes // this is the backups, which have a habit of crashing, so they get killed every 4 hours and reopened by the next cron job

Quote:
1 week cron job
STOP and START FileZilla Server FTP Server service // Filezilla drops connections when open a long time so this restarts the service once a week
Kill all netgraph.exe processes // when left open netgraph uses a lot of memory so once a week its killed
START netgraph.exe // this re-opens netgraph after being killed above


Most of the kill and start crons are designed to simply kill all unused processes that are left open by some operations, taking up memory. In the case of netgraph, it is killed and reopened once a week to clear old data from memory, as it doesn't do this itself. Over the course of a few weeks it takes up a few hundred MB of RAM. FileZilla Server seems to have issues keeping connections if open for weeks at a time, so a quick service restart of that, taking just a few seconds, solves this issue. It does mean anyone who is connected gets disconnected but its once a week so it doesn't really affect anyone that much.

So there you have it, thats how the server seems to fix itself!

_________________
Image

BetaArchive Discord: https://discord.gg/epK3r6A


Top  Profile  WWW
 PostPost subject: Re: The server is held together with cron jobs!        Posted: Fri Apr 03, 2009 2:51 pm 
Reply with quote
Donator
Offline

Joined
Fri Oct 26, 2007 5:12 pm

Posts
2461
For those who don't like reading, BetaArchive would simply like to advise its users that the server in its current state is held together with a combination of hot glue, masking tape and carefully placed rubber bands, so carefully placed that if even one were to be pulled or removed the whole server would disintegrate

and all that [censored]


Top  Profile
 PostPost subject: Re: The server is held together with cron jobs!        Posted: Fri Apr 03, 2009 2:52 pm 
Reply with quote
Administrator
User avatar
Offline

Joined
Fri Aug 18, 2006 11:47 am

Posts
12473

Location
Merseyside, United Kingdom

Favourite OS
Microsoft Windows 7 Ultimate x64
I've never heard it put like that before, but I suppose thats about right!!!

_________________
Image

BetaArchive Discord: https://discord.gg/epK3r6A


Top  Profile  WWW
 PostPost subject: Re: The server is held together with cron jobs!        Posted: Fri Apr 03, 2009 2:53 pm 
Reply with quote
Donator
Offline

Joined
Fri Oct 26, 2007 5:12 pm

Posts
2461
This is why OVH don't release pictures of their datacentres. You'd be horrified if you saw the physical state the servers in :P


Top  Profile
 PostPost subject: Re: The server is held together with cron jobs!        Posted: Fri Apr 03, 2009 2:56 pm 
Reply with quote
Administrator
User avatar
Offline

Joined
Fri Aug 18, 2006 11:47 am

Posts
12473

Location
Merseyside, United Kingdom

Favourite OS
Microsoft Windows 7 Ultimate x64
I can imagine it being a bit like this... lol.

Image

Nah, with all seriousness, these crons are necessary to make my life easier, otherwise I'd be sat there running scripts and restarting software all the time to keep it all 100% fully working.

_________________
Image

BetaArchive Discord: https://discord.gg/epK3r6A


Top  Profile  WWW
 PostPost subject: Re: The server is held together with cron jobs!        Posted: Fri Apr 03, 2009 2:59 pm 
Reply with quote
FTP Access
User avatar
Offline

Joined
Fri Oct 17, 2008 9:51 pm

Posts
127

Location
Germany
I like your "that process crashes from time to time so let's just kill it from time to time" attitude. :D

_________________
This signature intentionally left blank


Top  Profile
 PostPost subject: Re: The server is held together with cron jobs!        Posted: Fri Apr 03, 2009 3:06 pm 
Reply with quote
Administrator
User avatar
Offline

Joined
Fri Aug 18, 2006 11:47 am

Posts
12473

Location
Merseyside, United Kingdom

Favourite OS
Microsoft Windows 7 Ultimate x64
Has to be done! Its the easiest fix as well rather than solving the problem. The problems are in the software itself, which I didn't make, so I can't fix those problems, only the software creator can and in most cases they won't. In the case of Filezilla I think its the sheer volume of data passing through it that eventually causes the issues. It doesn't have a memory leak, it just stops accepting connections properly. mrpijey discovered this a few weeks ago and since restarting it, its been flawless again and speeds have increased.

_________________
Image

BetaArchive Discord: https://discord.gg/epK3r6A


Top  Profile  WWW
 PostPost subject: Re: The server is held together with cron jobs!        Posted: Fri Apr 03, 2009 3:11 pm 
Reply with quote
Donator
User avatar
Offline

Joined
Sat May 12, 2007 1:05 pm

Posts
5271

Location
The Collection Book

Favourite OS
Windows & Phone
happy dude wrote:
This is why OVH don't release pictures of their datacentres. You'd be horrified if you saw the physical state the servers in :P


They do have an video of there datacenter.

Al is put in racks and neatly put away.

And If I am downloading with Filezilla, the only time I know about the cron job is that the download restarts at the moment it would disconnected (well resumes).

Then the timer get's reset, so I was downloading for 10 minutes and it was back at 0 minutes.

_________________
Image
http://www.thecollectionbook.info
Subscribe to our Image for updates and like us on Image.

Reading Mode only, PM's possible.


Top  Profile  WWW
 PostPost subject: Re: The server is held together with cron jobs!        Posted: Fri Apr 03, 2009 3:18 pm 
Reply with quote
Administrator
User avatar
Offline

Joined
Fri Aug 18, 2006 11:47 am

Posts
12473

Location
Merseyside, United Kingdom

Favourite OS
Microsoft Windows 7 Ultimate x64
Yeah, the restart only happens once a week though, every Sunday at 4am.

_________________
Image

BetaArchive Discord: https://discord.gg/epK3r6A


Top  Profile  WWW
 PostPost subject: Re: The server is held together with cron jobs!        Posted: Fri Apr 03, 2009 3:46 pm 
Reply with quote
FTP Access
Offline

Joined
Mon Jan 28, 2008 4:53 am

Posts
428
What actually runs the cron jobs? Is there specific software out there that does it, or is it included in Windows Server? (I'm assuming you're using Windows Server, possibly 2003?).

Also, when you kill these services, what starts them back up? Do they just startup again automatically, or does the cron job take care of it?


Top  Profile
 PostPost subject: Re: The server is held together with cron jobs!        Posted: Fri Apr 03, 2009 3:54 pm 
Reply with quote
Administrator
User avatar
Offline

Joined
Fri Aug 18, 2006 11:47 am

Posts
12473

Location
Merseyside, United Kingdom

Favourite OS
Microsoft Windows 7 Ultimate x64
Its task scheduler in Windows. You can get software to do it but when windows has a service for it why bother?

When a task is killed, its one that is opened by something else or another cron. PHP for example is executed by the web server as and when a page is needed, so that opens again by itself even if a cron kills it. Netgraph for example needs to be opened by another cron, which is why after its killed its started again by the same cron job.

_________________
Image

BetaArchive Discord: https://discord.gg/epK3r6A


Top  Profile  WWW
 PostPost subject: Re: The server is held together with cron jobs!        Posted: Fri Apr 03, 2009 8:19 pm 
Reply with quote
FTP Access
Offline

Joined
Mon Jan 28, 2008 4:53 am

Posts
428
Ah. I see.

I've never messed with Task Scheduler, so I didn't even know what it can and can't do.


Top  Profile
 PostPost subject: Re: The server is held together with cron jobs!        Posted: Fri Apr 03, 2009 8:20 pm 
Reply with quote
Administrator
User avatar
Offline

Joined
Fri Aug 18, 2006 11:47 am

Posts
12473

Location
Merseyside, United Kingdom

Favourite OS
Microsoft Windows 7 Ultimate x64
Its simple really, you just tell it what file to run, at what times and how often. Nothing more to it than that except if you wanted to change permissions too, which you probably wouldn't in most cases. Its fairly simple to understand.

_________________
Image

BetaArchive Discord: https://discord.gg/epK3r6A


Top  Profile  WWW
 PostPost subject: Re: The server is held together with cron jobs!        Posted: Fri Apr 03, 2009 11:34 pm 
Reply with quote
Donator
User avatar
Offline

Joined
Tue Aug 12, 2008 7:37 pm

Posts
2381

Location
United States
Andy wrote:
Its the easiest fix as well rather than solving the problem.


This is a good quote, especially when the background information is not there :P.


Top  Profile
 PostPost subject: Re: The server is held together with cron jobs!        Posted: Sun Apr 05, 2009 7:54 am 
Reply with quote
FTP Access
Offline

Joined
Thu Aug 24, 2006 6:28 pm

Posts
352
I'm trying to get the cash together to rebuild my server (which just lost its 1TB drive) with a cheap quad core and a RAID card. I'm planning on Server 2008 with the Windows Home Server components (I can't believe no one's really tried yet to rip the files from the Home Server installation disc and make a standalone copy that can run on regular server yet); it'll be my web server, file server, backup server, download server, FTP server, firewall, virus scan and proxy server, router, and whatnot.

Anyways, I'll see if I have any good tips to give you afterwards. My linux server ran like a rock, restarting only when I let it install updates (until recently when it needs restarted every time lighttpd can't access the 1TB that's gone). My desktop only restarts when I install an update or a program, or those rare times that a game crashes. My last windows server almost never got restarted. Processes really shouldn't be crashing like that.


What really ticks me off is that the largest web programs out there, apache/php/mysql, require tedious manual tuning to run at peak efficiency. My $10 computer games automatically pick the best graphics settings for me, why can't these programs benchmark my system and tune everything for me, maybe doing some dynamic adjustments to keep the server up? Why hasn't someone made a program that will automatically balance resources between those three programs for max efficiency? You'd think they'd make a killing with something like that.


Top  Profile
 PostPost subject: Re: The server is held together with cron jobs!        Posted: Sun Apr 05, 2009 4:45 pm 
Reply with quote
Donator
User avatar
Offline

Joined
Wed Aug 27, 2008 12:52 am

Posts
890

Location
United States

Favourite OS
Windows Server 2008 Standard
Very interesting! It's simple to coordinate but is very very effective. What I don't understand is why some applications don't free up memory that it no longer needs

_________________
Image


Top  Profile
 PostPost subject: Re: The server is held together with cron jobs!        Posted: Sun Apr 05, 2009 4:46 pm 
Reply with quote
Administrator
User avatar
Offline

Joined
Fri Aug 18, 2006 11:47 am

Posts
12473

Location
Merseyside, United Kingdom

Favourite OS
Microsoft Windows 7 Ultimate x64
Thlump wrote:
What I don't understand is why some applications don't free up memory that it no longer needs


Because the coders didn't design it that way, or its not possible with the environment they used as its not advanced enough.

_________________
Image

BetaArchive Discord: https://discord.gg/epK3r6A


Top  Profile  WWW
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 17 posts ] 




Who is online

Users browsing this forum: No registered users and 4 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to:  

All views expressed in these forums are those of the author and do not necessarily represent the views of the BetaArchive site owner.

Powered by phpBB® Forum Software © phpBB Group

Copyright © 2006-2018

 

Sitemap | XML | RSS