BetaArchive Logo
Navigation Home Screenshots Image Uploader Server Info FTP Servers Wiki Forum RSS Feed Rules Please Donate
UP: 15d, 10h, 29m | CPU: 16% | MEM: 5398MB of 11747MB used
{The community for beta collectors}

Post new topic Reply to topic  [ 9 posts ] 
Author Message
 PostPost subject: Cron jobs... They're all over the place!        Posted: Sun May 17, 2009 7:55 pm 
Reply with quote
Administrator
User avatar
Offline

Joined
Fri Aug 18, 2006 11:47 am

Posts
12475

Location
Merseyside, United Kingdom

Favourite OS
Microsoft Windows 7 Ultimate x64
Hello all,

We haven't had a blog post for a while, and the issue of cron jobs keeps cropping up a lot on the server so I have decided its time to do something about it.

Its come to the point where the cron jobs are situated all over the place. There are some in the root of the BetaArchive htdocs, some in various sub-folders, and some not even within reach of the htdocs at all. Its time to centralise them so its less confusing for me and in the hope to remove a point of failure.

So how am I going to do this? Well its quite simple. A lot of the cron jobs simply output text into a text file from a PHP or Perl script. Since thats all they do, they don't have to be in any specific folder to do their job. This means its no problem to move them to one centralised folder such as "/cronitems" where they can, if necessary be sub-categorised into sub-folders.

As it stands there are at least 3 locations I can think of off the top of my head that contain cron job files, that don't need to be in that particular location. They can be anywhere, so why complicate things by putting them all over the place where a simple spelling mistake on such a long URL or folder structure could cause the script to fail? Thats the reason I'm planning to move them into a centralised location.

So which ones need to be moved? Well lets make a list:

  • - The FTP Servers list information
    All the data you see on this page, such as disk space, number of files, and whether the FTP itself is online, is all contained in text files. These files are written by a small PHP script that runs via a cron job every 60 seconds.
  • - The Total Current Archive Size box
    This box actually gets its data from two locations. The first is the hard disk space used by the beta files, this comes from a Perl script which outputs the data to a text file, ran by a cron job every 10 minutes. The second is the number of files. This comes from one of the text files of the cron job ran above.
  • - The server statistics display
    This box gets its data from several locations too. The first is a sub-domain of another domain linked to the server which contains the text files for the uptime and memory. The second is a text file located in the root of the BetaArchive htdocs for the CPU usage, and the last is another script that grabs the current bandwidth usage from the software monitoring it. These cron jobs run every 60 seconds.

Now that we have a list of cron jobs that can be moved, which ones can't be moved? Lets make another list:

  • - The session fix script for the forum
    Since this file relies on forum core files to work, it can't be moved and has to stay in its current location. This cron job runs every 10 minutes.
  • - The MRTG graphs
    Again this script relies on core files of MRTG to work, so it also can't be moved from its current location. The cron job for this also runs every 10 minutes.

Although the list of cron job scripts needed to be moved is small, the work involved in moving them is quite large. You might wonder why. Isn't it a case of moving the scripts and adjusting the cron job to match the new location? Well no, there is a lot more to it than that. Here is a list of things that need to be done:

  • 1. First of all the script does need to be moved, but I can't just move it, I have to copy it or the cron jobs will break, and this will also cause file read errors on some pages of the site and forum.
  • 2. Secondly I need rename the scripts to more meaningful names, and possibly edit the text file name output in some of the scripts to more meaningful names as well. Then I need to set up the cron jobs to point to the old and new locations at the same time so they both update ready for the next step.
  • 3. Third, I have to modify all the files that display information from the created text files to point to the new locations and check that they are working.
  • 4. Lastly, I have to disable the old cron jobs and delete the old files.

As you can see its a fair amount of work! Hopefully these changes, coupled with some minor script modifications along the way, will reduce server resource usage and allow more of the resources to go on serving you the pages of the site, files from the ftp, and much more!

_________________
Image

BetaArchive Discord: https://discord.gg/epK3r6A


Top  Profile  WWW
 PostPost subject: Re: Cron jobs... They're all over the place!        Posted: Sun May 17, 2009 8:47 pm 
Reply with quote
Staff
User avatar
Offline

Joined
Mon Jun 23, 2008 6:34 pm

Posts
692

Location
NY, USA

Favourite OS
Win 10 64-bit
It all sounds rather complicated... =/


Top  Profile
 PostPost subject: Re: Cron jobs... They're all over the place!        Posted: Sun May 17, 2009 9:58 pm 
Reply with quote
Administrator
User avatar
Offline

Joined
Fri Aug 18, 2006 11:47 am

Posts
12475

Location
Merseyside, United Kingdom

Favourite OS
Microsoft Windows 7 Ultimate x64
Well I've completed all of the changes stated above and everything seems to be OK. You might have seen the odd error as I was doing it but everything went smoothly! :)

Resource usage should be down now as the files are all static, rather than running the script every time the page loads like it was in some cases! Heres hoping!

_________________
Image

BetaArchive Discord: https://discord.gg/epK3r6A


Top  Profile  WWW
 PostPost subject: Re: Cron jobs... They're all over the place!        Posted: Sun May 17, 2009 11:00 pm 
Reply with quote
Administrator
User avatar
Offline

Joined
Fri Aug 18, 2006 11:47 am

Posts
12475

Location
Merseyside, United Kingdom

Favourite OS
Microsoft Windows 7 Ultimate x64
Its a good job I was messing with cron jobs today, I found the file count script wasn't actually counting all the files in the specified directory, it was getting to 2000 or so then not counting any more. I found a new script and that does work properly! I've replaced it so all is now well. We currently have over 3300 files! :)

_________________
Image

BetaArchive Discord: https://discord.gg/epK3r6A


Top  Profile  WWW
 PostPost subject: Re: Cron jobs... They're all over the place!        Posted: Mon May 18, 2009 12:34 am 
Reply with quote
Donator
User avatar
Offline

Joined
Tue Aug 12, 2008 7:37 pm

Posts
2381

Location
United States
Andy wrote:
We currently have over 3300 files! :)

:O, that is a TON more! Soon we will have 3333 files... :P


Top  Profile
 PostPost subject: Re: Cron jobs... They're all over the place!        Posted: Mon May 18, 2009 2:25 pm 
Reply with quote
FTP Access
User avatar
Offline

Joined
Fri Oct 17, 2008 9:51 pm

Posts
127

Location
Germany
Oh, I bet it's going to be over 9000! :P

_________________
This signature intentionally left blank


Top  Profile
 PostPost subject: Re: Cron jobs... They're all over the place!        Posted: Mon May 18, 2009 3:32 pm 
Reply with quote
Staff
User avatar
Offline

Joined
Mon Jun 23, 2008 6:34 pm

Posts
692

Location
NY, USA

Favourite OS
Win 10 64-bit
Tilka wrote:
Oh, I bet it's going to be over 9000! :P

Haha, classic.


Top  Profile
 PostPost subject: Re: Cron jobs... They're all over the place!        Posted: Thu May 21, 2009 2:17 pm 
Reply with quote
FTP Access
User avatar
Offline

Joined
Thu Jun 19, 2008 1:02 am

Posts
66

Location
Dundee, UK

Favourite OS
Mac OS X Leopard 10.5.7
Andy, can you call it a cron job if it's on Windows? :P Now isn't that technically a unix term?

None the less nice job on a faster BA :P


Top  Profile  WWW
 PostPost subject: Re: Cron jobs... They're all over the place!        Posted: Thu May 21, 2009 5:02 pm 
Reply with quote
Administrator
User avatar
Offline

Joined
Fri Aug 18, 2006 11:47 am

Posts
12475

Location
Merseyside, United Kingdom

Favourite OS
Microsoft Windows 7 Ultimate x64
Probably not but its a term thats easier to use in "techy talk". Scheduled task sounds lame! :)

_________________
Image

BetaArchive Discord: https://discord.gg/epK3r6A


Top  Profile  WWW
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 9 posts ] 




Who is online

Users browsing this forum: No registered users and 3 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to:  

All views expressed in these forums are those of the author and do not necessarily represent the views of the BetaArchive site owner.

Powered by phpBB® Forum Software © phpBB Group

Copyright © 2006-2019

 

Sitemap | XML | RSS