Server Transfers – Why I hate them and love them so much

There are lots of reasons to hate server transfers. Thankfully I don’t have to do them too often. In fact, I’ve only ever done a transfer 4 times and each time it was always different. Different things go wrong, the system setup was different and the timing was different.

Probably the biggest thing I hate about a move is the effort required and all of the collaboration of everyone involved in using the server. For example anyone who hosts on the server has to be included in the transfer plans so they can coordinate their own plans. In this instance I was lucky – only one person was delayed in getting back to me regarding their site.

So with a server transfer, where do you begin? Websites? DNS? Well lets list the things you need to change and then in what order.

Web Server
Applications
FTP
Database
Data
Mail Server
Cron Jobs
Backups
DNS

Personally I start with the applications I’ll need for the transfer, which primarily consists of an FTP client. Then I move onto configuration of the web server to get the main websites online as soon as they come over. Next are the databases, then FTP and mail server, and finally the data. Backups, DNS and cron jobs can wait until last.

Applications
This should be easy and goes without saying. You just need to install the applications you’ll need during and after your transfer.


The Web Server – IIS
Microsoft IIS is very powerful, but it’s not without its quirks either. Getting PHP set up should be easy, but Microsoft made IIS modularised so it’s not as easy as it sounds. You have to add the module to each host rather than just one global module for all hosts. This is fair enough though as it allows you to have individual configurations for each which allows you to both keep it secure and give different permissions for them.

Once it’s running, you’ll probably never touch it again except for the odd little tweak. When I tried to get IIS to run PHP it took me 2 hours simply because I hadn’t done it in over a year. I felt lost and had to engage the help of some other IIS users in order to get it running again. Once I had it though, each of the hosts went on without incident.


Database – MySQL
MySQL is very robust but when it breaks it breaks badly, especially when it won’t start. When this happens you have to consult and decipher the system logs just to understand what was going on. I thought I’d try to be smart this time around and upgrade MySQL at the same time as the transfer. That turned out to be a very bad idea.

The upgrade worked but then I couldn’t access any of the databases. I tried to revert back and then that version wouldn’t start. I removed both versions, rebooted and reinstalled. Finally that got it working but I was back at square one with no upgrade. I decided it wasn’t worth the trouble for now and just left it alone.

That charade itself took 2 hours from start to finish when it should have taken a mere 10 minutes. Can you see my anger and pain yet? My eyes at this point were already bloodshot!


Mail Server – hMailServer
Surprisingly this transition went smoothly. It just goes to show the better built software doesn’t it? I even managed to perform a full upgrade after the install too and everything has worked flawlessly. A little configuration change was needed to accept the new host but after that it hasn’t been touched since!

I love hMailServer for many reasons, one being its simplicity. I’ve used a few mail servers but this one came out top.


FTP – Filezilla FTP and Gene6 FTP
Because of the way I run things on the server, we have two FTP servers. One runs primarily BA (Gene6) while the other is for everything else. I use Gene6 because it has features that allow the merging of FTP servers externally into the same directory listing. That is how we linked two servers together. Both Filezilla and Gene6 both allow direct copy and paste of their setups, but Gene6 requires an install over the top to install its service.

I started both up but Gene6 wouldn’t start. This triggered my memory of last time I tried to start Gene6 up on a new server. Data Execution Prevention (DEP) was turned on and prevented it from starting. A quick trip to control panel to exclude the EXE and we were away.

It’s niggles like this that annoy server admins like me! Those bloodshot eyes just got a little redder...


Data – Transferring from old to new
This has to be one of the hardest parts, and the most time consuming. Transferring all of the data (all 1TB of it) from the old host to the new took over 48 hours. I began with the website data and got that over and done with, and transferred some of the websites to the new server and at the same time disabled them on the old one. This also required changing the DNS records on the old server to point to the new IP, but not a move of the DNS itself just yet.

The problem with transferring data between old and new is that you should do it when the current host is offline and not changing data. This is hard when you want to minimise downtime. Of course most admins would just shut the site down for a day or two, but I’m not your ordinary admin I managed to do it with zero downtime except for those who saw the DNS change and the site error 404 for a short time.

This takes skill and patience. You have to make sure that you get all of the changed data from the other side. The easiest way is by transferring it all twice. Send it all over enough to get the site running and then copy everything again to update it and synchronise. Sounds hard but it’s not in most cases. The harder part is the MySQL as explained above.


DNS – Propagation frustration
As many of you know DNS takes it’s time to propagate around the web. A simple DNS record change is easy; they’re normally effective within an hour of the change. But when it comes to the name servers being changed, the actual records that tell your computer which server to contact to get its DNS records, this takes time – sometimes 48 hours or more.

In order to this properly you have to be very prepared. You have to have both sides DNS records synchronised so they’ll still work after the transfer. Once you have that done you can change the zone records to the new IP of your hosting. Only then can you can start modifying the name servers. This is done at the root level of your domain through your registrar. Their servers then contact the root DNS servers around the world and your DNS begins to propagate to the new server.

While the propagation is in progress, the old DNS server should remain operational for at least 48 hours so those people who haven’t seen the change yet can still access the site. From that point on all you can do is wait and continue to check the site is updating. Many tools on the web will show you what name servers it’s seeing for your domain so it’s nice and easy to check.


Cron Jobs and Backups – Automation at its finest
After you’ve got everything working, you can set to work on the little things that keep the server running smoothly. This includes cron jobs and backups. These tend to be fairly easy. I have premade scripts and software to run the backups for me. All I have to do is make sure they’re present and pointing to the right folders, set up the cron and off we go. Usually it’s as easy as that.



Now you can see why server transfers can be so demanding and so damn annoying. I hate doing it in some ways and love it in others. I hate it when things don’t work but I’m a geek at heart, so I love trying out new tech and exploring new options. If I didn’t, I wouldn’t make a very good geek right? OK so I may not be as geeky as some of you lot, but its good enough for me

I hope everyone enjoys the new server and appreciate the effort that has been put into it. It’s certainly been a few hard days for me but everything is pretty much back to normal again. I’ll need to monitor things for a few days but then I do that anyway so it’s not a big thing for me!

As usual, questions and comments are totally welcome!

Tip Top Job Andy

A note, PHP/MySQL are easy, it is set up within 10 minutes and fully compatible / manageable from within IIS, install the ''Microsoft Web Platform Installer'', go to http://php.iis.net/ and click install.
Each site you make will run in its own php-cgi.exe / IIS worker process.
MySQL also installs via this one and from the start integrated with IIS/PHP.

Yea we know...........

@iBetaTosh, was that for me or Andy?
If it was for me, no need to post and i''ll just stop helping people in the future or give tips ok?

I know about this however the MySQL again would probably not work because of the upgrade issues, plus it does not install to the place I want it to.

PHP is also configured to my needs and wouldn't work through the web platform installer either, otherwise I would go that route. It's great for getting started quickly but not really good for well configured production machines.

Changing DNS records tend to be easier when the TTL is not set to 86400, which is irritating.

Well, it worked for me, but hey, I only run my website on my server, not multiple configurations.

But it is an great tip for people who start (or are starting) with Web Servers.

Google DNS showed the change almost instantly since I didn't notice any downtime, ISPs should just use google dns instead of their own dns servers, cheaper for them and better for the clients.

Businesses need to stop trusting Google with their critical data... Don't trust Google. Jeez.
If you think they got where they are today without selling your usage data then you're wrong.


Someone give this man (Riostar) a gold star Damn straight. I always like to drop the TTL (Time-To-Live) on records to 10 minutes or less, 48 hours+ before the move of the server, that way change over happens damn quickly for almost everyone. There's always a few ISPs that don't honour TTL and will define their own minimal update rate, but that's not my fault.

None the less Andy's method is well thought out and it seems the new server is working good

Now you're on gigabit let's see them DDoS gone!

Nice insight into the server configuration. Wouldn't imagine that you would run off Windows servers rather than Linux, I've only ever heard bad things about them, eg performance issues and that they're really high on resources.

Seems like you found a way to make it all work out in the end.

You only ever hear bad things because it's the Linux fanboi's that bash Windows servers and likely the people you've heard or read it from.

Actually Windows is a great server OS. I use Windows XP with Apache for my web server the State of California uses Windows Server with IIS and many more people are using Windows.

Agreed

Apart like me. But I installed PHP 5.(something) and Microsoft SQL Server 2000 instead of MySQL. Because phpBB3 didn't detect MySQL and stucks in MSSQL.

Never mind, it's working fine since I tested it.

That comment just made me wish Microsoft had public DNS.

@Andy: Maybe it's a dumb question, but when you say "cron jobs", are you referring to the "scheduled tasks" thing that's built into Windows? .. or have you found some other nifty utility that works like cron for Windows? Just wondering because usually "cron" refers to Linux.

@compgeke: I agree, I've been pretty satisfied with the performance of Windows as a server OS. I usually hook up my modem / internet connection to the Windows Server and use it as an RRAS router to route internet to the rest of the house.... and it works very well.

Yes, he's referring to the scheduled tasks you can set on a server, but it seems more common to say cron jobs even if you aren't using *NIX.

You are sadly mistaken (see http://w3techs.com/technologies/overview/web_server/all and http://w3techs.com/technologies/overvie ... system/all). Linux is by far the most common platform for webhosting, because it's free, more stable and often more secure. People can say their Linux server has been running for years straight, since when can a Windows server admin say that? Windows servers constantly receive patches due to their high attack vector which prevents the ability to maintain large chunks of uptime, in addition the fact that Windows server relies on a GUI makes it a much easier attack vector as opposed to Linux servers which are almost always terminal only and restricted with policies such as secret keys as opposed to simply relying on the old username/password authentication scheme utilized by windows.

Windows is becoming more popular, but no it's not the most popular at the moment. Linux is preferred for many reasons, scalability being one of them and because it's the "norm" in that industry.

I wouldn't go so far as to say Apache is more stable. In my experience IIS is much more stable but this can be down to a lot of factors anyway so you can't really debate this fact unless both are running the same setup.

Both Linux and Windows is capable of running for years straight, I've managed a full year on a Windows box as well, and frequently hit 6 months on my home server. The reason I don't go longer is because updates are often required to ensure the security of the system. If it's got years uptime, you can guarantee it's not secure.

Windows relying on a GUI is probably more secure because a terminal can do everything on a Linux box, it can't on a Windows box so "getting in" is much harder to begin with. You'll probably find a lot of Linux and Windows users do and don't use secured authentication, it's their preference.

There is no right and wrong here. Both do the job equally well but it all depends on what you want from the server, the applications you're running and what it's doing as a whole.

You might think that a Linux server running for years straight is insecure but you'd be wrong. That's actually one of Linux's benefits over Windows, its ability to have services updated and replaced without requiring a reboot. Pretty much any Linux service can be stopped for a update and then restarted afterwards without needing a reboot, and the kernel can even be updated without a reboot through the use of the 3rd party utility ksplice. So ultimately it is entirely possible to never need to reboot Linux at all ever short of replacing failing hardware.

As for the GUI making it harder to get in, I strongly doubt that. RDP is one of the most well known remote desktop services and I garuntee if you look at any Windows server firewall log you will see constant banging against that port. Then once someone gets in to the server through RDP they're given a classic username/password GUI prompt which makes it easy for brute forcing. I'm not a Windows server admin pro so maybe I'm unaware of all the possible security policies, but afaik the best you can do to defend against this attack on Windows is set a lockout policy which is more of a nuisance then anything.

Yet another Linux fanboi who always has to be right. Argument over. Topic locked.