Planned downtime

The place for anything at all...
Post Reply
User avatar
takyoji
Legendary
Legendary
Posts: 530
Joined: Tue Nov 27, 2007 4:46 pm
Gender: Male
Mood: Busy
Location: Minnesotan, dontcha' know!
Contact:

Planned downtime

Post by takyoji »

Tonight the forum will finally be upgraded to phpBB 3. This is planned to occur at 2am Eastern, 1 am Central, 12am Mountain, 11pm Pacific. During this period the forum will be disabled thus inaccessible. We can do the conversion process live without anyone being able to notice, however it would put a heavy load on the server and potentially cause the server to crash as I've heard from a member of the phpBB support team thus we'll just take the safe route by disabling the forum and doing the conversion process on a different server.

The reason why it has to be disabled is due to the fact that when we upgrade we need to have it convert all post, users, and so on into the phpBB 3 format. And since we'll be converting the posts on a different server, there can't be users writing new posts on this forum as those posts wouldn't be converted. Once the conversion is complete, the new forum installation will be uploaded to the website and finally be accessible.

So the whole point is: Don't go into cardiac arrest if you see a message "Forum disabled" tonight and you can still post right now, but you won't be able to when the conversion process is in progress.
Avatar original image by "sometimesong" of Flickr, released under the Creative Commons license (share-alike, attribution, non-commercial, no-derivatives)
Shadow Wulf
Site Admin
Site Admin
Posts: 7572
Joined: Thu Jun 23, 2005 3:17 pm
Location: Zephyrhills, Florida
Contact:

Post by Shadow Wulf »

phpBB3 here we come! I might not be able to experience it but oh well. :D
Every government degenerates when trusted to the rulers of the people alone. The people themselves are its only safe depositories. - Thomas Jefferson
Image Image
User avatar
takyoji
Legendary
Legendary
Posts: 530
Joined: Tue Nov 27, 2007 4:46 pm
Gender: Male
Mood: Busy
Location: Minnesotan, dontcha' know!
Contact:

Post by takyoji »

As you notice the forum wasn't upgraded yet because I was having hell last night and ran out of time. Last night I got to have a series of the following happen to me (of bad luck):
  • My own internet connection completely dying for 30 minutes right before I was ready to upload the converted data
  • The connection being lost 3 times while I was trying to upload the converted data (considering I'd wait like 15 minutes and it'd be about half-way done then the connection would be lost)
  • Accidentally re-enabling the forum thinking it was the forum on my test server thus I got all stressed out thinking someone else re-enabled the real forum when someone told me the forum was accessible again.
  • Figuring out that I didn't have the programs that was running the conversion process at a high priority considering that for some reason it would convert like 7 users per 5 minutes for some reason (considering we have like 1,600) then I finally realized it and it sped up to 400 users per 5 minutes for the conversion rate after prioritizing it higher (even though it was running a lot quicker earlier for some reason).
  • Ending up giving up at 4:30 am while I was trying to upload the converted data and finally going to sleep.
I've spent like 6 hours overall on manually converting our current forum's layout manually by hand. Thus 13 hours and 30 minutes of effort put into this whole process in one day. But anyway I'll work on it more tonight.

But otherwise uploading the converted database was very last thing considering EVERYTHING else was completed just fine and that was only the remaining part, so if it wasn't hell trying to upload it with the connection dropping several times our new phpBB 3 installation would finally be ready.
Avatar original image by "sometimesong" of Flickr, released under the Creative Commons license (share-alike, attribution, non-commercial, no-derivatives)
User avatar
outwarddoodles
Moderator
Moderator
Posts: 2670
Joined: Sat Apr 02, 2005 11:49 am
Custom Title: I'm here! What more do you want?
Gender: Female
Location: Ohio
Contact:

Post by outwarddoodles »

Oooh, exciting.

Anyone up for some PackChat on the official night of the conversion? I'll be there.

Also, to Takyoji: Thanks for the hard work you're putting into these forums. It's really appreciated!!
:D
Spongy
Legendary
Legendary
Posts: 829
Joined: Tue Feb 13, 2007 10:04 pm
Gender: Male
Mood: Indifferent
Location: The Moon

Post by Spongy »

Poor Tak. He's really having a hard time with this :P
Image
User avatar
Xiroteus
Legendary
Legendary
Posts: 1052
Joined: Sat Jul 01, 2006 10:24 pm
Gender: Male
Mood: Happy
Location: United States
Contact:

Post by Xiroteus »

You are putting a lot of hours into this.
User avatar
takyoji
Legendary
Legendary
Posts: 530
Joined: Tue Nov 27, 2007 4:46 pm
Gender: Male
Mood: Busy
Location: Minnesotan, dontcha' know!
Contact:

Re: Planned downtime

Post by takyoji »

Well, now it's all finally converted (or at least for the most part). Apparently it looks like a couple avatars are missing, but I'll work on that as well (SO DON'T RE-UPLOAD YOUR AVATAR, it'll be fixed). And I guess I'll have to transfer the birthdates and genders as well. Otherwise for the whole conversion process, it took 19 hours and 40 minutes in grand total. I've had a total of 7 problems I've overcome. I've written 2 little programs to fix those problems (one for transferring custom titles since it was a mod, and one for getting the proper sizes for avatars over 150px*150px since it made those appear smaller). I did this all in 1.5 entire days (one from 8am to 4am, another from 7pm to 5am). And yet I still have my sanity... or at least of what I notice...

Also, there's obviously a lot of new features, thus feel free to ask questions on how to use them if required.

*finally jumps in bed, realizes it's 5am, hears the birds chirping outside, but attempts to sleep anyway*
Avatar original image by "sometimesong" of Flickr, released under the Creative Commons license (share-alike, attribution, non-commercial, no-derivatives)
Sioux
Legendary
Legendary
Posts: 96
Joined: Fri Jun 13, 2008 12:01 pm
Custom Title: A life does not have to be human to be great.

Re: Planned downtime

Post by Sioux »

takyoji wrote:Apparently it looks like a couple avatars are missing, but I'll work on that as well (SO DON'T RE-UPLOAD YOUR AVATAR, it'll be fixed).


Thank you so much for doing all this takyoji. We all know it took a lot of your time and your hard work is greatly appreciated. :D
Don't worry about the avatars. I'm sure those that lost them(me included) will wait patiently for you to get them fix when you have the time.
User avatar
outwarddoodles
Moderator
Moderator
Posts: 2670
Joined: Sat Apr 02, 2005 11:49 am
Custom Title: I'm here! What more do you want?
Gender: Female
Location: Ohio
Contact:

Re: Planned downtime

Post by outwarddoodles »

Great job on the layout.

I had to take a double take when I logged on to notice the difference on the forums.
"We are not always what we seem, and hardly ever what we dream."
Kzinistzerg
Legendary
Legendary
Posts: 2335
Joined: Thu Feb 03, 2005 6:28 pm

Re: Planned downtime

Post by Kzinistzerg »

I noticed the new layout as well, and it looks nice! You might want to make us some sub-forums, however (not make, per say, just move them. Maybe).
KittyRose
Legendary
Legendary
Posts: 1254
Joined: Sun Dec 24, 2006 5:01 pm
Custom Title: The Spotted Feline Writer
Location: Here, There, and Everywhere
Contact:

Re: Planned downtime

Post by KittyRose »

I just noticed the new layout (I needed to do a double take as well) and it looks great.
DeviantART
Meez

:wolfpaint: Avatar created by Z
User avatar
IndianaJones
Legendary
Legendary
Posts: 528
Joined: Wed Sep 05, 2007 11:08 pm
Custom Title: Anime/Furry/Disney Fanatic
Gender: Male
Additional Details: I like transformations and humans being turned into animals, furries, and monsters.
Mood: Relief
Location: USA
Contact:

Re: Planned downtime

Post by IndianaJones »

It looks very slick, I like it. Much better than the old one.
Disney/Disneyland fans and theme parks!
http://micechat.com/

http://steamcommunity.com/id/ZanderFox/
Shadow Wulf
Site Admin
Site Admin
Posts: 7572
Joined: Thu Jun 23, 2005 3:17 pm
Location: Zephyrhills, Florida
Contact:

Re: Planned downtime

Post by Shadow Wulf »

Looks really good so far, takes a little getting use to but I like all the new add ons phpBB3 has. :D
Every government degenerates when trusted to the rulers of the people alone. The people themselves are its only safe depositories. - Thomas Jefferson
Image Image
User avatar
takyoji
Legendary
Legendary
Posts: 530
Joined: Tue Nov 27, 2007 4:46 pm
Gender: Male
Mood: Busy
Location: Minnesotan, dontcha' know!
Contact:

Re: Planned downtime

Post by takyoji »

KittyRose wrote:I just noticed the new layout (I needed to do a double take as well) and it looks great.
Actually it was supposed to look as close to the current layout at possible (but I didn't really hesitate to get every little page exactly how it is), also this isn't really the new layout. I just figured I'd go ahead and just upgrade the forum with the current design and put the new one on later.
Kzinistzerg wrote:I noticed the new layout as well, and it looks nice! You might want to make us some sub-forums, however (not make, per say, just move them. Maybe).
That's one of the things I was considering. We could probably even go into deeper categorization (if sane) so that people could subscribe to those forums and receive notices of new topics in those specific categories.
IndianaJones wrote:It looks very slick, I like it. Much better than the old one.
Nah, it's just the same IMO. xP
Shadow Wulf wrote:Looks really good so far, takes a little getting use to but I like all the new add ons phpBB3 has. :D
Now we'll actually be able to have the basic moderation features built in instead of being modifications (considering that the merge feature only worked on a specific theme). But yea, finally there's a handful of more things we can do with it as well.


As for those who are curious how the whole process went, it went like so:
  • Installed phpBB 3 on my test server
  • Downloaded the current layout from the phpBB 2 installation.
  • Spent about 6-9 hours modifying this layout to look like the current layout.
  • Downloaded all the files from the old forum (6,340 files, one-by-one)
  • Disabled the forum
  • Backed up and downloaded the whole database (160MB file containing every post (190,000 posts), topic, user, private message, etc)
  • Imported the database on my test server (which took probably 30 minutes to calculate the whole thing)
  • Ran the converter on the phpBB 3 installation on my server which probably took another hour or so perhaps (to convert all the posts, topics, users, avatars, etc). During the process my internet connection completely died (and the forum was probably disabled at the time as well) so I was freaking out that the forum would be disabled until my connection comes back at about 2:30am, which it luckily did come back 30 minutes later. When the connection came back someone told me that the forum was apparently re-enabled, so I thought some other admin re-enabled it even though I announced it would be disabled for the upgrade (but later realized that night that I accidentally re-enabled the live forum rather than the one on my test server).
  • Wrote a program to convert the custom titles from the old forum to the new forum manually and ran it (since it was a modification and wasn't built into phpBB at all).
  • Uploaded the converted files of the phpBB 3 installation onto The Packs Den.
  • Tried to upload a 131MB file of the converted database onto the forum. About 40% through it (which was like half an hour) the connection dropped and it started over from the beginning again. Then it dropped again, then tried uploading it again, but it dropped AGAIN. Then I split up the file into smaller parts, and just the same problem occurred. So I gave up for the night at probably 4:30am.
  • The next day I tried a different method which was by connecting directly to the server's database and having it insert each entry one-by-one (much slower process) and finally got it on the server after probably 4 hours or more (would have taken just 35 minutes if it could have been uploaded as one file, only if it wouldn't have kept losing the connection).
  • Realized that all avatars over the size of 150x150 were all being displayed with a size of 150x150 which couldn't be reverted. So then I wrote another program which checked all the actual sizes of the avatars and fixed the values in the database.
  • Made ANOTHER phpBB 3 installation on my test server.
  • Disabled the live forum again, but this time I only backed up posts, topics, private messages, and other things that were made after I backed it up earlier. But as I was backing up apparently there was a bug in the software that I was using the prevented me from getting some things the easy way, so I had to back up some things by hand. Then I finally converted those posts.
  • Started putting those newly converted posts on the server successfully.
  • Switched The Packs Den to the new forum and went to sleep at 5am.
  • Next day I realized a lot of the avatars were missing, checked on my computer for them, they weren't there apparently. Then I checked on the old forum on The Packs Den and there they were. So I wrote a script to list all the avatars the were missing and probably 60 avatars were missing. So I realized my stupid FTP client didn't copy ALL of the avatars as it apparently just didn't download some of them. Thus I downloaded the whole avatars folder again twice to make sure there were none missing this time. Then I wrote another program which converted those avatars into the phpBB 3 format and uploaded those. And that leads to the point of where I started writing this post. xP
If we didn't do the careful way (which is where it's all converted on a different computer rather than on live server) and if we didn't have the custom titles modification and all that, then the process would have simply been:
  • Remake the current layout for phpBB 3.
  • Upload phpBB 3 to the live website.
  • Start the converter and disable the old forum when the converter is near complete (disabled for probably 15 minutes).
  • Switch around the old forum with the new one.
  • Fix the avatar size of avatars over 150x150 (which would have needed a custom written script to do easily)
which could have probably taken 4 hours instead of 18 steps which involved writing probably 4 programs, bugs with software, connection problems, and like 13 hours of manual work. But hey, at least we did it the safe way which involved all that lovely stress (and note: there was no guides or anyone helping me to get around all these problems and to write these basic programs). :lol:
Avatar original image by "sometimesong" of Flickr, released under the Creative Commons license (share-alike, attribution, non-commercial, no-derivatives)
User avatar
outwarddoodles
Moderator
Moderator
Posts: 2670
Joined: Sat Apr 02, 2005 11:49 am
Custom Title: I'm here! What more do you want?
Gender: Female
Location: Ohio
Contact:

Re: Planned downtime

Post by outwarddoodles »

takyoji wrote:
KittyRose wrote:I just noticed the new layout (I needed to do a double take as well) and it looks great.
Actually it was supposed to look as close to the current layout at possible (but I didn't really hesitate to get every little page exactly how it is), also this isn't really the new layout. I just figured I'd go ahead and just upgrade the forum with the current design and put the new one on later.
Which is exactly why I was amazed at such a good job making the new one look exactly like the old -- to a point where I had to do a double take. I'd rather stick with the old layout and design specifically so I don't have to get used to a new one -- I think this design is beautiful.

Oh, by the way, for suggestions: I believe in the past we had a limit towards the size of pictures in signatures. I wont say names, but the limit has obviously been exceded by a couple members, and it's rather annoying. Is there a chance you could script something so large pictures in signatures will not appear?
"We are not always what we seem, and hardly ever what we dream."
User avatar
WerewolfKeeper3
Legendary
Legendary
Posts: 851
Joined: Wed May 02, 2007 1:01 pm
Custom Title: Darkness Surrounds me, Chaos commands me, and i see what others cannot
Gender: Male
Mood: Relief
Location: Somewhere...
Contact:

Re: Planned downtime

Post by WerewolfKeeper3 »

So that's what happened... cool... {Bow to the great and might Takyoji} Seriously, you need the kudos for doing all this... and my avatar didn't disappear either... cool! Hey, i can see more smilies too... hmm... this is gonna interesting...
No what you have are bullets in the hope that when your guns are empty I'm no longer standing. Because if I am, you'll all be before you've reloaded.
V, from V for Vendetta.

What a strange creature is man, that he cages himself so willingly?
-Athena from Appleseed (2004)
Set
Legendary
Legendary
Posts: 3236
Joined: Sun Dec 19, 2004 5:34 pm
Custom Title: Devil in disguise
Gender: Male

Re: Planned downtime

Post by Set »

Hm. *pokes at new options* Enable word censoring, Y/N? ...Does this mean I can actually see the crap I type now? :lol: :grinp:
Post Reply