It’s been over month since I posted my original update that the forums would be returning in “days, not weeks”. Unfortunately, as everyone has noticed, I made that statement prematurely. We’ve since encountered a number of issues that have roadblocked our restore of the forums. I want to talk about that process here so that everyone understands what we’ve been going through over the last month.

As you may know, we are transitioning away from vBulletin, our message board software, due to issues with its aging infrastructure. There are numerous problems with vBulletin that we just can’t solve, which caused us to take the forums down temporarily. Meanwhile, we had been working for many months on a new version of the ZU forums running on new, modern software with a fresh design inspired by our major redesign last year. We decided it was better to launch this new version of the forums in beta than attempt to rewrite vBulletin ourselves.

However, along the way, we have encountered numerous problems in the data import process. The ZU forums are massive: over 10GB in size. The import also involves converting the data from vBulletin format into the format for our new message board software. What this means is that each full import of the ZU forums takes roughly four days, uninterrupted.

If the import is interrupted for any reason, we have to start the process all over again. Because the import process is extremely sensitive (all posts, threads and more must pass through completely intact), hiccups can occur during the import process and cause us to have to start over. Sometimes these hiccups are merely annoying, but in other cases we’ve had to develop solutions to real issues that we did not anticipate. This has happened several times, which takes us right about to today.

We hope that the issue we’re currently solving will be the last, but only time will tell. Make no mistake: we’re working literally 24/7 to get the forums back up, with multiple team members around the globe in different time zones taking shifts to watch the status of the imports and provide immediate solutions if necessary. This is the absolute highest priority inside ZU staff right now.

Since we don’t know how long it will take to run a successful import (although we obviously hope this is the last issue we’ll be dealing with), we’re working on a temporary solution where everyone in the community can go until the forums are back online. I’ll make another post when that’s up as well.

Thank you, everyone, so much for being so incredibly patient. You are truly the lifeblood of ZU, and the heart of the Zelda community. Even though it’s taken inexcusably longer than expected, I still think you’ll agree it’s worth the wait. In the meantime, I’ll be answering any questions you have in the comments below.

~ Jason Rappaport.

  • Tuf Pic

    Best of Luck getting the forums back online!! πŸ˜‰

  • Concerned Citizen

    Will new ZU have social groups? Will the threads from existing social groups just be gone forever? Thanks.

    • The new forums will NOT have social groups. For as many cool new features as we are adding, we are also focusing the forums on being just that – great message boards. As such, “extra” features like social groups and user-specific image galleries are being eliminated.

      However, we do have copies of the social group data, so there’s always the possibility that we do something cool with it in the future – either within or outside the forums. So I won’t say they’re gone for good! At the very least, there’s a strong chance we find a way to put them up eventually for archival purposes.

  • MJ

    Thank you for another update. Much appreciated. πŸ™‚

  • Wolf Deity

    Wait the forums are only 10 GB?
    Is there something I’m not getting here?
    Also is the import the last step or are there more steps afterwords.

    • That’s 10 GB of pure text. Imagine a word document that size! It’s an almost mind-blowing amount of text. The import also has to “read” all of this text, which is what takes time.

      The import isn’t the last step, but it is the longest step and the one that is most out of our control. After that, it’s just cleanup and final preparations for launch.

      • Wolf Deity

        are you not importing Images and the like?

        • Temis

          Why would they? All the images on the forums are really just the urls of images (so, text), and the images they lead to end up being put on to the page you’re viewing

          • While this is true, we did at one time support attachments. We are absolutely porting these attachments over, regardless of how old they are.

          • disqus_Tommy

            Thoroughness is great, I’m very excited to see what your team has in store for us and the fact you’re taking time to do this properly and not just half-assed. While I really loved vBulletin that you used because I absolutely cannot stand version 4+, I’m very excited to see what your team has come up with in line of software itself.

          • disqus_Tommy

            Also a question, you said there were problems with vB that you just couldn’t fix anymore. What specifically was causing problems that was unfixable? Was the software the cause of forum slowdowns or was that hardware related?

        • We are, in fact, importing images! There are legacy attachments from almost a decade ago that we intend on preserving. I should add that the images are much easier to deal with than the text, too, and don’t factor in to the 10GB size I quoted earlier. Their total size is relatively small, though – we never supported big attachments, or attachments for very long in general.

        • I should add that the images are much easier to deal with than the text, too, and don’t factor in to the 10GB size I quoted earlier. Their total size is relatively small, though – we never supported big attachments, or attachments for very long in general.

      • We are, in fact, importing images! There are legacy attachments from almost a decade ago that we fully intend on preserving.

  • MJ

    What will the temporary community have?

    • We’re not sure yet! But it will be an easy-to-access location that doesn’t require you to register. It’ll just keep the community going for a short while, it’s nothing we want to have around forever. You’ll know more about it when we do!

  • The Fanatic of Zelda

    The new forums are NOT Discourse, right?

    (Be aware that I will consider anything other than a clear “No” as “Yes”)

    • The new forums are not Discourse! We rejected Discourse a long time ago because it was too different from what folks expect of a message board. Most of us on staff still don’t “get” Discourse, though we respect the team greatly and admire their accomplishments with Stack Overflow.

      Although I won’t say what software we’re using specifically (because it doesn’t matter), it is definitely not anything you’ve ever heard of or anything anyone else is or will be using. We did an exhaustive technology overview of every message board software out there and chose one based on our specific plans for the forums over the next several years. That means you’ll get a great forum from Day 1, but later on is going to blow your head off. We’ve thought VERY long-term about this, longer than I think most fan sites have thought about any project they’ve ever done. For us, the software that powers our community may as well be our community.

      If we’ve done our job right, you won’t really notice the software. It’ll just be a great place to have the community.

      • The Fanatic of Zelda

        Well, that’s a relief! And I’m excited to see what the new forums will be like. Definitely keep the surprise there.

        I guess this means I can also rule out IP.Board, SMF, XenForo, MyBB, PHPBB and other popular software, right?

        • Yup, it’s none o’ those. And that’s the last I’ll say about it!

        • disqus_Tommy

          I personally liked MyBB somewhat, it was a stripped down version of vBulletin. I haven’t tried it since their big changeover, at least I heard at one time they were really making massive changes to the software. But honestly, I would’ve been happy with something like that. But now since I’m only familiar with a handful of what you mentioned, I’m curious to see what it honestly could be. Maybe they somehow wrote their own software like InvisionFree did with ZetaBoards.

      • Flueworks

        But will the new forum support Markdown in any way, as opposed to the antique and horrible BBCode?

  • Temis

    I just thought I’d point out that THE ENTIRETY OF WIKIPEDIA takes up 42 GB! That means if ZU is 10 GB, then that’s almost One fourth of the size of Wikipedia!!

    • 17Haru17 .

      That’s a wonder. Wikipedia must have much better compression, or some such technological tomfoolery. At any case I’m pretty sure the forums will be up before we get any new Zelda U news ;P

      • Temis

        That’s actually 42 GB uncompressed. Compressed it’s like 9 or something I think. But the 10GB size of the forums is also uncompressed.

  • Tuf Pic

    Sure hope we can still keep our friendships, & hopefully make new 1’s, & I hope we DO get Social Groups back, in some form!!

  • Sageofsarcasm

    Thanks for the update, Jason.

  • Charley Pope

    Looking forward to the forums coming back. I joined a while ago, but have really gotten into Zelda a lot more since then. Hopefully it comes sooner then later. πŸ™‚

  • Green Cloud

    Great stuff. I’ve been checking daily lmao. Thanks for the update :).

  • Rotfl Master

    I’ll be checking back to see if the forums are ready in feburary, let’s see if you can finish by then πŸ™‚

  • SBOBET

    nice share, this is very good information for us, thank you SBOBET

  • KeeSomething

    Glad we got another update since I was a little worried the forums were just forgotten about (I should’ve known better). It’s strange to have all this Zelda news, but no ZU forums to gush over it with!

    I look forward to see what the new forum will be like! Hopefully it’ll be up before my 10 anniversary of posting here.

  • The Fanatic of Zelda

    Another random question hit me. Are you going to try and carry over past warnings and infractions?

    • Yes; all moderation decisions and permissions will carry over from the previous forums. Our goal is for the data to make it over as close to identical to the old forums as possible, and we’ve gotten pretty darn close in our tests minus a few manual permissions adjustments we need to make when the import is done.

  • Leo F

    This might be my lack of knowledge showing, but would it not make more sense to just launch ZU with no post history, just transfer accounts over? I certainly wouldn’t be upset at starting new versions of old threads. And then try the full import process on a not-live machine, and merge it when ready?

    Or am I asking a silly question? I don’t know much about running a forum. But I can only imagine that SOME sort of forum is better than none while you wait, in the interests of continuing to get ad revenue and traffic and stuff.

    • Aiko

      For the RPers, this would be terrible, as years of stories and characters and everything else we still use today would be lost.

      • Aiko

        Our ongoing RP threads would be missing until any possible merger as well, and if that wasn’t successful, we’d be out of luck.

    • Data imports don’t really work that way – if we’re going to go through the trouble of figuring out how to import one thing (like accounts), we may as well import everything because it all works the same way.

    • Guest

      As someone who joined ZU in 2007 (Lillia) and for whom past years and posts and rereading them occasionally do mean something, I would definitely prefer that all the history is transferred over- there is so much history between people in ZU, it’s not just a place where people discuss zelda and games but a place where people formed genuine friendships, and even where people ended up marrying (after forming friendships on ZU). I support the transfer of all past posts and threads, there is so so much history here.

  • 17Haru17 .

    It’s kind-of, sorta, extremely ironic that the threads the mods lock in such a draconian manner after only 3 months are what’s causing the hold-up. Gotta save those threads so they can be immediately locked if anyone posts in them -_-

    • Koulamatata

      Personally, I enjoy randomly getting likes on posts I made 8 years ago. Haha.
      That can’t happen if those threads cease to exist!

    • Actually, we found the source of the error: it was some erroneous duplicate thread subscriptions πŸ˜‰ we removed ’em, and the import is on its way again!

      • The Fanatic of Zelda

        Yay! Here’s hoping that’s the last error. We’ll know within a week…

      • Sweet, any chance you could let us know if that happens to be the last error? (you know, in four days or however long)

    • ich Will

      The mods only enforce the rules that the leaders made. But I do agree that some of our practices (including my own take on them) have been too strict.
      Though I do kinda think that as a mod, one should consider taking a step back at times and see if the rules are ok

  • Bob Rakole

    In my personal experience, every forum that has had a crash or has been offline for weeks has lost a chunk of its userbase for some reason. So I’m kinda scared.

    • I imagine we will likely have fewer people when it first comes back, but I imagine the biggest loss will be of the members who only appear for a few months. I highly doubt anyone who has been on ZU for a considerable period of time would be turned away by this.

      Either way, would it really be that horrible for the older users to have time to learn the new software before having to teach newbies?

  • GuineaMane

    I have a question!
    In another post I read that you guys lost a lot of stuff from the past couple of months. Does that mean that my FanFics in the Writing Forum is GONE?! I started writing in August of last year, so if my stories are not on the site anymore then I’m screwed! Please tell me you can recover all the FanFics in the Writing Forum!

    • The Fanatic of Zelda

      Everything between October 2012 and February 2013 was lost.

      If you started posting on August 2014 then you haven’t lost anything.

    • Your stories are just fine!

  • The Fanatic of Zelda

    Yet another question. Will the “random board that appears now then” make the trip too?

    • Being that the board which doesn’t exist is ZU tradition, I assume it would make the trip, as it would otherwise be quite a loss.

    • Silver

      Which board? You must be imagining things.

      • The Fanatic of Zelda

        Very funny.

  • Guest

    The beloved ZU… Don’t worry admins/ webmasters, I personally agree with what’s going on and ensuring the history is all transferred over. It’s evident that you’re working really hard, good luck! -Lillia

  • Ken Iizuka

    Will our clan and current friends list transfer over or do we have to redo that stuff once the forums come back

    • Silver

      Friends lists will transfer over, though they may work slightly differently to how they did on VB. Clans will as well, as I believe usergroups are also being ported.

  • hollander

    Hold on, this is waiting to be approved by Zelda Universe.

  • Contraltissimo

    Soooo I thought of a question that might be kind of silly, but that I feel a somewhat practical need to ask…. Before the forums went down, I’d been composing a somewhat lengthy theory. And in my composition I’d been using the vBulletin codings for things like italics, quotes, images…. Now, I don’t know anything about the DNA of varying internet message boards, but will the new system run on the same codings? Or would now be a good time for me to go through my theory and put it in regular html or whatever else?

    Also, again, thank you thank you THANK YOOOOU AAAAAALL for the blood, sweat and tears you are all pouring into the transfer! Thank you for taking tender loving care of our data!

    • To the best of my knowledge, everything should work since the new forums will use BBCode.

    • We are making sure that all previous BB codes will be compatible with the new forum software. That’s actually the next step once the import is done! We had a lot of custom BB codes that need to be recreated.

  • Zelda Paradox

    I have a question. What about Private Messages between users? Will they be backed as well, or deleted/not backed?

    • Silver

      Private Messages will be saved.

    • Private messages are absolutely transferring over.

      • Zelda Paradox

        Even the ones sent before the transfer?

        • The Fanatic of Zelda

          Well yes, otherwise it wouldn’t be “transferring”.

          • Zelda Paradox

            Ah, I see.

  • Ryano

    Thanks again to Jason and everyone that is working so hard on this update. It’s been tough waiting this long to discuss all the new Zelda stuff with everyone here, but I really appreciate the way you guys are handling the transfer and that you’ve been here answering questions and updating us along the way.

    I gotta ask though…did the latest import work?

  • Pingback: The ZU Forums temporary community is live on Facebook | Zelda Universe()

  • MORE UPDATES! We were able to get the import to run successfully without errors. Now the data needs to convert from vBulletin to our new software. We did encounter an error at this part – but the difficult part of actually getting the software to accept the data is done. The good news is, at this stage we don’t have to restart the entire import when an error pops up.

    The conversion process typically takes two days (it’s half of the total import process). So we’re getting closer, folks!

    • The Fanatic of Zelda

      Yay! So it’s possible the forums will be up by February 1st?

      • Sadly, that I still can’t say because we’re still dealing with errors that cause us to have to restart the conversion. I’ll be able to give a solid date when we have all the data imported and working!

        • The Fanatic of Zelda

          Still, I guess this step won’t take over a month?

          • Again, there’s no way to know because we could encounter errors. These have all been entirely out of our control, so I can’t make any estimations until we’re back in control of our own development!

  • Pingback: Third forum update: import successfully finished, forum return imminent | Zelda Universe()