It’s been a VERY stressful past couple of weeks for us here, but I think/hope the end of the server problems are finally here!

/me knocks on wood

We’ve certainly had our share of technology related issues over the years, just like most highly trafficked websites. Historically we’ve bounced back from any glitches rather quickly. This time around however the root cause of the problem was especially hard to track down, which made it that much harder to fix, and that much more painful/stressful.

Technology problems are very much like human problems. When your body aches, you go to a doctor and s/he asks you where it hurts. S/he then uses that information and a series of test to diagnose the problem, and from there provide a solution. That same process usually works with technology. When it breaks, it spits out an error message telling you the problem, you then run a series of tests to replicate the problem, and you use that information to provide a solution. Everyone lives happily ever after.

Well this time around the errors we were receiving from the server we’re very obscure, and seemingly random, with very little information. So naturally that made tracking down the cause EXTREMELY difficult. It wasn’t until two days ago that I was FINALLY able to pinpoint the problem, and more importantly replicate it. From that point it took less than 48 hours to get a patch from a 3rd party software vendor that we use to render images on the website. I verified the patch worked on a handful of test cases, and then deployed the patch to the live server. So assuming the patch works we should be all fixed, which I’m sure is a huge relief to everyone!

We’ll still be monitoring the server very closely over the next couple of days/weeks to make 100% sure there is no sign of this particular bug rearing it’s ugly head again! And if you see anything odd, please don’t hesitate to let us know!

Thanks again for all of you that reported the errors you were noticing to us. Believe it or not, it’s a huge help as it augments our own internal reporting tools. I know how frustrating it must have been for you when the site wasn’t working as expected, so for that I apologize. I can only offer my handful of gray hairs that I yanked out as proof that it was at least equally frustrating for me, if not more so! :)

We’ll also be extending all premium memberships free of charge for a full extra week as a good gesture to account for the sites downtime. I hope it can offer some sort of condolence, along with our appreciation of your patience and understanding! :)

10 COMMENTS

Leave A Comment
  • 1
    Cindy Sweet Says: August 10, 2010 at 2:04 am

    I used the time to write blogs and edit some pics, we all know you guys work so hard to make this site all that it is so a little website downtime was a small sacrifice…

    ……..sure a lot of us were suffering hot & cold flushes, shaky hands and severe signs of withdrawal…. but we’re feeling much better now thanks :P

  • 2
    Katie Says: August 10, 2010 at 4:17 am

    That was like double dutch to me Jason all this software malarky but I think I got the most part of it haha! Glad to see MFI’s on track :D I had to deal with ‘real life’ for a few hours when it took the bad turn haha!

    That’s pretty cool your doing that for the guys I’m sure they’ll appreciate it!

    Long live mfi :P

  • 3
    Judy Says: August 10, 2010 at 4:19 am

    The site being down gave me time to go to work. So it’s a win for me. LOL.. honestly though in the bigger picture viewer I use, I think the downtime was rather short in comparison to other sites. You all work hard at keeping us happy and it’s funny to read all the complaining blogs. Realizing how much MFI has impacted our lives that we can’t live without it for even a few hours much less a few weeks or so. I think this says something for how much this site really brings people together and making dreams come true.
    THANK YOU ALL SO VERY MUCH for keeping up on things and fixing the problems as quickly as possible all the while keeping us informed of how things are progressing.
    {{{HUGS}}} niceswty

  • 4
    ChrisUK Says: August 10, 2010 at 4:36 am

    I had no idea how long the site was down. As soon as i noticed, i developed shaking and sweats, then went into shock. Luckily it was back by the time i recovered.

    Phew … MFI withdrawal … giving up drinking would be easier!!

    Good work Jason.

  • 5
    Haley Says: August 10, 2010 at 7:17 am

    Wow Im so happy we’re back to normal I was not liking having nothing to do – AND having panick attacks LOL. Luckily when it went down a few times my chats were stil working – happy days LOL
    Thanks for fixing it :D
    Now I can breathe again!!!
    PHEW

  • 6
    Si. Says: August 10, 2010 at 1:40 pm

    Sums up the site these days for me… My “paid” subscription only expired a couple of days ago (which was cancelled for personal reasons) yet im not offered the same extra week of full membership like the others here.. yet most of the problems with the site occurred during my PAID subscription

    thanks guys nice work!!

  • 8
    Dan S Says: August 12, 2010 at 9:33 pm

    What Si. said.. mine expired only expired on the 8th… no extra week here.. sent support an email…

  • 9
    Jeffrey Says: August 29, 2010 at 8:09 pm

    It was painful for me. Wasn’t able to login to my account. Had a member I known for two months and she left me a message probably to tell me she was leaving MFI. But I didn’t get the message because 4 hours after I was able to get on she had deleted her account and her message was gone. I was left shock, sad and confused why she left. I will never have the answers to why she left. I’m upset by the whole experience I had with the site issues.

  • 10
    Jenna Says: September 01, 2010 at 6:30 pm

    I wonder if this particular issue is at all related to the issue today(September 01st)If so, I’d get on the phone with that third party software company and ask for a refund:)…although I am sure it doesn’t work that way:(