Our (mt) Media Temple fiasco

Yesterday, our site inboundpass.com suddenly went down. What looked like a simple problem solving issue, we’ll-quickly-bring-it-back-up by the Media Temple staff rendered our very busy site inaccessible for almost 5 freaking hours. Media Temple, you screwed us up big time. And also my other websites are still down for the past 13 hours.

So to dispel the rumors, no our site was not hacked. It was a stupid admin bubu.

This is a very bad time to go down. When thousands of people are looking at the site. This season has caused our server to creak under pressure and we’ve seen more than 700% increase in site traffic. Woah! (This is reserved for another blog post.)

So what happened?

  • At September 30, 10:42 PM (PDT) That’s approximately 1:42PM (+0800) Manila Time

REQUEST ID #449907 / STATUS: In Progress
downgrade from MySQL Grid Container

David Quitoriano (Administrative Contact)

Hi (mt), Thank you for the letting us try the MySQL Container for a few days during this spike of traffic.

We are looking at other options and will not not be continuing the MySQL Container Lite.

Please DO NOT upgrade automatically. Thanks.
@
2008.09.30 10:42 PM

So far so good.

  • At October 1, 4:25AM (PDT) (Approx. 7:25PM (+0800) Manila Time) I got relatively prompt response from Mike.

    In regards to your inquiry,

    The burst container process is a automated process and can not be manually controlled or edited for one specific client. As long as you are not using a tremendous amount the MySQL container you should not have any problems. Please keep in mind that if you get bursted three times you will be required to purchase a MySQL container.

    If you have any further questions regarding your (mt) Media Temple services, please feel free to contact us at any time.

    Best Regards,

    Mike Muscet
    Customer Support
    Check out the New User Forums: http://kb.mediatemple.net/questions/824/
    (mt) Media Temple
    <v> 877-578-4000
    <f> 310-564-2007
    @
    2008.10.01 04:25 AM

Still good.

  • The next day I was irked to received this email–

    On Oct 2, 2008 at 3:05 PM, (mt) Media Temple <no-reply@mediatemple.net> wrote:

    Dear David,

    Your new MySQL GridContainer Lite (srv# 289172) has been successfully activated and is now available for use with your (gs) Grid-Server (coloredcompany.com). All of your MySQL databases have automatically been moved into your new GridContainer environment. Your databases are now being served exclusively from within your MySQL GridContainer. No changes to your applications are needed to work inside the MySQL GridContainer. Your databases can still be managed the same way as before directly from inside the AccountCenter or using remote tools. In addition new tools which give you greater control and insight into your database have been added to the MySQL Container tab inside your AccountCenter Database Management Controls.

    If you require the assistance of a support agent, please start by opening a request inside the Account Center: https://accountcenter.mediatemple.net/

    Your request will be routed to the appropriate agent or group on our team.

    Best Regards,
    (mt) Media Temple
    Hosting Operations
    @
    2008.10.02 03:36 AM

Very good, just as I thought what would happen.

Now to lend you a piece of my mind. Media Temple was courteous to support us in our spike of traffic during the UAAP & NCAA Championships. They “temporarily” placed us on the MySQL Grid Contrainer Lite which costs monthly at $20. So all in all our monthly hosting at Media Temple would increase to $20 (gs) + $20 (Grid Container) for a total of $40.

Now at that rate why not get a (dv) Dedicated-Virtual $50 or a Slicehost at $38 a month. Gimme time to think it through but don’t pressure sell me.

So what ensued was a string of message exchanges to supposedly cancel this arrangement.

  • I sent a reply, David Quitoriano (Administrative Contact)

Ok then un-automate it.

I received this already. Thanks! (referring to the automated email I received which I copied below for their reference.)

~~~~
On Thu, Oct 2, 2008 at 3:05 PM, (mt) Media Temple <no-reply@mediatemple.net> wrote:

Dear David,

Your new MySQL GridContainer Lite (srv# 289172) has been successfully activated and is now available for use with your (gs) Grid-Server (coloredcompany.com). All of your MySQL databases have automatically been moved into your new GridContainer environment. Your databases are now being served exclusively from within your MySQL GridContainer. No changes to your applications are needed to work inside the MySQL GridContainer. Your databases can still be managed the same

Yada. Yada. Yada.

~~~~

  • Now for the kicker replies:

Grace Hildebrand (mt) Media Temple Customer Support

There is no way to un-automate the burst container, it cannot be manually controlled, as what the previous technician stated before.

Best Regards,
Grace
@
2008.10.02 04:03 AM

~~~~

David Quitoriano (Administrative Contact)

Ok Grace.

Let me be very clear. We will NOT purchase an additional MySQL GridContainer Lite Container (srv# 289172).

So it’s an automatic process. Nevertheless, we will decline in getting an additional MySQL GridContainer Lite Container.

Cancel Service Order # 289172.

Thank you.
@
2008.10.02 05:04 AM

~~~~

Grace Hildebrand (mt) Media Temple Customer Support

Order number: 289172 has been canceled per your request.

Best Regards,

Grace

@
2008.10.02 05:16 AM

~~~~

BOOM!

At this point tragically, inboundpass.com went down at a blink of an eye. Cold sweat on my brow. A growling WT*? tear through the silence of the premises.

Hello? Are you mad? Are you our of your mind? I guess, YES!

In frustration some more exchanges:

  • I sent this reply

my website is now DOWN. http://inboundpass.com

thank you very much
@
2008.10.02 05:27 AM

~~~~
Grace Hildebrand (mt) Media Temple Customer Support

You requested the cancellation, I did it for you.

If you need the service back, I can add it back.

Best Regards,
Grace
@
2008.10.02 05:35 AM

~~~~
David Quitoriano (Administrative Contact)

I cancelled the MySQL GridContainer Lite. If a service will go offline you should have given a heads up. I don’t know the inner workings of (mt).

1. I want http://inboundpass.com up immediately.
2. We would like the default MySQL that’s included with (dv) Hosting. <- I meant (gs) hosting
3. Cancel the MySQL GridContainer Lite – Service Order # 289172

Please.
@
2008.10.02 05:54 AM

~~~~
Drew Schatt (mt) Media Temple Customer Support

The database is now reachable via the phpmyadmin page on your account, so all of your sites should be working.

If you have any further questions regarding your (mt) Media Temple services, please feel free to contact us at any time.

Best Regards,
Drew Schatt
@
2008.10.02 09:10 AM

~~~~

Thank you. inboundpass is now up. We were down for almost 5 hours by my estimates. My other sites are still down.

~~~~

David Quitoriano (Administrative Contact)

Hi Drew,

Thank you for bringing up http://www.inboundpass.com. Our main site went down for more than 4 hours.

All my other sites are down as well, please revert the databases :
http://www.air21express.com.ph
http://dev.inboundpass.com
http://h2orepublic.com/index.php

http://livewire.ph
http://pmxaudio.com
http://staluciarealtors.com

(site list trimmed)

Please fix these asap to reverse this fiasco 😦
@
2008.10.02 01:57 PM

As of this writing, I hope they’re resolving the issues. My other sites are still down.

(mt) Media Temple is NOT great!

9 responses

5 10 2008
Spikeithard

Very funny and interesting exchange. At one point in time, we really did think about getting hosting from (mt) as they are a popular choice for web design companies. Good thing we spend almost a week of researching here and there. And learned that although they are one of the cheap and reliable hosting providers, they indeed lack in the customer service aspect.

(Nice meeting you guys over at the Dayo: Bloggers’ Meetup. More power to InboundPass!!!)

6 10 2008
Dave Q

Hi Marc,

Though I’d still recommend them (mt) for your hosting needs, technical know-how is still your best protection from booboo’s like these.

For the record, we’ve been happy with (mt) up until now. I’m hoping this will be the last major error.

cheers!

31 01 2009
Legal Hallucinogens

Hi Marc,

Though I’d still recommend them (mt) for your hosting needs, technical know-how is still your best protection from booboo’s like these.

For the record, we’ve been happy with (mt) up until now. I’m hoping this will be the last major error.

cheers!

25 09 2009
Andrew

You are complaining about $40 per month. If you website does not make enough money to cover that you should shut it down. You are lucky they do the move automatically. If you were on godaddy, they would have told you to move it your self. Good luck with that.

8 11 2010
Former MT employee

I was a former Media Temple Employee, and I just wanted to give you the scoop on the Company. First off, when techs get off the phone with you, they will turn to the person next to them, and say what a moron that person was. They do that A lot.

HERE IS ANOTHER BIG SECRET: They are not worth shit. Sure, they act bigger than they are, but it’s still a small call center in the middle of Culver City run by a bunch of spoiled brat retards, who shop at IKEA.

They boast they have the greatest team. I know their System Administrators are so coked up and full of speed, they fuck shit up. If you’re gay, you’ll get promoted, if not, you’re stuck in phone purgatory.

Spread the word, on how fucked up this company is.

19 10 2011
Please tell me you're joking...

You’re mad because they did something you asked them to do?

25 01 2012
high expectations

You emailed them insisting they turn off a service. They warn you it’s a bad idea. You tell them to do it anyway. It breaks your site. You contact them and they fix the problem within 5 hours. And you’re paying $20 a month for this service. And you’re pissed? I don’t get it…

25 01 2012
inboundpass

Hey guys this post is old and we’ve gone a long way from our cheap hosting. We are now 10x bigger riding on an army of VPSes. Thanks for your comment guys!

27 01 2012
high expectations

I noticed the date of the post after I submitted the comment. Congrats on your growth!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s




%d bloggers like this: