Not at all how I imagined it...
Well, I planned on coming home from work, relaxing for the majority of
the evening while enjoying my shows...aparently the stars had something
else in mind for me. I spent my entire night busting my hump
getting our database server back online. Aparently there was some
sort of power glitch at the data center we use for our production
servers and when the hiccup occured, the battery backups were supposed
to kick in, but instead they either caught fire or just died (got two
different stories) and all 3000 servers in that center went down for
the count. Because the systems were down, our notification system
didn't work so we had no idea what happened for approximately 30
minutes. When things started coming back online we were getting a
picture of what happend..and it didn't look good. Our database
server wasn't coming up and after repeated phone attempts to get the
tech on site to press the power button, I went down to see what I could
do. I got there only to find that the drive controller that
controls the C: drive died and the machine wouldn't be
rebooting. So that new machine that we wanted a few days to test
to see how well it worked...yeah..that's going live as I write
this. Our DBA is restoring the data from the backups and I've got
everything ready for the new machine. I'm just waiting for his
call so I can make it all live again..then go to bed!
Comments
No comments yet — be the first.