Engineering Leader · Private Pilot · Dad · PSU Hockey
OAT 72° XPDR 1200 GS 145 18:42Z
10
20
10
20
160150140130120110100
138
32003100300029002800
3,000
306090120150
90
My Life
Bio & background
PROFILE · SET
My Computers
Current rigs
UPTIME · 100%
My Projects
What I'm building
STATUS · ACTIVE
Résumé
Curriculum vitae
FORMAT · HTML

Not at all how I imagined it...

11/30/04 @ 4:11 am · By Steve Hoffman · Comments (0) · Leave one
Well, I planned on coming home from work, relaxing for the majority of the evening while enjoying my shows...aparently the stars had something else in mind for me.  I spent my entire night busting my hump getting our database server back online.  Aparently there was some sort of power glitch at the data center we use for our production servers and when the hiccup occured, the battery backups were supposed to kick in, but instead they either caught fire or just died (got two different stories) and all 3000 servers in that center went down for the count.  Because the systems were down, our notification system didn't work so we had no idea what happened for approximately 30 minutes.  When things started coming back online we were getting a picture of what happend..and it didn't look good.  Our database server wasn't coming up and after repeated phone attempts to get the tech on site to press the power button, I went down to see what I could do.  I got there only to find that the drive controller that controls the C: drive died and the machine wouldn't be rebooting.  So that new machine that we wanted a few days to test to see how well it worked...yeah..that's going live as I write this.  Our DBA is restoring the data from the backups and I've got everything ready for the new machine.  I'm just waiting for his call so I can make it all live again..then go to bed!

Comments

No comments yet — be the first.