It's up

Everything looks to be up.
The only known outstanding issue is when banner will be released from functional testing.

Almost done

Waiting for VCL and for migration of a few systems back to datacenter from CTB.

Most things are back up, working on the last few.

Most things are back up.

VCL, luminus dev and QA are still being worked on.

Also working on bringing some redundant components back online, but that will have no user impact

Luminis has been started

Luminis has been started. Having TE-EUC take a look at it to make sure it's clean.

Banner turned over for testing

DBAs have completed banner startup and initial testing. They have turned it over for testing to functional users.

Banner prod has been turned over to DBA

DBAs are working on starting banner production

We are now in the general startup sequence

ESX (vmware) servers are booting

These systems will take a while, they have a long POST cycle.

Compellent seems ok

Compellent seems ok, we are bring up a viprod box to verify LUNs

Having some problems with Compellent

Compellent having some problems. We are on the phone with support

Power is back

TE-PSE and TE-NET are done

Turning the room over to the electricians

Compellent down

Compellent down. Shutting down the SAN directors next. After that we will turn the room over to networking for their shutdown.

Compellent won't shutdown

The Compellent storage array is refusing to perform a clean shutdown.
We have a call in to the vendor.

Email service fixed

Email is operational again.

We think we found an additional dependency that we will need to work on long term.
But for now we have worked around it and email is responding.

Email has stopped working.

We are investigating the issue

Powering down VM infrastructure and SAN disk arrays

Commencing 10pm Procedures

DBA 8pm systems down

Shutting down all the 8pm db and app servers. Commencing copy of gsproddb volumes from the 9985 to the new compellent disk.

8pm shutdowns starting

We are starting our 8pm shutdown procedure. - Jason Berry

DataCenter Status

This blog will be used for internal updates on datacenter status.
This blog is hosted off site and will allow us to post updates even if we have problems.