Status update at 8:00 pm

As of 6:00 pm this evening, Oracle engineers have successfully reconstructed all of the failed components of the disk storage array that has been out of service since early Tuesday morning. They are now in the process of reconnecting these components to make all of them available to the storage array’s controllers. This process willContinue reading “Status update at 8:00 pm”

Major service interruption update at 12:30 pm

As of noon today, the process to export 23 reconstructed volume groups was completed, a failed disk controller was replaced and now dual controllers in the storage array are functioning normally. Five of 18 volume groups needing to be reconstructed have been successfully reconstructed. Recovery procedures are run with due caution and continued success byContinue reading “Major service interruption update at 12:30 pm”

Major service interruption update 7:15 am

At midnight this morning, all automated data reconstruction was completed successfully on 23 of 41 volume groups comprising the disk array. The remaining 18 of 41 volume groups will be reconstructed in a less automated fashion, as was expected. A process to export the 23 reconstructed volume groups is nearly completed. This process serves toContinue reading “Major service interruption update 7:15 am”

Major service interruption update and review at 2:05 pm

Yesterday, at approximately 4 am, we experienced a major hardware failure on one of the primary storage disk arrays used by many NJIT applications. We have been working since that time to restore services. As of noon today, approximately 70% of the consistency checks had completed. It is difficult to predict an exact time, butContinue reading “Major service interruption update and review at 2:05 pm”

Major service interruption update at 9:15 am

The procedure to reconstruct data damaged in NJIT’s enterprise disk storage array is taken much longer than initially expected.  Fortunately, there are no indications of any data loss, and data integrity is the highest priority.  The Gmail authentication service has been restored and restoring the Moodle authentication service is under investigation. At this time, theContinue reading “Major service interruption update at 9:15 am”

Major service interruption status update at 10:45 pm

Unfortunately, after careful analysis of the disk and data rebuild progress, and after discussions with Oracle engineers about the safest strategy for the remaining processes, we’ve determined that this service restoration might not be completed before 8:00AM Wednesday morning. Our highest priority continues to be the integrity of the data. Our work with the OracleContinue reading “Major service interruption status update at 10:45 pm”