As of 6:00 pm this evening, Oracle engineers have successfully reconstructed all of the failed components of the disk storage array that has been out of service since early Tuesday morning. They are now in the process of reconnecting these components to make all of them available to the storage array’s controllers. This process willContinue reading “Status update at 8:00 pm”
As of noon today, the process to export 23 reconstructed volume groups was completed, a failed disk controller was replaced and now dual controllers in the storage array are functioning normally. Five of 18 volume groups needing to be reconstructed have been successfully reconstructed. Recovery procedures are run with due caution and continued success byContinue reading “Major service interruption update at 12:30 pm”
At midnight this morning, all automated data reconstruction was completed successfully on 23 of 41 volume groups comprising the disk array. The remaining 18 of 41 volume groups will be reconstructed in a less automated fashion, as was expected. A process to export the 23 reconstructed volume groups is nearly completed. This process serves toContinue reading “Major service interruption update 7:15 am”
Normal Moodle services are now available. Disk array data recovery is proceeding successfully, but very slowly. This recovery process is being closely monitored. As soon as possible after it is completed, other services will be restored to their normal working conditions. Your continued understanding is appreciated.
Yesterday, at approximately 4 am, we experienced a major hardware failure on one of the primary storage disk arrays used by many NJIT applications. We have been working since that time to restore services. As of noon today, approximately 70% of the consistency checks had completed. It is difficult to predict an exact time, butContinue reading “Major service interruption update and review at 2:05 pm”
Earlier this morning, we got the Gmail authentication service restored, so you can log into Webmail by Google. (NJIT students and alumns are the typical users of Webmail by Google.)
The procedure to reconstruct data damaged in NJIT’s enterprise disk storage array is taken much longer than initially expected. Fortunately, there are no indications of any data loss, and data integrity is the highest priority. The Gmail authentication service has been restored and restoring the Moodle authentication service is under investigation. At this time, theContinue reading “Major service interruption update at 9:15 am”
Unfortunately, after careful analysis of the disk and data rebuild progress, and after discussions with Oracle engineers about the safest strategy for the remaining processes, we’ve determined that this service restoration might not be completed before 8:00AM Wednesday morning. Our highest priority continues to be the integrity of the data. Our work with the OracleContinue reading “Major service interruption status update at 10:45 pm”
Due to the complexities of a multiple component failure, it looks like we are still several hours away from service restoration. Our current estimation is around midnight. Thanks for your continued patience.
Repairs are continuing. Thus far there are no signs of data loss or corruption, however the consistency checking is proceeding slower than anticipated. The new estimated time for service restoration is 5 pm. We thank you for your continued patience.