[Clusterusers] keg rebuild attempt

[Wm. Josiah Erikson]

OK, I have set up most of keg and am rebooting it back to the old IP.
Most services should be available immediately, but I imagine several
(many?) things will be broken... let me know what's broken and stay tuned.

    -Josiah


On 3/5/15 2:46 PM, Wm. Josiah Erikson wrote:
> I have replaced the failed disks, recreated the RAID, installed Ubuntu
> 14.04, run updates, and am now copying the backed-up /helga over to the
> new keg and waiting for the RAID array to resync. These things will
> probably take all night. Seeing as how I'm sick and got in at like 2AM
> last night, I'm going home and will pick this up tomorrow - I'll keep
> you updated.
>     -Josiah
>
>
> On 3/5/15 11:51 AM, Wm. Josiah Erikson wrote:
>> Hi folks,
>>     I have failed in my attempts to get keg's RAID array back. I will be
>> rebuilding keg with Ubuntu 14.04LTS, and then manually configuring
>> everything (which shouldn't be too hard) and copying things back from
>> backups. I hope *fingers crossed* to have everything back up by the end
>> of tomorrow. If I don't, I will work on it over the weekend.
>>     Until then, tractor, helga, and the render farm in general will
>> continue to be down. My sincerest apologies for this inconvenience!!
>>     -Josiah
>>
>>
>> On 3/5/15 10:02 AM, Wm. Josiah Erikson wrote:
>>> Hi guys,
>>>     I am going to start working on getting keg back online. This will
>>> happen in two stages:
>>>
>>>     1. I am going to attempt to get the RAID array to re-assemble, so we
>>> can start again right where we left off. My initial attempts don't look
>>> promising, but I'm going to keep trying
>>>     2. Failing that, I am going to have to rebuild from scratch and then
>>> restore the backed-up data
>>>
>>>     IMPORTANT INFO:
>>>
>>>     We do not back up everything. The svn repo is of course backed up,
>>> and all of /helga/tube is backed up, but only because we made a special
>>> exception. Unfortunately, the only thing in /helga/<yourfolder> that is
>>> backed up is anything that you put in /helga/<yourfolder>/prot, which
>>> means that any existing renders, etc, will be gone if I have to go with
>>> #2, which looks increasingly likely.
>>>
>>>     The idea here is that only machine time is required to rebuild what
>>> you've lost if you've put all the source material in the svn repo. I
>>> hope that this is the case, but it still kinda sucks.
>>>
>>>     I forgot my phone today, but feel free to stop by the server room or
>>> shoot me an email back.
>>>

-- 
Wm. Josiah Erikson
Assistant Director of IT, Infrastructure Group
System Administrator, School of CS
Hampshire College
Amherst, MA 01002
(413) 559-6091