North American Network Operators Group|
Date Prev | Date Next | Date Index | Thread Index | Author Index | Historical
Re: Akamai DNS Issue?
Mark Radabaugh wrote:
I don't have much to offer for the "excessively complicated" caseBut you don't say how to avoid failures caused by massive confusion when maintaining a excessively complicated system....
(which I think the instant case is an example of), but there are
cases as complex and complicated with some justification in my history.
For those, the best solutions involved concepts like "canned, tested,
documented procedures", "quality control", "change management" (which
included "staging", "testing and verification", and so on.
We were not fond, in the "production" and "system test" environments, of people who made ad hoc changes of any kind.
Many years ago, I hand carried a patch through the approvals process,
group leader reviewed the purpose, urgency, test methods, test results,
and signed the sheet. District manager looked it over and asked "what
are the chances that this patch could fail?" I flippantly replied
"One in a million!".
He handed the documents back unsigned with the words "Seven times
in the Metro (Los Angeles, California) office tonight.
Requiescas in pace o email
Ex turpi causa non oritur actio