Everyone has an EPO story, and it never seems to involve an actual emergency.
An Emergency Power Off (EPO) pushbutton is often required by building safety codes at data center exit doors. The purpose for the EPO is to give someone an easy, manual way to instantly turn off power and cooling air flow to IT equipment in the event of an emergency, such as electrocution, fire, smoke, flooding, etc. Fortunately these life-safety emergencies rarely happen in data centers. But erroneous data center shutdowns by EPO systems for no good reason happen on a regular basis. It often happens because someone pushed a button they should not have pushed. But just as often, design flaws encourage EPO activation when it should not occur. Then there is the well-meaning technician working with incorrect wiring diagrams or whose screwdriver simply ends up in the wrong place.
In the not too distant future the EPO system may go the way of the dinosaur. EPO systems are generally not required for data centers that do not use access floor systems for supply air and power distribution, becoming the preferred design of choice. But for now, most legacy data centers and traditionally designed new centers need an EPO system. Fortunately there are many ways to reduce EPO shutdown risk.
Basic guidelines include avoiding normally-closed, held circuitry and avoiding the big red, unprotected mushroom button at the door next to the light switch. Use normally-open circuitry that does not require a power supply to hold relay contacts closed to keep power and cooling equipment operating. If the EPO power supply fails or loses power, or any part of the EPO wiring becomes loose or relay contacts bounce, the EPO may fail to work in the event of an emergency but at least it won’t take the data center down. EPO buttons should be recessed, non-locking and under translucent protective covers to prevent inadvertent bumping from EPO activation. Better yet, the cover can be hinged and include a sensor that activates horns and strobes as soon as someone begins to left the cover. This gives the person something to think about before their finger even gets close to the button.
Additional tactics can be used, such as building in a time delay between pushing the EPO button and when shutdown signals are sent to the data center. Fire alarm, smoke detection and suppression equipment is often wired to activate the EPO. Although this makes sense strictly from a life-safety perspective, it is typically not required by building safety codes and should be avoided to reduce risk of something less than an actual fire condition, such as a false alarm or equipment malfunction from shutting down the data center. For highly redundant data centers with A/B power distribution and redundant cooling equipment, two independent EPO systems can be installed. One button and set of relays and wiring only shuts down “A” side power and half of cooling. The other set only shuts down “B” side power and cooling. In this case, at least in theory, critical dual-power circuited IT equipment can survive a single EPO error or malfunction. When dual EPO systems are used, signage must make it clear that in an actual emergency both buttons must be pushed. You don’t want someone at the exit door trying to decide whether his partner is getting shocked by an A circuit or a B circuit.