UPS Monitoring and Clean Shutdowns

[Bruce Cloutier]

Along these lines....

We have been without power after storms here in Pittsburgh Tuesday evening. It is not back as of 9AM Thursday. We run our own servers and so... INTEG has fallen off the map. Our UPS likely did admirably for as long as it could. I am interested in how long it held out. The best timeline will come from the JNIOR that sits there.

We will see if anyone panicked once we are back online.

[Bruce Cloutier]

Ryan, check in with support here either through the chat or by email. I mentioned this to them. They were intrigued and Kevin is already trying to communicate with our APC power systems. But, yes, all of that is possible. With the JNIOR we can even use logic to take different actions if a show is in progress. Whatever you need. It all depends on how well you have integrated the JNIOR into your system. You are not confined to what you can achieve with the cinema application or with tasker. A specialized separate application might be appropriate. Like I said all you have to do is ask.

[Ryan Gallagher]

As long as you are dealing with a Series 4 JNIOR you can have any action occur after a delay when the input changes state. It can even not do anything if the input returns.

in the simple input detecting approach. Would it be trivial to tell it to wait 20 minutes from “the most recent” detection before sending a WOL signal. For example, this might be enough to avoid events where the power fluctuates as it recovers. Last thing you want is to trigger a wake in the middle of a series of brown/blackouts with a low battery state.

equally could you cancel a 30min shutdown wait timer if that threshold is met?

Hard coded timers get iffy if battery is depleted from a prior event. The “assume the worst” and shutdown immediately approach might be best. Set the wake timer to correspond to a battery charge time.

[Bruce Cloutier]

As I see it, NUT has a few advantages to JNIOR here. It can wait to trigger a shutdown when the UPS battery is more depleted (configurable), rather than assume the worst and shutdown immediately. Deciding when to power back on is probably in the cards too. But native support for cinema servers is probably a plus for JNIOR!

As long as you are dealing with a Series 4 JNIOR you can have any action occur after a delay when the input changes state. It can even not do anything if the input returns. And if there is no apparent way to accomplish that all you have to do is ask support and they'll find a way. A Series 3 would have been able to do something similar but we can no longer develop for it.

Battery status is a complicated thing. If we can communicate with the UPS we can create an application to trigger on UPS status (as to what it thinks the battery has left) and not even use the overly simple wall wart trick. You just need to ask INTEG because we have no one out there promoting features. You folks do realize that we don't employ sales or marketing people. Maybe in the future.

[Ryan Gallagher]

typically you can use a shared certificate or known root password to SSH into the dcp2000 and run a shutdown command. no extra software needed.
this way you don't need to modify the device at all and control it's shutdown by an external monitoring agent.

noted. So whether it is NUT or JNIOR doing the monitoring, a client side service on the DCP2000 is not mandatory, assuming you are fine letting the bios handle the power on as soon as power is restored.

Power on after interruption is kinda a separate topic, easiest when the device supports bios restore state, wake on lan, or has a factory or 3rd party management interface. But not a nut you have to crack if clean shutdown is all you are after. (Pun intended). :-)

[James Gardiner]

typically you can use a shared certificate or known root password to SSH into the dcp2000 and run a shutdown command. no extra software needed.
this way you don't need to modify the device at all and control it's shutdown by an external monitoring agent.

[Ryan Gallagher]

As I see it, NUT has a few advantages to JNIOR here. It can wait to trigger a shutdown when the UPS battery is more depleted (configurable), rather than assume the worst and shutdown immediately. Deciding when to power back on is probably in the cards too. But native support for cinema servers is probably a plus for JNIOR!

[Ryan Gallagher]

I can play with the JNIOR route too, own a couple now. That price point may be a harder pitch to the org, would have to pair it with my curtain and house light automation improvements.

[Bruce Cloutier]

We just stick a JNIOR power supply (or any) in an unprotected outlet and wire it directly to a JNIOR input. Assuming that the JNIOR is on the UPS you can easily program the input to trigger, delay a bit and then signal a shutdown. We have one that just sends an email to alert us to the power event. It emails also when the power returns. You can then get a count of events and calculate event duration from the IOLOG if you need fuel to complain to the power company or whatnot.

[Ryan Gallagher]

From a more DIY perspective, perhaps not integrator ready. I was not aware of NUT previously, and looks pretty handy.

https://networkupstools.org/

It is a open-source 3rd party alternative for various proprietary UPS monitoring and automated shutdown/start clients. While I don't have any servers that stay on in my booth, I do have two un-monitored UPS (that support monitoring), and getting a nice interface to load and battery status history would be cool even if that was all I got from it.

It seems i'm late to this party as there was forum activity mentioning DCP2000 experimental NUT support back in 2018
https://www.film-tech.com/ubb/f16/t001786.html

We do often leave our cinema server on overnight working on ingests, or it will stay on for the duration of a festival run. I'm not sure installing a NUT-client on a doremi is still possible, but if we eventually have a NAS that could be shut down cleanly etc. I imagine something like a PiKVM could act as an intermediary that supports sending proper shutdown sequence to doremi. I believe Doremi supports APC-mon integration (or did at one point with an available package), but we have Middle Adlantic UPS here.

Jeff did a nice video about NUT last month, which feels a bit more mature than it was in 2018?

I'm not sure the company will buy me a RPi to run the primary on, but I'm tempted to feel it out using my own and pitch the addition when I have time for projecting. Not having a NAS, too many useful things can be accomplished with a single board computer that stays on in the booth.