Chromebook Automatic Updates - Sanity from Chaos

Chromebook Automatic Updates - Sanity from Chaos
Photo by Solen Feyissa / Unsplash

The amount of school districts getting absolutely bulldozed by hackers is an unpleasant reality that keeps me awake at night. In no small part due to my insistence, our school district has been on an absolute security rampage this past year. An integral part of a good security posture is staying on top of patching & updates, and while you are likely familiar with all the options for Windows (WSUS, SCCM, even WUfB) ChromeOS devices are a different paradigm altogether.

Chromebooks can be left charging in carts for months, sudddenly pulled out and used for a few minutes, and then promptly slammed shut and thrown back into their sad little cave. Not only will your users expect to be able to use the device the literal second it turns on, they will also ignore notifications saying the Chromebook is updating, delay reboots as long as humanly possible, and you will be flooded with tickets from puppy-eyed end users wondering how not updating for 6 months could possibly make the "...internet not work."

But how bad is it really? Let's get some stats!

There are a bunch of different ways to get to the same place for some reason, but Reporting -> Devices -> ChromeOS & browser versions makes the most sense to me.
Here's a link for the lazy.

Then make sure to click ChromeOS Versions near the top right.

Clear out the auto-populated report, otherwise you won't see any of the devices that haven't reported or synced recently.

Finally, click All managed devices and Browsers near the left, since Google really, really wants you to only look at a single OU for some reason.

Make sure you're looking at the Active Devices column, not the Inactive Devices column. Hopefully all of yours are clustered near the top This isn't terrible but clearly we've got some work to do. I won't show you the ugly part...

What do I do?

Purely within the Google Workspace Admin console, there are a number of different settings which all affect how successful your devices are at applying the latest updates. Chromebooks CAN auto-update, but only under certain conditions.
They CAN automatically reboot as well, but only under certain (other) conditions.
What are those conditions? First lets talk about the possibilities...

A Chromebook can exist in a multitude of states:

  • Off
  • Sleep Mode
  • Sleep Mode + Logged in & Locked
  • Sleep Mode + Logged in & Unlocked
  • On
  • On + logged in & Unlocked
  • On + Logged in & Locked

What can affect these states:

  • A User deliberately logging in/out.
  • Closing the Lid
  • Physically powering off (or battery dying)
  • Going to Sleep
  • Inactivity timeouts

What (mostly) cannot affect these states:

  • Updates...

Conclusions:

  • We can pretty much ignore the Off state.
  • Sleep is a state, while going to sleep is a trigger.
  • All of this means if we want to control cause and effect, we have a grid of 5 Triggers x 6 States. That's 30 possible scenarios to account for. ugh. Luckily we can group quite a few of them together.

Why is updating so hard?

  • OS updates must first be downloaded and installed, but they will not apply until a device is rebooted (or powered off and back on separately of course).
  • Chromebooks can download and install updates as long as they are ON.
  • Supposedly they can also download and install while in sleep mode, but this has failed for us tremendously. In this regard I consider Chromebooks in sleep mode to effectively be off.
  • Chromebooks CAN automatically reboot to apply updates, but cannot do so while a user is still logged in.

What switches and knobs do we have to play with? I am glad you asked.

When any user in any state closes this Chromebook, should it Sleep, Shutdown, or Do Nothing?

We can force Chromebooks to reboot automatically regardless of whether or not anyone is logged in based on a time schedule.

Allow (or prevent) automatic sleep/shutdown for Chrome devices that are sitting idle at the login screen (with no one signed in.)

This is per-device. There are quite a few settings here.

  • Allow or prevent automatic updates.
  • Set target version to latest, or a specific older version.
  • Roll back to target version if current is newer.
  • Pull updates from Stable, Beta, or LTS channels
  • Scatter updates randomly, apply on a schedule (perhaps per OU), or update every device as soon as possible.
  • Set auto-update blackout windows.
  • Auto Reboot after updates (Only works after users are logged out!)
  • Allow/prevent updates over cellular connection.
  • Allow peer-to-peer updates (akin to Windows Delivery Optimization)
  • Enforce updates to at least a specific version.
  • Warn users before blocking access.

Again, a big one. This time, User-based settings:

  • Sleep, Logout, Shutdown, or Do Nothing when a user closes the lid while logged in.
  • Lock Screen on Sleep/Lid Close (or don't).
  • Set Idle timeouts when plugged in
  • Set idle timeouts when unplugged (on battery)
  • Sleep, Logout, Shutdown, or Do Nothing when Idle timeouts are reached.
  • Set idle timeouts to warn, dim screen, turn off screen, and/or lock screen.

So here's the situation...

  • Students use Chromebooks during class, shut them without logging out, and plug them in in the cart. Normally that'd mean they are asleep, and logged in, both of which prevent updates from completing.
  • Students in higher grades use their Chromebooks on campus, close them without logging out, and move from class to class, so we can't be too aggressive about what happens when the lid closes.
  • Students take their Chromebooks home in the evening, open them to do homework, plug them in overnight, and again close them without logging out.
  • Updating all the Chromebooks at once oversaturates the network.
  • Chromebooks updating during class is a no-go, as is rebooting.
  • Neither teachers, nor students will ever remember to manually turn off a Chromebook or log out. The only thing we can count on them doing is opening and closing it.

For the time being, and admittedly because our Chromebooks are not yet all that well organized within their OUs, we're going to try to keep things simple with a one-size-fits-all approach.

..and here's what we ended up doing.

  • Allow updates - Enabled. Obvious, although technically you could rely solely on manual updates if you were sufficiently masochistic.
  • Use latest version - Enabled. This setting is for moments when you have need of a specific number version. Remember there is a separate Release Channel option.
  • Do not roll back OS - Disabled. Useful in case of breaking changes. Haven't had need of this yet.
  • Release Channel - Stable. Choose based on your threat model and risk tolerance. We've had more issues with lack of updates than we have breaking changes.
  • Rollout Plan - Scatter Updates over 3 days (from release). This keeps the network from being completely overwhelmed.
  • Auto-update blackout - On. Only during school hours, this prevents any disgruntled teachers.
  • Allow auto-reboots - On. If an update has already been downloaded and installed, when a user logs out it will reboot and apply. Very seldom does anyone manually log out, and reboots are very fast so this causes no issues.
  • Allow automatic updates over Cellular - ON. For our Chromebooks with SIM cards, we have unlimited data plans so this is no concern.
  • Allow P2P updates - On. This helps congestion. Some have security concerns, but updates are digitally signed and there have been zero exploits so far
  • Enforce updates - On. Devices that are out of date are blocked from use until updated.
  • Minimum Version - On. Only a few devices are this far behind, and this is four major versions back as of writing.
  • Enforce with no warning - On. Once we get back on track we can relax on this a bit.

For device settings, when the lid is closed we do nothing. (Locking on lid close is a User setting.) Do not go to sleep, log out, or shutdown. This is to keep the device awake when users close the lid so we can apply updates. This does come at the cost of battery life.

Every day at 4AM, all the Chromebooks reboot out of an abundance of caution. This will allow them to flush their cache, and allow any stragglers to apply updates. We will likely nix this as we get closer to 100% compliance.

Idle devices on login screen CAN be shut down or go to sleep automatically. Not sure when you'd want this disabled; Maybe if using ONLY scheduled reboots? Digital Signage? Kiosks? It doesn't help us, but it could kneecap you if you didn't know about it.

Closing the lid locks the screen. (User settings) No sleep mode, nor poweroff. This allows updates to download even while closed & running on battery at home, but prevents students from being irritated as they take their Chromebooks from class to class.

Automatic logout after one hour (3600 Seconds). This is true whether plugged in or not. While plugged in and charging, this means when Chromebooks are shut and placed in the carts, they will stay ON to apply updates, then they will log out after 1 hour at which point the auto-reboot setting can apply. Then everyone gets another reboot at 4AM for good measure. Other settings are left empty so the user can configure them . On the other hand, if a one-to-one student has their Chromebook unplugged and closed over an hour, it's unlikely they're at school. We can safely take this chance to logout so any pending reboots can happen & updates can apply.

That's all I got.

Best of luck to you. I will let you know how it goes for us.