Azure Recovery Services Vault SQL backups stops working after Windows Server 2016 patch KB4565511

Oopps! Upgrade your browser pretty please. Oopps! Upgrade your browser pretty please.

UPDATE: You don’t have to uninstall the patch to get the backups working again. You only need to reregister the servers in the Recovery Services Vault!

After patching Windows Server 2016 with patch KB4565511, the Recovery Service Vault lost the ability to backup the SQL databases from the SQL Server hosted on that server. Recovery Services Vault usually alerts about failed backups, but when this happened it was completely silent.

So how can you check if yo’re having the same problem?:

  1. Go to the “Azure portal”.
  2. Go to your “Recovery Services Vault”.
  3. Under “Protected items”, go to “Backup items”.
  4. Go to “SQL in Azure VM”.
  5. Sort the list of Backup items by “Backup Status”.

You should now be able to find all the Backup items with the status “Not reachable”. If all of the backup databases on a server has this status, you might be affected by this bug.

 

To fix it, we had to roll back the Windows update. It worked right away for most servers, but for some we had to “re-register” the server in Recovery Services Vault.

An interesting thing about this bug is that if you query the affected SQL Server, it looks like the backups are being taken as usual. But you can’t see the files in Recovery Services Vault, so it seems like the backup command reaches the SQL Server but the backup file is lost somewhere on the way back to Azure. Which really highlights the importance of restore testing your SQL backups. Backups aren’t all that useful if you can’t restore them.