Periodic vs Continuous Backup

Mastering Backup and Restore in Azure Cosmos DB: Periodic vs. Continuous Backup

Introduction: The Criticality of Data Protection

In the realm of distributed database systems, data loss is not merely a technical inconvenience; it is often a catastrophic business event. Azure Cosmos DB, as a globally distributed, multi-model database service, is designed for high availability and low latency. However, high availability is not the same as data protection. Even with replication across multiple regions, accidental deletions, application-level corruption, or malicious updates can propagate across your entire distributed environment in milliseconds. This is why a robust backup and restore strategy is the final, essential line of defense in your database architecture.

Understanding the distinction between Periodic and Continuous backup modes is foundational for any Azure administrator or developer. Choosing the wrong strategy can lead to either excessive costs or, more dangerously, a Recovery Point Objective (RPO) that exceeds your business requirements. This lesson explores the technical mechanics, configuration patterns, and strategic decision-making required to manage data protection effectively in Azure Cosmos DB. By the end of this module, you will understand not just how to turn these features on, but how to architect them to match your specific recovery needs.

Understanding Periodic Backup Mode

Periodic backup mode is the legacy, default configuration for many Azure Cosmos DB accounts. It operates by taking snapshots of your data at defined intervals and storing them in a storage account. Because it relies on periodic snapshots, your data recovery is limited to the last completed backup point.

How Periodic Backup Works

In periodic mode, the system automatically takes a snapshot of your database every few hours. You have control over two primary parameters: the backup interval and the backup retention period. The interval determines how often a snapshot is taken, and the retention period determines how long that snapshot remains available for restoration.

• Interval: You can configure how frequently the system backs up your data. This is typically measured in hours.
• Retention: This dictates how far back in time you can go to restore your data.
• Storage: The backup data is stored in locally redundant storage (LRS) by default, though you can configure it for geo-redundant storage (GRS) for additional safety.

Callout: RPO vs. RTO in Periodic Backup In periodic backup, your Recovery Point Objective (RPO) is essentially the length of your backup interval. If you back up every 4 hours, your RPO is 4 hours—meaning you could lose up to 4 hours of data. Your Recovery Time Objective (RTO) depends on the size of your data and the time it takes for Azure to initiate and complete the restore process, which can take several hours depending on the volume of data being restored.

Practical Use Cases for Periodic Backup

Periodic backup is ideal for workloads where the cost of storage must be strictly controlled and where the business can tolerate a data loss window of several hours. It is commonly used for:

• Development and testing environments where data is non-critical.
• Read-only workloads where data changes infrequently.
• Archival databases that are rarely modified.

Understanding Continuous Backup Mode

Continuous backup mode represents a modern approach to data protection. Instead of relying on snapshots, it logs every transaction that occurs within your database. This allows you to perform a "Point-in-Time Restore" (PITR) to any specific millisecond within your retention window.

The Mechanics of Continuous Backup

When you enable continuous backup, Azure Cosmos DB begins streaming the change feed of every write, update, and delete operation into a highly durable storage layer. This eliminates the "gaps" found in periodic backups.

• Granularity: You can restore your data to a specific point in time, down to the second.
• No Backup Window: Since every change is logged, there is no "interval" between backups. You are protected from the moment an operation is committed.
• Restoration Process: When you initiate a restore, the system creates a new database account and replays the transaction logs up to the exact timestamp you specified.

Note: Continuous backup is currently supported for both the SQL (Core) API and the Azure Cosmos DB for MongoDB API. Always verify the latest API support in the Azure documentation, as features are updated frequently.

Why Choose Continuous Backup?

Continuous backup is necessary for mission-critical applications where data loss is unacceptable. Because it provides the ability to recover from accidental "drop collection" or "delete document" commands, it offers a level of safety that periodic backups simply cannot match.

Comparison Table: Periodic vs. Continuous

Step-by-Step: Configuring Backup Policies

Configuring Periodic Backup via Azure Portal

1. Navigate to your Azure Cosmos DB account in the Azure portal.
2. In the left-hand menu, under the Settings section, select Backup & Restore.
3. Ensure the Backup Policy is set to Periodic.
4. Adjust the Backup interval (e.g., set to 240 minutes).
5. Adjust the Backup retention (e.g., set to 7 days).
6. Choose your Backup storage redundancy (Locally redundant, Zone redundant, or Geo-redundant).
7. Click Save.

Configuring Continuous Backup via Azure CLI

If you are automating your infrastructure, using the Azure CLI is the preferred method. The following command creates a new Cosmos DB account with continuous backup enabled:

az cosmosdb create \
    --name MyCosmosAccount \
    --resource-group MyResourceGroup \
    --backup-policy-type Continuous \
    --location "East US"

Explanation of the command:

• --backup-policy-type Continuous tells Azure to skip the snapshot method and enable the transaction log streaming service.
• Once this is set, you cannot switch back to Periodic mode for that account. This is a one-way configuration change in many scenarios.

The Restoration Process: A Practical Guide

Restoring a database is a high-stakes operation. Regardless of the backup mode, the restoration process does not overwrite your existing database. Instead, it creates an entirely new Cosmos DB account.

Step-by-Step Restoration

1. Identify the need: Determine the exact timestamp (UTC) to which you need to restore.
2. Initiate Restore: In the Azure Portal, go to the Backup & Restore tab of your account.
3. Select Point-in-Time: If using Continuous mode, use the slider or input the specific timestamp.
4. Configure New Account: Provide a name for the new account. Note that the new account will have the same API (SQL or MongoDB) as the original.
5. Validation: Once the restore completes, the status will show as "Succeeded." You must then manually update your application connection strings to point to the new account.
6. Data Migration: If you only needed to recover a single container, you can copy the data from the new account back into the original account using tools like Azure Data Factory or custom scripts.

Warning: Restoration does not automatically restore your stored procedures, triggers, or user-defined functions (UDFs) in all cases. Always verify the metadata of your restored account immediately after the operation completes to ensure your business logic is intact.

Best Practices and Industry Standards

1. The Principle of Least Privilege

Access to initiate a restore is a sensitive permission. Only senior database administrators or DevOps leads should have the Microsoft.DocumentDB/databaseAccounts/restorableDatabaseAccounts/restore/action permission. Restricting this prevents accidental or malicious data overwrites.

2. Monitoring and Alerting

Do not assume your backups are running successfully. Set up Azure Monitor alerts for the BackupPlanChange or BackupRestore events. If a backup fails, you need to know immediately, not when you are in the middle of a disaster recovery scenario.

3. Testing Your Restore Strategy

A backup is not a backup until it has been restored successfully. Conduct "Fire Drills" every quarter. Create a test database, fill it with dummy data, and perform a full restoration to a separate account. Time how long it takes and document the process. This ensures that when a real disaster strikes, your team is not reading the documentation for the first time.

4. Storage Redundancy

If your application serves a global audience, always select Geo-redundant storage for your backups. If your primary region experiences a catastrophic failure, having your backups stored in a secondary, geographically distant region is the only way to recover your data.

Common Pitfalls and How to Avoid Them

Pitfall 1: Overlooking the "New Account" Requirement

Many developers assume they can restore a single container directly into their production database. This is a common misunderstanding. Azure Cosmos DB restores the entire account. If you have multiple databases in one account, all of them will be restored to the new account.

• The Fix: Plan your account hierarchy carefully. If you need granular restore capability, consider separating high-value databases into their own dedicated Cosmos DB accounts.

Pitfall 2: Ignoring Restore Time

Restoration is not instantaneous. For large accounts (multiple terabytes), the restoration process can take hours.

• The Fix: Include the duration of a full restore in your business continuity plan. If your SLA requires a 1-hour RTO, but your data takes 4 hours to restore, you must implement additional measures, such as maintaining a read-only secondary region or keeping hot-standby data copies.

Pitfall 3: Failing to Update Connection Strings

After a restore, your application will continue to point to the old, corrupted, or deleted account.

• The Fix: Use Azure Key Vault to manage your connection strings. When a restore is complete, update the secret in Key Vault to point to the new account endpoint. This allows your application to pick up the new connection details without requiring a code deployment or a service restart.

Deep Dive: The Cost Implications

Cost is often the deciding factor between Periodic and Continuous backup. Periodic backup costs are generally predictable and lower because you are only paying for the storage of a few snapshots. Continuous backup, however, incurs two distinct costs:

1. Storage Costs: You are paying for the storage of the transaction logs, which can grow significantly if your database has a high write volume.
2. Resource Costs: The background processing required to maintain the continuous stream of changes consumes compute resources, which is reflected in your monthly bill.

Callout: Calculating the Cost of Protection Before enabling Continuous backup, perform a cost analysis based on your average daily write volume. If your application performs 1 million writes per day, the log storage required for a 30-day retention period will be substantial. Always use the Azure Pricing Calculator to estimate these costs before enabling the feature in production.

Advanced Scenarios: Handling Global Distribution

When you use Azure Cosmos DB with multiple write regions, backup and restore becomes more complex. In a multi-region write environment, the continuous backup service captures the global transaction stream.

Multi-Region Considerations

• Global Consistency: When you restore a multi-region account, the restored account will be created in the same primary region as the original. You will need to manually add the secondary regions back to the account after the restore is finished.
• Conflict Resolution: If you are using custom conflict resolution policies, ensure those policies are documented. During a restore, the system will apply the same policies to the data as it is replayed.

Summary Checklist for Database Administrators

When designing your backup strategy, use this checklist to ensure you haven't missed any critical steps:

• Define RPO/RTO: Have you confirmed the business requirements for data loss and downtime?
• Choose Mode: Is Continuous backup required, or is Periodic sufficient?
• Set Retention: Is your retention period long enough to catch "silent" data corruption (e.g., a bug that slowly corrupts data over a week)?
• Redundancy: Is your backup storage configured for GRS (Geo-redundant storage)?
• Permissions: Have you audited who has the right to initiate a restoration?
• Testing: Is there a scheduled date for the next restoration drill?
• Automation: Is the backup configuration defined as code (ARM templates/Bicep/Terraform) to prevent manual configuration drift?

Frequently Asked Questions (FAQ)

Q: Can I switch from Periodic to Continuous backup? A: Yes, you can upgrade from Periodic to Continuous backup. However, you cannot downgrade from Continuous to Periodic. Once you choose Continuous, that account is locked into that mode.

Q: Does restoring an account impact the performance of the original account? A: No. Because the restore operation creates an entirely new account, there is no performance impact on your production traffic.

Q: What happens to my data if I delete the original account? A: If you delete the account, the backups associated with that account are also deleted. If you need to retain backups for compliance reasons, ensure you have strict "Resource Lock" policies on your production Cosmos DB accounts to prevent accidental deletion.

Q: Can I restore to a different subscription? A: Yes, you can restore to a different subscription within the same Azure Active Directory tenant. This is a common pattern for moving data from a production environment to a restricted sandbox for debugging.

Conclusion: Building a Resilient Data Strategy

The choice between Periodic and Continuous backup is a decision about the risk profile of your application. Periodic backup is a cost-effective, snapshot-based approach suited for less critical data. Continuous backup is a powerful, log-based solution that provides the granular protection required for modern, high-velocity applications.

As we have explored, the backup itself is only the first half of the equation. The second, and perhaps more important half, is the restoration process. A backup that cannot be restored in a timely or reliable manner provides a false sense of security. By following the best practices outlined in this lesson—specifically the emphasis on testing, monitoring, and automated configuration—you can ensure that your Azure Cosmos DB solution is truly resilient.

Remember that technology changes, and Azure Cosmos DB is no exception. Always keep an eye on the official Microsoft Azure documentation for updates regarding backup policies, as new capabilities for cross-subscription restores and enhanced retention options are frequently added. Treat your backup strategy as a living part of your application architecture, one that evolves alongside your data and your business requirements.

Key Takeaways

1. Understand your requirements: Always align your backup mode with your RPO and RTO requirements. Never choose a mode based on cost alone.
2. Continuous is for mission-critical: Use Continuous backup mode when you need point-in-time recovery to protect against accidental data loss or corruption.
3. Restore creates new accounts: Always plan for the fact that a restoration results in a new account, which requires updates to your application connection strings.
4. Test, Test, Test: A backup is only as good as your ability to restore it. Conduct regular, documented restoration drills to validate your procedures.
5. Secure your backups: Treat backup permissions with the same level of security as your production database access.
6. Automate: Use Infrastructure-as-Code to ensure backup policies are consistently applied across all your environments.
7. Monitor failures: Never assume a backup is working; configure alerts for backup failures to ensure immediate notification of any issues.