Disk Striping and Simple Volumes

Module: Design and Implement SAP Infrastructure

Lesson: Disk Striping and Simple Volumes in SAP Environments

Introduction: The Foundation of SAP Performance

When we talk about the architecture of an SAP system, we often focus on the application layer, the database engine, or the interface design. However, the true heartbeat of any high-performance SAP environment lies in the storage layer. SAP applications are notoriously I/O-intensive. Whether it is the massive sequential reads required for database backups, the heavy random I/O of a transaction-heavy ERP system, or the constant logging operations of the SAP HANA database, the underlying disk configuration determines whether your system feels responsive or sluggish.

Disk striping and simple volumes are the fundamental building blocks of storage management. These concepts dictate how data is physically arranged across your storage devices. If you configure these incorrectly, you create bottlenecks that no amount of CPU or RAM can overcome. A poorly architected disk layout can lead to "I/O wait" states where the processor sits idle, waiting for the disk to retrieve data. Conversely, a well-planned layout allows the system to process thousands of transactions per second without breaking a sweat.

In this lesson, we will peel back the layers of storage management for SAP infrastructure. We will move beyond basic concepts and look at how striping (RAID 0) and simple volumes interact with file systems, volume managers, and SAP’s specific storage requirements. By the end of this module, you will understand how to design a storage layout that matches the specific workload profiles of SAP HANA, SAP NetWeaver, and other critical components.

Understanding Simple Volumes: The Basics

A simple volume is the most straightforward way to present storage to an SAP operating system. In a simple volume configuration, a logical partition or a volume spans a single physical disk or a single logical unit number (LUN) provided by a Storage Area Network (SAN). The operating system sees this as a contiguous block of storage, which is then formatted with a file system like XFS, EXT4, or NTFS.

From a management perspective, simple volumes are easy to create, monitor, and maintain. Because there is a one-to-one mapping between the file system and the physical disk/LUN, troubleshooting is usually straightforward. If a disk fails, you know exactly which volume is affected. However, simple volumes have a major limitation: they are bound by the performance characteristics of that single disk or LUN.

If you have a 1TB simple volume on a single SSD, your maximum throughput is limited by the physical interface of that SSD. For many low-traffic SAP components—such as the /usr/sap/trans directory for small development systems or logs for non-critical interfaces—this is perfectly acceptable. However, for database data files or redo logs, a simple volume will almost certainly create a performance ceiling that limits your SAP system's growth.

Callout: Simple Volumes vs. Striping A simple volume provides simplicity and ease of recovery but limits throughput to the speed of a single disk. Striping (RAID 0) distributes data across multiple disks, which multiplies the throughput potential, effectively allowing the system to read and write from several disks simultaneously. While striping increases performance, it also increases the risk: if any one disk in the stripe fails, the entire volume is usually lost.

The Mechanics of Disk Striping

Disk striping is the process of splitting data into smaller chunks and writing these chunks across multiple disks simultaneously. This is technically known as RAID 0 (Redundant Array of Independent Disks, level 0). In an SAP context, we rarely use pure RAID 0 because the risk of data loss is too high. Instead, we use striping in conjunction with mirroring (RAID 10) or parity (RAID 5/6) to balance performance and reliability.

When you stripe data, you must define a "stripe size" or "chunk size." This is the amount of data written to one disk before the controller moves to the next disk in the set. For SAP HANA, the stripe size is a critical tuning parameter. If the stripe size is too small, you create excessive overhead for the controller. If it is too large, you fail to utilize the parallel nature of the storage array, and you end up with "hot spots" where one disk is working harder than the others.

Why Striping Matters for SAP

SAP HANA, for instance, requires high throughput for its data persistence layer. When the system performs a "savepoint," it flushes gigabytes of data from RAM to disk. If you are using a single disk, this operation will take a long time, potentially causing the system to throttle transaction processing. By striping that data across eight or sixteen disks, the savepoint duration is reduced significantly, allowing the system to return to peak performance much faster.

Implementing Striping with Logical Volume Managers (LVM)

In modern Linux-based SAP environments, we rarely deal with raw physical disks directly. Instead, we use the Logical Volume Manager (LVM). LVM provides a layer of abstraction that allows us to create stripes across physical volumes (PVs) with ease.

Step-by-Step: Creating a Striped Volume in Linux

Let us assume you have four physical disks (/dev/sdb, /dev/sdc, /dev/sdd, /dev/sde) and you want to create a striped logical volume for your SAP data files.

1. Initialize the Physical Volumes: You must first tell the system that these disks are available for LVM.

   pvcreate /dev/sdb /dev/sdc /dev/sdd /dev/sde
   

2. Create a Volume Group (VG): Group the disks into a single pool of storage.

   vgcreate vg_sap_data /dev/sdb /dev/sdc /dev/sdd /dev/sde
   

3. Create the Striped Logical Volume (LV): The -i flag specifies the number of stripes (number of disks), and -I specifies the stripe size (in KiB).

   lvcreate -L 500G -n lv_data -i 4 -I 64 vg_sap_data
   

4. Format and Mount: After creating the LV, format it with XFS (the recommended file system for most SAP HANA deployments) and mount it.

   mkfs.xfs /dev/vg_sap_data/lv_data
   mount /dev/vg_sap_data/lv_data /hana/data
   

Note: When using LVM striping, ensure that your stripe size matches the requirements of your storage array. For many enterprise-grade SANs, a stripe size of 64KiB or 128KiB is a standard starting point for SAP workloads. Always consult the vendor documentation for your specific storage hardware.

Performance Considerations and Tuning

Simply striping disks is not enough to guarantee performance. You must consider the alignment of your file system and the queue depth of your storage controllers.

File System Alignment

When you create a file system, it is divided into blocks. If the start of your file system partition does not align with the underlying disk's physical boundaries, you suffer from "misalignment." This means a single logical I/O operation might result in two physical I/O operations, effectively cutting your performance in half. Most modern tools like fdisk or parted handle alignment automatically, but it is always worth verifying with the blockdev --getalignoff command.

Queue Depth

The queue depth determines how many I/O requests can be outstanding at once. If your SAP application is sending thousands of requests, but your disk queue is set to a low value (e.g., 32), the system will bottleneck at the OS level. For high-performance SAP HANA systems, it is common to increase the queue depth to 128 or 256, depending on the storage controller capabilities.

I/O Schedulers

Linux uses I/O schedulers to manage how requests are ordered. For SSDs and NVMe drives, the none or kyber scheduler is generally preferred. Older schedulers like cfq (Completely Fair Queuing) are designed for spinning hard drives and can actually hinder performance on modern flash-based storage.

Comparing Storage Configurations

Best Practices for SAP Storage Architecture

When designing your storage, keep these industry-standard best practices in mind to avoid common pitfalls.

1. Separate Workloads: Never place SAP HANA data files on the same physical disks as your log files or operating system binaries. Each of these has a different I/O profile. Logs are write-heavy and sequential, while data files involve large sequential reads and random writes. Mixing them leads to I/O contention.
2. Use Enterprise-Grade Hardware: While striping can make cheap disks look fast, it does not solve the reliability issue. Always use hardware that is certified for SAP environments. SAP publishes specific hardware certification lists; if your storage is not on that list, you will face support challenges.
3. Monitor Latency, Not Just Throughput: It is easy to look at a dashboard and see "1GB/s" of throughput and think everything is fine. However, if the latency is high (e.g., above 5ms for HANA data), your application will still feel slow. Always monitor the await (average wait time) in iostat.
4. Automate Provisioning: Use configuration management tools like Ansible or Terraform to deploy your storage layouts. Manual creation of logical volumes is prone to human error, such as forgetting the stripe size or miscalculating the number of physical volumes.
5. Plan for Growth: If you use simple volumes, expanding storage usually requires downtime or complex migration. Using LVM allows you to add physical disks to a volume group and extend the logical volume online, which is a massive advantage for SAP systems that grow over time.

Warning: Be extremely careful when using LVM to remove a physical volume from a group. If you have not migrated the data off the PV using pvmove, you will corrupt your file system. Always verify the status of your logical volumes before performing any storage maintenance.

Common Pitfalls and Troubleshooting

Even experienced administrators run into issues with disk striping. Here are the most common mistakes and how to avoid them.

The "Single-Disk Bottleneck"

A common mistake is creating a large volume group and then realizing that all your volumes are physically located on the same set of disks. If you create two logical volumes on the same physical disks and both are busy, they will fight for the same I/O bandwidth. Solution: Use different volume groups for different SAP components. For example, keep vg_hana_data and vg_hana_log on physically separate storage controllers or disk sets.

Ignoring File System Fragmentation

Over time, file systems can become fragmented, especially if the volume is nearly full. This is particularly problematic for databases. Solution: Monitor your disk usage. For SAP HANA, it is recommended to keep your data partitions below 80-85% capacity. If you approach 95%, the overhead of finding free blocks for new data will degrade performance significantly.

Forgetting to Test Backup/Restore

Striping makes performance faster, but it also makes the rebuild time for a failed disk potentially longer if you are using RAID 5 or 6. Solution: Always perform a "dry run" of your disaster recovery plan. If a disk fails, do you know exactly how long the rebuild will take? Does the system remain usable during that rebuild? If not, you need to adjust your RAID level or your hardware redundancy.

Deep Dive: SAP HANA Persistence Layer

Because SAP HANA is an in-memory database, it is often misunderstood that "disk doesn't matter." This is a dangerous misconception. While the system runs in RAM, the persistence layer (the disk) is the ultimate source of truth. If the persistence layer is slow, the system cannot restart quickly after a crash, and it cannot perform savepoints efficiently.

For SAP HANA, the industry standard is to use RAID 10 for both the /hana/data and /hana/log volumes. RAID 10 provides the performance of striping with the safety of mirroring.

A Note on NVMe and Flash

With the advent of NVMe (Non-Volatile Memory Express), the traditional concept of "striping" is changing. NVMe drives are so fast that they often saturate the PCIe bus before they saturate the drive itself. In some modern SAP HANA configurations, you might not need to stripe multiple drives to get the required bandwidth; a single high-end NVMe drive might be sufficient. However, you still need to stripe them to ensure redundancy and to provide enough capacity. Always check the specific throughput requirements provided in the SAP HANA Hardware Configuration Guide.

Practical Example: Configuring a Filesystem for SAP NetWeaver

SAP NetWeaver (the application server) has different needs than HANA. It requires a large amount of space for the /usr/sap/trans directory (for transport files) and the /sapmnt directory (for shared binaries).

For these volumes, you do not necessarily need the massive I/O throughput of RAID 10. Instead, you might opt for RAID 5 or 6, which provides more usable capacity per dollar.

Steps for configuring a transport directory:

1. Identify the storage: Identify the LUNs assigned for transport storage.
2. Create the PVs: pvcreate /dev/sdb /dev/sdc /dev/sdd /dev/sde
3. Create the VG: vgcreate vg_trans /dev/sdb /dev/sdc /dev/sdd /dev/sde
4. Create the LV (No striping needed): lvcreate -L 2T -n lv_trans vg_trans
5. Mount and set permissions:

   mkfs.xfs /dev/vg_trans/lv_trans
   mount /dev/vg_trans/lv_trans /usr/sap/trans
   chown sidadm:sapsys /usr/sap/trans
   

Note that we did not use the -i flag here. For transport directories, which are mostly used for file transfers, the overhead of striping is not worth the minor performance gain. Simplicity wins here.

Monitoring Your Storage Infrastructure

You cannot manage what you cannot measure. To maintain a healthy SAP storage environment, you must implement a robust monitoring strategy.

• iostat -xz 1: This command is your best friend. It shows you the utilization, wait times, and throughput for every disk. If you see %util nearing 100%, your storage is saturated.
• df -h: Simple, but essential. Keep an eye on disk capacity.
• lvs / vgs / pvs: Use these commands to keep track of your LVM layout. Make sure you don't have "orphan" physical volumes or full volume groups.
• SAP Transaction DB02: This is the internal SAP tool for monitoring database storage. It will show you the growth rate of your data and log files, which is critical for capacity planning.

Callout: The Role of the Storage Admin In many organizations, the SAP Basis team and the Storage team are separate. This is a common point of failure. The Basis team knows the application requirements (e.g., "we need 500MB/s of throughput for the log volume"), while the Storage team knows the hardware (e.g., "we have 16 SSDs in a RAID 10 array"). The best SAP infrastructure is built when these two teams speak the same language.

Advanced Concepts: Storage Tiering

As your SAP environment grows, you will inevitably have "hot" data (data that is frequently accessed) and "cold" data (historical data that is rarely accessed). Modern storage solutions allow for "tiering," where the system automatically moves data between high-performance flash storage (for hot data) and lower-cost spinning disks (for cold data).

In an SAP context, you can implement this at the storage array level (transparent to the OS) or at the application level (using SAP HANA Data Tiering). When using storage-level tiering, be careful: if the storage array decides to move your "hot" HANA data file to a slow "cold" tier, your performance will collapse instantly. Always configure your storage policies to keep SAP database files pinned to the highest performance tier.

Summary and Key Takeaways

Designing SAP infrastructure is a balance between performance, reliability, and cost. By understanding the differences between simple volumes and striped volumes, you gain the ability to tailor your storage architecture to the specific needs of your SAP applications.

Key Takeaways:

1. Understand Your Workload: Not all SAP components have the same storage needs. HANA database files require high-throughput, low-latency storage (RAID 10), while transport directories and binary shares can tolerate simpler configurations.
2. Striping is for Performance: Use striping (RAID 0/10) when you need to exceed the throughput limits of a single disk. Always match your stripe size to the recommendations of your storage hardware and SAP.
3. LVM is Essential: Logical Volume Management is the industry standard for Linux-based SAP systems. It provides the flexibility to resize volumes, add disks, and manage snapshots without taking the system offline.
4. Avoid I/O Contention: Always separate your data, logs, and operating system binaries into distinct volume groups or physical storage sets to prevent them from competing for the same I/O bandwidth.
5. Monitor Proactively: Use tools like iostat and SAP's internal monitoring (DB02) to track latency and utilization. High latency is often a better indicator of performance issues than raw throughput numbers.
6. Alignment Matters: Ensure that your file system partitions are properly aligned with the underlying physical storage blocks to avoid unnecessary I/O operations and performance degradation.
7. Capacity Planning: Keep your database volumes below 85% capacity to ensure the file system has enough headroom to perform efficiently and avoid fragmentation.

By following these principles, you will build an SAP infrastructure that is not only fast but also reliable and easy to maintain. Remember that storage is the one component that is hardest to "fix" after the system is in production, so invest the time in the design phase to ensure your configuration is solid from the start.