ST06 (OS Monitoring)

Detailed Notes for ST06 (OS Monitoring) in SAP Basis

ST06: Operating System Monitor

Purpose: ST06 (Operating System Monitor) is a vital transaction in SAP Basis for monitoring the performance and resource utilization of the underlying operating system (OS) where the SAP application server is running. It provides a holistic view of CPU, memory, disk I/O, and network activity, helping administrators identify potential bottlenecks and ensure the smooth operation of the SAP system.

How ST06 Collects Data:

ST06 retrieves OS performance data using a program called SAPOSCOL (SAP OS Collector).

• SAPOSCOL: This is an independent, executable program that runs as a background process (daemon or service) on each SAP application server host. It collects OS performance data directly from the operating system's kernel and stores it in a shared memory segment.
• SAP System Integration: The SAP system's work processes read this data from the shared memory segment and display it in ST06. SAPOSCOL is typically started automatically when the SAP instance starts.

Key Information Displayed in ST06:

ST06 presents various views to analyze OS performance:

1. Overview (Initial Screen):

   • CPU Utilization: Breakdown of CPU usage (e.g., user, system, idle, wait). High "user" or "system" CPU and low "idle" indicate CPU bottlenecks. High "wait" can indicate I/O bottlenecks.
   • Load Average: Average number of processes waiting for CPU time over 1, 5, and 15 minutes. High values suggest CPU saturation.
   • Memory Usage: Total, free, used physical memory and swap space. High swap usage indicates memory pressure.
   • Disk Activity (I/O): Number of read/write operations per second, transfer rates. High values can indicate I/O bottlenecks.
   • Network Traffic: Data sent/received per second.
   • Top CPU Processes: Lists processes consuming the most CPU (useful for identifying rogue processes).
   • Operating System: Type and version of the OS.

2. Detail Analysis Menu (accessed via tabs or dropdowns, depending on SAP version):

   • CPU: More granular CPU utilization over time, per-processor statistics.
   • Memory: Detailed breakdown of memory usage (physical, swap, paging activity), memory allocation by processes.
   • Disk: Detailed disk I/O statistics per file system or logical volume (reads/writes, queue length, response times).
   • Network: Detailed network interface statistics (packets/errors per second, collisions).
   • File System: Usage of mounted file systems.
   • Processes: List of all running OS processes, their CPU, memory usage, and PID. This is similar to OS-level commands like top or Task Manager.
   • Configuration: Displays basic hardware and OS configuration details.
   • LAN: Network interface statistics.
   • Paging: Detailed paging and swap activity.

Important Metrics to Watch:

• CPU Idle: Should ideally be above 30-40%. If consistently low (e.g., <10%), the CPU is a bottleneck.
• CPU Wait: If high, it indicates processes waiting for I/O operations, suggesting a disk bottleneck.
• Load Average: For a single-core system, a load average above 1 can mean saturation. For multi-core, ideally, it should be less than the number of CPU cores.
• Swap Space Used: Should be minimal or zero. High swap usage severely degrades performance.
• Disk Busy % (or Util %): Percentage of time the disk is busy. Consistently high values (e.g., >80%) indicate an I/O bottleneck.
• Paging Activity: High paging (pages in/out) signifies memory pressure.

Actions and Features in ST06:

• Refresh: Updates the displayed data with the latest information from SAPOSCOL.
• History Data: View historical OS performance data (daily, hourly). This is crucial for trend analysis and identifying recurring issues.
• OS Command: Some versions allow executing basic OS commands (e.g., ping, ls) directly from ST06 (requires specific authorization and configuration).
• Display Alerts: If configured, ST06 can display OS-level alerts.

Important Considerations/Best Practices for ST06:

• Regular Monitoring: Incorporate ST06 into your daily/weekly health checks.
• Baseline Data: Establish a performance baseline for your system during normal operations. This helps identify deviations.
• Correlate with SAP Performance: Always correlate OS performance data from ST06 with SAP-level performance data from other transactions (e.g., ST03N for workload, SM50/SM66 for work processes, ST04 for database performance). A problem at the OS level often manifests as SAP performance degradation.
• Troubleshooting:
  • High CPU: Identify top CPU-consuming processes (both SAP and non-SAP). If SAP work processes are consuming high CPU, further investigate in SM50/SM66 and ST03N. If non-SAP processes, investigate OS-level.
  • Memory Issues: High swap, high paging indicates memory exhaustion. Check total physical RAM, em/initial_size_MB, and abap/heap_area_* parameters.
  • I/O Bottlenecks: High disk busy %, high CPU wait time. Investigate disk configurations, storage (SAN/NAS), and database I/O (ST04).
• SAPOSCOL Status: Ensure SAPOSCOL is running on all application servers. You can check its status from ST06 (via the "Tools" or "Goto" menu, depending on version) or at the OS level. If it's not running, ST06 will show outdated or no data.
• Patching SAPOSCOL: Keep SAPOSCOL updated to the latest available version via SAP Support Portal. Newer versions often include bug fixes, performance improvements, and support for newer OS versions.

---

Important Configurations to Keep in Mind (Related to ST06/SAPOSCOL)

1. SAPOSCOL Installation and Status:

   • Location: SAPOSCOL (saposcol.exe on Windows, saposcol on Unix/Linux) is located in /usr/sap/PRFCLOG (or C:\usr\sap\PRFCLOG on Windows).
   • Execution: It runs as a daemon/service. You can check its status and start/stop it from the OS command line (saposcol -s, saposcol -k, saposcol -l).
   • Permissions: Ensure SAPOSCOL has correct OS permissions to read performance data and write to shared memory.
   • Shared Memory: SAPOSCOL uses a shared memory segment to store data. If there are issues with shared memory, SAPOSCOL might not function correctly.
   • Configuration Note: Always verify SAPOSCOL is running on every SAP application server host. If ST06 shows outdated data, SAPOSCOL is likely the culprit.

2. Data Collection Interval:

   • SAPOSCOL collects data at a fixed interval (e.g., every 10 seconds). This interval is generally not configurable by Basis.
   • Configuration Note: Be aware that ST06 data is not real-time down to the millisecond but is based on these collection intervals.

3. OS Kernel and Patches:

   • Ensure your OS is at a supported kernel level for your SAP release.
   • Configuration Note: OS-level patches related to performance, networking, or memory management can significantly impact SAP system performance and should be applied as per vendor recommendations.

4. Hardware Sizing:

   • Proper sizing of CPU, RAM, and disk I/O capacity is fundamental. ST06 helps validate if your current hardware meets the workload demands.
   • Configuration Note: If ST06 consistently shows bottlenecks, it's a strong indicator that hardware upgrade or re-sizing might be necessary.

5. Disk Configuration (RAID levels, SAN/NAS):

   • The underlying storage configuration heavily impacts disk I/O performance.
   • Configuration Note: Using appropriate RAID levels (e.g., RAID 10 for performance-critical data) and ensuring sufficient throughput from SAN/NAS storage is crucial.

6. Network Configuration:

   • Network bandwidth, latency, and error rates (visible in ST06) impact distributed SAP systems.
   • Configuration Note: Ensure proper network infrastructure and configuration (e.g., full-duplex, appropriate NIC drivers).

7. Alerting (CCMS/Solution Manager):

   • OS performance metrics collected by SAPOSCOL can be integrated into CCMS (RZ20) or SAP Solution Manager for proactive alerting.
   • Configuration Note: Configure alerts for critical thresholds (e.g., CPU idle < 10%, swap usage > 0, disk busy > 80%) to be notified of issues automatically.

---

30 Interview Questions and Answers (One-Liner) for ST06

1. Q: What is the primary purpose of ST06?

   • A: To monitor the operating system performance of SAP hosts.

2. Q: What is SAPOSCOL?

   • A: A program that collects OS performance data for ST06.

3. Q: Where does SAPOSCOL store the collected data?

   • A: In a shared memory segment.

4. Q: Which metric in ST06 indicates the overall system workload over time?

   • A: Load Average.

5. Q: What does a consistently low "CPU Idle" percentage in ST06 suggest?

   • A: A CPU bottleneck.

6. Q: What does a high "CPU Wait" percentage in ST06 indicate?

   • A: An I/O bottleneck (processes waiting for disk/network).

7. Q: How do you check memory utilization in ST06?

   • A: Look at "Physical Memory" and "Swap Space" usage.

8. Q: What does high "Swap Space Used" in ST06 signify?

   • A: The system is running out of physical RAM and using disk as memory.

9. Q: How can you check disk I/O activity in ST06?

   • A: Look at "Disk Busy %" or "Reads/Writes per second".

10. Q: What does a "Disk Busy %" consistently above 80% indicate?

    • A: An I/O bottleneck.

11. Q: Can ST06 show you network traffic?

    • A: Yes, it shows data sent/received and error rates.

12. Q: How can you view historical OS performance data in ST06?

    • A: Through the "History" or "Detail Analysis Menu" options.

13. Q: What is the first thing to check if ST06 shows outdated or no data?

    • A: Check if SAPOSCOL is running on the OS.

14. Q: Where is SAPOSCOL typically located on the OS?

    • A: /usr/sap/PRFCLOG (Unix/Linux) or C:\usr\sap\PRFCLOG (Windows).

15. Q: How do you start/stop SAPOSCOL from the OS command line?

    • A: saposcol -s (start), saposcol -k (stop).

16. Q: Why is it important to keep SAPOSCOL updated?

    • A: For bug fixes, performance improvements, and OS compatibility.

17. Q: Can you see individual OS processes and their resource consumption in ST06?

    • A: Yes, in the "Processes" detail view.

18. Q: What is the purpose of correlating ST06 data with ST03N data?

    • A: To understand if OS bottlenecks are impacting SAP workload performance.

19. Q: What should be the ideal "Swap Space Used" value in ST06?

    • A: As close to zero as possible.

20. Q: If CPU Wait is high, what type of bottleneck should you investigate?

    • A: Disk I/O bottleneck.

21. Q: How frequently does SAPOSCOL collect data?

    • A: At fixed intervals, typically every 10 seconds.

22. Q: Does ST06 show physical memory usage or virtual memory usage?

    • A: Both, but emphasizes physical and swap space.

23. Q: What is the command to check the status of SAPOSCOL at the OS level?

    • A: saposcol -s.

24. Q: Can ST06 help identify if a non-SAP process is consuming too many resources?

    • A: Yes, in the "Processes" detail view by sorting on CPU/Memory.

25. Q: What is a "Load Average" of 2.0 on a single-core CPU likely indicating?

    • A: Severe CPU saturation.

26. Q: Is it possible to configure ST06 to send alerts for OS performance issues?

    • A: Yes, via CCMS (RZ20) or Solution Manager.

27. Q: What is the significance of the "Paging" metrics in ST06?

    • A: Indicates how much data is being moved between RAM and swap space.

28. Q: How does proper hardware sizing relate to ST06 monitoring?

    • A: ST06 helps validate if current hardware is sufficient or if upgrades are needed.

29. Q: Which component provides the data displayed in ST06?

    • A: SAPOSCOL.

30. Q: If a user complains about slow SAP system response, what is one of the first things you check in ST06?

    • A: CPU utilization, Memory (especially swap), and Disk I/O.

---

5 Scenario-Based Questions and Answers for ST06

1. Scenario: Users are complaining about the SAP system being very slow, especially during peak hours. You check ST06 and observe that "CPU Idle" is consistently below 5%, "Load Average" is very high (e.g., 10+ on an 8-core system), and the "Processes" view shows multiple SAP work processes consuming high CPU.

   • Q: What is the most likely bottleneck, and what immediate steps would you consider?
   • A:
     • Bottleneck: CPU saturation. The system's CPU resources are fully utilized.
     • Immediate Steps:
       1. Check ST03N to identify the specific programs/transactions causing high workload.
       2. In SM50/SM66, identify if specific work processes are running long-running reports or complex transactions.
       3. If possible, reschedule heavy background jobs to off-peak hours.
       4. Consult with functional teams or developers to optimize resource-intensive programs.
       5. Consider adding more application servers or upgrading CPU resources if it's a persistent problem.

2. Scenario: You notice in ST06 that "Swap Space Used" is increasing steadily throughout the day, and "Paging Activity" is high. CPU usage seems moderate.

   • Q: What does this indicate, and what Basis parameters would you investigate?
   • A:
     • Indication: Memory exhaustion. The system is running out of physical RAM and is frequently swapping data to disk, severely impacting performance.
     • Parameters to Investigate:
       1. em/initial_size_MB and em/max_size_MB (Extended Memory).
       2. abap/heap_area_dia and abap/heap_area_nondia (ABAP Heap Memory).
       3. ztta/roll_area and rdisp/roll_max_size (Roll Memory).
       4. Also, verify the total physical RAM on the server is sufficient.

3. Scenario: ST06 shows normal CPU and memory usage, but "CPU Wait" is consistently high (e.g., 30-40%), and "Disk Busy %" for specific file systems is near 100%.

   • Q: What is the likely performance bottleneck, and what actions would you take?
   • A:
     • Bottleneck: Disk I/O. Processes are spending significant time waiting for disk operations to complete.
     • Actions:
       1. Identify which file systems/disks are heavily utilized (e.g., database data files, log files, SAP work directory).
       2. Check ST04 (Database Monitor) for database-specific I/O statistics.
       3. Consult with storage administrators to investigate the underlying storage (SAN/NAS) performance, RAID configurations, or disk throughput.
       4. Consider relocating heavily accessed files or upgrading storage infrastructure.

4. Scenario: You log into ST06, but the data displayed is several hours old, or no data is shown at all. All other SAP transactions are working fine.

   • Q: What is the immediate suspicion, and how would you verify and resolve it?
   • A:
     • Suspicion: SAPOSCOL is not running or is malfunctioning on that host.
     • Verify & Resolve:
       1. At the OS level, navigate to /usr/sap/PRFCLOG (or C:\usr\sap\PRFCLOG).
       2. Check SAPOSCOL's status: saposcol -s.
       3. If not running, try to stop (saposcol -k) and then restart (saposcol -l) it.
       4. Check the dev_saposcol trace file in the same directory for errors.
       5. Ensure correct OS permissions for the PRFCLOG directory and saposcol executable.
       6. If issues persist, consider updating SAPOSCOL to the latest version.

5. Scenario: During a major data migration project, you notice a sudden spike in network traffic in ST06 on one of your application servers. No other typical OS bottlenecks (CPU, memory, disk) are observed.

   • Q: What could be causing this, and what steps would you take to investigate further?
   • A:
     • Cause: A significant data transfer activity is likely occurring, possibly an RFC call to another system, a file transfer, or a database connection to a remote database.
     • Investigation Steps:
       1. Check the "Processes" view in ST06 to see if any specific process (SAP or non-SAP) is consuming high network resources.
       2. In SM50/SM66, look for work processes with "RFC" or "CPIC" status, indicating external communication.
       3. Review background jobs (SM37) for any active data migration or interface jobs running on that server.
       4. Check network interface statistics in ST06 for errors or collisions, indicating potential network issues.
       5. Consult with the project team to understand the ongoing data migration activities.

---