In smaller cloud and microservice-based systems, observability feels straightforward—logs, traces and metrics usually align to tell a clear story. Strong observability helps teams catch failures early, trace dependencies and sustain performance as workloads grow. But at hyperscale, clarity can give way to chaos as data pours in from thousands of microservices. Even the most advanced monitoring tools often struggle to handle the volume and velocity of telemetry data.
To keep visibility from becoming a liability, organizations must rethink how they design, collect and interpret observability signals across distributed systems. Here, members of Forbes Technology Council share the observability challenges that intensify at hyperscale—and the strategies that can help teams preserve clarity, control and confidence.
1. Mitigate Concentration Risk Through Load Decoupling
Concentration risk from a cloud customer can be a challenge for hyperscalers. This is especially true when key customers (like a major retailer on Black Friday or an AI company training a large model) concentrate their load in a single region; they can saturate the shared physical resources faster than the hyperscaler’s auto-scaling can respond. Finding and decoupling affected CSCs can help address this. – Akash Verma, Google
2. Control Cardinality Explosion With Smarter Sampling
At hyperscale, observability suffers from cardinality explosion—too many unique labels and traces creating noise and cost. The solution: Focus on SLO-driven metrics, use tail-based sampling, enforce cardinality limits and aggregate at the edge. This keeps observability efficient and focused on what truly matters. – Dr. Sanjay Kumar, City of New Orleans
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?
3. Centralize Telemetry For Scalable Insight
At hyperscale, observability requires keeping vast telemetry data like logs, metrics and traces usable and cost-efficient. Storing it under one roof in an accessible, scalable and performant fashion lets organizations run AI and analytics directly from their telemetry data, spotting anomalies, problem areas and threats while future-proofing their infrastructure for data-intensive workloads. – Garima Kapoor, MinIO
4. Balance Trace Depth With Cost Through Targeted Sampling
Traces are vital for observability but become costly and hard to manage at hyperscale. Cloud tools can get unsustainably expensive, while self-managed options may result in heavy operational overhead. Two key mitigations: 1. sample traces to capture representative system behavior; 2. pair them with well-designed logging using consistently propagated transaction IDs. – Elliott Cordo, Data Futures
5. Filter The Noise To Focus On High-Value Signals
In manufacturing, the amount of data coming from machines, sensors and inspection systems can be overwhelming. You’ve got temperature readings, torque specs, vibration data, dimensional checks and, sometimes, thousands of data points per part. At scale, this creates a similar problem to what we see in cloud systems: too much noise, not enough signal. Focus on what matters most, and tune in to catch it. – Alexander Kwapis, FusionPKG, an Aptar Beauty Company
6. Contain Complexity With Smarter Microservice Limits
The complexity of managing microservices doesn’t scale linearly with the number of microservices—it scales exponentially. Mitigation requires a multipronged strategy: Limit the number of microservices; use traditional approaches where a sufficient observability strategy should be robust, yet lightweight; democratize observability-based ops, tools and skills in the organization; and exploit AI for heavy lifting and ops automation. – Mrutyunjay Mohapatra, Alix Partners
7. Secure Data In Motion With Holistic DSPM Practices
While protecting data at rest is crucial, keeping it secure as it moves across platforms, devices and users is even more important—especially as distributed cloud environments continue to grow. A competent, holistic approach to DSPM helps ensure that sensitive data is monitored, tracked and secured wherever it travels, from remote worker desktops to independent AI-driven microservice solutions. – Thyaga Vasudevan, Skyhigh Security
8. Unify Tracing And Compliance Across Microservices
At hyperscale, tracing requests across thousands of microservices creates blind spots. Use OpenTelemetry for unified instrumentation, enforce trace context propagation and apply smart sampling. Track AI model versioning, log usage and monitor drift. Embed AI observability tools with alerts. Strong observability and governance ensure performance, trust and compliance at scale. – Madhavi Najana, Federal Home Loan Bank Of Cincinnati
9. Align Container Costs With Business Outcomes
At hyperscale, container cost attribution is a major FinOps challenge due to shared resources and ephemeral workloads. Solving it requires consistent tagging, automated cost allocation tools and aligning spend with business metrics via service-level telemetry. We’ve implemented tools like IBM’s Cloudability to successfully address this challenge. – Kim Bozzella, Protiviti
10. Correlate Data Layers Through An ‘Observability Mesh’
At hyperscale, observability drowns in signal noise from too many logs, traces and metrics with no context. The fix is shift-left instrumentation and an “observability mesh” that correlates data across layers, applying AI/ML to surface anomalies and root causes, not just raw events. – Sai Krishna Manohar Cheemakurthi, U.S. Bank
11. Predict Dependency Drift Before It Breaks Systems
One challenge is ephemeral dependency drift. At hyperscale, microservices vanish fast, breaking dependency maps and hiding failure roots. It’s like chasing ghosts in a storm. Fix it with real-time dependency snapshots and AI to predict drift patterns. Teams see the true service web, catch issues early and keep apps humming, no matter how wild the cloud gets. – Durga Krishnamoorthy, Cognizant Technology Solutions
12. Maintain Visibility With Adaptive Trace Sampling
At hyperscale, one hidden challenge is observability loss. Long call chains get cut off by trace sampling, so the most complex workflows go dark. The fix is adaptive sampling and targeted instrumentation that keep business-critical paths fully visible, ensuring teams see the whole story, not just fragments. – Priyadarshini Balachandran, Walmart Global Tech
13. Consolidate Fragmented Signals For End-To-End Clarity
At hyperscale, stitching together a clear view from fragmented logs, metrics and traces becomes a major challenge. Signals flood in from countless services, drowning insight in noise. One fix: a unified observability platform with correlation logic and contextual tagging. It turns chaos into clarity, helping teams spot issues faster, trace root causes and keep systems steady under pressure. – Maman Ibrahim, EugeneZonda Cyber Consulting Services
14. Simplify Distributed Tracing With Edge Aggregation
Distributed tracing becomes exponentially complex at hyperscale due to massive data volume and cross-service dependencies. By implementing strategies like prioritizing error paths, using context-aware trace aggregation and deploying edge collectors that preprocess telemetry before central ingestion, storage costs can be reduced while maintaining diagnostic capability for critical transactions. – Saurabh Saxena, Amazon Web Services
15. Prioritize Signal Fidelity Over Data Volume
When microservice architectures reach hyperscale, the core observability challenge isn’t data collection, but signal fidelity. Billions of logs and traces can overwhelm dashboards, hiding the causal patterns teams need most. The answer is context-rich observability—correlating telemetry with business KPIs and layering anomaly detection—so insights rise above the noise and drive real action. – To Quang Duy, Newwave Solutions JSC
16. Restore Context To Speed Root-Cause Detection
At hyperscale, pinpointing failures is tough as metrics explode, traces break across queues and alerts add noise. The solution is context, not just more dashboards. Standardize trace IDs, apply smart sampling and connect observability data with code changes and business goals. This restores end-to-end visibility, reduces noise and speeds up root-cause resolution for large-scale systems. – Arun Goyal, Octal IT Solution LLP
17. Define ‘Golden Signals’ To Focus On Critical Data
At hyperscale, observability faces signal overload—millions of metrics across microservices create noise. It’s like tracking every sensor on the International Space Station—without prioritization, vital alerts get lost. Leaders can define “golden signals,” enforce cardinality budgets and apply sampling to spotlight anomalies—focusing on mission-critical data while controlling complexity. – Shelli Brunswick, SB Global LLC
18. Track Configuration Drift As A Core Telemetry Stream
Dynamic configuration drift in self-healing autoscaling microservice systems is one significant problem to solve. Some effective solutions are implementing runtime state introspection, tracking configuration drift as a first-class telemetry stream, a temporal drift replay engine, augmented observability pipelines with declarative shadowing, and suppressing drift-sensitive alerts. – Balaji Soundararajan, Adroitts
19. Leverage AIOps To Accelerate Event Correlation
At hyperscale, the number of unique dimensions in observability data (metrics, logs and traces) increases dramatically and breaks storage, cost and performance. The primary challenge is the event correlation and root cause detection that are critical for reducing MTTD and MTTR for applications. AIOps with products like BigPanda can address this at the speed and scale that business demands. – Ashish Anand, Marriott International
20. Connect Observability Silos With Context Propagation
At hyperscale, correlating signals across fragmented services is a major observability challenge. Metrics, logs and traces often live in silos, making root cause analysis slow. Implementing distributed tracing with unified context propagation across services helps pinpoint issues faster and links performance directly to user impact. – Hemanth Volikatla, SAP America INC.