Hindsight is 20-20. But that is only true for event data if hindsight comes with historical data to interrogate, historical patterns to observe, and the tools with which to interpret them. Traditionally, hindsight for event data has been unaffordable, unmanageable, and too slow to be useful. In a world where event data indexes grew in size to be multiples of the original data, hindsight has been pragmatically limited to 30 days for most companies.
That was then.
Today, Scalyr Hindsight, provides a new service built on Scalyr’s Event Data Cloud, complementing Scalyr’s Log Analytics, and offering customers the ability to retain event data indefinitely at low S3 pricing and to query it on-demand on a pay-per-terabyte scanned model.
The service is low-cost, easy-as-SaaS, and performant. Customers are able to use their own S3 buckets for retention (data is moved into Scalyr for analytics and visualization) or let Scalyr manage storage. Scalyr’s rich tool set includes dashboards and visualizations, PowerQueries for complex analytics and joins, team-based data-retention policies and segmentation, role-based access and control, powerful tools for event analytics and management. And it’s optimized for event data and its use cases.
Just as Hindsight delivers ad hoc queries on event data, Scalyr leverages its materialized-views engine (the Scalyr Summary Service) to maintain things like repetitive searches and dashboards. This means Hindsight can be an attractive alternative to streaming analytics solutions like Druid or Clickhouse, and it avoids the dimensionality curse that plagues other solutions.
At Scalyr, we prize service performance as highly as we do affordability, so there is no ‘cold’ data in Hindsight (see our description of the Event Data Cloud architecture that underpins Hindsight for how we accomplish this).
Hindsight is 20-20. But that is only true for event data if hindsight comes with historical data to interrogate, historical patterns to observe, and the tools with which to interpret them.
At Scalyr, we prize service performance as highly as we do affordability, so there is no ‘cold’ data in Hindsight (see our description of the Event Data Cloud architecture that underpins Hindsight for how we accomplish this).
Hindsight marries the two prevalent business models in the observability market: the subscription model (that offers unlimited queries for a specified duration tuned to SRE use cases), and the on-demand model that charges for storage (at S3 rates) and individual queries based on TB scanned. Retention policies can be applied to the data, so that data is retained for the required period, and deleted at the end of that period. This way, Scalyr meets the event data needs of the SRE and devops communities for unlimited analytics for 30 days (or less) and the data science needs of enterprises to retain and model event data over seasons and even multiple years for security, compliance and analytical purposes.
Initial customer use cases for Hindsight include security, compliance, ML and AI model-building, performance and behavioral analysis,and forecasting, as well as real time stream processing, transformation, and analysis.
Scalyr has opened the Early Access program for Hindsight and is welcoming customers and interested prospects. What could you do with 20-20 Hindsight on event data?
Contact us to learn more.
