In 2020 Pinterest was ingesting 1.7 TB of data daily, growing to 3 TB that year. Since then, data volumes haven’t just grown exponentially, they have exploded. Hundreds of terabytes per day is no longer some crazy number in 2022—you can draw a curve from there into the future. The big question now is not whether OpenSearch can support a few TB of data per day, rather what does OpenSearch need to look like to support hundreds of TB of data, and how soon? In this talk, we’ll present and explore some disruptive ideas for the next decade of OpenSearch.
