Discussing open-source innovation at Open Source Summit North America
This week I had the honor of speaking about OpenSearch at Open Source Summit North America, organized by the Linux Foundation. This annual conference brings together open-source technologists, business leaders, academics, and others with a rich program of educational sessions and colocated events, and the OpenSearch Project is proud to be a sponsor.
The Summit’s program of keynote presentations includes a unique format: a five-minute presentation that, as a speaker, is both challenging and very exciting. I decided to devote my time to talking about open source and innovation.
As an open-source project, innovation at OpenSearch starts with and is sustained by our community. That’s why continuing to grow and support that community is an important priority both for the project and for the leadership committee on which I serve.
As I shared in my remarks at Open Source Summit, we’ve made a lot of progress in growing the community that drives OpenSearch, with more than 2,000 contributors who sit outside of sponsor organization Amazon Web Services, from independent contributors to those from some of the biggest organizations in software and services, including Oracle, Aiven, Aryn, ByteDance, SAP, Intel, IBM, and more. We also see more than 100 new contributions per week on average from the community. The community of maintainers continues to grow as well, with over 200 maintainers spanning 19 organizations.
I also noted that there is more work for us to do to nurture and enable that community. If you’ve been following the project, you might have noticed some examples of this as we strive to be more open and transparent with our governance model and as we bring more and more of the project’s operations and decision-making into the open. In recent months, we’ve shifted our release meetings, security triage meetings, operations meetings, and other vital and ongoing discussions into the public for open participation. Trust that the project will continue to prioritize open communication and collaboration as one way to help foster this active and growing community.
During the talk, I highlighted the kinds of innovation the community has helped deliver to enhance OpenSearch’s efficiency, scalability, and durability. Features like segment replication and remote-backed storage, along with a host of optimizations to the query and indexing layers of OpenSearch, are just a few recent examples. Going forward we will continue to advance the OpenSearch core with functionality like concurrent segment search and much more with the community’s support.
I also talked about the project’s advancements in the area of log analytics, such as integrating OpenSearch with Apache Spark to allow querying operational observability and security data stored in data lakes. Further expanding the project’s utility for analytics and observability use cases are recent data source integrations with Prometheus as well as OpenTelemetry, Grafana, and more.
Search and generative AI are also areas of ongoing innovation within the project, with significant support from our partners and the broader community. One example is the new OpenSearch Assistant Toolkit, which enables developers to build generative AI experiences inside of OpenSearch Dashboards. Our new flow framework, launched this month, empowers users to create AI-augmented workflows by automating the work of accessing and configuring machine learning resources, and we continue to build upon the powerful combination of lexical and neural search through OpenSearch’s hybrid search capabilities. Our latest release brought performance improvements for the k-NN algorithms that power our vector search functionality, offering as much as a 50 percent reduction in memory footprint for k-NN indexes.
As I shared with the attendees at the Open Source Summit, continuing to build on the innovation behind OpenSearch takes a growing and vibrant community. If you aren’t actively engaged with OpenSearch right now, we would love for you to join our OpenSearch community, collaborate, and contribute. There are many ways to participate, including our community forums, Slack channels, and events we are having this year like OpenSearchCon Europe, OpenSearchCon India, and soon-to-be-announced OpenSearchCon North America. Thank you for your interest in OpenSearch—I hope we get a chance to work together!
