Resources

Frequently Asked Questions

Categories: General · Plugins · Upgrading
  1. 1. General

    1. 1.1 What is OpenSearch?

      OpenSearch is a fully open source search and analytics suite. OpenSearch includes OpenSearch (derived from Elasticsearch 7.10.2) and OpenSearch Dashboards (derived from Kibana 7.10.2).OpenSearch is the new home for the plugins and advanced features distributed with Open Distro for Elasticsearch. Open Distro for Elasticsearch will end with version 1.13

    2. 1.2 Why was OpenSearch created?

      Developers embrace open source software for many reasons, one of the most important is the freedom to use that software where and how they wish. Elastic ceased making open source options available for Elasticsearch and Kibana, releasing them under the Elastic license, with source code available under the Elastic License or SSPL. These are not open source and do not offer users the same freedoms. Because of this, we made the decision to create a fork from the last Apache 2.0 version of Elasticsearch and Kibana and provide OpenSearch under the Apache License, Version 2.0 (ALv2).

      ALv2 grants well-understood and permissive usage rights that match the freedoms people expect with open source software; freedoms such as being able to use, modify, extend, monetize, and resell the open source software where and how they want. For OpenSearch, we believe this will enable broad adoption and contributions benefiting all members of the community.

    3. 1.3 Why should I use OpenSearch?

      OpenSearch enables people to easily ingest, secure, search, aggregate, view, and analyze data. These capabilities have led it to be popular for use cases such as application search, log analytics, and more. With OpenSearch you benefit from having an open source product you can use, modify, extend, monetize, and resell how you want. At the same time, we will continue to provide a secure, high-quality search and analytics suite with a rich roadmap of new and innovative functionality.

    4. 1.4 Is OpenSearch suitable for production use?

      OpenSearch is currently in beta. We are working towards a release candidate and expect it to stabilize and be ready for production by early summer (mid-2021).

    5. 1.5 What license is OpenSearch released under?

      All of the software in the OpenSearch project is released under the Apache License, Version 2.0 (ALv2). The ALv2 license grants you well-understood usage rights for OpenSearch. You can use, modify, extend, embed, monetize, resell, and offer OpenSearch as part of your products and services. We have also published permissive usage guidelines for the OpenSearch trademark, so you can use the name to promote your offerings.

    6. 1.6 Is AWS equipped to maintain and advance a project like OpenSearch?

      When AWS decides to offer a service based on an open source project, we ensure that we are equipped and prepared to maintain it ourselves, if necessary. We bring years of experience working with Elasticsearch and Kibana codebases and have made upstream code contributions to both Elasticsearch and Apache Lucene (the core search library that Elasticsearch is built on). We have added several features in open source like security, alerting, anomaly detection, index state management, and trace analytics that are widely used and deployed in production by our community and customers. We are well equipped to maintain and advance the project ourselves. Also, the community-backed codebase will help accelerate new innovations and will allow everyone to move faster in improving stability, scalability, resiliency, and performance. Already many organizations including SAP, CapitalOne, RedHat, Logz.io, Aiven.io, Bonsai, Logit.io, Search Guard, and BAInsight have publicly backed OpenSearch.

    7. 1.7 What is the relationship between the existing Open Distro for Elasticsearch project and OpenSearch?

      The final version of Open Distro for Elasticsearch is version 1.13. Future versions of the plugins and advanced features distributed with Open Distro for Elasticsearch will be available in the OpenSearch project.

    8. 1.8 Is OpenSearch wire-compatible with Elasticsearch?

      Yes. OpenSearch is a fork of open source Elasticsearch 7.10. As such, it provides backwards REST APIs for ingest, search, and management. The query syntax and responses are also the same. In addition, OpenSearch can use indices from Elasticsearch versions 6.0 up to 7.10. We also aim to support the existing Elasticsearch clients that work with Elasticsearch 7.10.

      Note that while the OpenSearch API is backwards compatible, some clients or tools may include code, such as version checks, that may cause the client or tool to not work with OpenSearch.

      For more information on backwards compatibility, see upgrading FAQs.

    9. 1.9 Will the OpenSearch API change in future versions?

      All future OpenSearch 1.x releases will be backwards compatible with Elasticsearch 7.10, although some APIs will be deprecated. If functionality requires a breaking change, we will introduce a new major version of OpenSearch and provide tooling to make migrating to the new major version simple. When new features require adding APIs, we will work with the community to add support for these features in popular clients.

      For more details on deprecated APIs, see upgrading FAQs.

    10. 1.10 Do Elasticsearch clients like Logstash and Beats work with OpenSearch?

      Since OpenSearch is wire-compatible with Elasticsearch 7.10, any clients that currently work with Elasticsearch 7.10 should also work with OpenSearch. For details specific about open source Logstash and Beats compatibility please see the documentation.

      Please report any issues you experience with these clients or other clients in our project GitHub issues.

      For more questions related to upgrading clients, see upgrading FAQs.

    11. 1.11 Does OpenSearch include forks of Logstash and Beats?

      No. There is documentation on which versions of Logstash and Beats work with OpenSearch. We will be providing, more documentation on how to use them with OpenSearch and download links to OpenSearch compatible versions.

    12. 1.12 Can I upgrade from Elasticsearch to OpenSearch?

      Yes. You can upgrade using a rolling upgrade (one node at a time) process when upgrading from Elasticsearch versions 7.0 - 7.10 to OpenSearch. For Elasticsearch versions 6.x you will be required to perform a cluster restart upgrade. OpenSearch can use indices from Elasticsearch versions 6.0 up to 7.10. Indices on versions prior to Elasticsearch 6.0 or after 7.10 will need to be removed from a cluster being upgraded to OpenSearch or reindexed into a compatible version of Elasticsearch then upgraded to OpenSearch.

    13. 1.13 Which features of Elasticsearch do you plan to offer in the future? Is OpenSearch going to keep pace with the upstream Elasticsearch releases? How will this evolve?

      The future investments and plans for OpenSearch should be viewed as independent of Elasticsearch. Our roadmap will be driven by the needs of the community. Where possible, if new features in OpenSearch are similar to Elasticsearch, we will strive to make the APIs similar. The two projects, however, are distinct. The OpenSearch project is open source and is focused on providing the innovations that our community and customers ask for.

    14. 1.14 Can OpenSearch read indices from previous versions of Elasticsearch?

      Yes. OpenSearch is compatible with indices created from Elasticsearch versions 6.0 up to 7.10.

    15. 1.15 Will you be making contributions back to Elasticsearch open-source?

      No. Open source Elasticsearch development ended with 7.10 when it was moved to a non-open source license. OpenSearch software will move forward according to the needs of the community. That said, all OpenSearch software built for its customers and the community is released under the ALv2 license and we welcome anyone to use the software under the ALv2 terms and conditions.

    16. 1.16 I want to contribute, what can I do?

      We welcome contributions in multiple forms. You can submit pull requests, open issues, and leave comments on any of the OpenSearch GitHub repositories. You can join our community meetings. You can also leave comments in the community forum. If you have a specific way you would like to contribute and don’t know what steps to take, please leave a comment in our community forum.

    17. 1.17 How are new features, bug fixes, and other development decisions made?

      New software developed for OpenSearch is made based on project’s principles for development. To learn more, visit our website.

    18. 1.18 What will your governance model be? Are you looking at any foundations?

      At this time, there is not a plan to move OpenSearch in to a foundation. As we work together in the open, we expect to uncover the best ways to collaborate and empower all interested stakeholders to share in decision making. Cultivating the right governance approach for an open source project requires thoughtful deliberation with the community. We’re confident that we can find the best approach together over time. For now, AWS is the steward of OpenSearch. The principles of development define the guidelines for how decisions about the project are made. These principles will continue to be iterated and refined based on feedback.

    19. 1.19 How do I contribute functionality and get it on the public roadmap?

      If you want to add something to OpenSearch that is not in the public roadmap, that’s a perfect opportunity to contribute! There are a couple of things you can do.

      You can create a feature request in the relevant GitHub repo for the feature and socialize the request. People are always looking for in-demand features to build. A maintainer or someone else in the community may pick this feature up and work on it. As progress is made the maintainers of the repo will help get the feature onto the roadmap.

      Another option is to build the feature yourself. To do this create a proposal as a GitHub issue in the relevant repo and use the proposal template (thanks jkowall for contributing the template!). Offer your commitment to build it. The maintainers of the repo will work with you to figure out how best to proceed. That could be further discussion, design docs, or just starting to write the code. As the feature is developed, the maintainers of the repo will also work with you to incorporate it into the roadmap.

  2. 2. Plugins

    1. 2.1 Can I use OpenSearch plugins with the proprietary Elastic stack?

      The plugins are tested to work with OpenSearch. They have not been tested with Elastic’s proprietary software. As these plugins are open source, we do welcome anyone who wants to test them out with the Elastic Stack. However, we do not plan to invest in making the OpenSearch plugins work on the Elastic stack.

    2. 2.2 Can I install an OpenSearch plugin as a standalone plugin?

      Yes. You can install an OpenSearch plugin independently of the other plugins. For example, if you would like to use OpenSearch with only our security plugin installed, you can remove the other plugins using the OpenSearch plugin remove command.

    3. 2.3 Will older versions of Open Distro for Elasticsearch plugins still be available for open source Elasticsearch?

      Yes, all older Open Distro for Elasticsearch versions of plugins, from 0.7 to 1.13 will continue to remain available for download.

  3. 3. Upgrading

    1. 3.1 How can I upgrade an Elasticsearch OSS and Kibana OSS cluster with multiple nodes to OpenSearch and OpenSearch Dashboards?

      OpenSearch supports rolling upgrades in the same way as Elasticsearch OSS. You can deploy OpenSearch into a mixed cluster with Elasticsearch OSS or Open Distro for Elasticsearch nodes. One by one you can replace the legacy nodes with little to no additional manual work.

      In the same way as Kibana OSS, OpenSearch Dashboards does not support rolling upgrades, but it supports restart upgrades. You are able to stop all Kibana OSS instances, deploy a new OpenSearch Dashboards instance and direct traffic to it.

    2. 3.2 How can I upgrade a single node deployment of Elasticsearch OSS and Kibana OSS to OpenSearch and OpenSearch Dashboards?

      You are able to stop Elasticsearch OSS and Kibana OSS, install OpenSearch and OpenSearch Dashboards, manually configure those to point to your Elasticsearch OSS and Kibana OSS data, review and potentially update settings, then start OpenSearch with OpenSearch Dashboards.

    3. 3.3 Which versions of Elasticsearch OSS and Kibana OSS can I upgrade from to OpenSearch and OpenSearch Dashboards, directly?

      Elasticsearch OSS and Kibana OSS 6.8.0 to 7.10.2 and Open Distro for Elasticsearch (ODFE) 1.x.

    4. 3.4 Can I upgrade older versions of Elasticsearch OSS and Kibana OSS to OpenSearch and OpenSearch Dashboards?

      Elasticsearch OSS and Kibana OSS 5.x up to 6.7.2 can be first upgraded to 6.8.0, then it is recommended to upgrade to Elasticsearch OSS 7.10.2 or ODFE 1.13, before upgrading to OpenSearch and OpenSearch Dashboards.

      Note that the minimum supported index version for OpenSearch is 6.0. So, all the 5.x indices have to be re-indexed, before upgrading to OpenSearch.

    5. 3.5 Can I upgrade a non-OSS Elasticsearch and Kibana cluster to OpenSearch?

      Yes. However, functionality not available in Elasticsearch OSS and Kibana OSS continues to not be available. We recommend evaluating additional features available in OpenSearch that may provide similar functionality.

    6. 3.6 Can I upgrade from Elasticsearch and Kibana 7.11 or other versions released after the fork?

      No, this will not be supported in OpenSearch 1.0.

    7. 3.7 Will my ODFE plugins continue to work after upgrade?

      OpenSearch plugins based on the Open Distro for Elasticsearch (ODFE) plugins are included in OpenSearch 1.0, and are functionally backwards compatible with their predecessors.

    8. 3.8 At what point is the upgrade process complete?

      When all nodes are running OpenSearch and OpenSearch Dashboards 1.0.

    9. 3.9 Does an upgrade to OpenSearch require downtime?

      You can perform rolling upgrades from Elasticsearch OSS to OpenSearch, which does not require downtime. Kibana OSS upgrades require a restart which will cause downtime for Kibana OSS and OpenSearch Dashboards. If you have a single-node deployment and wish to upgrade it in-place, you will incur downtime.

    10. 3.10 Can I do a Blue/Green upgrade with data migration?

      Yes. You can choose to do a Blue/Green upgrade instead of a rolling upgrade.

    11. 3.11 Can I install elasticsearch-xyz-plugin in OpenSearch?

      An Elasticsearch plugin that depends on Elasticsearch JARs will not work without code changes to depend on OpenSearch JARs.

    12. 3.12 Can I rollback an upgrade half-way in progress (e.g. 2/4 nodes are running OpenSearch)?

      This is possible and has caveats. For example, upgrading Kibana OSS to OpenSearch Dashboards migrates the Kibana OSS indexes and does not allow rollback.

    13. 3.13 Are there any changes required for my existing Elasticsearch OSS clients to continue to work?

      While the OpenSearch API is backwards compatible, some clients or tools may include code, such as version checks, that may cause the client or tool to not work with OpenSearch.

    14. 3.14 Can I run a heterogeneous mixed OpenSearch and Elasticsearch OSS cluster?

      This only supported in OpenSearch for the purpose of a rolling upgrade.

    15. 3.15 Can I run a heterogeneous mixed OpenSearch Dashboards and Kibana cluster?

      No. This is not supported.

    16. 3.16 Is OpenSearch security enabled by default after an upgrade?

      Yes. Security is enabled by default.

    17. 3.17 What settings change during the upgrade?

      Environment variables that contain branded words such as ES or OPENDISTRO have been renamed.

      Cluster settings that contain branded words such as es. or opendistro. can continue to be used but have been deprecated.

    18. 3.18 What REST APIs change during the upgrade?

      REST APIs that contain branded words such as ES or OPENDISTRO have been deprecated.

    19. 3.19 How do I upgrade or migrate secure or system indices?

      There is no need to migrate secure or system indices for the upgrade to OpenSearch 1.0. This is because theses indices are not being renamed. In future upgrades, index aliases or migrations may be introduced for secure or system indices.

    20. 3.20 Is there binary compatibility between Elasticsearch-OSS and OpenSearch?

      While an OpenSearch node is able to join an Elasticsearch OSS cluster, namespaces and class names in OpenSearch have been changed. If your plugin code depends on Elasticsearch OSS JARs, you will need to upgrade those dependencies to OpenSearch JARs.

    21. 3.21 Where can I find more information and track progress on ensuring backwards compatibility?

      The umbrella issue for backwards-compatibility is OpenSearch#671. Issues across opensearch-project repositories are labeled backwards-compatibility.

    22. 3.22 Will Kibana work on top of OpenSearch 1.0?

      No. You will need to upgrade to OpenSearch Dashboards.

    23. 3.23 I have built a custom plugin for Elasticsearch, will it run without modification on OpenSearch?

      Because OpenSearch is backwards compatible, it will. If you find identify an incompatibility, please open and issue in OpenSearch.

    24. 3.24 Can I install kibana-xyz-plugin that only uses REST APIs in OpenSearch Dashboards 1.0?

      While the APIs have not changed, we have not tested kibana-xyz-plugin.

    25. 3.25 I have built a custom visualization for Kibana, will it be usable with OpenSearch Dashboards without modification?

      Because OpenSearch Dashboards is backwards compatible, it will. If you find identify an incompatibility, please open and issue in OpenSearch Dashboards.

    26. 3.26 Can OpenSearch read indices from previous versions of Elasticsearch?

      Yes. OpenSearch is compatible with indices created from Elasticsearch versions 6.0 up to 7.10.

    27. 3.27 Will the OpenSearch API change in future versions?

      All future OpenSearch 1.x releases will be backwards compatible with Elasticsearch 7.10. If functionality requires a breaking change, we will introduce a new major version of OpenSearch and provide tooling to make migrating to the new major version simple. When new features require adding APIs, we will work with the community to add support for these features in popular clients.