Populate Elasticsearch Index
This Page Shows How To Populate an Elasticsearch Index using Rules
Last updated
This Page Shows How To Populate an Elasticsearch Index using Rules
Last updated
This documentation is based on the FoodCrunch use case. Please open the link below alongside this page to understand the examples.
Introduction and Use CaseIf you are new to Rules, we highly recommend reading the following document before continuing:
RulesThe steps provided here are applicable to both self-hosted installations and Squidex Cloud.
In order to create this rule you will need a working installation of Elasticsearch. Elasticsearch can be installed either as a Docker container, or in Kubernetes. Refer to the following links for installation instructions:
Docker installation: https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html
Kubernetes Installation: https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-installing-eck.html
In this example, we are using the cloud version of ElasticSearch at https://elastic.co. You can subscribe to a 14 day trial of the Elasticsearch cloud here.
Squidex also provides an action for OpenSearch. The reason OpenSearch is also available is because Elasticsearch blocks all non-official drivers.
OpenSearch action has the same logic and works similarly.
Select your Elastic deployment from the dashboard.
Copy the endpoint value next to Kibana (2). This is the Server Url value which will be required later.
The default user is elastic. The password is the same that has been set during Elastic cluster deployment. Make a note of them, these will also be required later in the steps. In case there is a need to reset the password, it can be done from security settings of the deployment. Select Security from the deployment and then click Reset password.
Navigate to Rules (1) in the Squidex App and select + New Rule (2).
Select Content changed (3) as the trigger for this event. This will load additional settings for the trigger that can be configured such as Conditions.
In this example, we are setting this rule for the magazine schema only and triggering it for all events in the schema where the event status is Published. Click + (4) to add a condition, select the schema (5) from the dropdown (i.e. magazine in this case) and use the following condition (6) in Javascript format:
Next, select Elasticsearch (7) as the action.
Provide the ElasticSearch instance values here such as the Server Url, Index Name, Username, Password etc. Click Save when done.
In order to test the rule we can either create a new article or update an existing article. The article / content must be in Published status for the event to trigger.
On successful trigger, you will see a status message similar to the screenshot below.
On successful Rule execution, one can connect to (or query) the ElasticSearch cluster to validate the creation of the documents in the Index. In this example we are using the managed cluster option through https://cloud.eastic.co, hence the screenshots and the navigation instructions refer their platform. If you are using a self-hosted ElasticSearch cluster the screenshots may be different.
To view the documents, navigate and login to the Kibana endpoint (1) (you can find the same in your Elastic Cloud home page).
Post login, select Enterprise Search (2).
Next, click Indices (3) and then select the Index (4), which in this example is search-squidex.
Select the Documents (5) tab to view the documents created through the Rule.
A newly Published article appears as MagazinePublished and an Updated article will appear as MagazineUpdated.
You can choose to customize the document that is populated in ElasticSearch. We support plain Text, Script and Liquid syntax.
Below is an example of custom document that sends only the Title of the article. It is written in the Liquid syntax. You must create a valid json string, therefore it is good to use the escape formatter.
The content ID from Squidex is used as the Elasticsearch document ID.
Populating to ElasticSearch is an asynchronous operation. Hence, it may take a few seconds before the updated results appear on a search.
You can configure Squidex to use ElasticSearch as the full text store and search directly using Squidex search API. To do this set the following values:
For more details and other full text parameters, refer to the appsettings.json file at: https://github.com/Squidex/squidex/blob/master/backend/src/Squidex/appsettings.json#L30
