Reading Time: 5 minutes

What is Elasticsearch?

Elasticsearch is a distributed, full-text, open-source search engine. It provides multi-tenant capabilities in analyzing aggregate data types from sources like Logstash or Kibana. This application stores and indexes information, which can then be queried for specific data. It returns useful details about a particular program, log analysis, application performance data, or other information. 

Installation Order

To install the Elastic Stack, deploy these applications in the following order.

  1. Elasticsearch (install instructions)
  2. Kibana (install)
  3. Logstash (install)
  4. Beats (install instructions)
  5. APM Server (install instructions)
  6. Elasticsearch Hadoop (install instructions)

Installation 

In order to install Elasticsearch, see our kb article for more in depth instructions. Here are the basic installation steps.

  1. Download and unpack the Elasticsearch official distribution.
  2. Next, run bin/elasticsearch on Linux or macOS. Run bin\elasticsearch.bat on Windows.
  3. Then, curl -X GET http://localhost:9200/.
  4. Start more servers

What is Indexing?

Indexing is simply the process of adding data into Elasticsearch. Elasticsearch stores and retrieves this data in Apache Lucene indexes. We will not be discussing Lucene specifically in this article because we need to delve deeper into that application to truly understand the role Elasticsearch plays. This article is for primarily for new users to employ basic GET and PUT requests in Lucene.

Requests

Put

We send PUT requests when we know, or want to specify an ID of the data type. We can use POST if we want Elasticsearch to generate an ID for that item on its own. An Example of a simple POST command would look like this.

curl -XPOST 'localhost:9200/logs/test_app' -H 'Content-Type: application/json' -d'
{
"timestamp": "2020-08-20 09:10:11",
"message": "Test user is logged in",
"user_id": 2,
"admin": false
}
'

And the output looks like this:

{"_index":"logs","_type":"test_app","_id":"e8rHCnQBSXUbYazxinrq","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":1,"_primary_term":1}

Output

We can see in the output provided above that id was generated by Elasticsearch., We can also note the “version” and “created” fields as well. This implies that the test_app file was created using our POST command which did not exist before. Let’s review how we can index something using a PUT request.

curl -X PUT 'localhost:9200/app/users/4' -H 'Content-Type: application/json' -d '
{
  "id": 2,
  "username": "Dean",
  "last_login": "2020-08-20 09:10:11"
}
'

Using this command, we get this output.

{"_index":"app","_type":"users","_id":"4","_version":2,"result":"updated","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":1,"_primary_term":1}

With this entry, we indexed the user Dean under /app/users/ directory. 

All the data that was entered is sent as a JSON object. If we ask how this works without any data structure, the answer is succinct. Elasticsearch usually works as a NoSQL database, thus structure is not needed. Next, we will check how we can perform queries using Elasticsearch.

What is Querying?

Querying is simply using a crafted query search to locate specific information within a given location. Using a query, we can separate questions in elasticsearch into two main categories. 

  • Leaf Queries - Leaf queries are looking for specific values in certain fields. We can run Leaf Query independently and some are termed match, term, and range queries.  
  • Compound Queries - Compound queries are combinations of leaf queries and compound queries. Compound queries combine multiple queries into a single one we use to get what we're seeking more easily. 

Query Types

A complete classification of Elasticsearch queries looks like this:

  • Leaf Queries
    • Full text
    • Term queries
    • Geo queries
    • Span Queries
    • Joining Queries
    • Specialized queries
  • Compound Queries
    • Bool
    • Boosting
    • Dis max
    • Constant score
    • Function score

We do not have to know all these query types to run some basic commands with elasticsearch as many of these are advanced and require more than basic knowledge of the program. We list them here to get a better overall picture of elasticsearch’s capabilities. Let’s review some basic query commands that can be used with Elasticsearch. 

Note:
To query something with the Elasticsearch, we must index it first!

Examples

Since we indexed our app using the previous commands, let’s try to query it now. A simple command would look like this.

curl -XGET 'localhost:9200/app/users/4?pretty'

And this would be the output:

{
  "_index" : "app",
  "_type" : "users",
  "_id" : "4",
  "_version" : 1,
  "_seq_no" : 0,
  "_primary_term" : 1,
  "found" : true,
  "_source" : {
    "id" : 2,
    "username" : "Dean",
    "last_login" : "2020-08-20 09:10:11"
  }
}

Using this command, we queried all the users under “app”. It would return basic information such as username, id, time of last login and so on. Every field with the underscore in front of it is the meta field of our query. Under _source, we can see the original document that was indexed. The simplest way to query Elasticsearch is to use a “URI Search”. We provide a single search term or word and Elasticsearch will comb through every field of every document in our cluster. For example, we can use this query:

curl -XGET 'localhost:9200/_search?q=dean'

And Elasticsearch will find every field that contains the word “Dean” in it. On our end, the output query will look like this.

{"took":60,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":{"value":1,"relation":"eq"},"max_score":0.2876821,"hits":[{"_index":"app","_type":"users","_id":"4","_score":0.2876821,"_source":
{
  "id": 2,
  "username": "Dean",
  "last_login": "2020-08-20 09:10:11"
}
}]}}

Output

As we can see, the output produces both search terms, and the result of the search. Let’s do a quick breakdown of this information.

took = number of seconds that query took to complete

timed_out = pretty straightforward; false means that query didn’t time out, true means that it did time out

shards = number of documents that Elasticsearch went through and how many of them were searched successfully, how many were skipped, and how many failed

hits = how many fields actually met the query criteria, along with the meta information

In the example above, we searched for specific fields within all of our indexed information noted in Elasticsearch. We can narrow this query down further by providing the path to a limited subset of documents. This way, we can cut down on the query time as well. So, instead of: 

curl -XGET 'localhost:9200/_search?q=dean'

We will use:

curl -XGET 'localhost:9200/app/users/_search?q=dean'

The output would be the same, but in this case “took” (search timeframe) was only 3 seconds as opposed to the 60 seconds that we observed in the previous query. This is handy when we know the specific directory where some information is stored, but are not sure which document holds the information that we need. 

Now that we know the very basics of indexing and querying with Elasticsearch, let’s move on to the deletion of indexed documents.

What is Deleting?

In the same way we used previous commands, we can use the Delete function to remove data stored in our cluster. The difference is, we use DELETION via an HTTP request. 

curl -XDELETE 'localhost:9200/app/users/4?pretty'

Our output will look like this:

[root@host ~]# curl -XDELETE 'localhost:9200/app/users/4?pretty'
{
  "_index" : "app",
  "_type" : "users",
  "_id" : "4",
  "_version" : 1,
  "result" : "not_found",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 2,
  "_primary_term" : 1
}

In order to delete a single index, following command can be used.

curl -XDELETE 'localhost:9200/logs?pretty'

If we want to delete an entire document that has been indexed, we can use this command.

curl -XDELETE 'localhost:9200/path/to/document'

The response that we get in both cases will look like this.

{
 "acknowledged" : true
}

Conclusion

In this article, we covered some of the basic Elasticsearch commands, but barely scratched the surface of all its capabilities. Multiple query variations can be employed to search through our information, but we would need multiple articles to cover all of them in depth. We hope that this article provides some of the basics of Elasticsearch as it is only written as an introduction to this powerful and versatile search engine.

Join Us!

Contact us today at 1.800.580.4985 to speak to a knowledgeable Solutions Provider who can get you the info you need on any one of our product lines, to assist you in making an informed decision right away.

Too busy to talk? Click HERE to open a quick chat with us to find out more. Would you like the information in an email you can review at your leisure? Email us today to get solid advice on which product in our line up would best suit your needs.

We look forward to hearing from you!

Avatar for Dean Conally

About the Author: Dean Conally

I am a Linux enthusiast and console gamer, dog lover, and amateur photographer. I've been working at Liquid Web for a bit less than two years. Always looking for knowledge to expand my expertise, thus tackling new technologies and solutions one day at a time.

Latest Articles

Blocking IP or whitelisting IP addresses with UFW

Read Article

CentOS Linux 7 end of life migrations

Read Article

Use ChatGPT to diagnose and resolve server issues

Read Article

What is SDDC VMware?

Read Article

Best authentication practices for email senders

Read Article