Dashboard using ElasticSearch, Logstash & Kibana - 1

All set ? I will try my best to list down the exact steps followed by me to get the ELK stack working on windows. I have tried to get in as much details as possible , special thanks to my co-worker – Kiran, for helping me with this research

This guide was installed successfully on  - Windows 7 Enterprise

Let’s get started

1. Dowload ElasticSearch , Logstash and Kibana from here

2. Create folders in your local drive say c:\Elastic and extract all the folders. So, after it is done, here is how it looks

c:\Elastic\elasticsearch

c:\Elastic\logstash

c:\Elastic\Kibana

Optionally, you can create a new folder say – elasticlogs , which will house the log files, incase you want to dump it there

c:\Elastic\elasticlogs

In the Elasticsearch\bin folder , edit the elasticsearch.yml to include  -

http.cors.enabled: true

Create a logstash config file with


                input {
                file {
                  type => "XMLLogs"
                  path => ["C:/ElasticLogs/*.*","//networkpath/*.*"]
                }
                }
                output {
                 stdout {codec => rubydebug }
                elasticsearch {
                   host => "localhost"
                    protocol => "http"
                    }
                }

3. Download  and install the latest version of Java JDK and install it depending on your OS architecture

4. Set your Java path , to do this

Right click on the My Computer icon on your desktop and select properties

Click the Advanced Tab

Click the Environment Variables button

Under System Variable, click New

Set Java Home Step 2Bullet4.jpg

Enter the variable name as JAVA_HOME

Set Java Home Step 2Bullet5.jpg

Enter the variable value as the install path "C:\Program Files\Java\jdk1.8.0_06" (update your java version accordingly

Click Ok

Click apply changes

5. Install IIS

  1. First, go to Control Panel, and then click on Programs. You’ll see a link for “Turn Windows features on or off”
  2. If you expand the Internet Information Services tree node, you can see that there are a lot of options beneath it. You will probably want to explore these options, because even if you click on IIS, some of the necessary options for doing development aren’t checked.
  3. Once you’ve gone ahead and checked the items you want, and clicked OK, you’ll see this dialog for a while….
  4. Now when you navigate in your browser to localhost, you’ll see the new default page…

6. Open IIS Manager, by going to command prompt and using “inetmgr”  IIS manager will open, now stop the "Default Web Site"

iisstop

7. Create a new website for Kibana as shown below
Right click on "sites" in the left part of IIS Manager and click "Add Website" . In the end it should look something like this

IISwebsiteadd

You can add mime types as below if your IIS version is older

".": application/octet-stream
".json": text/html

8. Start elasticsearch by hitting this in command prompt C:\Elastic\elasticsearch\bin>elasticsearch

9. Check if your elastic search is working by pointing your browser to localhost:9200

You should get something like this.

elasticsearchreply

10. Now reboot your server so you can test that everything is automatically starting on boot.

kib1

Congrats you have installed it!

 

Some Tips:

To clear elasticsearch cache
1. Install curl - http://www.confusedbycode.com/curl/
2    run from cmd - curl -XDELETE "http://localhost:9200/_all"

Starting logstash
logstash.bat agent -f logstash.conf
 
To check if the logstash config file is correct with parameters
  logstash.bat agent -f logstash.conf --configte

Setting up the Logstash and ElasticSearch to run as an automatic service.

1.  Start elasticsearch and put it on autostart
Open a console and go to "c:\Elastic\elasticsearch\bin\" & type the following command:

service install

You should get service has installed

Now type – service manager and  something similar to below should appear

serv

 

You have to change on the tab the "Startup type" from Manual to Automatic and then press Apply this should make Elasticsearch start automatically on server boot.
This window contains some more options such as how much memory Elasticsearch will use you can find this on the "Java" tab I would suggest to make this fitting for your server if you have a servers that will handle a huge amount of logs I would increase the "Maximum Memory Pool: 1024" at least to a higher amount.
Before you close the window make sure to press "Start" so it actually will run right now

This is everything to start ElasticSearch automatically on boot to test that it is working open a browser and go to the url you used in IIS  or go to: //127.0.0.1:9200/
If you see a json string something like what you saw in step 9  picture then it means your Elasticsearch is running

2. Create a Logstash startup bat file

A “run.bat" should simply contain this line -

logstash.bat agent –f  logstash.conf

Now place your "run.bat" in the folder: C:\Elastic\logstash\bin

3. Start Logstash & Autostart it
For this step we need another small program to create a proper windows service so please go ahead and download "NSSM" (the Non-Sucking Service Manager)

Once you have the zip file simply unzip it and copy the file from the unzipped folder you now have: "nssm-2.24\win64" (nssm.exe) to "C:\Elastic\logstash\bin" so it should result in you having "C:\Elastic\logstash\bin\nssm.exe".
Now open a Command Prompt and type:

cd c:\Elastic\lostash\bin

nssm install logstash

Ensure you have the details filled as below

ser1

ser2

Also make sure that "Startup type" is on Automatic.

Now the following step is additional but recommended if you are 100% going to use Elasticsearch, click on the "Dependencies" tab and add the following: elasticsearch-service-x64

ser3

The reason for this step is that Logstash can do difficult if you have a Elasticsearch output but it is not actually running.
Now this is all so press on the "Install service" button to finish setting up a service for Logstash.

[​IMG]

Now, you actually can stop and start Logstash or Elasticsearch, easily from  "services.msc" as they are now windows services.

author

Vinay Jagtap

A hard core Technocrat with over a decade of extensive experience in heading complex test projects coupled with real time experience of project management and thought leadership. Extensive experience in Performance, Security and Automation Testing and development of automation frameworks and ability to setup and execute Global service centers and Center of Excellences for testing.

Get Free Email Updates to your Inbox!

www.CodeNirvana.in

Powered by Blogger.

Translate

Total Pageviews

Copyright © T R I A G E D T E S T E R