Skip to main content

LEK stack for node.js

LEK stack for node.js 

1) node access log file
1. node access log json formatting
  (/Users/mac/git/fuji/config/express.js)

app.use(expressWinston.logger({
     transports: [
       new winston.transports.DailyRotateFile({
 json: true,
 filename: config.logs_dir + '/access-',
 datePattern: 'yyyy-MM-dd.log'
})
     ]
   }));

2) logstash
1. logstash download
http://logstash.net/
extract file to mac:/data1/logstash-1.4.0

2. make config file
mac:/data1/logstash-1.4.0> vi logstash-node.conf
input{
   file{
       codec => json
       path => ["/Users/mac/git/fuji/logs/access*.log"]
       start_position => "beginning"
   }
}

output{
   elasticsearch {
       cluster => "locketCast"
       node_name => "logstash"
       host => "127.0.0.1"
       index => "updatelogs"
  }
}

cf.     index => "%{es_index}"
       index_type => "%{es_type}"
       document_id => "%{es_id}"

3. run logstash
mac:/data1/logstash-1.4.0> bin/logstash -f logstash-node.conf

3) kibana
1. download
https://github.com/elasticsearch/kibana
/data1/elasticsearch/kibana>

2. add ngix virtual host for kibana

brew install nginx
sudo nginx -s stop
sudo nginx

/usr/local/etc/nginx/conf.d> vi vm.conf

server {
   listen       7001;
   server_name  localhost;

   #access_log  logs/localhost.access.log  main;

   location / {
       root   /data1/elasticsearch/kibana;
       index  index.html index.htm;
   }
}

3. check out the site
http://localhost:7001

Comments

Post a Comment

Popular posts from this blog

Amazon RDS Blue/Green Deployments

In order to avoid some errors I experienced when proceeding as described in the official documentation, I describe what I did in order. 1) Modify parameters of source_database * error: Blue Green Deployments requires cluster parameter group has binlog enabled. RDS Parameter groups: source-params-group binlog_format => MIXED mysql> show global variables like 'binlog_format'; 2) Insert a row after rebooting the source database, to avoid this error. * error: Correct the replication errors and then switch over. Read Replica Replication Error - IOError: 1236, reason: Got fatal error 1236 from master when reading data from binary log: 'Could not find first log file name in binary log index file' => To Fix: You need to change the data in the source database. INSERT INTO dummy_table ( `favorite_id` , `favorite_order` , `user_id` , `board_id` ) VALUES ('100001', '1', '11111', '11111'); 3) Modify the param...
Post a Comment
Read more

Fluentd for mysql in AWS

(0) preparation ulimit -n If your console shows 1024, it is insufficient. Please add following lines to your /etc/security/limits.conf file and reboot your machine. root soft nofile 65536 root hard nofile 65536 (1) install Fluentd // “Ubuntu 12.04 LTS / Precise” curl -L http://toolbelt.treasuredata.com/sh/install-ubuntu-precise.sh | sh /etc/init.d/td-agent start/stop/restart/status // test curl -X POST -d 'json={"json":"message"}' http://localhost:8888/debug.test /etc/init.d/td-agent stop chown: changing ownership of `/var/run/td-agent/td-agent.pid': Operation not permitted chown: changing ownership of `/var/run/td-agent': Operation not permitted  * Stopping td-agent td-agent                                                                                   ...
Post a Comment
Read more

Install CoreOs on linode without VM

Install CoreOs on linode without VM 1. Add a Linode 2. Create a new Disk   CoreOS 3. Rescue > Reboot into Rescue Mode 4. Remote Access   Launch Lish Console 5. make an install script cat <<'EOF1' > install.sh # add needed package sudo apt-get update sudo apt-get install -y curl wget whois sudo apt-get install -y ca-certificates #sudo apt-get install gawk -y # get discovery url discoveryUrl=`curl https://discovery.etcd.io/new` # write cloud-config.yml cat <<EOF2 > cloud-config.yml #cloud-config users:   - name: core     groups:       - sudo       - docker coreos:   etcd:     name: node01     discovery: $discoveryUrl hostname: node01 EOF2 # get the coreos installation script #wget https://raw.github.com/coreos/init/master/bin/coreos-install wget https://raw.githubusercontent.com/coreos/init/master/bin/coreos-install # run installation chmod 75...
31 comments
Read more