Tracking YARN logs

By | June 26, 2017
Create script to get yarn logs

$ vim hadoop_logs.sh
#!/bin/bash

APPLICATION_ID=$1
CONTAINER_ID=$2
NODE_ADDRESS=$3

if [ $# -eq 1 ]; then
    yarn logs -applicationId ${APPLICATION_ID}
elif [ $# -eq 3 ]; then
    yarn logs -applicationId ${APPLICATION_ID} -containerId ${CONTAINER_ID} -nodeAddress ${NODE_ADDRESS}
else
    echo "you must specify 1 or 3 arguments <hlogs applicationId containerId nodeAddress>"
fi

Create a symlink

$ ln -s /home/user1/hadoop_logs.sh ~/bin/hlogs

To track the logs:
Syntax: hlogs applicationId containerId nodeAddress
Note: ‘hlogs’ is the alias

1) To get the full logs of the application

hlogs application_1498490223123_0123

2) To get logs to specific container

hlogs application_1498490223123_0123 container_e123_1498490223123_0123_01_095999 host1.example.com

Leave a Reply

Your email address will not be published. Required fields are marked *