How to read fsimage:
We can use Offline Image Viewer tool
to view the fsimage data in a human readable format. Sometimes, this becomes
more essential to analyse the fsimage to understand the usage pattern, how many
0 bite files are created, what is the space consumption pattern and is the
fsimage corrupt.
Donwload the fsimage:
hdfs dfsadmin –fetchImage /fsimage
This will download the latest
fsimage from Name node
16/12/27 05:40:43 INFO namenode.TransferFsImage: Opening connection to http://<nn_hostname>:50070/getimage?getimage=1&txid=latest
16/12/27 05:40:43 INFO namenode.TransferFsImage: Transfer took 0.23s at 89.74 KB/s
Reading fsimage:
We can read the fsimage in several
output formats.
1 .
Web is the default output format.
2 .
XML document
3 .
Delimiters
4 .
Reverse XML.
1 .
FileDistribution is the tool for analyzing file
sizes in the namespace image.
In this blog I will focusing on two output formats Web and
Delimiters.
To get the output on
web:
Run the oiv command with fsimage as input
file:
hdfs oiv –i /fsimage/fsimage_0000000000000005792
16/12/27 05:48:43 INFO
offlineImageViewer.FSImageHandler: Loading 9 strings
16/12/27 05:48:43 INFO
offlineImageViewer.FSImageHandler: Loading 64 inodes.
16/12/27 05:48:43 INFO
offlineImageViewer.FSImageHandler: Loading inode references
16/12/27 05:48:44 INFO
offlineImageViewer.FSImageHandler: Loaded 0 inode references
16/12/27 05:48:44 INFO
offlineImageViewer.FSImageHandler: Loading inode directory section
16/12/27 05:48:44 INFO
offlineImageViewer.FSImageHandler: Loaded 32 directories
16/12/27 05:48:44 INFO
offlineImageViewer.WebImageViewer: WebImageViewer started. Listening on
/127.0.0.1:5978. Press Ctrl+C to stop the viewer.
Now open another terminal and run the below commands to read fsimage.
hdfs dfs -ls
webhdfs://127.0.0.1:5978/
hdfs dfs -ls –R webhdfs://127.0.0.1:5978/
We can also get the output in
JSON format by using curl:
To get the output in
to an output directory:
hdfs ovi –p Delimited –i /fsimage/fsimage__0000000000000005792
–o /fsimage/fsimage.txt
We can read the data in fsimage.txt by running: head fsimage.txt from the local folder.
