当前位置： 移动技术网 > IT编程>数据库>其他数据库 > HDFS基础

HDFS基础

2019年02月10日  | 移动技术网IT编程  | 我要评论

1. hdfs shell基础

[root@master hadoop]# hadoop fs
usage: hadoop fs [generic options]
[-appendtofile <localsrc> ... <dst>]
[-cat [-ignorecrc] <src> ...]
[-checksum <src> ...]
[-chgrp [-r] group path...]
[-chmod [-r] <mode[,mode]... | octalmode> path...]
[-chown [-r] [owner][:[group]] path...]
[-copyfromlocal [-f] [-p] [-l] <localsrc> ... <dst>]
[-copytolocal [-p] [-ignorecrc] [-crc] <src> ... <localdst>]
[-count [-q] [-h] <path> ...]
[-cp [-f] [-p | -p[topax]] <src> ... <dst>]
[-createsnapshot <snapshotdir> [<snapshotname>]]
[-deletesnapshot <snapshotdir> <snapshotname>]
[-df [-h] [<path> ...]]
[-du [-s] [-h] <path> ...]
[-expunge]
[-find <path> ... <expression> ...]
[-get [-p] [-ignorecrc] [-crc] <src> ... <localdst>]
[-getfacl [-r] <path>]
[-getfattr [-r] {-n name | -d} [-e en] <path>]
[-getmerge [-nl] <src> <localdst>]
[-help [cmd ...]]
[-ls [-d] [-h] [-r] [<path> ...]]
[-mkdir [-p] <path> ...]
[-movefromlocal <localsrc> ... <dst>]
[-movetolocal <src> <localdst>]
[-mv <src> ... <dst>]
[-put [-f] [-p] [-l] <localsrc> ... <dst>]
[-renamesnapshot <snapshotdir> <oldname> <newname>]
[-rm [-f] [-r|-r] [-skiptrash] <src> ...]
[-rmdir [--ignore-fail-on-non-empty] <dir> ...]
[-setfacl [-r] [{-b|-k} {-m|-x <acl_spec>} <path>]|[--set <acl_spec> <path>]]
[-setfattr {-n name [-v value] | -x name} <path>]
[-setrep [-r] [-w] <rep> <path> ...]
[-stat [format] <path> ...]
[-tail [-f] <file>]
[-test -[defsz] <path>]
[-text [-ignorecrc] <src> ...]
[-touchz <path> ...]
[-truncate [-w] <length> <path> ...]
[-usage [cmd ...]]

查看某个命令的具体帮助信息：

[root@master hadoop]# hadoop fs -help test
-test -[defsz] <path> :
answer various questions about <path>, with result via exit status.
-d return 0 if <path> is a directory.
-e return 0 if <path> exists.
-f return 0 if <path> is a file.
-s return 0 if file <path> is greater than zero bytes in size.
-z return 0 if file <path> is zero bytes in size, else return 1.

这里尤其要注意，如果文件存在，返回结果是0

我们上传一个文件，然后用test命令测试：

[root@master hadoop]# hadoop fs -put /root/test test.txt

[root@master hadoop]# hadoop fs -ls .
found 4 items
drwxrwxrwx - hdfs hdfs 0 2018-02-10 22:22 quasimontecarlo_1518319340789_698036166
drwxrwxrwx - hdfs hdfs 0 2018-02-10 23:21 quasimontecarlo_1518322909671_1083050937
-rw-r--r-- 3 root hdfs 5 2019-02-08 21:52 test
-rw-r--r-- 3 root hdfs 5 2019-02-09 02:12 test.txt

[root@master hadoop]# hadoop fs -test -e test.txt
[root@master hadoop]# echo $?
0
[root@master hadoop]# hadoop fs -test -e /user/root/test.txt
[root@master hadoop]# echo $?
0

[root@master hadoop]# hadoop fs -test -e /user/root/test.txtt
[root@master hadoop]# echo $?
1

2. hdfs 编程基础

2.1 判断文件是否存在

import org.apache.hadoop.conf.configuration;
import org.apache.hadoop.fs.filesystem;
import org.apache.hadoop.fs.path;

public class hdfsfileifexist {
    public static void main(string[] args){
        try{
            //hdfs路径
            string filename = "/user/root/test.txt";
            configuration conf = new configuration();
            conf.set("fs.defaultfs", "hdfs://master:8020");
            conf.set("fs.hdfs.impl", "org.apache.hadoop.hdfs.distributedfilesystem");
            filesystem fs = filesystem.get(conf);
            if(fs.exists(new path(filename))){
                system.out.println("文件存在");
            }else{
                system.out.println("文件不存在");
            }
            
        }catch (exception e){
            e.printstacktrace();
        }
    }

}

上面的代码中fs.defaultfs的值，请查看hdfs配置文件 core-site.xml ，该文件位于hadoop安装目录的etc/hadoop目录下。

如果在windows系统中运行该程序，请在c:\windows\system32\drivers\etc\hosts文件中加上主机名master和ip地址的对应关系。

您可能感兴趣的文章:

如对本文有疑问， 点击进行留言回复！！

相关文章:

验证码：
最近更新的文章
大家感兴趣的文章
移动技术网