小编给大家分享一下SequenceFile如何实现读写,希望大家阅读完这篇文章之后都有所收获,下面让我们一起去探讨吧!
SequenceFile 读
public static void main(String[] args) throws IOException {
Configuration conf = new Configuration();
FileSystem fs = FileSystem.get(conf);
Path seqFile = new Path("/user/hive/warehouse/abc/seqfile.seq");
SequenceFile.Reader reader = new SequenceFile.Reader(fs, seqFile, conf);
IntWritable key = new IntWritable();
Text value = new Text();
while(reader.next(key, value)){
System.out.println(key);
System.out.println(value);
}
IOUtils.closeStream(reader);
}
声明顺序文件的Reader实例后,调用next() 方法迭代读取记录。最后需要关闭reader实例。
如果键值对读取成功,则返回true;如果已经读到文件末尾,则返回false。
SequenceFile 写
public static void main(String[] args) throws IOException {
Configuration conf = new Configuration();
FileSystem fs = FileSystem.get(conf);
Path targetPath = new Path("/user/hive/warehouse/test_url");
final Option optPath = SequenceFile.Writer.file(targetPath);
final Option optKeyClass = SequenceFile.Writer.keyClass(Text.class);
final Option optValueClass = SequenceFile.Writer.valueClass(BytesWritable.class);
final SequenceFile.Writer writer = SequenceFile.createWriter(conf, optPath, optKeyClass, optValueClass);
final Collection<File> listFiles = FileUtils.listFiles(new File("/data1/url/"), new String[]{"log"}, false);
Text key = null;
BytesWritable value = null;
for (File file : listFiles) {
key = new Text(file.getPath());
value = new BytesWritable(FileUtils.readFileToByteArray(file));
writer.append(key, value);
}
IOUtils.closeStream(writer);
}
通过CreateWriter() 静态方法创建SequenceFile 对象,并返回SequenceFile.Writer 实例。
拥有SequenceFile.Writer 实例后,就可以通过 append() 方法在文件末尾追加数据。
最后关闭实例。
看完了这篇文章,相信你对“SequenceFile如何实现读写”有了一定的了解,如果想了解更多相关知识,欢迎关注天达云行业资讯频道,感谢各位的阅读!