需求:获取oracle表增量信息,发送至udp514端口,支持ip配置
步骤:
(1)需要的jar oracle的 odbc5.jar(oracle安装目录 /jdbc/lib下查找)
(2)flume的开源包flume-ng-sql-source-1.4.3.jar 最新的好像是1.5的 小版本记不住了 这个下载地址直接csdn上就有
这两个jar 都拷贝到flume的lib下
(3)flume配置文件
a1.sources = r1 a1.sinks = k1 a1.channels = c1 #接收syslog配置 #a1.sources.r1.type = syslogudp #a1.sources.r1.port = 514 #a1.sources.r1.host = 0.0.0.0 ###########sql source################# a1.sources.r1.type = org.keedio.flume.source.sqlsource a1.sources.r1.hibernate.connection.url = jdbc:oracle:thin:@10.9.224.36:1521/soc # hibernate database connection properties a1.sources.r1.hibernate.connection.user = soc_app a1.sources.r1.hibernate.connection.password = soc_app a1.sources.r1.hibernate.connection.autocommit = true a1.sources.r1.hibernate.dialect = org.hibernate.dialect.oracle10gdialect a1.sources.r1.hibernate.connection.driver_class = oracle.jdbc.driver.oracledriver a1.sources.r1.run.query.delay=1 a1.sources.r1.status.file.path = /usr/local/flume a1.sources.r1.status.file.name = a1.sqlsource.status # custom query a1.sources.r1.start.from = '2017-07-31 07:06:20' a1.sources.r1.custom.query = select chr(39)||to_char(createtime,'yyyy-mm-dd hh24:mi:ss')||chr(39),msg from flume_ng_sql_source where createtime > to_date($@$,'yyyy-mm-dd hh24:mi:ss') order by createtime asc a1.sources.r1.batch.size = 6000 a1.sources.r1.max.rows = 1000 a1.sources.r1.hibernate.connection.provider_class = org.hibernate.connection.c3p0connectionprovider a1.sources.r1.hibernate.c3p0.min_size=1 a1.sources.r1.hibernate.c3p0.max_size=10 ###########sql source################# #自定义类型 a1.sinks.k1.type = com.neusoft.utils.syslogsink #配置发送ip a1.sinks.k1.destination = 10.9.224.36 a1.channels.c1.type = memory a1.channels.c1.capacity = 10000 a1.channels.c1.transactioncapacity = 10000 a1.sources.r1.channels = c1 a1.sinks.k1.channel = c1 #启动命令 ./flume-ng agent -c ../conf -f ../conf/flume-sql.conf -n a1 -dflume.root.logger=info,console &
由于是自己定义的sink 在我的博客中也有提到,所以配置了自定义的sink。
总结
以上所述是小编给大家介绍的flume监听oracle表增量的步骤详解,希望对大家有所帮助
如对本文有疑问, 点击进行留言回复!!
Navicate 如何导出数据库中的存储过程、事件、视图等?
每日一记:Oracle升级2020年4月份的数据库补丁Database Patch Set Update : 11.2.0.4.200414 (30670774)
Oracle 19.3 dataguard for Redhat 7.6
网友评论