Apache Seatunnel-Source plugin : Hive [Spark]

从hive中获取数据

Options

pre_sql [string]

进行预处理的sql, 如果不需要预处理,可以使用select * from hive_db.hive_table

common options [string]

Source 插件通用参数，详情参照 [Source Plugin]
注意：使用hive source必须做如下配置：

## seatunnel 配置文件中的spark section中：
env {
  ...
  spark.sql.catalogImplementation = "hive"
  ...
}

Example

env {
  ...
  spark.sql.catalogImplementation = "hive"
  ...
}
source {
  hive {
    pre_sql = "select * from mydb.mytb"
    result_table_name = "myTable"
  }
}
...

Notes

必须保证hive的metastore是在服务状态。启动命令 hive --service metastore 服务的默认端口的9083
cluster、client、local模式下必须把hive-site.xml置于提交任务节点的$HADOOP_CONF目录下(或者放在$SPARK_HOME/conf下面),IDE本地调试将其放在resources目录

文章列表

• Apache Seatunnel-Roadmap
• Apache Seatunnel-Sink plugin : Clickhouse [Spark]
• Apache Seatunnel-Sink plugin : Console [Spark]
• Apache Seatunnel-Sink plugin : Elasticsearch [Spark]
• Apache Seatunnel-Sink plugin : Email [Spark]
• Apache Seatunnel-Sink plugin : File [Spark]
• Apache Seatunnel-Sink plugin : Hbase [Spark]
• Apache Seatunnel-Sink plugin : Hdfs [Spark]
• Apache Seatunnel-Sink plugin : Mysql [Spark]
• Apache Seatunnel-Sink plugin : Phoenix [Spark]
• Apache Seatunnel-Source plugin : Fake [Spark]
• Apache Seatunnel-Source plugin : FakeStream [Spark]
• Apache Seatunnel-Source plugin : Hive [Spark]
• Apache Seatunnel-Source plugin : JDBC [Spark]
• Apache Seatunnel-Source plugin : Kafka [Flink]
• Apache Seatunnel-Source plugin : Kafka [Spark]
• Apache Seatunnel-Source plugin : Socket [Flink]
• Apache Seatunnel-Source plugin : SocketStream [Spark]
• Apache Seatunnel-Transform Plugin
• Apache Seatunnel-Transform plugin : Json [Spark]
• Apache Seatunnel-Transform plugin : SQL [Spark]
• Apache Seatunnel-Transform plugin : Split [Spark]
• Apache Seatunnel-seatunnel v2.x 与 v1.x 的区别是什么？
• Apache Seatunnel-start-seatunnel-flink.sh 使用方法
• Apache Seatunnel-start-seatunnel-spark.sh 使用方法
• Apache Seatunnel-下载、安装
• Apache Seatunnel-为 seatunnel v2.x 贡献代码
• Apache Seatunnel-完整配置文件案例 [Spark]
• Apache Seatunnel-快速开始
• Apache Seatunnel-插件开发
• Apache Seatunnel-深入seatunnel
• Apache Seatunnel-部署与运行