Skip to main content

mrs-hive3

Applicable EditionsTapData CloudTapData Cloud offers you cloud services that are suitable for scenarios requiring rapid deployment and low initial investment, helping you focus more on business development rather than infrastructure management. Free trial with TapData Cloud.TapData EnterpriseTapData Enterprise can be deployed in your local data center, making it suitable for scenarios with strict requirements on data sensitivity or network isolation. It can serve to build real-time data warehouses, enable real-time data exchange, data migration, and more.TapData CommunityTapData Community is an open-source data integration platform that provides basic data synchronization and transformation capabilities. This helps you quickly explore and implement data integration projects. As your project or business grows, you can seamlessly upgrade to TapData Cloud or TapData Enterprise to access more advanced features and service support.

Follow these instructions to ensure that the Hive database is successfully added and used in TapData Cloud.

Restriction Description ​

TapData The current version of the Hive system supports only the target.

Supported version ​

Hive3.1.2

Configuration description ​

Data source configuration​

  • Host/IP
  • Port
  • Database name Name of the database
  • Account and Password

Enables the CDC configuration​

Hive row-level operations UPDATE and DELETE are transaction operations. Therefore, you need to enable transaction operations in Hive and modify the configuration items in 'hive-site. XML' to take effect after the modification is complete.

<property>
<name>hive.support.concurrency</name>
<value>true</value>
</property>
<property>
<name>hive.enforce.bucketing</name>
<value>true</value>
</property>
<property>
<name>hive.exec.dynamic.partition.mode</name>
<value>nonstrict</value>
</property>
<property>
<name>hive.txn.manager</name>
<value>org.apache.hadoop.hive.ql.lockmgr.DbTxnManager</value>
</property>
<property>
<name>hive.compactor.initiator.on</name>
<value>true</value>
</property>
<property>
<name>hive.compactor.worker.threads</name>
<value>1</value>
</property>
<property>
<name>hive.in.test</name>
<value>true</value>
</property>

Connect test items​

  • Checks host/IP and port
  • Check the database name
  • Check account and password