Skip to main content

Manage External Storage

Applicable EditionsTapData EnterpriseTapData Enterprise can be deployed in your local data center, making it suitable for scenarios with strict requirements on data sensitivity or network isolation. It can serve to build real-time data warehouses, enable real-time data exchange, data migration, and more.TapData CommunityTapData Community is an open-source data integration platform that provides basic data synchronization and transformation capabilities. This helps you quickly explore and implement data integration projects. As your project or business grows, you can seamlessly upgrade to TapData Cloud or TapData Enterprise to access more advanced features and service support.

To facilitate the quick reading of task-related information subsequently, TapData stores necessary configurations, incremental logs of source tables, and other information related to the task in its internal MongoDB database. To store more data, you can create an external database to store relevant data.

tip

If you are using TapData Cloud, you can access this feature by navigating to Advanced > External Storage in the menu.

Prerequisites​

An external database intended for data storage has been created. Currently, MongoDB and RocksDB are supported.

Create External Storage​

  1. Log in to TapData Platform.

  2. In the left navigation bar, select System > External Storage.

  3. On the right side of the page, click Create External Storage.

  4. In the pop-up dialog, complete the configuration according to the instructions below.

    Create External Storage

    • Storage Name: Enter a cache name with business significance for easy identification later.
    • External Memory Type: Supports MongoDB and RocksDB.
    • Storage Path: Enter the database connection address, for example, for MongoDB refer to: mongodb:/admin:password@127.0.0.1:27017/mydb?replicaSet=xxx&authSource=admin.
    • Connect Using TLS/SSL: Choose whether to enable TSL/SSL encryption; if this function is enabled, you will also need to upload the client’s private key.
    • Set as Default: Choose whether to use it as the default external storage.

  5. Click Test; after passing the test, click Save.

    tip

    If the connection test fails as indicated, please fix it according to the page prompt.

Use External Storage​

You can enable the Shared Mining feature and select the recently configured external storage when creating a connection, as shown in the example below:

Select External Storage