To enhance the user experience, Tapdata Cloud continuously enriches and optimizes product features and rectifies known defects by releasing new versions. This article provides an update log for Tapdata Cloud, helping you grasp the new feature specifications more effectively.
- Added support for Shared Mining, allowing multiple tasks to share incremental logs from the source database, thus avoiding redundant reads and significantly reducing the load on the source database during incremental synchronization.
- The Shared Mining feature now supports using RocksDB as local external storage to extend storage for incremental logs.
- Improved the onboarding process for users from the Google Cloud Marketplace.
- Added a time filter option for the incremental phase in the Task Monitoring Page to quickly observe QPS during the incremental phase.
- Added warning messages for critical operations that might impact the database (e.g., filtering source table data).
- Refined the logic for unsubscribing after instance subscription expiration.
- Fixed an issue with the Primary-Secondary Merge Node where changes in the key conditions between the primary and secondary tables resulted in data not matching expectations.
- Added support for Capped Collections in data synchronization between MongoDB database.
- Data replication/transformation tasks now have import capabilities. Design your data flow process on MongoDB Relational Migrator, export it, and then directly import it into Tapdata data pipelines from the top right corner, enhancing the convenience of data pipeline design.
- Enhanced the new user onboarding process, including the ability to collapse prompts and return to previous steps.
- Fixed an issue where JS node model declaration settings showed incorrect prompts on the task editing page.
- Fixed an issue where the DROP COLUMN operation in Oracle to MySQL synchronization was not syncing correctly.
- Addressed an issue causing DDL errors when syncing from MySQL to ClickHouse.
- Fixed instability in tasks due to frequent WebSocket reconnections.
- Corrected several UI interaction experience issues.
- Added support for Time Series collections in MongoDB 5.x and above versions.
- Added support for preImage in MongoDB 6.x and above versions.
- Improved system prompts when enabling scheduled tasks while reaching the task limit.
- Fixed inaccuracies in checkpoints in multi-table data replication scenarios.
- Resolved issues with unsubscribed and deleted Agent instances continuing to report heartbeat information.
- Addressed known UI interaction experience issues.
- Added Azure Cosmos DB as a new data source, enabling full data synchronization to facilitate quick cloud data transfers.
- Upgraded data source connections, with SQL Server now supporting SSL connections, enhancing data security.
- Optimized field type adjustments in data replication tasks, allowing for direct selection of common types from the target database.
- Improved task source node settings, enabling customization of the number of rows read per batch in the incremental phase, catering to performance needs of incremental synchronization.
- Addressed issues with enhanced JS nodes failing or causing exceptions under certain scenarios.
- Corrected several UI interaction experience issues for better usability.
- Support for loading table comments on Oracle data sources, which can be enabled in the Advanced Settings when configuring the data source. This makes it easier to quickly identify the business meaning of tables through their comments.
- In the task monitoring page, support viewing QPS (Queries Per Second) information based on the size of events.
- Enhanced the display effects of resource management and the subscription center pages.
- When performing data source connection tests, support for displaying connector download progress is now available, helping to quickly grasp connection progress and pinpoint timeout issues.
- Fixed an issue where incremental information was not successfully cleared after resetting and rerunning a task.
- Fixed an issue where some SaaS data sources displayed incremental timestamps during full data synchronization.
- Enhanced Data Source Connection methods, supporting SSL connections for data sources like MySQL, PostgreSQL, Kafka, TiDB, MariaDB, etc., to further enhance data security.
- Improved user interface interaction logic.
- To better manage data duplication for updates on non-primary keys, Tapdata Cloud now supports creating unique indexes.
- Fixed an issue where data synchronization could fail when table names contain
- Fixed an issue where task exception messages did not include table names.
- Fixed an issue with incorrect judgment of task quotas and task count limits when specifying an Agent for a task.
- Added support for automatically creating sharded collections when MongoDB Cluster is set as the target.
- Add support for Unwind Processing Node, which can help you efficiently "unwind" each element in an array, converting each element into independent data rows.
- Added support for disabling node capabilities when configuring tasks. You can access this feature by hovering over a node, which can help reduce the cost of data flow during processing.
- When configuring data replication tasks, you can now quickly filter tables with or without primary keys through the "Selectable table range" dropdown. Tables with primary keys include those without primary keys but with unique indexes.
- Added a Demo data source to the onboarding guide flow for new users, helping you quickly complete the tutorial and set up your first data flow task.
- Optimized the front-end display effects of operation buttons on the engine interface.
- Fixed an issue where an error occurred in MongoDB as a target during an INSERT operation when there was no shard key.
- Fixed an issue where MongoDB did not support REPLACE properly, and the fields deleted by REPLACE could not be properly removed.
- Introduced the Create Materialized View feature for swift construction of real-time data models.
- Added capability to fetch read-only access information of subscribed MongoDB Atlas.
- Kafka data source now supports settings for replication factor and partition count.
- For synchronization between MongoDB instances, added support for
- During the task guidance process, when creating a connection for a fully managed Agent, instructions about the public IP address of the fully managed Agent have been added.
- Enabled rapid target node location through node search at the top of the data replication/data transformation configuration page.
- Fixed an issue where the wrong category of operation logs was recorded when restarting the Agent via the webpage.
- Added Python processing node, which supports customizing data processing logic through Python scripts. This offers improved performance compared to the JS processing node.
- Added a "Contact Us" entry point, making it easier for users to quickly reach out to technical support when faced with issues.
- Enhanced error codes for data sources, covering more scenarios and providing solutions.
- While setting up email alert notifications, added guidance for binding email addresses.
- Improved reminders and easy upgrade guide for when the task count reaches its limit.
- Introduced the Primary-Secondary Merge Node, enabling quick construction and real-time updates of wide tables, assisting you in achieving better data analysis.
- Real-Time Data Hub now offer a storage instances for free trial, with more new specifications available, including M10, M20, and M30.
- Added support for connecting existing MongoDB Atlas instances as data storage for the Real-Time Data Hub.
- Changed the display of help documentation on the right side during data source connection to embedded online documentation, assisting users in accessing the most recent help information.
- For core data sources (such as Oracle, PostgreSQL, etc.), improved the page parameter descriptions and guidance when creating connections.
- Fixed the issue where users couldn't view the monitoring page for previously run tasks.