Skip to main content

Enhanced JS Built-in Function

Applicable EditionsTapData CloudTapData Cloud offers you cloud services that are suitable for scenarios requiring rapid deployment and low initial investment, helping you focus more on business development rather than infrastructure management. Free trial with TapData Cloud.TapData EnterpriseTapData Enterprise can be deployed in your local data center, making it suitable for scenarios with strict requirements on data sensitivity or network isolation. It can serve to build real-time data warehouses, enable real-time data exchange, data migration, and more.TapData CommunityTapData Community is an open-source data integration platform that provides basic data synchronization and transformation capabilities. This helps you quickly explore and implement data integration projects. As your project or business grows, you can seamlessly upgrade to TapData Cloud or TapData Enterprise to access more advanced features and service support.

Enhanced JS nodes allow you to utilize all built-in functions for external calls, such as networking and database operations. If your requirement is solely to process and operate on data records, it is recommended to use standard JS nodes.

For detailed instructions on how to use enhanced JS nodes and explore various scenarios, please refer to the documentation and resources available for JS processing node.

tip

This feature is only supported for use in data transformation tasks.

DateUtil​

parse​

Description: Converts date strings in various formats to Date.

Example:

  • General Usage

    var dte = DateUtil.parse('2010-01-01 00:00:00');
  • Advanced usage: parse(dateString, timeoffset), that is, specify the time zone offset while converting.

    // UTC+08:00
    var dte = DateUtil.parse('2010-01-01 00:00:00', 8);

    // UTC+0
    var dte = DateUtil.parse('2010-01-01 00:00:00', 0);

determineDateFormat​

Description: Get the date format.

Example:

var format = DateUtil.determineDateFormat('2010-01-01 00:00:00');

timeStamp2Date​

Description: Converts the timestamp to a date string in the specified format.

Example:

var dteStr = DateUtil.timeStamp2Date(1592233019140, 'yyyy-MM-dd HH:mm:ss');

addYears/addMonths/addDays/addHours/addMinutes/addSeconds​

Description: Adds or subtracts the year/month/day/hour/minute/second of the date.

Example:

var dte = DateUtil.addYears(new Date(), 1);
dte = DateUtil.addYears(dte, -1);

sameYear/sameMonth/sameDay/sameHour/sameMinute/sameSecond​

Description: Compares the year/month/day/hour/minute/second of the date.

Example:

if ( DataUtil.sameYear(new Date(), new Date()) ) {
...
}

idGen/UUIDGenerator​

uuid​

Description: Generate uuid, if you use var str = uuid();, you can get a random string.

Example:

// Both methods below are available
var uuid = idGen.uuid();
var uuid = UUIDGenerator.uuid();

objectId​

Description: Generate MongoDB ObjectId.

Example:

// Both methods below are available
var oid = idGen.objectId();
var oid = UUIDGenerator.objectId();

objectIdStr​

Description: Generate MongoDB ObjectId String section.

Example:

// Both methods below are available
var oidStr = idGen.objectIdStr();
var oidStr = UUIDGenerator.objectIdStr();

networkUtil​

GetAddress​

Description: Network tool to get the IP address or MAC address.

Example:

// Get the MAC address of the first network interface
var mac = networkUtil.GetAddress("mac");

// Get IP address
var ip = networkUtil.GetAddress("ip");

HashMap​

put/remove​

Description: Hash dictionary.

Example:

var map = new HashMap();
map.put(β€œname”, β€œtest”);
map.remove(β€œname”);

ArrayList​

add/remove​

Note: Array type.

Example:

var list = new ArrayList();
list.add(β€œtest1”);
list.remove(0);

Date​

add/remove​

Description: Date type.

Example:

var dte = new Date();
var year = dte.getYear()+1900;

ScriptExecutorsManager​

getScriptExecutor​

Description: Get the data source executor.

Example:

var source = ScriptExecutorsManager.getScriptExecutor('mysql-connection-name');

ScriptExecutor​

execute​

Description: When performing operations on a database, the return value is of Boolean type. True indicates a successful operation, while false indicates a failed operation.

tip

Before executing, the source component indicates performing operations on the source database, while the target component indicates performing operations on the target database.

Example:

var result = target.execute({
database: β€œtest”,
collection: β€œuser”,
op: β€œupdate”,
filter: {id: 1},
opObject: {name: β€œuser001”, age: 20},
upsert: true
});

Parameter Description

  • For structured databases (such as MySQL), you can refer to the method: var result = source.execute({sql: β€œupdate test.user set name='user001' where id = 1”});

  • For MongoDB, the available parameters are as follows:

    • database: The name of the database on which the operation is being performed.
    • collection: The collection name.
    • op: The action to be performed (INSERT/UPDATE/DELETE).
    • filter: The conditions for updating or deleting data.
    • opObject: The specific data for insertion, update, or deletion operations
    • upsert: You can choose whether to use the UPSERT mode of MongoDB, which enables inserting data if it doesn't exist and updating it if it does. The default value is false.
    • multi: You can specify whether to update multiple records. By default, it is set to false.

executeQuery​

Description: When performing database query operations, the return value is an array type that represents the result set of the query.

tip

Before executing, the source component indicates performing operations on the source database, while the target component indicates performing operations on the target database.

Example:

var users = target.executeQuery({
database: β€œtest”,
collection: β€œuser”,
filter: {age: {$gt: 10}},
sort: {age: -1},
limit: 10
});

Parameter Description

  • For structured databases (such as MySQL), you can refer to the method: var users = source.executeQuery({sql: β€œselect * from test.user where age>10”});
  • For MongoDB, the available parameters are as follows:
    • database: The name of the database being operated on.
    • collection: The name of the collection on which the operation is being performed.
    • filter: The conditions for updating or deleting data.
    • sort: Sorting condition (optional).
    • limit: Limit on the number of output records (optional).

call​

Description: Executing stored procedures and functions is supported only by structured databases. This feature enables the execution of specific database stored procedures and custom functions. The return value is in the form of key-value pairs, based on the defined result of the stored procedure.

Example:

var result = source.call('demo' [{'param1':'aa'}])

Parameter Description

  • funcName: The name of the stored procedure or function.
  • params: The supported parameters for input.
    • mode: Parameter types for input, with the following values: in (default, for input parameters), out (for output parameters), and in/out (for parameters that are both input and output).
    • name: Parameter name.
    • value: The value of the parameter.
    • type: Parameter class type.

JSONUtil​

json2List/obj2Json/obj2JsonPretty/json2Map​

Description: JSON format conversion.

Example:

var d = new Date();
var json = JSONUtil.obj2Json(d)

HanLPUtil​

hanLPParticiple​

Description: Chinese word segmentation tool, two parameters need to be set in parentheses, the format is (String inputString, String language).

Example:

var d = HanLPUtil.hanLPParticiple('δ½ ε₯½', 'HK_T')

Parameter Description

  • inputString: A string that requires word segmentation.

  • language: the type of the language with the word segmentation, support:

    • CH_S: Simplified Chinese.

    • CH_T: Traditional Chinese.

    • HK_T: Traditional Chinese (Hong Kong).

    • TW_T: Traditional Chinese (Taiwan).

Returns: Array type, that is, the result set after word segmentation.

split_chinese​

Description: Chinese word segmentation tool, two parameters need to be set in parentheses, the format is (String inputString, String language).

Example:

var strs = split_chinese("θΏ™ζ˜―δΈ€δΈͺδΈ­ζ–‡ε₯子", "CH_S");

Parameter Description

  • inputString: A string that requires word segmentation.

  • language: the type of the language with the word segmentation, support:

    • CH_S: Simplified Chinese.

    • CH_T: Traditional Chinese.

    • HK_T: Traditional Chinese (Hong Kong).

    • TW_T: Traditional Chinese (Taiwan).

Returns: Array type, that is, the result set after word segmentation.

util​

strToBase64/base64ToStr/unwind​

Description: String format conversion.

Example:

// Convert the string to Base64 format
var b = util.strToBase64('aa');
// Split JSON arrays into hierarchy levels
var list = util.unwind(map, 'a.b.c');

MD5Util/MD5​

Description: MD5 encryption tool.

Example:

// Get the MD5 signature of a string, the second parameter indicates whether to convert it to uppercase
var b = MD5Util.crypt('aa', true);
// Or
var b = MD5('aa', true);

Collections​

sort/get/emptySet/emptyList​

Description: Collection tool classes, such as sorting, getting collections, etc.

Example:

// Sort the List
Collections.sort(list);
// Get an empty collection
var set = Collections.emptySet();

MapUtil​

getValueByKey/needSplit/removeValueByKey/containsKey/getValuePositionInMap/deepCloneMap/copyToNewMap/putValueInMap/recursiveFlatMap/obj2Map​

Description: Dictionary tool class.

Example:

// Get the value of a specified level from a given map
var a = MapUtil.getValueByKey(map, 'a.b.c');

sleep​

Description: The program sleeps for a specified duration, measured in milliseconds.

Example:

// Sleep for 10 milliseconds in the program
sleep(10);

rest​

get/post/patch/delete​

Description: Call HTTP methods (such as Get), format reference:

rest.get(url, header)
rest.get(url, header, returnType)
rest.get(url, header, connectTimeOut, readTimeOut)
rest.get(url, header, returnType, connectTimeOut, readTimeOut)
  • returnType: The default return result type is an array.
  • connectTimeOut: The connection timeout duration in milliseconds. The default value is 10,000 milliseconds (10 seconds).
  • readTimeOut: The read timeout duration in milliseconds. The default value is 30,000 milliseconds (30 seconds).

Example:

  • Get

    var result = rest.get('http://127.0.0.1:1234/users?where[user_id]=1', {status: 0}, {}, 30, 300);rest.get(url)
    rest.get(url, headers)
    rest.get(url, connectTimeOut, readTimeOut)
    rest.get(url, headers, connectTimeOut, readTimeOut)
  • Post

    var result = rest.post('http://127.0.0.1:1234/users?id=1', {}, '[array/object/string]', 30, 300);rest.post(url, parameters)
    rest.post(url, parameters, headers, returnType)
    rest.post(url, parameters, connectTimeOut, readTimeOut)
    rest.post(url, parameters, headers, returnType, connectTimeOut, readTimeOut)
  • Patch

    var result = rest.patch('http://127.0.0.1:1234/users/find', {}, {}, '[array/object/string]', 30, 300);rest.patch(url, parameters)
    rest.patch(url, parameters, headers)
    rest.patch(url, parameters, connectTimeOut, readTimeOut)
    rest.patch(url, parameters, headers, connectTimeOut, readTimeOut)
  • Delete

    var result = rest.delete('http://127.0.0.1:1234/users?where[user_id]=1', {}, 30, 300);

mongo​

getData/insert/update/delete​

Description: Add, delete and check data in MongoDB, format reference:

mongo.getData(uri, collection)
mongo.getData(uri, collection, filter)
mongo.getData(uri, collection, filter, limit, sort)

Example:

  • Query Data

    var result = mongo.getData('mongodb://127.0.0.1:27017/test', 'users', {id: 1}, 10, {add_time: -1});mongo.insert(url, collection, inserts)
  • Insert data, supporting input of arrays or objects.

    mongo.insert('mongodb://127.0.0.1:27017/test', 'users', [{id: 1, name: 'test1'}, {id: 2, name: 'test2'}]);mongo.update(url, collection, filter, update)
  • Update data

    var modifyCount = mongo.update('mongodb://127.0.0.1:27017/test', 'users', {id: 1}, {name: 'test3'});mongo.delete(url, collection, filter)
  • Delete Data

    var deleteCount = mongo.delete('mongodb://127.0.0.1:27017/test', 'users', {id: 1});