Enhanced JS Built-in Function
TapData Cloud offers you cloud services that are suitable for scenarios requiring rapid deployment and low initial investment, helping you focus more on business development rather than infrastructure management. Free trial with TapData Cloud.TapData Enterprise can be deployed in your local data center, making it suitable for scenarios with strict requirements on data sensitivity or network isolation. It can serve to build real-time data warehouses, enable real-time data exchange, data migration, and more.TapData Community is an open-source data integration platform that provides basic data synchronization and transformation capabilities. This helps you quickly explore and implement data integration projects. As your project or business grows, you can seamlessly upgrade to TapData Cloud or TapData Enterprise to access more advanced features and service support.Enhanced JS nodes allow you to utilize all built-in functions for external calls, such as networking and database operations. If your requirement is solely to process and operate on data records, it is recommended to use standard JS nodes.
For detailed instructions on how to use enhanced JS nodes and explore various scenarios, please refer to the documentation and resources available for JS processing node.
This feature is only supported for use in data transformation tasks.
DateUtilβ
parseβ
Description: Converts date strings in various formats to Date.
Example:
General Usage
var dte = DateUtil.parse('2010-01-01 00:00:00');
Advanced usage:
parse(dateString, timeoffset)
, that is, specify the time zone offset while converting.// UTC+08:00
var dte = DateUtil.parse('2010-01-01 00:00:00', 8);
// UTC+0
var dte = DateUtil.parse('2010-01-01 00:00:00', 0);
determineDateFormatβ
Description: Get the date format.
Example:
var format = DateUtil.determineDateFormat('2010-01-01 00:00:00');
timeStamp2Dateβ
Description: Converts the timestamp to a date string in the specified format.
Example:
var dteStr = DateUtil.timeStamp2Date(1592233019140, 'yyyy-MM-dd HH:mm:ss');
addYears/addMonths/addDays/addHours/addMinutes/addSecondsβ
Description: Adds or subtracts the year/month/day/hour/minute/second of the date.
Example:
var dte = DateUtil.addYears(new Date(), 1);
dte = DateUtil.addYears(dte, -1);
sameYear/sameMonth/sameDay/sameHour/sameMinute/sameSecondβ
Description: Compares the year/month/day/hour/minute/second of the date.
Example:
if ( DataUtil.sameYear(new Date(), new Date()) ) {
...
}
idGen/UUIDGeneratorβ
uuidβ
Description: Generate uuid, if you use var str = uuid();
, you can get a random string.
Example:
// Both methods below are available
var uuid = idGen.uuid();
var uuid = UUIDGenerator.uuid();
objectIdβ
Description: Generate MongoDB ObjectId.
Example:
// Both methods below are available
var oid = idGen.objectId();
var oid = UUIDGenerator.objectId();
objectIdStrβ
Description: Generate MongoDB ObjectId String section.
Example:
// Both methods below are available
var oidStr = idGen.objectIdStr();
var oidStr = UUIDGenerator.objectIdStr();
networkUtilβ
GetAddressβ
Description: Network tool to get the IP address or MAC address.
Example:
// Get the MAC address of the first network interface
var mac = networkUtil.GetAddress("mac");
// Get IP address
var ip = networkUtil.GetAddress("ip");
HashMapβ
put/removeβ
Description: Hash dictionary.
Example:
var map = new HashMap();
map.put(βnameβ, βtestβ);
map.remove(βnameβ);
ArrayListβ
add/removeβ
Note: Array type.
Example:
var list = new ArrayList();
list.add(βtest1β);
list.remove(0);
Dateβ
add/removeβ
Description: Date type.
Example:
var dte = new Date();
var year = dte.getYear()+1900;
ScriptExecutorsManagerβ
getScriptExecutorβ
Description: Get the data source executor.
Example:
var source = ScriptExecutorsManager.getScriptExecutor('mysql-connection-name');
ScriptExecutorβ
executeβ
Description: When performing operations on a database, the return value is of Boolean type. True indicates a successful operation, while false indicates a failed operation.
Before executing, the source component indicates performing operations on the source database, while the target component indicates performing operations on the target database.
Example:
var result = target.execute({
database: βtestβ,
collection: βuserβ,
op: βupdateβ,
filter: {id: 1},
opObject: {name: βuser001β, age: 20},
upsert: true
});
Parameter Description
For structured databases (such as MySQL), you can refer to the method:
var result = source.execute({sql: βupdate test.user set name='user001' where id = 1β});
For MongoDB, the available parameters are as follows:
- database: The name of the database on which the operation is being performed.
- collection: The collection name.
- op: The action to be performed (INSERT/UPDATE/DELETE).
- filter: The conditions for updating or deleting data.
- opObject: The specific data for insertion, update, or deletion operations
- upsert: You can choose whether to use the UPSERT mode of MongoDB, which enables inserting data if it doesn't exist and updating it if it does. The default value is false.
- multi: You can specify whether to update multiple records. By default, it is set to false.
executeQueryβ
Description: When performing database query operations, the return value is an array type that represents the result set of the query.
Before executing, the source component indicates performing operations on the source database, while the target component indicates performing operations on the target database.
Example:
var users = target.executeQuery({
database: βtestβ,
collection: βuserβ,
filter: {age: {$gt: 10}}οΌ
sort: {age: -1},
limit: 10
});
Parameter Description
- For structured databases (such as MySQL), you can refer to the method:
var users = source.executeQuery({sql: βselect * from test.user where age>10β});
- For MongoDB, the available parameters are as follows:
- database: The name of the database being operated on.
- collection: The name of the collection on which the operation is being performed.
- filter: The conditions for updating or deleting data.
- sort: Sorting condition (optional).
- limit: Limit on the number of output records (optional).
callβ
Description: Executing stored procedures and functions is supported only by structured databases. This feature enables the execution of specific database stored procedures and custom functions. The return value is in the form of key-value pairs, based on the defined result of the stored procedure.
Example:
var result = source.call('demo' [{'param1':'aa'}])
Parameter Description
- funcName: The name of the stored procedure or function.
- params: The supported parameters for input.
- mode: Parameter types for input, with the following values: in (default, for input parameters), out (for output parameters), and in/out (for parameters that are both input and output).
- name: Parameter name.
- value: The value of the parameter.
- type: Parameter class type.
JSONUtilβ
json2List/obj2Json/obj2JsonPretty/json2Mapβ
Description: JSON format conversion.
Example:
var d = new Date();
var json = JSONUtil.obj2Json(d)
HanLPUtilβ
hanLPParticipleβ
Description: Chinese word segmentation tool, two parameters need to be set in parentheses, the format is (String inputString, String language)
.
Example:
var d = HanLPUtil.hanLPParticiple('δ½ ε₯½', 'HK_T')
Parameter Description
inputString: A string that requires word segmentation.
language: the type of the language with the word segmentation, support:
CH_S: Simplified Chinese.
CH_T: Traditional Chinese.
HK_T: Traditional Chinese (Hong Kong).
TW_T: Traditional Chinese (Taiwan).
Returns: Array type, that is, the result set after word segmentation.
split_chineseβ
Description: Chinese word segmentation tool, two parameters need to be set in parentheses, the format is (String inputString, String language)
.
Example:
var strs = split_chinese("θΏζ―δΈδΈͺδΈζε₯ε", "CH_S");
Parameter Description
inputString: A string that requires word segmentation.
language: the type of the language with the word segmentation, support:
CH_S: Simplified Chinese.
CH_T: Traditional Chinese.
HK_T: Traditional Chinese (Hong Kong).
TW_T: Traditional Chinese (Taiwan).
Returns: Array type, that is, the result set after word segmentation.
utilβ
strToBase64/base64ToStr/unwindβ
Description: String format conversion.
Example:
// Convert the string to Base64 format
var b = util.strToBase64('aa');
// Split JSON arrays into hierarchy levels
var list = util.unwind(map, 'a.b.c');
MD5Util/MD5β
Description: MD5 encryption tool.
Example:
// Get the MD5 signature of a string, the second parameter indicates whether to convert it to uppercase
var b = MD5Util.crypt('aa', true);
// Or
var b = MD5('aa', true);
Collectionsβ
sort/get/emptySet/emptyListβ
Description: Collection tool classes, such as sorting, getting collections, etc.
Example:
// Sort the List
Collections.sort(list);
// Get an empty collection
var set = Collections.emptySet();
MapUtilβ
getValueByKey/needSplit/removeValueByKey/containsKey/getValuePositionInMap/deepCloneMap/copyToNewMap/putValueInMap/recursiveFlatMap/obj2Mapβ
Description: Dictionary tool class.
Example:
// Get the value of a specified level from a given map
var a = MapUtil.getValueByKey(map, 'a.b.c');
sleepβ
Description: The program sleeps for a specified duration, measured in milliseconds.
Example:
// Sleep for 10 milliseconds in the program
sleep(10);
restβ
get/post/patch/deleteβ
Description: Call HTTP methods (such as Get), format reference:
rest.get(url, header)
rest.get(url, header, returnType)
rest.get(url, header, connectTimeOut, readTimeOut)
rest.get(url, header, returnType, connectTimeOut, readTimeOut)
- returnType: The default return result type is an array.
- connectTimeOut: The connection timeout duration in milliseconds. The default value is 10,000 milliseconds (10 seconds).
- readTimeOut: The read timeout duration in milliseconds. The default value is 30,000 milliseconds (30 seconds).
Example:
Get
var result = rest.get('http://127.0.0.1:1234/users?where[user_id]=1', {status: 0}, {}, 30, 300);rest.get(url)
rest.get(url, headers)
rest.get(url, connectTimeOut, readTimeOut)
rest.get(url, headers, connectTimeOut, readTimeOut)Post
var result = rest.post('http://127.0.0.1:1234/users?id=1', {}, '[array/object/string]', 30, 300);rest.post(url, parameters)
rest.post(url, parameters, headers, returnType)
rest.post(url, parameters, connectTimeOut, readTimeOut)
rest.post(url, parameters, headers, returnType, connectTimeOut, readTimeOut)Patch
var result = rest.patch('http://127.0.0.1:1234/users/find', {}, {}, '[array/object/string]', 30, 300);rest.patch(url, parameters)
rest.patch(url, parameters, headers)
rest.patch(url, parameters, connectTimeOut, readTimeOut)
rest.patch(url, parameters, headers, connectTimeOut, readTimeOut)Delete
var result = rest.delete('http://127.0.0.1:1234/users?where[user_id]=1', {}, 30, 300);
mongoβ
getData/insert/update/deleteβ
Description: Add, delete and check data in MongoDB, format reference:
mongo.getData(uri, collection)
mongo.getData(uri, collection, filter)
mongo.getData(uri, collection, filter, limit, sort)
Example:
Query Data
var result = mongo.getData('mongodb://127.0.0.1:27017/test', 'users', {id: 1}, 10, {add_time: -1});mongo.insert(url, collection, inserts)
Insert data, supporting input of arrays or objects.
mongo.insert('mongodb://127.0.0.1:27017/test', 'users', [{id: 1, name: 'test1'}, {id: 2, name: 'test2'}]);mongo.update(url, collection, filter, update)
Update data
var modifyCount = mongo.update('mongodb://127.0.0.1:27017/test', 'users', {id: 1}, {name: 'test3'});mongo.delete(url, collection, filter)
Delete Data
var deleteCount = mongo.delete('mongodb://127.0.0.1:27017/test', 'users', {id: 1});