[Improvement] [Dynamic] Using the Datax task on the second dynamic node is not supported when using two dynamic nodes

[Alexios-cd]

Search before asking

• I had searched in the issues and found no similar issues.

What happened.

I use two dynamic nodes in a workflow, and even dynamic nodes contain DataX tasks.
Runtime discovery:
The first dynamic node runs successfully, and the second dynamic node runs unsuccessfully.
The reason for the failure is that the second dynamic node accepts the parameters of the first dynamic node, and the java in datax is not compatible with the passing form of the parameters.

log is：
[INFO] 2024-01-03 16:43:20.119 +0800 - Final Shell file is :
#!/bin/bash
BASEDIR=$(cd dirname $0; pwd)
cd $BASEDIR
${PYTHON_LAUNCHER} ${DATAX_LAUNCHER} --jvm="-Xms1G -Xmx1G" -p "-Dp1='f2' -Dsystem.task.definition.name='datax_out' -Dsystem.project.name='null' -Dsystem.biz.curdate='20240103' -Dsystem.task.instance.id='742' -DStartParams='{"p1":"f2"}' -Dsystem.task.definition.code='12139625072448' -Dsystem.datetime='20240103164320' -DdynamicParams='{"p1":"f2"}' -Dsystem.project.code='12139492154560' -Dsystem.workflow.instance.id='127' -Dsystem.biz.date='20240102' -Ddynamic.out(data_in)='[{"dynParams":{"p1":"fw"},"outputValue":{},"mappedTimes":1},{"dynParams":{"p1":"lr"},"outputValue":{},"mappedTimes":2}]' -Dsystem.workflow.definition.name='data_out' -Dsystem.workflow.definition.code='12139504069824'" /tmp/dolphinscheduler/exec/process/default/12139492154560/12139504069824_3/127/742/127_742_job.json
[INFO] 2024-01-03 16:43:20.119 +0800 - Executing shell command : sudo -u dolphinscheduler -i /tmp/dolphinscheduler/exec/process/default/12139492154560/12139504069824_3/127/742/127_742.sh
[INFO] 2024-01-03 16:43:20.125 +0800 - process start, process id is: 18875
[INFO] 2024-01-03 16:43:21.125 +0800 - ->

DataX (DATAX-OPENSOURCE-3.0), From Alibaba !
Copyright (C) 2010-2017, Alibaba Group. All Rights Reserved.


/bin/sh: -c:行0: 未预期的符号 `(' 附近有语法错误
/bin/sh: -c:行0: `java -server -Xms1g -Xmx1g -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/home/setup/datax/log -Xms1G -Xmx1G -Dloglevel=info -Dfile.encoding=UTF-8 -Dlogback.statusListenerClass=ch.qos.logback.core.status.NopStatusListener -Djava.security.egd=file:///dev/urandom -Ddatax.home=/home/setup/datax -Dlogback.configurationFile=/home/setup/datax/conf/logback.xml -classpath /home/setup/datax/lib/*:.  -Dlog.file.name=742_127_742_job_json -Dp1='f2' -Dsystem.task.definition.name='datax_out' -Dsystem.project.name='null' -Dsystem.biz.curdate='20240103' -Dsystem.task.instance.id='742' -DStartParams='{p1:f2}' -Dsystem.task.definition.code='12139625072448' -Dsystem.datetime='20240103164320' -DdynamicParams='{p1:f2}' -Dsystem.project.code='12139492154560' -Dsystem.workflow.instance.id='127' -Dsystem.biz.date='20240102' -Ddynamic.out(data_in)='[{dynParams:{p1:fw},outputValue:{},mappedTimes:1},{dynParams:{p1:lr},outputValue:{},mappedTimes:2}]' -Dsystem.workflow.definition.name='data_out' -Dsystem.workflow.definition.code='12139504069824' com.alibaba.datax.core.Engine -mode standalone -jobid -1 -job /tmp/dolphinscheduler/exec/process/default/12139492154560/12139504069824_3/127/742/127_742_job.json'

[INFO] 2024-01-03 16:43:21.129 +0800 - process has exited. execute path:/tmp/dolphinscheduler/exec/process/default/12139492154560/12139504069824_3/127/742, processId:18875 ,exitStatusCode:1 ,processWaitForStatus:true ,processExitValue:1

What you expected to happen

The Java command does not recognize parameters of type ${dynamic.out(TaskName)} because it has "()"

How to reproduce

Place two dynamic nodes like I did, and place the datax task of the custom template in the second dynamic node.

Anything else

No response

Version

3.2.x

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[github-actions]

Search before asking

• I had searched in the issues and found no similar issues.

What happened

I use two dynamic nodes in a workflow, and even dynamic nodes contain DataX tasks.
Runtime discovery:
The first dynamic node runs successfully, and the second dynamic node runs unsuccessfully.
The reason for the failure is that the second dynamic node accepts the parameters of the first dynamic node, and the java in datax is not compatible with the passing form of the parameters.

log is：
[INFO] 2024-01-03 16:43:20.119 +0800 - Final Shell file is :
#!/bin/bash
BASEDIR=$(cd dirname $0; pwd)
cd $BASEDIR
${PYTHON_LAUNCHER} ${DATAX_LAUNCHER} --jvm="-Xms1G -Xmx1G" -p "-Dp1='f2' -Dsystem.task.definition.name='datax_out' -Dsystem.project.name='null' -Dsystem.biz.curdate='20240103' -Dsystem.task.instance.id='742' -DStartParams='{"p1":"f2"}' -Dsystem.task.definition.code='12139625072448' -Dsystem.datetime='20240103164320' -DdynamicParams='{"p1":"f2"}' -Dsystem.project.code='12139492154560' -Dsystem.workflow.instance.id='127' -Dsystem.biz.date='20240102' -Ddynamic.out(data_in)='[{"dynParams":{"p1":"fw"},"outputValue":{},"mappedTimes":1},{"dynParams":{"p1":"lr"},"outputValue":{},"mappedTimes":2}]' -Dsystem.workflow.definition.name='data_out' -Dsystem.workflow.definition.code='12139504069824'" /tmp/dolphinscheduler/exec/process/default/12139492154560/12139504069824_3/127/742/127_742_job.json
[INFO] 2024-01-03 16:43:20.119 +0800 - Executing shell command : sudo -u dolphinscheduler -i /tmp/dolphinscheduler/exec/process/default/12139492154560/12139504069824_3/127/742/127_742.sh
[INFO] 2024-01-03 16:43:20.125 +0800 - process start, process id is: 18875
[INFO] 2024-01-03 16:43:21.125 +0800 - ->

DataX (DATAX-OPENSOURCE-3.0), From Alibaba !
Copyright (C) 2010-2017, Alibaba Group. All Rights Reserved.


/bin/sh: -c:行0: 未预期的符号 `(' 附近有语法错误
/bin/sh: -c:行0: `java -server -Xms1g -Xmx1g -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/home/setup/datax/log -Xms1G -Xmx1G -Dloglevel=info -Dfile.encoding=UTF-8 -Dlogback.statusListenerClass=ch.qos.logback.core.status.NopStatusListener -Djava.security.egd=file:///dev/urandom -Ddatax.home=/home/setup/datax -Dlogback.configurationFile=/home/setup/datax/conf/logback.xml -classpath /home/setup/datax/lib/*:.  -Dlog.file.name=742_127_742_job_json -Dp1='f2' -Dsystem.task.definition.name='datax_out' -Dsystem.project.name='null' -Dsystem.biz.curdate='20240103' -Dsystem.task.instance.id='742' -DStartParams='{p1:f2}' -Dsystem.task.definition.code='12139625072448' -Dsystem.datetime='20240103164320' -DdynamicParams='{p1:f2}' -Dsystem.project.code='12139492154560' -Dsystem.workflow.instance.id='127' -Dsystem.biz.date='20240102' -Ddynamic.out(data_in)='[{dynParams:{p1:fw},outputValue:{},mappedTimes:1},{dynParams:{p1:lr},outputValue:{},mappedTimes:2}]' -Dsystem.workflow.definition.name='data_out' -Dsystem.workflow.definition.code='12139504069824' com.alibaba.datax.core.Engine -mode standalone -jobid -1 -job /tmp/dolphinscheduler/exec/process/default/12139492154560/12139504069824_3/127/742/127_742_job.json'

[INFO] 2024-01-03 16:43:21.129 +0800 - process has exited. execute path:/tmp/dolphinscheduler/exec/process/default/12139492154560/12139504069824_3/127/742, processId:18875 ,exitStatusCode:1 ,processWaitForStatus:true ,processExitValue:1

What you expected to happen

The Java command does not recognize parameters of type ${dynamic.out(TaskName)} because it has "()"

How to reproduce

Place two dynamic nodes like I did, and place the datax task of the custom template in the second dynamic node.

Anything else

No response

Version

3.2.x

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct