Can Dolphinscheduler log stack trace or detailed error log when task failed?

[WilliamChen-luckbob]

Here is the thing:

So far we decide to use DS to integrate different tasks which are coded by different languages ( We have multiple developing departments using different languages).

Comparing with Airflow, our Python developers complain about the error log in Dolphinscheduler for there is no error stack trace when the dag failed.

It is quite difficult to find out what's going on when a dag was crashed by exception, we have to code acpects into our code and then try to collect the instance id of each running dag. I though it might be a good practice to integrate error-stack-trace or api-calling-trace by plugins or a super interface in DS if there is something wrong running dags.

And that should be a proper way to support by DS.

Otherwise, there might be another way like setting a log server, we upload stack trace by plugins such as logtail or someting if we need.

For another idea, we use Aliyun SLS for log, is there a way to do a log implement that let us contain a dag-instance-id and task-instance-id in our log so that we can directly query detailed log from DS (actually DS querying SLS in the backend)