Replies: 10 comments 2 replies
-
|
Example for Basic Project
基础类项目示例
|
Beta Was this translation helpful? Give feedback.
-
|
Example for Advanced Project
进阶类项目示例
|
Beta Was this translation helpful? Give feedback.
-
Resource center optimization
资源中心优化
|
Beta Was this translation helpful? Give feedback.
-
|
Master/Worker/ API log optimization
master/worker/api模块日志优化 描述：由于DS日志不方便真实定位问题，需要将打印日志进行规范及优化 |
Beta Was this translation helpful? Give feedback.
-
TopicAdd file reader plugin for DolphinScheduler Python API label
DescriptionCurrently, DolphinScheduler Python API only could pass string format arg as tasks' parameter, such as our tutorial we only pass a string to shell task type. But generally, users hope the command is stored in other resources(for easier maintenance and integration), such as local file systems, GitHub, GitLab, Amazon S3, Aliyun OSS and etc. Maybe we could add trick like
@Resource(
type="github",
id="https://github.com/apache/dolphinscheduler/tree/dev/script"
)
with ProcessDefinition(
name="tutorial",
schedule="0 0 0 * * ? *",
start_time="2021-01-01",
tenant="tenant_exists",
) as pd:
task_parent = Shell(name="task_parent", command="install.sh")
task_child_one = Shell(name="task_child_one", command="start-all.sh'")
task_child_two = Shell(name="task_child_two", command="stop-all.sh")
task_union = Shell(name="task_union", command="status-all.sh")And task task_parent will load file content from URL https://github.com/apache/dolphinscheduler/blob/dev/script/install.sh and pass it to parameter Output Standard
Requirements
What Can You LearnWe wish everyone joining SoC could learn some things from the project. When you finish this project, you could learn:
课题为 DolphinScheduler Python API 添加资源文件加载插件 标签
描述目前，DolphinScheduler Python API 只能传递字符串格式传递任务的参数，例如我们的 tutorial 我们只将一个字符串传递给 shell 任务类型。但通常来说，用户希望具体的执行代码储存在其他系统中（希望更好维护和集成），例如将具体文件存储在本地文件系统、GitHub、GitLab、Amazon S3、阿里云 OSS 等。也许我们可以添加语法糖让用户更加简单的操作外部文件
@Resource(
type="github",
id="https://github.com/apache/dolphinscheduler/tree/dev/script"
)
with ProcessDefinition(
name="tutorial",
schedule="0 0 0 * * ? *",
start_time="2021-01-01",
tenant="tenant_exists",
) as pd:
task_parent = Shell(name="task_parent", command="install.sh")
task_child_one = Shell(name="task_child_one", command="start-all.sh'")
task_child_two = Shell(name="task_child_two", command="stop-all.sh")
task_union = Shell(name="task_union", command="status-all.sh")任务 task_parent 将从 URL https://github.com/apache/dolphinscheduler/blob/dev/script/install.sh 加载文件内容并将其传递给参数 产出
对报名者要求
从中学到什么我们希望每个加入 SoC 的人都能从项目中学到一些东西。当你完成这个项目时，你可以学到：
|
Beta Was this translation helpful? Give feedback.
-
Topicdetect and store all of application ids of yarn jobs automatically, instead of extracting them in task logs with a regexp Label
DescriptionDolphinScheduler support a wide spectrum of tasks, such as python, shell, etc. A basic function is to collect and kill all of the yarn jobs submitted by these tasks. Output Standard
Requirements
What Can You LearnWe wish everyone joining SoC could learn some things from the project. When you finish this project, you could learn Yarn, Hive, Spark SQL, Aspect-Oriented Programming 课题自动收集、保存作业提交的Yarn作业的application id 标签
描述DolphinScheduler支持各种各样的作业，比如Python、Shell等，其中一个很基础的功能就是收集这些作业创建的Yarn Job的application id。现在的实现方式是用一个正则表达式从作业的日志中解析，但这并不是一个完美的实现方式，本课题的任务就是用一种新的方式去自动收集、报错Yarn Job的application id信息，当然了，必须对用户透明，即对用户创建的作业无侵入。 产出
对报名者要求
从中学到什么我们希望每个加入 SoC 的人都能从项目中学到一些东西。当你完成这个项目时，你可以学到：
|
Beta Was this translation helpful? Give feedback.
-
|
@gabrywu Hi, May I ask is this feature implemented now? |
Beta Was this translation helpful? Give feedback.
-
|
Yes |
Beta Was this translation helpful? Give feedback.
-
TopicAdd workflow demos for DolphinScheduler server initialization. Label
DescriptionAfter users start the DolphinScheduler service, they can use the demo-tool program to preset workflow demos,, including but not limited to simple shell tasks, logical tasks (switch, dependent, subprocess, condition), parameter passing and other functions, which can guide users more easily use DolphinScheduler. Output Standard
Requirements
What can we learnWe hope everyone who joins SoC will learn something from the project. When you complete this project, you can learn:
课题为 DolphinScheduler 添加服务初始化的工作流demo 标签
描述用户在启动DolphinScheduler服务后，可以使用demo-tool程序预置工作流demo，包括但不限于简单的shell任务、逻辑组件任务（switch、dependent、subprocess、condition）、参数传递等功能， 引导用户更方便地使用DolphinScheduler。 产出
对报名者要求
从中学到什么我们希望每个加入 SoC 的人都能从项目中学到一些东西。当你完成这个项目时，你可以学到：
|
Beta Was this translation helpful? Give feedback.
-
TopicAdd Java task type for DolphinScheduler Label
DescriptionProvides Java type scheduling tasks, including Jar and custom Java code, and defines the Resources, Libs resource files, and JVM parameters required by Java runtime through the WEBUI, cover timed task calls Java program scenarios and enhance user-friendly configuration. Output Standard
Requirements
What can we learnWe hope everyone who joins SoC will learn something from the project. When you complete this project, you can learn:
课题为 DolphinScheduler 添加 Java 任务类型 标签
描述提供Java类型的调度任务，包括Jar和自定义Java代码两种方式，通过WEBUI定义Java运行时所需要的Resources、Libs资源文件和JVM参数，覆盖定时任务调用Java程序场景并提升用户方便配置。 产出
对报名者要求
从中学到什么我们希望每个加入 SoC 的人都能从项目中学到一些东西。当你完成这个项目时，你可以学到：
|
Beta Was this translation helpful? Give feedback.
-
TopicSupport test tasks Label
DescriptionAt present, it is difficult to verify whether the workflow configuration is correct before the new/modified workflow goes live, and can only be verified by executing it in the online environment, so it is necessary to support the test environment to execute tasks. Output Standard
Requirements
What can we learnWe hope everyone who joins SoC will learn something from the project. When you complete this project, you can learn:
课题支持测试任务 标签
描述目前新增/修改的工作流在上线前很难验证工作流配置是否正确，只能通过在线上环境执行来验证，所以有必要支持测试环境执行任务。 产出
对报名者要求
从中学到什么我们希望每个加入 SoC 的人都能从项目中学到一些东西。当你完成这个项目时，你可以学到：
|
Beta Was this translation helpful? Give feedback.
-
TopicAdd authorization management of read and write permissions for DolphinScheduler's project and resource center LabelSupported language: English DescriptionAt present, DolphinScheduler's control of permissions is relatively simple, and it is hoped to increase the control of operation permissions to separate read and write permissions when authorizing projects and resource centers. Output Standard
RequirementsFamiliar with Java、Spring、Vue3、TypeScript What can we learnWe hope everyone who joins SoC will learn something from the project. When you complete this project, you can learn:
课题为 DolphinScheduler 的项目和资源中心增加读写权限的授权管理 标签
描述当前DolphinScheduler对于权限的管控相对比较简单，希望在进行项目和资源中心授权时增加操作权限管控来进行读写权限的分离。 产出
对报名者要求熟悉Java、Spring、Vue3、TypeScript 从中学到什么我们希望每个加入 SoC 的人都能从项目中学到一些东西。当你完成这个项目时，你可以学到：
|
Beta Was this translation helpful? Give feedback.
-
Hi, all, DS has participated in the OSPP 2022 event [1] and can publish 8 projects.
If you have good ideas, you can post your ideas here for discussion. The idea includes 3 contents
Deadline May 10, 2022.
[1] https://mp.weixin.qq.com/s/3ROTqTSE9DlfClHkPQbwTw
Beta Was this translation helpful? Give feedback.
All reactions