java 集成 dolphinscheduler

Java集成DolphinScheduler

介绍

DolphinScheduler是一款开源的分布式可视化DAG工作流任务调度系统，用于解决大数据任务调度和依赖管理问题。它支持多种任务类型，如Shell任务、Python任务、Hive任务等，并提供了丰富的监控和告警功能。

本文将介绍如何在Java项目中集成DolphinScheduler，以便能够通过Java代码动态创建和管理工作流任务。

集成步骤

步骤1：添加依赖

首先，在你的Java项目的pom.xml文件中添加DolphinScheduler的依赖：

<dependency>
    <groupId>org.apache.dolphinscheduler</groupId>
    <artifactId>dolphinscheduler-api</artifactId>
    <version>1.4.0</version>
</dependency>

步骤2：创建DolphinScheduler的客户端

接下来，你需要创建一个DolphinScheduler的客户端，用于与DolphinScheduler进行交互。可以创建一个单例的客户端，以便在整个项目中共享该客户端。

import org.apache.dolphinscheduler.api.client.DolphinSchedulerClient;
import org.apache.dolphinscheduler.api.enums.ExecuteType;
import org.apache.dolphinscheduler.api.exceptions.DolphinSchedulerException;
import org.apache.dolphinscheduler.api.model.Result;
import org.apache.dolphinscheduler.api.utils.FileUtils;
import org.apache.dolphinscheduler.api.utils.StringUtils;
import org.apache.dolphinscheduler.api.utils.ZookeeperUtils;
import org.apache.dolphinscheduler.api.utils.ZookeeperUtils.ZkClient;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class DolphinSchedulerClient {

    private static final Logger logger = LoggerFactory.getLogger(DolphinSchedulerClient.class);

    private static final String ZOOKEEPER_QUORUM = "localhost:2181";
    private static final String ZOOKEEPER_ROOT = "/dolphinscheduler";

    private static final DolphinSchedulerClient INSTANCE = new DolphinSchedulerClient();

    private DolphinSchedulerClient() {
    }

    public static DolphinSchedulerClient getInstance() {
        return INSTANCE;
    }

    public Result createProcessDefinition(String name, String description, String taskNodeJson) {
        try (ZkClient zkClient = ZookeeperUtils.getZkClient(ZOOKEEPER_QUORUM, ZOOKEEPER_ROOT)) {
            DolphinSchedulerClientBuilder clientBuilder = new DolphinSchedulerClientBuilder()
                    .setZookeeperQuorum(ZOOKEEPER_QUORUM)
                    .setZookeeperRoot(ZOOKEEPER_ROOT)
                    .setZkClient(zkClient);

            try (DolphinSchedulerClient client = clientBuilder.build()) {
                CreateProcessDefinitionRequest request = new CreateProcessDefinitionRequest()
                        .setName(name)
                        .setDescription(description)
                        .setTaskNodeJson(taskNodeJson)
                        .setExecuteType(ExecuteType.STANDALONE);

                return client.createProcessDefinition(request);
            } catch (Exception e) {
                logger.error("Failed to communicate with DolphinScheduler server", e);
                throw new DolphinSchedulerException("Failed to communicate with DolphinScheduler server", e);
            }
        } catch (Exception e) {
            logger.error("Failed to connect to ZooKeeper", e);
            throw new DolphinSchedulerException("Failed to connect to ZooKeeper", e);
        }
    }
}

步骤3：创建工作流任务

现在，你可以使用上述客户端来创建工作流任务了。下面是一个创建工作流任务的示例代码：

public class WorkflowTaskExample {

    public static void main(String[] args) {
        DolphinSchedulerClient client = DolphinSchedulerClient.getInstance();

        String name = "Example Workflow";
        String description = "This is an example workflow";
        String taskNodeJson = FileUtils.readFromResource("workflow_task.json");

        Result result = client.createProcessDefinition(name, description, taskNodeJson);

        if (result.isSuccess()) {
            System.out.println("Workflow task created successfully");
        } else {
            System.out.println("Failed to create workflow task: " + result.getMsg());
        }
    }
}

步骤4：创建工作流任务定义文件

在上述示例代码中，taskNodeJson是一个工作流任务的定义文件。你需要根据具体需求创建该文件。下面是一个示例的工作流任务定义文件：

{
    "globalParams": [],
    "tasks": [
        {
            "type": "SHELL",
            "id": "task_1",
            "name": "Task 1",
            "params": {
                "resourceList": [],
                "localParams": [],
                "rawScript": "echo Hello, DolphinScheduler!"
            },
            "desc": ""
        }
    ],
    "