|
|
@@ -1,105 +1,70 @@
|
|
|
-Dolphin Scheduler Official Website
|
|
|
-[dolphinscheduler.apache.org](https://dolphinscheduler.apache.org)
|
|
|
-==================================================================
|
|
|
+# Apache Dolphinscheduler
|
|
|
|
|
|
[](https://www.apache.org/licenses/LICENSE-2.0.html)
|
|
|
-[]()
|
|
|
+
|
|
|
[](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler)
|
|
|
[](https://twitter.com/dolphinschedule)
|
|
|
[](https://s.apache.org/dolphinscheduler-slack)
|
|
|
+[](README_zh_CN.md)
|
|
|
|
|
|
-## Features
|
|
|
+## About
|
|
|
|
|
|
-Apache DolphinScheduler is the modern data workflow orchestration platform with powerful user interface, dedicated to solving complex task dependencies in the data pipeline and providing various types of jobs available `out of the box`
|
|
|
+Apache DolphinScheduler is the modern data orchestration platform. Agile to create high performance workflow with low-code. It is also provided powerful user interface,
|
|
|
+dedicated to solving complex task dependencies in the data pipeline and providing various types of jobs available **out of the box**
|
|
|
|
|
|
The key features for DolphinScheduler are as follows:
|
|
|
|
|
|
-- Easy to deploy, we provide 4 ways to deploy, such as Standalone deployment,Cluster deployment,Docker / Kubernetes deployment and Rainbond deployment
|
|
|
-- Easy to use, there are four ways to create workflows:
|
|
|
-
|
|
|
- - Visually, create tasks by dragging and dropping tasks
|
|
|
- - [PyDolphinScheduler](https://dolphinscheduler.apache.org/python/main/index.html), Creating workflows via Python API, aka workflow-as-code
|
|
|
- - Yaml definition, mapping yaml into workflow(have to install PyDolphinScheduler currently)
|
|
|
- - Open API, Creating workflows
|
|
|
-
|
|
|
-- Highly Reliable,
|
|
|
- DolphinScheduler uses a decentralized multi-master and multi-worker architecture, which naturally supports horizontal scaling and high availability
|
|
|
+- Easy to deploy, provide four ways to deploy which including Standalone, Cluster, Docker and Kubernetes.
|
|
|
+- Easy to use, workflow can be created and managed by four ways, which including Web UI, [Python SDK](https://dolphinscheduler.apache.org/python/main/index.html), Yaml file and Open API
|
|
|
+- Highly reliable and high availability, decentralized architecture with multi-master and multi-worker, native supports horizontal scaling.
|
|
|
- High performance, its performance is N times faster than other orchestration platform and it can support tens of millions of tasks per day
|
|
|
-- Supports multi-tenancy
|
|
|
-- Supports various task types: Shell, MR, Spark, SQL (MySQL, OceanBase, PostgreSQL, Hive, Spark SQL), Python, Procedure, Sub_Workflow,
|
|
|
- Http, K8s, Jupyter, MLflow, SageMaker, DVC, Pytorch, Amazon EMR, etc
|
|
|
-- Orchestrating workflows and dependencies, you can pause/stop/recover task any time, failed tasks can be set to automatically retry
|
|
|
-- Visualizing the running state of the task in real-time and seeing the task runtime log
|
|
|
-- What you see is what you get when you edit the task on the UI
|
|
|
-- Backfill can be operated on the UI directly
|
|
|
-- Perfect project, resource, data source-level permission control
|
|
|
-- Displaying workflow history in tree/Gantt chart, as well as statistical analysis on the task status & process status in each workflow
|
|
|
-- Supports internationalization
|
|
|
-- Cloud Native, DolphinScheduler supports orchestrating multi-cloud/data center workflow, and
|
|
|
- supports custom task type
|
|
|
-- More features waiting for partners to explore
|
|
|
-
|
|
|
-## User Interface Screenshots
|
|
|
-
|
|
|
-
|
|
|
-<img width="1100" src="https://user-images.githubusercontent.com/15833811/197348110-1653ea32-ce07-436c-a0b8-6ac1af80aea5.png">
|
|
|
-
|
|
|
-
|
|
|
-
|
|
|
-
|
|
|
+- Cloud Native, DolphinScheduler supports orchestrating multi-cloud/data center workflow, and supports custom task type
|
|
|
+- Versioning both workflow and workflow instance(including tasks)
|
|
|
+- Various state control of workflow and task, support pause/stop/recover them in any time
|
|
|
+- Multi-tenancy support
|
|
|
+- Others like backfill support(Web UI native), permission control including project, resource and data source
|
|
|
|
|
|
-## QuickStart in Docker
|
|
|
+## QuickStart
|
|
|
|
|
|
-Please refer the official website document: [QuickStart in Docker](https://dolphinscheduler.apache.org/en-us/docs/3.1.2/guide/start/docker)
|
|
|
+- For quick experience
|
|
|
+ - Want to [start with standalone](https://dolphinscheduler.apache.org/en-us/docs/3.1.5/guide/installation/standalone)
|
|
|
+ - Want to [start with Docker](https://dolphinscheduler.apache.org/en-us/docs/3.1.5/guide/start/docker)
|
|
|
+- For Kubernetes
|
|
|
+ - [Start with Kubernetes](https://dolphinscheduler.apache.org/en-us/docs/3.1.5/guide/installation/kubernetes)
|
|
|
|
|
|
-## QuickStart in Kubernetes
|
|
|
+## User Interface Screenshots
|
|
|
|
|
|
-Please refer to the official website document: [QuickStart in Kubernetes](https://dolphinscheduler.apache.org/en-us/docs/3.1.2/guide/installation/kubernetes)
|
|
|
+* **Homepage:** Project and workflow overview, including the latest workflow instance and task instance status statistics.
|
|
|
+
|
|
|
|
|
|
-## How to Build
|
|
|
+* **Workflow Definition:** Create and manage workflow by drag and drop, easy to build and maintain complex workflow, support [bulk of tasks](https://dolphinscheduler.apache.org/en-us/docs/3.1.5/introduction-to-functions_menu/task_menu) out of box.
|
|
|
+
|
|
|
|
|
|
-```bash
|
|
|
-./mvnw clean install -Prelease
|
|
|
-```
|
|
|
+* **Workflow Tree View:** Abstract tree structure could clearer understanding of the relationship between tasks
|
|
|
+
|
|
|
|
|
|
-### Build with different Zookeeper versions
|
|
|
+* **Data source:** Manage support multiple external data sources, provide unified data access capabilities for such as MySQL, PostgreSQL, Hive, Trino, etc.
|
|
|
+
|
|
|
|
|
|
-The default Zookeeper Server version supported is 3.8.0.
|
|
|
-```bash
|
|
|
-# Default Zookeeper 3.8.0
|
|
|
-./mvnw clean install -Prelease
|
|
|
-# Support to Zookeeper 3.4.6+
|
|
|
-./mvnw clean install -Prelease -Dzk-3.4
|
|
|
-```
|
|
|
+* **Monitor:** View the status of the master, worker and database in real time, including server resource usage and load, do quick health check without logging in to the server.
|
|
|
+
|
|
|
|
|
|
-Artifact:
|
|
|
+## Suggestions & Bug Reports
|
|
|
|
|
|
-```
|
|
|
-dolphinscheduler-dist/target/apache-dolphinscheduler-${latest.release.version}-bin.tar.gz: Binary package of DolphinScheduler
|
|
|
-dolphinscheduler-dist/target/apache-dolphinscheduler-${latest.release.version}-src.tar.gz: Source code package of DolphinScheduler
|
|
|
-```
|
|
|
+Follow [this guide](https://github.com/apache/dolphinscheduler/issues/new/choose) to report your suggestions or bugs.
|
|
|
|
|
|
-## Get Help
|
|
|
+## Contributing
|
|
|
|
|
|
-1. Submit an [issue](https://github.com/apache/dolphinscheduler/issues/new/choose)
|
|
|
-2. [Join our slack](https://s.apache.org/dolphinscheduler-slack) and send your question to channel `#general`
|
|
|
-3. Send email to users@dolphinscheduler.apache.org or dev@dolphinscheduler.apache.org
|
|
|
+The community welcomes everyone to contribute, please refer to this page to find out more: [How to contribute](docs/docs/en/contribute/join/contribute.md),
|
|
|
+find the good first issue in [here](https://github.com/apache/dolphinscheduler/contribute) if you are new to DolphinScheduler.
|
|
|
|
|
|
## Community
|
|
|
|
|
|
-You are very welcome to communicate with the developers and users of Dolphin Scheduler. There are two ways to find them:
|
|
|
+Welcome to join the Apache DolphinScheduler community by:
|
|
|
|
|
|
-1. Join the Slack channel [Slack](https://asf-dolphinscheduler.slack.com/)
|
|
|
-2. Follow the [Twitter account of DolphinScheduler](https://twitter.com/dolphinschedule) and get the latest news on time
|
|
|
-
|
|
|
-## How to Contribute
|
|
|
-
|
|
|
-The community welcomes everyone to contribute, please refer to this page to find out more: [How to contribute](docs/docs/en/contribute/join/contribute.md).
|
|
|
-
|
|
|
-## Thanks
|
|
|
-
|
|
|
-DolphinScheduler is based on a lot of excellent open-source projects, such as Google guava, grpc, netty, quartz, and many open-source projects of Apache and so on.
|
|
|
-We would like to express our deep gratitude to all the open-source projects used in DolphinScheduler. We hope that we are not only the beneficiaries of open-source, but also give back to the community. Besides, we hope everyone who have the same enthusiasm and passion for open source could join in and contribute to the open-source community
|
|
|
+- Join the [DolphinScheduler Slack](https://s.apache.org/dolphinscheduler-slack) to keep in touch with the community
|
|
|
+- Follow the [DolphinScheduler Twitter](https://twitter.com/dolphinschedule) and get the latest news
|
|
|
+- Subscribe DolphinScheduler mail list, users@dolphinscheduler.apache.org for user and dev@dolphinscheduler.apache.org for developer
|
|
|
|
|
|
# Landscapes
|
|
|
|
|
|
@@ -111,6 +76,3 @@ DolphinScheduler enriches the <a href="https://landscape.cncf.io/?landscape=obse
|
|
|
|
|
|
</p >
|
|
|
|
|
|
-## License
|
|
|
-
|
|
|
-Please refer to the [LICENSE](https://github.com/apache/dolphinscheduler/blob/dev/LICENSE) file
|
Jay Chung
