Browse Source

[Feature-8591][Python] Add pre-commit to run basic test before commit (#8843)

* add `.pre-commit-config.yaml` in the pydolphinscheduler module and configured basic check rules like `isort`, `black`, `flake8`
* add some instructions in `DEVELOP.MD`
* Fix some mistake and add some docs on it

Co-authored-by: Jiajie Zhong <zhongjiajie955@hotmail.com>
天仇 3 years ago
parent
commit
a308fa3bef

+ 2 - 2
dolphinscheduler-python/pydolphinscheduler/.flake8

@@ -36,5 +36,5 @@ ignore =
     # Conflict to Black
     W503   # W503: Line breaks before binary operators
 per-file-ignores =
-    src/pydolphinscheduler/side/__init__.py:F401
-    src/pydolphinscheduler/tasks/__init__.py:F401
+    */pydolphinscheduler/side/__init__.py:F401
+    */pydolphinscheduler/tasks/__init__.py:F401

+ 47 - 0
dolphinscheduler-python/pydolphinscheduler/.pre-commit-config.yaml

@@ -0,0 +1,47 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# See https://pre-commit.com for more information
+# See https://pre-commit.com/hooks.html for more hooks
+
+default_stages: [commit, push]
+default_language_version:
+  # force all python hooks to run python3
+  python: python3
+repos:
+  - repo: https://github.com/pycqa/isort
+    rev: 5.10.1
+    hooks:
+      - id: isort
+        name: isort (python)
+  - repo: https://github.com/psf/black
+    rev: 22.1.0
+    hooks:
+      - id: black
+  - repo: https://github.com/pycqa/flake8
+    rev: 4.0.1
+    hooks:
+      - id: flake8
+        additional_dependencies: [
+            'flake8-docstrings>=1.6',
+            'flake8-black>=0.2',
+        ]
+        # pre-commit run in the root, so we have to point out the full path of configuration
+        args: [
+            --config, 
+            dolphinscheduler-python/pydolphinscheduler/.flake8
+        ]

+ 11 - 0
dolphinscheduler-python/pydolphinscheduler/DEVELOP.md

@@ -115,6 +115,17 @@ maybe you could follow [Black-integration][black-editor] to configure them in yo
 Our Python API CI would automatically run code style checker and unittest when you submit pull request in
 GitHub, you could also run static check locally.
 
+We recommend [pre-commit](https://pre-commit.com/) to do the checker mentioned above before you develop locally. 
+You should install `pre-commit` by running
+
+```shell
+python -m pip install pre-commit 
+```
+
+in your development environment and then run `pre-commit install` to set up the git hooks scripts. After finish
+above steps, each time you run `git commit` or `git push` would run pre-commit check to make basic check before
+you create pull requests in GitHub.
+
 ```shell
 # We recommend you run isort and Black before Flake8, because Black could auto fix some code style issue
 # but Flake8 just hint when code style not match pep8