-
Notifications
You must be signed in to change notification settings - Fork 4.8k
[Feature-13052][Task Plugin] Support Apache Seatunnel Connector-V2 #13086
New issue
Have a question about this project? No Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “No Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? No Sign in to your account
Conversation
Codecov Report
@@ Coverage Diff @@
## dev #13086 +/- ##
============================================
+ Coverage 39.20% 39.22% +0.01%
- Complexity 4273 4274 +1
============================================
Files 1069 1069
Lines 40311 40311
Branches 4634 4634
============================================
+ Hits 15804 15811 +7
+ Misses 22729 22722 -7
Partials 1778 1778
📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
SonarCloud Quality Gate failed. |
Hi @zhuangchong @CalvinKirs Do you have time to review this pr? |
Could you provide a test? or your local running screenshot? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have view and test the code. Find a bug need to fix, when save seatunnel process definition.
Class SeatunnelParameters
need add follow code for json parse.
@JsonTypeInfo(use = JsonTypeInfo.Id.NAME, visible = true, property = "engine")
@JsonSubTypes({
@JsonSubTypes.Type(value = SeatunnelFlinkParameters.class, name = "FLINK"),
@JsonSubTypes.Type(value = SeatunnelFlinkParameters.class, name = "FLINK_V2"),
@JsonSubTypes.Type(value = SeatunnelSparkParameters.class, name = "SPARK"),
@JsonSubTypes.Type(value = SeatunnelSparkParameters.class, name = "SPARK_V2"),
@JsonSubTypes.Type(value = SeatunnelEngineParameters.class, name = "SEATUNNEL_ENGINE")
})
The screenshot that i run in my machine. It can work for seatunnel engine to run ETL job. @CalvinKirs |
@caishunfeng @CalvinKirs Hi, This PR hasn't been merged for a long time, is there any consideration? I would be happy to help with anything. |
@shangeyao This is a great PR, very useful. Can you continue to complete it? |
@shangeyao 你好,我在使用的时候,我用的seatunnel、flink版本不支持run-mode参数,可否在创建任务页面及构造job 配置的任务增加一个run-mode枚举none ,不拼加run-mode参数,这样使用范围更广一些,感谢 |
@shangeyao @caishunfeng @CalvinKirs @chaplinthink Hi ,生成seatunnel 脚本的时候,是否可以加上任务名称,现在提交到fink的job名字都是默认的seatunnel 看着很难受,seatunnel可以接受--name参数,可以把工作流的名称拼接上 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1
LGTM
@chaplinthink, considering this PR hasn't been merged for a long time, I merged it and you can submit a new PR to fix it, thx |
Purpose of the pull request
close #13052
Brief change log
support flink-v2, spark-v2, seatunnel-engine
Verify this pull request
This pull request is code cleanup without any test coverage.
(or)
This pull request is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(or)
If your pull request contain incompatible change, you should also add it to
docs/docs/en/guide/upgrede/incompatible.md