Open sourceDataAdvanced
Apache Airflow
Workflow orchestration platform for data pipelines, scheduling, and observability.
Apache Software FoundationAbout 2 hrpythonorchestration
Browse by topic
数据工程
10 items
Open sourceAIIntermediate
OpenAI Python SDK
Official OpenAI Python SDK for typed clients, streaming responses, and API compatibility.
OpenAIAbout 2 hrpythonapi
Open sourceDataAdvanced
Apache Spark
Large-scale data engine for distributed computing, SQL execution, and performance optimization.
Apache Software FoundationAbout 2 hrdistributed-systemssql
Open sourceDataBeginnerReady to start
Improve Operators Documentation Page
This is an issue that is reserved for the Airflow Summit "Contributors" Workshop. This is denoted with the label contributors-workshop. Out of respect for the organizers and participants of this workshop, please do not implement a PR that addresses this issue. > > If this issue is still open following Airflow Summit,…
apache/airflowAbout 4 hr数据工程文档
Open sourceDataBeginnerReady to start
Sensitive data from extra fields is visible in Edit Connection window
3.0.3 After upgrading Airflow from version 3.0.2 to 3.0.3 sensitive data in extra fields and extra fields JSON is visible in edit connection page. Sensitive data should be masked with Enter Edit Connection window in Airflow UI. Ubuntu 24.04 Other Docker-based deployment Airflow deployed with Docker Swarm. Settings in…
apache/airflowAbout 4 hr推理部署数据工程
Open sourceDataBeginnerReady to start
Can't select the desired dag version for a backfill dag run
3.0.0 I am making a bunch of edits to a dag, and I want to re-run a backfill for a specified date range. When I do, Airflow performs the backfill with the version of the dag that originally ran, and there's seemingly no way in the UI to specify that I want to re-run using the latest dag version. The latest version sho…
apache/airflowAbout 4 hr数据工程
Open sourceDataBeginnerReady to start
DAG Import Errors message is dangling in web-interface
Other Airflow 2 version (please specify below) 2.10.2 We have this message in the Airflow's web-interface no matter what. This DAG can exist or not - the error still shows. We have found many different topics across the Internet for various Airflow versions. Some of them from 2018. So users constantly witness these da…
apache/airflowAbout 4 hr数据工程
Open sourceDataBeginnerReady to start
AIP-82: Add common messaging system providers
related: #49938 Several users have pointed out that the Event-Driven Scheduling feature introduced in Airflow 3 currently supports only Amazon SQS. So I add Kafka common message queue provider in #49938, but there are still quite a few common providers that should be useful for airflow users to implement Event-Driven…
apache/airflowAbout 4 hr数据工程云与基础设施
Open sourceDataBeginnerReady to start
Add support for Papermill using https
github I want to connect to a remote kernel in JupyterHub. Issue #34840 seems to provide access with transport set to "transport". I would like to make access throught https and token. 2.9 ubuntu 20.04 Official Apache Airflow Helm Chart - create a papermill connection - try to connect with https host and aditional tok…
apache/airflowAbout 4 hr数据工程
Open sourceDataBeginnerReady to start
job table missing `dag_id` and `executor_class` for BackfillJob type
2.9.1 When using the airflow dags backfill command to execute a backfill for any DAG, I noticed that the dag_id and executor_class columns in the job table are empty(null). Results in entries in the job table like this: However, in versions before 2.6.0, the entries in the job table included the dag_id and executor_cl…
apache/airflowAbout 4 hr数据工程