Data Foundations
Independently obtain, inspect, and organize data.
- Excel or Sheets
- SQL queries
- Python and Pandas
Learning direction
Data Analytics
Start with problem framing, data cleaning, and statistical judgment to produce explainable, reproducible analysis.
Skill map
From foundations to real delivery
Analytical Thinking
Turn a domain question into a testable analysis question.
- Descriptive statistics
- Sampling, bias, and correlation
- Metric definition
Communication and Validation
Make charts and conclusions withstand scrutiny.
- Data visualization
- Reproducible notebooks
- Boundaries and counterexamples
Real Question
Complete an analysis useful to a campus group or public-data audience.
- Stakeholder interviews
- Analysis report
- Publishing and retrospective
Put the skills to work
Recommended learning and practice items
View all in this direction →Open sourceDataBeginnerReady to start
Sensitive data from extra fields is visible in Edit Connection window
3.0.3 After upgrading Airflow from version 3.0.2 to 3.0.3 sensitive data in extra fields and extra fields JSON is visible in edit connection page. Sensitive data should be masked with Enter Edit Connection window in Airflow UI. Ubuntu 24.04 Other Docker-based deployment Airflow deployed with Docker Swarm. Settings in…
apache/airflowAbout 4 hr推理部署数据工程
Open sourceDataBeginnerReady to start
Support for podman in breeze tooling
Enable breeze to utilize podman for container management. Given the open source nature, lack of licensing issues, OCI-compliant images, and improved security and performance, enable podman as a container management option for breeze tooling.
apache/airflowAbout 4 hr性能优化云与基础设施
Open sourceDataBeginnerReady to start
control state of individual taskflow in mapped task-group
Originally posted by ntnhaatj August 20, 2023 Hi, as my issue was raised here which also included depth-first execution model context, I would like to open an issue to have more discussion on this matter. motivated from the depth-first execution model implemented in mapped TaskGroup: At present, the upstream / downstr…
apache/airflowAbout 4 hr
Open sourceDataBeginnerReady to start
Allow passing config_path as a parameter to KubernetesExecutor
Currently, when using executor_config with KubernetesExecutor, the config_file path must be hardcoded. If we try to pass it dynamically using a Jinja template ("{{ params.config_path }}"), it results in an error. Example of a working static configuration: However, if we attempt to pass it dynamically, it fails: This w…
apache/airflowAbout 4 hr云与基础设施
Open sourceDataBeginnerReady to start
Can't select the desired dag version for a backfill dag run
3.0.0 I am making a bunch of edits to a dag, and I want to re-run a backfill for a specified date range. When I do, Airflow performs the backfill with the version of the dag that originally ran, and there's seemingly no way in the UI to specify that I want to re-run using the latest dag version. The latest version sho…
apache/airflowAbout 4 hr数据工程
Open sourceDataBeginnerReady to start
Generate database ERD using chartdb
https://github.com/chartdb/chartdb is very popular project and generate nice ERD schemas (~though not sure if we can use it due to AGPL-3.0 license ?~) We should update our generation script to use it Note: This library is to be used only for development. We are not distributing it to users thus we are OK with having…
apache/airflowAbout 4 hr数据库
Open sourceDataBeginnerReady to start
DAG Import Errors message is dangling in web-interface
Other Airflow 2 version (please specify below) 2.10.2 We have this message in the Airflow's web-interface no matter what. This DAG can exist or not - the error still shows. We have found many different topics across the Internet for various Airflow versions. Some of them from 2018. So users constantly witness these da…
apache/airflowAbout 4 hr数据工程
Open sourceDataBeginnerReady to start
AIP-82: Add common messaging system providers
related: #49938 Several users have pointed out that the Event-Driven Scheduling feature introduced in Airflow 3 currently supports only Amazon SQS. So I add Kafka common message queue provider in #49938, but there are still quite a few common providers that should be useful for airflow users to implement Event-Driven…
apache/airflowAbout 4 hr数据工程云与基础设施