Integrations — Orchestration
NerveStax and Airflow
Agents read your DAGs and runs, diagnose failures, and propose changes as pull requests.
What we do
What NerveStax does with Airflow
NerveStax connects to the Airflow you already run. Agents read DAGs, runs, task instances and logs, work out why a run failed, and write DAG changes into your repository as a pull request. A change can be tested by a verification DAG in your own Airflow before you review it. Triggering, clearing, pausing and backfilling all wait for a person.
-
Reads DAGs, runs and task logs
DAG source and structure, run and task history, durations, import errors and cluster health. Task logs reach the agent with password, token and key patterns redacted.
-
Diagnoses a failed run
An agent walks the failed task, its tries and its logs, compares the run with earlier ones, and names the likely cause with the evidence it used, rather than a retry count.
-
Proposes DAG changes as pull requests
New or edited DAGs are validated, then arrive as a pull request in your repository. Nothing is written into your Airflow deployment behind your back.
-
Tests a change in your own Airflow
An installed harness can run a verification DAG against a labelled pull request in your own Airflow, on an isolated plane, so you read the change with a real result attached.
-
Schedules by tag
A model carries one tag naming its cadence, and each cadence renders one DAG that builds exactly those models. Agents read and lint that vocabulary; people decide what a cadence is and when it runs.
-
Triggering and backfills wait for you
Triggering a DAG run, clearing tasks, pausing or unpausing and running a backfill each stop and ask a person first. An agent can prepare one; it cannot start one.
Background
About Airflow
Apache Airflow is the open-source scheduler most data teams use to run their pipelines: DAGs written in Python, executed on a schedule, with a UI and API over runs, tasks and logs.
How it connects
What connecting it looks like
-
01
Add your Airflow
Give the workspace the API URL and credentials for the Airflow you already run. They are stored encrypted and never shown back.
-
02
Point it at the DAG repository
Tell the project which repository holds the DAGs, so proposed changes arrive as pull requests in the right place.
-
03
Install the test harness
The harness that runs verification DAGs for labelled pull requests is itself installed through a pull request you review, using a revocable machine token.
-
04
Point failures at on-call
Send Airflow failure callbacks to the alert webhook and every one is triaged read-only before anyone is escalated.
Scope
What this does not cover
- Airflow only. Dagster and Prefect are not supported.
- Trigger, clear, pause, unpause and backfill are approval-gated, and an agent never edits your deployment directly — changes land as pull requests.
- One production Airflow per workspace. The pull-request test harness runs on a separate, isolated plane.
Questions
Airflow and NerveStax
Q01Does the agent change my Airflow directly?+
No. DAG changes are written into your repository and arrive as a pull request, which agents cannot merge. The actions that touch a running deployment — trigger, clear, pause, unpause, backfill — stop and wait for a person to approve each time. Everything else the agent does with Airflow is reading.
Q02Which Airflow can it connect to?+
The one you already run, wherever it runs, as long as NerveStax can reach its API. Agents read DAGs, runs, task instances, logs, import errors and cluster health through that API. There is no managed Airflow to move to and no scheduler of ours to adopt.
Q03Can it test a DAG change before I review it?+
Yes, with the harness installed. When a pull request is labelled, the harness runs a verification DAG in your own Airflow on an isolated plane and reports what happened back into the conversation and onto the pull request, so the change comes with a result rather than an assurance.
Q04Can it run a backfill?+
It can plan one and it cannot start one. A backfill is a gated action: the agent shows what it intends to run and waits for a person to allow or deny. The same applies to triggering a run or clearing task instances.
Q05What does it do with my task logs?+
It reads the tail of a failed task's log to diagnose it. Password, token and key patterns are redacted before the model sees anything, and that redacted text is sent to your own model provider under your own key. Nothing is stored with a third-party analytics or error-tracking service.