Workflow Extensibility Guide

Table of Contents

• 1. Design Goal
• 2. Grid Workflow Extensions
• 3. Multi-Run Orchestration (Sweeps/Studies)
• 4. Runtime Completion Extensions
• 5. Data-Driven Particle Closure Integration
  • 5.1 Offline (Recommended First)
  • 5.2 Tightly Coupled Inference (Runtime)
• 6. Guardrails for Safe Growth
• 7. Related Docs
  • CFD Reader Guidance and Practical Use
    • What To Extract Before Changing A Case
    • Practical CFD Troubleshooting Pattern

This page captures practical extension directions that are already compatible with the current architecture.

1. Design Goal

Keep the pipeline easy to extend while preserving a strict, debuggable contract:

YAML -> generated artifacts -> C ingestion -> runtime consumers

This contract is already explicit in pages 14/15/16 and the ingress audit tooling.

2. Grid Workflow Extensions

Supported today:

• External grid ingestion (grid.mode: file) with validation and non-dimensionalization in picurv
• Pre-run generation (grid.mode: grid_gen) via generators/grid.gen

Extension-friendly next steps:

• Add additional generator wrappers under grid.generator (new grid_type aliases)
• Add stronger pre-run grid quality checks in Python before solver launch
• Add optional strict checks that block run if metrics exceed thresholds

3. Multi-Run Orchestration (Sweeps/Studies)

Implemented in current picurv:

• run --cluster <cluster.yml> for Slurm-backed single-run scheduling
• sweep --study <study.yml> --cluster <cluster.yml> for matrix expansion + array submission
• post-stage dependency chaining (afterok)
• run/study manifests and submission metadata for reproducibility
• metrics aggregation and basic plot generation from existing outputs

Extension-ready next steps:

1. add more metric extractors (custom CSV schemas, multi-file reducers),
2. add sweep resume/retry policies,
3. add scheduler backends beyond Slurm without changing C contracts.

4. Runtime Completion Extensions

The current runtime already supports safe early shutdown for walltime and signal events. Physics-based completion is a separate future extension and should keep that same checkpoint discipline.

Extension-ready next steps:

1. add optional convergence-stop criteria under solver.yml -> solution_convergence,
2. limit deterministic fixed-state stopping to mode: steady_deterministic,
3. use explicit tolerances, minimum samples, and dwell windows before declaring completion,
4. keep mode: transient diagnostic-only because nonzero drift can be the expected physics,
5. route any automatic completion through the existing final-write/safe-checkpoint path.

For statistically steady turbulence, prefer postprocessing-heavy turbulence analysis after runtime window metrics indicate stationarity; runtime completion criteria should remain lightweight and conservative.

5. Data-Driven Particle Closure Integration

5.1 Offline (Recommended First)

• Use solver/post outputs as training/inference datasets.
• Keep ML scripts external (Python pipeline).
• Reinject inferred coefficients/flags through YAML for later runs.

This requires minimal C changes and is ideal for rapid iteration.

5.2 Tightly Coupled Inference (Runtime)

For in-solver inference:

1. add runtime-selectable closure model interface in ParticlePhysics.c,
2. expose model type and parameters in schema,
3. map/validate in picurv,
4. ingest in setup.c,
5. wire in particle update path.

Keep deterministic fallback as default for robustness.

6. Guardrails for Safe Growth

• Keep new user options structured first; reserve passthrough for temporary gaps.
• Keep ingestion mostly in setup.c/io.c.
• Update mapping docs and ingress manifest together with code changes.
• Require at least one template/example update per new user-visible feature.

7. Related Docs

• User contract: Configuration Contract (YAML -> Generated Artifacts -> Runtime)
• Ingestion map: Developer Ingestion Map
• Extension playbook: Configuration Extension Playbook

CFD Reader Guidance and Practical Use

This page describes Workflow Extensibility Guide within the PICurv workflow. For CFD users, the most reliable reading strategy is to map the page content to a concrete run decision: what is configured, what runtime stage it influences, and which diagnostics should confirm expected behavior.

Treat this page as both a conceptual reference and a runbook. If you are debugging, pair the method/procedure described here with monitor output, generated runtime artifacts under runs/<run_id>/config, and the associated solver/post logs so numerical intent and implementation behavior stay aligned.

What To Extract Before Changing A Case

• Identify which YAML role or runtime stage this page governs.
• List the primary control knobs (tolerances, cadence, paths, selectors, or mode flags).
• Record expected success indicators (convergence trend, artifact presence, or stable derived metrics).
• Record failure signals that require rollback or parameter isolation.

Practical CFD Troubleshooting Pattern

1. Reproduce the issue on a tiny case or narrow timestep window.
2. Change one control at a time and keep all other roles/configs fixed.
3. Validate generated artifacts and logs after each change before scaling up.
4. If behavior remains inconsistent, compare against a known-good baseline example and re-check grid/BC consistency.