The Conductor Script: pic.flow

Table of Contents

• 1. General Usage
• 3. sweep: Parameter Study via Slurm Arrays

pic.flow is the workflow orchestrator for PICurv. It validates YAML inputs, generates C runtime artifacts, and runs or schedules solver/postprocessing stages.

1. General Usage

python3 scripts/pic.flow [COMMAND] [ARGS...]

Primary commands:

• init
• build
• run
• sweep
• validate

Help:

python3 scripts/pic.flow --help
python3 scripts/pic.flow run --help
python3 scripts/pic.flow sweep --help
python3 scripts/pic.flow validate --help

5. Run Solver and Postprocessor

python3 scripts/pic.flow run [STAGES] [INPUTS] [OPTIONS]

Stages:

• --solve
• --post-process

Inputs for --solve:

• --case <case.yml>
• --solver <solver.yml>
• --monitor <monitor.yml>

Inputs for --post-process:

• --post <post.yml>
• either same invocation with --solve, or --run-dir <existing_run_dir>

MPI/local options:

• -n, --num-procs

Cluster/Slurm options:

• --cluster <cluster.yml>
• --scheduler slurm (optional explicit selector)
• --no-submit (generate scripts/manifests only)

Preflight options:

• --dry-run (resolve and print launch/artifact plan only)
• --format json (machine-readable output for --dry-run)

Local example:

python3 scripts/pic.flow run --solve --post-process -n 8 \
  --case my_case/case.yml \
  --solver my_case/solver.yml \
  --monitor my_case/monitor.yml \
  --post my_case/post.yml

Slurm example (generate + submit):

python3 scripts/pic.flow run --solve --post-process \
  --case my_case/case.yml \
  --solver my_case/solver.yml \
  --monitor my_case/monitor.yml \
  --post my_case/post.yml \
  --cluster my_case/cluster.yml

Slurm example (generate only):

python3 scripts/pic.flow run --solve --post-process \
  --case my_case/case.yml \
  --solver my_case/solver.yml \
  --monitor my_case/monitor.yml \
  --post my_case/post.yml \
  --cluster my_case/cluster.yml \
  --no-submit

Dry-run example (no file writes):

python3 scripts/pic.flow run --solve --post-process \
  --case my_case/case.yml \
  --solver my_case/solver.yml \
  --monitor my_case/monitor.yml \
  --post my_case/post.yml \
  --dry-run --format json

3. sweep: Parameter Study via Slurm Arrays

python3 scripts/pic.flow sweep \
  --study my_study/study.yml \
  --cluster my_study/cluster.yml [--no-submit]

Behavior:

• expands parameter matrix from study.yml
• materializes case directories under studies/<study_id>/cases/
• generates solver_array.sbatch and post_array.sbatch
• submits post array with afterok:<solver_jobid> dependency (unless --no-submit)
• aggregates metrics and emits plots in studies/<study_id>/results/

4. Validate Inputs Before Launch

python3 scripts/pic.flow validate \
  --case my_case/case.yml \
  --solver my_case/solver.yml \
  --monitor my_case/monitor.yml \
  --post my_case/post.yml

validate does not launch solver/post and does not create run/study artifacts.

6. Inspect Generated Artifacts

Single run (run):

• runs/<run_id>/config/*.control, bcs*.run, whitelist.run, profile.run, post.run
• runs/<run_id>/scheduler/solver.sbatch, post.sbatch (cluster mode)
• runs/<run_id>/scheduler/submission.json (cluster mode)
• runs/<run_id>/manifest.json

Sweep (sweep):

• studies/<study_id>/cases/<case_i>/...
• studies/<study_id>/scheduler/case_index.tsv
• studies/<study_id>/scheduler/solver_array.sbatch
• studies/<study_id>/scheduler/post_array.sbatch
• studies/<study_id>/scheduler/submission.json
• studies/<study_id>/results/metrics_table.csv
• studies/<study_id>/results/plots/*.png (if plotting enabled and matplotlib available)
• studies/<study_id>/study_manifest.json

9. Next Steps

• Config contract: Configuration Contract (YAML -> Generated Artifacts -> Runtime)
• User workflows: User How-To Guides
• Extensibility: Workflow Extensibility Guide
• First-run path: Tutorial: Your First Simulation (Flat Channel)
• Troubleshooting map: Common Fatal Errors and Fixes
• Smoke/CI verification details: Testing, Smoke, and Quality Gates Guide