|
PICurv 0.1.0
A Parallel Particle-In-Cell Solver for Curvilinear LES
|
picurv is the workflow orchestrator for PICurv. It validates YAML inputs, generates C runtime artifacts, and runs or schedules solver/postprocessing stages. It is also the primary user-facing contract layer: many defaults, aliases, and translation rules are enforced here before the C solver starts.
After make all, bin/picurv is a launcher for scripts/picurv (the single source of truth). The launcher uses .picurv-venv/bin/python when bootstrap created a managed environment, then a bootstrap-recorded or user-provided PICURV_PYTHON, and finally python3. Source etc/picurv.sh to add bin/ to your PATH and expose scripts/ as a fallback so picurv works from any directory even if bin/picurv is temporarily absent before make conductor recreates the launcher. If bin/picurv does not exist yet, run ./scripts/picurv build or make conductor.
Primary commands:
initbuildsync-binariessync-configpull-sourcestatus-sourcerunprecomputesummarizesubmitcancelsweepvalidateCore idea:
case.yml, solver.yml, monitor.yml, and post.yml are modular profiles.case.yml can be paired with a quieter monitor.yml, a different solver strategy, or a different post recipe without changing the physical setup file.Help:
Behavior:
examples/<template_name>/ into a new working directory,--dest,.picurv-origin.json with the source repo path and template name,.picurv-execution.yml for optional site-specific launcher overrides,.picurv-execution.yml when the source clone already has one, otherwise from inert defaults,bin/ directory via PATH.Binary pinning (--pin-binaries):
--pin-binaries is passed, simulator and postprocessor are copied into the case directory,bin/ originals at runtime (resolve_runtime_executable checks the invocation directory first),picurv itself is never copied — it is always used from PATH and is safe to update mid-run since it only launches the C binaries.Equivalent manual step (after init): picurv sync-binaries --case-dir <case>.
Examples:
Behavior:
Makefile via make,make all when you do not provide an explicit make target,.picurv-origin.json when run from an initialized case,logs/build.log in the source repo,make manually.Direct make all keeps the traditional stdout-only behavior. When you want the same source-repo build plus a warnings-only artifact, use:
This writes:
logs/build.loglogs/build.warnings.logExamples:
These commands are intended for case directories created by init.
sync-binaries copies simulator and postprocessor from the source repo bin/ into a case directory for version-pinning (optional — normally binaries are resolved via PATH):
sync-config refreshes files from examples/<template_name>/ into the case directory. By default it preserves user-modified files and only copies missing files:
--prune is conservative: it removes only files previously recorded as template-managed that no longer exist in the source template. User-created case files are not pruned. sync-config does not copy execution.example.yml into a case; instead it creates .picurv-execution.yml only when the case does not already have one.
pull-source refreshes every local branch with a configured upstream, then restores the branch you started on, so you can update code without leaving the case directory:
status-source inspects source commit drift, copied binary drift, and template-file drift before you decide what to sync:
For older cases that do not yet have .picurv-origin.json, pass --source-root /path/to/PICurv. For sync-config, also pass --template-name <example_name> if the template cannot be inferred.
status-source --format json payload highlights:
source_repo_root, case_dirlast_known_source_git_commit, current_source_git_commit, source_commit_changedbinaries:case_bin_current, case_bin_different, case_bin_missingconfig:case_current_files, case_modified_files, case_missing_filestemplate_removed_since_last_sync (when template-managed tracking is available)Stages:
--solve--post-processInputs for --solve:
--case <case.yml>--solver <solver.yml>--monitor <monitor.yml> (logging, diagnostics, profiling, output cadence, directories)Inputs for --post-process:
--post <post.yml>--solve, or --run-dir <existing_run_dir>MPI/local options:
-n, --num-procs1.PICURV_MPI_LAUNCHER, MPI_LAUNCHER, nearest .picurv-execution.yml, nearest legacy .picurv-local.yml, then default mpiexec.Staging/execution options:
--cluster <cluster.yml>--scheduler slurm (optional explicit selector)--no-submit (stage run artifacts without starting the local or Slurm backend)Preflight options:
--dry-run (resolve and print launch/artifact plan only, including PETSc diagnostic flags and log paths)--format json (machine-readable output for --dry-run)Local example:
In this command, solver runs with 8 ranks; post-processing is still forced to 1 rank.
Slurm example (generate + submit):
Stage artifacts without starting execution:
Add --cluster my_case/cluster.yml to stage Slurm scripts instead of local commands.
Follow-up execution/submission from existing artifacts:
Post-only continuation examples:
post.yml.start_step. Keep the full desired analysis window in post.yml, then let --continue move the launch cursor inside that window:420, PICurv launches only the fully available prefix in the requested stride. A later --continue run picks up the newer steps after the solver produces them.Qcrit or changing the statistics output prefix, PICurv treats that as a new recipe lineage and starts again from the configured start_step.runs/<run_id>, it is refused immediately instead of racing on viz/ or statistics/.Graceful shutdown note:
cluster.yml -> execution.extra_sbatch.signal requests an early warning signal, PICurv also traps SIGUSR1, SIGTERM, and SIGINT, then writes one last snapshot at the next safe checkpoint even when the normal recording interval has not been reached.cluster.yml -> execution.walltime_guard; keep execution.extra_sbatch.signal as fallback protection for preemption/manual termination or jobs that may not reach the warmup window.signal: "USR1@300" for srun-launched jobs, or signal: "B:USR1@300" plus exec mpirun ... for direct mpirun batch launches.Runtime stream logs:
runs/<run_id>/logs/.runs/<run_id>/scheduler/ for both local and Slurm launches.Behavior:
Examples:
Typical sources:
logs/Continuity_Metrics.loglogs/Particle_Metrics.log (per-step Lost plus run-local Lost Total when available)logs/Momentum_Solver_Convergence_History_Block_*.loglogs/Poisson_Solver_Convergence_History_Block_*.loglogs/solution_convergence.log (mode-specific speed/KE drift and L2 norms)logs/Profiling_Timestep_Summary.csv when enabledlogs/Runtime_Memory.log when monitor.diagnostics.runtime_memory_log.enabled is truelogs/PETSc_*_Solver.log / logs/PETSc_*_PostProcessor.log when file-backed PETSc diagnostics are enabledscheduler/*_solver.log or scheduler/solver_*.out for sampled particle snapshot previewsWhen particle snapshots are available, summarize reports sampled diagnostics such as:
Dry-run example (no file writes):
Common run use cases:
--solve --post-process--solve--post-process --run-dir ...--no-submitsubmit --run-dir ...--dry-runprecompute materializes deterministic case artifacts without launching the solver. It uses the same case.yml generator settings as run --solve:
grid.mode: grid_gen writes the configured dimensional PICGRID,prescribed_flow.source.type: generated delegates dimensional .picslice profile generation to scripts/profile.gen and writes profile.info,prescribed_flow.source.type: field_slice extracts a dimensional .picslice from an old Cartesian ufield*.dat plus its old PICGRID,config/precompute.manifest.json records generated paths and profile stats.The output layout mirrors runs/<run_id>/config/ so inspected artifacts can be reused later with grid.mode: file and prescribed_flow.source.type: file.
Behavior:
--no-submit artifacts without regenerating configs or scripts,scheduler/submission.json to locate staged Slurm scripts or local command tokens,solve, post-process, or both,all is selected,launch_mode: local,--force is explicitly provided.Examples:
Notes:
--run-dir supports local and Slurm staged runs,--study-dir remains Slurm-only,--dry-run prints the exact local command or sbatch plan,--force is the opt-in path for deliberate resubmission.Behavior:
scheduler/submission.json from an existing run directory,solve and/or post-process,scancel <job_id> for the selected stage set by default,--graceful, sends scancel --signal=USR1 <job_id> for solver jobs so the runtime can write the latest safe off-cadence step at the next checkpoint,--graceful is present,Examples:
Notes:
scheduler/submission.json,--graceful requests solver shutdown and final output; if a job is wedged or not reaching runtime checkpoints, rerun without --graceful to hard-cancel it,--dry-run is useful when you want to confirm the recorded stage/job mapping first.Behavior (new study):
study.ymlstudies/<study_id>/cases/solver_array.sbatch, post_array.sbatch, and metrics_aggregate.sbatchpost_array.sbatch with a single-task allocation and a forced one-rank launcher commandafterok) → metrics (afterany) chain (unless --no-submit)Behavior (--continue):
resolve_restart_source)Behavior (--reaggregate):
validate does not launch solver/post and does not create run/study artifacts.
What validate is for:
picurv preserves a C-side default intentionally.This section is intentionally exhaustive and mirrors the current argparse contract in scripts/picurv. Use it as the authoritative option reference when writing docs, examples, wrappers, or CI jobs.
run:
--solve (requires --case, --solver, --monitor)--post-process (requires --post; requires --run-dir when --solve is not selected)--case <path>--solver <path>--monitor <path> (logging, diagnostics, profiling, output cadence, directories)--post <path>--run-dir <path>-n, --num-procs <int> (solver stage only; post is forced to 1 rank/task)--cluster <cluster.yml> (enables Slurm mode)--scheduler <name> (must be used with --cluster; must match cluster.yml:scheduler.type)--no-submit (stage files and submission metadata without starting execution)--dry-run (no file writes; plan only, including diagnostic artifacts)--format {text,json} (dry-run output format)validate:
--case <path>--solver <path>--monitor <path>--post <path>--cluster <path>--study <path>--strict (adds additional checks for selected roles; documented below)precompute:
--case <path>--output-dir <path> (defaults to precomputed/<case-name>)submit:
--run-dir <path> or --study-dir <path>--stage {all,solve,post-process}--force--dry-runcancel:
--run-dir <path>--stage {all,solve,post-process}--dry-runsweep:
--study <study.yml> (required)--cluster <cluster.yml> (required)--no-submit (optional)--continue (required)--study-dir <path> (required)--cluster <cluster.yml> (optional; overrides original cluster resources)--reaggregate (required)--study-dir <path> (required)init:
<template_name>--dest <dir>--source-root <repo>--pin-binaries (copy simulator/postprocessor into the case for version-pinning)build:
--source-root <repo>--case-dir <case_dir>MAKE_ARGS... are passed directly to makesync-binaries:
--case-dir <case_dir>--source-root <repo>sync-config:
--case-dir <case_dir>--source-root <repo>--template-name <template>--overwrite--prunepull-source:
--case-dir <case_dir>--source-root <repo>--remote <git_remote>--branch <git_branch>--no-rebasestatus-source:
--case-dir <case_dir>--source-root <repo>--template-name <template>--format {text,json}--strict does not change baseline schema validation, but it adds file-system consistency checks for selected roles:
--post:source_data.directory is not <solver_output_dir>, the resolved directory must exist.--study:study.base_configs are loaded and revalidated as real case/solver/monitor/post bundles.Use --strict in CI/pre-submit checks when validating reusable profile libraries or study manifests.
picurv run --dry-run --format json emits a deterministic plan payload with these top-level keys:
mode ("dry-run")created_at (ISO timestamp)launch_mode (local or slurm)warnings (list)inputs (resolved absolute paths)stages (stage-specific launch plans)artifacts (predicted files/directories, deduplicated)run_id_preview / run_dir_preview (when known)solver_num_procs_effectivepost_num_procs_effectivenum_procs_effective (currently mirrors solver count)For file-backed grid modes, artifacts includes the planned staged grid path (config/grid.run). For grid.mode: grid_gen, it also includes the generated PICGRID path (config/grid.generated.picgrid by default) plus any configured stats_file or vts_file. Dry-run still does not run scripts/grid.gen or write these files. For generated prescribed-flow profiles, artifacts includes the dimensional generated .picslice, the solver-scale staged .picslice, and profile.info.
Stage entries under stages.solve and stages.post-process include:
mode (local or slurm)num_procs_effectivelaunch_command (tokenized command list)launch_command_string (shell-ready display string)Additional stage fields:
script (Slurm script path in cluster mode)command / command_string (local staged command tokens/display string)source_data_directory (post stage source directory resolution)restart_source_directory (solve stage, when restart source is resolved)Dry-run guarantees:
Validation and CLI usage errors are emitted as one-line, machine-parseable records:
Current normalized error code set:
CLI_USAGE_INVALIDCFG_MISSING_SECTIONCFG_MISSING_KEYCFG_INVALID_TYPECFG_INVALID_VALUECFG_FILE_NOT_FOUNDCFG_GRID_PARSECFG_INCONSISTENT_COMBOThis contract is exercised by Python tests and should remain stable for wrappers and CI parsers.
PICurv is intended to be used with reusable profile libraries.
Typical pattern:
case.yml focused on physics, grid, BCs, and run duration,solver.yml focused on numerical strategy,monitor.yml focused on logging, diagnostics, profiling, and I/O cadence,post.yml focused on analysis outputs,Examples:
case.yml + a lighter monitor.yml for fast debug runs,case.yml + a stricter solver.yml for convergence checks,case.yml + multiple post.yml recipes for different analysis outputs,solver.yml reused across many cases when the discretization strategy is stable.This is why picurv treats these roles as separate inputs instead of one monolithic file.
For prebuilt reusable profiles, also see the local guides under:
config/solvers/config/monitors/config/postprocessors/config/schedulers/examples/master_template/picurv resolves simulator and postprocessor at launch time using this precedence:
picurv script (e.g. case-local copies from --pin-binaries or sync-binaries), it is used first.bin/ directory — the default location after make all.bin/picurv is a launcher for scripts/picurv. This means:
scripts/picurv), so code changes never drift,make conductor recreates the launcher (idempotent),.picurv-venv Python,etc/picurv.sh adds bin/ to PATH so picurv works from any directory.Rebuilding while jobs are running:
picurv (the Python script) mid-run is always safe — it is only used to launch jobs, not during solver execution.simulator/postprocessor (make all) overwrites the binaries in bin/. If a Slurm job references bin/simulator by absolute path and has not yet started, the running binary may be replaced before execution begins.--pin-binaries at init time or sync-binaries before submission. Case-local copies are isolated from repo rebuilds.Recommended workflow for concurrent development and production:
Single run (run):
runs/<run_id>/config/*.control, bcs*.run, post.run, plus optional whitelist.run / profile.run sidecars when enabledruns/<run_id>/logs/* (runtime logs and metrics written by solver/postprocessor)runs/<run_id>/scheduler/solver.sbatch, post.sbatch (cluster mode)runs/<run_id>/scheduler/solver_<jobid>.out/.err, post_<jobid>.out/.err (cluster mode, after submission)runs/<run_id>/scheduler/submission.json (cluster mode)runs/<run_id>/manifest.jsonSweep (sweep):
studies/<study_id>/cases/<case_i>/...studies/<study_id>/scheduler/case_index.tsvstudies/<study_id>/scheduler/solver_array.sbatchstudies/<study_id>/scheduler/post_array.sbatchstudies/<study_id>/scheduler/solver_<array_jobid>_<taskid>.out/.err, post_<array_jobid>_<taskid>.out/.errstudies/<study_id>/scheduler/submission.jsonstudies/<study_id>/results/metrics_table.csvstudies/<study_id>/results/plots/*.png (if plotting enabled and matplotlib available)studies/<study_id>/study_manifest.jsonThis page describes The Conductor Script: picurv within the PICurv workflow. For CFD users, the most reliable reading strategy is to map the page content to a concrete run decision: what is configured, what runtime stage it influences, and which diagnostics should confirm expected behavior.
Treat this page as both a conceptual reference and a runbook. If you are debugging, pair the method/procedure described here with monitor output, generated runtime artifacts under runs/<run_id>/config, and the associated solver/post logs so numerical intent and implementation behavior stay aligned.