PICurv 0.1.0
A Parallel Particle-In-Cell Solver for Curvilinear LES
 
Loading...
Searching...
No Matches
Functions | Variables
extract_artifact_topology Namespace Reference

Functions

list normalize (list paths, Path workspace, str run_id)
 Replace unstable path components with logical tokens.
 
dict run_plan (Path workspace, Path case_dir, str monitor, list extra, str post="post.yml")
 Invoke the dry-run planner for one scenario and return its raw plan.
 
Path init_case (Path workspace)
 Initialize a shipped case inside a scenario workspace.
 
str flat_post_recipe (Path case_dir)
 Write a post recipe requesting a non-canonical output directory.
 
dict map_to_identities (list artifacts, dict contract)
 Map each normalized artifact token onto a declared logical identity.
 
dict build_snapshot ()
 Build the normalized topology snapshot across scenarios.
 
int main ()
 Write or verify the artifact topology snapshot.
 

Variables

 REPO_ROOT = Path(__file__).resolve().parents[2]
 
str CONTRACT_PATH = REPO_ROOT / "tests" / "tooling" / "artifact_topology.json"
 
str SNAPSHOT_PATH = REPO_ROOT / "docs" / "generated" / "artifact_topology_snapshot.json"
 
str PICURV = REPO_ROOT / "picurv_cli" / "picurv"
 

Detailed Description

Extract a normalized run-artifact topology snapshot from the CLI's own planner.

Function Documentation

◆ normalize()

list extract_artifact_topology.normalize ( list  paths,
Path  workspace,
str  run_id 
)

Replace unstable path components with logical tokens.

The workspace root, the generated run id, and any embedded timestamp differ on every invocation. Fingerprinting the raw plan would therefore report drift on every run; fingerprinting the normalized shape reports drift only when the topology actually changes.

Parameters
[in]pathsPlanned artifact paths.
[in]workspaceTemporary workspace root used for the plan.
[in]run_idGenerated run identifier.
Returns
Sorted, normalized artifact tokens.

Definition at line 23 of file extract_artifact_topology.py.

23def normalize(paths: list, workspace: Path, run_id: str) -> list:
24 """!
25 @brief Replace unstable path components with logical tokens.
26
27 The workspace root, the generated run id, and any embedded timestamp differ on
28 every invocation. Fingerprinting the raw plan would therefore report drift on
29 every run; fingerprinting the normalized shape reports drift only when the
30 topology actually changes.
31 @param[in] paths Planned artifact paths.
32 @param[in] workspace Temporary workspace root used for the plan.
33 @param[in] run_id Generated run identifier.
34 @return Sorted, normalized artifact tokens.
35 """
36 normalized = set()
37 for raw in paths:
38 text = str(raw)
39 text = text.replace(str(workspace), "<workspace>")
40 text = text.replace(run_id, "<run_id>")
41 text = re.sub(r"\d{8}-\d{6}", "<timestamp>", text)
42 normalized.add(text)
43 return sorted(normalized)
44
45
Here is the caller graph for this function:

◆ run_plan()

dict extract_artifact_topology.run_plan ( Path  workspace,
Path  case_dir,
str  monitor,
list  extra,
str   post = "post.yml" 
)

Invoke the dry-run planner for one scenario and return its raw plan.

Parameters
[in]workspaceScenario workspace.
[in]case_dirDirectory holding the initialized case.
[in]monitorMonitor file name inside the case directory.
[in]extraAdditional CLI arguments for the scenario.
[in]postPost recipe file name inside the case directory.
Returns
Parsed plan mapping.
Exceptions
RuntimeErrorwhen the planner fails.

Definition at line 46 of file extract_artifact_topology.py.

47 post: str = "post.yml") -> dict:
48 """!
49 @brief Invoke the dry-run planner for one scenario and return its raw plan.
50 @param[in] workspace Scenario workspace.
51 @param[in] case_dir Directory holding the initialized case.
52 @param[in] monitor Monitor file name inside the case directory.
53 @param[in] extra Additional CLI arguments for the scenario.
54 @param[in] post Post recipe file name inside the case directory.
55 @return Parsed plan mapping.
56 @throws RuntimeError when the planner fails.
57 """
58 result = subprocess.run(
59 [
60 sys.executable, str(PICURV), "run", "--dry-run", "--format", "json",
61 "--case", str(case_dir / "config" / "case.yml"),
62 "--solver", str(case_dir / "config" / "solver.yml"),
63 "--monitor", str(case_dir / "config" / monitor),
64 "--post", str(case_dir / "config" / post),
65 *extra,
66 ],
67 cwd=workspace, capture_output=True, text=True, check=False, timeout=180,
68 )
69 if result.returncode != 0:
70 raise RuntimeError(f"dry-run failed:\n{result.stdout}\n{result.stderr}")
71 return json.loads(result.stdout)
72
73
Here is the caller graph for this function:

◆ init_case()

Path extract_artifact_topology.init_case ( Path  workspace)

Initialize a shipped case inside a scenario workspace.

Parameters
[in]workspaceScenario workspace.
Returns
Path to the created case directory.
Exceptions
RuntimeErrorwhen initialization fails.

Definition at line 74 of file extract_artifact_topology.py.

74def init_case(workspace: Path) -> Path:
75 """!
76 @brief Initialize a shipped case inside a scenario workspace.
77 @param[in] workspace Scenario workspace.
78 @return Path to the created case directory.
79 @throws RuntimeError when initialization fails.
80 """
81 case_dir = workspace / "case"
82 result = subprocess.run(
83 [sys.executable, str(PICURV), "init", "flat_channel", "--dest", str(case_dir)],
84 cwd=workspace, capture_output=True, text=True, check=False, timeout=120,
85 )
86 if result.returncode != 0:
87 raise RuntimeError(f"picurv init failed:\n{result.stdout}\n{result.stderr}")
88 return case_dir
89
90
Here is the caller graph for this function:

◆ flat_post_recipe()

str extract_artifact_topology.flat_post_recipe ( Path  case_dir)

Write a post recipe requesting a non-canonical output directory.

Runtime path injection must replace this request with the stable output/visualization/<recipe_id> home.

Parameters
[in]case_dirCase directory to write into.
Returns
File name of the variant recipe.

Definition at line 91 of file extract_artifact_topology.py.

91def flat_post_recipe(case_dir: Path) -> str:
92 """!
93 @brief Write a post recipe requesting a non-canonical output directory.
94
95 @details Runtime path injection must replace this request with the stable
96 `output/visualization/<recipe_id>` home.
97 @param[in] case_dir Case directory to write into.
98 @return File name of the variant recipe.
99 """
100 source = (case_dir / "config" / "post.yml").read_text(encoding="utf-8")
101 source = re.sub(r'^(\s*)output_directory:\s*.*$', r'\1output_directory: "viz"',
102 source, flags=re.M)
103 (case_dir / "config" / "Flat_Viz_Analysis.yml").write_text(source, encoding="utf-8")
104 return "Flat_Viz_Analysis.yml"
105
106
Here is the caller graph for this function:

◆ map_to_identities()

dict extract_artifact_topology.map_to_identities ( list  artifacts,
dict  contract 
)

Map each normalized artifact token onto a declared logical identity.

An artifact the contract does not name is reported rather than ignored: an unmapped path is a documented-layout gap, which is exactly what a topology contract exists to surface.

Parameters
[in]artifactsNormalized artifact tokens.
[in]contractParsed topology contract.
Returns
Mapping of identity id to matched tokens, plus an unmapped list.

Definition at line 107 of file extract_artifact_topology.py.

107def map_to_identities(artifacts: list, contract: dict) -> dict:
108 """!
109 @brief Map each normalized artifact token onto a declared logical identity.
110
111 An artifact the contract does not name is reported rather than ignored: an
112 unmapped path is a documented-layout gap, which is exactly what a topology
113 contract exists to surface.
114 @param[in] artifacts Normalized artifact tokens.
115 @param[in] contract Parsed topology contract.
116 @return Mapping of identity id to matched tokens, plus an `unmapped` list.
117 """
118 logical_roots = {
119 "<run.root>": r"<workspace>/runs/<run_id>",
120 "<run.config>": r"<workspace>/runs/<run_id>/config",
121 "<run.post_recipes>": r"<workspace>/runs/<run_id>/config/post\-recipes",
122 "<run.inputs>": r"<workspace>/runs/<run_id>/inputs",
123 "<run.runtime_logs>": r"<workspace>/runs/<run_id>/logs",
124 "<run.scheduler>": r"<workspace>/runs/<run_id>/scheduler",
125 "<run.solver_output>": r"<workspace>/runs/<run_id>/output",
126 "<run.analysis>": r"<workspace>/runs/<run_id>/output/analysis",
127 "<run.visualization>": r"<workspace>/runs/<run_id>/output/visualization",
128 }
129 rules = []
130 for record in contract["artifacts"]:
131 rule = record["path_rule"]
132 pattern = re.escape(rule)
133 pattern = pattern.replace(re.escape("<workspace>"), r"<workspace>")
134 for token, resolved in logical_roots.items():
135 pattern = pattern.replace(re.escape(token), resolved)
136 pattern = pattern.replace(re.escape("<role>"), r"[^/]+")
137 pattern = pattern.replace(re.escape("<ext>"), r"[^/]+")
138 pattern = pattern.replace(re.escape("<name>"), r"[^/]+")
139 pattern = pattern.replace(re.escape("<recipe>"), r"[^/]+")
140 pattern = pattern.replace(re.escape("<run_id>"), r"<run_id>")
141 pattern = pattern.replace(re.escape("<n>"), r"[^/]+")
142 rules.append((record["id"], re.compile("^" + pattern + "$")))
143
144 # Specific rules must win over wildcard ones. A configurable directory such as
145 # `<run.root>/<configured log dir>` matches any single segment, so first-match
146 # order let it swallow manifest.json, output, and scheduler.
147 def specificity(item) -> tuple:
148 """!
149 @brief Rank a rule so literal path rules are tried before wildcard ones.
150 @param[in] item Tuple of identity id and compiled pattern.
151 @return Sort key placing more specific rules first.
152 """
153 pattern = item[1].pattern
154 return (pattern.count("[^/]+"), -len(pattern))
155
156 ordered = sorted(rules, key=specificity)
157 mapped: dict = {rid: [] for rid, _ in rules}
158 unmapped = []
159 for token in artifacts:
160 for rid, pattern in ordered:
161 if pattern.match(token):
162 mapped[rid].append(token)
163 break
164 else:
165 unmapped.append(token)
166 return {"mapped": {k: sorted(v) for k, v in mapped.items() if v}, "unmapped": sorted(unmapped)}
167
168
Here is the caller graph for this function:

◆ build_snapshot()

dict extract_artifact_topology.build_snapshot ( )

Build the normalized topology snapshot across scenarios.

Returns
Snapshot mapping.

Definition at line 169 of file extract_artifact_topology.py.

169def build_snapshot() -> dict:
170 """!
171 @brief Build the normalized topology snapshot across scenarios.
172 @return Snapshot mapping.
173 """
174 contract = json.loads(CONTRACT_PATH.read_text(encoding="utf-8"))
175 scenarios = []
176 workspace = Path(tempfile.mkdtemp(prefix="picurv-topology-"))
177 try:
178 case_dir = init_case(workspace)
179 flat_post = flat_post_recipe(case_dir)
180 cases = [
181 ("fresh_local_solve_and_post", "monitor.yml",
182 ["--solve", "--post-process"], "post.yml"),
183 ("fresh_local_solve_only", "monitor.yml", ["--solve"],
184 "post.yml"),
185 # A non-canonical request proves the runtime path is still fixed.
186 ("post_output_request_is_canonicalized", "monitor.yml",
187 ["--solve", "--post-process"], flat_post),
188 ]
189 for name, monitor, extra, post in cases:
190 plan = run_plan(workspace, case_dir, monitor, extra, post)
191 artifacts = normalize(plan.get("artifacts", []), case_dir, plan["run_id_preview"])
192 scenarios.append({
193 "scenario": name,
194 "launch_mode": plan.get("launch_mode"),
195 "artifacts": artifacts,
196 "identities": map_to_identities(artifacts, contract),
197 })
198 finally:
199 shutil.rmtree(workspace, ignore_errors=True)
200 return {
201 "default_layout": contract["default_layout"],
202 "isolation_enforced": False,
203 "logical_artifacts": [a["id"] for a in contract["artifacts"]],
204 "scenarios": scenarios,
205 }
206
207
Here is the call graph for this function:
Here is the caller graph for this function:

◆ main()

int extract_artifact_topology.main ( )

Write or verify the artifact topology snapshot.

Returns
Process status code.

Definition at line 208 of file extract_artifact_topology.py.

208def main() -> int:
209 """!
210 @brief Write or verify the artifact topology snapshot.
211 @return Process status code.
212 """
213 parser = argparse.ArgumentParser(description="Extract the run artifact topology snapshot.")
214 parser.add_argument("--check", action="store_true", help="Fail if the snapshot is stale.")
215 args = parser.parse_args()
216
217 try:
218 snapshot = build_snapshot()
219 except (RuntimeError, subprocess.TimeoutExpired, json.JSONDecodeError) as error:
220 print(f"Artifact topology extraction failed: {error}", file=sys.stderr)
221 return 1
222
223 content = json.dumps(snapshot, indent=2, sort_keys=True) + "\n"
224 if args.check:
225 if not SNAPSHOT_PATH.is_file() or SNAPSHOT_PATH.read_text(encoding="utf-8") != content:
226 print(
227 "Run artifact topology has changed. The planned artifact set no longer matches "
228 "the recorded snapshot.\n"
229 " Review the pages that document run layout, then refresh with:\n"
230 " make docs-topology",
231 file=sys.stderr,
232 )
233 return 1
234 print(f"Artifact topology snapshot is current ({len(snapshot['scenarios'])} scenario(s)).")
235 return 0
236
237 SNAPSHOT_PATH.parent.mkdir(parents=True, exist_ok=True)
238 SNAPSHOT_PATH.write_text(content, encoding="utf-8")
239 total = sum(len(s["artifacts"]) for s in snapshot["scenarios"])
240 print(f"Wrote artifact topology snapshot: {len(snapshot['scenarios'])} scenario(s), {total} artifacts.")
241 return 0
242
243
int main(int argc, char **argv)
Entry point for the postprocessor executable.
Here is the call graph for this function:
Here is the caller graph for this function:

Variable Documentation

◆ REPO_ROOT

extract_artifact_topology.REPO_ROOT = Path(__file__).resolve().parents[2]

Definition at line 17 of file extract_artifact_topology.py.

◆ CONTRACT_PATH

str extract_artifact_topology.CONTRACT_PATH = REPO_ROOT / "tests" / "tooling" / "artifact_topology.json"

Definition at line 18 of file extract_artifact_topology.py.

◆ SNAPSHOT_PATH

str extract_artifact_topology.SNAPSHOT_PATH = REPO_ROOT / "docs" / "generated" / "artifact_topology_snapshot.json"

Definition at line 19 of file extract_artifact_topology.py.

◆ PICURV

str extract_artifact_topology.PICURV = REPO_ROOT / "picurv_cli" / "picurv"

Definition at line 20 of file extract_artifact_topology.py.