diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 6f5dc44..7868f6a 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -40,8 +40,10 @@ jobs: cargo test --release -p arm-lab cargo test --release -p arm-lab-demo --bin multi_query_bench cargo test --release -p arm-lab-demo --bin randomized_eval + cargo test --release -p arm-lab-demo --bin oxmpl_baseline cargo run --release -p arm-lab-demo --bin multi_query_bench -- --check cargo run --release -p arm-lab-demo --bin randomized_eval -- --check + cargo run --release -p arm-lab-demo --bin oxmpl_baseline -- --verify-artifacts - name: Build demo (no GL runtime needed to compile) run: cargo build --release -p arm-lab-demo diff --git a/Cargo.lock b/Cargo.lock index aa46ac0..f3179af 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -313,7 +313,10 @@ dependencies = [ "arm-lab", "mujoco-rs", "nalgebra 0.35.0", + "oxmpl", + "rand 0.9.5", "rerun", + "sha2 0.10.9", ] [[package]] @@ -5927,6 +5930,18 @@ version = "4.3.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "d211803b9b6b570f68772237e415a029d5a50c65d382910b879fb19d3271f94d" +[[package]] +name = "oxmpl" +version = "0.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "11988e3c34bfcacbc4d96265f1da76f9bd3a30fcc92ad62e4046cc6315e980dc" +dependencies = [ + "getrandom 0.3.4", + "rand 0.9.5", + "rand_chacha", + "web-time", +] + [[package]] name = "parking" version = "2.2.1" diff --git a/README.md b/README.md index 03b6476..8a7edd7 100644 --- a/README.md +++ b/README.md @@ -2,9 +2,9 @@ [![Main CI](https://github.com/Dylan-Gallagher/arm-lab/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/Dylan-Gallagher/arm-lab/actions/workflows/ci.yml?query=branch%3Amain) -A serial-chain manipulator stack written **from scratch in Rust** — no ROS, no MoveIt, no kinematics library, no off-the-shelf planner. +A serial-chain manipulator stack whose production algorithms are written **from scratch in Rust** — no ROS, MoveIt, kinematics library, or off-the-shelf production planner. Independently maintained OxMPL is pinned only in a separate evaluation binary for a cross-implementation baseline. -**Evidence index (simulation; sampled where collision claims are involved; no hardware validation):** randomized evaluation [protocol](docs/randomized_eval_protocol.md) / [report](docs/randomized_eval_results.md) / [queries](docs/randomized_eval_queries.csv) / [planning](docs/randomized_eval_planning.csv) / [tracking](docs/randomized_eval_tracking.csv) · [multi-scene report](docs/multi_query_results.md) · [planning CSV](docs/multi_query_planning.csv) · [tracking CSV](docs/multi_query_tracking.csv) · attached payload [protocol](docs/attached_payload_protocol.md) / [results](docs/attached_payload_results.md) · corner-stop [protocol](docs/corner_stop_protocol.md) / [results](docs/corner_stop_results.md) · [robustness report](docs/robustness_results.md) · [main CI](https://github.com/Dylan-Gallagher/arm-lab/actions/workflows/ci.yml?query=branch%3Amain) · [citation](CITATION.cff) +**Evidence index (simulation; sampled where collision claims are involved; no hardware validation):** randomized evaluation [protocol](docs/randomized_eval_protocol.md) / [report](docs/randomized_eval_results.md) / [queries](docs/randomized_eval_queries.csv) / [planning](docs/randomized_eval_planning.csv) / [tracking](docs/randomized_eval_tracking.csv) · independent planner [protocol](docs/oxmpl_baseline_protocol.md) / [report](docs/oxmpl_baseline_results.md) / [paired CSV](docs/oxmpl_baseline_results.csv) / [validation](docs/oxmpl_baseline_validation.md) · [multi-scene report](docs/multi_query_results.md) · [planning CSV](docs/multi_query_planning.csv) · [tracking CSV](docs/multi_query_tracking.csv) · attached payload [protocol](docs/attached_payload_protocol.md) / [results](docs/attached_payload_results.md) · corner-stop [protocol](docs/corner_stop_protocol.md) / [results](docs/corner_stop_results.md) · [robustness report](docs/robustness_results.md) · [main CI](https://github.com/Dylan-Gallagher/arm-lab/actions/workflows/ci.yml?query=branch%3Amain) · [citation](CITATION.cff) ![Demo 3 — UR5e pick-and-place around a pillar. IK, RRT-Connect, corner-stop S-curves, PD + velocity feedforward](docs/demo3.gif) @@ -58,6 +58,12 @@ Position PD meets the numeric gates in **0/18** tracking cases. Desired-velocity Both nominal controllers replay every successful canonical trajectory. Position PD meets the numeric gates in **0/138** cases. Velocity feedforward meets them in **104/138** (75.4%, 95% Wilson 67.6--81.8%) and passes the added zero-sampled-penetration gate in **90/138** (65.2%, 95% Wilson 57.0--72.7%). Thirty-seven of 276 controller replays contain sampled penetration, with 3.2141 mm maximum depth; these failures are retained rather than hidden. See the pre-result [protocol](docs/randomized_eval_protocol.md), generated [report](docs/randomized_eval_results.md), and complete [query](docs/randomized_eval_queries.csv), [planning](docs/randomized_eval_planning.csv), and [tracking](docs/randomized_eval_tracking.csv) CSVs. The generator is uniform in compiled joint limits under the stated conditions, not uniform in Cartesian workspace or representative of real tasks; collision checks remain discrete and all evidence is simulation-only. +## Independent planner cross-check (simulation) + +An outcome-blind [protocol](docs/oxmpl_baseline_protocol.md) pins independently maintained **OxMPL 0.6.0** before comparing its RRT-Connect implementation with the committed in-repository rows on all 217 blocked-direct queries and the same five seeds. The in-repository planner succeeds on **271/1,085** paired trials; OxMPL succeeds on **290/1,085**. The paired cells are 271 both, 0 in-repository only, 19 OxMPL only, and 795 neither. The predeclared exact two-sided McNemar/binomial calculation is 0.00000381, but it is descriptive only: five seeds share each query, violating independent-pair assumptions. At the query level, at least one seed succeeds for 56/217 blocked queries in-repository and 58/217 with OxMPL. + +Every nominal external path is rechecked with a fresh arm-lab collision checker at 0.05-rad spacing: **290/290 pass**, with zero invalid-path outcomes. Two consecutive full runs on the recorded Fedora/Ryzen host agree in every status and path digest after normalizing wall time. This corroborates every in-repository success in the frozen sample and shows limited complementary recovery, but it is not a compute-matched or universal planner ranking: the in-repository artifact uses 2,000 iterations, OxMPL uses a predeclared 250-ms timeout, and the external evaluator consumes the eight-decimal serialized query vectors. The integer seeds and replicate indices are paired, but the two planners use different PRNGs and therefore do not receive identical random samples. See the generated [report](docs/oxmpl_baseline_results.md), complete [1,085-row paired CSV](docs/oxmpl_baseline_results.csv), and [validation record](docs/oxmpl_baseline_validation.md). + ## Layout ``` @@ -105,6 +111,10 @@ cargo run --release -p arm-lab-demo --bin multi_query_bench -- --check # predeclared 300-query randomized evaluation; write or verify all raw outcomes cargo run --release -p arm-lab-demo --bin randomized_eval -- --write cargo run --release -p arm-lab-demo --bin randomized_eval -- --check + +# independent external RRT-Connect baseline; full pinned-host repeat or fast artifact audit +cargo run --release -p arm-lab-demo --bin oxmpl_baseline -- --check +cargo run --release -p arm-lab-demo --bin oxmpl_baseline -- --verify-artifacts ``` Requirements: Rust stable, a C++ toolchain, and (for `--render`) `ffmpeg` on PATH. On Linux without system MuJoCo, `mujoco-rs` auto-downloads MuJoCo 3.9 at build time. Set `MUJOCO_DOWNLOAD_DIR` to an absolute directory before building and add its downloaded `lib/` directory to `LD_LIBRARY_PATH` before running; see the [mujoco-rs docs](https://github.com/davidhozic/mujoco-rs). @@ -131,9 +141,10 @@ Requirements: Rust stable, a C++ toolchain, and (for `--render`) `ffmpeg` on PAT - [x] Multi-scene, multi-query planning and tracking extension with raw CSVs - [x] Pair-scoped attached-cube collision proxy for Demo 3 carry - [x] Predeclared 300-query randomized planning and nominal-tracking evaluation +- [x] Predeclared independent OxMPL comparison on all 217 blocked randomized queries ## License & assets -Code: Apache-2.0. The vendored `assets/ur5e` model and meshes are from [MuJoCo Menagerie](https://github.com/google-deepmind/mujoco_menagerie) (UR5e description © 2018 ROS Industrial Consortium, BSD-3; see `assets/ur5e/LICENSE`), with a local modification adding offscreen buffer size to `scene.xml`. `scene_cluttered.xml` and `scene_pickplace.xml` are original to this repo. +Project code: Apache-2.0. The separate comparison executable depends on non-vendored [OxMPL 0.6.0](https://github.com/juniorsundar/oxmpl) under BSD-3-Clause. The vendored `assets/ur5e` model and meshes are from [MuJoCo Menagerie](https://github.com/google-deepmind/mujoco_menagerie) (UR5e description © 2018 ROS Industrial Consortium, BSD-3; see `assets/ur5e/LICENSE`), with a local modification adding offscreen buffer size to `scene.xml`. `scene_cluttered.xml` and `scene_pickplace.xml` are original to this repo. Research users can cite the software using [`CITATION.cff`](CITATION.cff). diff --git a/crates/arm-lab-demo/Cargo.toml b/crates/arm-lab-demo/Cargo.toml index 331d632..27e5fd1 100644 --- a/crates/arm-lab-demo/Cargo.toml +++ b/crates/arm-lab-demo/Cargo.toml @@ -10,7 +10,10 @@ publish = false arm-lab = { path = "../arm-lab" } nalgebra = { workspace = true } mujoco-rs = { workspace = true, features = ["renderer"] } +oxmpl = "=0.6.0" +rand = "=0.9.5" rerun = "0.36" +sha2 = "=0.10.9" [lints] workspace = true diff --git a/crates/arm-lab-demo/src/bin/oxmpl_baseline.rs b/crates/arm-lab-demo/src/bin/oxmpl_baseline.rs new file mode 100644 index 0000000..b87c2e0 --- /dev/null +++ b/crates/arm-lab-demo/src/bin/oxmpl_baseline.rs @@ -0,0 +1,1095 @@ +//! Predeclared cross-implementation RRT-Connect comparison. +//! +//! See `docs/oxmpl_baseline_protocol.md` for the frozen design and claim +//! boundaries. OxMPL is independently maintained; every returned path is +//! revalidated with arm-lab's collision checker before it counts as success. + +use std::collections::{BTreeMap, BTreeSet}; +use std::fmt::Write as _; +use std::path::{Path, PathBuf}; +use std::sync::{Arc, Mutex}; +use std::time::{Duration, Instant}; + +use arm_lab::plan::{edge_free, path_length}; +use arm_lab::{Chain, CollisionChecker}; +use mujoco_rs::prelude::*; +use oxmpl::base::error::{PlanningError, StateSamplingError}; +use oxmpl::base::goal::{Goal, GoalRegion, GoalSampleableRegion}; +use oxmpl::base::planner::{Planner, PlannerConfig}; +use oxmpl::base::problem_definition::ProblemDefinition; +use oxmpl::base::space::{RealVectorStateSpace, StateSpace}; +use oxmpl::base::state::RealVectorState; +use oxmpl::base::validity::StateValidityChecker; +use oxmpl::geometric::RRTConnect; +use rand::Rng; +use sha2::{Digest, Sha256}; + +const ASSET_DIR: &str = concat!(env!("CARGO_MANIFEST_DIR"), "/../../assets/ur5e/"); +const QUERY_SHA256: &str = "c07b6227183a55e273120e37c15e86d99f717dc8f47b6a9d3d989d05415b5452"; +const PLANNING_SHA256: &str = "b739009c0a4b3691bc910f67ad079a944e1b16c83d341bcd078179bda6c13055"; +const EXPECTED_BLOCKED_QUERIES: usize = 217; +const REPLICATES: usize = 5; +const PLANNER_SEED_BASE: u64 = 202_608_180_000; +const MAX_DISTANCE_RAD: f64 = 0.25; +const GOAL_BIAS: f64 = 0.05; +const EDGE_RESOLUTION_RAD: f64 = 0.05; +const JOINT_LIMIT_INSET_RAD: f64 = 0.05; +const SOLVE_TIMEOUT: Duration = Duration::from_millis(250); +const ENDPOINT_TOLERANCE_RAD: f64 = 1e-9; + +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +struct Scene { + index: usize, + name: &'static str, + file: &'static str, +} + +const SCENES: [Scene; 3] = [ + Scene { + index: 0, + name: "open_floor", + file: "scene.xml", + }, + Scene { + index: 1, + name: "offset_pillar", + file: "scene_cluttered.xml", + }, + Scene { + index: 2, + name: "tabletop_pillar", + file: "scene_pickplace.xml", + }, +]; + +#[derive(Clone, Debug, PartialEq)] +struct QueryRow { + scene: Scene, + query_index: usize, + start: Vec, + goal: Vec, +} + +#[derive(Clone, Debug, PartialEq, Eq)] +struct InHouseRow { + success: bool, + seed: u64, +} + +#[derive(Debug)] +struct Inputs { + queries: Vec, + in_house: BTreeMap<(usize, usize, usize), InHouseRow>, +} + +#[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord)] +enum OxmplStatus { + Success, + Timeout, + PlanningError, + InvalidPath, +} + +impl OxmplStatus { + fn as_str(self) -> &'static str { + match self { + Self::Success => "success", + Self::Timeout => "timeout", + Self::PlanningError => "planning_error", + Self::InvalidPath => "invalid_path", + } + } + + fn parse(value: &str) -> Self { + match value { + "success" => Self::Success, + "timeout" => Self::Timeout, + "planning_error" => Self::PlanningError, + "invalid_path" => Self::InvalidPath, + _ => panic!("unknown OxMPL status: {value}"), + } + } + + fn succeeded(self) -> bool { + self == Self::Success + } +} + +#[derive(Clone, Debug, PartialEq)] +struct ResultRow { + scene: Scene, + query_index: usize, + replicate: usize, + seed: u64, + in_house_success: bool, + oxmpl_status: OxmplStatus, + oxmpl_detail: String, + elapsed_ms: f64, + waypoints: usize, + path_cost_rad: f64, + path_sha256: String, +} + +#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)] +struct PairCounts { + total: usize, + in_house_success: usize, + oxmpl_success: usize, + both: usize, + in_house_only: usize, + oxmpl_only: usize, + neither: usize, +} + +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +enum Mode { + Run, + Write, + Check, + VerifyArtifacts, +} + +fn main() { + let mode = parse_mode(); + let inputs = load_inputs(); + + if mode == Mode::VerifyArtifacts { + verify_committed_artifacts(&inputs); + return; + } + + let rows = run_study(&inputs); + validate_result_layout(&inputs, &rows); + let csv = render_csv(&rows); + let report = render_report(&rows); + + match mode { + Mode::Run => print!("{report}"), + Mode::Write => write_artifacts(&csv, &report), + Mode::Check => check_artifacts(&csv, &report), + Mode::VerifyArtifacts => unreachable!(), + } +} + +fn parse_mode() -> Mode { + let args: Vec = std::env::args().skip(1).collect(); + match args.as_slice() { + [] => Mode::Run, + [arg] if arg == "--write" => Mode::Write, + [arg] if arg == "--check" => Mode::Check, + [arg] if arg == "--verify-artifacts" => Mode::VerifyArtifacts, + _ => panic!("usage: oxmpl_baseline [--write|--check|--verify-artifacts]"), + } +} + +fn docs_dir() -> PathBuf { + Path::new(env!("CARGO_MANIFEST_DIR")).join("../../docs") +} + +fn load_inputs() -> Inputs { + let docs = docs_dir(); + let query_bytes = std::fs::read(docs.join("randomized_eval_queries.csv")) + .expect("read frozen randomized queries"); + let planning_bytes = std::fs::read(docs.join("randomized_eval_planning.csv")) + .expect("read frozen randomized planning rows"); + assert_eq!(sha256_hex(&query_bytes), QUERY_SHA256, "query input hash"); + assert_eq!( + sha256_hex(&planning_bytes), + PLANNING_SHA256, + "planning input hash" + ); + + let query_csv = std::str::from_utf8(&query_bytes).expect("query CSV UTF-8"); + let planning_csv = std::str::from_utf8(&planning_bytes).expect("planning CSV UTF-8"); + let queries = parse_queries(query_csv); + let in_house = parse_in_house(planning_csv); + + assert_eq!( + queries.len(), + EXPECTED_BLOCKED_QUERIES, + "frozen blocked-query count" + ); + assert_eq!( + in_house.len(), + EXPECTED_BLOCKED_QUERIES * REPLICATES, + "frozen paired in-repository trial count" + ); + for query in &queries { + for replicate in 0..REPLICATES { + let key = (query.scene.index, query.query_index, replicate); + let row = in_house.get(&key).expect("paired in-repository row"); + assert_eq!( + row.seed, + planner_seed(query.scene.index, query.query_index, replicate), + "paired seed" + ); + } + } + Inputs { queries, in_house } +} + +fn parse_queries(csv: &str) -> Vec { + let mut lines = csv.lines(); + let header = lines.next().expect("query CSV header"); + assert_eq!( + header, + "scene,scene_file,query_index,generator_seed,accepted_draw_index,\ + cumulative_rejected_start_collision,cumulative_rejected_goal_collision,\ + cumulative_rejected_short,start_q_rad,goal_q_rad,distance_rad,direct_path_free" + ); + let mut queries = Vec::new(); + for (line_index, line) in lines.enumerate() { + let fields: Vec<_> = line.split(',').collect(); + assert_eq!( + fields.len(), + 12, + "query CSV line {} columns", + line_index + 2 + ); + if parse_bool(fields[11]) { + continue; + } + let scene = scene_from_fields(fields[0], fields[1]); + queries.push(QueryRow { + scene, + query_index: parse(fields[2], "query index"), + start: parse_vector(fields[8], "start"), + goal: parse_vector(fields[9], "goal"), + }); + } + queries.sort_by_key(|row| (row.scene.index, row.query_index)); + queries +} + +fn parse_in_house(csv: &str) -> BTreeMap<(usize, usize, usize), InHouseRow> { + let mut lines = csv.lines(); + let header = lines.next().expect("planning CSV header"); + assert_eq!( + header, + "scene,scene_file,query_index,variant,replicate,seed,direct_path_free,status,\ + plan_elapsed_ms,iterations,nodes,shortcut_waypoints,path_samples,path_cost_rad" + ); + let mut rows = BTreeMap::new(); + for (line_index, line) in lines.enumerate() { + let fields: Vec<_> = line.split(',').collect(); + assert_eq!( + fields.len(), + 14, + "planning CSV line {} columns", + line_index + 2 + ); + if fields[3] != "default" || parse_bool(fields[6]) { + continue; + } + let scene = scene_from_fields(fields[0], fields[1]); + let query_index = parse(fields[2], "planning query index"); + let replicate = parse(fields[4], "planning replicate"); + let seed = parse(fields[5], "planning seed"); + assert!(replicate < REPLICATES, "unexpected planning replicate"); + assert!( + matches!(fields[7], "success" | "unconnected"), + "unexpected in-repository status" + ); + let key = (scene.index, query_index, replicate); + assert!( + rows.insert( + key, + InHouseRow { + success: fields[7] == "success", + seed, + }, + ) + .is_none(), + "duplicate in-repository row" + ); + } + rows +} + +fn parse_vector(value: &str, label: &str) -> Vec { + let values: Vec = value.split(';').map(|field| parse(field, label)).collect(); + assert_eq!(values.len(), 6, "{label} dimensions"); + assert!( + values.iter().all(|value| value.is_finite()), + "{label} finite" + ); + values +} + +fn parse_bool(value: &str) -> bool { + match value { + "true" => true, + "false" => false, + _ => panic!("invalid boolean: {value}"), + } +} + +fn parse(value: &str, label: &str) -> T { + value + .parse() + .unwrap_or_else(|_| panic!("invalid {label}: {value}")) +} + +fn scene_from_fields(name: &str, file: &str) -> Scene { + SCENES + .iter() + .copied() + .find(|scene| scene.name == name && scene.file == file) + .unwrap_or_else(|| panic!("unknown scene/file pair: {name}/{file}")) +} + +fn planner_seed(scene_index: usize, query_index: usize, replicate: usize) -> u64 { + PLANNER_SEED_BASE + 10_000 * scene_index as u64 + 10 * query_index as u64 + replicate as u64 +} + +fn run_study(inputs: &Inputs) -> Vec { + let mut rows = Vec::with_capacity(EXPECTED_BLOCKED_QUERIES * REPLICATES); + for scene in SCENES { + let model = Arc::new(load_model(scene)); + let chain = Chain::from_mujoco(&model, "ur5e", "wrist_3_link", "attachment_site") + .expect("extract UR5e chain"); + let bounds = planning_bounds(&chain); + let mut space = RealVectorStateSpace::new(chain.dof(), Some(bounds.clone())) + .expect("bounded OxMPL state space"); + let extent = space.get_maximum_extent(); + space.set_longest_valid_segment_fraction(0.5 / extent); + assert!( + (space.get_longest_valid_segment_length() * 0.1 - EDGE_RESOLUTION_RAD).abs() < 1e-12, + "OxMPL edge resolution configuration" + ); + let space = Arc::new(space); + + let scene_queries: Vec<_> = inputs + .queries + .iter() + .filter(|query| query.scene == scene) + .collect(); + println!( + "{}: {} blocked queries, {} paired trials", + scene.name, + scene_queries.len(), + scene_queries.len() * REPLICATES + ); + for query in scene_queries { + for replicate in 0..REPLICATES { + let key = (scene.index, query.query_index, replicate); + let in_house = inputs.in_house.get(&key).expect("paired row"); + let result = run_oxmpl_trial( + &model, + &chain, + Arc::clone(&space), + &bounds, + query, + replicate, + in_house, + ); + println!( + " query {:03} rep {}: in_house={} oxmpl={} ({:.2} ms)", + query.query_index, + replicate, + if in_house.success { + "success" + } else { + "failed" + }, + result.oxmpl_status.as_str(), + result.elapsed_ms + ); + rows.push(result); + } + } + } + rows +} + +fn load_model(scene: Scene) -> MjModel { + let path = format!("{ASSET_DIR}{}", scene.file); + MjModel::from_xml(&path).unwrap_or_else(|error| panic!("load {}: {error}", scene.file)) +} + +fn planning_bounds(chain: &Chain) -> Vec<(f64, f64)> { + chain + .joint_limits() + .into_iter() + .map(|limit| { + let (lower, upper) = limit.unwrap_or((-3.0, 3.0)); + let inset = (lower + JOINT_LIMIT_INSET_RAD, upper - JOINT_LIMIT_INSET_RAD); + assert!(inset.0 < inset.1, "joint limit survives inset"); + inset + }) + .collect() +} + +#[derive(Clone)] +struct PointGoal { + target: RealVectorState, +} + +impl Goal for PointGoal { + fn is_satisfied(&self, state: &RealVectorState) -> bool { + l2(&state.values, &self.target.values) <= ENDPOINT_TOLERANCE_RAD + } +} + +impl GoalRegion for PointGoal { + fn distance_goal(&self, state: &RealVectorState) -> f64 { + l2(&state.values, &self.target.values) + } +} + +impl GoalSampleableRegion for PointGoal { + fn sample_goal(&self, _rng: &mut impl Rng) -> Result { + Ok(self.target.clone()) + } +} + +struct MujocoValidity { + bounds: Vec<(f64, f64)>, + checker: Mutex>>, +} + +impl StateValidityChecker for MujocoValidity { + fn is_valid(&self, state: &RealVectorState) -> bool { + if !numeric_state_is_valid(&state.values, &self.bounds) { + return false; + } + !self + .checker + .lock() + .expect("OxMPL collision checker lock") + .collides(&state.values) + } +} + +fn run_oxmpl_trial( + model: &Arc, + chain: &Chain, + space: Arc, + bounds: &[(f64, f64)], + query: &QueryRow, + replicate: usize, + in_house: &InHouseRow, +) -> ResultRow { + let seed = planner_seed(query.scene.index, query.query_index, replicate); + assert_eq!(seed, in_house.seed, "paired seed before OxMPL run"); + let goal = Arc::new(PointGoal { + target: RealVectorState::new(query.goal.clone()), + }); + let problem = Arc::new(ProblemDefinition { + space: Arc::clone(&space), + start_states: vec![RealVectorState::new(query.start.clone())], + goal, + }); + let validity = Arc::new(MujocoValidity { + bounds: bounds.to_vec(), + checker: Mutex::new(CollisionChecker::new(Arc::clone(model), chain)), + }); + let mut planner = RRTConnect::new( + MAX_DISTANCE_RAD, + GOAL_BIAS, + &PlannerConfig { seed: Some(seed) }, + ); + planner.setup(problem, validity); + let start_time = Instant::now(); + let solved = planner.solve(SOLVE_TIMEOUT); + let elapsed_ms = start_time.elapsed().as_secs_f64() * 1e3; + + let (oxmpl_status, oxmpl_detail, waypoints, path_cost_rad, path_sha256) = match solved { + Ok(path) => { + let path: Vec> = path.0.into_iter().map(|state| state.values).collect(); + let waypoints = path.len(); + let cost = path_length(&path); + let digest = digest_path(&path); + let mut validation_checker = CollisionChecker::new(Arc::clone(model), chain); + let detail = validate_path(&path, &query.start, &query.goal, bounds, |q| { + validation_checker.collides(q) + }); + match detail { + Ok(()) => ( + OxmplStatus::Success, + "validated".to_string(), + waypoints, + cost, + digest, + ), + Err(detail) => (OxmplStatus::InvalidPath, detail, waypoints, cost, digest), + } + } + Err(PlanningError::Timeout) => ( + OxmplStatus::Timeout, + "solve_timeout".to_string(), + 0, + 0.0, + String::new(), + ), + Err(error) => ( + OxmplStatus::PlanningError, + planning_error_name(&error).to_string(), + 0, + 0.0, + String::new(), + ), + }; + + ResultRow { + scene: query.scene, + query_index: query.query_index, + replicate, + seed, + in_house_success: in_house.success, + oxmpl_status, + oxmpl_detail, + elapsed_ms, + waypoints, + path_cost_rad, + path_sha256, + } +} + +fn planning_error_name(error: &PlanningError) -> &'static str { + match error { + PlanningError::Timeout => "solve_timeout", + PlanningError::NoSolutionFound => "no_solution_found", + PlanningError::PlannerUninitialised => "planner_uninitialised", + PlanningError::InvalidStartState => "invalid_start_state", + PlanningError::UnsampledStateSpace => "unsampled_state_space", + } +} + +fn numeric_state_is_valid(values: &[f64], bounds: &[(f64, f64)]) -> bool { + values.len() == bounds.len() + && values.iter().zip(bounds).all(|(value, (lower, upper))| { + value.is_finite() && *value >= *lower && *value <= *upper + }) +} + +fn validate_path( + path: &[Vec], + start: &[f64], + goal: &[f64], + bounds: &[(f64, f64)], + mut collides: impl FnMut(&[f64]) -> bool, +) -> Result<(), String> { + if path.is_empty() { + return Err("empty_path".to_string()); + } + for state in path { + if state.len() != bounds.len() { + return Err("invalid_dimension".to_string()); + } + if state.iter().any(|value| !value.is_finite()) { + return Err("non_finite_state".to_string()); + } + if !numeric_state_is_valid(state, bounds) { + return Err("state_out_of_bounds".to_string()); + } + } + if l2(&path[0], start) > ENDPOINT_TOLERANCE_RAD { + return Err("start_mismatch".to_string()); + } + if l2(path.last().expect("non-empty path"), goal) > ENDPOINT_TOLERANCE_RAD { + return Err("goal_mismatch".to_string()); + } + if collides(&path[0]) { + return Err("start_collision".to_string()); + } + let mut scratch = vec![0.0; bounds.len()]; + for segment in path.windows(2) { + if !edge_free( + &segment[0], + &segment[1], + EDGE_RESOLUTION_RAD, + &mut collides, + &mut scratch, + ) { + return Err("edge_collision".to_string()); + } + } + Ok(()) +} + +fn l2(a: &[f64], b: &[f64]) -> f64 { + if a.len() != b.len() { + return f64::INFINITY; + } + a.iter() + .zip(b) + .map(|(left, right)| (left - right).powi(2)) + .sum::() + .sqrt() +} + +fn digest_path(path: &[Vec]) -> String { + let mut hasher = Sha256::new(); + hasher.update((path.len() as u64).to_le_bytes()); + for state in path { + hasher.update((state.len() as u64).to_le_bytes()); + for value in state { + hasher.update(value.to_bits().to_le_bytes()); + } + } + format_digest(hasher.finalize()) +} + +fn sha256_hex(bytes: &[u8]) -> String { + format_digest(Sha256::digest(bytes)) +} + +fn format_digest(digest: impl AsRef<[u8]>) -> String { + digest + .as_ref() + .iter() + .map(|byte| format!("{byte:02x}")) + .collect() +} + +fn validate_result_layout(inputs: &Inputs, rows: &[ResultRow]) { + assert_eq!( + rows.len(), + EXPECTED_BLOCKED_QUERIES * REPLICATES, + "result row count" + ); + let mut keys = BTreeSet::new(); + for row in rows { + assert!(row.replicate < REPLICATES, "result replicate range"); + let key = (row.scene.index, row.query_index, row.replicate); + assert!(keys.insert(key), "duplicate result row"); + let input = inputs.in_house.get(&key).expect("result input pair"); + assert_eq!(row.seed, input.seed, "result seed"); + assert_eq!( + row.in_house_success, input.success, + "result in-repository status" + ); + assert!(row.elapsed_ms.is_finite() && row.elapsed_ms >= 0.0); + if row.oxmpl_status == OxmplStatus::Success { + assert_eq!(row.oxmpl_detail, "validated"); + assert!(row.waypoints >= 2); + assert!(row.path_cost_rad.is_finite() && row.path_cost_rad > 0.0); + assert_eq!(row.path_sha256.len(), 64); + } + } +} + +fn aggregate(rows: impl Iterator) -> PairCounts { + let mut counts = PairCounts::default(); + for row in rows { + let oxmpl_success = row.oxmpl_status.succeeded(); + counts.total += 1; + counts.in_house_success += usize::from(row.in_house_success); + counts.oxmpl_success += usize::from(oxmpl_success); + match (row.in_house_success, oxmpl_success) { + (true, true) => counts.both += 1, + (true, false) => counts.in_house_only += 1, + (false, true) => counts.oxmpl_only += 1, + (false, false) => counts.neither += 1, + } + } + assert_eq!( + counts.total, + counts.both + counts.in_house_only + counts.oxmpl_only + counts.neither + ); + counts +} + +fn exact_mcnemar_p(in_house_only: usize, oxmpl_only: usize) -> f64 { + let n = in_house_only + oxmpl_only; + if n == 0 { + return 1.0; + } + let tail = in_house_only.min(oxmpl_only); + let ln_choose = (0..tail).fold(0.0, |sum, index| { + sum + ((n - index) as f64).ln() - ((index + 1) as f64).ln() + }); + let mut probability = (ln_choose - n as f64 * 2.0f64.ln()).exp(); + let mut cumulative = probability; + for value in (1..=tail).rev() { + probability *= value as f64 / (n - value + 1) as f64; + cumulative += probability; + } + (2.0 * cumulative).min(1.0) +} + +fn success_histograms(rows: &[ResultRow]) -> ([usize; 6], [usize; 6]) { + let mut per_query: BTreeMap<(usize, usize), (usize, usize)> = BTreeMap::new(); + for row in rows { + let entry = per_query + .entry((row.scene.index, row.query_index)) + .or_default(); + entry.0 += usize::from(row.in_house_success); + entry.1 += usize::from(row.oxmpl_status.succeeded()); + } + assert_eq!(per_query.len(), EXPECTED_BLOCKED_QUERIES); + let mut in_house = [0; 6]; + let mut oxmpl = [0; 6]; + for (left, right) in per_query.values() { + assert!(*left <= REPLICATES && *right <= REPLICATES); + in_house[*left] += 1; + oxmpl[*right] += 1; + } + (in_house, oxmpl) +} + +fn render_csv(rows: &[ResultRow]) -> String { + let mut csv = String::from( + "scene,scene_file,query_index,replicate,seed,in_house_status,oxmpl_status,\ + oxmpl_detail,oxmpl_elapsed_ms,oxmpl_waypoints,oxmpl_path_cost_rad,\ + oxmpl_path_sha256\n", + ); + for row in rows { + writeln!( + csv, + "{},{},{},{},{},{},{},{},{:.8},{},{:.8},{}", + row.scene.name, + row.scene.file, + row.query_index, + row.replicate, + row.seed, + if row.in_house_success { + "success" + } else { + "failed" + }, + row.oxmpl_status.as_str(), + row.oxmpl_detail, + row.elapsed_ms, + row.waypoints, + row.path_cost_rad, + row.path_sha256 + ) + .expect("render result row"); + } + csv +} + +fn parse_results(csv: &str) -> Vec { + let mut lines = csv.lines(); + let header = lines.next().expect("OxMPL result header"); + assert_eq!( + header, + "scene,scene_file,query_index,replicate,seed,in_house_status,oxmpl_status,\ + oxmpl_detail,oxmpl_elapsed_ms,oxmpl_waypoints,oxmpl_path_cost_rad,\ + oxmpl_path_sha256" + ); + let mut rows = Vec::new(); + for (line_index, line) in lines.enumerate() { + let fields: Vec<_> = line.split(',').collect(); + assert_eq!(fields.len(), 12, "result line {} columns", line_index + 2); + rows.push(ResultRow { + scene: scene_from_fields(fields[0], fields[1]), + query_index: parse(fields[2], "result query index"), + replicate: parse(fields[3], "result replicate"), + seed: parse(fields[4], "result seed"), + in_house_success: match fields[5] { + "success" => true, + "failed" => false, + _ => panic!("invalid in-repository result status"), + }, + oxmpl_status: OxmplStatus::parse(fields[6]), + oxmpl_detail: fields[7].to_string(), + elapsed_ms: parse(fields[8], "OxMPL elapsed time"), + waypoints: parse(fields[9], "OxMPL waypoints"), + path_cost_rad: parse(fields[10], "OxMPL path cost"), + path_sha256: fields[11].to_string(), + }); + } + rows +} + +fn render_report(rows: &[ResultRow]) -> String { + let pooled = aggregate(rows.iter().cloned()); + let p_value = exact_mcnemar_p(pooled.in_house_only, pooled.oxmpl_only); + let invalid = rows + .iter() + .filter(|row| row.oxmpl_status == OxmplStatus::InvalidPath) + .count(); + let (in_house_histogram, oxmpl_histogram) = success_histograms(rows); + let in_house_any: usize = in_house_histogram[1..].iter().sum(); + let oxmpl_any: usize = oxmpl_histogram[1..].iter().sum(); + + let mut report = String::new(); + report.push_str("# Independent OxMPL RRT-Connect comparison\n\n"); + report.push_str( + "This result follows the outcome-blind `docs/oxmpl_baseline_protocol.md`. \ + It compares the in-repository planner with independently maintained \ + OxMPL 0.6.0 on all **217 frozen blocked-direct queries**, five paired \ + seeds each (**1,085 trials**). Every nominal OxMPL solution was \ + independently revalidated at 0.05-rad edge spacing before counting \ + as success.\n\n", + ); + writeln!( + report, + "Pooled seed-trial success: **{}/{} ({:.2}%)** in-repository and **{}/{} ({:.2}%)** OxMPL.", + pooled.in_house_success, + pooled.total, + percent(pooled.in_house_success, pooled.total), + pooled.oxmpl_success, + pooled.total, + percent(pooled.oxmpl_success, pooled.total) + ) + .expect("render pooled headline"); + writeln!( + report, + "Paired outcomes: **{} both**, **{} in-repository only**, **{} OxMPL only**, and **{} neither**; the predeclared exact two-sided McNemar/binomial calculation is **{p_value:.8}**. Because five seeds share each query, this calculation is descriptive only and does not satisfy independent-pair assumptions.\n", + pooled.both, pooled.in_house_only, pooled.oxmpl_only, pooled.neither + ) + .expect("render paired headline"); + + report.push_str("## Results by scene\n\n"); + report.push_str( + "| Scene | Trials | In-repository success | OxMPL success | Both | In-repository only | OxMPL only | Neither |\n\ + |---|---:|---:|---:|---:|---:|---:|---:|\n", + ); + for scene in SCENES { + let counts = aggregate(rows.iter().filter(|row| row.scene == scene).cloned()); + writeln!( + report, + "| {} | {} | {} | {} | {} | {} | {} | {} |", + scene.name, + counts.total, + counts.in_house_success, + counts.oxmpl_success, + counts.both, + counts.in_house_only, + counts.oxmpl_only, + counts.neither + ) + .expect("render scene result"); + } + + report.push_str("\n## Per-query repeat distribution\n\n"); + report.push_str( + "| Successful seeds out of five | In-repository queries | OxMPL queries |\n\ + |---:|---:|---:|\n", + ); + for successes in 0..=REPLICATES { + writeln!( + report, + "| {successes} | {} | {} |", + in_house_histogram[successes], oxmpl_histogram[successes] + ) + .expect("render repeat histogram"); + } + writeln!( + report, + "\nAt least one of five seeds succeeded on **{in_house_any}/{}** blocked queries in-repository and **{oxmpl_any}/{}** with OxMPL. OxMPL returned **{invalid} invalid nominal paths** after independent checking.\n", + EXPECTED_BLOCKED_QUERIES, EXPECTED_BLOCKED_QUERIES + ) + .expect("render query recovery"); + + report.push_str( + "## Frozen configuration and integrity\n\n\ + - Inputs: query CSV `c07b622...b5452`; in-repository planning CSV `b739009...13055`.\n\ + - OxMPL: crates.io `oxmpl = 0.6.0`, crate archive `11988e3...980dc`, archive VCS commit `6ee31f4...e13`.\n\ + - Both planners use 0.25-rad extensions, 0.05 goal bias, the frozen paired seeds, and the same MuJoCo emitted-contact state predicate.\n\ + - OxMPL 0.6.0 motion sampling was configured to at most 0.05 rad; every returned segment was then rechecked independently at 0.05 rad.\n\ + - OxMPL received a fixed 250-ms wall-time budget; the in-repository artifact used 2,000 iterations. Runtime is intentionally not compared.\n\ + - OxMPL consumes eight-decimal serialized query coordinates; the original in-repository rows used pre-serialization values, a maximum representational difference of `5e-9` rad per coordinate.\n\n", + ); + + report.push_str( + "The exact host, two-run resource measurements, artifact hashes, and \ + deterministic comparison gate are recorded in \ + [`docs/oxmpl_baseline_validation.md`](oxmpl_baseline_validation.md).\n\n", + ); + + report.push_str( + "## Interpretation boundaries\n\n\ + - This is a cross-implementation comparison on one frozen cohort, not evidence that either planner is universally better.\n\ + - Five seeds for one query are repeated algorithm trials, not independent task draws. The retained seed-level McNemar/binomial calculation is descriptive, not a valid population significance test; no population confidence interval is attached.\n\ + - Path lengths are retained in the CSV only as diagnostics. The in-repository path is shortcut and densified; OxMPL returns a raw tree path, so cost is not a fair quality comparison.\n\ + - OxMPL's timeout is wall-clock-bound. Two consecutive runs on the recorded host must agree, but materially different hardware can change borderline timeout outcomes.\n\ + - Seed integers and replicate indices are paired, but arm-lab and OxMPL use different PRNGs; they do not receive identical random samples.\n\ + - Collision checks sample configurations and MuJoCo emitted contacts. They do not certify continuous avoidance or positive clearance.\n\ + - This is deterministic simulation evidence: no hardware, grasp-success, uncertainty, or sim-to-real claim is introduced.\n\n\ + ## Reproduce\n\n\ + ```bash\n\ + cargo run --release -p arm-lab-demo --bin oxmpl_baseline -- --write\n\ + cargo run --release -p arm-lab-demo --bin oxmpl_baseline -- --check\n\ + cargo run --release -p arm-lab-demo --bin oxmpl_baseline -- --verify-artifacts\n\ + ```\n\n\ + `oxmpl_elapsed_ms` is observational wall time and the only field normalized by `--check`.\n", + ); + report +} + +fn percent(numerator: usize, denominator: usize) -> f64 { + 100.0 * numerator as f64 / denominator as f64 +} + +fn write_artifacts(csv: &str, report: &str) { + let docs = docs_dir(); + std::fs::write(docs.join("oxmpl_baseline_results.csv"), csv).expect("write OxMPL result CSV"); + std::fs::write(docs.join("oxmpl_baseline_results.md"), report) + .expect("write OxMPL result report"); + println!("wrote independent OxMPL comparison artifacts"); +} + +fn check_artifacts(generated_csv: &str, generated_report: &str) { + let docs = docs_dir(); + let committed_csv = std::fs::read_to_string(docs.join("oxmpl_baseline_results.csv")) + .expect("read committed OxMPL result CSV"); + let committed_report = std::fs::read_to_string(docs.join("oxmpl_baseline_results.md")) + .expect("read committed OxMPL result report"); + assert_artifact_equal( + "OxMPL deterministic result fields", + &normalize_elapsed(generated_csv), + &normalize_elapsed(&committed_csv), + ); + assert_artifact_equal("OxMPL report", generated_report, &committed_report); + println!("OxMPL pinned-host artifact check passed; only wall time was normalized"); +} + +fn verify_committed_artifacts(inputs: &Inputs) { + let docs = docs_dir(); + let committed_csv = std::fs::read_to_string(docs.join("oxmpl_baseline_results.csv")) + .expect("read committed OxMPL result CSV"); + let committed_report = std::fs::read_to_string(docs.join("oxmpl_baseline_results.md")) + .expect("read committed OxMPL result report"); + let rows = parse_results(&committed_csv); + validate_result_layout(inputs, &rows); + assert_artifact_equal("OxMPL result CSV", &render_csv(&rows), &committed_csv); + assert_artifact_equal( + "OxMPL report derived from CSV", + &render_report(&rows), + &committed_report, + ); + println!("OxMPL frozen input, artifact schema, and report derivation checks passed"); +} + +fn normalize_elapsed(csv: &str) -> String { + let mut lines = csv.lines(); + let header = lines.next().expect("result CSV header"); + let columns: Vec<_> = header.split(',').collect(); + let elapsed_index = columns + .iter() + .position(|column| *column == "oxmpl_elapsed_ms") + .expect("OxMPL elapsed column"); + let mut normalized = format!("{header}\n"); + for line in lines { + let mut fields: Vec<_> = line.split(',').collect(); + assert_eq!(fields.len(), columns.len(), "result CSV column count"); + fields[elapsed_index] = ""; + writeln!(normalized, "{}", fields.join(",")).expect("normalize result row"); + } + normalized +} + +fn assert_artifact_equal(label: &str, generated: &str, committed: &str) { + if generated == committed { + return; + } + let mismatch = generated + .lines() + .zip(committed.lines()) + .position(|(left, right)| left != right) + .map_or_else( + || generated.lines().count().min(committed.lines().count()) + 1, + |index| index + 1, + ); + panic!("stale {label}: first mismatch at line {mismatch}"); +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn parses_blocked_query_and_default_planning_rows() { + let query_csv = "scene,scene_file,query_index,generator_seed,accepted_draw_index,cumulative_rejected_start_collision,cumulative_rejected_goal_collision,cumulative_rejected_short,start_q_rad,goal_q_rad,distance_rad,direct_path_free\nopen_floor,scene.xml,7,1,2,0,0,0,0;1;2;3;4;5,1;2;3;4;5;6,2.0,false\nopen_floor,scene.xml,8,1,3,0,0,0,0;1;2;3;4;5,1;2;3;4;5;6,2.0,true\n"; + let rows = parse_queries(query_csv); + assert_eq!(rows.len(), 1); + assert_eq!(rows[0].query_index, 7); + + let planning_csv = "scene,scene_file,query_index,variant,replicate,seed,direct_path_free,status,plan_elapsed_ms,iterations,nodes,shortcut_waypoints,path_samples,path_cost_rad\nopen_floor,scene.xml,7,default,0,202608180070,false,success,1,2,3,4,5,6\nopen_floor,scene.xml,7,zero_goal_bias,0,202608180070,false,success,1,2,3,4,5,6\n"; + let parsed = parse_in_house(planning_csv); + assert_eq!(parsed.len(), 1); + assert!(parsed[&(0, 7, 0)].success); + } + + #[test] + fn planner_seeds_match_frozen_formula() { + assert_eq!(planner_seed(0, 0, 0), 202_608_180_000); + assert_eq!(planner_seed(1, 99, 4), 202_608_190_994); + assert_eq!(planner_seed(2, 42, 3), 202_608_200_423); + } + + #[test] + fn path_validation_checks_endpoints_bounds_and_edges() { + let bounds = [(-2.0, 2.0), (-2.0, 2.0)]; + let good = vec![vec![0.0, 0.0], vec![0.5, 0.0], vec![1.0, 0.0]]; + assert_eq!( + validate_path(&good, &[0.0, 0.0], &[1.0, 0.0], &bounds, |_| false), + Ok(()) + ); + assert_eq!( + validate_path(&good, &[0.1, 0.0], &[1.0, 0.0], &bounds, |_| false), + Err("start_mismatch".to_string()) + ); + assert_eq!( + validate_path(&good, &[0.0, 0.0], &[1.0, 0.0], &bounds, |q| q[0] > 0.7), + Err("edge_collision".to_string()) + ); + let out_of_bounds = vec![vec![0.0, 0.0], vec![3.0, 0.0]]; + assert_eq!( + validate_path(&out_of_bounds, &[0.0, 0.0], &[3.0, 0.0], &bounds, |_| false), + Err("state_out_of_bounds".to_string()) + ); + } + + #[test] + fn paired_aggregation_retains_all_four_cells() { + let scene = SCENES[0]; + let combinations = [(true, true), (true, false), (false, true), (false, false)]; + let rows = combinations + .into_iter() + .enumerate() + .map(|(index, (in_house, oxmpl))| ResultRow { + scene, + query_index: index, + replicate: 0, + seed: index as u64, + in_house_success: in_house, + oxmpl_status: if oxmpl { + OxmplStatus::Success + } else { + OxmplStatus::Timeout + }, + oxmpl_detail: String::new(), + elapsed_ms: 0.0, + waypoints: 0, + path_cost_rad: 0.0, + path_sha256: String::new(), + }); + let counts = aggregate(rows); + assert_eq!(counts.total, 4); + assert_eq!(counts.in_house_success, 2); + assert_eq!(counts.oxmpl_success, 2); + assert_eq!(counts.both, 1); + assert_eq!(counts.in_house_only, 1); + assert_eq!(counts.oxmpl_only, 1); + assert_eq!(counts.neither, 1); + } + + #[test] + fn exact_mcnemar_matches_small_known_cases() { + assert_eq!(exact_mcnemar_p(0, 0), 1.0); + assert_eq!(exact_mcnemar_p(1, 1), 1.0); + assert!((exact_mcnemar_p(0, 5) - 0.0625).abs() < 1e-12); + assert!((exact_mcnemar_p(2, 8) - 0.109375).abs() < 1e-12); + } + + #[test] + fn elapsed_normalization_changes_only_wall_time() { + let csv = "a,oxmpl_elapsed_ms,b\nx,12.3,y\n"; + assert_eq!( + normalize_elapsed(csv), + "a,oxmpl_elapsed_ms,b\nx,,y\n" + ); + } + + #[test] + #[should_panic(expected = "stale test artifact: first mismatch at line 2")] + fn stale_artifact_is_rejected() { + assert_artifact_equal("test artifact", "header\nnew\n", "header\nold\n"); + } +} diff --git a/docs/oxmpl_baseline_protocol.md b/docs/oxmpl_baseline_protocol.md new file mode 100644 index 0000000..12d4cd7 --- /dev/null +++ b/docs/oxmpl_baseline_protocol.md @@ -0,0 +1,170 @@ +# Independent OxMPL planner comparison protocol + +Frozen before adding the evaluator, its external dependency, or any +result-bearing OxMPL artifact on 2026-08-18 (Europe/Dublin). + +## Purpose + +The existing randomized study compares direct interpolation, the repository's +RRT-Connect implementation, and a goal-bias ablation. This extension adds an +independently maintained implementation of the same planner family so that the +repository's evidence is not limited to self-comparison. + +The comparison is deliberately narrow. It asks whether two seeded +RRT-Connect implementations return independently revalidated paths on the +same frozen blocked-query trials. It is not a universal planner ranking, a +runtime-equivalent benchmark, or an optimality study. + +No query, seed, collision rule, planner parameter, timeout, outcome field, or +promotion gate below may be changed after an OxMPL outcome has been observed. +Every trial and failure will be retained. + +## Frozen inputs + +- Query cohort: `docs/randomized_eval_queries.csv`, SHA-256 + `c07b6227183a55e273120e37c15e86d99f717dc8f47b6a9d3d989d05415b5452`. +- In-repository planning artifact: + `docs/randomized_eval_planning.csv`, SHA-256 + `b739009c0a4b3691bc910f67ad079a944e1b16c83d341bcd078179bda6c13055`. +- Cohort: all 217 queries whose frozen `direct_path_free` field is `false` + across the three shipped scenes. Direct-free queries are excluded because + the in-repository planner returns the shared direct-interpolation solution + before running RRT. +- Replicates: exactly five per query, with the existing seed + `202608180000 + 10000*scene_index + 10*query_index + replicate` for + `replicate = 0..4`; 1,085 paired trials in total. +- In-repository comparator: the already-committed `default` row at the same + scene, query, replicate, and seed. Its configuration remains 2,000 maximum + iterations, 0.25-rad extension length, 0.05-rad edge sampling, 0.05 goal + bias, 40 shortcut attempts, and densified output. + +The existing CSVs are inputs, not regenerated by this study. A hash mismatch +is a hard failure. + +The seed integers and replicate indices are paired, but the random samples are +not: arm-lab and OxMPL use different PRNG implementations. Pairing therefore +blocks on query and deterministic replicate label; it does not claim common +random numbers or sample-by-sample stochastic equivalence. + +The external evaluator consumes the query vectors exactly as serialized to +eight decimal places in the frozen query CSV. The committed in-repository rows +were generated in the original evaluator before that serialization, so any +corresponding coordinate can differ by at most `5e-9` rad. This representational +difference must be disclosed and prevents calling the paired inputs +bit-identical, although it is several orders of magnitude below the 0.05-rad +collision-sampling resolution. + +## Independent implementation + +Use the crates.io package `oxmpl` exactly at version `0.6.0` (BSD-3-Clause): + +- crate archive SHA-256: + `11988e3c34bfcacbc4d96265f1da76f9bd3a30fcc92ad62e4046cc6315e980dc`; +- archive VCS commit: + `6ee31f498a8901d6c612962fb9dba65e70bc2e13`; and +- planner: public type `oxmpl::geometric::RRTConnect`. + +For every trial: + +- construct `RealVectorStateSpace` from the compiled UR5e joint limits with + the same 0.05-rad inset used by the in-repository planner's uniform sampler; +- use the exact frozen start and point goal; +- use the repository's unchanged MuJoCo emitted-contact state predicate; +- set OxMPL maximum extension distance to 0.25 rad and goal bias to 0.05; +- seed OxMPL with the frozen paired seed; +- set the OxMPL longest-valid-segment fraction to `0.5 / extent`, which makes + OxMPL 0.6.0's internal `0.1 * longest_valid_segment_length` motion samples + no farther than 0.05 rad apart; and +- allow 250 ms per `solve` call. + +The 250-ms timeout is fixed before results. It is a bounded operational budget, +not an assertion of equal compute: OxMPL terminates by wall time whereas the +in-repository planner terminates after 2,000 iterations. Wall time, node count, +and iteration count are therefore not comparable headline metrics. + +## Independent result validation + +Treat an OxMPL path as a success only if all of these checks pass: + +1. `solve` returns a non-empty path; +2. every state has six finite values inside the compiled joint limits; +3. the first and last states match the frozen start and goal within + `1e-9`-rad L2 distance; and +4. every returned segment passes the repository's independent `edge_free` + check at 0.05-rad spacing with a fresh collision checker. + +Retain one of these statuses per OxMPL trial: + +- `success`; +- `timeout`; +- `planning_error` for another OxMPL error; or +- `invalid_path` when a nominal solution fails any independent check. + +An invalid path counts as a failed trial. Do not repair, shortcut, or replace +an OxMPL path. Record its returned waypoint count and raw joint-space polyline +length only as diagnostics. Record wall time as observational and +non-byte-stable. + +## Predeclared analysis + +Primary analysis uses the 1,085 paired seed trials: + +- success count and rate for each implementation; +- paired counts: both succeed, in-repository only, OxMPL only, neither; +- exact two-sided McNemar/binomial test on the two discordant counts; and +- status counts by scene. + +Also report the per-query histogram of successes out of five and the number of +blocked queries with at least one success. Five replicates on one query are not +five independent task draws, so do not attach query-distribution confidence +intervals to seed-trial rates or claim that McNemar generalizes beyond this +frozen cohort. + +Path length is diagnostic only. The in-repository path is greedily and +randomly shortcut, while OxMPL returns its raw tree path. Do not headline or +interpret their cost difference as planner quality. Likewise, do not compare +wall times as if the stopping criteria were equal. + +## Artifacts and determinism gate + +Add one bounded `oxmpl_baseline` executable with `--write` and `--check` +modes and commit: + +- `docs/oxmpl_baseline_results.csv` -- every paired trial and retained status; + and +- `docs/oxmpl_baseline_results.md` -- the predeclared aggregates and scope + limitations. + +The executable must test the frozen input hashes before evaluating or checking +artifacts. `--check` must regenerate and compare every deterministic field, +normalizing only OxMPL wall time. Run the full study twice consecutively on the +same pinned host. Any status flip, path-content mismatch, invalid returned +path, or other deterministic-field mismatch fails the promotion gate. A +wall-time-bound baseline can still differ on materially different hardware; +the report must say so even if the pinned-host repeat is exact. + +Unit tests must cover input parsing, seed pairing, endpoint/edge validation, +paired-count aggregation, exact McNemar calculation, time normalization, and +stale-artifact rejection. CI may validate the frozen artifact schema, hashes, +and unit tests without rerunning all wall-time-bound trials; the two exact-host +runs and their environment are the authoritative outcome-reproducibility gate. + +## Promotion and interpretation rules + +- Preserve and report the study whether OxMPL wins, loses, ties, or exposes a + defect in the repository planner. +- No unfavorable outcome permits changing the cohort, seeds, timeout, + configuration, or validation rule. +- Merge only after formatting, strict linting, all workspace tests, evaluator + unit tests, frozen-input and artifact-schema checks, two consecutive + pinned-host outcome-identical runs, and a result-independent source review. +- If a correctness defect appears, retain the raw study on this evidence branch + and fix the defect in a separately declared follow-up rather than silently + rerunning the comparison. +- Public wording must say "cross-implementation comparison on 217 frozen + blocked queries" rather than "better planner" unless the exact scope is + immediately stated. +- The collision predicate samples configurations and MuJoCo emitted contacts; + it does not provide continuous collision avoidance or positive clearance. + This remains deterministic simulation evidence with no hardware, grasp, + uncertainty, or sim-to-real claim. diff --git a/docs/oxmpl_baseline_results.csv b/docs/oxmpl_baseline_results.csv new file mode 100644 index 0000000..115a5dd --- /dev/null +++ b/docs/oxmpl_baseline_results.csv @@ -0,0 +1,1086 @@ +scene,scene_file,query_index,replicate,seed,in_house_status,oxmpl_status,oxmpl_detail,oxmpl_elapsed_ms,oxmpl_waypoints,oxmpl_path_cost_rad,oxmpl_path_sha256 +open_floor,scene.xml,0,0,202608180000,failed,timeout,solve_timeout,250.01317200,0,0.00000000, +open_floor,scene.xml,0,1,202608180001,failed,timeout,solve_timeout,250.00094200,0,0.00000000, +open_floor,scene.xml,0,2,202608180002,failed,timeout,solve_timeout,250.02131300,0,0.00000000, +open_floor,scene.xml,0,3,202608180003,failed,timeout,solve_timeout,250.03468300,0,0.00000000, +open_floor,scene.xml,0,4,202608180004,failed,timeout,solve_timeout,250.04080200,0,0.00000000, +open_floor,scene.xml,1,0,202608180010,failed,timeout,solve_timeout,250.01401200,0,0.00000000, +open_floor,scene.xml,1,1,202608180011,failed,timeout,solve_timeout,250.00467200,0,0.00000000, +open_floor,scene.xml,1,2,202608180012,failed,timeout,solve_timeout,250.00275200,0,0.00000000, +open_floor,scene.xml,1,3,202608180013,failed,timeout,solve_timeout,250.02813200,0,0.00000000, +open_floor,scene.xml,1,4,202608180014,failed,timeout,solve_timeout,250.00032200,0,0.00000000, +open_floor,scene.xml,2,0,202608180020,success,success,validated,7.93579700,105,25.99999992,c8ea317b1631edf7ae23d2518e27d3dcbe14b560c305d14324d003de38498f2c +open_floor,scene.xml,2,1,202608180021,success,success,validated,4.54864200,86,21.22872949,3c4b1950e0d177309c9557fd8d84c12d723015e330688c43bfed70c8813a5b3f +open_floor,scene.xml,2,2,202608180022,success,success,validated,3.31836400,68,16.72832681,68d666eabc9caf21623e53d61efe096810db94660a7794bd2975c9860a0f1069 +open_floor,scene.xml,2,3,202608180023,success,success,validated,5.04372100,85,20.99999790,3d8b523cf321b37ac0fdae1ba3b716ab1e165c048376c0de76f0043a233b2c24 +open_floor,scene.xml,2,4,202608180024,success,success,validated,5.48175100,104,25.75000000,d02192e2dbd0e3e46276cea73e39b1cc2f6dd3bed3780a3eef03ea38dc36d8fb +open_floor,scene.xml,6,0,202608180060,failed,timeout,solve_timeout,250.03649200,0,0.00000000, +open_floor,scene.xml,6,1,202608180061,failed,timeout,solve_timeout,250.01349200,0,0.00000000, +open_floor,scene.xml,6,2,202608180062,failed,timeout,solve_timeout,250.03340300,0,0.00000000, +open_floor,scene.xml,6,3,202608180063,failed,timeout,solve_timeout,250.04228200,0,0.00000000, +open_floor,scene.xml,6,4,202608180064,failed,timeout,solve_timeout,250.04368200,0,0.00000000, +open_floor,scene.xml,7,0,202608180070,failed,timeout,solve_timeout,250.02805200,0,0.00000000, +open_floor,scene.xml,7,1,202608180071,failed,timeout,solve_timeout,250.02185200,0,0.00000000, +open_floor,scene.xml,7,2,202608180072,failed,timeout,solve_timeout,250.01846100,0,0.00000000, +open_floor,scene.xml,7,3,202608180073,failed,timeout,solve_timeout,250.06399200,0,0.00000000, +open_floor,scene.xml,7,4,202608180074,failed,timeout,solve_timeout,250.00834200,0,0.00000000, +open_floor,scene.xml,8,0,202608180080,failed,timeout,solve_timeout,250.01909200,0,0.00000000, +open_floor,scene.xml,8,1,202608180081,failed,timeout,solve_timeout,250.01544200,0,0.00000000, +open_floor,scene.xml,8,2,202608180082,failed,timeout,solve_timeout,250.02958200,0,0.00000000, +open_floor,scene.xml,8,3,202608180083,failed,timeout,solve_timeout,250.00808200,0,0.00000000, +open_floor,scene.xml,8,4,202608180084,failed,timeout,solve_timeout,250.02303200,0,0.00000000, +open_floor,scene.xml,9,0,202608180090,failed,timeout,solve_timeout,250.00566300,0,0.00000000, +open_floor,scene.xml,9,1,202608180091,failed,timeout,solve_timeout,250.05174300,0,0.00000000, +open_floor,scene.xml,9,2,202608180092,failed,timeout,solve_timeout,250.01515200,0,0.00000000, +open_floor,scene.xml,9,3,202608180093,failed,timeout,solve_timeout,250.02236200,0,0.00000000, +open_floor,scene.xml,9,4,202608180094,failed,timeout,solve_timeout,250.03608200,0,0.00000000, +open_floor,scene.xml,11,0,202608180110,failed,timeout,solve_timeout,250.00467200,0,0.00000000, +open_floor,scene.xml,11,1,202608180111,failed,timeout,solve_timeout,250.00517200,0,0.00000000, +open_floor,scene.xml,11,2,202608180112,failed,timeout,solve_timeout,250.04771200,0,0.00000000, +open_floor,scene.xml,11,3,202608180113,failed,timeout,solve_timeout,250.01460200,0,0.00000000, +open_floor,scene.xml,11,4,202608180114,failed,timeout,solve_timeout,250.03536200,0,0.00000000, +open_floor,scene.xml,12,0,202608180120,success,success,validated,1.87710700,47,11.39973281,3b836c29035488aa3838827f6cfa8d313d2f050cfaa3a68e3027391883247370 +open_floor,scene.xml,12,1,202608180121,success,success,validated,12.12943900,57,13.96637315,75a29504aa89f08ad9a0af2ac94707aa34b26a9d1f136a8135dd93fc19026083 +open_floor,scene.xml,12,2,202608180122,success,success,validated,11.23238200,115,28.50000000,ef12da05f4af968d19c8077c31adada12573959c7d2f5c7249304bc06150ca28 +open_floor,scene.xml,12,3,202608180123,success,success,validated,5.04470100,94,23.24999997,4f9fa0dfe3d8c58c141d98a2dcfccf0423641f91ba2ace889a6f5c7014e9a897 +open_floor,scene.xml,12,4,202608180124,success,success,validated,2.84903600,50,12.22376741,0f44e6af91841ab7eb996504fdc12467c7acc46916b8039337562aece0fcc1bf +open_floor,scene.xml,13,0,202608180130,failed,timeout,solve_timeout,250.02186200,0,0.00000000, +open_floor,scene.xml,13,1,202608180131,failed,timeout,solve_timeout,250.00263200,0,0.00000000, +open_floor,scene.xml,13,2,202608180132,failed,timeout,solve_timeout,250.02310200,0,0.00000000, +open_floor,scene.xml,13,3,202608180133,failed,timeout,solve_timeout,250.02661100,0,0.00000000, +open_floor,scene.xml,13,4,202608180134,failed,timeout,solve_timeout,250.04041200,0,0.00000000, +open_floor,scene.xml,14,0,202608180140,failed,timeout,solve_timeout,250.04105200,0,0.00000000, +open_floor,scene.xml,14,1,202608180141,failed,timeout,solve_timeout,250.02124200,0,0.00000000, +open_floor,scene.xml,14,2,202608180142,failed,timeout,solve_timeout,250.03915200,0,0.00000000, +open_floor,scene.xml,14,3,202608180143,failed,timeout,solve_timeout,250.00454200,0,0.00000000, +open_floor,scene.xml,14,4,202608180144,failed,timeout,solve_timeout,250.03001200,0,0.00000000, +open_floor,scene.xml,15,0,202608180150,success,success,validated,19.12686800,151,37.50000000,7ebd9318d77d0422bea4ec23b3303dfe32940a461e27403bfa99f6558c5c40ed +open_floor,scene.xml,15,1,202608180151,success,success,validated,12.26064900,138,34.25000000,50bd3d65303665f1704a5971438db34811eb42cebcab5f475aec6bd5e815747c +open_floor,scene.xml,15,2,202608180152,success,success,validated,10.28760200,132,32.75000000,e8dfca7bbb745dcf6d95ba83f87bfd60c3b5265b8eae9e0b10bb63153e24329d +open_floor,scene.xml,15,3,202608180153,success,success,validated,2.72858500,68,16.70204445,960cea399860583dd838223fd93e1d90b4bcbae54e7ff3f608bd7d759aac4144 +open_floor,scene.xml,15,4,202608180154,success,success,validated,3.22475500,65,15.93692591,d6742b9b8b027c5980f9fd071dc3336b62598b6414cd6dd08511e2501fc4a805 +open_floor,scene.xml,16,0,202608180160,failed,timeout,solve_timeout,250.01942200,0,0.00000000, +open_floor,scene.xml,16,1,202608180161,failed,timeout,solve_timeout,250.00604300,0,0.00000000, +open_floor,scene.xml,16,2,202608180162,failed,timeout,solve_timeout,250.04989200,0,0.00000000, +open_floor,scene.xml,16,3,202608180163,failed,timeout,solve_timeout,250.03266200,0,0.00000000, +open_floor,scene.xml,16,4,202608180164,failed,timeout,solve_timeout,250.01252200,0,0.00000000, +open_floor,scene.xml,17,0,202608180170,failed,timeout,solve_timeout,250.01333200,0,0.00000000, +open_floor,scene.xml,17,1,202608180171,failed,timeout,solve_timeout,250.02946200,0,0.00000000, +open_floor,scene.xml,17,2,202608180172,failed,timeout,solve_timeout,250.05168200,0,0.00000000, +open_floor,scene.xml,17,3,202608180173,failed,timeout,solve_timeout,250.00066300,0,0.00000000, +open_floor,scene.xml,17,4,202608180174,failed,timeout,solve_timeout,250.01957200,0,0.00000000, +open_floor,scene.xml,18,0,202608180180,failed,timeout,solve_timeout,250.03567200,0,0.00000000, +open_floor,scene.xml,18,1,202608180181,failed,timeout,solve_timeout,250.03293200,0,0.00000000, +open_floor,scene.xml,18,2,202608180182,failed,timeout,solve_timeout,250.03444200,0,0.00000000, +open_floor,scene.xml,18,3,202608180183,failed,timeout,solve_timeout,250.03373200,0,0.00000000, +open_floor,scene.xml,18,4,202608180184,failed,timeout,solve_timeout,250.04999200,0,0.00000000, +open_floor,scene.xml,20,0,202608180200,failed,timeout,solve_timeout,250.01668200,0,0.00000000, +open_floor,scene.xml,20,1,202608180201,failed,timeout,solve_timeout,250.03712200,0,0.00000000, +open_floor,scene.xml,20,2,202608180202,failed,timeout,solve_timeout,250.03578200,0,0.00000000, +open_floor,scene.xml,20,3,202608180203,failed,timeout,solve_timeout,250.01645200,0,0.00000000, +open_floor,scene.xml,20,4,202608180204,failed,timeout,solve_timeout,250.01265200,0,0.00000000, +open_floor,scene.xml,21,0,202608180210,failed,timeout,solve_timeout,250.00288200,0,0.00000000, +open_floor,scene.xml,21,1,202608180211,failed,timeout,solve_timeout,250.00066200,0,0.00000000, +open_floor,scene.xml,21,2,202608180212,failed,timeout,solve_timeout,250.00575200,0,0.00000000, +open_floor,scene.xml,21,3,202608180213,failed,timeout,solve_timeout,250.01188200,0,0.00000000, +open_floor,scene.xml,21,4,202608180214,failed,timeout,solve_timeout,250.00880200,0,0.00000000, +open_floor,scene.xml,22,0,202608180220,success,success,validated,4.04107300,73,17.97878974,dd8bc9936ab6ecb0cae8359424777e28e349a7e56364cc04a02f9a49bba45fd1 +open_floor,scene.xml,22,1,202608180221,success,success,validated,7.61533700,102,25.24999957,54dfd4dadadaf5e0134ef3e5c5b64f31268dd5fe0dfa0989485bfde7efabc5d2 +open_floor,scene.xml,22,2,202608180222,success,success,validated,14.47997600,97,23.96477017,7b891a60b308beb4367b46ebf1127092a15a326d7d740e09b7a6db05af4e01ff +open_floor,scene.xml,22,3,202608180223,success,success,validated,13.06386800,112,27.73187971,851ce5d81fa047f43a83043c996f70657c4648675df4a72231f3de7147cdcb3c +open_floor,scene.xml,22,4,202608180224,success,success,validated,4.42750300,106,26.24992146,b1f2f503111220e2af1a1392b9dc2b3934131f77183d89d106b99e9836f06af2 +open_floor,scene.xml,23,0,202608180230,failed,timeout,solve_timeout,250.00374200,0,0.00000000, +open_floor,scene.xml,23,1,202608180231,failed,timeout,solve_timeout,250.01639200,0,0.00000000, +open_floor,scene.xml,23,2,202608180232,failed,timeout,solve_timeout,250.00192200,0,0.00000000, +open_floor,scene.xml,23,3,202608180233,failed,timeout,solve_timeout,250.02243200,0,0.00000000, +open_floor,scene.xml,23,4,202608180234,failed,timeout,solve_timeout,250.00687100,0,0.00000000, +open_floor,scene.xml,24,0,202608180240,failed,timeout,solve_timeout,250.03184200,0,0.00000000, +open_floor,scene.xml,24,1,202608180241,failed,timeout,solve_timeout,250.01485200,0,0.00000000, +open_floor,scene.xml,24,2,202608180242,failed,timeout,solve_timeout,250.00871200,0,0.00000000, +open_floor,scene.xml,24,3,202608180243,failed,timeout,solve_timeout,250.04562200,0,0.00000000, +open_floor,scene.xml,24,4,202608180244,failed,timeout,solve_timeout,250.03736200,0,0.00000000, +open_floor,scene.xml,25,0,202608180250,failed,timeout,solve_timeout,250.02349200,0,0.00000000, +open_floor,scene.xml,25,1,202608180251,failed,timeout,solve_timeout,250.04298200,0,0.00000000, +open_floor,scene.xml,25,2,202608180252,failed,timeout,solve_timeout,250.04919200,0,0.00000000, +open_floor,scene.xml,25,3,202608180253,failed,timeout,solve_timeout,250.03743200,0,0.00000000, +open_floor,scene.xml,25,4,202608180254,failed,timeout,solve_timeout,250.06437200,0,0.00000000, +open_floor,scene.xml,26,0,202608180260,failed,timeout,solve_timeout,250.05268200,0,0.00000000, +open_floor,scene.xml,26,1,202608180261,failed,timeout,solve_timeout,250.02449200,0,0.00000000, +open_floor,scene.xml,26,2,202608180262,failed,timeout,solve_timeout,250.04927200,0,0.00000000, +open_floor,scene.xml,26,3,202608180263,failed,timeout,solve_timeout,250.01859200,0,0.00000000, +open_floor,scene.xml,26,4,202608180264,failed,timeout,solve_timeout,250.04484200,0,0.00000000, +open_floor,scene.xml,27,0,202608180270,failed,timeout,solve_timeout,250.01433300,0,0.00000000, +open_floor,scene.xml,27,1,202608180271,failed,timeout,solve_timeout,250.00972200,0,0.00000000, +open_floor,scene.xml,27,2,202608180272,failed,timeout,solve_timeout,250.00059200,0,0.00000000, +open_floor,scene.xml,27,3,202608180273,failed,timeout,solve_timeout,250.01358200,0,0.00000000, +open_floor,scene.xml,27,4,202608180274,failed,timeout,solve_timeout,250.03577200,0,0.00000000, +open_floor,scene.xml,28,0,202608180280,failed,timeout,solve_timeout,250.02327200,0,0.00000000, +open_floor,scene.xml,28,1,202608180281,failed,timeout,solve_timeout,250.00974200,0,0.00000000, +open_floor,scene.xml,28,2,202608180282,failed,timeout,solve_timeout,250.01374200,0,0.00000000, +open_floor,scene.xml,28,3,202608180283,failed,timeout,solve_timeout,250.04368200,0,0.00000000, +open_floor,scene.xml,28,4,202608180284,failed,timeout,solve_timeout,250.00188200,0,0.00000000, +open_floor,scene.xml,29,0,202608180290,success,success,validated,6.42756900,52,12.37792760,00447d3d129575f639eba0f8e80274300d80c5fa5c7643c170f87639c220bddf +open_floor,scene.xml,29,1,202608180291,success,success,validated,3.48247500,83,20.50000000,18c4721df8e50b3825d821457f68afbb2b9db16e181d912620ebb10da08784ab +open_floor,scene.xml,29,2,202608180292,success,success,validated,12.12050000,126,31.25000000,60bc91a7aacc70f0577f5228d1019c808b02bbc3071d617c897cd1b4cff5a71c +open_floor,scene.xml,29,3,202608180293,success,success,validated,8.19395700,106,26.25000000,f7101792b9bd2c9771c953edc5d335fbc4f658392da3cddbfd9ad78289c832b1 +open_floor,scene.xml,29,4,202608180294,success,success,validated,11.18910100,76,18.74999694,78f4f1cf9280b8bc1e9e66480c33d8625119300dd3f3f6937c5691de8f19a962 +open_floor,scene.xml,30,0,202608180300,failed,timeout,solve_timeout,250.01605200,0,0.00000000, +open_floor,scene.xml,30,1,202608180301,failed,timeout,solve_timeout,250.00705200,0,0.00000000, +open_floor,scene.xml,30,2,202608180302,failed,timeout,solve_timeout,250.05032200,0,0.00000000, +open_floor,scene.xml,30,3,202608180303,failed,timeout,solve_timeout,250.02995200,0,0.00000000, +open_floor,scene.xml,30,4,202608180304,failed,timeout,solve_timeout,250.03551200,0,0.00000000, +open_floor,scene.xml,36,0,202608180360,failed,timeout,solve_timeout,250.00365200,0,0.00000000, +open_floor,scene.xml,36,1,202608180361,failed,timeout,solve_timeout,250.06902200,0,0.00000000, +open_floor,scene.xml,36,2,202608180362,failed,timeout,solve_timeout,250.00046200,0,0.00000000, +open_floor,scene.xml,36,3,202608180363,failed,timeout,solve_timeout,250.04122100,0,0.00000000, +open_floor,scene.xml,36,4,202608180364,failed,timeout,solve_timeout,250.03410200,0,0.00000000, +open_floor,scene.xml,40,0,202608180400,failed,timeout,solve_timeout,250.01006200,0,0.00000000, +open_floor,scene.xml,40,1,202608180401,failed,timeout,solve_timeout,250.02034200,0,0.00000000, +open_floor,scene.xml,40,2,202608180402,failed,timeout,solve_timeout,250.00899200,0,0.00000000, +open_floor,scene.xml,40,3,202608180403,failed,timeout,solve_timeout,250.03254200,0,0.00000000, +open_floor,scene.xml,40,4,202608180404,failed,timeout,solve_timeout,250.01693200,0,0.00000000, +open_floor,scene.xml,42,0,202608180420,failed,timeout,solve_timeout,250.04235200,0,0.00000000, +open_floor,scene.xml,42,1,202608180421,failed,timeout,solve_timeout,250.01909200,0,0.00000000, +open_floor,scene.xml,42,2,202608180422,failed,timeout,solve_timeout,250.00993200,0,0.00000000, +open_floor,scene.xml,42,3,202608180423,failed,timeout,solve_timeout,250.06183200,0,0.00000000, +open_floor,scene.xml,42,4,202608180424,failed,timeout,solve_timeout,250.04650200,0,0.00000000, +open_floor,scene.xml,43,0,202608180430,success,success,validated,9.73754400,128,31.75000000,348068a7b6c3e45d18e80ab57a88ea339eb7714f68ec3803fa4560cb5c647508 +open_floor,scene.xml,43,1,202608180431,success,success,validated,24.08187000,130,32.25000000,11859f5ef5bfecaf76825fca32d6f4f8920b4a024aa98ebcb2d48530440d49e8 +open_floor,scene.xml,43,2,202608180432,success,success,validated,6.27543000,81,19.99999859,05875a71b9c2c3d3c9d2f75c27034e45781d4aa450bb0538070c6cf5eaf1324d +open_floor,scene.xml,43,3,202608180433,success,success,validated,6.93014800,111,27.50000000,6131d7eb488278f8390e1446c96b223333947e8bcd71f9aea445e173067beb44 +open_floor,scene.xml,43,4,202608180434,success,success,validated,3.04034500,65,15.99152433,91ad334123d2edb6589ef58b4ac3b8d886b0d2765a29d7e0d45e91317b9e7eb8 +open_floor,scene.xml,44,0,202608180440,success,success,validated,7.76773700,128,31.75000000,cc747fc34b6a4ea25d5b2a99be9e24a2144f972796c88540f792197376718382 +open_floor,scene.xml,44,1,202608180441,success,success,validated,7.75661700,133,33.00000000,b36615831f86dec9997fc28ea3f63f636c29852e794037377a69cab20ef3a7ae +open_floor,scene.xml,44,2,202608180442,success,success,validated,4.00261300,101,24.99999265,635e066710c560c96a039f4c42657dc1415281e8d447fd6b645ae6ceac665cf8 +open_floor,scene.xml,44,3,202608180443,success,success,validated,15.20409500,135,33.50000000,cb7379b109324104a48c03e13339332bb71db27c51299eb65cf2c97628ece826 +open_floor,scene.xml,44,4,202608180444,success,success,validated,12.50309000,149,37.00000000,c64b0d9e56ba463b2a16dcd38aa2eaa56d94e7c6408df1dd19109ed099095ecd +open_floor,scene.xml,45,0,202608180450,failed,timeout,solve_timeout,250.01745200,0,0.00000000, +open_floor,scene.xml,45,1,202608180451,failed,timeout,solve_timeout,250.00338200,0,0.00000000, +open_floor,scene.xml,45,2,202608180452,failed,timeout,solve_timeout,250.03510200,0,0.00000000, +open_floor,scene.xml,45,3,202608180453,failed,timeout,solve_timeout,250.03108200,0,0.00000000, +open_floor,scene.xml,45,4,202608180454,failed,timeout,solve_timeout,250.03527200,0,0.00000000, +open_floor,scene.xml,46,0,202608180460,failed,timeout,solve_timeout,250.03380300,0,0.00000000, +open_floor,scene.xml,46,1,202608180461,failed,timeout,solve_timeout,250.00271200,0,0.00000000, +open_floor,scene.xml,46,2,202608180462,failed,timeout,solve_timeout,250.00960200,0,0.00000000, +open_floor,scene.xml,46,3,202608180463,failed,timeout,solve_timeout,250.00290200,0,0.00000000, +open_floor,scene.xml,46,4,202608180464,failed,timeout,solve_timeout,250.00770200,0,0.00000000, +open_floor,scene.xml,47,0,202608180470,failed,timeout,solve_timeout,250.04221200,0,0.00000000, +open_floor,scene.xml,47,1,202608180471,failed,timeout,solve_timeout,250.02405200,0,0.00000000, +open_floor,scene.xml,47,2,202608180472,failed,timeout,solve_timeout,250.00587200,0,0.00000000, +open_floor,scene.xml,47,3,202608180473,failed,timeout,solve_timeout,250.00778200,0,0.00000000, +open_floor,scene.xml,47,4,202608180474,failed,timeout,solve_timeout,250.00406200,0,0.00000000, +open_floor,scene.xml,48,0,202608180480,failed,timeout,solve_timeout,250.00528200,0,0.00000000, +open_floor,scene.xml,48,1,202608180481,failed,timeout,solve_timeout,250.00786200,0,0.00000000, +open_floor,scene.xml,48,2,202608180482,failed,timeout,solve_timeout,250.02978200,0,0.00000000, +open_floor,scene.xml,48,3,202608180483,failed,timeout,solve_timeout,250.01436300,0,0.00000000, +open_floor,scene.xml,48,4,202608180484,failed,timeout,solve_timeout,250.00957300,0,0.00000000, +open_floor,scene.xml,49,0,202608180490,failed,timeout,solve_timeout,250.02331200,0,0.00000000, +open_floor,scene.xml,49,1,202608180491,failed,timeout,solve_timeout,250.02621200,0,0.00000000, +open_floor,scene.xml,49,2,202608180492,failed,timeout,solve_timeout,250.04927200,0,0.00000000, +open_floor,scene.xml,49,3,202608180493,failed,timeout,solve_timeout,250.03843200,0,0.00000000, +open_floor,scene.xml,49,4,202608180494,failed,timeout,solve_timeout,250.04172200,0,0.00000000, +open_floor,scene.xml,50,0,202608180500,success,success,validated,4.48660300,62,15.24561601,f1729eabdf0eaa86108814151bfcd309897190f3067615d63044ae67b5262f63 +open_floor,scene.xml,50,1,202608180501,success,success,validated,5.43054100,64,15.65443472,b0e04fae6fad0f44bfc32a05d9a4ccfba88c5e2a2ffa378e90a51309744a1ca3 +open_floor,scene.xml,50,2,202608180502,success,success,validated,4.40528200,68,16.72828734,1e0f40f96c92a61f89bee7d899cff288f0a21dd7c2c389492a3f051dd2997ba1 +open_floor,scene.xml,50,3,202608180503,success,success,validated,9.08767500,85,20.99882840,91d47a8e5a78005e18dd27125cbbd6bbc314ec162c1e6fe47b13f64ced67ca54 +open_floor,scene.xml,50,4,202608180504,success,success,validated,7.05095800,89,21.99999997,cb8017ea51b858ea050e87fe766536b29e8826cada10d0c7e4a3ea33f614389a +open_floor,scene.xml,51,0,202608180510,failed,timeout,solve_timeout,250.04318200,0,0.00000000, +open_floor,scene.xml,51,1,202608180511,failed,timeout,solve_timeout,250.06132200,0,0.00000000, +open_floor,scene.xml,51,2,202608180512,failed,timeout,solve_timeout,250.01742200,0,0.00000000, +open_floor,scene.xml,51,3,202608180513,failed,timeout,solve_timeout,250.06115200,0,0.00000000, +open_floor,scene.xml,51,4,202608180514,failed,timeout,solve_timeout,250.05273200,0,0.00000000, +open_floor,scene.xml,52,0,202608180520,failed,timeout,solve_timeout,250.02758200,0,0.00000000, +open_floor,scene.xml,52,1,202608180521,failed,timeout,solve_timeout,250.01605200,0,0.00000000, +open_floor,scene.xml,52,2,202608180522,failed,timeout,solve_timeout,250.02482300,0,0.00000000, +open_floor,scene.xml,52,3,202608180523,failed,timeout,solve_timeout,250.01706200,0,0.00000000, +open_floor,scene.xml,52,4,202608180524,failed,timeout,solve_timeout,250.02991200,0,0.00000000, +open_floor,scene.xml,56,0,202608180560,success,success,validated,18.40354900,88,21.73124645,c75a8a4db3e0ac0eab841e5e8ba6665922f311d3813971bfb129cd0891bbbd93 +open_floor,scene.xml,56,1,202608180561,failed,success,validated,21.16755500,133,32.99999990,b2f01def4cab0533ffb7cde716727619cd055ce9958bcc992780203bf3a55194 +open_floor,scene.xml,56,2,202608180562,failed,success,validated,8.31192700,68,16.73870613,3300be9fddcb2521d77286c31b02bd4cdafccab5abe2de59f39d6d897669ed2f +open_floor,scene.xml,56,3,202608180563,failed,success,validated,24.09706000,99,24.49749763,5298e3ee0a1727369a6214833b4e0839c78852f58ff462b41aacb2b503410726 +open_floor,scene.xml,56,4,202608180564,success,success,validated,11.16433100,84,20.74014884,2fc660a3da928ed8fcbc6479f7e7ecd103609fb3e7b996a84b7f21c7ae66cc92 +open_floor,scene.xml,57,0,202608180570,failed,timeout,solve_timeout,250.00190300,0,0.00000000, +open_floor,scene.xml,57,1,202608180571,failed,timeout,solve_timeout,250.03005200,0,0.00000000, +open_floor,scene.xml,57,2,202608180572,failed,timeout,solve_timeout,250.04940200,0,0.00000000, +open_floor,scene.xml,57,3,202608180573,failed,timeout,solve_timeout,250.00552200,0,0.00000000, +open_floor,scene.xml,57,4,202608180574,failed,timeout,solve_timeout,250.01652200,0,0.00000000, +open_floor,scene.xml,58,0,202608180580,failed,timeout,solve_timeout,250.00158200,0,0.00000000, +open_floor,scene.xml,58,1,202608180581,failed,timeout,solve_timeout,250.01899200,0,0.00000000, +open_floor,scene.xml,58,2,202608180582,failed,timeout,solve_timeout,250.01752200,0,0.00000000, +open_floor,scene.xml,58,3,202608180583,failed,timeout,solve_timeout,250.00062200,0,0.00000000, +open_floor,scene.xml,58,4,202608180584,failed,timeout,solve_timeout,250.06320200,0,0.00000000, +open_floor,scene.xml,59,0,202608180590,failed,timeout,solve_timeout,250.02597200,0,0.00000000, +open_floor,scene.xml,59,1,202608180591,failed,timeout,solve_timeout,250.03178200,0,0.00000000, +open_floor,scene.xml,59,2,202608180592,failed,timeout,solve_timeout,250.00967200,0,0.00000000, +open_floor,scene.xml,59,3,202608180593,failed,timeout,solve_timeout,250.04523200,0,0.00000000, +open_floor,scene.xml,59,4,202608180594,failed,timeout,solve_timeout,250.01875200,0,0.00000000, +open_floor,scene.xml,60,0,202608180600,success,success,validated,7.04667800,109,27.00000000,78218fc09ce0b9d1144f0e23fe2e6ae5d82e4ba08008e3dfc1fd863a394045a6 +open_floor,scene.xml,60,1,202608180601,success,success,validated,9.62401400,106,26.25000000,16c6c21be39d83ddd92e2a5fdfec06d91f2d5622ea0cc1dbe3a9560cddc557e5 +open_floor,scene.xml,60,2,202608180602,success,success,validated,8.14802600,140,34.75000000,098a194939c3f20be0c4837285405f395ac72c1351d5470170489893e02ceba7 +open_floor,scene.xml,60,3,202608180603,success,success,validated,3.52315400,75,18.49999913,74a47f76bfd50a3f1ed31daa1dff7831d8f8d977834acbd288dafba5d7a3bbab +open_floor,scene.xml,60,4,202608180604,success,success,validated,13.45754800,137,34.00000000,fb5f81fcb49c33728f7e47ba31b0513346d25b89079eca868fee0b7de49560a1 +open_floor,scene.xml,61,0,202608180610,success,success,validated,2.58460600,38,9.17871628,8248c185d81025f188c8f93ad7e77b48602d4d8da6933b01859daeac36ba4d8e +open_floor,scene.xml,61,1,202608180611,success,success,validated,4.52331300,39,9.16970794,4fdf2524565e8ecf5233ef4022c43cf763e3462be273a3cb0ac0f2bc703eea7f +open_floor,scene.xml,61,2,202608180612,success,success,validated,3.19645400,54,13.25000000,9247c84f3f726fc7edaef2d1bddc77b193f15520d17824b70b4d8f984cc441fb +open_floor,scene.xml,61,3,202608180613,success,success,validated,4.51135200,44,10.42828896,87561a4178168c5654191939b6ea2867c337b80726164f5811049de76563d33e +open_floor,scene.xml,61,4,202608180614,success,success,validated,4.30673300,38,8.92275263,1b07ec660dbdc72415c7a52f2d9c1bf9cdeda29a6bb94698e262cddd75d098af +open_floor,scene.xml,63,0,202608180630,failed,timeout,solve_timeout,250.06217200,0,0.00000000, +open_floor,scene.xml,63,1,202608180631,failed,timeout,solve_timeout,250.02050200,0,0.00000000, +open_floor,scene.xml,63,2,202608180632,failed,timeout,solve_timeout,250.01635200,0,0.00000000, +open_floor,scene.xml,63,3,202608180633,failed,timeout,solve_timeout,250.02277200,0,0.00000000, +open_floor,scene.xml,63,4,202608180634,failed,timeout,solve_timeout,250.00138200,0,0.00000000, +open_floor,scene.xml,64,0,202608180640,failed,timeout,solve_timeout,250.05259300,0,0.00000000, +open_floor,scene.xml,64,1,202608180641,failed,timeout,solve_timeout,250.00153200,0,0.00000000, +open_floor,scene.xml,64,2,202608180642,failed,timeout,solve_timeout,250.01761200,0,0.00000000, +open_floor,scene.xml,64,3,202608180643,failed,timeout,solve_timeout,250.02164200,0,0.00000000, +open_floor,scene.xml,64,4,202608180644,failed,timeout,solve_timeout,250.03935200,0,0.00000000, +open_floor,scene.xml,67,0,202608180670,failed,timeout,solve_timeout,250.01320200,0,0.00000000, +open_floor,scene.xml,67,1,202608180671,failed,timeout,solve_timeout,250.01099200,0,0.00000000, +open_floor,scene.xml,67,2,202608180672,failed,timeout,solve_timeout,250.02160200,0,0.00000000, +open_floor,scene.xml,67,3,202608180673,failed,timeout,solve_timeout,250.03261200,0,0.00000000, +open_floor,scene.xml,67,4,202608180674,failed,timeout,solve_timeout,250.01659200,0,0.00000000, +open_floor,scene.xml,68,0,202608180680,failed,timeout,solve_timeout,250.00944200,0,0.00000000, +open_floor,scene.xml,68,1,202608180681,failed,timeout,solve_timeout,250.01178200,0,0.00000000, +open_floor,scene.xml,68,2,202608180682,failed,timeout,solve_timeout,250.07278200,0,0.00000000, +open_floor,scene.xml,68,3,202608180683,failed,timeout,solve_timeout,250.02245200,0,0.00000000, +open_floor,scene.xml,68,4,202608180684,failed,timeout,solve_timeout,250.01159200,0,0.00000000, +open_floor,scene.xml,70,0,202608180700,failed,timeout,solve_timeout,250.04809100,0,0.00000000, +open_floor,scene.xml,70,1,202608180701,failed,timeout,solve_timeout,250.00132200,0,0.00000000, +open_floor,scene.xml,70,2,202608180702,failed,timeout,solve_timeout,250.02205200,0,0.00000000, +open_floor,scene.xml,70,3,202608180703,failed,timeout,solve_timeout,250.02296200,0,0.00000000, +open_floor,scene.xml,70,4,202608180704,failed,timeout,solve_timeout,250.03388200,0,0.00000000, +open_floor,scene.xml,74,0,202608180740,failed,timeout,solve_timeout,250.02710200,0,0.00000000, +open_floor,scene.xml,74,1,202608180741,failed,timeout,solve_timeout,250.00557200,0,0.00000000, +open_floor,scene.xml,74,2,202608180742,failed,timeout,solve_timeout,250.07701200,0,0.00000000, +open_floor,scene.xml,74,3,202608180743,failed,timeout,solve_timeout,250.01451200,0,0.00000000, +open_floor,scene.xml,74,4,202608180744,failed,timeout,solve_timeout,250.05411200,0,0.00000000, +open_floor,scene.xml,76,0,202608180760,failed,timeout,solve_timeout,250.01456300,0,0.00000000, +open_floor,scene.xml,76,1,202608180761,failed,timeout,solve_timeout,250.01370200,0,0.00000000, +open_floor,scene.xml,76,2,202608180762,failed,timeout,solve_timeout,250.03252200,0,0.00000000, +open_floor,scene.xml,76,3,202608180763,failed,timeout,solve_timeout,250.01363200,0,0.00000000, +open_floor,scene.xml,76,4,202608180764,failed,timeout,solve_timeout,250.01127200,0,0.00000000, +open_floor,scene.xml,78,0,202608180780,success,success,validated,7.60375700,105,25.99999969,0d3fb48548abd3f051a503ea6351b84d4fc06422f709a19cda2be5247948bd5c +open_floor,scene.xml,78,1,202608180781,success,success,validated,14.01726700,145,36.00000000,af98fdb7434f618b3215b41a778cf735644f60abd7d606cff5c243811789fa02 +open_floor,scene.xml,78,2,202608180782,success,success,validated,11.84146000,138,34.25000000,27c444350410ac5dc3717e111f0255f1fe8b943174469fd729b2da4591b88490 +open_floor,scene.xml,78,3,202608180783,success,success,validated,6.71170900,88,21.75000000,7f0550834aad62f36a69d59ab48337ba9e976f6e6ca6635cdc5e3864db73f4ad +open_floor,scene.xml,78,4,202608180784,success,success,validated,11.80498000,124,30.75000000,312294e71a5288d11ecb3c59d9b3d26ea8d94851d88c3791a431ed85642c1d76 +open_floor,scene.xml,80,0,202608180800,failed,timeout,solve_timeout,250.03455200,0,0.00000000, +open_floor,scene.xml,80,1,202608180801,failed,timeout,solve_timeout,250.00523200,0,0.00000000, +open_floor,scene.xml,80,2,202608180802,failed,timeout,solve_timeout,250.05121200,0,0.00000000, +open_floor,scene.xml,80,3,202608180803,failed,timeout,solve_timeout,250.02304200,0,0.00000000, +open_floor,scene.xml,80,4,202608180804,failed,timeout,solve_timeout,250.00103200,0,0.00000000, +open_floor,scene.xml,83,0,202608180830,failed,timeout,solve_timeout,250.05640200,0,0.00000000, +open_floor,scene.xml,83,1,202608180831,failed,timeout,solve_timeout,250.03936200,0,0.00000000, +open_floor,scene.xml,83,2,202608180832,failed,timeout,solve_timeout,250.00048200,0,0.00000000, +open_floor,scene.xml,83,3,202608180833,failed,timeout,solve_timeout,250.03001200,0,0.00000000, +open_floor,scene.xml,83,4,202608180834,failed,timeout,solve_timeout,250.00032200,0,0.00000000, +open_floor,scene.xml,84,0,202608180840,failed,timeout,solve_timeout,250.00048200,0,0.00000000, +open_floor,scene.xml,84,1,202608180841,failed,timeout,solve_timeout,250.04356200,0,0.00000000, +open_floor,scene.xml,84,2,202608180842,failed,timeout,solve_timeout,250.00258200,0,0.00000000, +open_floor,scene.xml,84,3,202608180843,failed,timeout,solve_timeout,250.00899200,0,0.00000000, +open_floor,scene.xml,84,4,202608180844,failed,timeout,solve_timeout,250.02018200,0,0.00000000, +open_floor,scene.xml,85,0,202608180850,failed,timeout,solve_timeout,250.03227200,0,0.00000000, +open_floor,scene.xml,85,1,202608180851,failed,timeout,solve_timeout,250.02323200,0,0.00000000, +open_floor,scene.xml,85,2,202608180852,failed,timeout,solve_timeout,250.02535200,0,0.00000000, +open_floor,scene.xml,85,3,202608180853,failed,timeout,solve_timeout,250.00910200,0,0.00000000, +open_floor,scene.xml,85,4,202608180854,failed,timeout,solve_timeout,250.00431200,0,0.00000000, +open_floor,scene.xml,86,0,202608180860,failed,timeout,solve_timeout,250.00583200,0,0.00000000, +open_floor,scene.xml,86,1,202608180861,failed,timeout,solve_timeout,250.02076300,0,0.00000000, +open_floor,scene.xml,86,2,202608180862,failed,timeout,solve_timeout,250.01896300,0,0.00000000, +open_floor,scene.xml,86,3,202608180863,failed,timeout,solve_timeout,250.02279200,0,0.00000000, +open_floor,scene.xml,86,4,202608180864,failed,timeout,solve_timeout,250.00389200,0,0.00000000, +open_floor,scene.xml,87,0,202608180870,failed,timeout,solve_timeout,250.05131200,0,0.00000000, +open_floor,scene.xml,87,1,202608180871,failed,timeout,solve_timeout,250.00410200,0,0.00000000, +open_floor,scene.xml,87,2,202608180872,failed,timeout,solve_timeout,250.02062200,0,0.00000000, +open_floor,scene.xml,87,3,202608180873,failed,timeout,solve_timeout,250.04186200,0,0.00000000, +open_floor,scene.xml,87,4,202608180874,failed,timeout,solve_timeout,250.01245200,0,0.00000000, +open_floor,scene.xml,88,0,202608180880,failed,timeout,solve_timeout,250.00823200,0,0.00000000, +open_floor,scene.xml,88,1,202608180881,failed,timeout,solve_timeout,250.00915200,0,0.00000000, +open_floor,scene.xml,88,2,202608180882,failed,timeout,solve_timeout,250.05484200,0,0.00000000, +open_floor,scene.xml,88,3,202608180883,failed,timeout,solve_timeout,250.02005200,0,0.00000000, +open_floor,scene.xml,88,4,202608180884,failed,timeout,solve_timeout,250.06430200,0,0.00000000, +open_floor,scene.xml,90,0,202608180900,failed,timeout,solve_timeout,250.04532200,0,0.00000000, +open_floor,scene.xml,90,1,202608180901,failed,timeout,solve_timeout,250.01463300,0,0.00000000, +open_floor,scene.xml,90,2,202608180902,failed,timeout,solve_timeout,250.04694200,0,0.00000000, +open_floor,scene.xml,90,3,202608180903,failed,timeout,solve_timeout,250.05496200,0,0.00000000, +open_floor,scene.xml,90,4,202608180904,failed,timeout,solve_timeout,250.06240200,0,0.00000000, +open_floor,scene.xml,91,0,202608180910,success,success,validated,2.44260600,15,3.11148881,b20887bd379081f1a6b4d446c0027ddb87dba99ad10b1422b7e6b0665e24469c +open_floor,scene.xml,91,1,202608180911,success,success,validated,2.16326600,38,9.24603714,b12437cdffaf5e406717b7a1e144c72a8e5503e1cdbae7bbc3fcc9229579e715 +open_floor,scene.xml,91,2,202608180912,success,success,validated,4.89768200,15,3.21057761,123f5372fb704419965877665b4a18bd0394d360e3d47c8a28e8d1af19f59abd +open_floor,scene.xml,91,3,202608180913,success,success,validated,0.63448900,19,4.49998300,69bad26f938d0347223ffbff8f98dfe42b8ccfe6d0ddb699edd42e80888aed73 +open_floor,scene.xml,91,4,202608180914,success,success,validated,4.24231300,92,22.75000000,d916df1f7e61f64ebee5c41e5530abcd0b736a2945addfec0f6be4afff5dae69 +open_floor,scene.xml,92,0,202608180920,failed,timeout,solve_timeout,250.00390200,0,0.00000000, +open_floor,scene.xml,92,1,202608180921,failed,timeout,solve_timeout,250.06431200,0,0.00000000, +open_floor,scene.xml,92,2,202608180922,failed,timeout,solve_timeout,250.00928200,0,0.00000000, +open_floor,scene.xml,92,3,202608180923,failed,timeout,solve_timeout,250.00196200,0,0.00000000, +open_floor,scene.xml,92,4,202608180924,failed,timeout,solve_timeout,250.06921200,0,0.00000000, +open_floor,scene.xml,95,0,202608180950,failed,timeout,solve_timeout,250.03696200,0,0.00000000, +open_floor,scene.xml,95,1,202608180951,failed,timeout,solve_timeout,250.05416200,0,0.00000000, +open_floor,scene.xml,95,2,202608180952,failed,timeout,solve_timeout,250.01266200,0,0.00000000, +open_floor,scene.xml,95,3,202608180953,failed,timeout,solve_timeout,250.01648200,0,0.00000000, +open_floor,scene.xml,95,4,202608180954,failed,timeout,solve_timeout,250.00407300,0,0.00000000, +open_floor,scene.xml,96,0,202608180960,failed,timeout,solve_timeout,250.04715200,0,0.00000000, +open_floor,scene.xml,96,1,202608180961,failed,timeout,solve_timeout,250.01714200,0,0.00000000, +open_floor,scene.xml,96,2,202608180962,failed,timeout,solve_timeout,250.04666200,0,0.00000000, +open_floor,scene.xml,96,3,202608180963,failed,timeout,solve_timeout,250.00253200,0,0.00000000, +open_floor,scene.xml,96,4,202608180964,failed,timeout,solve_timeout,250.03715200,0,0.00000000, +open_floor,scene.xml,98,0,202608180980,failed,timeout,solve_timeout,250.01045200,0,0.00000000, +open_floor,scene.xml,98,1,202608180981,failed,timeout,solve_timeout,250.00827200,0,0.00000000, +open_floor,scene.xml,98,2,202608180982,failed,timeout,solve_timeout,250.03219200,0,0.00000000, +open_floor,scene.xml,98,3,202608180983,failed,timeout,solve_timeout,250.03622200,0,0.00000000, +open_floor,scene.xml,98,4,202608180984,failed,timeout,solve_timeout,250.03253200,0,0.00000000, +open_floor,scene.xml,99,0,202608180990,success,success,validated,8.68035600,72,17.35649937,9fa9fd122e8db7931d0e86cc43a4ec31649c71af375721ca4c8fa0a2685fe570 +open_floor,scene.xml,99,1,202608180991,success,success,validated,8.00967700,104,25.74999970,c4e1af67d078257fcb0f7cdee98d20abbdf620587575f09795fa5c12de47ce61 +open_floor,scene.xml,99,2,202608180992,success,success,validated,11.43675100,76,18.30886556,871f32c64c9b98d76ffd85812868c63abb63e2e090e550fe675a1a4d7e802ce2 +open_floor,scene.xml,99,3,202608180993,success,success,validated,6.35790900,72,17.74959791,91cbfa13b4cd4cfb7028551c7f22f49b3651edd8038ed7df4a66aafe1fcdb9f8 +open_floor,scene.xml,99,4,202608180994,success,success,validated,7.79891700,119,29.50000000,0e706ec02858a04217c6ca8c0ae0e8ebefec10419568d0214e23af4dc9a4f033 +offset_pillar,scene_cluttered.xml,0,0,202608190000,failed,timeout,solve_timeout,250.01299200,0,0.00000000, +offset_pillar,scene_cluttered.xml,0,1,202608190001,failed,timeout,solve_timeout,250.02087200,0,0.00000000, +offset_pillar,scene_cluttered.xml,0,2,202608190002,failed,timeout,solve_timeout,250.04231200,0,0.00000000, +offset_pillar,scene_cluttered.xml,0,3,202608190003,failed,timeout,solve_timeout,250.00658200,0,0.00000000, +offset_pillar,scene_cluttered.xml,0,4,202608190004,failed,timeout,solve_timeout,250.00228200,0,0.00000000, +offset_pillar,scene_cluttered.xml,1,0,202608190010,failed,timeout,solve_timeout,250.02027200,0,0.00000000, +offset_pillar,scene_cluttered.xml,1,1,202608190011,failed,timeout,solve_timeout,250.01037200,0,0.00000000, +offset_pillar,scene_cluttered.xml,1,2,202608190012,failed,timeout,solve_timeout,250.01990200,0,0.00000000, +offset_pillar,scene_cluttered.xml,1,3,202608190013,failed,timeout,solve_timeout,250.05242200,0,0.00000000, +offset_pillar,scene_cluttered.xml,1,4,202608190014,failed,timeout,solve_timeout,250.00146200,0,0.00000000, +offset_pillar,scene_cluttered.xml,2,0,202608190020,success,success,validated,2.11613700,55,13.49911820,a44dde3e00417e2334effd1a5608d1dc5268307dbfba095ecc65382c156ab17d +offset_pillar,scene_cluttered.xml,2,1,202608190021,success,success,validated,3.77893400,73,17.96437133,6bbceacb0350ffe1ee12a4bfc22e7f92ee8de5073203636055de5a59bcf5c7a5 +offset_pillar,scene_cluttered.xml,2,2,202608190022,success,success,validated,8.98111500,121,30.00000000,e14f5d9bf644e435926ea0db03b594c760ac4ab81d2dfbd0476e0686daa7e28d +offset_pillar,scene_cluttered.xml,2,3,202608190023,success,success,validated,12.07139000,51,12.05199828,f89f32a0e79c1d8ce8b532103decae271f4ca04d4f370dbb1474c84cd02bbe52 +offset_pillar,scene_cluttered.xml,2,4,202608190024,success,success,validated,11.65877000,110,27.25000000,7255ee3652aa9df90d7cdf5df96c4ba60f5d9cf24427ba55de708703a883e81d +offset_pillar,scene_cluttered.xml,3,0,202608190030,failed,success,validated,54.94350800,99,24.49005561,4d4e163cc6709923ed42b420462493a351236975204d5981f4d858251df94525 +offset_pillar,scene_cluttered.xml,3,1,202608190031,failed,success,validated,153.83060200,198,49.23326580,ecdbbcb0e77403b653ca9f5782045536845315e176b1664962350d2e0abd20ff +offset_pillar,scene_cluttered.xml,3,2,202608190032,failed,success,validated,85.97951600,90,22.19170119,fb1fbdc1aa6101d06ca209488817e89951d8d2016baade8196b64148bf816057 +offset_pillar,scene_cluttered.xml,3,3,202608190033,failed,success,validated,108.41260800,142,35.23759979,434402b978f045e8b9da392a3870b05324994a6af1365ac3c3867fd1d5bc355b +offset_pillar,scene_cluttered.xml,3,4,202608190034,failed,success,validated,99.27750500,140,34.68906436,37135cb585e14c9778387057d49e30c187a848feed60e7fc514d0a069020ee4a +offset_pillar,scene_cluttered.xml,4,0,202608190040,failed,timeout,solve_timeout,250.01896200,0,0.00000000, +offset_pillar,scene_cluttered.xml,4,1,202608190041,failed,timeout,solve_timeout,250.01698200,0,0.00000000, +offset_pillar,scene_cluttered.xml,4,2,202608190042,failed,timeout,solve_timeout,250.02119200,0,0.00000000, +offset_pillar,scene_cluttered.xml,4,3,202608190043,failed,timeout,solve_timeout,250.02024200,0,0.00000000, +offset_pillar,scene_cluttered.xml,4,4,202608190044,failed,timeout,solve_timeout,250.00147300,0,0.00000000, +offset_pillar,scene_cluttered.xml,5,0,202608190050,failed,timeout,solve_timeout,250.00310300,0,0.00000000, +offset_pillar,scene_cluttered.xml,5,1,202608190051,failed,timeout,solve_timeout,250.01948300,0,0.00000000, +offset_pillar,scene_cluttered.xml,5,2,202608190052,failed,timeout,solve_timeout,250.03897200,0,0.00000000, +offset_pillar,scene_cluttered.xml,5,3,202608190053,failed,timeout,solve_timeout,250.00125200,0,0.00000000, +offset_pillar,scene_cluttered.xml,5,4,202608190054,failed,timeout,solve_timeout,250.01916200,0,0.00000000, +offset_pillar,scene_cluttered.xml,7,0,202608190070,failed,timeout,solve_timeout,250.00598200,0,0.00000000, +offset_pillar,scene_cluttered.xml,7,1,202608190071,failed,timeout,solve_timeout,250.00435200,0,0.00000000, +offset_pillar,scene_cluttered.xml,7,2,202608190072,failed,timeout,solve_timeout,250.06064200,0,0.00000000, +offset_pillar,scene_cluttered.xml,7,3,202608190073,failed,timeout,solve_timeout,250.04020200,0,0.00000000, +offset_pillar,scene_cluttered.xml,7,4,202608190074,failed,timeout,solve_timeout,250.00603200,0,0.00000000, +offset_pillar,scene_cluttered.xml,8,0,202608190080,success,success,validated,6.97144800,70,17.22657731,10e6943cc1eff6eca22cfddf24db04a35f762583b086fc4c1d6bb79aa4a340ed +offset_pillar,scene_cluttered.xml,8,1,202608190081,success,success,validated,14.60548600,94,23.24693645,9eeba5083614bd6b774ff4b45b53af992a3f390a35b60be7d0b04601e15d1c22 +offset_pillar,scene_cluttered.xml,8,2,202608190082,success,success,validated,4.62932300,54,13.22952815,8ca1040e9ba70a1c40e19a2f3c049aca90a8098f58dab5af2f20a289bf59c706 +offset_pillar,scene_cluttered.xml,8,3,202608190083,success,success,validated,16.32821200,107,26.47838166,79d862e170ea523361dd34a69b82d820f9b4b7bb414a2cfa1d46ac60dd715958 +offset_pillar,scene_cluttered.xml,8,4,202608190084,success,success,validated,9.28820500,59,14.37608916,f85e512c0ed2e84dc83d97e02eb5607dde9f781b30d65c3e1390be08fe5be5e2 +offset_pillar,scene_cluttered.xml,9,0,202608190090,success,success,validated,13.81278700,126,31.25000000,331a3f573b542c4493479185ec33d168567d63c97272faec1bfdd9dd05cec518 +offset_pillar,scene_cluttered.xml,9,1,202608190091,success,success,validated,9.50365400,95,23.49999996,ca66df418edcf837606c755ef2fd3c5022ba5ff062b8eb4083ae1440ee4a42bf +offset_pillar,scene_cluttered.xml,9,2,202608190092,success,success,validated,12.54778900,61,14.71863324,9450b84673108434815fa6bbb71cceac4511856f5ef2674aedf8acca27460fb5 +offset_pillar,scene_cluttered.xml,9,3,202608190093,success,success,validated,3.38269500,68,16.68946517,db1964158c7ab90561b7fbf5b2c9a4db25863a5ff5a8cbc1e94474b61ecdd917 +offset_pillar,scene_cluttered.xml,9,4,202608190094,success,success,validated,20.42741600,112,27.74999994,b2c37f8e457ec7adb454aeb2353a5a97c5dd307fc890476d8b10b64b8ca0a33a +offset_pillar,scene_cluttered.xml,10,0,202608190100,failed,timeout,solve_timeout,250.00553200,0,0.00000000, +offset_pillar,scene_cluttered.xml,10,1,202608190101,failed,timeout,solve_timeout,250.00169300,0,0.00000000, +offset_pillar,scene_cluttered.xml,10,2,202608190102,failed,timeout,solve_timeout,250.01643300,0,0.00000000, +offset_pillar,scene_cluttered.xml,10,3,202608190103,failed,timeout,solve_timeout,250.01586200,0,0.00000000, +offset_pillar,scene_cluttered.xml,10,4,202608190104,failed,timeout,solve_timeout,250.01193200,0,0.00000000, +offset_pillar,scene_cluttered.xml,11,0,202608190110,failed,timeout,solve_timeout,250.01594200,0,0.00000000, +offset_pillar,scene_cluttered.xml,11,1,202608190111,failed,timeout,solve_timeout,250.02667200,0,0.00000000, +offset_pillar,scene_cluttered.xml,11,2,202608190112,failed,timeout,solve_timeout,250.00401200,0,0.00000000, +offset_pillar,scene_cluttered.xml,11,3,202608190113,failed,timeout,solve_timeout,250.03154200,0,0.00000000, +offset_pillar,scene_cluttered.xml,11,4,202608190114,failed,timeout,solve_timeout,250.01271200,0,0.00000000, +offset_pillar,scene_cluttered.xml,12,0,202608190120,failed,timeout,solve_timeout,250.00826200,0,0.00000000, +offset_pillar,scene_cluttered.xml,12,1,202608190121,failed,timeout,solve_timeout,250.00896200,0,0.00000000, +offset_pillar,scene_cluttered.xml,12,2,202608190122,failed,timeout,solve_timeout,250.00899200,0,0.00000000, +offset_pillar,scene_cluttered.xml,12,3,202608190123,failed,timeout,solve_timeout,250.00787200,0,0.00000000, +offset_pillar,scene_cluttered.xml,12,4,202608190124,failed,timeout,solve_timeout,250.00483200,0,0.00000000, +offset_pillar,scene_cluttered.xml,13,0,202608190130,success,success,validated,4.99661100,70,17.19090094,62fb7711e8e04d5a6c3e19c4fea0047aa8f743baa037574c39eadc335166e5a0 +offset_pillar,scene_cluttered.xml,13,1,202608190131,success,success,validated,4.80295200,86,21.24999956,888237223ffe15594e684e6ff5453e63941341cde07d5fc194c54c4a5069713b +offset_pillar,scene_cluttered.xml,13,2,202608190132,success,success,validated,6.89714800,98,24.25000000,53bc96a352c359fc0bc7c6fdc76e3d9eaeae896a7509c7ff3e9ecaf5c94dd021 +offset_pillar,scene_cluttered.xml,13,3,202608190133,success,success,validated,4.92806200,79,19.49928451,5a89cd5d0b7208a634868aaf4e8474ac57ce68fa560ca217e74e8adbdcd23e6b +offset_pillar,scene_cluttered.xml,13,4,202608190134,success,success,validated,2.93326500,59,14.45211722,a8bcbae0b60642556a2f3f4d3cee64ecad72e864f965dd289157fc96344301dc +offset_pillar,scene_cluttered.xml,14,0,202608190140,failed,timeout,solve_timeout,250.01682200,0,0.00000000, +offset_pillar,scene_cluttered.xml,14,1,202608190141,failed,timeout,solve_timeout,250.03329200,0,0.00000000, +offset_pillar,scene_cluttered.xml,14,2,202608190142,failed,timeout,solve_timeout,250.01164200,0,0.00000000, +offset_pillar,scene_cluttered.xml,14,3,202608190143,failed,timeout,solve_timeout,250.00103200,0,0.00000000, +offset_pillar,scene_cluttered.xml,14,4,202608190144,failed,timeout,solve_timeout,250.01204200,0,0.00000000, +offset_pillar,scene_cluttered.xml,15,0,202608190150,success,success,validated,7.31100800,50,12.22544925,28b57ab336c230193217b3ffecdc774d17b727f37486906661e9a8a4cb99b37c +offset_pillar,scene_cluttered.xml,15,1,202608190151,success,success,validated,5.26975100,46,11.18391188,425e9f62ea55c4902779b58ce5af30dd30feb84423f1b43a24b20a3da19560c5 +offset_pillar,scene_cluttered.xml,15,2,202608190152,success,success,validated,3.97455400,46,11.18578913,c51f36d8538c3eb56d9ec67d043dec652c0fca505a0c2e3bf1f28b60e3b7d3ea +offset_pillar,scene_cluttered.xml,15,3,202608190153,success,success,validated,5.76145100,54,13.20756248,cff6ffd31c16b496f2116e2835484276258419224e37065a0e6bb90e39383969 +offset_pillar,scene_cluttered.xml,15,4,202608190154,success,success,validated,2.85962500,45,10.98354768,da52cc23606f7fcee7215b3a35de165db0380899ec4565a6a5cfe6bd3fc7f91e +offset_pillar,scene_cluttered.xml,16,0,202608190160,failed,timeout,solve_timeout,250.01340200,0,0.00000000, +offset_pillar,scene_cluttered.xml,16,1,202608190161,failed,timeout,solve_timeout,250.00723200,0,0.00000000, +offset_pillar,scene_cluttered.xml,16,2,202608190162,failed,timeout,solve_timeout,250.02559200,0,0.00000000, +offset_pillar,scene_cluttered.xml,16,3,202608190163,failed,timeout,solve_timeout,250.02004200,0,0.00000000, +offset_pillar,scene_cluttered.xml,16,4,202608190164,failed,timeout,solve_timeout,250.01447200,0,0.00000000, +offset_pillar,scene_cluttered.xml,17,0,202608190170,failed,timeout,solve_timeout,250.07095200,0,0.00000000, +offset_pillar,scene_cluttered.xml,17,1,202608190171,failed,timeout,solve_timeout,250.06937200,0,0.00000000, +offset_pillar,scene_cluttered.xml,17,2,202608190172,failed,timeout,solve_timeout,250.01200200,0,0.00000000, +offset_pillar,scene_cluttered.xml,17,3,202608190173,failed,timeout,solve_timeout,250.05713200,0,0.00000000, +offset_pillar,scene_cluttered.xml,17,4,202608190174,failed,timeout,solve_timeout,250.01059200,0,0.00000000, +offset_pillar,scene_cluttered.xml,18,0,202608190180,failed,success,validated,36.22709900,121,29.96083092,8d06bfb0843f2fa676a3b82e62af6a01a23ac24c69bcc1a390573b1d6ea5bb8d +offset_pillar,scene_cluttered.xml,18,1,202608190181,failed,success,validated,46.99470200,120,29.74487126,170a488d83de88f33e39d764ed3ded6a9d45f48cd69d282c0680190023d5aed2 +offset_pillar,scene_cluttered.xml,18,2,202608190182,failed,success,validated,30.82960800,129,31.93870559,a84be84a88fe03aeaecd64a7df7a252259a617ec2346716e782b1627931a760a +offset_pillar,scene_cluttered.xml,18,3,202608190183,failed,success,validated,43.48404700,104,25.65825660,8121fa733ead767a9a247ac6419ff8c24969897744adcc63d74f0886f9d9cb08 +offset_pillar,scene_cluttered.xml,18,4,202608190184,success,success,validated,39.00206500,109,26.99624863,8cfebb831c50610f7de580baf1a7449fa836372bf8563d6e804156ba617ceac5 +offset_pillar,scene_cluttered.xml,19,0,202608190190,failed,timeout,solve_timeout,250.04271200,0,0.00000000, +offset_pillar,scene_cluttered.xml,19,1,202608190191,failed,timeout,solve_timeout,250.01891200,0,0.00000000, +offset_pillar,scene_cluttered.xml,19,2,202608190192,failed,timeout,solve_timeout,250.01086200,0,0.00000000, +offset_pillar,scene_cluttered.xml,19,3,202608190193,failed,timeout,solve_timeout,250.01265200,0,0.00000000, +offset_pillar,scene_cluttered.xml,19,4,202608190194,failed,timeout,solve_timeout,250.02569200,0,0.00000000, +offset_pillar,scene_cluttered.xml,20,0,202608190200,success,success,validated,19.19017800,123,30.50000000,4c1287504d94acf7b17ea8e94465ab40e6a68d2354c8611bb25f312fc934b0ab +offset_pillar,scene_cluttered.xml,20,1,202608190201,success,success,validated,8.36242600,112,27.75000000,0a3a3b5ba15b782b4cecec7fe5e16fb6fc2e7eff7a0669d696339c06dde22fd3 +offset_pillar,scene_cluttered.xml,20,2,202608190202,success,success,validated,22.05407300,100,24.75000000,583fd083d4f14b1816cbe5a1b4da9c7758429578c9613ab2971ddac1db3b08e7 +offset_pillar,scene_cluttered.xml,20,3,202608190203,success,success,validated,9.84122400,107,26.50000000,f940826dbc0d4d354b87454b94d1bb31b65f163f809cdab3eb6b613d40e5650c +offset_pillar,scene_cluttered.xml,20,4,202608190204,success,success,validated,10.05402300,85,20.99999993,ab73e159e5f7bc7d338dff79aae5aad57abe9645f51a33f810ed35ecaa42d460 +offset_pillar,scene_cluttered.xml,21,0,202608190210,failed,timeout,solve_timeout,250.00525200,0,0.00000000, +offset_pillar,scene_cluttered.xml,21,1,202608190211,failed,timeout,solve_timeout,250.06457200,0,0.00000000, +offset_pillar,scene_cluttered.xml,21,2,202608190212,failed,timeout,solve_timeout,250.01284300,0,0.00000000, +offset_pillar,scene_cluttered.xml,21,3,202608190213,failed,timeout,solve_timeout,250.04196200,0,0.00000000, +offset_pillar,scene_cluttered.xml,21,4,202608190214,failed,timeout,solve_timeout,250.03584200,0,0.00000000, +offset_pillar,scene_cluttered.xml,22,0,202608190220,success,success,validated,7.44710800,92,22.70222420,6a968c09ba84945ce37dbe1f1682a5176fe8b660f0ece19d3a84933083193618 +offset_pillar,scene_cluttered.xml,22,1,202608190221,success,success,validated,15.91506400,112,27.74220284,25e7bf639f3202c7ebbf68310a9475666b24828e4d06eedd678598a69fde4d64 +offset_pillar,scene_cluttered.xml,22,2,202608190222,success,success,validated,29.40604100,96,23.53636266,16e990e2abc8818683c8f5b14081e3d72e6ccc43ed6e060f164138453126a05d +offset_pillar,scene_cluttered.xml,22,3,202608190223,success,success,validated,8.82882500,109,26.99747328,89f5588cbfb436153860e87eb10f2fe828e443c4cd2b8f564fd71978367ef5f1 +offset_pillar,scene_cluttered.xml,22,4,202608190224,success,success,validated,15.99229300,124,30.74777888,d02fbdc8e75e41484c64409c2ce5a8413272111c7c7cb8676ade0e07c1602275 +offset_pillar,scene_cluttered.xml,23,0,202608190230,failed,timeout,solve_timeout,250.02806200,0,0.00000000, +offset_pillar,scene_cluttered.xml,23,1,202608190231,failed,timeout,solve_timeout,250.06107200,0,0.00000000, +offset_pillar,scene_cluttered.xml,23,2,202608190232,failed,timeout,solve_timeout,250.03572200,0,0.00000000, +offset_pillar,scene_cluttered.xml,23,3,202608190233,failed,timeout,solve_timeout,250.05121200,0,0.00000000, +offset_pillar,scene_cluttered.xml,23,4,202608190234,failed,timeout,solve_timeout,250.00142200,0,0.00000000, +offset_pillar,scene_cluttered.xml,24,0,202608190240,success,success,validated,6.78275800,94,23.24999991,3600c7a88b00b1ddea104e76200b89ee004507a5a4eb4f6d46c8f49553e2e7a7 +offset_pillar,scene_cluttered.xml,24,1,202608190241,success,success,validated,8.02642600,77,19.00000000,c0527721b805b466d876b9fa503da3a87d0af26c471d4006c881a589b80ec1bc +offset_pillar,scene_cluttered.xml,24,2,202608190242,success,success,validated,5.34907100,97,24.00000000,56614365df16e80a39e820ad2e920881a2176364deab148f36d168cbf98d1588 +offset_pillar,scene_cluttered.xml,24,3,202608190243,success,success,validated,10.09832300,106,26.25000000,e560949f18b240490b54e7e718be933fcc597efc4d035bd72723250025678dcb +offset_pillar,scene_cluttered.xml,24,4,202608190244,success,success,validated,6.88626800,75,18.49999974,a8b7c80bb82a39aee28996b7063acd723c492ff629982fd1b015a99963092ca5 +offset_pillar,scene_cluttered.xml,27,0,202608190270,success,success,validated,1.87332700,54,13.21297189,a23693f372df9c7bf96614e5c9ee60c179757c451379020296c7003d2929f65d +offset_pillar,scene_cluttered.xml,27,1,202608190271,success,success,validated,9.63155400,123,30.50000000,d093f3fabf628e3a29cda57a22a70f1c49a32b6afdb2dbcf8d865f1adbe3a959 +offset_pillar,scene_cluttered.xml,27,2,202608190272,success,success,validated,11.58864100,125,31.00000000,24a1b8737a293d1a422b6c1f983205516f69f46644014c43a43a928d266815fa +offset_pillar,scene_cluttered.xml,27,3,202608190273,success,success,validated,7.60307700,111,27.50000000,b4062aad894759fed78fe3aeb577f39e0043de32f10597fcd5a4d6a19bf107b2 +offset_pillar,scene_cluttered.xml,27,4,202608190274,success,success,validated,5.50849100,64,15.73944070,3a54c553722d4f33d7834d4a0a49f9461bc99041a48991ad92017a165952483c +offset_pillar,scene_cluttered.xml,28,0,202608190280,failed,timeout,solve_timeout,250.03785200,0,0.00000000, +offset_pillar,scene_cluttered.xml,28,1,202608190281,failed,timeout,solve_timeout,250.00320200,0,0.00000000, +offset_pillar,scene_cluttered.xml,28,2,202608190282,failed,timeout,solve_timeout,250.00446200,0,0.00000000, +offset_pillar,scene_cluttered.xml,28,3,202608190283,failed,timeout,solve_timeout,250.03308200,0,0.00000000, +offset_pillar,scene_cluttered.xml,28,4,202608190284,failed,timeout,solve_timeout,250.04759200,0,0.00000000, +offset_pillar,scene_cluttered.xml,29,0,202608190290,success,success,validated,18.72341900,115,28.49986355,0d5bb3dad2969ad213aa0f4dd4244400508cbb75dbed9ce70862dd72098e4ea0 +offset_pillar,scene_cluttered.xml,29,1,202608190291,success,success,validated,175.55448700,111,27.46483517,25099e47585b98848d4bbd11e079e5cb87f47263e3fd130ccf2964d1b2a0657e +offset_pillar,scene_cluttered.xml,29,2,202608190292,success,success,validated,6.38653900,67,16.46462488,12edbb305c303eebd1f03674abdf1c8bb75fc5c6455e7b4919c404995caaec67 +offset_pillar,scene_cluttered.xml,29,3,202608190293,success,success,validated,174.94037700,245,60.94591780,a03e96347438d7f50b827af38da423322b4f65add2f5a57a50e2cd5a4c9c6b49 +offset_pillar,scene_cluttered.xml,29,4,202608190294,success,success,validated,8.78365600,101,24.99888507,3fed0b7c2c776404e62921320eb2a4fc6ca6c566a6f82c45b737994279b72425 +offset_pillar,scene_cluttered.xml,30,0,202608190300,failed,timeout,solve_timeout,250.02186300,0,0.00000000, +offset_pillar,scene_cluttered.xml,30,1,202608190301,failed,timeout,solve_timeout,250.00371200,0,0.00000000, +offset_pillar,scene_cluttered.xml,30,2,202608190302,failed,timeout,solve_timeout,250.00441200,0,0.00000000, +offset_pillar,scene_cluttered.xml,30,3,202608190303,failed,timeout,solve_timeout,250.06083200,0,0.00000000, +offset_pillar,scene_cluttered.xml,30,4,202608190304,failed,timeout,solve_timeout,250.04491200,0,0.00000000, +offset_pillar,scene_cluttered.xml,31,0,202608190310,failed,timeout,solve_timeout,250.04705200,0,0.00000000, +offset_pillar,scene_cluttered.xml,31,1,202608190311,failed,timeout,solve_timeout,250.04558200,0,0.00000000, +offset_pillar,scene_cluttered.xml,31,2,202608190312,failed,timeout,solve_timeout,250.02552200,0,0.00000000, +offset_pillar,scene_cluttered.xml,31,3,202608190313,failed,timeout,solve_timeout,250.00223200,0,0.00000000, +offset_pillar,scene_cluttered.xml,31,4,202608190314,failed,timeout,solve_timeout,250.01699200,0,0.00000000, +offset_pillar,scene_cluttered.xml,33,0,202608190330,failed,timeout,solve_timeout,250.02680200,0,0.00000000, +offset_pillar,scene_cluttered.xml,33,1,202608190331,failed,timeout,solve_timeout,250.04693200,0,0.00000000, +offset_pillar,scene_cluttered.xml,33,2,202608190332,failed,timeout,solve_timeout,250.03718200,0,0.00000000, +offset_pillar,scene_cluttered.xml,33,3,202608190333,failed,timeout,solve_timeout,250.01390200,0,0.00000000, +offset_pillar,scene_cluttered.xml,33,4,202608190334,failed,timeout,solve_timeout,250.04000200,0,0.00000000, +offset_pillar,scene_cluttered.xml,36,0,202608190360,failed,timeout,solve_timeout,250.01462200,0,0.00000000, +offset_pillar,scene_cluttered.xml,36,1,202608190361,failed,timeout,solve_timeout,250.01395200,0,0.00000000, +offset_pillar,scene_cluttered.xml,36,2,202608190362,failed,timeout,solve_timeout,250.01337200,0,0.00000000, +offset_pillar,scene_cluttered.xml,36,3,202608190363,failed,timeout,solve_timeout,250.03076200,0,0.00000000, +offset_pillar,scene_cluttered.xml,36,4,202608190364,failed,timeout,solve_timeout,250.01915200,0,0.00000000, +offset_pillar,scene_cluttered.xml,37,0,202608190370,failed,timeout,solve_timeout,250.00828300,0,0.00000000, +offset_pillar,scene_cluttered.xml,37,1,202608190371,failed,timeout,solve_timeout,250.01354300,0,0.00000000, +offset_pillar,scene_cluttered.xml,37,2,202608190372,failed,timeout,solve_timeout,250.00189300,0,0.00000000, +offset_pillar,scene_cluttered.xml,37,3,202608190373,failed,timeout,solve_timeout,250.01541200,0,0.00000000, +offset_pillar,scene_cluttered.xml,37,4,202608190374,failed,timeout,solve_timeout,250.01069200,0,0.00000000, +offset_pillar,scene_cluttered.xml,38,0,202608190380,success,success,validated,29.03197200,61,14.95020071,fba4995076357a80bf4d6edc0463bc3461a6dbade86a010656d537e78b49362a +offset_pillar,scene_cluttered.xml,38,1,202608190381,success,success,validated,10.34634300,46,11.24803136,eaa3a3db3f7917fc324ae8c407ddde73fcf9a1cbcaed5a9f58370eca1b07cc78 +offset_pillar,scene_cluttered.xml,38,2,202608190382,success,success,validated,40.17609300,84,20.74995479,fdd2cb30ffe4ab218cf9c96f31b0f856fbabc1e3ad889ef2bf507c6aa8729489 +offset_pillar,scene_cluttered.xml,38,3,202608190383,success,success,validated,6.08042000,42,10.16130204,5e0fb60f5057c8afe6146c104924e20df4c27fccff61024c2c3ea279863c9be3 +offset_pillar,scene_cluttered.xml,38,4,202608190384,success,success,validated,34.87107200,50,12.23786362,d8d61b27b72807ad878689b6b70e8cf18fd9b0e8765d658a21e01c90109857a0 +offset_pillar,scene_cluttered.xml,40,0,202608190400,failed,timeout,solve_timeout,250.02264200,0,0.00000000, +offset_pillar,scene_cluttered.xml,40,1,202608190401,failed,timeout,solve_timeout,250.04356200,0,0.00000000, +offset_pillar,scene_cluttered.xml,40,2,202608190402,failed,timeout,solve_timeout,250.04557200,0,0.00000000, +offset_pillar,scene_cluttered.xml,40,3,202608190403,failed,timeout,solve_timeout,250.02239200,0,0.00000000, +offset_pillar,scene_cluttered.xml,40,4,202608190404,failed,timeout,solve_timeout,250.00991200,0,0.00000000, +offset_pillar,scene_cluttered.xml,41,0,202608190410,success,success,validated,6.54709900,101,24.99999999,195470b961551bda524dcd74495e904c315f252fa494b7dfff16e294bb9f4498 +offset_pillar,scene_cluttered.xml,41,1,202608190411,success,success,validated,5.04323100,95,23.49999980,bc8f29c684f2d5d0f616cabbcbde9a475b541b19db76a69cdf5d17f53936ed60 +offset_pillar,scene_cluttered.xml,41,2,202608190412,success,success,validated,8.35727600,143,35.50000000,10d43af1f3ceae89b00a3f3e90c25c497871cfe23f531e8387935007de7bd264 +offset_pillar,scene_cluttered.xml,41,3,202608190413,success,success,validated,5.51246100,116,28.75000000,cab61184920c1bfdffef08d701c6186c3a39b480c0dcf4994b6f23f818f032c9 +offset_pillar,scene_cluttered.xml,41,4,202608190414,success,success,validated,6.79161800,136,33.75000000,0e9d296d4943681966dbef2879c43810e10fb940d4b3caec7c9f9f2c0991db47 +offset_pillar,scene_cluttered.xml,42,0,202608190420,failed,timeout,solve_timeout,250.00381200,0,0.00000000, +offset_pillar,scene_cluttered.xml,42,1,202608190421,failed,timeout,solve_timeout,250.04714200,0,0.00000000, +offset_pillar,scene_cluttered.xml,42,2,202608190422,failed,timeout,solve_timeout,250.01363200,0,0.00000000, +offset_pillar,scene_cluttered.xml,42,3,202608190423,failed,timeout,solve_timeout,250.02393200,0,0.00000000, +offset_pillar,scene_cluttered.xml,42,4,202608190424,failed,timeout,solve_timeout,250.02202200,0,0.00000000, +offset_pillar,scene_cluttered.xml,43,0,202608190430,failed,timeout,solve_timeout,250.00722200,0,0.00000000, +offset_pillar,scene_cluttered.xml,43,1,202608190431,failed,timeout,solve_timeout,250.00240200,0,0.00000000, +offset_pillar,scene_cluttered.xml,43,2,202608190432,failed,timeout,solve_timeout,250.00873200,0,0.00000000, +offset_pillar,scene_cluttered.xml,43,3,202608190433,failed,timeout,solve_timeout,250.00469200,0,0.00000000, +offset_pillar,scene_cluttered.xml,43,4,202608190434,failed,timeout,solve_timeout,250.00451200,0,0.00000000, +offset_pillar,scene_cluttered.xml,44,0,202608190440,success,success,validated,6.73348800,105,25.94453672,80df1c9bf9f7af7e7055dc3d9d7b663c8f7721dec9b5e32e39eda6619c60e408 +offset_pillar,scene_cluttered.xml,44,1,202608190441,success,success,validated,9.14800500,142,35.25000000,cf8201fdceccf85a84288c82455dc409020c148a411528f6b262437047de8b5e +offset_pillar,scene_cluttered.xml,44,2,202608190442,success,success,validated,10.71428200,143,35.50000000,e8f130fdade46c4508e28c0c6672d746a424fb94c67c83899d313aaf89e49fa4 +offset_pillar,scene_cluttered.xml,44,3,202608190443,success,success,validated,3.97855400,92,22.74566541,84980cabd24069a077941aef1dd813c6424b228279830538893a6de018112efd +offset_pillar,scene_cluttered.xml,44,4,202608190444,success,success,validated,10.51142200,138,34.25000000,cfa6c845cd6ca95cffb3668ae1425187924027b562eb67a2b4cbccc500102468 +offset_pillar,scene_cluttered.xml,45,0,202608190450,failed,timeout,solve_timeout,250.02128200,0,0.00000000, +offset_pillar,scene_cluttered.xml,45,1,202608190451,failed,timeout,solve_timeout,250.00567200,0,0.00000000, +offset_pillar,scene_cluttered.xml,45,2,202608190452,failed,timeout,solve_timeout,250.02229200,0,0.00000000, +offset_pillar,scene_cluttered.xml,45,3,202608190453,failed,timeout,solve_timeout,250.01265200,0,0.00000000, +offset_pillar,scene_cluttered.xml,45,4,202608190454,failed,timeout,solve_timeout,250.00835300,0,0.00000000, +offset_pillar,scene_cluttered.xml,46,0,202608190460,failed,timeout,solve_timeout,250.02567200,0,0.00000000, +offset_pillar,scene_cluttered.xml,46,1,202608190461,failed,timeout,solve_timeout,250.03432200,0,0.00000000, +offset_pillar,scene_cluttered.xml,46,2,202608190462,failed,timeout,solve_timeout,250.02758200,0,0.00000000, +offset_pillar,scene_cluttered.xml,46,3,202608190463,failed,timeout,solve_timeout,250.05941200,0,0.00000000, +offset_pillar,scene_cluttered.xml,46,4,202608190464,failed,timeout,solve_timeout,250.00643200,0,0.00000000, +offset_pillar,scene_cluttered.xml,47,0,202608190470,failed,timeout,solve_timeout,250.01787200,0,0.00000000, +offset_pillar,scene_cluttered.xml,47,1,202608190471,failed,timeout,solve_timeout,250.03425200,0,0.00000000, +offset_pillar,scene_cluttered.xml,47,2,202608190472,failed,timeout,solve_timeout,250.03982200,0,0.00000000, +offset_pillar,scene_cluttered.xml,47,3,202608190473,failed,timeout,solve_timeout,250.02184200,0,0.00000000, +offset_pillar,scene_cluttered.xml,47,4,202608190474,failed,timeout,solve_timeout,250.04535200,0,0.00000000, +offset_pillar,scene_cluttered.xml,49,0,202608190490,failed,timeout,solve_timeout,250.00614200,0,0.00000000, +offset_pillar,scene_cluttered.xml,49,1,202608190491,failed,timeout,solve_timeout,250.01378200,0,0.00000000, +offset_pillar,scene_cluttered.xml,49,2,202608190492,failed,timeout,solve_timeout,250.01523200,0,0.00000000, +offset_pillar,scene_cluttered.xml,49,3,202608190493,failed,timeout,solve_timeout,250.00832200,0,0.00000000, +offset_pillar,scene_cluttered.xml,49,4,202608190494,failed,timeout,solve_timeout,250.03035200,0,0.00000000, +offset_pillar,scene_cluttered.xml,50,0,202608190500,success,success,validated,146.60900500,130,32.20882317,433d68ffe5eb08489fd5159667d90e44772ec22c2ce4c74f7e3b3cf58b177cb1 +offset_pillar,scene_cluttered.xml,50,1,202608190501,success,success,validated,142.89256200,82,20.23988620,55e9b1e3397a4a3501d66574432d72c0429e2c2f002f3d3f08ccdf27e452235e +offset_pillar,scene_cluttered.xml,50,2,202608190502,success,success,validated,85.07536800,140,34.72485912,e244a05e15f666e6b5b053abd0431b8360ddbe76a85fb84f71adbf075101a736 +offset_pillar,scene_cluttered.xml,50,3,202608190503,success,success,validated,153.31660300,110,27.15845150,31181341d1a666a6d3edc6eac2f1f10900654c5a83b4fd4ef752a5f436523692 +offset_pillar,scene_cluttered.xml,50,4,202608190504,success,success,validated,93.30822400,132,32.74257175,a40fd3ee65e68de27615e7e3c95f4712350e5ade1bcf8abcf689228596926517 +offset_pillar,scene_cluttered.xml,51,0,202608190510,success,success,validated,6.01868000,83,20.50000000,d8123f1370a76497ba92017cda1c834696d9c641f4a161cfc4fee5e395cfdaed +offset_pillar,scene_cluttered.xml,51,1,202608190511,success,success,validated,3.85486400,53,12.95899575,7350af850729cc908fc7faa7cb08f7eb61ac5d5337f0bc5b5c51e6bff01d6e64 +offset_pillar,scene_cluttered.xml,51,2,202608190512,success,success,validated,2.15966600,53,12.96453167,959c6ea1e7259f8bd680962fa5e2a501ebcc477a468986e8e55a0b8dab865a55 +offset_pillar,scene_cluttered.xml,51,3,202608190513,success,success,validated,6.23556000,83,20.49999989,8206d98aa371b2f45d8550a5ded87b32c1e57ead2384f92e3b4281113582b9e9 +offset_pillar,scene_cluttered.xml,51,4,202608190514,success,success,validated,4.04402300,56,13.70704832,b39d35393e40983f552202c7e846122282b6532c798caf9baba166478d12b608 +offset_pillar,scene_cluttered.xml,52,0,202608190520,success,success,validated,8.68615500,102,25.24693598,6f7e481bf142c2e32d014e65496b4d9e16c204e9052d0257bee134bcbda073d2 +offset_pillar,scene_cluttered.xml,52,1,202608190521,success,success,validated,7.51555800,102,25.21698137,427fc52c1684edabe2125554ea0b1880273abd77e06bd96112801a3aebf1756a +offset_pillar,scene_cluttered.xml,52,2,202608190522,success,success,validated,7.20191700,103,25.49561569,a29d50229ac1372690c0eca62e08d97ee9ea5cacf759a07767cb7647cc877c96 +offset_pillar,scene_cluttered.xml,52,3,202608190523,success,success,validated,14.44429600,147,36.50000000,0211d35021a1de094fb8149ac40a3f1c38a1c3e3a69200f31b0fa03a7eb456aa +offset_pillar,scene_cluttered.xml,52,4,202608190524,success,success,validated,5.87219000,89,21.99999985,f2aaa25c6d083cc05f55175fade40bb325f443fe4e453c39036a9d3a3286b2fe +offset_pillar,scene_cluttered.xml,53,0,202608190530,failed,timeout,solve_timeout,250.00413200,0,0.00000000, +offset_pillar,scene_cluttered.xml,53,1,202608190531,failed,timeout,solve_timeout,250.00806200,0,0.00000000, +offset_pillar,scene_cluttered.xml,53,2,202608190532,failed,timeout,solve_timeout,250.00775200,0,0.00000000, +offset_pillar,scene_cluttered.xml,53,3,202608190533,failed,timeout,solve_timeout,250.02226200,0,0.00000000, +offset_pillar,scene_cluttered.xml,53,4,202608190534,failed,timeout,solve_timeout,250.02916200,0,0.00000000, +offset_pillar,scene_cluttered.xml,54,0,202608190540,failed,timeout,solve_timeout,250.00447300,0,0.00000000, +offset_pillar,scene_cluttered.xml,54,1,202608190541,failed,timeout,solve_timeout,250.04189200,0,0.00000000, +offset_pillar,scene_cluttered.xml,54,2,202608190542,failed,timeout,solve_timeout,250.00175200,0,0.00000000, +offset_pillar,scene_cluttered.xml,54,3,202608190543,failed,timeout,solve_timeout,250.00741200,0,0.00000000, +offset_pillar,scene_cluttered.xml,54,4,202608190544,failed,timeout,solve_timeout,250.00460200,0,0.00000000, +offset_pillar,scene_cluttered.xml,56,0,202608190560,failed,timeout,solve_timeout,250.02960200,0,0.00000000, +offset_pillar,scene_cluttered.xml,56,1,202608190561,failed,timeout,solve_timeout,250.02073200,0,0.00000000, +offset_pillar,scene_cluttered.xml,56,2,202608190562,failed,timeout,solve_timeout,250.01785200,0,0.00000000, +offset_pillar,scene_cluttered.xml,56,3,202608190563,failed,timeout,solve_timeout,250.01477200,0,0.00000000, +offset_pillar,scene_cluttered.xml,56,4,202608190564,failed,timeout,solve_timeout,250.00515200,0,0.00000000, +offset_pillar,scene_cluttered.xml,57,0,202608190570,failed,timeout,solve_timeout,250.03857200,0,0.00000000, +offset_pillar,scene_cluttered.xml,57,1,202608190571,failed,timeout,solve_timeout,250.05582200,0,0.00000000, +offset_pillar,scene_cluttered.xml,57,2,202608190572,failed,timeout,solve_timeout,250.06161200,0,0.00000000, +offset_pillar,scene_cluttered.xml,57,3,202608190573,failed,timeout,solve_timeout,250.05350200,0,0.00000000, +offset_pillar,scene_cluttered.xml,57,4,202608190574,failed,timeout,solve_timeout,250.07351200,0,0.00000000, +offset_pillar,scene_cluttered.xml,58,0,202608190580,failed,timeout,solve_timeout,250.00812200,0,0.00000000, +offset_pillar,scene_cluttered.xml,58,1,202608190581,failed,timeout,solve_timeout,250.05620200,0,0.00000000, +offset_pillar,scene_cluttered.xml,58,2,202608190582,failed,timeout,solve_timeout,250.01616200,0,0.00000000, +offset_pillar,scene_cluttered.xml,58,3,202608190583,failed,timeout,solve_timeout,250.01074200,0,0.00000000, +offset_pillar,scene_cluttered.xml,58,4,202608190584,failed,timeout,solve_timeout,250.02295200,0,0.00000000, +offset_pillar,scene_cluttered.xml,59,0,202608190590,failed,timeout,solve_timeout,250.01960200,0,0.00000000, +offset_pillar,scene_cluttered.xml,59,1,202608190591,failed,timeout,solve_timeout,250.03401200,0,0.00000000, +offset_pillar,scene_cluttered.xml,59,2,202608190592,failed,timeout,solve_timeout,250.06169200,0,0.00000000, +offset_pillar,scene_cluttered.xml,59,3,202608190593,failed,timeout,solve_timeout,250.04316200,0,0.00000000, +offset_pillar,scene_cluttered.xml,59,4,202608190594,failed,timeout,solve_timeout,250.00159200,0,0.00000000, +offset_pillar,scene_cluttered.xml,60,0,202608190600,failed,timeout,solve_timeout,250.05674200,0,0.00000000, +offset_pillar,scene_cluttered.xml,60,1,202608190601,failed,timeout,solve_timeout,250.01279200,0,0.00000000, +offset_pillar,scene_cluttered.xml,60,2,202608190602,failed,timeout,solve_timeout,250.01506200,0,0.00000000, +offset_pillar,scene_cluttered.xml,60,3,202608190603,failed,timeout,solve_timeout,250.01846200,0,0.00000000, +offset_pillar,scene_cluttered.xml,60,4,202608190604,failed,timeout,solve_timeout,250.04931200,0,0.00000000, +offset_pillar,scene_cluttered.xml,61,0,202608190610,success,success,validated,10.04575300,85,20.95066402,64d2d1ed842962ea73ab45d0e7e173690f4755640d011194e16733eb6a342e3e +offset_pillar,scene_cluttered.xml,61,1,202608190611,success,success,validated,9.87371300,124,30.72821214,6d8a2d71d1bd67de06458608aa916dfea4551d5a6f45313b7182aed1d1ef1c8c +offset_pillar,scene_cluttered.xml,61,2,202608190612,success,success,validated,12.99700800,74,17.81100009,89cc6591f4879d8726b8cb3d3f3fb2f548b8750d19ed7137a18fb3dd13e0f4ee +offset_pillar,scene_cluttered.xml,61,3,202608190613,success,success,validated,8.68779500,89,21.97826603,c7d9b29d0211cddedf77aa8d016d0cf9796678c7611cebb1730ce18c8e2e6ae8 +offset_pillar,scene_cluttered.xml,61,4,202608190614,success,success,validated,7.73916700,90,22.23765540,3a4917f1c1cd045349aaef57763e6d713f64d8e15b74d6c1d9b3632fd43b27f6 +offset_pillar,scene_cluttered.xml,64,0,202608190640,success,success,validated,15.25153500,158,39.25000000,9881fda73fa076f7c6c4db1d86136d7fb87a8bf8ea3b6d44ff26884781faf476 +offset_pillar,scene_cluttered.xml,64,1,202608190641,success,success,validated,10.78231200,143,35.50000000,abb1d3e02f8e3d07b9e07f8e4ad15c654850383ceb0bf17b2757034d899fe1af +offset_pillar,scene_cluttered.xml,64,2,202608190642,success,success,validated,7.69892700,101,25.00000000,5ba123d9de867a08848df95d0db828782ef313ff0da37e7845e99a78d7c37104 +offset_pillar,scene_cluttered.xml,64,3,202608190643,success,success,validated,10.43841300,120,29.75000000,030f55c8eff5752aedbcfb77961125644487a96fd631d51694719141eb6a12e4 +offset_pillar,scene_cluttered.xml,64,4,202608190644,success,success,validated,10.55554200,145,36.00000000,1ea97887ab325e36826d515c3ebac4c8ce21e19d8f6c64fd520886fcd2635092 +offset_pillar,scene_cluttered.xml,66,0,202608190660,failed,timeout,solve_timeout,250.01202200,0,0.00000000, +offset_pillar,scene_cluttered.xml,66,1,202608190661,failed,timeout,solve_timeout,250.01156200,0,0.00000000, +offset_pillar,scene_cluttered.xml,66,2,202608190662,failed,timeout,solve_timeout,250.02187200,0,0.00000000, +offset_pillar,scene_cluttered.xml,66,3,202608190663,failed,timeout,solve_timeout,250.00267300,0,0.00000000, +offset_pillar,scene_cluttered.xml,66,4,202608190664,failed,timeout,solve_timeout,250.01283200,0,0.00000000, +offset_pillar,scene_cluttered.xml,68,0,202608190680,failed,timeout,solve_timeout,250.00408200,0,0.00000000, +offset_pillar,scene_cluttered.xml,68,1,202608190681,failed,timeout,solve_timeout,250.00839200,0,0.00000000, +offset_pillar,scene_cluttered.xml,68,2,202608190682,failed,timeout,solve_timeout,250.00191200,0,0.00000000, +offset_pillar,scene_cluttered.xml,68,3,202608190683,failed,timeout,solve_timeout,250.01193200,0,0.00000000, +offset_pillar,scene_cluttered.xml,68,4,202608190684,failed,timeout,solve_timeout,250.02561200,0,0.00000000, +offset_pillar,scene_cluttered.xml,69,0,202608190690,failed,timeout,solve_timeout,250.04439200,0,0.00000000, +offset_pillar,scene_cluttered.xml,69,1,202608190691,failed,timeout,solve_timeout,250.02089200,0,0.00000000, +offset_pillar,scene_cluttered.xml,69,2,202608190692,failed,timeout,solve_timeout,250.03870200,0,0.00000000, +offset_pillar,scene_cluttered.xml,69,3,202608190693,failed,timeout,solve_timeout,250.00575200,0,0.00000000, +offset_pillar,scene_cluttered.xml,69,4,202608190694,failed,timeout,solve_timeout,250.00431200,0,0.00000000, +offset_pillar,scene_cluttered.xml,70,0,202608190700,success,success,validated,39.33839400,129,31.99999663,d69a928bc1f97bfe9df3f746043e2b94c4d6a9f3dc5f190516b2dcadceec6c14 +offset_pillar,scene_cluttered.xml,70,1,202608190701,failed,success,validated,23.60368000,132,32.74433426,68493b57e04b27b9dae33d7bff440097029777969590f02e6ba299e1a4411e87 +offset_pillar,scene_cluttered.xml,70,2,202608190702,success,success,validated,21.73708400,167,41.50000000,3fe2c07278109dad39ef01e05b8229382f67fe3a85cf852c1329fe53c70f3021 +offset_pillar,scene_cluttered.xml,70,3,202608190703,failed,success,validated,8.73312600,71,17.37238231,bd9c2811faecfbe9d9c6e40c3f7984057caad4a96fa029d60d306e2791d73060 +offset_pillar,scene_cluttered.xml,70,4,202608190704,success,success,validated,29.22476100,129,31.99897423,5ee95529a41c533aad09444457a9e210a819a7e51d13d9627892dc249c470a89 +offset_pillar,scene_cluttered.xml,72,0,202608190720,success,success,validated,2.51828600,64,15.73841493,66fe88d87f204cd291ffb299e7ccc810de4af3e3e3311bca113130f08304e404 +offset_pillar,scene_cluttered.xml,72,1,202608190721,success,success,validated,9.26566500,110,27.25000000,18b819be453e2387a03f058d12cdac3bf046d09694db9b076039dbf399ea1a52 +offset_pillar,scene_cluttered.xml,72,2,202608190722,success,success,validated,7.08390800,78,19.24998205,1f84ac85c23c8b285ff9851d8fd8676833c95c7232e9b569924451737388897a +offset_pillar,scene_cluttered.xml,72,3,202608190723,success,success,validated,2.36949600,56,13.70879636,aef204d66970ce4a989f3a72bbf20accaec0a7e7499089c896f23301b153594c +offset_pillar,scene_cluttered.xml,72,4,202608190724,success,success,validated,12.26662000,84,20.74999954,12ecc3a465d8029edc0bc44c37454cff7daa7bd9f3b14f8e16aeb46cc4f31981 +offset_pillar,scene_cluttered.xml,73,0,202608190730,failed,timeout,solve_timeout,250.05805200,0,0.00000000, +offset_pillar,scene_cluttered.xml,73,1,202608190731,failed,timeout,solve_timeout,250.06062200,0,0.00000000, +offset_pillar,scene_cluttered.xml,73,2,202608190732,failed,timeout,solve_timeout,250.04523100,0,0.00000000, +offset_pillar,scene_cluttered.xml,73,3,202608190733,failed,timeout,solve_timeout,250.01584200,0,0.00000000, +offset_pillar,scene_cluttered.xml,73,4,202608190734,failed,timeout,solve_timeout,250.00135200,0,0.00000000, +offset_pillar,scene_cluttered.xml,74,0,202608190740,failed,success,validated,23.80649000,135,33.44838003,228ab1097263d9d79be91ad44e9bcc48ea9d7165890cd662ae983b09926c33b9 +offset_pillar,scene_cluttered.xml,74,1,202608190741,failed,success,validated,77.28176100,201,49.93892094,ed007d70a3e57bc8c2ae577848509c78a1d3d0f7b41730d2aaaa07ad2af5198f +offset_pillar,scene_cluttered.xml,74,2,202608190742,failed,success,validated,100.55402200,150,37.24726737,73e24d465d69fe4f53dc2abc9a22dcc68b4f855b714510e9e3b0401c97ee2f6b +offset_pillar,scene_cluttered.xml,74,3,202608190743,failed,success,validated,94.16805300,100,24.73538807,054d0b3121a58346b0b39c70a7e1600bcb21b8283d756f0aad17ac0f87a2d348 +offset_pillar,scene_cluttered.xml,74,4,202608190744,failed,success,validated,107.38960100,119,29.49293392,0d616f8bd13a965342cc4e69bdf09f48e1a81f89d2fe200ed9b0d9c7c6af3c7d +offset_pillar,scene_cluttered.xml,75,0,202608190750,failed,timeout,solve_timeout,250.05435200,0,0.00000000, +offset_pillar,scene_cluttered.xml,75,1,202608190751,failed,timeout,solve_timeout,250.02463200,0,0.00000000, +offset_pillar,scene_cluttered.xml,75,2,202608190752,failed,timeout,solve_timeout,250.00294200,0,0.00000000, +offset_pillar,scene_cluttered.xml,75,3,202608190753,failed,timeout,solve_timeout,250.00732200,0,0.00000000, +offset_pillar,scene_cluttered.xml,75,4,202608190754,failed,timeout,solve_timeout,250.00365200,0,0.00000000, +offset_pillar,scene_cluttered.xml,77,0,202608190770,failed,timeout,solve_timeout,250.00635200,0,0.00000000, +offset_pillar,scene_cluttered.xml,77,1,202608190771,failed,timeout,solve_timeout,250.01899200,0,0.00000000, +offset_pillar,scene_cluttered.xml,77,2,202608190772,failed,timeout,solve_timeout,250.05854200,0,0.00000000, +offset_pillar,scene_cluttered.xml,77,3,202608190773,failed,timeout,solve_timeout,250.01001200,0,0.00000000, +offset_pillar,scene_cluttered.xml,77,4,202608190774,failed,timeout,solve_timeout,250.01351200,0,0.00000000, +offset_pillar,scene_cluttered.xml,78,0,202608190780,failed,timeout,solve_timeout,250.04033200,0,0.00000000, +offset_pillar,scene_cluttered.xml,78,1,202608190781,failed,timeout,solve_timeout,250.02394200,0,0.00000000, +offset_pillar,scene_cluttered.xml,78,2,202608190782,failed,timeout,solve_timeout,250.01360200,0,0.00000000, +offset_pillar,scene_cluttered.xml,78,3,202608190783,failed,timeout,solve_timeout,250.00666200,0,0.00000000, +offset_pillar,scene_cluttered.xml,78,4,202608190784,failed,timeout,solve_timeout,250.02376200,0,0.00000000, +offset_pillar,scene_cluttered.xml,80,0,202608190800,failed,timeout,solve_timeout,250.01450200,0,0.00000000, +offset_pillar,scene_cluttered.xml,80,1,202608190801,failed,timeout,solve_timeout,250.06241200,0,0.00000000, +offset_pillar,scene_cluttered.xml,80,2,202608190802,failed,timeout,solve_timeout,250.00511200,0,0.00000000, +offset_pillar,scene_cluttered.xml,80,3,202608190803,failed,timeout,solve_timeout,250.03141200,0,0.00000000, +offset_pillar,scene_cluttered.xml,80,4,202608190804,failed,timeout,solve_timeout,250.07344200,0,0.00000000, +offset_pillar,scene_cluttered.xml,81,0,202608190810,failed,timeout,solve_timeout,250.00043300,0,0.00000000, +offset_pillar,scene_cluttered.xml,81,1,202608190811,failed,timeout,solve_timeout,250.01339200,0,0.00000000, +offset_pillar,scene_cluttered.xml,81,2,202608190812,failed,timeout,solve_timeout,250.01017200,0,0.00000000, +offset_pillar,scene_cluttered.xml,81,3,202608190813,failed,timeout,solve_timeout,250.02496200,0,0.00000000, +offset_pillar,scene_cluttered.xml,81,4,202608190814,failed,timeout,solve_timeout,250.02101200,0,0.00000000, +offset_pillar,scene_cluttered.xml,83,0,202608190830,success,success,validated,4.78986200,94,23.24999671,a0f30d7eb505d4f72e6c1c528518b6c60fe29bff369dbb4e61eafb86079b367d +offset_pillar,scene_cluttered.xml,83,1,202608190831,success,success,validated,3.28679400,78,19.24995215,f731636c4f75b4a8424444a6c5512d10afb8dee152741a93da96ebd66ddf43d8 +offset_pillar,scene_cluttered.xml,83,2,202608190832,success,success,validated,4.38402300,76,18.74408936,ef448ba6aa18ecd119d32b35359301885cedbe86bc4b261c63ddd228d81c58aa +offset_pillar,scene_cluttered.xml,83,3,202608190833,success,success,validated,3.15477400,58,14.19817261,2437c44bc84d1d43fa7efc74629bee923772d6a30f70dd894f926be0b407e813 +offset_pillar,scene_cluttered.xml,83,4,202608190834,success,success,validated,3.61058400,70,17.18320503,4c3aaa461e1a6a4b5521d9492c340054226b9f9345b7325e7766ed22d552fda1 +offset_pillar,scene_cluttered.xml,85,0,202608190850,failed,timeout,solve_timeout,250.01510200,0,0.00000000, +offset_pillar,scene_cluttered.xml,85,1,202608190851,failed,timeout,solve_timeout,250.01216200,0,0.00000000, +offset_pillar,scene_cluttered.xml,85,2,202608190852,failed,timeout,solve_timeout,250.00081200,0,0.00000000, +offset_pillar,scene_cluttered.xml,85,3,202608190853,failed,timeout,solve_timeout,250.01386200,0,0.00000000, +offset_pillar,scene_cluttered.xml,85,4,202608190854,failed,timeout,solve_timeout,250.01692200,0,0.00000000, +offset_pillar,scene_cluttered.xml,86,0,202608190860,failed,timeout,solve_timeout,250.01352200,0,0.00000000, +offset_pillar,scene_cluttered.xml,86,1,202608190861,failed,timeout,solve_timeout,250.01045200,0,0.00000000, +offset_pillar,scene_cluttered.xml,86,2,202608190862,failed,timeout,solve_timeout,250.04048200,0,0.00000000, +offset_pillar,scene_cluttered.xml,86,3,202608190863,failed,timeout,solve_timeout,250.04917200,0,0.00000000, +offset_pillar,scene_cluttered.xml,86,4,202608190864,failed,timeout,solve_timeout,250.01242200,0,0.00000000, +offset_pillar,scene_cluttered.xml,87,0,202608190870,success,success,validated,5.99364000,106,26.24999998,0d042b2613fa19e1d0e66f727e9d59e5150ea5085649405cb434a1fc6cd35a95 +offset_pillar,scene_cluttered.xml,87,1,202608190871,success,success,validated,8.09124700,120,29.75000000,56cd2183ada22abc860acd40dc57d502f18d699799721f61794a74a4f3e2959a +offset_pillar,scene_cluttered.xml,87,2,202608190872,success,success,validated,6.54665900,95,23.49999986,44750d72815290a730a18143df63d711a6fd23e43e1396b611438148cd62d85a +offset_pillar,scene_cluttered.xml,87,3,202608190873,success,success,validated,4.23418300,86,21.24999942,5e9cb516225272f71662524b56b33641d0d35860e5789a1cde0358b95dca4c41 +offset_pillar,scene_cluttered.xml,87,4,202608190874,success,success,validated,6.33557000,119,29.50000000,3695c4fbabb907d8d44315333a87ef90f0d24f758359df35c67f3dfd79c83356 +offset_pillar,scene_cluttered.xml,88,0,202608190880,success,success,validated,21.87499300,86,20.78723811,99a40dd0c81bc8d415fda2f31071cb99419354422afad99239a22f37b44e2729 +offset_pillar,scene_cluttered.xml,88,1,202608190881,success,success,validated,9.88015300,129,31.99999993,f063e217e9bc527b04de82c7dbfd34e2f46189a0e2c65aee316864e811db83b0 +offset_pillar,scene_cluttered.xml,88,2,202608190882,success,success,validated,4.08648300,83,20.49557250,da8e6ea4dd9f8bb7d8aa184b6591349a3fd2bd87930f761d8ec481df0040b22c +offset_pillar,scene_cluttered.xml,88,3,202608190883,success,success,validated,8.56832600,106,26.24482243,4a65416730e4026350b1d78db1cd76915692c1b2e8f81aea05de2e8a48eba298 +offset_pillar,scene_cluttered.xml,88,4,202608190884,success,success,validated,7.09005800,108,26.74508266,22a030fb2659b5b9a586b44f0fb440fa2083679e5b4fc2878adebc4c71c355ba +offset_pillar,scene_cluttered.xml,89,0,202608190890,success,success,validated,7.42616700,97,23.99999999,ee81e52058fa47b906f41c25b5b3f0ef4fc5b6449663a9668a1afd586e41a7d3 +offset_pillar,scene_cluttered.xml,89,1,202608190891,success,success,validated,7.37997800,132,32.75000000,42cb641514ec9248c6ea639ad1b2019b0ffeb7d99378a417d9a8c2dc3cd22015 +offset_pillar,scene_cluttered.xml,89,2,202608190892,success,success,validated,3.97267300,48,11.74950787,7c6b12fbcf1def7bd1b1288c6fcfd129435624ea25974195fa73f008e60c1337 +offset_pillar,scene_cluttered.xml,89,3,202608190893,success,success,validated,8.75003500,118,29.25000000,4e9548f5c8aae8e688d1a764f25afedefa750662524683491153fbc036c5efc8 +offset_pillar,scene_cluttered.xml,89,4,202608190894,success,success,validated,10.11900300,73,17.96547131,59f98b6d8f89f2f2fae2caa0cec8ccc4f0ce3bb97509ad64d3d35afae2e3d2ff +offset_pillar,scene_cluttered.xml,91,0,202608190910,failed,timeout,solve_timeout,250.04747200,0,0.00000000, +offset_pillar,scene_cluttered.xml,91,1,202608190911,failed,timeout,solve_timeout,250.00147200,0,0.00000000, +offset_pillar,scene_cluttered.xml,91,2,202608190912,failed,timeout,solve_timeout,250.01394200,0,0.00000000, +offset_pillar,scene_cluttered.xml,91,3,202608190913,failed,timeout,solve_timeout,250.03384200,0,0.00000000, +offset_pillar,scene_cluttered.xml,91,4,202608190914,failed,timeout,solve_timeout,250.00332200,0,0.00000000, +offset_pillar,scene_cluttered.xml,92,0,202608190920,failed,timeout,solve_timeout,250.01330200,0,0.00000000, +offset_pillar,scene_cluttered.xml,92,1,202608190921,failed,timeout,solve_timeout,250.05507200,0,0.00000000, +offset_pillar,scene_cluttered.xml,92,2,202608190922,failed,timeout,solve_timeout,250.01099200,0,0.00000000, +offset_pillar,scene_cluttered.xml,92,3,202608190923,failed,timeout,solve_timeout,250.00182200,0,0.00000000, +offset_pillar,scene_cluttered.xml,92,4,202608190924,failed,timeout,solve_timeout,250.00858200,0,0.00000000, +offset_pillar,scene_cluttered.xml,94,0,202608190940,failed,timeout,solve_timeout,250.01239200,0,0.00000000, +offset_pillar,scene_cluttered.xml,94,1,202608190941,failed,timeout,solve_timeout,250.02984200,0,0.00000000, +offset_pillar,scene_cluttered.xml,94,2,202608190942,failed,timeout,solve_timeout,250.05648200,0,0.00000000, +offset_pillar,scene_cluttered.xml,94,3,202608190943,failed,timeout,solve_timeout,250.04671200,0,0.00000000, +offset_pillar,scene_cluttered.xml,94,4,202608190944,failed,timeout,solve_timeout,250.06637200,0,0.00000000, +offset_pillar,scene_cluttered.xml,95,0,202608190950,failed,timeout,solve_timeout,250.01632300,0,0.00000000, +offset_pillar,scene_cluttered.xml,95,1,202608190951,failed,timeout,solve_timeout,250.02770200,0,0.00000000, +offset_pillar,scene_cluttered.xml,95,2,202608190952,failed,timeout,solve_timeout,250.05518200,0,0.00000000, +offset_pillar,scene_cluttered.xml,95,3,202608190953,failed,timeout,solve_timeout,250.02399200,0,0.00000000, +offset_pillar,scene_cluttered.xml,95,4,202608190954,failed,timeout,solve_timeout,250.01691200,0,0.00000000, +offset_pillar,scene_cluttered.xml,96,0,202608190960,failed,timeout,solve_timeout,250.01279200,0,0.00000000, +offset_pillar,scene_cluttered.xml,96,1,202608190961,failed,timeout,solve_timeout,250.01609200,0,0.00000000, +offset_pillar,scene_cluttered.xml,96,2,202608190962,failed,timeout,solve_timeout,250.04179200,0,0.00000000, +offset_pillar,scene_cluttered.xml,96,3,202608190963,failed,timeout,solve_timeout,250.05880200,0,0.00000000, +offset_pillar,scene_cluttered.xml,96,4,202608190964,failed,timeout,solve_timeout,250.04236200,0,0.00000000, +offset_pillar,scene_cluttered.xml,97,0,202608190970,success,success,validated,9.91970300,116,28.69177959,02975dbdad9c7e2def0191bc4f77fc2d73424b2b4c9e75b19da244867407b28e +offset_pillar,scene_cluttered.xml,97,1,202608190971,success,success,validated,8.08492600,114,28.20905277,6a55746d48c2916c18794c99dcf396fe25384021c1d55e7f5c4ae7d4e8893d61 +offset_pillar,scene_cluttered.xml,97,2,202608190972,success,success,validated,10.48130200,109,26.98469743,6d418797eafd8e7136022d3db6ed237cbca337ed4ad5f9c779dc9cac56777667 +offset_pillar,scene_cluttered.xml,97,3,202608190973,success,success,validated,10.32140200,102,25.22204204,4e869de17799dd59476b355d9d3f2e8a81d491e45102b080df467fd6ee6f9f41 +offset_pillar,scene_cluttered.xml,97,4,202608190974,success,success,validated,17.99342000,142,35.24999896,899e112ed76651b71d2e35f40d1fa10c8427a5a4523898e820e93ea7ae3421a9 +offset_pillar,scene_cluttered.xml,98,0,202608190980,success,success,validated,4.11125400,65,15.94035323,7ac2c6b9f10d9d091dfb201db56b8f4ebf1fbc960b853ea4fc43e9f98f9117c2 +offset_pillar,scene_cluttered.xml,98,1,202608190981,success,success,validated,9.37511500,123,30.50000000,56cc945244313b7f8549c3b5254640158659a2555cff87185c56354bf94ceb94 +offset_pillar,scene_cluttered.xml,98,2,202608190982,success,success,validated,4.75325200,92,22.74994544,c5f6b52749a9d248c89b74efc01719389dc5ae6a29a20a964ddb1b74a6e38fb2 +offset_pillar,scene_cluttered.xml,98,3,202608190983,success,success,validated,11.36668100,58,13.92814863,b7ea65e8fd7f73da6bebdbaeca192bb022f0bfa1a5a8d69c9fff3cc68b7a9e21 +offset_pillar,scene_cluttered.xml,98,4,202608190984,success,success,validated,10.21759300,56,13.41827210,2ced5a07956a3348aaa3af917b761959fde67511201db195adc4854fc6c2b7d1 +offset_pillar,scene_cluttered.xml,99,0,202608190990,failed,timeout,solve_timeout,250.00772200,0,0.00000000, +offset_pillar,scene_cluttered.xml,99,1,202608190991,failed,timeout,solve_timeout,250.02608200,0,0.00000000, +offset_pillar,scene_cluttered.xml,99,2,202608190992,failed,timeout,solve_timeout,250.00820200,0,0.00000000, +offset_pillar,scene_cluttered.xml,99,3,202608190993,failed,timeout,solve_timeout,250.00580200,0,0.00000000, +offset_pillar,scene_cluttered.xml,99,4,202608190994,failed,timeout,solve_timeout,250.00733200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,0,0,202608200000,failed,timeout,solve_timeout,250.04766200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,0,1,202608200001,failed,timeout,solve_timeout,250.00534200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,0,2,202608200002,failed,timeout,solve_timeout,250.03738200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,0,3,202608200003,failed,timeout,solve_timeout,250.00168200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,0,4,202608200004,failed,timeout,solve_timeout,250.06652200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,1,0,202608200010,failed,timeout,solve_timeout,250.02014100,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,1,1,202608200011,failed,timeout,solve_timeout,250.03466200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,1,2,202608200012,failed,timeout,solve_timeout,250.02756200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,1,3,202608200013,failed,timeout,solve_timeout,250.02536200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,1,4,202608200014,failed,timeout,solve_timeout,250.03239200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,2,0,202608200020,success,success,validated,17.69043000,148,36.75000000,8817919e90d2b18f2d62634241bbffb6c4c56d2f06f38536a5e7ab9941311b3c +tabletop_pillar,scene_pickplace.xml,2,1,202608200021,success,success,validated,14.19728600,148,36.75000000,1666ad786c50d3dc453c57e8a54f7d8396f434e03439b2b1874af721e985a3db +tabletop_pillar,scene_pickplace.xml,2,2,202608200022,success,success,validated,5.07552100,96,23.74999984,fea5dc6f014f40a2315fb9149c8195a6f62775daaf7261f33160086ed4be3064 +tabletop_pillar,scene_pickplace.xml,2,3,202608200023,success,success,validated,16.41467200,127,31.50000000,402eab56e91fbfec0f32ee97a50db5097cefc621d9ea5a5b09291503eb3890bf +tabletop_pillar,scene_pickplace.xml,2,4,202608200024,success,success,validated,3.05224500,73,17.96749190,b96ebbf9e7aaa18e7b75be77ae7b8c591b1571a4ae0d5fb23438c9bc4b77024e +tabletop_pillar,scene_pickplace.xml,3,0,202608200030,failed,timeout,solve_timeout,250.01226200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,3,1,202608200031,failed,timeout,solve_timeout,250.04038300,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,3,2,202608200032,failed,timeout,solve_timeout,250.01458200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,3,3,202608200033,failed,timeout,solve_timeout,250.00095200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,3,4,202608200034,failed,timeout,solve_timeout,250.07142200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,4,0,202608200040,failed,timeout,solve_timeout,250.04805200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,4,1,202608200041,failed,timeout,solve_timeout,250.01253200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,4,2,202608200042,failed,timeout,solve_timeout,250.07828200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,4,3,202608200043,failed,timeout,solve_timeout,250.00098200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,4,4,202608200044,failed,timeout,solve_timeout,250.02926200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,6,0,202608200060,success,success,validated,8.79999500,80,19.71522554,dca266372e8dfb199d943dd9f251c1e981a3bfd0914f732c5447449400b07cd1 +tabletop_pillar,scene_pickplace.xml,6,1,202608200061,success,success,validated,6.90668800,92,22.74999998,dc22c54f83b9e0330b187115bd1258c513c650d620f5cce83bfa7462b9e1b77d +tabletop_pillar,scene_pickplace.xml,6,2,202608200062,success,success,validated,23.81989000,152,37.75000000,71fd4179558e35bc02c6cca9d51a2789caf88156789742deaf9c622e763b7f00 +tabletop_pillar,scene_pickplace.xml,6,3,202608200063,success,success,validated,9.27695400,115,28.49813793,de82e0c5b3a2e3f5c3d76ebb2eba43622e656a413bd233502e3b1a1e2f4944c1 +tabletop_pillar,scene_pickplace.xml,6,4,202608200064,success,success,validated,12.05190000,99,24.49999763,eb85af391e1b0b280f6b659dcc543f233fabcd3447f72dac07aefb4e6f2559a0 +tabletop_pillar,scene_pickplace.xml,8,0,202608200080,failed,timeout,solve_timeout,250.00665200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,8,1,202608200081,failed,timeout,solve_timeout,250.00476200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,8,2,202608200082,failed,timeout,solve_timeout,250.00332200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,8,3,202608200083,failed,timeout,solve_timeout,250.01197200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,8,4,202608200084,failed,timeout,solve_timeout,250.01813300,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,11,0,202608200110,failed,timeout,solve_timeout,250.05180200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,11,1,202608200111,failed,timeout,solve_timeout,250.00700200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,11,2,202608200112,failed,timeout,solve_timeout,250.05509200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,11,3,202608200113,failed,timeout,solve_timeout,250.01146200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,11,4,202608200114,failed,timeout,solve_timeout,250.07695200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,12,0,202608200120,failed,timeout,solve_timeout,250.01041200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,12,1,202608200121,failed,timeout,solve_timeout,250.02957200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,12,2,202608200122,failed,timeout,solve_timeout,250.05806300,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,12,3,202608200123,failed,timeout,solve_timeout,250.02608200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,12,4,202608200124,failed,timeout,solve_timeout,250.00706200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,14,0,202608200140,success,success,validated,9.06011500,71,17.47244011,de6d5210c32d5df2a2c651ca26a8183a8a1381a5dfc3abf8cf34f718040aeb79 +tabletop_pillar,scene_pickplace.xml,14,1,202608200141,success,success,validated,7.89965600,75,18.39322296,4de737d64c278fbb685fd4420e8391924b29b78e89cc82c5650cc80f7fb2df60 +tabletop_pillar,scene_pickplace.xml,14,2,202608200142,success,success,validated,6.82587900,99,24.49999963,cb27645d81f953244c2d1d4b305310eaa549c83adbf82957fbfc59c949237cca +tabletop_pillar,scene_pickplace.xml,14,3,202608200143,success,success,validated,10.50370200,116,28.75000000,92a093e4b043b590a4be261a1f19fd83db7c6b6144aece58b1fb383da87c854a +tabletop_pillar,scene_pickplace.xml,14,4,202608200144,success,success,validated,7.36393700,59,14.48954717,8d8ca1a1989ffadee586da9c193c1d36d8e2291f88d1345c56ff5b591b0c50ed +tabletop_pillar,scene_pickplace.xml,15,0,202608200150,failed,timeout,solve_timeout,250.05110200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,15,1,202608200151,failed,timeout,solve_timeout,250.04030200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,15,2,202608200152,failed,timeout,solve_timeout,250.00254200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,15,3,202608200153,failed,timeout,solve_timeout,250.00426200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,15,4,202608200154,failed,timeout,solve_timeout,250.03565200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,18,0,202608200180,failed,timeout,solve_timeout,250.01247200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,18,1,202608200181,failed,timeout,solve_timeout,250.03769200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,18,2,202608200182,failed,timeout,solve_timeout,250.00497200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,18,3,202608200183,failed,timeout,solve_timeout,250.01961200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,18,4,202608200184,failed,timeout,solve_timeout,250.07187200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,19,0,202608200190,failed,timeout,solve_timeout,250.02775200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,19,1,202608200191,failed,timeout,solve_timeout,250.03817200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,19,2,202608200192,failed,timeout,solve_timeout,250.02030200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,19,3,202608200193,failed,timeout,solve_timeout,250.04078200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,19,4,202608200194,failed,timeout,solve_timeout,250.04937200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,20,0,202608200200,failed,timeout,solve_timeout,250.01505200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,20,1,202608200201,failed,timeout,solve_timeout,250.00342300,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,20,2,202608200202,failed,timeout,solve_timeout,250.00432300,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,20,3,202608200203,failed,timeout,solve_timeout,250.00523300,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,20,4,202608200204,failed,timeout,solve_timeout,250.01992200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,22,0,202608200220,success,success,validated,9.14464500,104,25.75000000,5352d3398ff76624181fc148ae68e8433257df4c8bfe07285ef638809fc8beab +tabletop_pillar,scene_pickplace.xml,22,1,202608200221,success,success,validated,2.22960600,52,12.72696321,8eb424479558483c7306214c25e543d592f7bcb08fbe3293395917514b25b708 +tabletop_pillar,scene_pickplace.xml,22,2,202608200222,success,success,validated,10.61770200,122,30.25000000,8d6ceb31c4faa69d102ab764be9cb3c968ffb95d984da1d9ee6ca1995656a99e +tabletop_pillar,scene_pickplace.xml,22,3,202608200223,success,success,validated,11.56009000,118,29.25000000,5e4a747b8c34f4a0b3f7c1402731b684ff84cd4d37d4ffbb933ae373069d6f59 +tabletop_pillar,scene_pickplace.xml,22,4,202608200224,success,success,validated,7.64577800,95,23.50000000,2b8bc597106d1550f75744f40e7de90f3a6c42bd42796a65d9c0d3cae1663499 +tabletop_pillar,scene_pickplace.xml,23,0,202608200230,failed,timeout,solve_timeout,250.00244200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,23,1,202608200231,failed,timeout,solve_timeout,250.01417200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,23,2,202608200232,failed,timeout,solve_timeout,250.02289200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,23,3,202608200233,failed,timeout,solve_timeout,250.02384100,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,23,4,202608200234,failed,timeout,solve_timeout,250.01616200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,24,0,202608200240,failed,timeout,solve_timeout,250.00929200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,24,1,202608200241,failed,timeout,solve_timeout,250.01021200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,24,2,202608200242,failed,timeout,solve_timeout,250.02575200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,24,3,202608200243,failed,timeout,solve_timeout,250.03391200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,24,4,202608200244,failed,timeout,solve_timeout,250.06708200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,25,0,202608200250,success,success,validated,1.86876700,46,11.21451767,edd5ca45158d849fa5fc90c3879b960fd4b63a1f6c944c149c4fcaf1b3d694d9 +tabletop_pillar,scene_pickplace.xml,25,1,202608200251,success,success,validated,5.30770100,85,20.99998781,18967d35a4fdc362049b6076bb4d7b4e86097820e92217511d73abe508e29fca +tabletop_pillar,scene_pickplace.xml,25,2,202608200252,success,success,validated,2.04933600,45,10.98083065,0febc9c5c71b446cabc90c5d6eae115e5aa46b5d67422c6535c7cf243cd67b63 +tabletop_pillar,scene_pickplace.xml,25,3,202608200253,success,success,validated,8.03019600,84,20.74630575,9a27e643dfcfe21ee3d833bf24e6e03278fdf777bfc8d931cad9dc6932899103 +tabletop_pillar,scene_pickplace.xml,25,4,202608200254,success,success,validated,2.05639600,47,11.45941780,6061ba13f98fe3f544d53e4e875424ecf08f362beaad38dc74baee685c9fae36 +tabletop_pillar,scene_pickplace.xml,26,0,202608200260,failed,timeout,solve_timeout,250.00123200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,26,1,202608200261,failed,timeout,solve_timeout,250.01669200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,26,2,202608200262,failed,timeout,solve_timeout,250.01374200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,26,3,202608200263,failed,timeout,solve_timeout,250.03070200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,26,4,202608200264,failed,timeout,solve_timeout,250.00957200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,27,0,202608200270,success,success,validated,22.24321300,99,24.47221228,fb283232418849dd1732e5aee12220d7161142c801aad67f6efa6e0acab9d8b3 +tabletop_pillar,scene_pickplace.xml,27,1,202608200271,success,success,validated,23.44302000,117,28.99472756,a9bc40cbb746fa52a8def4d92652e8e81d37b257ee5214c94b7946cb4491794c +tabletop_pillar,scene_pickplace.xml,27,2,202608200272,success,success,validated,35.57577100,151,37.46469314,2707c5dbd129de17222794e92d14c658a650e4315dc299f9df820422b01d85f1 +tabletop_pillar,scene_pickplace.xml,27,3,202608200273,success,success,validated,20.51343600,126,31.21214060,168720e7a73e4bb7008fc22c5f4f88bbfdfe5d723fdb5d1e62cc5b8481aeac75 +tabletop_pillar,scene_pickplace.xml,27,4,202608200274,success,success,validated,10.16352300,74,18.23380031,708daabd820b322b5418822e2c94b3d1c5159e0071878d397bc65b1e70645886 +tabletop_pillar,scene_pickplace.xml,28,0,202608200280,failed,timeout,solve_timeout,250.00605200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,28,1,202608200281,failed,timeout,solve_timeout,250.05904200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,28,2,202608200282,failed,timeout,solve_timeout,250.00429200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,28,3,202608200283,failed,timeout,solve_timeout,250.03618100,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,28,4,202608200284,failed,timeout,solve_timeout,250.02835200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,30,0,202608200300,failed,timeout,solve_timeout,250.03693200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,30,1,202608200301,failed,timeout,solve_timeout,250.03371200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,30,2,202608200302,failed,timeout,solve_timeout,250.04241200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,30,3,202608200303,failed,timeout,solve_timeout,250.02572200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,30,4,202608200304,failed,timeout,solve_timeout,250.07159200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,31,0,202608200310,failed,timeout,solve_timeout,250.05004200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,31,1,202608200311,failed,timeout,solve_timeout,250.05888200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,31,2,202608200312,failed,timeout,solve_timeout,250.00314200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,31,3,202608200313,failed,timeout,solve_timeout,250.01392200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,31,4,202608200314,failed,timeout,solve_timeout,250.01198200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,33,0,202608200330,failed,timeout,solve_timeout,250.00176200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,33,1,202608200331,failed,timeout,solve_timeout,250.00191200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,33,2,202608200332,failed,timeout,solve_timeout,250.00355200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,33,3,202608200333,failed,timeout,solve_timeout,250.02196200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,33,4,202608200334,failed,timeout,solve_timeout,250.00562300,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,35,0,202608200350,failed,timeout,solve_timeout,250.04466200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,35,1,202608200351,failed,timeout,solve_timeout,250.09471200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,35,2,202608200352,failed,timeout,solve_timeout,250.00167200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,35,3,202608200353,failed,timeout,solve_timeout,250.02815200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,35,4,202608200354,failed,timeout,solve_timeout,250.04252200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,37,0,202608200370,failed,timeout,solve_timeout,250.01015200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,37,1,202608200371,failed,timeout,solve_timeout,250.05341200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,37,2,202608200372,failed,timeout,solve_timeout,250.01101200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,37,3,202608200373,failed,timeout,solve_timeout,250.02358200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,37,4,202608200374,failed,timeout,solve_timeout,250.00154200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,38,0,202608200380,failed,timeout,solve_timeout,250.00622200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,38,1,202608200381,failed,timeout,solve_timeout,250.01791200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,38,2,202608200382,failed,timeout,solve_timeout,250.05748200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,38,3,202608200383,failed,timeout,solve_timeout,250.01035200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,38,4,202608200384,failed,timeout,solve_timeout,250.01027200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,40,0,202608200400,failed,timeout,solve_timeout,250.00301200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,40,1,202608200401,failed,timeout,solve_timeout,250.01073200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,40,2,202608200402,failed,timeout,solve_timeout,250.00502200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,40,3,202608200403,failed,timeout,solve_timeout,250.06644200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,40,4,202608200404,failed,timeout,solve_timeout,250.00776200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,41,0,202608200410,success,success,validated,15.94543300,103,25.49381094,9199f6636d8d8b53034638445e0c3a618adb2dad4277b7feeecff7fa663291bb +tabletop_pillar,scene_pickplace.xml,41,1,202608200411,success,success,validated,5.46294100,88,21.74858484,7085c444fd8172df06b45e4922709758c75a75a96c1a84abbea7455ea8adc516 +tabletop_pillar,scene_pickplace.xml,41,2,202608200412,success,success,validated,10.26766300,100,24.72981022,2d4f5eb152eb25a0c34db82f88a76ed0e4dcd8389cbd82d51ae52451216d043e +tabletop_pillar,scene_pickplace.xml,41,3,202608200413,success,success,validated,3.03205500,60,14.70184507,954ce11cf528e7caa9f32cdda5990ab96f5f2842d0425ce8c223e7afe4647531 +tabletop_pillar,scene_pickplace.xml,41,4,202608200414,success,success,validated,7.88160700,109,26.99998865,0788d692e880c8140595a695b0ae7df64a12a45b97c679965aa081df356df9d3 +tabletop_pillar,scene_pickplace.xml,42,0,202608200420,success,success,validated,2.82696500,69,16.99088084,3fefc64632d50921efc826bbdfbe14883e16aebd40498a659dcc64e35ad9bc9d +tabletop_pillar,scene_pickplace.xml,42,1,202608200421,success,success,validated,3.24936400,69,16.99041651,0eda8fe29cfd587f15c47d700c0a69e8aa0a77e0cf4ad695dd5d9b2480e2751b +tabletop_pillar,scene_pickplace.xml,42,2,202608200422,success,success,validated,14.39030600,117,29.00000000,3c80679b893681948dc9d0d9980faff9c144dd68eb229ee420070238fdf1f44b +tabletop_pillar,scene_pickplace.xml,42,3,202608200423,success,success,validated,5.15110200,67,16.42430954,44737b9172246824ac625943ab0055203d88716e8f778ade318a331e4e24e83d +tabletop_pillar,scene_pickplace.xml,42,4,202608200424,success,success,validated,5.66058100,107,26.49999999,84f7016fbbdf24e27e2a0fb37b307f44cfafe0299304f12777389eb651744afa +tabletop_pillar,scene_pickplace.xml,43,0,202608200430,success,success,validated,13.82577700,74,18.20571594,c0fb67976da717797a6500ac9b8e8a0e613f50d8aac89cd30b61bc43dc181ab3 +tabletop_pillar,scene_pickplace.xml,43,1,202608200431,success,success,validated,5.41117100,50,12.24317701,209dabb3d51f96e7ed09659bf30756c1e066b17994769f3185e75443a091025b +tabletop_pillar,scene_pickplace.xml,43,2,202608200432,success,success,validated,12.73815900,60,14.74900865,e18c55135a42f85c16af73a75cd845f86a33d77628696459d3f6aecb43143cc8 +tabletop_pillar,scene_pickplace.xml,43,3,202608200433,success,success,validated,4.03776400,44,10.44310190,141ad9c807e403eac5bb5ad938d3fc88c7464b93dd8457e11ce6f4467bbf2b73 +tabletop_pillar,scene_pickplace.xml,43,4,202608200434,success,success,validated,8.85786600,64,15.73714967,ef46b6e53ccbc86d82007361b935ee95d5aaf67d2ede7af1099a40007a116086 +tabletop_pillar,scene_pickplace.xml,44,0,202608200440,failed,timeout,solve_timeout,250.00569200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,44,1,202608200441,failed,timeout,solve_timeout,250.00610200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,44,2,202608200442,failed,timeout,solve_timeout,250.00612200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,44,3,202608200443,failed,timeout,solve_timeout,250.00502200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,44,4,202608200444,failed,timeout,solve_timeout,250.00027200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,47,0,202608200470,failed,timeout,solve_timeout,250.00855200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,47,1,202608200471,failed,timeout,solve_timeout,250.00449200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,47,2,202608200472,failed,timeout,solve_timeout,250.05167200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,47,3,202608200473,failed,timeout,solve_timeout,250.02418200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,47,4,202608200474,failed,timeout,solve_timeout,250.01438200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,48,0,202608200480,failed,timeout,solve_timeout,250.06469200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,48,1,202608200481,failed,timeout,solve_timeout,250.04433200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,48,2,202608200482,failed,timeout,solve_timeout,250.00796200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,48,3,202608200483,failed,timeout,solve_timeout,250.00133200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,48,4,202608200484,failed,timeout,solve_timeout,250.03358200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,49,0,202608200490,failed,timeout,solve_timeout,250.05882200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,49,1,202608200491,failed,timeout,solve_timeout,250.00987200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,49,2,202608200492,failed,timeout,solve_timeout,250.06014200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,49,3,202608200493,failed,timeout,solve_timeout,250.00875200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,49,4,202608200494,failed,timeout,solve_timeout,250.03946200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,50,0,202608200500,failed,timeout,solve_timeout,250.01902200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,50,1,202608200501,failed,timeout,solve_timeout,250.04522200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,50,2,202608200502,failed,timeout,solve_timeout,250.03682200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,50,3,202608200503,failed,timeout,solve_timeout,250.02026200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,50,4,202608200504,failed,timeout,solve_timeout,250.01640200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,51,0,202608200510,failed,timeout,solve_timeout,250.03478200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,51,1,202608200511,failed,timeout,solve_timeout,250.03617200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,51,2,202608200512,failed,timeout,solve_timeout,250.01992200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,51,3,202608200513,failed,timeout,solve_timeout,250.02143200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,51,4,202608200514,failed,timeout,solve_timeout,250.01075200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,52,0,202608200520,success,success,validated,6.82151800,64,15.74999988,ca62ab8c67b95485f86c7cb985be49f955e41ca56d73b9ff1d7338dec10be540 +tabletop_pillar,scene_pickplace.xml,52,1,202608200521,success,success,validated,30.43400000,94,23.24999639,7468a0a2ae5d6357bc4a16fe362b68f7190c2d2b05ef9aead62549c6d28cdc4a +tabletop_pillar,scene_pickplace.xml,52,2,202608200522,success,success,validated,16.76442200,70,17.22947701,bee9c9fdac4c576219b607ffd82fb39fd919b404c32a2247db2aa79c52378df0 +tabletop_pillar,scene_pickplace.xml,52,3,202608200523,success,success,validated,7.09849800,66,16.21607733,4dd7fc3b6da191d2ed693004e18bc38ebb92c04d02c7088890e53bc5edb78ab6 +tabletop_pillar,scene_pickplace.xml,52,4,202608200524,success,success,validated,5.77809000,49,11.99942226,bb87fb27ce1036264dcb1b2a882450d644fcb7cbdc7aa63afea10389b489d1b3 +tabletop_pillar,scene_pickplace.xml,54,0,202608200540,failed,timeout,solve_timeout,250.00429200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,54,1,202608200541,failed,timeout,solve_timeout,250.01743200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,54,2,202608200542,failed,timeout,solve_timeout,250.03309200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,54,3,202608200543,failed,timeout,solve_timeout,250.00288200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,54,4,202608200544,failed,timeout,solve_timeout,250.03576200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,55,0,202608200550,success,success,validated,22.65738200,175,43.50000000,a55e23273511b0fac2a5601915462d911610771df5fe1be6b185c1e07159e60e +tabletop_pillar,scene_pickplace.xml,55,1,202608200551,success,success,validated,12.16286900,98,24.21576421,e2cafa8876fb2ee7c640c2b0e44b1edf92296eeda0b2ad9d73320f0957c81f28 +tabletop_pillar,scene_pickplace.xml,55,2,202608200552,success,success,validated,6.52622900,95,23.49999997,27d1e8717a391523b4a01a5a39a4d1fe930587cb275439e6282afdb2aebf5c56 +tabletop_pillar,scene_pickplace.xml,55,3,202608200553,success,success,validated,10.34208300,114,28.25000000,3bf950c150b08ebf94d3f7ad9a45bc6e7a3f0d57668e6b174481b50c92a04ed2 +tabletop_pillar,scene_pickplace.xml,55,4,202608200554,success,success,validated,12.57547900,114,28.24999998,d05e4dc4d67e92104ad0488a2b3fe92a7b68b6d18308c08f2871c71dd18d090d +tabletop_pillar,scene_pickplace.xml,57,0,202608200570,failed,timeout,solve_timeout,250.00890200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,57,1,202608200571,failed,timeout,solve_timeout,250.01177200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,57,2,202608200572,failed,timeout,solve_timeout,250.01566100,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,57,3,202608200573,failed,timeout,solve_timeout,250.05728200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,57,4,202608200574,failed,timeout,solve_timeout,250.05646200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,58,0,202608200580,failed,timeout,solve_timeout,250.05156200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,58,1,202608200581,failed,timeout,solve_timeout,250.03400200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,58,2,202608200582,failed,timeout,solve_timeout,250.00976200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,58,3,202608200583,failed,timeout,solve_timeout,250.03563200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,58,4,202608200584,failed,timeout,solve_timeout,250.05841300,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,59,0,202608200590,failed,timeout,solve_timeout,250.01358200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,59,1,202608200591,failed,timeout,solve_timeout,250.00661200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,59,2,202608200592,failed,timeout,solve_timeout,250.02530200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,59,3,202608200593,failed,timeout,solve_timeout,250.01244200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,59,4,202608200594,failed,timeout,solve_timeout,250.00647200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,60,0,202608200600,failed,timeout,solve_timeout,250.04163200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,60,1,202608200601,failed,timeout,solve_timeout,250.02988200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,60,2,202608200602,failed,timeout,solve_timeout,250.04453200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,60,3,202608200603,failed,timeout,solve_timeout,250.05051200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,60,4,202608200604,failed,timeout,solve_timeout,250.00884300,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,61,0,202608200610,failed,timeout,solve_timeout,250.00689200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,61,1,202608200611,failed,timeout,solve_timeout,250.02845200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,61,2,202608200612,failed,timeout,solve_timeout,250.02091200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,61,3,202608200613,failed,timeout,solve_timeout,250.03392200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,61,4,202608200614,failed,timeout,solve_timeout,250.03940200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,63,0,202608200630,failed,timeout,solve_timeout,250.05799200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,63,1,202608200631,failed,timeout,solve_timeout,250.01361200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,63,2,202608200632,failed,timeout,solve_timeout,250.01850200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,63,3,202608200633,failed,timeout,solve_timeout,250.04202200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,63,4,202608200634,failed,timeout,solve_timeout,250.02572200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,64,0,202608200640,failed,timeout,solve_timeout,250.04502200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,64,1,202608200641,failed,timeout,solve_timeout,250.06433200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,64,2,202608200642,failed,timeout,solve_timeout,250.01548200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,64,3,202608200643,failed,timeout,solve_timeout,250.00763200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,64,4,202608200644,failed,timeout,solve_timeout,250.00973200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,66,0,202608200660,failed,timeout,solve_timeout,250.03899200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,66,1,202608200661,failed,timeout,solve_timeout,250.00761200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,66,2,202608200662,failed,timeout,solve_timeout,250.06315200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,66,3,202608200663,failed,timeout,solve_timeout,250.00232200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,66,4,202608200664,failed,timeout,solve_timeout,250.04433200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,67,0,202608200670,failed,timeout,solve_timeout,250.00570300,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,67,1,202608200671,failed,timeout,solve_timeout,250.02366200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,67,2,202608200672,failed,timeout,solve_timeout,250.01140200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,67,3,202608200673,failed,timeout,solve_timeout,250.05096200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,67,4,202608200674,failed,timeout,solve_timeout,250.05672200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,68,0,202608200680,failed,timeout,solve_timeout,250.09755200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,68,1,202608200681,failed,timeout,solve_timeout,250.03931200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,68,2,202608200682,failed,timeout,solve_timeout,250.03369300,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,68,3,202608200683,failed,timeout,solve_timeout,250.04681200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,68,4,202608200684,failed,timeout,solve_timeout,250.03229200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,69,0,202608200690,failed,timeout,solve_timeout,250.00962200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,69,1,202608200691,failed,timeout,solve_timeout,250.01307200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,69,2,202608200692,failed,timeout,solve_timeout,250.04381200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,69,3,202608200693,failed,timeout,solve_timeout,250.00600200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,69,4,202608200694,failed,timeout,solve_timeout,250.00942200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,70,0,202608200700,failed,timeout,solve_timeout,250.00104200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,70,1,202608200701,failed,timeout,solve_timeout,250.02760200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,70,2,202608200702,failed,timeout,solve_timeout,250.02590300,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,70,3,202608200703,failed,timeout,solve_timeout,250.04692200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,70,4,202608200704,failed,timeout,solve_timeout,250.02073200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,71,0,202608200710,failed,timeout,solve_timeout,250.05303200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,71,1,202608200711,failed,timeout,solve_timeout,250.00741200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,71,2,202608200712,failed,timeout,solve_timeout,250.00448200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,71,3,202608200713,failed,timeout,solve_timeout,250.01685200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,71,4,202608200714,failed,timeout,solve_timeout,250.00792300,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,73,0,202608200730,success,success,validated,11.77156100,108,26.74999978,fae086df57a973fffc9a140ab8a277c42c323a9eeb97be6eec4d86793f154057 +tabletop_pillar,scene_pickplace.xml,73,1,202608200731,success,success,validated,10.29616300,66,15.82245250,488b352330dba825c433f1f87f4af8824f019625e77cd92f872c4c2f6af00b60 +tabletop_pillar,scene_pickplace.xml,73,2,202608200732,success,success,validated,9.34917400,106,26.24969821,227344b582a4fbd14667d332f5613ae8464262dd601188127f08ac2df21e6374 +tabletop_pillar,scene_pickplace.xml,73,3,202608200733,success,success,validated,10.25340200,76,18.64896789,73487f96c786608cfc42f84ca48fa30a64c6b4a0d23b79a4ca97fe467083979a +tabletop_pillar,scene_pickplace.xml,73,4,202608200734,success,success,validated,7.18453800,85,20.97131824,3059d15f6579dfe0c4be2529aefd9511890404666f2d4f518c8f8164c876b1f1 +tabletop_pillar,scene_pickplace.xml,74,0,202608200740,failed,timeout,solve_timeout,250.02142200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,74,1,202608200741,failed,timeout,solve_timeout,250.01231200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,74,2,202608200742,failed,timeout,solve_timeout,250.01507200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,74,3,202608200743,failed,timeout,solve_timeout,250.02521200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,74,4,202608200744,failed,timeout,solve_timeout,250.00093200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,75,0,202608200750,failed,timeout,solve_timeout,250.00948300,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,75,1,202608200751,failed,timeout,solve_timeout,250.00595300,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,75,2,202608200752,failed,timeout,solve_timeout,250.01442200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,75,3,202608200753,failed,timeout,solve_timeout,250.00150200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,75,4,202608200754,failed,timeout,solve_timeout,250.05512200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,77,0,202608200770,failed,timeout,solve_timeout,250.00935200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,77,1,202608200771,failed,timeout,solve_timeout,250.00333200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,77,2,202608200772,failed,timeout,solve_timeout,250.00288200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,77,3,202608200773,failed,timeout,solve_timeout,250.00977200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,77,4,202608200774,failed,timeout,solve_timeout,250.00763200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,78,0,202608200780,failed,timeout,solve_timeout,250.00717200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,78,1,202608200781,failed,timeout,solve_timeout,250.00313200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,78,2,202608200782,failed,timeout,solve_timeout,250.01701200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,78,3,202608200783,failed,timeout,solve_timeout,250.02648200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,78,4,202608200784,failed,timeout,solve_timeout,250.02326300,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,80,0,202608200800,failed,timeout,solve_timeout,250.01657300,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,80,1,202608200801,failed,timeout,solve_timeout,250.00136300,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,80,2,202608200802,failed,timeout,solve_timeout,250.00962200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,80,3,202608200803,failed,timeout,solve_timeout,250.00711200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,80,4,202608200804,failed,timeout,solve_timeout,250.01042200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,81,0,202608200810,failed,timeout,solve_timeout,250.05300200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,81,1,202608200811,failed,timeout,solve_timeout,250.02318200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,81,2,202608200812,failed,timeout,solve_timeout,250.05194200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,81,3,202608200813,failed,timeout,solve_timeout,250.00114200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,81,4,202608200814,failed,timeout,solve_timeout,250.01823200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,82,0,202608200820,success,success,validated,6.94730800,64,15.61663190,7e7fc4ce87ff8ba16cc4fa898d026a07d612599454aa07ea6e874195a776bcf4 +tabletop_pillar,scene_pickplace.xml,82,1,202608200821,success,success,validated,2.73009600,58,14.24713465,3e090e6ce407e7c560f90c34217d0a7edb085c74c12f0d780010262014fd81b5 +tabletop_pillar,scene_pickplace.xml,82,2,202608200822,success,success,validated,8.48379600,87,21.49607404,52465994a5c98d8b3c44d0762855ac90bead04179d4a96e4409816e6b388d6a1 +tabletop_pillar,scene_pickplace.xml,82,3,202608200823,success,success,validated,4.23056300,63,15.49117832,51dadd73b3774b5b8ef5b70cdabfa311a549e5452ef0c6a804a1e073d9e5d6d2 +tabletop_pillar,scene_pickplace.xml,82,4,202608200824,success,success,validated,15.31188400,54,12.80970721,cac6ec9466b6130143f18ac4d5c37a0a062366610eee5c52b09b39353b601e62 +tabletop_pillar,scene_pickplace.xml,83,0,202608200830,failed,timeout,solve_timeout,250.07849200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,83,1,202608200831,failed,timeout,solve_timeout,250.02814200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,83,2,202608200832,failed,timeout,solve_timeout,250.03392200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,83,3,202608200833,failed,timeout,solve_timeout,250.00037200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,83,4,202608200834,failed,timeout,solve_timeout,250.05372200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,86,0,202608200860,success,success,validated,11.35388100,45,10.61600707,4cb764a03ec172d4ac53632702d7f55ef058b2fd1628464d2cef2810168b7de2 +tabletop_pillar,scene_pickplace.xml,86,1,202608200861,success,success,validated,9.23996500,47,11.05771964,59e7249e89861f15c164eeed9bc1174200d2e40a09e019cb47ee2930eec36bed +tabletop_pillar,scene_pickplace.xml,86,2,202608200862,success,success,validated,6.66120900,99,24.50000000,c35fcf69a65de2603568669a68184fb16de91f1a9a18571bb2a956bb06620000 +tabletop_pillar,scene_pickplace.xml,86,3,202608200863,success,success,validated,5.44272100,48,11.65729667,2f2f22e46c115b8416d7e061d5705444fe343cc0215ef2ac1d4d0bd1ccea9b9e +tabletop_pillar,scene_pickplace.xml,86,4,202608200864,success,success,validated,8.42221600,45,10.51922547,59ef6c0ebf02bc5eec47f91158f8e52ba5e29f1daf42968926b80ab7c52c05f2 +tabletop_pillar,scene_pickplace.xml,90,0,202608200900,failed,timeout,solve_timeout,250.02133200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,90,1,202608200901,failed,timeout,solve_timeout,250.01098200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,90,2,202608200902,failed,timeout,solve_timeout,250.02035200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,90,3,202608200903,failed,timeout,solve_timeout,250.03650200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,90,4,202608200904,failed,timeout,solve_timeout,250.04011200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,92,0,202608200920,failed,timeout,solve_timeout,250.00366200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,92,1,202608200921,failed,timeout,solve_timeout,250.02643200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,92,2,202608200922,failed,timeout,solve_timeout,250.03521300,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,92,3,202608200923,failed,timeout,solve_timeout,250.00367300,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,92,4,202608200924,failed,timeout,solve_timeout,250.00531200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,94,0,202608200940,failed,timeout,solve_timeout,250.00260200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,94,1,202608200941,failed,timeout,solve_timeout,250.05234200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,94,2,202608200942,failed,timeout,solve_timeout,250.00479200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,94,3,202608200943,failed,timeout,solve_timeout,250.06288200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,94,4,202608200944,failed,timeout,solve_timeout,250.04808200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,95,0,202608200950,failed,timeout,solve_timeout,250.01014200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,95,1,202608200951,failed,timeout,solve_timeout,250.04869300,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,95,2,202608200952,failed,timeout,solve_timeout,250.05634200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,95,3,202608200953,failed,timeout,solve_timeout,250.02193200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,95,4,202608200954,failed,timeout,solve_timeout,250.00380200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,96,0,202608200960,success,success,validated,7.22133800,112,27.74999996,79d906025190042bd248614c4b81daf601e2bdafe6066cfeedfafe0621cb488b +tabletop_pillar,scene_pickplace.xml,96,1,202608200961,success,success,validated,5.33778100,80,19.74995601,4e59385b74c825b70ddc8df5d7327cea0964c8b7f311f0748c7bda98e2788d46 +tabletop_pillar,scene_pickplace.xml,96,2,202608200962,success,success,validated,11.38594100,150,37.25000000,c0d78539c3b131dd32bf7e6bbe351c053ac322961541c69e46c49085b6e90f75 +tabletop_pillar,scene_pickplace.xml,96,3,202608200963,success,success,validated,8.42312600,120,29.75000000,207f949c188cabc5bcbc028d5e46aadf1588a94f9f78331a9be46c08a071f024 +tabletop_pillar,scene_pickplace.xml,96,4,202608200964,success,success,validated,11.53757100,145,36.00000000,ddd573c332348d526a82373329db0caf5f0a0000ed5fd722ec520ddd43a66643 +tabletop_pillar,scene_pickplace.xml,97,0,202608200970,failed,timeout,solve_timeout,250.05974200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,97,1,202608200971,failed,timeout,solve_timeout,250.00937200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,97,2,202608200972,failed,timeout,solve_timeout,250.00707200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,97,3,202608200973,failed,timeout,solve_timeout,250.01593200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,97,4,202608200974,failed,timeout,solve_timeout,250.04934200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,99,0,202608200990,failed,timeout,solve_timeout,250.00793200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,99,1,202608200991,failed,timeout,solve_timeout,250.06398200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,99,2,202608200992,failed,timeout,solve_timeout,250.01567200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,99,3,202608200993,failed,timeout,solve_timeout,250.04849200,0,0.00000000, +tabletop_pillar,scene_pickplace.xml,99,4,202608200994,failed,timeout,solve_timeout,250.03958200,0,0.00000000, diff --git a/docs/oxmpl_baseline_results.md b/docs/oxmpl_baseline_results.md new file mode 100644 index 0000000..2c026d9 --- /dev/null +++ b/docs/oxmpl_baseline_results.md @@ -0,0 +1,58 @@ +# Independent OxMPL RRT-Connect comparison + +This result follows the outcome-blind `docs/oxmpl_baseline_protocol.md`. It compares the in-repository planner with independently maintained OxMPL 0.6.0 on all **217 frozen blocked-direct queries**, five paired seeds each (**1,085 trials**). Every nominal OxMPL solution was independently revalidated at 0.05-rad edge spacing before counting as success. + +Pooled seed-trial success: **271/1085 (24.98%)** in-repository and **290/1085 (26.73%)** OxMPL. +Paired outcomes: **271 both**, **0 in-repository only**, **19 OxMPL only**, and **795 neither**; the predeclared exact two-sided McNemar/binomial calculation is **0.00000381**. Because five seeds share each query, this calculation is descriptive only and does not satisfy independent-pair assumptions. + +## Results by scene + +| Scene | Trials | In-repository success | OxMPL success | Both | In-repository only | OxMPL only | Neither | +|---|---:|---:|---:|---:|---:|---:|---:| +| open_floor | 335 | 67 | 70 | 67 | 0 | 3 | 265 | +| offset_pillar | 400 | 129 | 145 | 129 | 0 | 16 | 255 | +| tabletop_pillar | 350 | 75 | 75 | 75 | 0 | 0 | 275 | + +## Per-query repeat distribution + +| Successful seeds out of five | In-repository queries | OxMPL queries | +|---:|---:|---:| +| 0 | 161 | 159 | +| 1 | 1 | 0 | +| 2 | 1 | 0 | +| 3 | 1 | 0 | +| 4 | 0 | 0 | +| 5 | 53 | 58 | + +At least one of five seeds succeeded on **56/217** blocked queries in-repository and **58/217** with OxMPL. OxMPL returned **0 invalid nominal paths** after independent checking. + +## Frozen configuration and integrity + +- Inputs: query CSV `c07b622...b5452`; in-repository planning CSV `b739009...13055`. +- OxMPL: crates.io `oxmpl = 0.6.0`, crate archive `11988e3...980dc`, archive VCS commit `6ee31f4...e13`. +- Both planners use 0.25-rad extensions, 0.05 goal bias, the frozen paired seeds, and the same MuJoCo emitted-contact state predicate. +- OxMPL 0.6.0 motion sampling was configured to at most 0.05 rad; every returned segment was then rechecked independently at 0.05 rad. +- OxMPL received a fixed 250-ms wall-time budget; the in-repository artifact used 2,000 iterations. Runtime is intentionally not compared. +- OxMPL consumes eight-decimal serialized query coordinates; the original in-repository rows used pre-serialization values, a maximum representational difference of `5e-9` rad per coordinate. + +The exact host, two-run resource measurements, artifact hashes, and deterministic comparison gate are recorded in [`docs/oxmpl_baseline_validation.md`](oxmpl_baseline_validation.md). + +## Interpretation boundaries + +- This is a cross-implementation comparison on one frozen cohort, not evidence that either planner is universally better. +- Five seeds for one query are repeated algorithm trials, not independent task draws. The retained seed-level McNemar/binomial calculation is descriptive, not a valid population significance test; no population confidence interval is attached. +- Path lengths are retained in the CSV only as diagnostics. The in-repository path is shortcut and densified; OxMPL returns a raw tree path, so cost is not a fair quality comparison. +- OxMPL's timeout is wall-clock-bound. Two consecutive runs on the recorded host must agree, but materially different hardware can change borderline timeout outcomes. +- Seed integers and replicate indices are paired, but arm-lab and OxMPL use different PRNGs; they do not receive identical random samples. +- Collision checks sample configurations and MuJoCo emitted contacts. They do not certify continuous avoidance or positive clearance. +- This is deterministic simulation evidence: no hardware, grasp-success, uncertainty, or sim-to-real claim is introduced. + +## Reproduce + +```bash +cargo run --release -p arm-lab-demo --bin oxmpl_baseline -- --write +cargo run --release -p arm-lab-demo --bin oxmpl_baseline -- --check +cargo run --release -p arm-lab-demo --bin oxmpl_baseline -- --verify-artifacts +``` + +`oxmpl_elapsed_ms` is observational wall time and the only field normalized by `--check`. diff --git a/docs/oxmpl_baseline_validation.md b/docs/oxmpl_baseline_validation.md new file mode 100644 index 0000000..74554ee --- /dev/null +++ b/docs/oxmpl_baseline_validation.md @@ -0,0 +1,89 @@ +# OxMPL baseline validation record + +Validation date: 2026-08-18 (Europe/Dublin). This is a simulation-only +execution record for the predeclared +[`oxmpl_baseline_protocol.md`](oxmpl_baseline_protocol.md). + +## Exact environment + +- OS: Fedora Linux 43 Workstation, x86-64; +- kernel: `7.1.4-104.fc43.x86_64`; +- CPU: AMD Ryzen 5 3600, 6 cores / 12 threads; +- Rust: `rustc 1.97.1 (8bab26f4f 2026-07-14)`; +- Cargo: `cargo 1.97.1 (c980f4866 2026-06-30)`; +- MuJoCo: 3.9.0 through `mujoco-rs 5.0.0+mj-3.9.0`; and +- external planner: crates.io `oxmpl 0.6.0`, archive SHA-256 + `11988e3c34bfcacbc4d96265f1da76f9bd3a30fcc92ad62e4046cc6315e980dc`. + +No other CPU-heavy evaluation was intentionally run concurrently. + +## Two consecutive outcome runs + +The protocol requires one result write followed immediately by a full +regeneration/check. Both commands used the release profile and the same +MuJoCo runtime: + +```bash +cargo run --release -p arm-lab-demo --bin oxmpl_baseline -- --write +cargo run --release -p arm-lab-demo --bin oxmpl_baseline -- --check +``` + +| Run | Exit | Wall | User CPU | Max RSS | Deterministic result | +|---|---:|---:|---:|---:|---| +| write | 0 | 3:26.12 | 205.70 s | 254,184 KiB | 1,085 rows written | +| immediate check | 0 | 3:25.27 | 204.23 s | 256,772 KiB | exact match after wall-time normalization | + +The check compared every status, failure detail, waypoint count, raw path +cost, and SHA-256 digest of the exact returned floating-point path. Only +`oxmpl_elapsed_ms` was normalized. There were no status flips or path-content +mismatches. The slowest successful first-run solve was 175.554487 ms, leaving +74.445513 ms before the predeclared 250-ms boundary; timeout cases remain +wall-clock-dependent on materially different hardware. + +## Retained outcomes and artifacts + +- in-repository success: 271/1,085; +- OxMPL success: 290/1,085; +- paired cells (both / in-repository only / OxMPL only / neither): + 271 / 0 / 19 / 795; +- independently invalid OxMPL paths: 0; +- `oxmpl_baseline_results.csv` SHA-256: + `3b44b2a05283b5da752ab1464091d03c03702afda8323f8718df55927c37d840`; +- `oxmpl_baseline_results.md` final SHA-256: + `31d894331e4b9f1d72d9d6f8f2df67826ccb48e174820efeeb5f2a5a68668c56`. + +The fast `--verify-artifacts` mode checks frozen input hashes, the 1,085-row +schema and paired keys, re-renders the CSV, and derives the complete Markdown +report from the committed rows without rerunning the wall-time-bound planner. + +## Source and chronology audit + +The result-bearing runs used evaluator commit +`8bfe273f13f151a8adf0b6f5bf32503c20538bbd`. Its first parent chain contains +the protocol freeze (`ed48da6`), the matched-limit clarification (`10ee6fc`), +and the serialized-coordinate disclosure (`9a793e8`), all committed before the +evaluator dependency and implementation. The working tree was clean before +the first `--write` run. + +A fresh source audit mapped the evaluator to every frozen requirement: +cohort/input hashes, 217 blocked-query filter, 1,085 paired keys and seeds, +0.25-rad extension, 0.05 goal bias, 250-ms solve budget, 0.05-rad internal and +independent edge sampling, endpoint/bounds/finite checks, retained status +categories, exact-path digests, and wall-time-only normalization. Formatting, +strict workspace Clippy, all workspace release tests, both existing artifact +regenerations, the new artifact verifier, and an independent CSV invariant +script passed. + +Post-run evaluator changes are confined to generated-report wording: they add +the validation-record link and explicitly downgrade the seed-level McNemar +calculation because seeds share queries. Post-run protocol edits correct the +public OxMPL type path and clarify that the paired integer seeds feed different +PRNGs. No planning, collision, timeout, validation, parsing, aggregation, or +raw-CSV logic changed after results were observed. + +## Claim boundary + +This validates exact-host repeatability and independent sampled-path checking. +It does not establish cross-hardware timeout invariance, equal compute, +continuous collision safety, positive clearance, hardware behavior, or +sim-to-real performance.