From d422e66156869712b30447a266483d4982ec481a Mon Sep 17 00:00:00 2001 From: Amantux Date: Sat, 26 Sep 2026 05:45:42 +0000 Subject: [PATCH 01/54] chore: untrack committed .pyc bytecode files 66 __pycache__/*.pyc files were committed despite .gitignore already covering *.py[cod] and __pycache__/. Untrack them with git rm --cached; bytecode regenerates fine on next import. --- openblade/__pycache__/__init__.cpython-310.pyc | Bin 378 -> 0 bytes openblade/__pycache__/config.cpython-310.pyc | Bin 5855 -> 0 bytes .../api/__pycache__/__init__.cpython-310.pyc | Bin 244 -> 0 bytes openblade/api/__pycache__/main.cpython-310.pyc | Bin 8600 -> 0 bytes .../__pycache__/routes_archive.cpython-310.pyc | Bin 6142 -> 0 bytes .../routes_inventory.cpython-310.pyc | Bin 2029 -> 0 bytes .../api/__pycache__/routes_jobs.cpython-310.pyc | Bin 2013 -> 0 bytes .../__pycache__/routes_restore.cpython-310.pyc | Bin 3974 -> 0 bytes .../__pycache__/routes_tapes.cpython-310.pyc | Bin 2861 -> 0 bytes .../routes_volume_groups.cpython-310.pyc | Bin 2374 -> 0 bytes .../__pycache__/__init__.cpython-310.pyc | Bin 279 -> 0 bytes .../catalog/__pycache__/db.cpython-310.pyc | Bin 11355 -> 0 bytes .../catalog/__pycache__/models.cpython-310.pyc | Bin 19650 -> 0 bytes .../__pycache__/repository.cpython-310.pyc | Bin 54496 -> 0 bytes .../cli/__pycache__/__init__.cpython-310.pyc | Bin 158 -> 0 bytes openblade/cli/__pycache__/main.cpython-310.pyc | Bin 8010 -> 0 bytes .../domain/__pycache__/__init__.cpython-310.pyc | Bin 248 -> 0 bytes .../domain/__pycache__/backends.cpython-310.pyc | Bin 4073 -> 0 bytes .../domain/__pycache__/errors.cpython-310.pyc | Bin 6293 -> 0 bytes .../domain/__pycache__/models.cpython-310.pyc | Bin 7355 -> 0 bytes .../domain/__pycache__/policies.cpython-310.pyc | Bin 3352 -> 0 bytes .../domain/__pycache__/states.cpython-310.pyc | Bin 2167 -> 0 bytes .../fuse/__pycache__/__init__.cpython-310.pyc | Bin 179 -> 0 bytes .../fuse/__pycache__/cache.cpython-310.pyc | Bin 2073 -> 0 bytes .../fuse/__pycache__/filesystem.cpython-310.pyc | Bin 3742 -> 0 bytes .../__pycache__/__init__.cpython-310.pyc | Bin 270 -> 0 bytes .../__pycache__/discovery.cpython-310.pyc | Bin 4005 -> 0 bytes .../hardware/__pycache__/ltfs.cpython-310.pyc | Bin 11867 -> 0 bytes .../hardware/__pycache__/mtx.cpython-310.pyc | Bin 6438 -> 0 bytes .../hardware/__pycache__/runner.cpython-310.pyc | Bin 3195 -> 0 bytes .../hardware/__pycache__/safety.cpython-310.pyc | Bin 983 -> 0 bytes .../hardware/__pycache__/sg.cpython-310.pyc | Bin 2306 -> 0 bytes .../jobs/__pycache__/__init__.cpython-310.pyc | Bin 257 -> 0 bytes .../jobs/__pycache__/format.cpython-310.pyc | Bin 3597 -> 0 bytes .../jobs/__pycache__/inventory.cpython-310.pyc | Bin 1732 -> 0 bytes .../jobs/__pycache__/queue.cpython-310.pyc | Bin 3803 -> 0 bytes .../jobs/__pycache__/restore.cpython-310.pyc | Bin 5940 -> 0 bytes .../jobs/__pycache__/verify.cpython-310.pyc | Bin 2723 -> 0 bytes .../jobs/__pycache__/worker.cpython-310.pyc | Bin 976 -> 0 bytes .../__pycache__/__init__.cpython-310.pyc | Bin 240 -> 0 bytes .../__pycache__/changer.cpython-310.pyc | Bin 576 -> 0 bytes .../simulator/__pycache__/drive.cpython-310.pyc | Bin 731 -> 0 bytes .../__pycache__/faults.cpython-310.pyc | Bin 2970 -> 0 bytes .../__pycache__/library.cpython-310.pyc | Bin 17578 -> 0 bytes .../__pycache__/ltfs_volume.cpython-310.pyc | Bin 15715 -> 0 bytes tests/__pycache__/__init__.cpython-310.pyc | Bin 151 -> 0 bytes .../conftest.cpython-310-pytest-8.3.2.pyc | Bin 963 -> 0 bytes tests/e2e/__pycache__/__init__.cpython-310.pyc | Bin 130 -> 0 bytes ...archive_restore.cpython-310-pytest-8.3.2.pyc | Bin 6451 -> 0 bytes .../fault/__pycache__/__init__.cpython-310.pyc | Bin 132 -> 0 bytes ...fault_injection.cpython-310-pytest-8.3.2.pyc | Bin 9923 -> 0 bytes .../__pycache__/__init__.cpython-310.pyc | Bin 138 -> 0 bytes .../test_catalog.cpython-310-pytest-8.3.2.pyc | Bin 18252 -> 0 bytes .../test_jobs.cpython-310-pytest-8.3.2.pyc | Bin 7591 -> 0 bytes ...mulator_backend.cpython-310-pytest-8.3.2.pyc | Bin 17696 -> 0 bytes ...imulator_stress.cpython-310-pytest-8.3.2.pyc | Bin 4523 -> 0 bytes .../__pycache__/__init__.cpython-310.pyc | Bin 135 -> 0 bytes ...bot_transitions.cpython-310-pytest-8.3.2.pyc | Bin 6193 -> 0 bytes .../safety/__pycache__/__init__.cpython-310.pyc | Bin 133 -> 0 bytes tests/unit/__pycache__/__init__.cpython-310.pyc | Bin 131 -> 0 bytes ...t_domain_models.cpython-310-pytest-8.3.2.pyc | Bin 5239 -> 0 bytes ...fuse_filesystem.cpython-310-pytest-8.3.2.pyc | Bin 4997 -> 0 bytes ...ardware_parsers.cpython-310-pytest-8.3.2.pyc | Bin 13824 -> 0 bytes ...hardware_safety.cpython-310-pytest-8.3.2.pyc | Bin 5762 -> 0 bytes ...st_safety_gates.cpython-310-pytest-8.3.2.pyc | Bin 3092 -> 0 bytes ...ate_transitions.cpython-310-pytest-8.3.2.pyc | Bin 2486 -> 0 bytes 66 files changed, 0 insertions(+), 0 deletions(-) delete mode 100644 openblade/__pycache__/__init__.cpython-310.pyc delete mode 100644 openblade/__pycache__/config.cpython-310.pyc delete mode 100644 openblade/api/__pycache__/__init__.cpython-310.pyc delete mode 100644 openblade/api/__pycache__/main.cpython-310.pyc delete mode 100644 openblade/api/__pycache__/routes_archive.cpython-310.pyc delete mode 100644 openblade/api/__pycache__/routes_inventory.cpython-310.pyc delete mode 100644 openblade/api/__pycache__/routes_jobs.cpython-310.pyc delete mode 100644 openblade/api/__pycache__/routes_restore.cpython-310.pyc delete mode 100644 openblade/api/__pycache__/routes_tapes.cpython-310.pyc delete mode 100644 openblade/api/__pycache__/routes_volume_groups.cpython-310.pyc delete mode 100644 openblade/catalog/__pycache__/__init__.cpython-310.pyc delete mode 100644 openblade/catalog/__pycache__/db.cpython-310.pyc delete mode 100644 openblade/catalog/__pycache__/models.cpython-310.pyc delete mode 100644 openblade/catalog/__pycache__/repository.cpython-310.pyc delete mode 100644 openblade/cli/__pycache__/__init__.cpython-310.pyc delete mode 100644 openblade/cli/__pycache__/main.cpython-310.pyc delete mode 100644 openblade/domain/__pycache__/__init__.cpython-310.pyc delete mode 100644 openblade/domain/__pycache__/backends.cpython-310.pyc delete mode 100644 openblade/domain/__pycache__/errors.cpython-310.pyc delete mode 100644 openblade/domain/__pycache__/models.cpython-310.pyc delete mode 100644 openblade/domain/__pycache__/policies.cpython-310.pyc delete mode 100644 openblade/domain/__pycache__/states.cpython-310.pyc delete mode 100644 openblade/fuse/__pycache__/__init__.cpython-310.pyc delete mode 100644 openblade/fuse/__pycache__/cache.cpython-310.pyc delete mode 100644 openblade/fuse/__pycache__/filesystem.cpython-310.pyc delete mode 100644 openblade/hardware/__pycache__/__init__.cpython-310.pyc delete mode 100644 openblade/hardware/__pycache__/discovery.cpython-310.pyc delete mode 100644 openblade/hardware/__pycache__/ltfs.cpython-310.pyc delete mode 100644 openblade/hardware/__pycache__/mtx.cpython-310.pyc delete mode 100644 openblade/hardware/__pycache__/runner.cpython-310.pyc delete mode 100644 openblade/hardware/__pycache__/safety.cpython-310.pyc delete mode 100644 openblade/hardware/__pycache__/sg.cpython-310.pyc delete mode 100644 openblade/jobs/__pycache__/__init__.cpython-310.pyc delete mode 100644 openblade/jobs/__pycache__/format.cpython-310.pyc delete mode 100644 openblade/jobs/__pycache__/inventory.cpython-310.pyc delete mode 100644 openblade/jobs/__pycache__/queue.cpython-310.pyc delete mode 100644 openblade/jobs/__pycache__/restore.cpython-310.pyc delete mode 100644 openblade/jobs/__pycache__/verify.cpython-310.pyc delete mode 100644 openblade/jobs/__pycache__/worker.cpython-310.pyc delete mode 100644 openblade/simulator/__pycache__/__init__.cpython-310.pyc delete mode 100644 openblade/simulator/__pycache__/changer.cpython-310.pyc delete mode 100644 openblade/simulator/__pycache__/drive.cpython-310.pyc delete mode 100644 openblade/simulator/__pycache__/faults.cpython-310.pyc delete mode 100644 openblade/simulator/__pycache__/library.cpython-310.pyc delete mode 100644 openblade/simulator/__pycache__/ltfs_volume.cpython-310.pyc delete mode 100644 tests/__pycache__/__init__.cpython-310.pyc delete mode 100644 tests/__pycache__/conftest.cpython-310-pytest-8.3.2.pyc delete mode 100644 tests/e2e/__pycache__/__init__.cpython-310.pyc delete mode 100644 tests/e2e/__pycache__/test_mock_archive_restore.cpython-310-pytest-8.3.2.pyc delete mode 100644 tests/fault/__pycache__/__init__.cpython-310.pyc delete mode 100644 tests/fault/__pycache__/test_fault_injection.cpython-310-pytest-8.3.2.pyc delete mode 100644 tests/integration/__pycache__/__init__.cpython-310.pyc delete mode 100644 tests/integration/__pycache__/test_catalog.cpython-310-pytest-8.3.2.pyc delete mode 100644 tests/integration/__pycache__/test_jobs.cpython-310-pytest-8.3.2.pyc delete mode 100644 tests/integration/__pycache__/test_simulator_backend.cpython-310-pytest-8.3.2.pyc delete mode 100644 tests/integration/__pycache__/test_simulator_stress.cpython-310-pytest-8.3.2.pyc delete mode 100644 tests/property/__pycache__/__init__.cpython-310.pyc delete mode 100644 tests/property/__pycache__/test_robot_transitions.cpython-310-pytest-8.3.2.pyc delete mode 100644 tests/safety/__pycache__/__init__.cpython-310.pyc delete mode 100644 tests/unit/__pycache__/__init__.cpython-310.pyc delete mode 100644 tests/unit/__pycache__/test_domain_models.cpython-310-pytest-8.3.2.pyc delete mode 100644 tests/unit/__pycache__/test_fuse_filesystem.cpython-310-pytest-8.3.2.pyc delete mode 100644 tests/unit/__pycache__/test_hardware_parsers.cpython-310-pytest-8.3.2.pyc delete mode 100644 tests/unit/__pycache__/test_hardware_safety.cpython-310-pytest-8.3.2.pyc delete mode 100644 tests/unit/__pycache__/test_safety_gates.cpython-310-pytest-8.3.2.pyc delete mode 100644 tests/unit/__pycache__/test_state_transitions.cpython-310-pytest-8.3.2.pyc diff --git a/openblade/__pycache__/__init__.cpython-310.pyc b/openblade/__pycache__/__init__.cpython-310.pyc deleted file mode 100644 index d8cc7e1c5824e2b15e700cbaa13b31d1689c9ac3..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 378 zcmYk0Jx;?g7=|4uX`v7XSQtCf!Eq`Q+DHiTv%|m^30aQqmX;*899R6@gA1^62riP9 zi7PN+ODOP3eyqoqU%xVnM!?SN_-_4S_fIGP2UbP& zr7*ojTC{(9JsEaGLbzmvbO+MLmo7dor8KS4RB6{$`woFaXyIPasv_ha=xulC@V&;S zR+v?E!v#t((l!G1g>hQWvw6CpC`slyN~gy&lnXjTSs!KqZ`mZeIpob-os=&Mhz>~U5mw)0-kU=c;9atTm)c>VfKFJHf{$4(@o z5_~Rw`r*dpm?V9UjloY48h0c~c76>EmFg0eDKX_1sS_FbfElO<;9oJ7R>HS zekIpaMvx}y&}T9{k)p%!#PAF9Laqa3!&`=6i zr-lD3y1U-MCk8(C>~b*KbT;nY(;#`i7xjo4$I3UY%Q;HLkvU<66#X z09uXpwoY@;+-bY*gT?RMR>XoD9mlm> zJk+!tSGQpG>pG*)bY}1f9%$2Xn~fUSiV8-zZCG!?6M#|fphvdBZo1a;Kr#XwNY?+p9el$KO*=Gz+bFSh%qn0 zx{lV5iE*YuK2x6*Iys?pTwQZA+;aO_kqty%xqXpfIIv;`g8emDey?D>SX6PypfH8BGu4)}-a%H)+ zROS)WexjMireX46LvO5ug@`|mHW?3r^*m`>Pk78{JSj84y1TmfZnMORaSr z0|Er_TH9`dL^0hphYun3<>>#-R1!QVfC?iU_NwMKTZY|nHOByNrp_0)(2fr^)^+QN z!S?qyFM(O|G0K{o#{NE@>H$L8mc}|(X*&x0p4yoz*_OeR-7%@L*=!hIgc@=?ZPR!k z)GkfU%*=ei6W-XKj>8qF;WT-g*{iVfhNHPmx16@kTpo46S#>554eGSjv;Zt$_eJ1kZm-L9Y<+GTP!T) z9~5iaz4HA6JB8|Jkeozv4oMctMI_Tma!BToTt;#Q$yFrpBDs#_CX(+U`7V<0A-MyD zXL_?0?msBzYvqdepi-=|JZj7%SwOOg1j8%KASoc}WuJivd=xq+;pgBYqEeE?<%}Gc zM|<4POyE3zZ)%FoFTp;8Bu$&Or=cF3#1uJlTQ8b#ieAR3U87u4lBPHc+@whXXG zmvs!t%EWef$gdfA;ECb#Am*z!51RHfgJlEkJ(!B&(s+RB&jw(@RKz0?G&VI1CWyQf z1`>yysmLjrJ%R?i2BZPM-iH8(IaBNdf%o=-727U(F6~ITdMDv3Rmly~09Bxp3L=Fb zm8!e)B4hyky^x>FJ97I3YGy+veyF@sD9so1iv{h?!@doZ^q>S@!X;pua}Y`d{6{eO z2T>f3ksbLJl#sjfLqPo_C~y$KJvoQflF9@&S;EQ8EQUFYHlcv>LO|b4c;wa-!!kZ@ zv)ju!3&bgje421?KthAdFR07_-V1`vq3V7e3#`iy`d5vp|gZ1BG$QYhu;i-pA+L(@F~7r!3@ zL*G(_qaP42`M2%UPc~?q4#tA02$0MI_Kp2v-0N8ZM*6N5V65myIgA-J0DDfm!3`DS zpF|b30ceFVh9JdVAMM-0vKJ;6%J*0DHSKQrES$!b|7v(mmf)1TmfXZCS(Y5vg9fo>T~(AWXaFE&q3Vv z5T?PWUDl4A$gAe#n=P3;E%m{D?1~5b~ov`7vKUDdfj{@+n__VsKZm zj+4Ip6hxJuQ3)iccLG4q`0} PBd=*%WdT$j|%oEa-nO^e_05i}aFM@wDHbL4@Fm zoSUGtbdJ9B8MZFF$rm`z6?aJdUv*RB|6S0!Mz8x?H{4;A{up8y=>DH)$bP59hx?u} zpI>+nZlDYkIX~W)ym#-{7%2y!2dO?B**K`wV)D^VGZmj~mNU#C;7pZCr7?G-C z-Bjfz*};^>J_IGE0qLeJb)dTLr&f`*(D(<5D76^-#<%b2R z4IeHq6simh1cr_#3Rh95io%qq`T#7>W97)EQ0oK|EL!|V$(+=CgHE9@+GV~xwl`cGGD`Mb-dyFvkrJ5m1*DpU$ps9wDhE}#OI#r$x8gko)} z3Zd*lsisxRE~7H}CIh8`D4|pZS_&awj|a$CNrn776D0r2sAMk_BLB>U z$v-j?^7mwvd_w-F#K>PWaq^c`f_#}wl0PSh$e-dV@<(-;{6S5V->W0!ckxm3TXjtN z4f!8**r%{4D3QcPL7;%Q1#yvK#V!YKEqJ^Rba~Y6@>qp;dAx{qdAu)W$9YK8sNL}1 z9jRoJ=bsMf< z;sCMAWlbbFsdsxh;1tDn2hz7VGI3hrTi$EU+ZYNRbaO?DlQ3Ko@tY2$gYh7|Pz;6x RDLFM64Mh{tXjDx`{sX9Mm4N^N diff --git a/openblade/api/__pycache__/__init__.cpython-310.pyc b/openblade/api/__pycache__/__init__.cpython-310.pyc deleted file mode 100644 index 0722d4f0c64ce104bc5488f4a7c53f7d942f8850..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 244 zcmd1j<>g`kf~g9zSt>yKF^Gc<7=auIATH(r5-AK(3@MDk44O<;vTljRC5{1}3Pt&) zC8@;n9du>c_`t=4F<|$LkeT-r}&y%}*)KNwov{qZs54 G4h8_Ktv`bR diff --git a/openblade/api/__pycache__/main.cpython-310.pyc b/openblade/api/__pycache__/main.cpython-310.pyc deleted file mode 100644 index e8221cb7270a4becb01327af912f5060c7714496..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 8600 zcmb7JTWlOhcJ1zYayTSKilUyD#(G#9nH=h2$+BJ+pAv0qNRyOox7Y48r;21#-P7ah z9!i{0h#d++5=6-^5+vE!n~+{0h=T+O5Fi2a021s^K9aBe^>01`1Nq2bunN}Bxz#<7 zrWOe@gSma{)UE2OTet4LRpa*b

~%KN?!UenwILjS8Kg3<`G?MfLtOsVGd*6{a%H zRvk^(RN@nMLQmkCw3ALsPdRBl?PT)o=Pw)0Mp-Xr;p-Rtz} zeUi`G`<#BgU-Dh{fU{rUFZrB3=p4`wI79l7b5K9%9MTUthxNnG5&ejBR6pt*(~mi? z=&v}(_2bS7{e)A{3(l+htIkRNq;pC?<($?}J7@GW&TIN>&RPAeb51|!6!oHWUO(@= zuD|XK>%-27KH`-0k~6B0Iv4Z{&PDyAb4kDCT-Gl;Z|HA0SM)2+RsE_QPq+Q1b4`CM z;;)0hA^E(0)48SJa&GIlWx2(kC%{jPIQ zzvs;8GtN8uJ5sjKe%G1RXI15{!uGNLXA0}*`?nMNeV$?ie3tEhuIXhq$PPSHcw&8S zXPynQgS*yLVCc75HE&zLrU5{a}1G~X)wy~Y}`SAL+T~*Hf zeSWdYV|GQz-eR|7vTucwsJ(*Px7nRo?WJaG^g7h!A$u6IV{E)F3v5(k6EW~0J@A+dWg?1;o>V%SlMy%WQZN$lMic2Z)qG3=DY?#Hl##L6-3 zRf)~Tu+tKoZ)4C~si`;cZ*=fCqBZ(}y$9>`*&%HrOZ30QV6r z_Io@HxyNjoRixa<66*o>gmJbau}>s618kG6vqus$Bz6l}gH@R;v0nf-&uYMakqLHA zVkS#0D#fkF$!XK`$L41WX02vh71OueYGK6{g}EB9j@u^VrEkbz7S*8JtX5rLN_gdB zGRQ7tD!fLmgUkZ|w9Y+0$S!cN=2ksk)PkWk->(_GDP*jfRc3P$oED+5 zv0>U4lbydTg_jRSojz=dO$p%&)bviwEiB%*7`l6G3JxUy&f;9T^=>uj8aF+D-(}n` zo(%eIYgw3L)2O&rpFi<~T-EfbuwtzS{lcyL+%rtaHq1(edtR`=6Ji@?wZd7@*9q4B zwOGSt(wY|xb;vNwT&=pEZ&kdQC==EOL_1&0H+a>LzjnnE4oxi>=;$(S+t~nTJlzvChUsMpgWRZcbC0ZbvR86N@`Zy{)!jmBiWj8pR`+w zWwX*6n1BhoP+If7p1H#P&DPTSW{n$et=$y?UQ4=Ovt5(56tQ6mzi!$FUOgDSdc(Er z4mVa&ta-4yw8RB${m3F08&-ucxsP~t+7-(%PF|Jv8XO?A6Rr*0H{7bdY4FI5yH~m2 zw8>&uFgR|C3QYFCUA6pS9BHWm-YZj7f&$lX826tnS`TC z1wFFo=B^GBo-cwFC*=ic#(mSW$tPtEqiSMRj38$ij?3yc@p;4e6ss52^chBnCo_z% z6)}YoNb=&$sBm3>v^Bj^?4YW&*io})jXI`PE!8%~Ei{jG8ZKF!jAnh)%^9r7V`*l$jw7wNutLBHSme%bjv~J%6^Ep;*3yJ9rjt}>g zry5hYv~@*Va6)oP*lb|IWp66Jx~}acSQpD7y-n^atoxBBhFM3n0^y(oFH4N_#>gFMjo&6>PSVno);mF zz6HL54aCLByo=Dk**Le#U0QtV_SU)LaKW+Aptj~#xhz)Dy)eNx2IjeNERT%Zi&_Ug zZL^vxKMcAVBAM;h94t$agP&&g3U%B#<(N-K%vFBt>g5Y>Ub=AM!i|Det=M(O7wgND zF6}U{ad51~1Qk1O*d%+u%ksZ?Tx* zs1s`>a1fLGc_J|lq{bQLBXnTWv{+Brv8i*2MvWu}lMie^xhOb4*@C4+jIXe5cg$c#p5NjiL@K@{@g zy~Ei5A~}k*u#p8BF)8OuQo7N|$+FB98+pk`Nn%(NrgLFUGmmR@lU%f(p_uf6UZamH zI+De(GRzViy;RxG+`>MaDO;n9YGnc#3LCvP>rKJY?thn8hxz>1H9_h1vjiJ(n_T04bHGp z54M753Bj|35gK^C&&4ye5|==NG%SuX?=)LB>cSS=RMQH_VOSe%_bM}U*~jlfP)1+v zv%EH-iC?AOk~Py`6SNb9B^V^^(pNe(+QoYOC%PHcg| z%yUv--eb#OE2`qxsAkXT4jNW?V4%1Q-?@yGgNE@V$ayuc*SGH7N-PXj(HC(m)SlvNJv8#%cTbQM+OQ5c}rB)MuVoTjtpTZ}>FX2LFSKCVB zn~Xa&g{k)*Od<036EOB*>7GPp7N#Z<<^fI5EsW1h zA~c1?^4t=j2j$5iPqh}d(iKSqWN)N|oXC}ueXE3G~7UmYhw>&71Jsg{v9UGsW zDrUaKsNw8}{CbsprN-rlTv#icEdjU~ zNWjy$by493OXemk78mW;l^W+3P;YN=*^0ISg0OE@R}0dTK-DJ*kXC&_LgQ8r6J)6f_h-0)3#2*paCGv+vc0k}Ba+Zf! z@6(YwNYX9{vdAgw%ZO-l56cjTE}m1HJ0L-phHE%xEy#sKCO+(V&#oMhopM#A`V!rm6;ss0gy(Al_Gscjc-3OWFJS%=k{-{)Y~n94n17~Gg}Gc3ra$~$?z_gN|Nqz;``TA;1-Y-p7wB31DM)0f z;?D?>hDs|a{t_feOIa_pS~A#XYAS72`h-0SJ@J1G?w`;}l%kXA=G3&7{Z>n8KO0Dj zzoJ*FhPaNWeC12VOb0r-RVg#233`CQxS#rdvj^5XYsa6csS2SmOEfrY(I z{2d}vrG4VZ#FIl2*ND79A#h{g?Ci{ibmc^5bd+1)~lvT@i-aaZUNt{n*27IByLDe ze=Gh5Vu6McTmTbW0@~iKJB5uO|KurAB5nJjuN1ZKq$z!9K2_vh`8Hk*kpQB3%Ur zyRAIMfI|nrj+?44$$2Nrf#yD<84#2G}$prISd|1$B(a0}A9hv=^pnhrz8ZxVWk zNV|vMqWWaV8>DH_D!q($5t;&0~SOEz#EBJ>3#|J?xA;Yh{w2|`;jzpEEm;PDsgbbGCh=PHaq&ykv zf`iQpr}h6qsp1MxkiQK2o3%*f(qo zej)=z_7kDQhd4lFh{!=AWM1Mhkt0No5;;cX6(YxpoFGyl@+uM9i{ccK(?re?d5y?f zBIk${iJT|$I+0-_BScCd!LiuDOVPddDzf6o=B5}Wk>03egqPbv;paBNUnagG>eVX# z&>83`G+P~sw`%TEIz>wlA!vUG``c1DyZP9S$nE4IvEv-c?Cjk8QkT#RCw^rX^bNw?79 z_oN4$rg3I(h8NxVV;)Z9E0Cm;*Rt9nIF5Yu{l1nzomZ(mtmeD-CGx3sYCubWn@oqP zFK&mC=4QL{YPLJC4rB(^yYDWh<%$0xZ}|gxp4U9F@{%eisThvwe5Yp@%j-@}@9FN- zr+ZGH@0`;eRVqadzwfmx?f*EfY5z`@XMZLtA8Q(m|A}cD*Bp&AuKO&|9i0)K^K(JY z$pwaEsJh{sf#q01-pQ-F>DxiUDJa_Vi$TdLDLU_$gNjoLs!lbSa3+G9Q&a7>KN(Cp zQ^B+|t?C7TCOG08QFPIt4URfTgE?nT)l2@d;J9;K(PjTcaMC%Y=nCl5&S^zg{WpTT zQ&;qa|7P%(^OmA({&#}6owtK`oOhTeDg(_qb4%ls|ID0sMFF+<_#g3Ee)NIiH254p z_K5M1_;>i*4=m^G4>kS{Kl4E2XT;PX=bS|SUH%^G@2UFxs5kgo)X%E=2mFuuIX(|b z=lDgwz!yQyi!<#7eqMYaYVAdS;bBfx#oT~xG5#m~L;k%7=72fp70(~Mg69(dL;iim zbAivTYmFcDW|w5^mUlgiOB^(9ajDa}5`~Glm!zd_ zk+`j_0=~&lJYTFt$tO`a=?HT@fc}nULMkZDtB^+tDU5U6s(rdh3?p zU0=|!!uqYIacs;c21O3|qp_$SCOeB0=j^ z*6T{qitY&6-1J3i-f8+>aWFAbb-iC-fu2)Kili&UG#4kb1xOwKh^D!&5ZJDR=mTzD zO_maEk0p9Lx6gL9aJsLx4b?se%A|JA9#W6h*Y>OtAL#tPwr6sk=k|4&h_M14SV_+d zx1yf){^H_Ci;F398n(<~sx%iyccuE$TsLXSH>oWzMaOTpL|SadBQr@0UL0*jGH62a z@)Xz}Yw|t(9V{%!C`uMShdEsKn_MhJpf(lN?0D#ey~HD>&UbdwqDw+y1wrQ|1jJ<& zHmkBCtFf|PWAZE-Xw{~molo;ayO1@srxy0v^|+)_X(^7nvL)OO8F2Yd{P>Md*5@e}i`6Q1Rs5$p4k_D#iaN2vRtdt5nXg4Q*r3>@%p}>RU?n(_CBE`ex@O zXZ_qSHLmxKUuwTHcJmkso_3z+`Z_lf8+2j7`g&5lqsjBCUK+5FaSi?E*Sh>Zw@}OX z^W1)v-zSe+=l?iM$k;9PO{{~-uli=cz$+JN#pR@uR0rCC zjb_PsV!yBNP4MbIlOeBQt_iHo#;DpOYoB30d`V++rB@h2f!BM}b@+~`o~f~})L2jR zLwu!HNW1UL6HwF>jzW^*Hu9;F>~uu7A9hE2k@S+Cw1EAU$p*QaN^67{RRvG9Ekzs? zBI2%}^z7jh>OK9Vde5w%yHsC*w=uEP*M=Ki%r;s%s#3PVA0BSAJcfm&23dH+N_7ut zJ+!dYjs=jZmPG4-4TMRWhyBIPZIR}=NC5D-QIiuSWJ-iF+@R})Ug9-flk5bk^NiUi;Ctm8I+M zXP;ko*Eg0nZmch_r^RlE9EIYkH9O=}l_V@Wgzb_nTgg)TjPH7I%|-bEwV$JcU?AtI zSfGNeKF!fGKB6itTa#C*AgE?V15{ddH>Jn7g_}e)-5IpSmr+y;Wo9#j&EPeduIo%k zeFo^%neiLr>1^3hZ9235&$fuu)@?S2UV4lJ`}gw45Yxi@+>dfV` z{RYwqk>mQs>_3Np14e1eX)tBqe)wf8kB!uJ-LMIOyKo_H5RroiU3T5u-KIb6nQ+}_ zLZ0hB^WjPb^z};q4G_P!P!!>O%1pJrv}|&WACp^N1M&K03xgZne8?P=Tlk>-MNI&= zj?Hb<3aA;p2#(T2-6`-g+JIbM#ivfBbI%8)rH(fYZ!diR69qPw76UJIeGzUaw|WKN zi<3Wxi@TXt{AMWJO<)yqE|#-cZfZb9;>mrG2z)T~ttP;h>ahA8aGq8+eS~*6hK;+y z<_fH81FVwtBEO*GODd{V`~?+CrnSDYc5QW8-lUdID#nI2*6&y7aPTVEPWrXk6ijFi zjti|XX?bKElC<0~)9K|95xy(CH`4zbi-d9deu1T%z&xk`gDsuJ* zMWrqBAU6-A0NOQDEHPZ6eI_6vglhYm%=fizwyq_N^4ZF*-)l zhr}d?;@USDa$_`|JzMbz4`Cmr;Dnd$GHP>;l`P&o8D1MX_f-NddKRaa)Aq zUPoSPlW;Of{r+*Sch2>YhK<6iJB))QEuO!KjcxkzrTN3LDLdQ9$t?^nDSgX-%iw7ziydfqh=gRQ-M-=v6j8|e@9uCE-h1*9)DarQ|D zVP6?YT9U#?wtI&ho1~SMbGbA&0G<-9GjveAz|kKF&ilEp2;jK5(}wbmwO>InY&z7|g>$P%fI zl9gZO2JBmw7(d%Va>Lr@o`sYEeR%WSc%<*^ySd(UVkbyNRGaw_R@*M(EfGI9tvoQ9 z_N5li8(LD?0RVP!`0KruRQuRA8Z*cZWls<{w~|`l7_fb`CX?V%ZKFA}KAwMQG3{F< zB)i5>jJxb^wzetaMe1b^)0x|!X;?eSFxzFkln<+wgPzG`BuA)^+r02l|5DQ-(ZEA1 zTG!5N@z3wVi0JtWuQH|L%0YQuWsr5kUj3k0$7!j)MW+sV0jh22b*l73L1mTLa^hYqEl9Bi6UMpgMO;jX%$<*%6^V!` zr0DmHCPfAdkB%5@Jk^b1A_3YkCyQ;CvV0JihY}LvWs;{=E~_&r8Y4 zLpYuy$Wj~R5|AdcK3;wW94mXLiT4>eK2mTb8D{$!-Zw^4K-V)F$(z*2E4=zjIR48^ zaGY3?JJ>bZL-B+>UGKZY93_!nN7TtN*2xokFtmCzsl(mIN&Q0ox#R~py^FC(*Hvnt z;n*DGILl@Z6BxwdV%QB_PWJ|}M<;P)pvt)ak`|+KAL34nzn zN=2QDL)go|p(>puUIx9wdm$cOugJe8uD_$=g$&~82=p)9syqg7dogEz|G;(8*4dROM;aic~)*NyC0hrC-_q-8ApIJZfdfut0v7@Mu#+2PGF@mjJ%9aS1zIlKj$ zr(qM+Ha%|J#FY^aF3pa*)RB|aeRSN-qoC=9^XmR*7!_y7y@*%w{4lx7&WhjCyebsV zy}&j$Rl-L)^I|vM`DI6&)353pUk7IBAspe}LfwJN>on_qD$Y`&0Hk(>wkgX`DZ@iz zAH_*Jsla_<@q)W_<;wEv#`0Aaqm)HHgC2#W1`V?>W@p?>bV7+gLt$vRAHxk<8P6R3 u4{55GaeHRzwr;UvPM@s)rnXQsYj)8rTD6jut3EXr)d`&$zq9gFPyZjY`rX(7 diff --git a/openblade/api/__pycache__/routes_inventory.cpython-310.pyc b/openblade/api/__pycache__/routes_inventory.cpython-310.pyc deleted file mode 100644 index 9c67df38b08fde2aefdb2cee4f5cf1c22c4eafa9..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 2029 zcma)7&u<$=6rS1NUfYQiw-6|_<~LT6LoXaq6tyK%)dK;w;<6}OZFVNX(Rz0~yN*Z{ zB#?8>feU9i_QwCfe=%21962CCDpbBVb`qymMc0})@6CI=Gv7DwJs0hE$ngC6YisYO z$Jp;=%s(!S+l=w*Cn$nNOmJakd}KuyC*98MkrO#1H*$^bWZuY+e51QrFlt1NQ5c1s zNoT^M<~=4n;U6>MOMhZVE!cr*z-}134LcM~*iB6umfzO8xhv~9of;{9asIJk+$M(Jhay?T z!IJeAxRJK7)|jv{JD6R}9%dgW4s`2bR@4tQDD=fuBp`Gob$aGMo>9`z9 zaT->&>K*kqi(G1dJ5ec)xcS)8K^*7FNXD@a<9JkvaYlM8jvtMaZ01>v<2mSY{FtdL zIHwV(w^mhA)T>{BfV){DL3yA8yxhj9NC{cM1K#3h^~J<| zg3y;C1_T8}5atHv5JWX~dqWNPE;xO)|Z#z~P0WbEySdvB8*bPNOPZ zyVT85G0yAQ7@KO5BF}l4YFBCP6^JfI)qApX|BLty?05~M zqCRkj7*|T`r+gJpnDr^f@U(ad}BE$4rp=@uiuk(FC8m|3h&LR>0Kn{b36Ok z@FDF?m$J)_*j5+2(k>js?5>t-2f97sIm%V=6Z=Q&k%ir?&AzRO2}KEDlBw^S^nPYl z(M_RESc8wXL-#S4PLxSHT%+qri&44Whh23bJZ5>;6Nfu|nLGb_PY>Us#&Q8=Ei2L_ ztJVh>?>>i)3c#RShhorOn@1Q#J4t&pJ7QLmGr zgy{RG8m4~ZBFM(C-TfpR%W0UU-ov_fP2xSng8BfiTNo7;8{h}lKMq>PGWYOvo_J6D zj(QU=)gy78#0>~ENPl+Ix^d2r$;r9!4==xCsds4MCW(vMHkZ-9#Pi#Ts0gw3eP~;_ z4RaXJIrRc3aQ^oGQXj#&iKCehQYNc+Nl;Dnm$V0P7{N_+|zzjr# gE?Ap3|8=Utir#Wq%j#Gy-a)=~?2gw7TK)?E52kd;y#N3J diff --git a/openblade/api/__pycache__/routes_jobs.cpython-310.pyc b/openblade/api/__pycache__/routes_jobs.cpython-310.pyc deleted file mode 100644 index c4cd5dfc772bec531668e94737f287636885d720..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 2013 zcmZ`)TW{P%6rLI1y_ak@Elny43zW->lJtQmh@!R>0hL%vS|sEuvYegS$cw#p$6l#f zHd4z*qAz(s;te4sZ{&CIFXojeegLS-18~lGlSUN0Iv39wANzdgo3mM?;S;#lK5P%$ z4k5qc&H2lMn;V4C;uml@2?(d0B{XFLqv#unk(z;-T7i|?ft@;mqu0&EO})U=zLofC zEvTjSpiYS}XC!ERLb%PHeZn2#&5WQa8p8$dik4`u5bhl?;c$OOg(d1UI$(U6FY@-j zHKW0z-nk^2dRG;ve1*TnFX?!j`}auq^5o*3Y`asGQ8_L;V#M<-9+k!F?^s%ws#-J} zWe|+BQPDM(ck}LNo7uP&QaQIp4*Lt$`1Jn$yB|O5iyU_;>&vl_yYQ%YD=Nh285c>H zDgS1kf0T_%@u*a_9Z`n;>ZXtHIe!^&a|2=A1V@m70+j4408kj5tyk8VTNA7W>o!;i ztUF)#=GF&W3rt>zSOa}1xT$JT@TMqos7NTID5YXCR~AAblsn9}!*Vwl6CWa-X{VQs zgtCN`nVc;0xL-bi^=qAMdno$lmU2^3Mm#Db<@cq43_OfV<&Se%(|+&72S85~`ANJj zBe@&KylW{p3`bEa!ch5Pm}Y#OpkEKehvO)jM;5~HTnoZbUI1K?x-!KWN%~7Gf zV_LQT4R}PpjY^XbKTN&_>OGJG8*vpd1a81-NU|N}oqbu?s1jg3%0wns8FVh4rVEi} zOoO1a$F}9x>_6OwN!v_fY~rnb3(vu^*;97q$QrD_Krp2DuUOyL;5^Zl9nvW|0%A>> zCf3E0A?>E*C&G=N4Q?K=?`UaE88CzaL)g#;aX^4{L+{WBwyDJ{tV3dR74{DJ3kMX}OlCNP@ zlkRo&ka>XPyXYg1@=a7{Q$TY^uEN@Nkm6-f>~WpB^a2v)>{k)Tex`SJ?W!$!q^q2G zoaHO1UIhhl{W=8t-*LpbyQG{j`mdbCIn!?qq_QbVN-Ku?)SElmtv$T=9!n5o?LYNBr7 dxX`D_Bz3mT>a@jd)-qZ(o7uGaSN+m5{Rf-O#kc?f diff --git a/openblade/api/__pycache__/routes_restore.cpython-310.pyc b/openblade/api/__pycache__/routes_restore.cpython-310.pyc deleted file mode 100644 index 69e77bb38e1870b0bb3f74cd4abb8a2ccc1a0287..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 3974 zcmaJ^OK%*<5uTp+&OXVdXiB2wwVqkaA{D_&96?YlQH&!+q8O4&;DLd`aJxy(dLHYZ zUOg7WKoNE2H5bbOiR6+$kxTwXUvttervPRGB~?9KiW1_?VtcB)tE;=~@l{P$uX`Fi z)u`V8>!har9Xm%K3p$@`8p&sgrcuq;D4}{x65ZDc(nf3~hHoUMZ>qi-TZ!%4ind}W zaeY_Oc3eq3-%F}~HL3Zvr0&<%xDz*$rr%Vw8@G}Pe?rlfcrt1G?PSWIQhhHzmQ4E} zD7p&tj6b93T6{b?;h#`+JwBPt`m@O?{}j=fx2^f7A8EAlC*prdFVQxgdTII{dW=rL zBJ>m5p&z}p{g1xT=oxzUrAE)P=CA1e}Ak#JUzCmb>~LYD=ZfoXPukNcRDPkgDgr#KKB~+*(I_XrfDWZ5oKxK z)ul^A!90&#|QU*-saws0b+Zqld`f1mc6%e+P4F%)Uxy=S3Hkb8Zt*tFDKJBpq z3X^sYQinOLs@w{5c0Z#m#!i^T0ZuX4&AT}mEM%!*Plc>*G74fPN$YVK57~Zmtf@}XSpqdln=>)wsWfuId*GLc z57M~T0($H{APVr=p~kC1+a*Hp8+&9+OWTFkH`VwIP!^6;vV${rq3zmZKA@dFZP%hY zHTHC{u(B^OBSFT9ncU^A4R?XXB#W9DoK^*p&^ul~> zWm$>xY$M}I2)gDo!1fo7pM}@{{5;PxG5-Lpd@Bwqo6mq+S5!ENU=rLQA1ifk@JxCE z3I!V{1C*23pm9lqc%(&YdW-P$Fo37F80Q&y(1F!?9mb^%rVS#>5dmba7m6^>HiH2; zpsY}ksz`nTX7@eN^jP~Q+Tc*_2oNZs3V)^ce*l5%2GuHs6EH&G+_ee<%C`$!DZWj$ zRjsfFvy>FZw;I(8^IPpZbIXC1z|(iAQRvhZF3^>2Qs}~atnnYKes!CqglaHveXaA) zs0}@*aH#vr*+WpO6go)sl-#P4Yj2?(r_c+dFlp_Txm77FI0xZh7FJQA_3L=zny8D$ zwzf^iyOcb6T5 zqOoH>4VrmPq2cJKT}PVY*?^VmcLzMmcqE?53Ye3NH3WuwZyX+~3q08-2+c77@?k7S z?!lg&k$w%gy?mnskYRyc?;M!5dSF(B6@VN#5mL&idP4wojE%gW@^U#~5n=^<) zbsjtfSUeTd0bk3*O(q@61O#*5ZE;O>K4!Rp%0wTG*V ztI`_|5EK+oD;yv!DoJqYRPd81%T}_KGV>=9fU3tY;`ooSL67BKY(B;YeOwxNjGtil z1~$LI2E(#!&H*JG!8(uVCJRJ{+ns^2{1!BgN{zV0BvbG=iLUEJhyD~qqE5{3%{S9E zQ;q4w{nbU24?YR?z{ith*mxLzw;Q-`y8=s4hW|C;P$@$5b~~XP@S37xQ;X+ z6woHn7QAg71DAAo6BaG;OVFyTg1)mM@3Athm5%W8V0$+R(lB8`0DuxC86rH;wIKLr z7{=vHDWMqzN0L|&@cStIt&$Xlno@iJ28i5+#?xv9e`@rHFL@eRk6x0&CGj%w#*3P8 zDND8&Q%F-oM(<|p;Qip4IV%l*XxOE$$(QgD53pIr29NST&$$8!YOfV zpUgv!Ye1Z-9Z)MQ1`VoA3(1p05b!c#_H^JMKNTPl3jc;y!#Mu~0--j5%+e=YI)E1D zqe-C)19A}5Qakz!Qjos2Ya`?VmqX20`kuaJs8P4j0X9u)ZCLrZS0L_&tM#iVEhm`>q7FlO%P$OHdl zD6_B&qhH;I>Md=HY4O4T;Y2=*fa=lXk`cXdI1o+<^0}xLY`=9^(=D2t=+=K8|Qz%tGysfg4?eaZvRv%1E|xb z1o0uzRqD~|4(93F(&*yRY}Uzm=ZNcfqP&Bqi#VfQoj(L?0Z1u8Tvxa)p~+>xY5FK{$%iO%QbGlif}!R0Rw=NeTO>u#yWty?QlxxU{_TU}14} z_2#X6i^1a3+MTs8`2wEp01p*PV!+BK0_XTviBg2k3`qOnifue0d=F?G^Sd--v61r@k`o6hM}?m% z7-i*cDcUM=8P8glo@HnvxbfS#+`sI)0aZ*9EQv0uw4H8^0gPXO7IeWjF zgbX<@L#4r<_GE>#4G2^yO^<}Q!s{k0!D3u9eFd`f6dL_`&B8*-2Ke=e`KD$9B$Ez; zF`0(8rGOlu9NzZ6UK#fR-v4DjZ#c+$kQk|HtN?ccRK0!HlsYk^8BNElT((?uB1Od%@ zCCG0Kp@0UE=ZIaZd@UHXLTJAo17{UbPKEd3pCP3~qichIsZT2Y1&YS1L*-wE{L&m`xsVeoGOu2_8r)o1SX^FPysfNw#KR7g wopi64N%IDRh04{Y2IaSoId2AAD8>YkxbG`?@mx9pZ>8d})=Of)P!zy8o! zf9zYh3%g*|Y=*nZVd(x6M>F=;LCMKWQ;M+TFm5B@DxH$2+w!`|zp?gBi%E=@3 z=EZ4o#^}u&y|Z$<(gSVcRDR&pi{hM^GukJ``8!tc(&o9Fp(<4@*5q6oX5pGlWme9~ zObqfkD~pBS2;3gi%`nUIGA!dfD|%e}*FXRGc0Men(%vmOfPMib|3+BIPjev?2rVV= z4`U^Ra9Dl~`Pf)4qgZvxk}1>zIAa4a`k=-qMqMp5K*2h6(YlAhf#j!)%PQYQhTb!(?UBNxNmJ5cW+!iHq_}fPPm`hO4VG z0$jl;7=^BX8>%eM)(SO6``d952npq3RK^dau9<9FQ8QS4j#R~+qIYLNW{EjHmR{`k zYfw2NzxLdQTQxoNp@}IncQsb&Z6rIHL(ZudkqaO%v1cC3=P?TESQHclgWkQ^s@Ji4 zgjI*L-h*{@J&EL~@ZH1R;N}&>D3V1{MZrtKqUblzh&VFXU!JS79_fwTUrq9`ysJGS z%P>xgm$-8g#*u+V9i+-JI7DxE*&>R(twu#g2`>}f5~gd&EQ(P$-l*gi?bfc68r%dlgXzhCs@ixZK)EJI~dBmvYKeo24b?Ybg)VgKuFhF9j9N5O8Np*3Dwp|9Z zh7EV8wpnRyv2}*QF>DM6qlV!M>xug#zt3?P>$z2sC$LjdZK*DiNg~rAJ9BpjUH6~OdD0B_PAV3bk~|8NVj0Z% z41~dp!ZJ+qHQhO+-(%`6GJdbgC{s0W;~k-6!pa}IyzaYyD0;hja`F37ac(bG?QK!0 zjVo%HRbEpiL*YkC|GA9^aG)P~y5s+Qf175M)sf;Hqv@=M;%p7_M zmc4%%9sB{b3+g=lVYkNjhlaVk>Rl{scE%VX)2Ux*kFG;>i~|r+BG)hsL~nw54iWOv zTDPA9rt$zd0LHF>vDGE)P*jEL4Uk@4y@SaV7yV`B8MP%-mxr}&^&Tx5&ryNJFezW; z_Q;1%pl{k2e>mmOF!esZmIq<9eZ_IJ{0#!MgeP`6Je;iS$lsF9<{) zRhQ=K5DfyCrlS1P-fD=;b`UFqPf?ob`d~wZSs6!);M8aLF31JwEsQS6Fip(IiJ}&) zPaV`5f2GjZi}FZe`nduw^%8wD9GHfs@F7ZF%X|-2b#F4wI78NJ8~2-bsN_uTAhU9? zlo$rIeK?!?GsRzDsy+>tslydlK^&{ayS&9(7+vP_uHE(VtIfJIY^Lqn?Z2H4XPmW} K^Ox&Q{rxYBHk4KX diff --git a/openblade/api/__pycache__/routes_volume_groups.cpython-310.pyc b/openblade/api/__pycache__/routes_volume_groups.cpython-310.pyc deleted file mode 100644 index a7c7e9b555e3c8e25a339970ac14510f39222d2b..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 2374 zcmaJ?OK;mo5Z+xo&|{ICfyA9zT8`{IRv|DQYjG_ z|NZH9|F8-98orm7Uf`~% zi`op~m3<}(!ri6olwaX>KE7AjrQSIBOn^^K*xieKV&6FRxymQ`l=hj_KGULh=06&I zmS58IQ~dH>(tLAs_S-lb^u$LSG9Dz2#XFxjM4u;dr=O**e-WN0Rpp@HkFy}_#Qn5s zsv>x;#Dh#oW#1GD_@}D!*}Z#rZaoM^f?mo>!F`az+Tx8M6<@_%L`|kji%GH+_cQSz zQ{@ej`C+bvxY6h^K)DXNcrposARdK$ux}s>!r*M4a06mD)pQ>FZ47-$iXam!;{HIS znKIH$DzhK-#PXr39N$OL_f^UFdodqGD7(IYe-K0`Gh@C#a!TJnByt?~)~};a+cJ)` z_SbOM8&SYTI|gY@OF`0sNdP1Df6z$!8p*VkJd|blg9dhm}~LaIDG2 zH0^Bkmjao|4&N9aO1dzu?6p9K0Ozu-g8DqhT$(@|Vs-(Ai$m>|Tc+2*?9EaET|~EJmF)`wq}wRnAGI>1#rrhBX*v>cFc+9O_U& z39U!84b@YXC);4!M5OGKf|2O2EGe>q>>thOOpAU&c;@jbFd zy5x6yKyH#<3I{co&n#o$3H^fH#bpf;HekW3yG=8)MY|Lp29LqXzVQ>g&)^T}p&@6$ zQ9~l%0I6x}L!GxW#aeT!petr1m6%0Un#WUxaW7f83c5zfB8=6Kt>evWICrk#=A$?a zqI9A4d?8TI4udR+;tf@QaY>WP%NXWqp-Wvq?|^M~xFy@MOov&|BwCwHUV@J*=kx~p za-nRQlP*M{h68cNhN`jt5`;$x6rc=Pvjd>>fXa)Rf%v24SD^H71~>QF$!=e&_Oug$SW5J4N@4;$hE;FBIQiS?~4F0!B#$p$D* zN4|}=8n&NN#^%YwnALcwo5|T}EXq88I-!Pq7Y!P}w@^9<;)h^E$WkOV+o=~OX_Zb; ziT2HE`$ud$+wDLzJ>MGX9U^YSfnDPi!|y=>3|Wrh0o^vXpyUQX@{Hvup1~0Pz@uvd z>V*Tiz+PBM)Azq1k7;IUFQaS2*LjZb&w#Id4>sK_HK6%7*5g5+%O)6d7>gQ4`2iX< zXtM$5{t4y%EItdKX2TTvY8bIq40)mD*J7z^IZS>y?1*!)KCQuv?YNc$u-2h%xCn0x zjLX0_ofm8p@(R9?K}{=98FuCK^B5IWhK>dEkb7c}c&X>KvVEV&VSb~m51RpN diff --git a/openblade/catalog/__pycache__/__init__.cpython-310.pyc b/openblade/catalog/__pycache__/__init__.cpython-310.pyc deleted file mode 100644 index 134a98e004546a633ad723d4579fdd0766854b38..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 279 zcmYk1K~BRk6hxhFh>&^*Z@X}vii9LmRUx{E1v?~U{n|gRE628M7Xvyb%_5*W9q9z)bIp_NDn z7bpaI(66ZIpbbU}R?!3=Da5Gvc0`-v@?lY&uY^!e3h_n4ETb>>6L=aumj+G>S;bj! zF0n_C_TGR(Uhlf5s|R3J)%C1i-L$MnXj#*%W;ImU-1aSxGC-0C4~oAf=6vKX19lgH c{433c&{iiQmj3-*u1BX%1~=2xICUw>FO76fv;Y7A diff --git a/openblade/catalog/__pycache__/db.cpython-310.pyc b/openblade/catalog/__pycache__/db.cpython-310.pyc deleted file mode 100644 index c41fb279a9741f5af2ed3da21570f5bce35f7e9a..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 11355 zcmcgy-E$kqk>6ba3lRL)w=Bh4%T7eZ=7(&_wp@xUvT(ZBT$ki2NtJYUl`8)iS9SLn+~ZbWQdf0VdAXOo>EbBAp8Wuz#gYG%ue_8^mO<1%(Oc?T9)wnm3DjU%A6$qHx+t6c@(}WNwW990B%W4T9cT}6kB$b zHAN;kXXl*!T3*0;yWps6s(=f2(J8H!oT0TLr@U4cb*eq=jI50axM+_$V{2p1_}aKA zm+T2=a&1z;L-rBp=-N>Mm+fQD@wMZChm8|##5~DHuM&KUjR77P@M$&ycoOi}*>QH_ zL4NHe_BuPoP6Il_-e70gSwLsmG&{#MKwn`qs6EeKeNb3?na#4Vvag}`9GhcPtO7`5 z!z)tdV*8y1!#8S%XKJ45x0>3z%Qb(~)KI3tKl`xx*z0y z^PZ1(YE`_bC_!-rn4lDlcp=~z+a^bo@(OiP|HyT1BK!gP#sB2BkeOw(>3Wv$^4+EO zXh=-!XiF}27WLXb+e`7&sGr+$vr#>&S#;}$-7b4Rk6#saR?}PD4%FyBhnIoMAuZmh z%Or;ALvR7VYottP0d%B?P@hLK9|0tjQnB2l+zRR;`utM+n0Lp9kY6_}H-z4sot=GL z?!td0l8_7dF}UyDwfxPX$jzp0)XmRjK8d%VOZ+tc9^13N)10l_MvIx#bFJ67WhHbUYNIb{Egs0+BmdE70%+C;0lgz67pF&=5 zXl`u_bACG|upHV6ft>)52yBQ%Wyvd&6^bfQL!ktd^-Z(Bt$T)J>i#D5ie0bBoN)Om zdXOx_$4h{MV|s|E*WE?~{p$wb@QAgD)e^CKj#}nPUHZ5q&XVc;HVRB$kZzyblZnd? zP3tR6=}0^H_o+^yy8H6O+#_jE=_p(Itpdy8oBt`i2y7c8xuejtQdp`e{1_;+^88zDq?5-?;h&9KaznPHy@J42*;s=812&#uk44yON9F5mqN4&E|7*4vUJD$iNt>v zy+0P#bqf4H7(Oo#D@9o@Zx>207!Bv z?#rEGG}7rXpTnj${@}_*`E}>npv|S32KUdbO|gT3_j_eWkC3 zrC!;P6ItZCcg5_h*vkLGi}YYV}*y<@WIR=a;X2dwzN9tv4${F)CCF!LYY!a0b8G zYWQ)%YA_Q{rUr6w!SMaTXxOUBT^e&R6nPDc^$nXmDGmE5`GWHREE@JGW?tS?U=8CP z`6uK@;XymfFN6@fBZcR>2oyexM=%`9LHBo?SO9|w;S;UK2KXfTcp;i_{pG8)_z2-# z2CDF2s-!B4tb9?*<^Q58?N_q}mEZ;w4a37^dI&E}eao$7&nTx!;4}0P%MgOirxzst z7V552{UtntamdTJ3|seF^*SP+_Ddh!yt!DNU(%Ltu4+rS78kXJ>h<|si>umPP_Cmf zx7ddHx3jR2uu!(G8aMc^Zn5@Mq!ik6^?G%=x^%6&q9v~zh(Q`)a>`(o7X5G_! zQR5}YLb$oqPjo8Lt4e#Sx9g|prnh_JJFthc_S7fhC*-`JpVUybsT4X<1B0=kK3cx< z;r#L^+IOp;yfLsX8L}2xRd-TvE#3Iut?FP5(`nfTl#OD}6y{(QEcc{Djh0h0d5kl- zspB#;O*YZ83oCFG2G^{&E)2%!u5D_o)sI)ZC{1oTP@oJ1&*cbOI0Iv`&=Qp%+=baN zYBu&9nWk`YFkZOJV7lRJ3sAY$8y{v!xYcBEj(rVNb5}{#1d}QK=yP(tzWrG}ft8qH zo&?)o=+*61cfHEJ&pcqakjT7UeP)VXr1V6!&-aq)`k`Kq+DrSJdKXJiMkXx1`$C)Q zzNr{L#O5b83;UQa`2zCN#s;ES*S6}G>1A7RY?Ftx;K3RwdfgR^Q*4D(Z(ph;c0U*k zTa$HbL*Me;Mrc_n-+$rn_cE;O1?#-=;1F|AHxPB`j9WYZ-}0wDuh`YZ(tm-H{}R1a zO3BmCr~MzPMh93zs8v}8ux;)pGCI%#b_~0P&}V=BO!1cqg6lC!j@u0^4{@gZqn{V;g}eU#AEuMo1>S z8}0OXw~X!`+A@W1hyta-o$grNZy{z28^_Khj*wC-Kt{;3xy!K6gE;k?xoPZJIJ=Ff zFGb?QotD`$Gk6Y1g+s;-(?3vP@n$G$E{srwgy}we4iF=(3>102`hAKvVh@~^=kacCi`VIh58g9dPF~n*)Ajs) zT0NtY4KS_@!|1pESf2+s_KhYEIn0p10mho?BEn+}$7p6bm&}Mu>{!LAND?Ne)P635 zTilb_dTrOodX{PF8MxSd`zS&@=U$#R1HFd}U&2NpGUy&dOr~M*d_R-faud6KgvwpP9zZoR%LC?tPUjA( zYL0K=%p%(_h#MHM<>)Y?H!r`vPfKB*HG|h_NzEoGnz`%iWXO?S3>N5tMgSrim!QZ8icv%gEJJ!Y?s2*0@5WP=sB01< zT^>ApB=F}C=AOxj(xy3>1q7hJ)esR=mg9f+h{d+uyRcer*fx&TG97wG=;j#r;yrRE z>YIZ{5e94F!n@Ck5NnO^>OJGmm{r^%^1GPU=;)-oJ4`KZ7rbq&smBNJ8RWQmr)6=R zv5^@*|D-z>(g1L(e>iMhg5m(|(47l+WV;k;iepQ1Txli0kQrOVDv?|oh@DKP5HceT z(p5(sR30v55Lhu2&%ff+>X}@892R}u0F~!Qe(G8z+qg2KrB=h(LC;*&Bdv#N6>MJAsbx{*# z__#cnNIXGIZ#*{cxZ-H+sIa^xy`G6%4SG4592qC=Psi=kugX#G2e!%L5|PvWTQD9U zHTT^deuG9Ot`Enbx*M$EjrU)8l#u^0A|I0#x=9pY_5aF6RZ;(<7DRncc~X)~Pw@VW zbWORCM+U`Nq@CnJC;ud$3rm91ll&9)2+t2U{XuwCw*Mv zZY!LU5kxwNchb)zK|z*90*sG)ZnQe*%alAJOTO9(-ysFWFA~i1zsVteCc@^Id?b9Z z!d1y1>d5Oz+vq6&CjU$}WWT&M{BXnXa0G<&q5L z%;oVfV5qPvt6!80C8*m7>c;Tn8!-!Y0DT|2#6!dJ9PumE01ihDZ{KyjeqJ8|H$eNlx2L4fh2|PBWT(3r&M!l;g(5wOp57Afh@*>lU9|6pf@gNB(T}ugq=~^wUB{wQ^szM5)R_343 zOC_u#6^UUIQRNlnI_k`=>v~Yu^-#feos*=XV6GTG`8={<$}00WsYI;f!tjo5*pw@> z8|o*m4Z_0*ligP{J*i&7SWi7N0Yi-?-3`Wj-jG2FwGU2pr5-Lz*Um5H+l8eLq8||S@iR&fPO*xJJ2tx68a_S&!Jydlju)Me<%7CHHH3^^mm~@ zt!B`lk^XMF-0os`jD3Px|}O z->(j!e?a;N(7#FDjQ-8izX|Eug<3 z{X^&*vkE%P+zeD=BqJO8l3;nyKe;fM8)N%BWOaFHC?^gGqe~vjF538}Wk@?rHA3FQCH$Tv7TyWZJ)keKvZ*^Cz?XGfKZgr)rt8eW&od;WW_C{IlwCtl@X!aU>;_$yHIw#2oMKqkqQWU%p zQ&G@<40|zV#<}$Re!?nl=h`jscjc{T-s0`d;YB=BvR3QV+fJ=!(zP0PY|vsmTdO@W zsJFb4M6IT}jauzWL{H1m|FP%BmL}A= zlTh*dBETkrn@|a0Qz{8;%EL~m^mDPLwBRzqOyV*si+MQ@2Nnk|EAxsTt|Yje;L3uF zt4YD-<<1qs6>w)>TLi-=)U@DAPWhspSpnN8)a(!|BRlY%Q!+Al9~d7meU~Y`xw)+C zdhHD$w|b}Hm{Q9kYE2L0o|})EOiwr4m{z+4%8fTW%6aNq#uDMNl4Z(<$C?T*RHM&bn9B#TMhH7PgfVa9p}8sxsGnu zTeZ%hz3k}q)Y(QIY*209btP&N%2}xoT746jH*FH$6Pj@i?$ac8lXknhU+Ze+=xbST z{bexcd6RHDd^&Zl=%IZ!K5xDSDhp`{16%= zxnWF=L!8EiBPKv?36EA|P;@bsdTx9vsfbStKINn@W>iL~-RFQDRUFkC*2t?e=1yX6 z4vNgj76R%nLOtzJbI*=H6J06^?d~?2$0+ukhjn+*Gr7yssjaf-p4;?=xOJ!SgJrXetvYJT_`mP4d20y}hjJ_jB)GggFt(M%F8# z$2jay0O`zOSm;Mr*&+>`zA~owutNsWd_DI4-@?{m0xoP)r|KFyL9IF_8~$4B8Ml7f zS+Fz+Zp$y%8<%RmZjXHAxF*r4yA24z^%;+(OMH^^!JKeqjicF7WbF}9wKJRP|heF)pl4WA!d z+;9=Mksp`jMpAw+rk|g9SM=%V#mHlkXyl2Qikyw;6BvS)O)S@S11cKdKX&-2Nd@01 z^v1{mCUxJc({Y~a>HF{F>XiPm_f#V52gC2nTg=6a>q5G?_a6?fqBx@fag;udYaw&S zU8lA3^4lc+C0@t7@QYMp8)?Fyus9#nZ@{>IBfxz5Wzs|6#UYUyA{a<7`WS(Th=Bj~ zapHvesmSHo^mm|Tl06MQsQYUgX7FgKa5n~S$HygcQYy^FW}}bU z>*at=0~w-E^3o~+iQ#Gz;Iw{#{Wk$zD~4r*d$_6eevaJ)U<%=}T2rl8HtBU-04Wtn zqCf*fp9dN^9bBbaWbDXO@0-p_TH{bHrF43>#sw{Y`GG=uzd2O6pmsf1f zVRu84ULS{nw?=4P=wj;Jbja$At30^M;xob*mfzw|R``u0W8f_6?vii^`If5;b@VuR z)8Y!kmDVQpaYeyR$+dAcEj-NYOChV&+S9a1%f3YldwSBTT%1xnF?TxP&KWqOQMDJi zSyja|?Zdj@NOgb?r@94_VF#SdggONNoCkm2svpMIibT9|{`^iZCHy%#)QVd%=;`?} z%TA)=sol+Zuin34*;3dg?KamP%NbWsazX*VXXBDPXxH2e^*fK>tsmtbg;05KlL1;Q zICz6j-|vX|ZPLroSeI&C@M@J83E$`Jd}spdZT)txk?Pg8)9FJBSjOGRa=L~&>yw)q z?!0iC$?}T_?m;GG4y>2!t=8>Tc*{=TS?KECg?eXwCOrM{(3ZyDkhAsc`9w*v$*~Us zql-Nez7(cMDsVv-=jXP~?(g7Tgvsk{k;g*Zuh?}6RtPQ-Gzngr=i8hjqI(ELh?Ca% z%3R7dU!hi_vmWzXwx^1P@g0i+YyX%3PpEX00H6#yNjvCkXk3o~fGbKRn}UL+GfL+5 z6s)LM;wc7*x6Wr==F%Z0F7Gib3~$WKhL}GGM(4F*<}ct{&MT@4uCf}ISM za)mJ`HHVpCYRudP7TpaN-J>t!>zTaJQt*;*oLe-yhA_t+ApCuA@CCh8A1xb-q$ zZe{X@BN>2ik_(3{O8S-gmu5YUR{`nI#^jA}P3~X73fDvK&|fK#RoKIfE5ufSv&3(S z9QLH(!g?+(dXhw51W8&AmvizL3o_H{F8ZP4j$mb^0;VbjTCJ>hN-&ka=%kthLGLnS zZ|p9c@r&K%S8zSIYZN}iy%wdD#j0CTuScv&+_TnSgWdlGFB8j*Z}wX3o$o??(CODf zj`i9_x7)EABJOD2)gim zjc}Rw6>870CC{?P)+m`T8*!GS?;;pJuCzqv_c8Dud_pAe2UcuQ1x6p8D9+nN^5a0B zq=bJ58ly=bL_84+GR7suNMdK;G3iO_6r?>vk0f~FCCu_n1TsE@UKWxzhu4uerL*-F zr@wZtd&%io>4C7CplW^M+;gzG=kP4zXqdPlJ=#jqos@vG$Uu|t*Y&E?uXzsbdd@Q% zR`8M$15Cr&ZfeJs;8^BWUe9}ZOL1@C$Cl9ivuusp@poh3Blx%s-cZX{V$`wasOHF~ zJr1)2!JKk8KOvel^+A>8?JFxSUr#niU?l)97i5azK^MVV^_kpJdOa&K#7Tf>=8RF;`m`=*)g!PS#$7OVQs?}X^?v89S!kDxDw8iW^T zP6-YgS#b6_r)5R!62k%TuAY&Zdp69Rm6>#txuU&#hv2H9ynUd&{U&t|%JQwfH+NS} zM)YR4S3`c~oPH19nMZ9v1!^~ejtH_HCeG*)h@rR4 z7D>S%4T21c%NXwRnwg-0Y^tlG--eDWf+fwahU1DL4;<5hD{_kgE*DW{E%Jdu76g$& z^)?=CvW;%LM{CyxfM3lh{T@D&(D?VVMH9kWrXOJIg9IXxhF^luufLDr`w2co@L_@< zAP62x{~){Mh>`5_ehhpPAD3U(XP3Dk_B)FfW5G5aOgJ0Wyk=`g`YDW#5>_Bv#DPIh z6DkhEn-GEki2@H%1$&tz~(VCE97ID#Oh;W5go?5HV-n?$CILu zIYOo=Bg!+QW+gT}sdj+)cACTk?bf46ADC3V-KrrTpT!@ z8KlvG9!0eD5|eSzq8y%&+8aNDcIZV@37KRQa#ppQo!>xDbY?oXUi7~8VSg}CnLoni zUQK20<9&o)KgO0gQMa-syz>dRURh=SD5s1P;2*`n&*S6H0+1qSCGJQXwAP|1OwQUw zgA+iWFL8p8{YBe^h!rNwlI^XztQlx^e-xgQgM zmy^ZLEcs7AG&U*ww!FW7ftQ{j_zc0QSI#^99tU{`;e^Hc{HquR+@w&4#RytNA$yrWB1+32ZT!WKzt30p+zXu>v04l?;^{piAlx_bet(if688kHbx zP=EgzUWfQAChoNB%@(4N_5%|Lq^JsV5E#k$hnYzpYe*NNc7<<3V$xbUSLkkI7N@aN`4qi z%|gQOFj>mT2YOBMq)b7id_apML_G~ylORWs*!QDn6M4(Tr^U^xkwG-c+WPI!vn6Kj zvuJJF;~0>qu*ch_?Qd~8q3x;gvu~ttlNDOTa}J|Z3GG~lZ>ScpBT4ylyqOpf)bo}^ zpJ$6-#S-B!uqCF%FS0d?53xxTQ@F({}-!SGM(A@g#toF-ds%aot!E1 zjO=96%^pl(h37;m-uBb!zrH zj-0XSMUDe`l1h)x`VG9ua2(IN%4u2)2KPr2>ThEyi>SX6f9g5U|~0^ z5?)`~(PQ&c0H^o zFX~HNWB8&(AuJF#>XnUpTVKTFEUa}YWVK4dkH5}EHAlC8Q?JA5w%!y{3lh7ttc9qn zN?~8qCdGXf(lr^`Q)!2{(~O}=a=39OeYxHmI6~-)^XXS1>fgrmSr15d=y%v1wFh6u zzzp|*nH}F`)5LE5-Jizjw!XU`cYF=SjVfaIN+yZjJBy4q++=PZlhqqV$UE?iHYQ_5 z$SV2>!ExTpB#0J*43S~+RkoH(hn!3CeureZUE=!+R@uXi&!gcJA4LU7k7Jv*Kd?-$ zzorB7oo*R}y*AV3$1GwZ;nN{OkYddwgoDj=`C(-{4+je%8myJUT4K(#^&P{IGBam+ znP;tF))(6p@36uuj;t4PB}otaL1uf8vRrFj$v8sHoHr@+Y=}9ob_=c|I2%mncjJAh zlm0l8$_QSws2*I>X5ohw z#S(d(y4=)#l-OgK16~=ki6h`Wa7_8anz9L-+Ol(@ei;#NRG5g=_fj`ab*&t#mIwDi zpF3vKYoLBvGCY#vPN5K5)w-S58Zy|82FxsZsG&!>I6w0$Jhx>)=l8i&e?UOTQ~wcx zXo5dx>rV;(jNmT_zD6MCm;_Wu$-}Q=$|1646%DfGTx=Nd_O*lW%4*Kuno)aLXSAH$ zy5t#TjFdFkEN~dWyeRAkeq%x!<gPn;($Q<)5>-xBg;B3KU9HrNsHzn(N1(V*j(=yMp zGc#L!mKk{#%g#P-R&ZcvM8AC84#AeQmDtI`*&!yj)m%f-kd$N? zzL@o+Qnq5NUGGWEU-AtmEtM%=)mI+}j#{>8%qICijHfZhMz;t18>L}hN{Xj}I?WZz z4yif|80LDarz9RGqD@qz1h4)Ex6^-1@OK1{5&S*D*9jy`@C~-;dfRN(KeF{rg5V)W zS}}i#Dc^>76C|qUB(J-XU&B4PRJ>+u-aUrZu8Vg^6_VIUXv)u|3(eYjR=xVgC3ap` z=J^?4tA_KTh%M=zA+3VU^E0~@$Onsu{T}csd7N`gg0t>F^Q*+!LP;CNjU$RXRI!$K z{rgnpTFbjWZc1=g75lhp!GWLE9`N5@@fe0RM%H5p7f9WBebza=9IJ1q!G^6C>85_p)|?b)j2QT=J7mH;9h!I_|L0hu-RoVKfSU- z*;DAQthAaP2bvG+5a}3UH=LZH^Yn7g+P(t3%4ZYy63+T(cIBakf%vHUTLh$S{VxRn zO2C`zea`Nh+n)|M1!SVHl3)VGe{|3i7lGw(ktip26(s^ zxk_;}HTjF{7cs8?li)K1qpY|4F~C|K{a`hGI0G=n z8{9=OL69a86ZF5?66PZtnu33v(USrui%Mp{tjX0~&ao<34lS6XZyuv84u=?4*F@{# zJP?YnP13LB^g`*b<1nie;@2$K%X^0dYp}3gocGzvw0ytQVBT5=cWPUG}@Lcy9$HUfr6?Vy$7!|Hgu8~i zr?$!B_~0t)QKSS)GEAoEA498omzi>a8QPGA0JnnRK8~0VFLN(7dFVW3rAqy@D~AJ- zqs8R1Z?cDQGaCeAXdowrs5P8@;K|}mTX?v4QVf#cUCyqM<%Tlhf<97-Ta3E<*swlV~aZR#AsQ04QkBSl6O^WKTi7JA>;`m+dU zL+^9;rmYFY1%uq9#RZsSLPF8@+`BDnV3urSh;Vc67Do;|QNciV_(+1q+f11?sO3B` z?vxK@ZgFzYJLf(;{g?KB>VoIrZOH>fftmr!&YHqoJRj&WRD3aMRSKLUkf>DEY%X4S zJ2&@D1bod1*xH%3zH6PBWnw4br}2Oeb}(#XeuG^J&7@JT&=_bQwb``A*GtTwz+Hrx zc{0~Jb>0^Wq=CuU6AE4};`Z}j@!g*yW;q~L7bAq#KXc#ydsvsLU$)L83IIDha;KavG zi%PVC6CXDtI2$x4Y;b{Zuf9@@At{xFKoc=7u?|uxC4SavH^FK z;EsaF?*NbADURwH8@R+70qdyx1{#WeUjudgDN?+tCdj-s=Pm=Ac1hg;Bgv4J% znpi)_G4Vn_#1_qR6Q?S4LqX%fBGrG+aTWvin;lzmFES+$yR`cxOka(L)4EDIf}n8y zDKCp>ru{G8@76+Bqf*(rt2T3+qwcObPvPeP;0~B#7ZUDC6hFhCwG;AQL8}r+!9^&E zC5W9>ubW3@znp~k`UP>Jd9=g+x`vHpLDSpEgp!xAviD;QA~H>Z-wlU;#U$ac73AfB zus>JXV^_2$+TL~mLaXVYvFaj!=_ep$QrZ;W$+gJSC z{|obg>&zd~bffsYJ}U$K*aLpoN0aRII|=S17~}W^YW1fR?e8eroe=*k z!O3#;Og2z2Q1U{$UPWB25IF)ibh#hKz{CHMb8 C3|l(@ diff --git a/openblade/catalog/__pycache__/repository.cpython-310.pyc b/openblade/catalog/__pycache__/repository.cpython-310.pyc deleted file mode 100644 index 3cb3ff13e18c168511b45e602f0eed423d6af7b1..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 54496 zcmc(I37i~9b@xn9&$+X^qt$A4TVq+WwYCml@`Y^c_JO^YCE4IL_IR{CE6r+WXVfz* zd6$(rWWYEW%n=}9!zy4v93g}wA%Vm|2shzAIzWs$f`KG}4L0jT-~a!r&zYW;Oup~S z?yp~0)l^qky?XWPy;rZQtDT*h2>w>T^45{}W}=awFyjBy5!o1tMC%VmBM~Dq7%`%T zHX5yHgIbi~*l27pCU<>M$6X(dR}zB>`7J)0tRx4M5>AY!D(S(rgp;G0O2=S_gj1v0 zO6Oo_rE9QD;_1=uN^US$nKL*?;+fH&%G|-ZmEOVL%DlmOmA=8g%KX9kl?8(fDhme} zRu&B|sw^H{Tv;->q_T8yY2}o`Q!2{_msRqE`O2w-r&g8^F0Y(6cv@w};EKx1!IhQM z2T!k@F?dF0)!?d1|6sqoqhs{U%2|VFNjN*Yy0T_)jf6W#*H+dIu9I-r=-HKX2G6OS zJ9uv8yutG-=MSD=SwFbGa>3vQl?w+itXwpBQRU*niz}B5UQ&6^;BzV)1~*hL9lW%1 z+2CdJd^et7GI+U!a|kaT+$iBW2%j>zNy0q{FB{w(jg;1mY#G@KxNT(XK5cNjafUJ9 zSa7F4c!jabSYj+i=t`sCSZ3rAx~h~gPBoU_6TKp0oMx=JGh(bbq8*72UXA!l<8;JN zKcWpj*NE+k^q)Doa@Y0OY#JTfSE@{`*;5?aUo!GT#YSH~6^s&4yVv_Li(UTU3>HmiCO7 zM~$81$i2NhTDoei-YAX@m8=Y7J4-`V)39P!SNB-m*Od2|MRUT<(S2=ktUO$*H*PGM z^19A}Vtq>y4c}^(50rzPH zwXbNFtQkL|UxJAvd)wes%j{?eG$C9!d%+}L)oG}f?E zjPI?#hI8WZw>-NN+lVy$#t?V9RdtK-J%=y;`cg;^c10n2k(*MVwd zdv$!w*k+nl(@As?u?xM&4(jj~+uvp7JXF|NnBRhP`|b0ZFli_EI4Ia!8ZM8ONg~!9 zU(@Yu3w%ktN=0*M-_Fv3abVKw@uSAh-5W%ZohoSCQ8FuKML7~D+Wfy0d3HSK^%R>w zUbi}fU6fS_RoVNaX0K4YIZxgC)NMX)R=iR*N~2byx@QFBYbDH5W855@nPZfP8aLM) z=IXrq<(46^NB5867_Kd3oTnhrh)jW59?{H>smKTzs1ds-x(jL3+R+Gxr9NO~H zUbn`qRvT-stChw!j~0#6y7Tv3wEm*=hl^{@KKr7J*PMUOc^9s^xKzAw&H9Ut_2&$q zU0Q#^#TTxt51GZr(7syHSa+zgrdq47tD=BCQh?1O*7;aOZNi#cP?c^}3#@U06&pZx z>Ir~MG#gDt&C3zA;s=YP(;{+M|Yp0na)Ds`*->Rs0j zYz3b%N@{dy=0^N%ZYCh+6u1l_D_7t%swGn2-AJu>0q9XZs-ZbuXpnZ?+|JxFv}Fi^ zz#r=`bnPmCB_>=W7R`ex?HagSItVX;j4;NO6t>5Q27eTb_@S8BN_anU%;Wry+8LJWQLijI?wZ=MxyAghiagK2=!a0QB zYMgJZM|e*9g7k&yi_#aTFG1>kxb8Q$8QbrS5B4DZHsdPeYUG-W@Y{{&8rLA)i}18D zU|fgrJjt`exE|p?gpV3Kja>-ONBA#|8;lzfUV!jBjOQCSA-oXbcN#YvFF<$^!Vlnj zmvNg>#QPQ_{BC2&F!0+Fgx_Nf8+#F6YFyy{@4JuqJ7zSD@jDZPry%@Ze^BmLvQj<8I^6@cd~A{|$QY z)d;UZ_`~S2dl6oV@JG-~uS58Bgg*+bzX{|BV07BD@~q z&l;aMzJM?>^e@Jjj4va6A;OT>j+aTx*oiyK;24{W_%4^i*(be1<^PTl$HWp)`fl7G!nG9FDad~V?mxzr$8{>=H{$*< zE>iId#Gi-zPjHc9&p`b7xc?N_&v21a`%$NxaQ`_jQY5*>AnyNxi&VH4@tbk~1ujzD zIf%ai_eXI35*MlH0+ewJ?!UrC%6Ja)+=@H%{2CXj;!@-(;QpVuNC6v>=QiB`3)g?+ zV()K8o+9qQ!Np$RiadL8|1B=|@D+#;;r=^Z?A5CfH*o(ZF81Yf5ijBXC@%KhwTKVn z{unOy75UU&+<%XY{jd}9eYpPt7x8)n;$_?)$3?t7AMp{~pTPA;T*Si*P{;ka&){O4 zZ$+L_+@Hk7_T7ef1^1_LvF<~NkKz6_F4k=r@ha}m;NrW>h}UpGj*IWwkNAOmH6wyB z%dH@8B2F6Rb2Y^4h-(O&xax>E?v17!;NBM?J%&q1`axWWP}gz9;|NdSnnd1%h$j%9 z!gUz&Lx_`}@4)pU#BY~!P#)4#xJcVC#`O}EHGy~r;g{k%g1nQ6cOd*STrWrbg@|Vn zekHEE5uZZ56X8F@^(w><8yD<~tm=a7;nT_-J<&?ml`?0hF`=640W6*3U5iwQ=apnc zQF{uKTp`kkQu;+GWP=Si?yrw#|--<&x8`@8G5?bT1LfpdL~v>_-uGN?Hj>Y?F&zy~e5aS|=k!X{deX`Fv#u^s#bdLc|dfbWDm*ZaXN^ z$8phE0$DCv--EzphB6*5Z7O;c|A%AGi;VR_I*m2-VQnHGjU3jak;zm;pVHn6f$M&# z3Y0UT1=oA(9*udnf{E+CWBAMnZ< zPbpu_Lq;tdc?!6;4l=pAP0q3E*qTzM)|l|;?oW!OY;HwEtT^?Kx|QBNQB#7lmAR23 zy9muzrd%)Enzy+D+56+>t+<)D5m4$9q1uX-ZXasyl!XPUCUj6>whqCM8h3$cuj09S z4}cc!jrK%y(bS8z<6T-8WbMhbLi(&-IaUl3s!5&A?JR^NB)D!|(phZdRCI(_6qnYB z0qT1rN6>L2aa;*pNn9yhXf(X|04wlUZgkSu^w9;5{ z&uGc8diG5iP|u)b7xt9)6%UrHrqw+}b4CFaytmYl!cAqNaFV3Jj^fY|2C4m~A&jer zlEpLQHOlIg%lo^{GEiiW5ODa2j$@88R3R86V2_zKf&&C5L7kvMFivm~;FzYUV;>Wm zbgVf-1vOYy3Q!uUFW!z^q9W@W)vOGz*8jxsCQl0?2~w25EYagzsDQ&(sNl^Alrap^ zpOM!?dcZx3J840~9g5~i40qCljyq{0jyvfhfjem*sR6^=OEF-Z@S;a^*9Y`hJ zWN|0$bmHEHdl&BAxOd~8!##)l9Ng!;9tv)M&wzA+m6QgR4e1N$OVx36sH8~4iR`Pw zdXn~Ns>4r9-AzT?3js|s^g@Cug2M!N5KwV8UqtX?f|n4yl;8-# zodhor#}Fao!E$AhNAavos2pa8QwlNwixTs^oTG#w#?= zf=%oz4ehUwR|@F=bJt&BUeAmjLKfBGVIWcPZ)FraA=B{nz$=hBv>O&kHwYb>Vu*FR z;_Z0CH=~OtxhcQw50z)>JXX zj_tLhU8UP=Rg*J``ng+3ON?o-&%>lW-+61nnV@8jgn170RT=g#JSyx5cBd{0I2jGA zi{w!E;#n0RE+W~mx>0hY4DMAQms!r5Gd@;4h~GGC79^{cI#e{rz#>ua5-sF~3Z}#2 z-jYY;K!C&MI2kTc3{9BQy+EYmTd!qEm>iL7zK-Da1aBbtbAmS#yorEQjrkV@Zy|Uq z!F>ex6TFRp!^E5>I7;xB1n(g5xZgV&4P$|7>^aCBsqS&c-iPo+{RIF)Hkb`zg9)&~ zgv|!CV1rq(!7SKd7HlvpY*71ey+@7SjBj2}{jBZC4XKd12$g9Wxh{je4Zni6T?Sc8 z$@CnnW#z8daz+#~GDfOBB9gU*@{22|)lQX(yrZ^K(zCUGxhtV9CPEvm>k*Rze80^} zvsmpa4}ofg*SUQPYBE8M=7eL+w6Yj0PNc3#5tPVPCyHBY66J)6Wgw^RFbgBTcf#C( z7g{NY_gYyjnl%dbQfUk%=Jl~7dA%FCebn_HMrFD@#?X5S-be6$0v2t4fZ(qPJc0?M zm3h+20%mT_xJcUl&dZTQL8oMf?KjuQXa!2qN^$;MPWlm$P=Nu zYP)jGr6{Nc9aw@;=vaY(A06H(L8@>;AS0pF(1fCP5xA+bSsgWZfwbUa!Gn;ShZVlF}UT?s%yJDZxq)kAuTYUaZCy zibE8qtrUYsr95VJ7Heg+9&7hh=sk>Cox;vJkC31DcPcD@hG_j~g3l8C3&H0IK2Puk z0%7W3WXKz`Ut%;3^X zh2s?UE=jMCR>gFe7M)4t(Yukwd=>L$sdqs|Qc_HUay~}Xz5M+Me}^|p9WE4LX4Gq> zqhJ7lJ&;deWQknSV_JOR-w612^9KY!B=`}5`#SUA8GVSreT5=}A2W>-f}(|=Fw|De z$cYwMV1pGMtS23ioPs(@m}$nZlh8CnoutJyvpi%}3iiBL3aP(45u`*zfDm#|>@2R-fC5-4!_!t2Mxqli$DO}6=654*$OHj1-n(k=iE-ZGG@v?VDUkx`F4c?SG{*T0>c*!}D z`hr7I<UyeISW?A zRzFmRM@S0UMD2!~i@bk?mG~9Gr38-^9Kv>4oreah2RD3Eil{BLFBoC+P-i76$)KT=M zgkwkPqQq3)PF2z@1T#URqrv^j`%nBao2GXc7HUFf z$Tp=}8Ml06rmx9c)Akh#9kE3CqbpF2238D@0J#lO6839YZ_|KSeL6mc@N|NnbZQkb zX|m@8byob)K4@xc#=8lr=>OYGLl{0*PhBXffaOZ`d;yDO@U>Iee3BXUdZXg^%#xwh z%cT~#B{rriz|^DUF1=XB3NsytJ=C_b=F_Y^njHa+xD7F%VXBY}n_(U&pg?}iKioid zE%x#yVQ+`MO~l>_Ub5lnK-5|*ii53|%s}b;bA-YjNsvV|z zCo+B@|U0V^fr6E&ByA-J$~)oViB`ORn zeSEN`C+p)tg{&R-Mi)dUSG6+8@W%q3NwI`n_PJygI}w1NC)hsMho@pjWISSaki8qx zt&v;04(nhb+QEpqxPcX4hNtv}GASJB1Y)#lhk zQZA~!xIL3-Q%(SlF_gzksOM}1Tr^@5w$TVW5#EZFAjdT@O&^O5Y~PLfip^InGyz{; zEM}Q(y??Arn_C*IGPgALPBxb2F4fqq*ha%RVVmd8Q}Jw|rM6(AT1rW&sijf44Wm6E z5coaaLoyP=u*rd8-JWGMFhf6)z)hGeqd#HgIGFuwgHjDe41Yit!HJ>>-J4RZROxn% z6BrJyxY+EQku=w4_x_S!MFer)QV1dy*p#rcbv%VUKhtxg=j49EsezQ?@|765$`Nvs zI6_cS-Dx5@osww++@*=PusbNAQ+=wNlu5W0W^YBPx8=JgTsxY1tvMHIZ^ZQ|i%Do% zG+lc-7Y`F{n)_pBjxbbT(`R9(=g-n;@R=wS%E(A+8mc5vkpWZf@H5q{VybMoxsa`3 zZB6RJCN}HXOeMvTH`QS04)PIT-%fM8tsY^Eej=#94cS~Yz;F8L1@SP$MC*e}OSywp zq5?fc9^m|Oq_{K6ZbaOX;;_~}juP;P=sj9+oQx%fYedbZl-dfqjG?$YZW6(9qj|OH zPgC$w+^ubU zB7bxhs^yMs2z^4>?6>$dMm7jL3BrcN;7JQQ?(&_=3!`RMxtIVy4(uX=VF!x+0K4IC z0&Lnw_LO2Y2o^g6Zxc{&OhDu7AIDjkIs zl%I}58VXQHAv2W(?7*`fh3r%skP?hO-DM9leY$%}%9-j=rmeKE$uxKnes8e(j3eZ`Dy4BIJ7mF@TeMN}b= zix6{Y+%b0LYe6f&M!x` z=`3RydU}I{qYFbq2QG8qWOEcJgUhFRKH51GT?yAJ4_%@O1~F`&hkS~0lM7%|4wf5< zK&nOiOKLK2$Ne0^e?6Z}D9l(Le9lIb8PW@u2r2%7SN|oR7iljW?TfNs7LcZ&T962H z*%M~z8w&MD&SiDuqh%U363QKkn?<>4Xl6-c9L{GqH}}@+JhdY6yrl?neVbVz1ZB_UzWr^t1Vl|D zkgNvcXS6&0v?66q}OT(Kt1;6L-ssfRTdUD?c8$Z zRX1+iTDWRp*X~UNTecN&+r4d<$hZua2A2+tYGy%3r4y$}{u4hQ*tKIVi; zrkn+X>{Jt@BI7Q_0qHlORJ3Op8W5CYE#BlB2Nu}GXff27eF_X!WfuiFk1Q-XjzP;D zYEQ{F1$&xX-G-coY|axX!b-~QS2hA72Y8ID)|w00ftbj!3-<)vzRacvwKYeKGFV91 z1U|1xIHDoAY^*}d%*A+SGQ{C4^oh3huWv=I+D*U8ATymrU#D=TVF1>^{~{wD(^;7$ z(bv=$Je98d1UL}&M|8baY%*uDwzHOl9C;=TIm(Qo6f}E&I@{@UWzB&-hoVi05H4gB zcr<__N-MlXLU}#{uI0k9X3l9~GMS1(*3j;UG>HvB_QoPiHYu@X#2gdwfj*#&F~{l| zn~qb-l2t=&8S1o2GEZX5&t&pA#3Mh9S=i`@nnFA+C!U5G68gbeE6M^0bE)tlp&#Kq zS$~_F`D>4D;}oJ+7`vDVqkux)CWjqZTx`TXtPqs7KCC<)f5gghgb(>;yntN38NcFC z1z36-QB1bL*AdFum|&r{>M*LSH3i7lvig>woxl#IDS08RDY$q@ojn?WrC!J;kpkFm;2p$vf!LZu)`r$9_)|FLI37N-JWkvXw%Z9<~zOH(hnjwyi!~$?Aq5 zFAode_-vE?u{YQ(<@b6Mv696j8gQZT0-S` z5}+CA4PvEh8H8LbgdrQNh6PAcl%f!#3xX`7deeb^b#;$D>@q$=5`-*wDx@5ISPM1I z4l*R9Wb?F#f;n#wp{Ja^u16XWWn&)^#K}NW5{+3Ij@5b`&7Xp#aBs(b(?%ONYM;KJ4l+PQ357nKW|jbgrP1!W+Z zuDL)5*%nL{JsCCorm#>4`Nv(La@T7>{cYE4q`ySe5h-UoJK!0%M>u3VRoxMKBvY8s zBeFZ&UvIFtXlJ5EE%xrkt!NNAhq(hII?#GD52Yz{5^=gDYMzQ3JJq>^6_**Zi6<@Q zx`r)r(}m2xs5FTRn$N|PlWSTlHp_FOLn*5xXBgK~j&Ou}K3VD(>X~mrfaFA5IrS7k z*CD-u-h#G|wRs6?ZYbu4Fi!1X;|xKkPs1V*fpbF~+CD@T3Mxm%kkm+zWN>xtpy8{$&7pV>s=AO_7EqE_n0Z9A{peibGkaU&6qDn13aGE4cUW9L?FV|sGTpb2=|bDQAYhqO{S43W<_Geo#*!K}c#)bD3ySY#$T88WkZ8E2L$ zW9@c4rL5|!^9)~Qi=shh8IU?(;HxAZEM>B5(S{nDGN?^yBeAVe5apyM87P!e>#LwW zVWr}&5&Q52=mbW3VMMH_nE~?^Y$gTIfQ^X_qgo+0CU*B;VC$A{$E-GjSAk3GBQY7J zoS=GX6)c5`%VFe zg~?{FXJG4Ela^s&~FJ&|{pNA4B7YmJ-&gl*l*#2cJsTk25WmFgw$BdP7ifO{GrJUS?_AnP;(`?cLph1l&`XR$7{h34} z@kc#P^|Y%557B5bChP=M(qGUD@3p#b0+KstN|FPZIS z()7fDcJsb@T5AVe%DCmIVs*igJ=cB+NHR}hceOKUg+3PGJ#03wEu2fKtvS5M9oFLc zMneaz=CGuXXdKE%qUL<+IzU5vqejcXB@e0dMI5*<;lS+*AGkL{7d-&3Vh;fG&xn|i zA!?;J>@AI9XNq~*CrL46p4y9b4AtJ_nd6h&S_ea`0i#eJE?^6?edL{QpgJYP07GJj zf8yALTEA;Dz}W7bF3mSndj3Q&{W$C1gSwO2xt&wjQZVg`5$4=y--a3Sl)Sn5nA286 zFZ+WXZ>}VM5u(~EtCkB9R+d;1Alg!F2OkWwO?$yno%Dg@Rk3pLSrY0Q$lBI4?awV| zP|VD4uKtM>3v>te;hqLj?3t3`0t;X%k{ z6rEzJh?ZEXo{B?pjA8FlypV3t_zoKx!r2A{P^gvhLbp7Tb6q*+*w4lx{lo_*GXtA; z<<)TkSc)kUHSBAIdBZ~XUM8IbV0EcOL!I+{B6q}-#k1%(Kea?w&e7cB*?2%;O znMgk4kgK#RPsW5&`lP3B<6{YU+hC6wmmw#3Kmiiuh=b;SoHJg-{sDhiewm|c3Cifg zGIvTWBecq?LndiC5`Xy9TD!TWq!y0p44g+fQLvMz9rK$b40RgH(KRw1b4|Vx>Xgv& zsbA$lVpy0GP9ioU8F8Lb*XAjLhiMXa6WQXRBKpPpDE9|w?xWP>sJK(S=Ek_N0oU^6 z%v=hED3WZ^WEN+Md%Mt5;LiR|_^ak4S zCvprn$Ke=1VAGmVKi6RczW4uv6&AJ;;xHSM-{ec$^Nu&-*Z&RP0e;d6&e7@wrnrvH zIl9R?r15#_F7q!)pfXXsg=C#mtYal{_g4ONIYS<)3%z*!BF&?Z)xi!fw=nPLbHC$r zo~7RA4bmsK@u#;j)7t@X8XEWKHTP7XZzTIV>sPZehLpn?LQKlx#kJ)Q=3laYEhAl; z-!jsrRqy2kP2|?+8?U}bJV}asC-R5czQZ3epO5S_3wVZC;2`XP`D?x7akUst3TzVhPmuX>G-V(;fMg9 zVkZ`0v&5e^061y|8^8@!Jd#c+Z%xQ@XLFS&2GQ0vX3_zLJQ1!Zv&`+j!UN+ofROE| zQve~(s!JbNVO)=YGJo;yZ`fO&u;xO|0p=x{m1&27pAa>!R+Pv9@dm_SAyZ)egt$;R z_#i{gh~Pk)mt&%rORc5)QdU~#s}SEtcfL}2XSy*-No~W% z9Y(clDsLmvvf|GCBvo(fdU1Vs>vk%uCNonTKW{gu$$|qkACPBXjbHy)<|N9!oibj; zH9087ra2c0V%01}guqfNw{A5LtwK`hJR}t0%|jU}Ms51Oh|laHkcp^e4idcI#@~H& zkg7&gZ(xDNl}jbV3JyhX(hL_%MJQc9t57P%JllaOjcpm2(vH6i;U78QP?Pta=Z?#e z$~9-6nJ>e#Wu^%qpC~!{=()g%>w{u%9q_#M?qb*DJe8xeYn1GhRtT9@TC5Dzz`%y0 zkMq&0#;TUnW4qlm`l$tdecz{Jqy#3s?9 z^b}aBPie)e{{fQ|N1y?Lx%j6jio5nn9J0L_8!&k+FD!MQH^g$4g49H9A%v5NZ?_S& zXwj)P=+ZH4jL>jU4SXXS;@~vW39UK?iC7!y!o3UkZrr^jdtyVN`KAPsoC#>KwuKU zZf74GdypZQBdRfE`!2{7FB~z947z=+kd&~EgnTAS%Rn+qK}lE2Iq4$!-wWlPLJMdm zWPWWbH9#qNgEyUqi{pYQmob{331NoaKK{mvzW~K586~7TO?gABgXg1l@?Jm3)Ne22e@IA@%hBu(EOy+Fz~&XvkU)`3BqN9>~0@h_aJ_3>lI9q!8u0FXD}t-5g%i? z)e66(Ta@B=vVI{jFP84NKZFYy{I_1qk7JjU4wyjPwXc?6G5WCDzLCz}rv;+$Kim<31mE zaI^w0aKL?t0s8ph!dlgiEfhYu$mN5JT|NkPl6=to7oeeksYNk*D^`V}uY@u{>8&sZ zD8wbSILZEdA;A>EVS+mdJkIzcMqf_*&8;ucFpNTph{ zhgJ$jIOp#)9~t7Ux&;^@&!3>{4czYN&G;-C;A_}%?HJ&j*&tzn&uNuRTvo!xc6%#h zxX@?&g3sftgzE{r5RLPdOq+%ONzO3-=b20WuYVmX>~6B-#_i?tFt6~?s)u&RmiA0g znnUzCyjuK@!X1asGu{?RLe0#{K@Z0EowV!6cbfKb-sB2qYLQIRkapEhWee)kessEdE0XTSz z{40_#C154_lG8s=IQ{c{oPJ*IRy#INI6d~$*%B}236aAiO7?X){X&c1E!^%psf03n z>A5gwFC$4fzKl8H_;&+$=ARLW*OYAFxrfms1RDunP0+;Hg=d5@cE6d~*Lla-`>o=w z$Icnzim#8Yhfs!(t@{d>x(19jRC{T#*+N!&BLacZ;TR}oU5XZ4EJQ}s(IZ%ngwAI# zM{>s-_qstnX{1C=;ohDI9UQQ7#dB=*f$j40-H`G;*+@FzI~*Ef@fTcq=)5|BPG=vh zj48<^>_hYh5k*~j)BZ$LkLgwK){32G$ZmqpWv@A~39oz|cZ*GB@Jh zDwlKBPE^Y_o}htg>3=4dKR`USo0h(c+MRe>f;Ez#JQeTI*<|zie3eW_;|zJOsYe){ zWp0w+XP=w=b@(D{{S^?#tTU4TiO`ELlttG)0JD-r)9S7$8K{{*z4*b}4;QN}nB@FR}S{Y=OiU*s+BYTWH4? zNo*170g=407D)KBx9$c6r-i{cc zZ;Cjr_AWbi8h%@WoGWCiSm{m`r@K?d8GfZ;l}r^o)d+yQcj#0hV@~ROS$wOj~aK@aF_?BzP0Sn+g7cfS$?E?mGf9;k=WNJx=fhfv|)V%{o5% z{0`_@oY_5tjo`dP#X|ji^N~_EI2)-&P?%dn*{N&K_PrK7{eD*WZ3J&8m?n4! zLC6m?9X!g>UI39??p(%sERBm}YYX}DUm*~f%TiM*agMR)vMG3d<6zztcT`};O5vQ7 znE3329udQyZHcYK=_e_e_vly?HCN*lloU2VB_wtpPD4pS0me}S`0WyI;)fcX8sXs( zog+G6m%#23eP0aT-`Co2%8lVo$eWY;&5=3M_3r5LO^-r!RP>a1A*AL_iXjLRoZti7 z*Im2gnr*w`uWf?6AkMwt*KqTYOD4`f{)bq)6x^=3P@cHdH^ch+Q#ON)*Jom&Cf2*F zL!3(ZkyadXl7c^`|H9eJ>i_9XJfUUaC_A~XRdt&6O#!}&x4Qk`W))p9>Wb;6MbYC@ zqCfz(1IsU!`AF0!&(1#H@A?IJJOwjJ)u!bX%9K&-G z?p)ZI2W}@OXmRI)F-Ev$){fbRXgg-l1$L}ka<(-}+qq6;syVR%+7$neMds;faI?Xg zu1Cr|{Rf7G5;rpR*90Cn{Sc#nL-1jOj}Uy6;9~@TOYnCDB9?@4#E;`wM=kpayIlmw zu=(CO$;bEggfr1;KA~JKgnR5zF++!0+BWA9vxdDt%vUf@6T-I{X4@qmZd4SsNt0}& z;?LQnu-3={O1?L4A|20WRP^Pwl4|e;w*XOHaEbVrh`7$bj=Eqw|4!3yG4=rV?uySM z_v%80;AdW7uvWf`(fX-ZN3a80sSk0iZs^KKvBA@a^bs5*1<#@mJ4Vu`ucG>mqz$hl zJ4VuuW2HsMN(;Rhb!eJ#uvi@{E%Cw}I~T2xajduK&^qH-Z_%N3#Xz_ja+Lpt=6?{|ZOQrnF%jDiP157PU`2XoCuf_1`ug<>lJjcxobnv* z!Q4$7!?P#o)}lk%9T;=8z7g;9FI*@ENiEMF#z`h}HmuJ!cR1#$NK>(AX{qDTXsZ(K z;ShKbNNc-P;ltIeRyrAmP3q8P?2ViF{-#w5d$mh`rzXTBsSnlF%+tJ9;jch}y~RPd z2s(dH`;`j+Ggi|6vPo!vY+-*;ApM_BKlbFqNeG+Xg6p7&hp!HBx`3y#ZN0&&sBOK0 zT}VdQPl1{KIp!Hepcn-Nm9lg^hiM{%%UmJ*BR;{jml1rD;3WkANbpYtj}wSM{2)RO z`~Nk2s4e?&HRwu`}TI0eNh5t(yKO^ZoWmWjB}D(l-=hWvnzW^4Z5}$W9U- zfQ?1JB=21K!^q(QpuP>I|D%yZM713`Q|`#w&?=~jp}4t?;lsy1M@~BQ8NSfl>mnU0 zb_MCdSq73kBPNAkvO`6J^Tv@n_qnAy`JgV+P{}C+^w3C+9H4a8ToVxI=OTAs=(# z${IS(yWokyF{t%w)Wk+QYL`epwGP)j5(B;m!F4!1<$4^5uK|4y!q?t3e3~#;z%`Ps z9b`>v2OEpv3lX1Be}t?ryjW_t#2($TX{h)d-GY(GG!(0&zz2+hiOK7+T@Zfvd29_+ zb4b3J-+}#<*Q&3V<-r-(=5OSOkGV4|k1thL$M^2b@0c)(_wk6 zdPJjt4>Nlb>M=&&OYlB|jRfx}*g^0C0_vp7r$QuDk2`%3KRDcp``_Agr@6Lamzz(W zZ?oM&n%;^cdb9*oEpAfKp3#ou#HT-|(%wl=JYd9;QI%@XDPR3B5CI?ykBZ5VC6b*FBM7HJHU%|`onaHp5e*|zf z_O9TY#ATy@ZGLB|F>a34^Zbr^zC7%mowIhpF;dG6K09TQAt-`kCmypWd5?Vd@%JVb zQ_kr2_uxC(gMG-Eh^hYY`j7s{VVj+~{_igW%4X?+c_zxR-RN3+rFCa6TZTT0)|HO{ z*HGz5GNPGSv3HK{Vw~8PUlQhW_ZNH#1HUEInTc+Wl)2W~Na`FbO7~$B$01?$mMe5wlb?`RE z$Ck$GE*-7T+w0k&snvOVi{)zaHFx2mv{ue^oYYn>pVXGGY(4NViEg0lp=>)6*TW1> zN5kNN`ysl6sP#J_#(YeSG}m^v{v4@_=7%Mb$xB-2YjyZ$fQZ6p>-owrN0_5*BbO;# zNc`6*9bbg8v0`o`@0wtp?jn#G`aKK@bG(?pddyMe=F6FOB9ZwqW@!C5t%ym+hZ*aCa=^EHoo?Ka1Sc@;Tj<_{I?_O?-DEx(Cip8|6dY|OKX z{f@|PVp#Z^z3dzFGn_9F!=#BYo@Fmths-4*C7Cb4D+59mn388=SVFu;aX4C%j+)ch z3Q0^I0Hff6#SkZ=ldCt6kM4&gL7BqCjT}dsuMQJ6b^F(nu@+wNzhN7125{UFSOa`s z3nwv7G`D@>9+X`OnF8w;F+jyEh0}1cKjf^|G1m6k7BWIv9vM+jp4PZKoL;HmU^tw_ zguI5sWO%HoI-?m|1aZc_N`nzlxL^#YI7dck9|1W{cc}g4K+xtbJeoB?L*QZRTL}1o z$^^L;_+H5nUZMrj6rc zWqj5!&#$hcvG#5TX{QJfLL?rS5Isu>!e$lnyKsqpL`4&;5E+^cbG~rZL2P7@vsa^E zxZq=XYcEr7_CGcR;&=ZjEDhKaWOKDw8A*hRp zbOHZqHma{Q3?Hvn>Ny=VG_u~KIP~W}ZN9hOFi>B)0dC{xo$@^l^o6MY(!-o?74LLp;nuSc#Di>dks}NHXE~df zLfG}6iy%1lWZL7L;hGpohb@Vcn$IB`C}VU98f@t`q=~*~ES;Y|4m6b0BTJEC!1*jp z!r-|cM$DEoK+N|+A8spY=omWZ_#}-)sHBlJQelz?WC^T?f%Ico6VtIK7Q>ntB@Rc% zh++4UUg&91H;(fZEs;p@#R5nqw;`7!k@O)KY=iL0*Wr`TfwaIKNDE!bWRWeIWM-30 zh%;yCh-9*RyYnsS(9dXp5m@wO74u7sHpwf@(!W^AE9>}sTY2Ra{u(B)e1<>#Gl7hx ze_`lz1fM7P0)Z#UNR?!uOKm(W-j|Wjk!I2y@=uZMyoH9)!Dvez#LFe5!mTJID9yN1 z&ExQq_yeSxF27XcSn!yptb6u4LWrA5a&TEF*Fr5txF)2{|XntV3)uBE}k5812P6S#TLrYPd zveI3NfL~BkmNn>w_%0wUU(ix`lmb&_ zsQg5{K$toi`N>SMM?Aqvc=`e}%T$l2HwjMC{USI?_dhHC?MYN&OxzZ$go!;{v5Ku8 zC+%!Nzn_d)<>%lk+@6EmA{O7#!Y3TASie9BuAPWQxBY_HJ8sJXpiHT*KQU~w#p4jva{OX8J{Cv;J zh)p?LlquHAI6E3`ejBNYgddch!Yx6RmDCETYYz&s6g0oWO39mM_13fHre#e6iPtq@ z!jmABSa5`=USc6kc#=a}+%|s#ZB`Ogn?qXMoYElyh7l%4ojqHvWq*i~-*x4tHRrCs zAiu9z-{%>1qy;klO3!;UuNZZtgKc?Aes3#J$wzD4TGO(__k4!!6oKlDR?`Rts+K1L zO{EOQ=Hec$Z%4o{Lxp+P(5uGtJ!|!^aeUDx&k_mB?OZGrmPV^%d$A5JLX)#1O>VdL zu(+I3slb!$;nd&_|Vt3iS8G0u63%4V?CnaXP-|phnxTCPAc_GtSPCe_<+_%`wEb8XWV={|{@RQ~TEv*rBto4t{*48?X z5X#1k^U(sWRhHPxb|{A!F;$Hg>y0&_JxK873dH$hrG{?Mlv)nZRtP0un?MKq>B zWPY3AI|Sb)kP-hqhMFV+{@TA#Ndl|+ds|6>E3?g#z&DxJ_RXG5a3S+aMWhaoGx`L< z_W?kMw=%pWL>y=mD;;lr!A3!_@iS7}Wq3X)3$(MNc=C4&ADQ&65w-W=_r2^9gB4|~ zL%;MQlTHo4s!0dSgtT@>O&H>)v4V}SPN;=jeBDLTm)Plu%YrVJ8`T#?czP9@`FtF^rTO2km<=sCiVQGGQL^hEc@m;F^PgEXj{XhckP8>@Pd<@ zu-IE|C#+sb<+D3l`w$sEWpygw6QMK_Fod*($eQM@woW8AEy?fR(_u51Iw!_orzBeN zXcyX;!Z#@>H0E&N&@89L_@A}3YAwLD0y;_xHV2Psm2tF`Fe&IK0qsO>n1EB9z4S~r zwF+A(kYdDO^i`WG@Qv=HXb@6U_WB5O;L9UmEnHHD_J8t>3;ER&w4<|tiQ#$CQl!M* z#ysl(w#|Sh;TVo-(dC#Hn1jVt6vp7fBBU?Iz11yL)s`-yin%4>fvfbIVxFH-1c{c+ zVxwFg%PaDQ4_{uqGh~kXF^TSB0%2l5Vd$LzIHSm7Sh&o^3^6CXK1Dq}mkGklM7j}U zLY`^z%pVvMru1(NZ6x>yf*%n4kl;rIcN4sd;NJ-zBDj~}2!Y4n+SSpwg7Z^6;V3?* zhH$ztdvTYomB_~?tF=DRl*qJXby&5rKQiuDd$@X|c6az09mC3zJz#POIoUJ#LP1u5 zs=?trgRHIptaUVr6Z9q*P+L~Q^F*ZJ#@^c`{S&4Rbx`=XF|mQXM$E|}hHqcK7CRk9 zi7k{b=>&N)b~?Hyy5}A=Z)xIMPdx5x3Vpxp;%IiRBs{%QbO~O&w!1j-R1&!GBV%LthL~U_`bM}=t z*z2Gt>b(=&_QHnaQu8ztd)wjozc2`Gd!N+yWbJ)jjjgpVw+35Czlb60-^!dA>q~3}O7j(4VD{;XNqgMwpP%8K}=~ zta%>eVKT&q6UZ5m$8=PcI;SwM_ai{V=A95IDEu9Uu0z&?BLU<0=?0AZMzVp=X^&`t zF@d_kxWI(Kq`;KG^oX`MCC{del#zCyRryuU4DxXGTHhOkTc!s0%pUGeneJ%7)6K{% zL_P3(CAP^U$ZpxRd($=7U7>#H_F|iF zxayj%R(AJw&)qgqxOUUPrYlg|rW>|iwOhR^MwTx=giizOwlTr#hFev^>wr2w=&f*B z7k9v>4zYS#(OqT#Y(=}eH1#Q6?00!eOHvR@!1LSxuRVcZa{gbmr}U@vU#6D#K@`$n ztUnX>?c&x~nk(?S77+;bP@|Fa8b5P7!rc)N71W0XTx zYyBvSd@cYx&oGZ+mkeE}LDI0j#k!!8QLkYizg4pA0{5OFK?EkrJ;FG2~AxbnksgRmU8pKyqYj%!45vD zWd0X^-EICi^MEz-Lntx_2=aczR0;i-p-lwmxoN*+=uv{l+_c{_bRB_IgCiM;e=?Z* zluGs9BY7oNQ_m$7mu`BF??P!!&z%~YI-e!jr7c0&IXF>7Bx%wetkfm~d3}c}#`|4T zm-~uY!Emqo{;8C3Hy@V9et;oKTg`V#2$BGnav5DkAfbLYbS6WanB^=tByXg-2aOzX zOO(`ekqP#N^Lf+F^?XELB0pZh)R=iGQ*mLkO{q;yQn$;QETN4IZDPUl*d|2zDGw<$ zVP4N4)UV9v6@RVdQ_tMSux`pJcwP>vovvEqrAoD0^2%6EsjaG#*TcA2IK1aOQKA0m zZVc>Ew?Y+$gyy6P_Q3akf0&K%im-ogxX+I>M0TK>)6G|;%!k~k2Hc!d(k3^jl%)G| z%KKl4?CdN)<>h@5Qxob`-litXQ$lJP^?6odS4}9OO~CinPkBa=p_?z~o4oef#3I|` zyi^u9tdSu-pZzT<`ama{obZ{AA16pg@l-F*tq;6s{@(#P)yF$xt zO0`R9O0`R98sIi2z0@PYH`rxKYD;Y%3g(rva;BH+m32RJt^sb;k!%uryPG!6(2w}* zI~elbW54|&rh3@4Q-8t-TKfFwO!e~G?ekK_4C6f|zek0`f?!0s7LMvi`maM@r zK$`emvOwQJNvM@Vd0}{knqFd8Nc5SO3dz#6S9xi&u6&VNQk%-P2VGMM^0tKOrOLXu zSC%~`$tq=2ZDQsr5W@$8f`Ghq6H)L@0txwvQWgQ43ewf`g!fXrCoLDxVTg+j3`hkq zwF>>qt32iqHhye|Yqv=-ELxOQ&`uS#$*al+G!Z@9lqz}eFn63_1eX(ddar~oBdI(efGSI!6p)HwIZV%{vpv~)xR&AS$!4>eY%bfG?aI#6 zx}8O|+!NUa$jSeXYzjG2xfDXW*5T&(eYQV`&EL7>T6gYvb_q&N;Y#Pa7|JCPie(eo zSawm4a1MUWqO=&^m2&UMZ^ydjp2*I}dlFrV+@c)bm`mi+x%s^r+!I>A^X9(CbF1*% z{9K}UK`x$~lS}uWp6l-I%gyO!{>8Z@evRiABD^w}>_uI2%W|3ADUzN>YD!~jcCADQVY)Gs>m+9>goMC96NBoI|ENp_kfZaFP4r)5B?|?k zI#eiFnL=TB9Q)`>g@TEmve!y)M#$>mHuE8P3)M??GsfR&CE>?=8~aAfdrXSs*lsmm zHe?5~m8l;XEshTDD^(`UB(tZOq038MTQw`DSSK?4WdST|V|zx6MrrL3-h{>EwH1E- zsBTgbFvUdB$xs(THv#o#Q>+A(cg?v3y#(_J`Uu1dK}l0S>8v)wFJ^QJf!GqnbRg!L zJkw4Eu;zP}F{%|DCbAaWT&iZ>Tps?bUDMO}>%x|2<+H|EXc42NHP`zcU%OdLhmN|r zlCM6U;0%IQ1S0u}1alUns|nT+tR+}Sa5lj?1m_Zbjf4C<1fM7P0YQymBY!=@&~*ft zGrFDNYJzJCH~_;&rj^=&){T#rI3Gpo7Xj#ztd_ziJhf^Kzd>x~HMHzP#y3VWaAN#V T#1pVqWfJ&Lr}SL3C-r{-9zn#4 diff --git a/openblade/cli/__pycache__/__init__.cpython-310.pyc b/openblade/cli/__pycache__/__init__.cpython-310.pyc deleted file mode 100644 index 8b8659a764790415cde41c7c51007a57db5ce6a4..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 158 zcmd1j<>g`kf{&|svV?*3V-N=!FakLaKwQiLBvKfn7*ZI688n%yc$|Ga6$%oQvlG)( z_53s$Z?VV6r{pKc$FF24Vgf1!6TdX|i}LeJ^!*D`^PF-LQ&RQwfvhAjD>)}qKR!M) dFS8^*Uaz3?7Kcr4eoARhsvXG2VvrFW3;>d`B+&o> diff --git a/openblade/cli/__pycache__/main.cpython-310.pyc b/openblade/cli/__pycache__/main.cpython-310.pyc deleted file mode 100644 index 3df4d2922a8cc0d9472f4fd116a7604fc071576f..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 8010 zcma)B-E$k)b>A;67E6#KsSi?;W5bF+0!5^3=c{cLSGH`$aV`@3$_zPusUJH16Z+oiYhP|3`=pu9^ug0r_f3N&gy?yg(!}t%Xl>e%zTs90dtJ{VljCDhp!b;4JwQiY2 z+ljqyvis{CbONn^|{Wm^S z&qgQPPerH1qB#B#TySkkoG8_n#mQ3bv^Z6&oe@i=+F7xzYoaU8il-hp>z@&+cvhSP z^|a`T=fv}%o)I62rZ^9(A*8qzHkczIany<=HRt$ght_qMvu!=lLa+RH4}EbX&&Zr+RfHb zcfJ|sxAAMg*4=K}YOW=tFpq*(+RdZ;xtiIE@^NKWM%kp=wAA9&RCdDrO4{9wiNK<2 z=6bjp<=Z#XJ5d+wE?!GpcixFNWGJ^^4_jyyXg$WQH?CeUng-8~Puj@CUepL>>vnuM zYS1}knk12=v)+!9-eB>1-03G_4)%)PY_RlRn)Ex-%9c#~y+#TVXbuPS`h<;psl2n9 zr1!GH!fVCSlcu@1(~U;l&3aLbSQso`3G*;Xw@Piw>3|P9LpSV1gZkale#(WY!M&;$ zr&dlo-N;n6rI%5zjPkzhK3NcPE5DiL@>6!z z&)PS3jNBd?ZAZ84TDWU-tyz&SMyYC|#j9#@_iogMV7Ap#Fs6e|nRbHF7{4Hb822Dg zVg84PJdK|x&tH^jnqM4W#6^g)yFtoxv6aLZJ7L^i?QN^Y(#pMXJ4r+F45nuein=*( z%CjKkG74pbRe6%C1n4ab#psXttl-PMz5-%s?it|I8X7~ZZMH4K^an)SZy3A4^rGf) z&3WWN0+qEEiEQ*o%ac5=llU%x*fNI3t;HR4XtuF@+mv&;MZddd*A&zOC-KPs8|woS zr?m#{uVzV_Lo%wmfvZeKq`b7%>i6PEG#&XFtSRA;3`v(GiEG$^wfd5BXdX*Z7iZ9= z{7Xrk<*l^Sd$oyrHiyDBEZ6zPqUo5d`qjBlC%{j*{dK%YaHU0+#vJ% zMG(yEsb9rw+X2UwwgaA>3D1u5goL4iX=Y8$=xPRth}29c?RWElD@|8^78{iW@EoeD zayLx+k$fI>(~=Et;w7fUA^e_e~9 z=l(a3kffF^d=LDQVzHjm$#Y}RgoiZG8p1jY`8VygfxkL7mBBH>34Q$4c9wP_6j~J6 zm3E?%O!$1PC-cE%O3(uue+}a^+7!k_daRrz-^}C;lWS==I%G+t_I$!iAP6I1M;W1rgG!1z^G=lCBH+Rguy5|i6Nv93eth|OAa78$%TRS#rdy^2Qvi6wZ ziF^h@$%NUaUkHqao2g1pK3h3&-j8$n6--eTLe)&w;EM%IkO5TH7$Zn_tPC)k-=WdC zEr#LA4u=Iv3zh`=b~h>a(VY+NHAx08 z3C>}wm+_;VsC7GKrsQu?A3KMMZH}#gE&E#-!HR#z;<|F;Zk#_RlO0r7Ru~K>1vTFj{ZClHGJ7{{sn|~t~)lo1Pnx& z53L<1hvUf|a!R|Fu$GKn>kZ>peWx-cji?A4U+1UJ&#Zlz*as(acZi^Q$IYul8^6_| zGpq*Q&_!7zy6vMzzcacOzq-9%)aylkwy4hrwR~<^-GzDP$1u~~#XkA`-hzNW{?rie z&_r1sS}47rRmWyrTZ0wXy8TX|5!5GAON<4m>}ywkPuVYhNerGQ8OLD~4i=8E&OipR1%th_thQawrtj7fJlx z$I2g(<@=So>6>Rvxq*su`(5h&R1Rf8C*o@;&MBzj)6OZsk6s*+q^8d~cAhe%wh02V zqeGm9ph2Hum-sX z9WX>29V5-nR#BN4BElm0e)#lJumnRqjX+P>CCNBz@Yu*vvl8d7=KQy@+EIFo@brD^ zBD9fjQgNgL=)*ojKL$lof)$eW70P->c1TGhZk%Ejwz@s| ziFOrVdiJ8U%52wkTJ^}@H^*MXIh;_DigA)bPMDJn7mH$nT~j9!%n33{N{L1ZfVnZn z9|!;cfD!E7PZ)z2b^aW1z8+}aj!JrvlUc}2h->8)>X#vYqWhDJ)Ox&luqZEh)D_5- zrqDvNhXP0CMJs%QspG2S<<*Mg{XJ%7brf^ZHV4RIeFB)z&@iY9xxFI$-2)dxYUbn1 z7xM-PQnS&;0QV_$96@4nIdm=f`uNTuey<}{a}OzB(^a+c1BY_+P~wS3Ergq!2aFW4 z`@K+h&|)OX5oXSG1TjZ-ROCIB->3NgnB*{9+z75AthNR`_t zpyiYuBbHCXF1Jj}b>!2FE&dUsGXlUx7OA642p69xmgoEr1U#e_$wAW#6FDV)cG~X% zT9dfPc0dv6MBMC&3Sq+%ELeioM$F!>QT|`j|Kbgx-)cn}@{|tB5J5~;M0pq|W4$9W z$sH;#Q}GrRU!j5{+#eA2M^yYVib5t@8b?9@NF)V^|BN}TW%Csh!@SnAKS8UiM%@qk zQ9l~Yt*n%vpU7iV_N0nODQ#4o8homRKKe3N{B;t7GFokaT1p?1l-Qb(p7Se#L-|W{ zM(fabu(13g6_=@)5)K>TpHasPD3phHvVdpXa+g{Oz?6;FY0Vi&e}xeK6$3ebnXe!Z z{4r_+hjc?(dHShb%LlZ_pQ0F@_Pf;Z8=hAF1x>X1jPfsO;*Y49Lh@=6mHr$0^Z_j# z@UKu)E^!-+11Lf8J$B)jQJGxI1ile86$qj6fy)`R5kgcBBSb}m9n1O8Y3E8LWh#}| zfg%e$N|rpAHhooL)lyY@?j`ZlQGWI1u{h*UXxv|;0H8i3`Y4(B#JF~D|Bf-)G762N zj(&$Bga0{Rg#H%>|F`H=R+1eOQxzQyrU$LNET)lnBj1)+Iii7m7f%~UJQosZuy-R@ z!rrZ-pU@8aw(p~N*w2IK+R_*~pjASO<~RhwNu3gR;>- zTULu_0iEAJbL0p-QwG{K>DS824Sw0O!(I>ILpcsn^9Q0UYVnI)pXixUp3jqj92rlV z87VV0*XTuo{D#yIN=tO|3si7Mc9j@K%+TqCU7=r++;@BQuN_XyIo;*Fk5eqp6gbM} zXpSQeNgkKo3PU%85}QrQ#@N~@dQsk`PAd9I^s0$kMzHG`b<4Brcv|zU6V^O_@!>jC z#giL7yV1wD{fd9ucYN3NF~jjKAGzt=FTJyX2_PYLc Ny>pm@zWI6c{{f~<9uoin diff --git a/openblade/domain/__pycache__/__init__.cpython-310.pyc b/openblade/domain/__pycache__/__init__.cpython-310.pyc deleted file mode 100644 index 566ede15211a661b2e731c73a0bc95b8bdd34a2c..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 248 zcmYk1v5LY#5QaB4h67*W*yak0bt9LU1VoO*S_?ZtmPuxVktDm9T{+@=_z=F5t#4uF zCY3+TKm5h~e4{3l58`yChtrqak3PK1bK%F0@tq;r6ZCduI=XzWRSjiBZKtK7&>%HX zNZZl&TuC$AOuhS92voTcH`0#W@nFwCZfni+3eG55z*(!5Wm_$q-wIIjFiWB&EPw?; zlCp6A9kWzI%%T*dc@fBHk;dHQE!h010r|gWN*k_RK{gcNfW?I4!Lm3Iky~0x?2wmA*yUx*S2T6d*|!N%ycU(6tW6D*M9%y*H>vpd4-ed zCkc}sc%wgnFvV0@f~j7@S54K|Of7+P&C~s)neS6GUrUnnfiQ_r>M33GvX2Fn01Gvd`* z_CRUnj_>b_FbBl}7cI<}1PdplUvCJHJVN7tXgsk2Rg>T1q z!=UHl0Ms0GYx+sSWCz~pPas?|6Od7rCAij8naZ>mx~VgrCs`8S1W)x+lnvcX6UUfv zGb{_b2I-z~b1VOAS79BR{`zrd z;FdwJ!i&oa*to=&fm0pn)ka^huyep&U^Unk=Vfse-Cf&x%mXGnTx`KaJUu0evk&kooBhs(cruBgX6Kw zq6QPOfCSxaX<`w_=a8HqbB7UFf~nNK&}O8v0@LH#j2$SIx!4wucuXdo6__wD&H+vh z0$d!85CmPHfgZ}wibsyl!_)onDPNwT%{q!w2a_>=aQv>Is>7hpi)RJ{MOi|phMvtP zccLQGm{m;Y8IdpJ+LLW8D3iuGMHB)G>EcXC#~L!-wS@!s&faN(UyfY9ItO^6=LU>6 zht)l@Mk`Zn$cQU5M$FSbtr_CuIrw#2adL5k8Wi(O6TdIdfu2!r)umxsfei)El3C00LpJbm zoVTne1KS%)1ojaE&yLif9E&1ODZT{O#HoSgBP1mxSAa+@iiP+Dr=Q|f_gr9nhI42~ z6wyL*vWaO#*FgmYc!Nb4YeOljjRgKx;cxhVB_UA9f%lCEjo5)V`U}W(4P+|UnZ}b) z^>mhmiju<0$WlBFTm!giW&k$>eTYGNI?J*gaI(b7^BgPiJV**XgB4i`B*l>=50V^x zV~i?bi6m9hFT>h3wgCDSLezK_YGR!=fKwZB7J#$JmVi^wEVB!21uTQFxwfmYWq38- z!2ht)Id=X<(p+TAPz^829Cj9?QYi3X%DgkxA&4rl&U$^)`MwSxP9=tJmsF2h5R%iz zu1=w^4uul#SjOKA^2QvsH4op?g5Oe2y^rVgo-I_)89jc&1xIcVuYKx@)|CA``+ z-bgA_!*VQj#-rHLVWIJZ<98H{m-?_^5bh@Ei7g~2d!Is_eJ0$PTRrF%@w^OM1NFEu zZN`bpX|cuj9Hs;&?uiZr{w52bAWZcquMB-5SOPhQFPi& z&MgEv9z@XITe02es}qdHX^bh24}z)+j-8zwJc9eTZ#!;$)R`b&oJM?GaZi>;MbZj| z^*-UfgHZFro29&Uk76FJp$;YXjSY07Kuuz6Nze}X0=2Ipp}>`KMA!V6I4U5?BUwjs z6A5-K0y|HULGl%nTS#(9ZX>}HA-+XI8Q+1COpQO z2^IhTo4jiFwW>{KC;?7G!}_4AI_YJBn^KFf1+KT(YRI%@vCx6amd1d4@GEWNfK0_l zeaPg{Ui?)o@W+%~ob>Ul!#;i$uQx9mRxc`eGVzPG-^>=Mj3>=}a$rlbhb{F1+xOC?cyUiwu@l*PmkB{3n(uPe_dfGeU3T$K~R z72spyIPh_ctH3A3B=DrHqW>81l$Zvdw)i;kNihRFWAO>#SuqDZXTP5WJ|*UX=PjNB zJ}nl27c8CzJ|h-^7wz{aftSQG@Urcn0X{2MfLAP@1wJRv1E05e4)}t&2z=4vQ^2d@ z67VI9=YcN^2Fxrz4SYph1-@$W0`N6)9r(JHe+KwHaRc~X6E}fx z+VRW4x5PT|y6ry;To(=ChQ%wu8{#(bZ9D!P@CV{U;16y8dEk%4$G{(3d;$25_yqV9 zi!X{_$g=oUuKHE+nYjBy_4y@nZ?{zY{MFUHVP6U+RK$arNv)Mmn6ET@(wFi5kPF%H z-qZwohYnb6_!+8FlBMTmXvBmlKd*=h%vm{{%fH>C`Cgs8&lBn9ZfyTpJ)CT{Vjjs> z>u|c&ij+u0dY@^vzDaqQ^_*z6gz{RgA4_@?Qtaa&^$o3**-)MOz|?)oO`UHj@az!J z4Y!63w}wW4s3-6tY3|STm@G|}ZOeaMj;3{i{)I>J0S^PQYq*hnn#W0C0u{Tp+}Jcm zMty@l<1q`|NS4r0X7aI(ZC$}+BkJyLirP+EGI$}Av#5~K;x6~48U7|vAsu*nkJ z&blOhz!P?0M@TVF{tU#@&&j8^Liy)*BoDvhS`4_B4}yeuLMhyST&FJ9%lcTX43n|# z$L26k#eUEo$7UAtNsSyELWOJ=?{n=bAs+`x#EsW;L>gIZE>@!x#|G7wiREPnGa(NG zPmZHB2NH`pB|?RimLBNfKsJ@u@{JSXx1dqBCzHxPuYbY9prg4SGC`lzWhf&V8|+m6 zE#A{mSjrvA4x|qJ07p7K&G5Iwg9N%a=*gIwo@7ve&5hD^A-xc%15>7XS%S}SFcI%F zfdlJmiCq{3rf0u$y@{PsOD;eP-}qWZw%bmTG}xwRg1F1DiO}^lKB7Xi_NgP1Fb+1* zun!=k?0m#(pcNYIH0vx;m^4V3mVTOmMGz;3WGbI^Q<)_Awqd^G953M=`U5Gp*A zg$J||PdqQ}WBZ(vtk^aNGCZVp4@p}aX&fpXG-r|PWd{F?avp>Vd7Rl+yB_S`>phkt;BMEi zY%7+y)(u3&cpvtOPR0*-5RywCM`9UbKg&rFDkL#W=l(Da!%RFuL;`EzsqY6~K#t9a z=V3{okN3G}A@G;EctVBZmzq84?I&sUb{tqeYR){T=c0g*Js2Ms$-Qjm6bKbkSZ;D{ zbRfENTbYN@@l5|O=k3;u`_x08CUgDc&fy&7ZsjZp6|z{$r9JWeFo>l)k4t$o>852p9L_`PqnrbwLJrH%V#-CD-0)C!pK{}(GXEA4aFgK-sG0TA4q$x)LK}q< z1T1V0OobypBnV^@u`)qCamPlJhQloOWwSG_25{gQNj}Et6yZc5`Y=p6kYQLy`U=gz z%i3)ZPTt6tkF#%Si5M%|Z5wK03_*@Z;m8KXcKd0nenZ8n11m8uw|+U+`Gb7E*pL_#c64KRcotXHHS2w)Jo z3kso~)VLU`If8A10Pd-%uVNSyh}yJ(qAkLHYLY-m7MKRxv2m4!Dit9GG#gzDBkBbl z>*M^82^Ajl%3~N2MG_o#+IRZobgj9EpwE#lAd5K4Z#vxDm(Y9i2(e-T)}hUY(?hm~ z>xP}GW^?BP4|x&nzRtN4D&)HK$jaNYYHx$0y^4IfiKP84CPxVvY$OV7GP2HWlw4=H z1s$ac|4X=D1+U-cED05|TyXr{2|DM&PvP2f{Oy2so|@i{RhiEo7bh-5+I}vZP@(MkU#rgBqUX>uk~~fvg8-F_BUC8v@tff-GEV|24!?(=|Do{FmyE=eKn3BT@sRIY& z*W}h8!YLvQy;^$|XIkvMJtZ$_XKRTY4#%Y8a9A71<|^b5ayEqCfz6KUD6<#9pi}b( z8%@p3xb}Bqa}Dyp&Djuo2R8O@@$Y+MFBR2-M*rBG9$w-?nvpdk9k3KhZ9zu?1WiX0uj=!m0{ zFOd-A^qy^RKHK|hYkTieb8~O&0ZUTtBSTKu+6&c58gLj&*D^Qm3Uql%?Wjgao_{#H z{*$5@coH~g-1aFVz>)NMLM}(EwA*u@WHTPROhbl9W&ZZN-?OHQB8;`>z}9d{Jz%$( z<^vY(hsIAfBJ&a<8X20dt+OY)cHFx4lsn}2BbROB#-G&$j8Y+`C-D7t{s`$Aiz&ju zv^X*vp!p4B>+4Yb2{)PaTy^D3fxlLF6H(2wSm4aiR$?j2MBu z2c-wh>EkA1XTnL!zAgQr%mKM|%9QS54E(P2Djy73Y%NlZRwkwsSZR<2X*3Mu!_av4J#@EYXPyiBc%ZiD|g&6=!8pS#p`# zrEM`z8`Mo(v`x_~z0#s~(*x>>wrGL=EAzFV`q|(7rTx8GQlh10-H@2!?0d6wy!U&@ z4ESd=%N3k;$bS0P=ahSkzfg?c!`EmRg{EDPf0IXP0HU?HHF`l zmu{$4wV_oth4^VtZ|GIMVN?zIta@g{s#>zGd6`DGnr-B&Ir*%6`NmLnNJ+ewU{_?e)dQedECWiWOBn$r%W|OPx|D;Uads4aj^0k*R;rKp z(Vt+)KtCqwNBihcvI6J@Nq?e`{uDb7`f*7=)<=JuodErWq(9k5e}(x2+17uoZmKQHOW`{sPR(;bVX` zIRNt;PQ5vSmAhUr-rC|>Or1x$M-3y9R-&(=NX}r9A4T`UB8Rv&Va<|ABT=hvqZMum z&GWGhCemx|08i^q-E&#FsPhpF#}869ib|-Zg{75CLYrTlo`ulT=U12UbYXgKzBIcl zt*G@ly>$%LVU&C1!g|}lCWt<86Z{EqX`u^#yBS6cHQUVwJzHtfn`U;9m=&dya#o(3*Fcm&*d=|l3W$j)py z&1)`?wv?_j&lOt3zfN#gUCw=8E^2#ao}QbXF9~g7@gl)fEmbOumDq)z!sI7V0s~bf zYe<@?gBMKw;}_hs(5k_7EFLEId4H{IpMWi+HS4_HBnuQ~&2O|k7uw;+F`^iW5M*<= z=3jTYvxbeDHK$o~Jqavx?C zrq4qt=FD_?rZhhYd}s0A^C$rsQV)D&M-U6~28)HX{BW^UO=dterqCgpm8}*$m&0ot zGLw$WqnDyp$w+R-^tll7>CbY=6m5bIssZ3X9UMD}LS4zt( zixt4&!eV7%dPQgu!Q6#Q!kjB#ES1568O|+`FB4ki(_$eqgMGb-5+ILwKq68m#2qXX zasc~fg8iq3DGhZ2&P$lwb(rlpy)9wM<_%sC-EvXgyEn5^nx3^6%k!6nC7W+n=2l9v ztFBjp_w;Urz_LtmAUc&XZR*wKaax6NppJYdBT_wneX{Ov(zO01m9qks9Oi)vlB4WZ+xpS&JW1-Ow!or$caBn)b(Az4UEjV5a zKCs<@HPi~V4M@8dx;)?xCPKWSAECBeEtiXoeIBA-oGroTg}L7LJQz}KLs%DS2PJZs zLi5))-CBsgcHkm;2)S)jwi3ABy0DsLdtSW*_3ngnPH{=(Y`f(LpBln{2$>|O<;1p9E-G>#+X~+VdGFLZmP*Sc{Rbp$P3a8n zPA7Tg?Id5LcL@f?Zu=^=_l1Wc+wM#5Y@2)Zo=?^8o@Gl0WE+eHlxVvNBWWe`8Ye1E z^}r{Erzrvf`N|l?rGT|G(5C`@8qlVrZJ?b40<*xM#!`SoUYN_s<)X-usy9QSQ*LM1 zS^0jxhm%_cf}zmYBBWkGBI5g=F#KAr-6HGU5yn0h&?;wWm1I<;0u&^iZRPD0&xS}l z38oU5$#Q4BH(-QJiKoJi4L96yd7(Rbp&k@kaNqFK#T37ZQTcVMu2I#bYFDmOq$F^{ zt#nVrz)$)=og+JYeh@*tg^v63D*g@)D`%1ONfqwSva3CgewS$~N~bJnFRNsge3BD= zp#JWW5y$Bd`btAzDth@$V3ddS(q|~ zPLYu*8gLFuc{{z=kIL=F?XQEE*HL7;Nc%G&&aA?TK5&103tX&tc)q^I9lkZ!i?IAE z7{c-*G8Wz$^6Ei&c}F)_gi2TpItL`ZZ^u`JCN~wx{7K~CsVH&l?udU81NXsTS3&QC zAu;kL>OoHfmmQo3cULV?g2IwGz@e+ zUL}CuOQOmqI-`9PkM}xkD`JSb>rUGX?R9u}q&m?YNz9;kIU-GmvQAFQz3T<82%~F) zIHT^(5P2F?@^4cm(eoX&M4Ddb-z8!knmz!7jH>JF4#QdwE1amYQko(!@)o!cQ;MrG z4R;DUEJl?c&qR`Jp`C$Bvd9WaVL92hWIK=c5L7iREXqhK$cj0OT?1)~e-&$x+TmZL zR@~OU0fvOwH__-E+#PjC4g6d5?p3NtYWI~1ZsJ#@TR6FM3S5y@TLI#uy|xv)fk=p< z+J;+uJ7_oTV8i+BnU}=Sb%`ze8i9+Pb}_Tpn)hgiX5k-E^+T$DOw~`QirdHc!FUfP z$fB|mhm*rF_pCCkaN@*@Xo}QCZuDVlLQv|n9b8!TtV_qWmC7twl)*Akq{*_8!dz=J zQ_ZqGo`#^Z;c5;SQz{#wHq>`eWcw4!*wuX+1%4fy;8KBy`o|rlLjEK47x{G{NFJ;> zgFf~xf=rVZagR=cA@W1A_3dO?7^~&w(#quIDdZ&-?_>2rxS}F0+(<+nWnlUNy&@50 zhEB`1C5$MCfe)2#zBAOjKq_>@d8U_U+fG=_KLj=()A({G381_5NW;sdoBxtppHLNt zqMw36Mk5b;gd#HZVI@yEBHlhBr9hk_9MN7Mh9g(o)M4NnkU(D8+_b8ho(G24wsvrp z#pS2UvgE0e1ohB{bO==G0;+j7jP?QP%ZH>dA7&$<9PB!XQIWz?GIqH$pTaYI54(zh zmBnxa@q{4})i?b$JH+8t9*L!(2hQJ(A$M>P*i7Bsi6LoD=LL~!`ZqWw!XniUYq5$V zaZA@gsE^*VYB(n1;s6*KI4}PVb*Fg6e@j&y254yMaA;^@wOfQn8nxW95G1;ozSQAaqln?EK1wW%3=AUQ6qYJTK*Ik^8+$E$M@$ zj*ytKI&OTWUiNIJr&OTyzN3A^#u$q*k^EuC|K)A@&q&Tc zr|K6}#p#<6DuH~5dPo;6{oPJ1=I;`NZZ<@kZi0m}&dG~+Z#kvQEf;gK5dAB?Dl;QF z=FUibl^B1ciiYBUrz&oBe*xnYl$}USILs@f3MWpiw5P~}PJ+9u{+;WH6nr2K7cQkj zDl%H6OrXGjcCXBaaCwgFaQq=raRa1^{07Fs(Xd^sxj`WF7F_kC=^*Wvsmvc_2L?;6 zdq@!X-c3iFm$}HF(3Ag@s<@5*0}L8#=LBp}R!G$1N&5F-y^V_z{CmJDl8PjEImKxl za(aK6xB*Xr5@n=hRgu$6ySUciI`!2lIMT60id+Uw4K5ZprQD+aSE-_mN8W?WBa=L5 zME(RIj!+=upU~-U6(-JIqD$D&aZjF!`1{oQfGWzDWU%ApIXKy@)STQ|oFI_W17!H4 z*FZ5*dURt{ F`7d*N8ukDH diff --git a/openblade/domain/__pycache__/policies.cpython-310.pyc b/openblade/domain/__pycache__/policies.cpython-310.pyc deleted file mode 100644 index 45dfd451feb63cb90a68ca0d88a735b383d1d300..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 3352 zcmai$-)|eo5y$Tr$s;Mzvg*XKod(yYZNMZV<;IHTz$qk|Yy!lg0wrkx0>O24yOhp8 z-qE|u$Rd=N+OK)+Q&7Nu%)fN6ee%Chq)5NBr(;N{PL9~Q-Pzsa&hBSsHd|e7Sa|*! zeEp!0+P|r>_;_gSps1%1+!|Wk=1yV{ZS!`94&H9!j=iC0Tf*ZWuY6~JWbrDmJ+*l4 ziSxuBRu=R=uR~uq`s#wd!JE)GjlQ;^U*W6JuNuA2ozJayYu5gpjf6f8CRq|ku~5M% zlfnK(qb$_BtwGz)>zrv8B}}O=(5jAPHWO(?y^D{F#tw=)fDjgp zw1&=?FnQ<-PgJ;b7bfEE;*G^$vrzXI>J`*$&{glk^r0{6ytYSUd}?Fq^}NxOrw7yY zvxKFy4|+mHGCqU>C&x?&Ed6=u!B_iV-tPrG3r=EvoK1Bg9*bzI z$s?ij%WO0fkrq5WWHQRQP`Q7?WQr*&Z^bHvM$2iW<8QnqbNkN5t())N z-nb*!?TxKFeCy_DQ*6C|=XMtx$aHi(VZ3{yH?oQ9Vq?-n;-Jg3F^kh~Nl9mNDv4$v zMXfJ6LxGH>xqr~!GfvYs~!%)iJUigb2&KNTA|F%f8L2W$!~WIQ7v zSfC~%ibru2ES&MrFm>{ZnItPPA#WADl$7KR@puwTp+cr#ylZ1XQ|qK;A+JiIr!tMm zHOhzTRg`-u>J12OY5N%>!E^h}IsJ17s;$>DLU!&T! zLi;LSc@6a8zf`_z)*4P_qTuGaM@F~Zg;_Uf>Kr5a3eEP9L`q&Sev6h`fpBoL4t@>0 z`s3`%!d7<5p%ILoKU8;6j7=?6wDrt_71jg$p#>Z4LEDi5^u?m>+*2YMU04R(>>HNE z{7)FE=zA@D`OnJ%gQb6`k@J5K&^R;yu3AO39XsFJHg=Y~PwamnC}BO`K+=*C%+785 zTbw1v`TA03a`xJJj4Wt5W(om>j+tXC-^9fH>fBK}g`;!?O;qv^^jSy#9xqeBV52b* zTiwP8Mcza%Kzf!x8pQw14mw%va`p`@h) zgxa|}kmAvl4ktJi$Yp{O8YY-%&jQURqLZ)T{FcHycF3f`&>V4Y<*P+b$Yh8(EbE438C6(zkQ~Yop`N9O=CxgMAt+ z(@`LO8GC1Tpqw1c>cJsW<58xALAxT~g$V@}?@;ZJBsL*{yrH>UDi)|Qzz|8>D2h;Q zI1TXZIBT}|6PUhn@f16AwClgjvf)g2=x<*N>)Yt1MVg>h!n$I=>PYIj04bf>4h_Er z@yn1JK#`7sCGbRIsqrdKs*ztl5G?taNsjA*_-KmCtn=wqY0wu;a76-#I>SMpf=iYJ z$LAv#6H3eAdjz;5WFwxxe%_`DVL2fbDLYIAH?ao68DwI3nw>y*G#05oXL2zP6Q5KJ zlkcNq__M~giGW~_yANH}0)?(xM?^FPqUEV;Xd$g4K9ZHSS+C%+h=a?~k=R@|4RFF~ zf{7#_N=}3GmCuR1wzy@zSQ)zvmbph?n*E&y{GY8-rEdrOpY;cO_jh~!aBuhCC;dV1 zL$d^y@`bL0{_g$oiLkf>x;%4UgjPk-=Z)3zv2ca zX1~+}ZsHT>xG5!+9^l(8M^aC_UL***XP5+>H|KYGp(L?lE^dXdkfh4DmfP$d4D~pOm3QO9B z4fRqN{Rce-1=>r2{H4A2)N^iak#u$`D~g+@6lOR(JDi>QW@Z(nQkDYGPxd$MACrpm zyI71qW3YGx6#j-3M&0O|brUI6Zzq8) zps^MnNXZrwYoJ&fA4(}90a-2VCF*H9L3KJw^(Cc=mlQfh5)Tw=w5Ne)zz6WbY8Kve zEfu(sgmw;20}eFZLY`Ki2VVyGGKRi%@MX3afEIz4z&@51z%#6Y#NM*hFN>A*vjToh z@M8}B7~lsJ^AgopNfLHfVRtp&B^F&JV?g8d21${0(;^dO@&~QH20J?J=u)ebDKZV( zo1is7YYenAWEQkGXidZ>mmg8K-#+Q%=IpY==hJrdt!w91u+u2>}ZfCc4$=1E*2HpB} zcRLq0m>V@(J(uLZjj};6%mv`!SUQM{o|Djp>%DV6>kQh0I`0s^1)+o@PE*xT8kuMr zX>ZvaC>j{p)gxqqtVDGhgh+`{AFw3oQd|ZYa8fTmuee@F?T_@5?KSPF^PWb$|L;g?KXT-z z_+(YEHDG)!4k;d^vDjEH{)LBVUs>d35MWf4HhO`B@xBt_whE-R@R`5`UEHt8>9cF> zl-hycITy9{A{*)%x=ko?V4wP(z$LQaGJtg$3|+*NJSxfgFXWl%Dy|=t_wt22*T2{+ z9u)GB_)SnVTd->|Pc~fNi5)xA=?)l$qAKH9LmJZGu)DCx+~tPjkf7l>T#wFso`1qb z3HZWrLe}F6S*PNX3+3Q`V7D6(+6kGsVq~mw0!FdqR1eBEZh|0hH*OZbep;^7APeKL zSgjUIPq?{%P}nPhB)gbv#A`%sN<=Z?IIiypaIL|09YTzW!r{}}b9P&7YLAbq&)FSu zqU<&?qud$*=2Xff@>e{OFIM0~*(2i59d4nB#iJt{UN&6Z1ln;BbR;+ZwoE{oi>F=NTz|+G{SH3C}Hc(yfvSg I*Os+^045<9Bme*a diff --git a/openblade/fuse/__pycache__/__init__.cpython-310.pyc b/openblade/fuse/__pycache__/__init__.cpython-310.pyc deleted file mode 100644 index 36207cc0de31767dd022421404cc7ad8e4e4a7f4..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 179 zcmd1j<>g`kf{&|svebd}V-N=!FakLaKwQiLBvKfn7*ZI688n%y6x>3CU3D|_iVHG} zQd1Q25_40F3lfu46*5wD3Q~)T_53s$Z?VV6r{pKc$FF24VghOh6Th_di}LeJ^!*D` y^PF-LQ&RQwfvhAjE3LFRRX;vHGcU6wK3=b&@)n0pZhlH>PO2Tq8O0zoI2Zu9;4M!8 diff --git a/openblade/fuse/__pycache__/cache.cpython-310.pyc b/openblade/fuse/__pycache__/cache.cpython-310.pyc deleted file mode 100644 index 2d39f3c56aa33e4b2b8430bcab71c109fba771df..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 2073 zcmZ`)Pj4GV6rY)$U9XMfByE~MO$(-g%Boc&km#X82sNVgfDBSA5WcL|o^ibKde_X3 zo5b2&NRhZy{QwBbKISXUl~cb02f}+}k~ntR)y%$m^XL8E@4b1OEiF|DJj<_Z2Mw2y zzi=@7I57B{5Sl*%L69a9RIr5hS(8zeoy2K6@Z|~byG@spZ;5b)_lgMblr=r!JSRbE z^x(U!9VPWnoXC1RYIkM5lc{?5Sg1%#QPTS#V%{5Am9^g&m4m87J4ucuMM_N)2yhscf?uNGNk?d&t`RAZSw1 zQ(BM^+5tUaz!q@$8u=q&%W1qgW2Q8S6nLK-UmnG|4zph1D0EIiRETl=Jun?G!*iK* ztnQ4c8ch9te^+Ih-u-?c(9aUD(q&%jFo8Sxw2BHJKuqi?M77U>A+6fsjvFjA}7A@*sv z4AKR^J$Z~1Iqq7gqs=*N(;&bi!&_OFEMP*yEbn+oLB@b%LdSFhSk7o8U}_EY#Lv^paKw3Ntq4SEanu%ZTrM(PL^?8V>sZUYjpi>N44V(l!g&G01)LM7aIU98;fx*O zOz4~12`~7V7i>Z)`3cg4!+YvbQouF(jN~63f*#+Wp`k9|jTfp6hZ}a-3f%wPo2HVg zqMlr6>xEMI}HLRV-N-pV;=EB|mdTAK@w zU4AVj9&Eeh;CC2Ya=;3PVaZoCT!(`GLH=Z?^j8X6M}YTls>srqPsj->pShm#$xMfzKLS=*Vo@T>Te4|vR z)LqnbB_m-<-6-!Sam#plH~RdGN2c7BMfQrk zK5S|k2xdqyRxoS|Z9>JuY_XK6Pp68jgtMN^-FjEPjNAG?EX^@b5OkID^M%y1%NhZ< zWkg|GsSEqmz;Bt0nIlJ@q+hH~{;}f`M z51PGyH3|7QcFtcMI(G=6c>#nV9U`b;3GK5Eqexqc)#n{VJ zvNRb6yRj+;Q4(y#iOh$2A^X94)ZLaMSRV$(mJGU45hdB?^0%l&i|TrmrdbgcahB%L ztqI_FlPJ$yO!J3Pv887o4wQVDqXjg2Hu z?BcrTFAF+%(16Q8NYbHTMD|B8owNj7CBlL?hqn!H2i`8c zJ$TnZbzjeZRw?{(tnQ_S8fq&ql;&yFmzu9WdbB~!_>_qzG4>~zJT{S_9ZS3+0#OtG z3%kR`n_@;ZU+|7C-V$?S9w!5g%wZcmM6Nr*F`64_cV-@3w&KOgJ@+{$G)RvF;J=I?JSjDbny5>n!$4y z9liiWNlOxvf|d~7KrulclVjR$S?VTmRSg<#8}Thm^IRqy72_6iC&pLTaUQrZPU9jB z7onG<(Hz>OqnoGUdZyL#c%qtzNhep6I4}NkDUiPG;y%Y0?gJmQ@BaW%z|QyS9;>jv3#Pgi2wmbzQA(!E7eyIK+HJ+JU^?miV9_VPthD_vp3 zdVc8%=YW@1>8^t0(k-oBs(OYGk}qeedA25iEdRx=w6S|_%Am3s@+&|{3 z0dEC>NYorQXvimBWR5SS@r%dkYMclrAq{Nl28dO!!%z#X1}`HFpu(1z!c%TtO``s~ zh}J%YsX2;llF^T+or+$B5L{kOvTl^*Yd}=rfn;l24qG!OrNr;IVD%s7H z$c-Mlu4Dp;wJXCS)Ba~sGN{l@+dBYqxwaCSnoKpiYbVcCA%%8!ac17CsQ{HilW0H9 zR6j}pb%b_Ny;$a$AGjK>gYOVANPFW&jWx92-HMZnb=e^9sisS?K>iD85IH=`=>qjB zgOG0k&0ac;K|g>#GdGOU#&@2>V6Oj?eZ63fZv0Oxo?N?>4?!82Y(AeAHmr>(YO>;M zAjVmNPCjIXg;@U`I57QhjlUV&$HZQAVk zxxxeC0_-4MB90jYzXb&5&>=0+sH|Wm%VuH3Iy7cI0&|V?%BfKDm+*;^s@?!f*Wf5L zSSs`^9>v@FpdaR2(NBK%ApmZGlQC$Gs+Se4!?#=JB$wh7m}Ky^L9fx#?P+b!&DC7U zywJ8eb1$3)cyxI180G#B#5v`vhyj2BB|)uk;A}8zT0Q%S=1YZB(tkjF{1f8qln1zm zJV03wqi)YG?Ov_t3kJ7!0JRt(Ze8#r`nTquQ#yx@5@Gz`h{X3ve(xEK zS4doqwe0rj(;&`+tg3823Z4$+Kn9gLpl}M-+{I{0cEB+t%#aIv|ue_0Vm49?z?GR@~19sW8QqaiUrUrur2&Xgq`s1eIN> zFdMZ~3D$Oh8;c%}2e-DLsko3W_neD=jRH(abFBDu9laE%rcTq&R+NfFUJe0{`u!e0 z%mZi;ay$br06j087RG!7ZvQ1`bN}V+RZ)%t)Ti+Ag+Rb4M_4g|fal6{`kYni{|!B6 z?NMt|RK2Rc1;?OVQ~w@QJ^N6qew^pvm#PFkAKAF}CAAbr^B3@#h~}Wv==yX-8w*~@ zP2BNh*N7m`>3Gko4PEsB8aUtrq)l2~f(%EBU|z&A@HT#Oc{m#(I@TU6bEsjbu0 z#_Un13c`|0;aQ+=BcE4`osA-HOphW~O^+f1!3t)(MuLfxGg)^o^Nm4q^#AR^?5nP> z;I!AU!OBm&VJNb07^*jM2tikUgv}Z@7-q&Z>JLbrna)sH?x{s+uELX}T#sn%f|Z-wnhAuORYF&ruE6R})I2B;QSBXQK1OC;;5Gr;3*fIMm$EcQZ{{9%~OA(YfcId+xdSp09h<&gV@9 z&v(}9MzN?U|Dew3qoH$KQCRqQAY9?hS6Zyjn3AQQ>Z^4XZOTuzw7SNKrupfXUe_g_ z_A@P`ZUEQoS)Qqz+^CPa>Q0VlcUe6zF%ww9)%Y0Cy=V6op67*k3NLJ{+pIo5LLcWv z(2J5@^lAfNANd}aPN14fN0j7HFvoi1g>ySGDR_0 zsyF4kYT@Bt_=?)OCv((T)*t5Y;=58^xHtF5^0n#TSypv!`m$wJxVu>ixnr(e8=2m6 zxz}y|$$EI_Q8K-H`ReqQ#Y>lx9$>Y4tva0DYV4uSY9eUrW>xMZayl-}*%S11@yYVT zr;o}{=BE=`4=&fP3a*|y zaSIo7Coh!F_BNhVA6D}kt{uJYxU#^xX$(R)f@ck)LPZOdhN#bO4txi)L8F0yz(Y0w>x2_ z4FNBufb~IUs37PY?M+v_o$S078La;mzVJAJsTiuk4!$`id4Arq=P>aXEUUp%(xR|X zCN3^|FNGuC&JXwSspxxQw6brw)!1mPJ3-SGuo)NLrW<}LaIX=qzz`QLfYn%s9a>Ij zNgTu4`zlPZFNk&$;4;3@07#C7gAg^=+uf+sjYM8{gd255AWosvKziJJvcSgi-6iM! zCy=JH#rn#Qva8H1tDP;iukNs2Wh>QBh2yXFNZkRp%eJ(>##tAe0!;%=+eveEmk}>U z)BUs@J^xzor!j&VJ36H3eO+Ab>s*^q0>;@^Cd%~nZN}5DG%*KydO$b&Y{!VQktyZs zyKI{c@XQEqjNrJiwwYWD7nW?t25m0cm89iCGx`}`cn>b7V0VlHs(1RaBSP0oG-e^j zThbzt2;>M92#`_*u53jd0VtXOEvGn2 zb3UUvW3d@_d@u67zzyRxT*B+bWA@zq{PIGrW-l)wiYW{blOQ~igr;bBJK&{x$N_E; z>rMx;!KHT#a+6Y4oS`+&0w4i6Z){p1)(DCc5luQRJdb4O2HB(IMC+m4SF*#sfEMJ4 zajruev=gP>1SKS(sX2t1##CmaRoO8K3#`aaNNSEHFNGp;s(7`3=|5@(L0&C>1#rM! z(;(0%-Sq(wZ`55+9(LD$YDb0JX%QS0ZpL1v#ThuRI-&F#+SG(HGjsy2ck(BkAPK-1 zlOI*AB`ASN+m|#!=|Fr<;39!CKnc>(#2LhF#|xUGLL*forJL3@Sosl}eC%5B_tM8OPVt1Q7JGFFxomi=2tPwlypb6I1 zECQZwpf-G^3boJvroN_HjNuqb+4F%(kN%*>>SPtUO5Qt%k7CYHervQ_ommfpzo(_X zPSX4KkJjcMJziX}7i+bb>mFfZ8Ti`i0I-TZS+0?^4mC{?AAu| z?D|ohPr`cGHIuH)3x^$ISg9Zp%c`LRaUsbxR9$98oyqryqq>1iT2zlQ6(c&brT&BX zJs6RYDD}w8Ui_M{Q$IRs_+sM%d0~s%sWJ*ppr@&2}EfDnS+g)q==oRyb)5e&MB&D k;Nr^R(^M^^X_}5}=|?R^(9ny_$QPNZn`g|tsTH(;1NZW}nE(I) diff --git a/openblade/hardware/__pycache__/ltfs.cpython-310.pyc b/openblade/hardware/__pycache__/ltfs.cpython-310.pyc deleted file mode 100644 index bcd35374fcaae967480d701a9b3a750f308a653b..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 11867 zcma)C*>4=ldGGG&>6vK`hlePMqK@s=WzDWgQ5#9t%Cx+zYb9HvWQtmOXIJU8rkdo8 z=Af#3R?>JBCoAPbpddD4CyuoV8RWrA0@yhaBnT3~c?ghyq27WZ4-4eYhyly7o!|G> zTpUr(4%szT)m1fB-|_p7?)vbsYvA)u?^gMh6Nd4h)ad`%XuM?@W^~Xr3}FbfVpPq7 zX&M=7Srw~b@pqz-z;~izSCfULN%!oEQ*{bXHC0G)J6TCrGlfjmEx6ovDnr$5Absmg+=dqPn%PwYsgat-8Iiy*gQ#=e6jXV)XEa0sp4Vh>t-(8`MgVz1bT zQLhQ>x{*Kl2~jX_s-d7(tH(iHs@I}q*+K}}RbmEpz}TE^w7o-uNDHI#gWg)nMV;(S86 z=j4NhM(tW9sHyP_by*GKv-R3ii7Gg%8e0g1%Eds6dw~q!X`q6Qk5lL6(rp>Y2WNuf zov=o|?)6|P#9FmbHoxb-|HkYaC#GkQPal2LJNm|{*;)MG@@UIPkIl}`Uh>X`AC`)t zS1CpDssCs6(Mz5;d&)Z^!Vix`@vL{~(wQsXjmwJ*mu_5{UOaa)|0(7l?an{C$^7O= z?qaYSdbc(4PkGlaUY(sidikX8E(WVXu@pZzg`;5e`s3Yx5QmX_^XkHtQ{H^t)ztIu z)f*Mz-IJv_JjLhrD#7ira@Z@?M0np@4P;y*(jN9mf_iPYd8I?0qnAF2UO)7OYjZs1 z^atYg{Hb;iNC2vxW>S1`{q;P$KR84~MHm%jX_Zz!J%j%IB3R-q$$B%a6-kEsKM6G6 zVkAH%Gzz9L3f2uELYNTN9CRPwcKbVtZwH!?QZDUJvrQ#RwOA#2e<~5hQaO5OYN&4` z^LbmPeZLk|L*G}f?^o-hQK5R)_wP1>O8d@;@Aqk!?>{r-UTlnu=k+6Tb@d1|ts%nc z*(1}kUXQ0C$h9+-K!it*-9CBzDC8&& z8LAR!_`sMroX@**4?5*5RCrXpO2ucWI7G#3RPemajrl8AZ_F>AKPO+O&MvofZ6EdR zM{x{Sltn>g%z}(rbKK-R%BeTv)kdrj=EyK^$lCK!LWB2vVs@)YC+U2WsAO=RCCR;w z%Chm$Y#D3Dx^d39b?BkhvewLX<6)weh(_)@v9*TUy7|y<*}`mqRCk@0xn{@7*lF2o zDPgUfk4&y%6o1?6rbs?P%QR>#c;Phj*8&-Zo)+(E+MuVm?m@Y#TEk-yH=3E3Q`nT; zMa6Cu`NUvHTDj3`r4(06wJ=gi=uK%AdU5XjyO++M_ZQA9CklZ;E6NF0A)tb#PO~_< zEbEQcd{UBVs+8VNq|!3D=hJe^<}wM~F0S-3gm|I00d=C#CkglUOw5Vm&*NU?p|HW6 zv}v1`>Ehes8ug&>r_Rsqlstt&MO+&`U4KHLyCeO=EF?l37Bww0k8N1uBwBk!PK9WlCm>H4BdV2H}epaizpYko-H1*9t$v|d8eFGylPNWiYZfW*kRaBmS8Yj4_e zAPVEST(i07MGnmLZC)P0EY@nR2;*pOAR-Zzkd=ve7?l{L5?0Ij4(v|L{EqRl-7=O< zVT)vJu378GazZ#FCDKpG3T5VD)ELB?DgEs=ZLePQn%mBW(5FT*E`1n!fOqZa%ruYQ zBvTw|;i!gjz-AZX>eGQBEAl7OnSghAX1RfA&)KotMFTj;}d(l6)@SeYL zV_qJmU09l+3@SaC-A8k`Xae|ciSSq9#NEP;h{V~lEHg)njBm%ZerhH0{iEi=O}Mwo zGAfH*cD+Ek1q;eC6h*?|&L@QmUChZEDy>9Y9jrAd0ih^!)&yS>Lo7QFlPw#HvM07g zQaCNBPb(=>PZN(4P#U|OKnth=g#mWAoU%=qoEC1)6ho*daXA>1)id(LkcH|UQlX|b zEpXMEsp)B)rWh_ZbD+vJl)%%c1HD-=iH^z!FQZQm!?wsXXtQB!4(oy5QUI4)JWI$+ zG?Y{Z$|G;0@hm0J(eU$B5H;jGC{!wH6pLXL@dQ~wS3cGIF?oXqlB#K?qlM%bsTknq zP@4@Tj1}CE$QY;7t~pBl<3AU0oC6|x-!dKhzFLR3RwU%=d>m9dtoz{# z8Ze!VWWsDQYOy-nA19V=n9l^5wzH3Ec9&@hFwJf=EeWP2!8Ba*wKmg|U|JGPv-_AP zPviIG7f?J;131VB{CqQUSk@Pt+d52RzXS>zh%q;fotmdt(F~RE(UkX5JahV1`xK1HX(y?(>SHYNL)uO{HSu!Jyt;{=ZCvN2{E$8t zhj91kB!2}xM523(&FwFeh*3-i7$bXh8SAGDk7#bABXIR4l##b-jRCnF>1~_m{xOXa zw4uGf@QggTn83i!hp?j~bP?%E$9zKQjAaXcn*~9efS|!_0v!;8Jhd1bEQlGrXyE`G zY>1f+F~cSIi4?FQjXEKVgE5&lHn_ls2Jd{aInp6nqsBeJ)Se_=#y(h*RHl796wO?_ z+6IH3?7;uxsh>h@QtdUqj~|FwUYvASUdXZ9pvhC4(6YV2eBRN}L9nioWTTIQ`;2WA zBWTg)8v8J0Wb)Zf?=vtlvh!(p%{f|ve}`ui$RR9X<>NK^@Lc=FbJChjpJL?DK+vI5 zzAx%UiV-O+DHUO|SL%Wt5=kD3yhTNkijWFIuRfV&=_T_hKZoMWxHRz%GKE>wO5@7m z+Ky{0U85$Cqy;y82rl2ENaL?i>Gy6)I-i>2i(*uaJ$4Y`eMxK;+mJCzipyfVm_*GH zSHv!{8?}^}hZnmSwY0b@_KO3kWyCe{ns^;GR~*LQ^ixZGSsWK9u-cGV5ZA@$#N6X# zAuATe=f!D^8x}XjTjFiha^hWaMw~@$MBEhT#Cg<4#aF~daS63CaTq?*dun8X5;X7$ zJ1+8BPTc?{sk7@E$16dsp&%mota#N>lmahaL8!63qQ{=Se96<{jXVfKT!+Uz55r5Qv(MaCsreZcD(4668zKpTm^V_6PnI@R1h?aL5!m_6Qv*ymhbM}pY;cTQ$ z%vZt^irHHxB6BM?f&EsQ!lJ~a@kpPF(-(PL-osFps0R17=ui$~!`^LT;NY&uWmpZs zpxUzENs9ajCPuUY7coq8@8GsyoS=QSp3_Ur>3sS2?ZP(>}w@`Bp*wl8;n&N z0(&;-18jq-HIS!~hv^1oXdoP2p6k;wrn4XVfCg>kz5EjT{tFjlC*01q18SpY|HMg= z6i`TLo6T*5q|2E*mYj>-wV_rr&bE+j!2Drc zxpf=i#YlPdVX8KQ`(rP;KmIUxeND=wcVJLTBrS{!EtB)dAgnM_nz-K|=P$~Ca!VG={AsI1=Lo%jClGQt+BvrOwAepTyvvB_HbNCuQ-f%7xK1)yBi5Bt^6_2UdMFk1I%IcV2r$3dWi77Wx2M~d5 z#{XbabPNTh#zyg%LmcLslW0vML*k%rq27tsoS$Udib8RCbL*g@@S5`XvEBx>Buhxw zS+at}f-b-SE!U7@0#<5qh7~M3>j_esa!Q^=E4`M%T~`<^V#eAKkIp_K#j@k!a;|0j zBN#UdvA4?O)YBfh1(p@-atqP8tmB9C6Kbn3!@G?VtU$l<)cj`0i}}rT9e^X^m`beH zSNqs@0DUYHJz&yi_xI=qu}E#f4=y1kwZdk{Z;%|o>1J*csnl?LBW-TI6v1eB*?gY0N~+lct<4P+3M&6ym6DLI5nAf1L+u0#Z;>#%R#L&e)+3N$jS>SQrBvHZ{2U-Z1zCF#E9Q z?VlVeu7t%qQKRZdE5RGbPdwkU;d!LjqQsXBg{8dOq0K==-R=edil0q&EZCZBQ# zhPFGpv2Hxf#9Lb# zAa;8`Fnc@d_)XL%%RAz!wVm;<@@^#j?I%fKadN(S@GL)t($DVT^m+j&6{aZ#(M}%{ z3@pm8;fR3W9I*xRTC7Xa#o!D%B|c#;<@YILJ46|MSaabAF>v@Bn9TUBQPgD=y}58{ z@w^(j7RqWVitr3iV=*Zft3(Bn5d4AO+K#pd^0#QUKca$xk=2pa>^15erQ&y~Ah48w zfI_8Kf@lTaI|Q8t0#+ovFG|alwIPdCDqgd{3lc+@nMmX-=x|+6@;rQ-I2xtLFXgj`MweD$1pcT2%Ce#>g*TQ$4$zC z5#HLkoS(ete)cQcNIN_0!*gO@*i=5B+C=u-0{uM8lHU>U1h&!nkfwK^JN-EtExacI z7v(@~#Bfnaq=3QfrSp@hE$p1&Ia3Rc3q%|#ur+5Lx{dGDIzk*1(aPNsidi7UX}t!y zg6qg`GG19P={0NZJ9bmk_Fg`>z>RhpTgwfcK`wf2a6Z3NG1MzUBiB0C%%MA{U7 z7iUopI~0-pHX8ElRItBnIaqzel9V{Pg-q``01fS4^VUX zqi^fywne<87Rp5WXOq^8c-F(oF3?~db{e~mqeKU2q9X}%vjH@1su9H#pxPusPRr_Z zB9m))5c3e3qOwy?^|_CUH9!f`HipxNCl4;I6c4RssdsI79nZj@axN;Lnt!fe3LuH{ zp1bU`EddyW1_ou*<`J1Ot-g;<=5t!d=@HPwBHO)qh5cn?G~;~5R@Dfk#Q znUjA(wZEi-auW!8i4^Qf^jQ+pJUnHUrR7qM+{XvxR}QdAWth-}V^@e-lbGW&YevBy zQa_z|-bTsh9fL-vJ68^4-Ubh!RHkbY%4l>MMb1i(QDECk7-ch~*iO)+*>yNKab^wC z64)`cZp2x5^SVDr-aI{L1Cmo5m7JIVfM1mVhyw3GSaB1!mYLCoB}$Bo(o#ud^4MnV zk^e+%uu{*~jm6OFs!C%lHyJMF_h>uqFpS}nx75SO zf2QBtX#V97(DyzuhEj4p#t>6Tmymz{-{!<1TWF&jJW9ec!frOtv#0>8I^h;H=B445 zL0N}&dRU3j^daWSUqu08a*pLUsN*-O_-*R#ODbq*d*CQv#h?{B3WnwUs6J_XN zK3?0oj%Ui`IX$K)IEaPU&2xxP5$(3`k~UyVMmN!hIw#p^<%mKII>fJofVZ-EnZl&i zC9)Xg#+I#jVXE$en4N$ip;CCnNpQiUDgPC7;cr?qvs%G65=3R@mctqzz01@0i7X_8 z)~|U+^TZ}38Fcz*<|_4KP>D{@bQe+PYv2rFC+9iv-ch5fl5)9Hzun+u8~p(qwa2Z z$jZPs`f+v_`f~2D>*O4sqqU5=LMPZ%sTnFrFp)t96s#e`3QQ!E3s?a9S@X}$BhVKj zmvSob`b(=U5^a|SuM~wk-00gn@A|u#@m10xQe?t6c4k2GX|($!{0N%pWv207nW?nl z{Oj}<;8FuGd*LlWY0*5gq{@LTxq;e+4( z(McGRB<)l7u(vB`Dm|y)xI7K68PN$8S-cpc-)N)+7GB7+4w(}!y_jcPwqvL8a>38j U$x+jC;Qgg@6S;=*wcLsS1MVd6UjP6A diff --git a/openblade/hardware/__pycache__/mtx.cpython-310.pyc b/openblade/hardware/__pycache__/mtx.cpython-310.pyc deleted file mode 100644 index 038978b5dd824ccb8a4a99b95fa5c80082fe4865..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 6438 zcmcIoOLN=S6~>Do2vU?}JATFvVmpahS5M2erMj+dNwL-H;fl(Ov5i2y5JiOqmoX<%erw&|`1{+E^baa@|1=bCNRk}nWJzKYlWS64F3GZ#p^{QlN{V=^B^7VArqxrW zluRQl6zt{mdv0uuN%?nWaN4TRQBj zTSr(9(zgUh?T!jA5AK-Y`oSF++yJ-}Zi)@EL(k>g5m!`nTD6cAA}erj<^!_n5`r1$PExo@M9I=G;@I^bS*sQsMn?NKXYh z&N`mwhfY}ayJx?Isvhnnh>L zU2b}w%Ugdii=oe*ifc`;J6^@*)(CSSjs%s_sWEGYS0B2;kFD`ht61~Hz`9_KT66V= z&%=?~$Mmua<_uO~t#HQ+*3f&l=GI*=v__|9>y7Y@@2{>jEM z*}EfK_a_;jfYuWv(=?0;-QmwmwGi-hGpyOtpAiMDMyVnqB)v|m2Uh!gKN|Y(u z?6TkV!t3|g#lntzlQ2O1b<0FNwk97O9DTdPRhbwonZOO5YAv`ve6eu8HNLZ^PEhVl z)3eV9oo)2$(hi>1>D}O=QCA&a_L+PA^9T2W`xgrQJ{2xoz-GUXajA8!rs6yEwZ$<1WPc^JBH@g>hP_ zugYR=)%R<$;g`$JM%87n%pl}eWG(uh8>idLjCJmYP41O}K080+X>HRupI#mX6H1Y^ zA^RDkTMCn3$WP^>w5=@0Dm;8!BI6Mp;wOQO&ct4tm-4yiLux-z&yS5#hwq5sGvqpT*w!dmRwYH?+w!2O#h>Zrv zig@^hpP->=<6Y!96%CLnJV6e`lnlj?4OvkJWFhJ8B{$Ig-+Kwpe2p3Km<&87#rohg zIpHxnOT%OGaSq!ThaI^L&tTBlU?Lo=Ol&>YYvNeoeUJ&OF1Ac~R>ya;f#2k1*Csy` z3hU`Se1Jw6ByyO@TSRtMBD6tjny31cAg$xwi|aNHKLv3i&5=t<{4|j>MBX8Co`^-H zr_SGnbQVuQPTJ9VF3~#4UemgY${UoQT?g~uX&n|w2@9Br-K>bx47nUqnPw2Hh{9M; z@UV5LRt>_>yZ3w_GLeb220OA&c;Amxr1F4Md?tp=CWaF@iqL~J!Og{jexRm)L_IGO z*|mb;gel&G;8%F|IA%_UW0E9JvjkK5`!s-v6!MTg)~667)lHM?$S$F@B}dW=c~g!6 zZ%qUjlZsN*hLJ7x1xf{FQMg$_i?jBvo6FNnGqd*cY^(<^z;r!MRk+`5VD}-fHVO*= zF~*HkbpnE}eb5%-yxnkk;M#3?E1@o+cry`Vnw*#UIJMGoP&Jz2o|ThhP%wy;hHZ?q{NZOKKNHfy?Xj6~$E%}ACnU2!Iv9I!>vZY38?9+%0rZ#0h5gA(= z(`e^mO7JOy%}u3U(nU!{PrcnU6J;G*L&rGE~RR$b2Y;*~okWlisNWY2tIx|HnjM9fus78$3 zJqpV`SKDLcBRSHdRHQ^I>wBIeNNHWXOG>vAty(zk>ed(a@Ubcr22#2V!z1x*zAIjSZ*iHI+#NjpN(#C=l9U>B&O zKxAjwLO-ITQLBcvs^9}6Gsx96A8YR3L> zd1--~4pU#8_UNMrkyY6ttOjE|Q=SPXf4MK-I zU^>9cWL$FW(AR)sE$R(ahme;HVghgtZO zA>Qdu& zRh%5YXN#K+jmls(oHwL5N|fn4!+u?5s@_9<1MvBiKcQ7}aw29-`J|w&Q?ILQrw7tw zN}&Ygq{KQm9m;g#SUS~XuHat@@GrMA`@Zf7pT)nSDilCGh3eh-E1 zY?s*tc8YvMcq?TmLPva=x}}3=x$Fjk7=+uX>UjCi8pM$NBkH%C@=&V4DL42to|lwT zCiY5@3wZb4Knh3)hbO*u^mQB39(vj-c!Ql_p~PA}>_nFO0qvwWy~I-aj-`?c{yC}O zJ`u7^Y$Vk#YxN~u+d%vQHTeRhqggU67wRQr9jMtd1b-#rDu_tw_X6{9dkyCyj<<2TO0wdXNEz>Q#ruGXTk zL50CCdVcoi;@slx_+(e*t@ib3VXjyt*@-UM%G|>2(nl+HaRq7Za-1&v z^#)LuU%_PjDv_TOnItkqgbtP@hb85SL^e4ZN$&?4EHL}yYI?h4o8IX-rgu9o^mgX; zYn>XgI6d)sa7sG$U8^(093C?Fk;QAwW3|`h8;kiYcxAERm>!c(1IBeW1LGM!du(^+ zU>vdrj2p+a(`0nZYR$eyIa@@}Bzaz_MD>bXwqYEwL?vmKNO=}yJ*oaIm3Xe_HlKd| z<&*9Bk6&(YKizC^Z3Cy*6~&0>DQ-0VOq6cJEVnv@Sse<>6K*s52$X`}f!>AQ8}~Kn zeHaIzl*1N`xDKe|vMX$kHD5U$kIjSFi@Nco==YPHZE`uxl)%HZlZaiZ?Ov{QU2ruN zd0H^8Z4h01Qn6yFOwWbTZZ|6urRR8-3?yf<AFM860js|XJN6HjV|5$|rchi5rpkWkhRTQmQpS=qNEy{p-4t2Q zAEeRlHXij5h!T7jTP1icCMEbSrpi9HP8>#*d*YSkl)Rygmh=teV`;y!{^6YhD4}0l zPJ85!9T1JesGIaM&Z1i)eLS*T4Kh(o&vkN@0q2INZ+#|CWT_#!;SuYeUli7I&n-&4tMQMe1GLpcN9j-lli=m`zOdQ6L z_a+VW9OK;t@Gw@rJ_ooHCTcEX1>$1zAdSfI1Vu%b=~%%C5C|TSSu4V~+Aqg3>)Dr?3h;*2Onw9%hK8W~$v8 zg@Ozvnfb5XElY(bjZ^EWre^o%N=)~lbRZ0%G{5$a{A#tVU9*k?*s00F(%+je185gm zgU$YxUbjviILqI=A`T449MY3DkmLMZ^C$d=>t#@#-fE7%!az#SBPfEZ8413}Qx!qx z_VV2b3_a}gT**ikQSxokV^J?h2DlAGJHuVMHu?yi5%q^sMLQfeL$BP9;K~4WwTr2M zBYS0KWmJQJd>AduQ8bRu$J=7aZ%1;!HyGf7{D7xJ++AC{;5hR@US5=p5T+`$n*(b5 zSVoV7?l!_)hIDPL`n&1*JJ@dnKDh}^z2=iT)Gi0A z*A4jSUlyoG>x2Tz((&v4iFgksB=_*0C`dhUSIcF2lF3#W5>Vz-)vcfl5v(eo?n? z;m`0%w773MB&4)PN%KF#tjyzGp*u7Y77GMMT|-wxxK>{N9Hzu0oS&K8jky#y&IG$$ z{oceyB^;Js;bMazZ4gTbPY+T)z@V?{JD&PJLAC#=y0UzZS4xTZkXeh;^Ls`JUJI#j KH%YU(;QtRd!Pmq9 diff --git a/openblade/hardware/__pycache__/safety.cpython-310.pyc b/openblade/hardware/__pycache__/safety.cpython-310.pyc deleted file mode 100644 index e56ae30746f318545a210700f6f4a88073fa938b..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 983 zcmZuw&2G~`5cb-Rla`kLNPsGFFh?I8r)twS5=F#~MI}m#!T~7~G8^xtb!vOvwcV-_ zC*aDFhv1d=%86G%La=KmB^9hRvMXaypyUC8GDBq>CxOCw>7m~R` z5oM?lMAQhy>QIffp@yWdMS5Ht)=FECjJQ6m3%dpyF`z!yNu4x4>%#_FdZSp&(L{Dx6WhOTmT*kv};0+gk(*jJ}46z;VM!<-c`U8|07UjYlpArMRB_p2>`R1wV|&wN?fpJVYMDVgPuDo~X9;`fAJ3%1YX zg;{l8pH*P0ppNSj1%L40DojoklT`EC6^;t95Hp1LO<`-|^YhZl)fVk<_P4; zNDhxPKb2EWUo*XtKud=GJAfMkO|~i2rm7+p>AzLoK)SGuUu;8c_pK#f_kBVG-{@nO7g-V%u`_lQw9;mz zb+AjWqAI!O9qeN+c>+${c!jxg$}3PD=pJcdD>$Q?neLu%My;=VT1l4fLK)(56Sm!0ee)HX>_a_iT{O zUWKX04z*55uPP-QBv(omkZMwLK&msHR;l}(tSGcb>(3Ngf2uwuy{QTN6rBe9w6sst zOY}Uw@J#QWp_l2q^gWQyQnjlz-hYK((jcN5#Bq`ZS(wCW8}tedvY;OYX$s1j4hy2y zfYT=dXDcHR!02>?UDg@JG2@57IcXNe6c)Tz`_6W&b9dhHyv;kKAi8nQ^Oo3t*k@jL zFl6&y77Ur^b=Gg+xbAst+y3H4dj&!JjHmGS8`plh_9blIVKGg(7g82y;cnQMuFZR` z+v^@SJf0_X)X)A0*EFF{cIKPS z1u9+k(dC;z{CF?DKifE*Io+pn$G&v0xrJ%i1c50%LY1Dn4e7`R?d1k8!(%Y29~IH`)SyUACE$Q zAha~&qB=o=FpIAfR_VI1$9}b6=)(5>I2bVB7mn`_ASw~cuJ1pF1SnUge19UveE)^Q z&w`_J_-kg4CrLK9F=X-WD4=XE0qG|xJuK61F5PPm54Z(;E<#W7PCAOM+QcRI7GV|~$kT-t7WA}v4c7kizL3tL?K})0Ku=8= z&^z!9ZNf}O*>IGVr*f8!c-$wjGHz86aML12@OKa>IU-a&QlUfqj`UnPR!)>9<^I}{ zksHV4L^(2ZGrjQG&D3KpH&4ism0MIB5q>4Nj&-V^5Q!pyFiM1-+Y<3rZUe$75h@_e z5>d%360w?B00F1UGc-V0CBn%aiTFNu00G5HB6L95Imy*r%XMl$*MZW?;ai&lPnq{1 z-SdIG2VTKKQ&^+;=Qw#13#HAWq$(Vv$&mx@!qCu9wN`|a4x=!O!kDGPfFuovV#;4y z-f1l^`&)N6m;KJNFjE%r{zGBx@nkfFjNDmnFKu*isp@ZbHkP&*w@`6m*lBfJ8*LC| zgd0`gjX{))1HzpYC0vkvImJtwdatWxpr(IE`!iNG7;0z+YkjA z@>h8CplDfHIA9Qcs5JI|5QXq}VZx&Pzz5%?tD@HZc71bg+3&9S@DC9hEQmS}LZBsy zea3mh;na-cDZ2bHeiaVqvoOFXvUC(>uhGgsg|+XXrbRI;20Tiyz2ZN z9I(1H?BZ}~m2W`a$U$l<=X?g N;|RC&Q*k~dXMb63MS=hT diff --git a/openblade/jobs/__pycache__/format.cpython-310.pyc b/openblade/jobs/__pycache__/format.cpython-310.pyc deleted file mode 100644 index 92d947d322d60313f0cf781aaac71062f9f4bac5..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 3597 zcmZ`+OK%&=5uTnGht!CYEy-t}xAK$uvRjg?~?FbuElcoAS%UP#J?$!0K`Zb~E1 zgYIrh7C~JiUlPEVoB|{Pq@({*Uvt`XfE>IDlB#Y}q$OvF>8XBqSAAbqv6`K2Sn%BX zy~6MKMXgiQbuSN! zdZ%7AIt^X-^O+*-gj%lUvqiJhEap0M#e8R8w*~p7;&SJ*mh1UKvDjHGmO4ws`pTjW zI`hV&Gskx4NQp`z5d4QT&3Zr)%tX>ww3f))U9}tNST#8(Lu#ucJt~WI>_X1 z#9sIEEX`z;R%JKCL0MI{zsH-lh?+^tvXZ(@K=snoL?(H)v%z|m$YjM2TefO`oo(}k zA3jOam#m~}=IgEJo1+SJFFr|lT2b~*CSdV&_g9=(T-|)G546Ev^)t@I)2Yq%0N8Bq zMUQd4^#&9DT(%t5c*YMm`sKHIQmV^lh3T1UW;5wBdAL=5v30*yqD^*XdR4l&VD;V!mVh^>ffcPqw#}_2k>g04L}%Ld)5g# zwT^6L>l@JCp8d=^2DXlB4%VJnM-FJ$*(W@bE_H|Guh!q~WAXF4e;7JJ5zX;t(d z;Yfj-Mb_YFA1avg!1ZQJ(FybBExc zLzmtj@9A6D$zc9B<;$`+8E^F)i*LWab)z4-tUnYEqNp&gdc3CPJjYslhgImTsn5KwUOhgdUqNFc(E1nHl zBzH40Mi+2u9wOogU{T?;*|s>cxHV_YIuG#r(5f}) zhTGi113_%TII8Mr3tm9hC<(X)!^Kaa2uT3HMR*ZY4M3Ux{SekdK$j0#bIEyZjn-Ii z1t#7gqRc(f59olTV;x_>>LL^^cjDDd>kVx6r&{$b(3zNigeq_yLP?nvERIzpj*E)+ zbCkn4e$`KM)5DP^{0l6eV1W^Dsz7haF%UK36}W5wbZRu33%YfEas(BQE~ZnGDeIlP z6^7pT%NG)*Xc%EgpBz8Mct3uSY3MdI(|!lSkemP)bZ8+%7lOPaZHc_nY32|D*57R4 zl1pvsz4eYf=>fky3`h`w>{AFDm7&2_92Y~GH?w2>c&|z)*hvyZGM)*n~e0@WC*Q_BrDt7CZ88NM>W~$ z)eD0dX7i)Chptau90I&#xiDJ3xJ$3XzaLEKaA;Vi^bd2nzwTU})Dfx!58guUna~IaoV|%R0TKCS- zXS?U5+rE46+w6vCDi^n%E1pidko~t9vrF|(lx4Y!RFY>B)b`6rMQMKd1E1zH!5d&~ zeV2?xByL|s@vl5%F<#wX73Gh?%E^!dZHonlk~6fK$NYOTbXXn8?C8CU%sCe~NzApE zmiWSf``Q&;%|xc1vpnOmwG{|u{J+HK!B;G3K|*+>bMl6sb*XT{*0r=Fd3s@#H$?1! z?O{4llj$H%qZ#A9{XtLUx$1pC<=KlgVtjBoIypKyyoh@H`zNQp!-LO{d#5})?j4=7 zql1fme)PrZ@j%8RQt{O^VuRmQFQ3Xm4gezqFwRGEkUTzee|oF^Fif&Ug(33%8(9bR z?&-f%?>~CQ6g(am%J1e$#$-uXkf*4_S=Tbj>3S6ra>qC(*>Y=us(qP7Q+buECLXq6 z`wt3_K4)FWz?!chJiOt<)~#VKSv28csDm(^O+f9%W#tL1CKLXl&5~ICB$asH$wy-ztJm+80|r81Qw|Rx6l6(O zWNsqU>lJzZ(Y#jF#x}ENL_E1}6m;n*V_DD@ox98~+|VuTrB}FPdqpWJ z9OfL8xdXn|*f$7X;jIylu6t_)u&fbk|BKL5^#b}bAq#u%7ydF>5y1Q4XKP-c2Z&SD zk%QVP0_H6&aZv>0MrH%QU_mQX@S)`T$AXTV%hrk@L$zB#bmsqQL0i@DK0Ck7;?BC0 zI#p$Wr1Fls;+;4WN+j$OCDY@5frWOay(9}|nya^{Y9BNvk{hnuVRa>xFe13rwVNoN zacHr2btFRxfjQIuC=zkbxa{tUT|}}aSGtDCQhVz|-HaI9;DV15;DhMeh9m8o*l^*J zgAorC#V4R&MG|%K452;ArG$~C-8i325~W?43&mM?r>v+a;8c=;S-`<<&Cx+Nx?Yjj zwK7$=%ujrR_ddmKQ^lr}+G3VLL$9Z32>sp$FS!q!Z@JW=0sJhwM_t-5^0w9hdqAI= zlNJ>RaOte;Zfai`x>ena_ralC*IpR1JTANLVg@r2x=!F+&>vQlr(b=`YG9iN(oZ)o rET2S4)*qGATbBKejboT>WomLR2gjFXn}3BRA<^WJh6O*`vRnQ?xA?cl diff --git a/openblade/jobs/__pycache__/queue.cpython-310.pyc b/openblade/jobs/__pycache__/queue.cpython-310.pyc deleted file mode 100644 index f7f3d2e8a415fb4b9f664cc007c7c860358218d2..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 3803 zcmZ`*TXWmS6~-;X_>6-ynn)Sb_<02Vt1XuVf4;P+^!dRX6z=VZYd~r z{lGWRUQ+H>f{GcJl7;SKu-L5z)ov}Qb(exAGw&z$?sBl)Z3K<(ncxhwWJQ#ZS+FA6 zlU2F)lm%x$u|!2IJhsHbF%Q-S-?3VY!z;IX@Ag$1%Pec|r@PHB2XY{r2c7(WQ^_nH zs8}}BgPv5`{Z7A`t0;cZ>D}AA8*T|EIl*h7-Kp?MJ!66NZoTpRn(xfAUMvY zzjLEomy3d5v&1sKqdTqUi-tJ!*bAy+Mb?TH@x3Qj#Tvd}nwvc<)-hY3>N#RVyn&hJ znVC1mIm|T7Ue3!i_bjYHPO*NJ`Qv85x*<*2(B8%6t9!R!GP8<$Uns( zw?-^yBYuqN{e~aeBWs_3i?|!n$R64I&QO4ftRSURjM|9#`tCo?Vd~t3!v#ENHHbR% zS&da*I=ueE;lndi_$RvBBLj3qO!Q6;G63zyfDSMKH0#?YvxZfS*obWa;$3*e3Ctb9 z&H3RbvyM3Q+}s|qW0>g?7npg-57+^j;qt5*$WhcXmIuNU#QyJOkp&b5XUi_YZ(^)% z`BglRyL3oYtW4z2f2x4vX<*d7oHJ)+pW+t_{ zCgrJjXuTH16bu2OHA=V)e^)RVSx*4uDLZoDy3UCA7e>xLAKAJ6#L3-#?Y&wg$4zRwtMM;a?NOv$LcahE$@WDleG*1 zx-?#J-Xh~45v|);!q8f=IfyOD^}ero4x91kVja>)wJ|9L9!-0j9H+u~53`2e%fJbL ze!`7y5WK<~Fm(>0%pch5Ty6{a`gfLqrM|QN;T*F^@aV!FUh51$nRkYk-5z?TmNkbP z%}&;g5=w}Nri_Z_?xCr(6Zc=kzO)S&Rpg12uqO*B(}h?A((Ei&m<=w@g^ z;6aP5&Q>5dzaOql-G4^)lzGThQ}4fn2Vf^{Dtm-|8Nq|pb&f}{pnaC)U1w!M}1;RSQy!L`qq^wO5o>HC$1^#2|*A(Lym@pVG z6c`QhYXv$`zm}sXqbJagOb^=e6tdT#e0v@x^7?Rf+|bO?JcfQ2|5g;9&&ElC$bAKM zLDje*ehQ6@noJ*@QdZx5K7;v`>i@@l8{Zqu8wKVO1EVXZX1{{$HApYcBQ5yE6105- z>gP0~T3rZJmzE}sjG?sgdhDXe&l3XIUmFFxh0l-kd0W~jW&Aq6TNO6Z+VpJ@&BYUa(AvW5os4d(~ zMNs6#XiWdqZr^HuqRThGie*3Vq&@8=={aVi;N`g8PZD+aC zUb?$4(cw^gq23~4>IhW|V-aH@ZKK0e@6+7RNc@7t6%th46tyemop>s zJGBHsPMN&};iCt<;5EEP$#K1xC?qev@x^)N#aeZ()NmW#k6iOrYeSd9P^2*cm52*) zps2IZOL3Yc*gaUvwxZqGY)3DcrZI`KOlCzvq-s-?K@Y7ZeNH~B&rVy`Es=JkPH)Q; z@ysLxlk-i>Q}Fz%GOR3S|K06Sa& diff --git a/openblade/jobs/__pycache__/restore.cpython-310.pyc b/openblade/jobs/__pycache__/restore.cpython-310.pyc deleted file mode 100644 index 3c977af1720f0a54e3f90e318edb520bda5fd5bd..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 5940 zcmb7INpsxB6~+z>W`>(cYPUv~my9hg+N>ptBUy`6jzr3ooK(h15O5k4L17s;Fp|iC zm(Z1}R7ICuZB>y+Uy>h?s{D{XCMVxgshk*F%J&*G9Ey^i3Z~F_{rdGXZ-1TE>#l<5 zBKub4#?@9;!gf@P>#e$kooFZ?ZVksHtr6L;M5FOoYb@T^+9&&NG#*d1 zCgRD~r0iFt{qcd;0SVWlgYlu(Aqm%`!|{>U5eW}PN8{I8uf@k&$5bUSK2+E+8~I#e zBOBU=+GsN2Ok$3+b8Lc5er~j0XYa9t>=2+6>^wWdjskjvU0@A%9MGHWB0Is} z0Q45S#HQFZptso%*(^H+=m*&6G&_S;&unO|COdmqX{1z0$`0hm@n#gx;(g(X5nM%(o8>&ybLP^=!{+~Pq7emK!U@qV&c%P zaFP3b^#i~CAV`>~-nw`5Zm)ykiEBR3dC2Yu^Gi!nm;^UCPdU)bLHj}0jcQM!0~J zr>q+hT=Trox_;Cf8TPzg67@U|!&AOe_%VEBvz^T3ac2fH?y_KNc4msFX+AaI36c*Y zp9M2#7cZQ;Zhi|xiuHk zX%vk)=({ij(!!?64}+Q?q2?$xuTk?RH9MRwp$6fOqoD~IIl%Hi%YRLHVeugE@gKcma=J*f85gWNA)qMeOS7{=Z8a@UvxJ_ckGLtJ zZr&D^FiV$G4x14+4?2YayR9j`p_2mq!Q3e)z(6uvN?(m{>6HRJli&!)a6QlZbDVe$-3l}`b&>pTq_AC^Zc z_al9N8%TK;D9kA$u(K%5ag>)%K?jznAcoT^zEI@;8umYR^3=)xuB2FdcO8|x%9VY_ zpt;$*ppyH#DnA2Q*gc&Q1}R0GcI!XH@G4%p`yT=1N}+D3>sqdEYK69?uIq))l&;F( z1V%3uU<^P;q0u=DqjM-E(P~WJ(wM=_FLh>p2{Eh6fW9!~3fFl)Lh6R_EYHy4k8Qa(J{g^VYrjsf)sjQkXhx>YTy{KaWP3(szHQ z@r%@}mK3)G7W$$>oonP|If;?aplKSSN-Vota;%)?B5>rl3qwV6crx z+@U{|&7yHsSEbibHC4kigs<_f@yvMUR-HZ;-!awr#(8Eu=O1FywhO!H`HME0L;oSw zAh#}6g8}HNh7K`m1*}>MbzF*Bhi$`jb_8#fiqaGOG(=7+Gz(g@P;#pvfg{?K^nG#D zw`oG;A&UZj2|Eg7IgJsu`H#?%LTcIrneZPI_9``3sKG%Msqnjb%cad+r=gqF{4a*m z-nT%Q4WV(=Au!8@$3LVx&-qV*5_T`{@rvw4Wmdq+a)lA)OBpA7;;*CGp3J+QDEJf% zKG_&>`B!aXZU3PsO zq5cj`T{m-BX`vQcZsg{QRp^BYY6GG$pIFRd_7hW82uCY6ow;pyCv$0RR~up0NXrC9 zTFZ8|7YnBlhQ6|dopwC3#ewol28&uRB?*|fxGMkrJ!CDz+z+$BOA~L&4BR!j5@|43K zc>*t7@{%Bc1Nj8h*-136?MS;*}Z#?WD< zB)@^LDRfBEVCoip8esU#6?oKNYU}1Y2-P+hQZz1Q;KwTU|uWgbmATU*NFoZ zjV1Swv7Nr;eJQ0Exjs~Ky*a#grkBk%{C-&zu*MSXps~c$xU6gP6r;Ne8)#NHNy6m* zoN$82os{PR6IN+G{0@|he@yyUX{Rwf3kGi@3`(^>L4$8lvp~&VG;7sq}W#0^9wnr!+u{ zzNOVt;w}?OX)rR8}<3vT4KU8rDyPK5cd;Ej-$sBnI>8$xEEj7H8B$pMHs5}buWm$|+HcMOcItZIX z{8M~(ITh(+h`U7BPB{}!pBWIr&R>_wAbA|&!j1WnM|F^}mi+|5L=5}w92Jh&KZS4y zLfIRRh9P}acd{V%AdMW0W(As>GoscIiw@wm0BLYo*7HfdraHRz+)>A$8=Cg5W{zNF zQaz+O8ab`jm1GMqJ@aX7Wp;W&C zpkM80ia4CoTUK#j8xab|L-h`bx{har-A>AERumi}7?P04Q=S53IrZonxk z=WkJS4$TYRmSU)e_ZppnDjWDX#e>&Dib*ZV-KXSaDPtzy_z59`ZXkmaC z+8i<}c`!1_W=)1t;kIFBNMWIMoFbX97jE3WH@|R0m=FC3vY={Is)DS32MiM$iYuG^ z1*RhPS|vKU{BLO)9au8qC8+gsP!>=nXB#wZ2Z0{X}~XF$!W-PU>DXZU8!XEo>^oBP3M*4a;2Z_tS>1`$;w3amF$r@ zliWu}5DCdM&(l7&(i=|Dn%`@zW&9b;K2EdW5w_>Cv|V2JEOl{bfa@Mk7AvYd*K|R~ z$&bhzfU@|v$+=TfP5dl_Lw)CVX4>zBWjS_afa~Ss)3_z*-PV<<`N07ROJhGwrWebr zyG$nQGEEozUm}t>CRrK8>U7$dNpN`;M47Y_E`!zhU=?V4hN#UoelEg(in;5Wo=0p4YCL-LoSspWdwdm5NYeH y)L1x|%Nx1(k+v!sDTblcw3;?TF+igzAR$;L!J3ZO8c|)_HRSK8TXBz#UHUJ-#E5eM diff --git a/openblade/jobs/__pycache__/verify.cpython-310.pyc b/openblade/jobs/__pycache__/verify.cpython-310.pyc deleted file mode 100644 index fcc79aac9da7b2659dc88398fbe54cadaa92070d..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 2723 zcmaJ@OOG4J5$<^pXUSdeL$b7y<8dCgX^6X$qxDLNAjj6@BvXQ<_(9tl45ypoQ1hU= zn_h7-3j|pMi4Q{Vc>udd{iVL<ANf{_>>=8?*l0`C}u!ysVQzj(r z(Ygovt;wBwLIt&YZ#H4do=_j~ zFsX59{Q1n+L|T6&L&zSrzVfNPZ(bu`czb4L9N3k4Xc*+->c@~_8-DwiL15lFBzMVq zBHr2oEB-!(`)I!7^pwd5$vlGN2ZpkWVIZ#lL=FHXhXy3b?4^XX@ypws#q*9rB+&y>h;-&6|^S|2e zUSjcfck2?}IzK$ew%)z8y(tnN%Vac(>E@2?mJ_jA!UhAq!MGfV%`XunvyI72`Ioae zAJF&;1WSYoLqK+aB1Z8$Tknc#zHvD#lQl?li!0j8hVIKdU%l3av&;6I*;^)Er=`u|d`59FF*{~dp zSt7riIt+6eZk;-)Pbhs%0DO9~Z+~v?*?SHo#@+2#79@^Hb8H=&d*1FgB+{cre{_s9Kq$PNM6xzzg`0?XfhPg8Cfhsd%a(oMH4vRMdh6V_-8Qz=3ykP*O(hVzOF zp+E4Hi46_R-tNh(BbGb>G6?fjM)9F)A zj`YtP;HesSuD^dRy4Ab+sanrc0eCKijEe-8L7zItsVA5a(8QE=pGofEO1(Ms3+zK! z1vadc97|X=bSuFop_Y<)z|owVf{)^C%4$$+7V@S=9nVV;cYoW{AYZ6a{avIhBhnBD z>cRaPagGD|q>W?yupv1SpMSKFoiidz-)OQ<7!8BHY?AlNm z=D*h)zWKw2VSj7>-hcMPHuqbg$`-#4n|EwYy1HxfMT}|*XM?(gk?ra_M$)W{89B3l zcl4~+{1PNsqw8GH>d*3jSL<`$!8NCp7e%y8qDTc%G@L@yW>Lhk$#CQWfgc5ZzsWCQT`-kPw`D4U*$jG_;Wrswff?hagA=k|kt0-c6gb_PT3ZRjS_F zzrY=iz3_wf%89?giJ47Hdtjs)znxvryfNM7cTCb1B-p|dIEuy zgmB84qy37ec zP8UTb#a6v6^JA?_)z({QN`Do4@EiJ(J~iP%CS^900&sA+ti-2G19aZ08ScX4Ln~Tg zf{&tZWpkM0d-pAb#Xflb27(|71r>}_;c&*CxyxPdU4a7b3y%lFKO!QSvnPZz5ejdG z=ROa42TpakgS?IZ*^)HgsqO7+(X_gpxj#kR6jhwdtmPsa#*tR4iQZSDd@VCB;+@&vXm4kpMZ@9VINI5Mz8j52 zwi}Jce6&3uiqVVlZd~U&Yx3hNsxoFaK9hoBK zE-Gr8K)9u`D8 z8p&xCrF{s-Yg)K)dJ_|OT`j`aI)H$y8c&C0lRk57Zqi>fL7H-vL$8foNh$JXp~`xY z&2n4c_|39flt&szVjlk+_kgP+Ta*J}Dr9Z>m`8WJ_%R0d8m&lUU)%A2^i242CHNKm RtE5JOF6przMGx7g^B2!*-tPba diff --git a/openblade/simulator/__pycache__/__init__.cpython-310.pyc b/openblade/simulator/__pycache__/__init__.cpython-310.pyc deleted file mode 100644 index b18d9a383524f78689adaa4425a52bc9e85f6f38..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 240 zcmYjMu?oU46ii(#f|Hw*Ylk!~wzdTk@dI>l6C|X05vy$zniSFh@FV;sSO3DvR1iFP z_qgGXJ2@Nbh66-#^CwFGE~z}X_NcN=Yssx6t3%_L|1ENamH+H;6PyW_;5dUg zJOm=1WeL%|VTKy(UvdoTw!0vi$Z-n3) zUV(?`Tksrv?TJ_DspAdMj5OnK?D020w%Xci5|pR*`OPz~f2^}9E;>_G^BE10hyVqa z(3C}tTI?iF>P9Xll6@oKg7-+kTd@_5JgCh{r~a@vU!==K6iT<_Ymr?^-8O|NWcLLb zNr#pnKolZQgfZy&U)M2n8xq@yO>xkfqMA==B#9_i#LkFxZrsR|j;sNl5MY?sDO;y1 zzI|mYy(;CbQ!g8wXCjrHmrc%71aG!FXvUxvY0ieWGq z!(jiiCkF@PVQ6A4iugJg5Z)Dm%1x+{;G!ZhZ{rJJ$Jfp8wSATbs!RVM!vn)n5@TiNNqyfb9-9r|6pnraj-wcH%t?lG z5)*7IpTT7W-H?P-zBbugs zReRezVeLasH_F<%QAQik@76x01&&A zQ>uJWuS_q>48L~Hmme-as+LJOCehlgmKU4V<@uR1yZeRIk-0PO?@d(YWdsx4a}f@s zP)S*Z7o{i`8RH@vZKvbu=!%8I;dB;`PF_vInP8J}Jmce&tDzXbo=qa1C{`t(OU9$G zRVYgxNg&uZ1p5CDqo(yi`Axa-u92*bm;m|Y8GeF~aFbv4{o13H%LICF-!E5aMWDET uaZfzpGG}QqfI*2&KXl)8+)zC|6MC4+3n|A;*MR$M_$_TDs%_ diff --git a/openblade/simulator/__pycache__/faults.cpython-310.pyc b/openblade/simulator/__pycache__/faults.cpython-310.pyc deleted file mode 100644 index dbc94d82c49099918e79abc1a053b2fdd127c2f5..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 2970 zcma)8OLH5?5uTY{EZ(F<>J`UvkV;|)wg^y`Xo+P;5e!MJK|)q1%z5SHEt zJi8E!DDXsHb#W@U9K@r3LH^5JbIM;(l~w7UCBcW3sw`>;-7`JYudjP2u2xF~o`2My z?fxrI$iI;}|KxyqLnaLQRgLuNCIv=JDgX`5lr&S~BZtT1oqHJu9zVbLy9 z@-^WW&mR$<7bTHDH0`o5VXnZ7FjqWeht#g{(iW+g$BS$3C`fC5v@5*SkE5CwM}7bK zNV=Nq$Fi0VL@n{dQQ)SrZ2Sl8B6X_DZWP6-mP+bO6}g+bUf?E)%JqE_a1b%u(I|v} zV>R0MaNGHZ0`mwRy$6IKHU(!Hr?$bFZF0lTanrVV&dzhoF7Uiv}ZKDqY=^ek!KyRc|1? zACpn&gnkmbsW(v7D0cc%2qzi3o>22L-icEuh`k>jj91`$shaf$ZuDHpsg_C@gU%C^ zI`yh5I8Nk-!f{l|al)970;DUB^V7%;PF5;UyQ`0#wbk{f+na4Qv)#GCS08V#|E=xx z);HSS?VhS`bhkUbsdV+*&GlY;%3f_fX}7-F+TL(B*0(lRd#xv`+UYuLo9(u<_1D!_ zTg`8Fzv}j!r`^^!POIBld%E80soB<()y~)L&8e2kMi+FRnDm;+-$KIl@J2S%aM*;b zkGNP^Y%a(+P8YgE5q%Z7Tr}_PJXm^gukS7_ELdgvn3dE$Be+8;#-T|Bm z@{bU35fG<<+lHF54PlBLXD=xL&JcNSt`GoTfm=Z5f&LY*@amBX5d478@+&}<7Per0i*^w#i*Wpr@6zNDt%EZGw1~EE)kjCJ(FnBXSOvo;IMJJTgV+PK# z&RU(x-@}43o{JQkftiS)FaHQLXJC;v6euTycobxX`VmYdc=!w*-_+h@VQHL`c{8JZ zO;UNg23_Qthi&8ZLE3;<8}=a(*ej1t%sVEJ$sq+(88`lkW}2M~n{gw0wS`{Y0>X!l z&kEnO1A4$FbeBxn-`P*FQM>S(s@w}V7>T-}SYr`-90F41Mda=Tf?syCw1Q@PaX5U4 zanWAhBI8fadzr@0ZW2e06=)-FkUVUh?RlMrQ~m+=){QK*IcQN~l1))rhn34p^Cu_% z4mDaZ%rTQ%Z^!fR(W>%8cOP5in+#0T0H5II0s!!vGzCBpm6t2ZmlXkLiIqAroU_ax$Xw;}obv%5yIvyc}u`|LayqJNizNP`ggj zA`m)B*G`iJd+8BLb*5hs$x^W9kfwA3U$`uJb#~!AL>nevpEn1@0F$0O05I>8pU=`Q zZ^IgZx&eQ(iNmz%WjHRbBqGZ%uIl{Gk@4O0k?zYl{GSDDU$1T*q349bA^QO_yE z3lh166D8^Hor)oxvm-7?Wa=+^l6(Mz$qKYWEow3AZ7IhnqchC3) zy76scfg+qIN0f( zE0;W~yoWZKs=zjt1?8fN; diff --git a/openblade/simulator/__pycache__/library.cpython-310.pyc b/openblade/simulator/__pycache__/library.cpython-310.pyc deleted file mode 100644 index 24d063d4d501649f60beeec66049ce917760a41e..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 17578 zcmbt+d5|2}dEazTPtUQ}V6j;2F0R2r5<}r2c!Po|;-nw}3kHW6j06n_`v$-a=UBY% z1&Hw=h6vh{6TysZ$EC7J)vhDQW_(rR#IaL3l*CRdcm8pb>ZB|us+3Em_>Y9Hl)NN} z{C?l-Ik!P6m7SVz-retg@4MgYZH|p)V))zh(_gIq__H4>@%ltzqMj{e>+1^Z>XU`Z z`uf89`cz>`M&DT5P@gVL*Ebe6)@KSc_1VH~eXcN9-&ELC-(1*S-%{98-&)uzZD(rR z>bXL$zP+%$zN4_CzO%4X%4ci4>W>s2k@Q^c(fVVB$0WU}w!5A$xZ82^;b+9}k>bv!nudsUg2k zy?LX2)l{Zl3KOwT;s>8HC3g;YN)C1sxtB0jveNMUQo|3@x?HMNJioFKq^^~;x>?fY z)+SY1@IUYQdQYyGx`N00mo5d_;`x)OPtLzweC6z!6DLm{KNpOb-&m|w7Ak&Gm20J2 ztE191*Urs18|A>&Wq(OGf`pVVfU>%OaikLX9wp7afJ7SXgJV7t(1H0)3NsL<2 z%|;NvTK0nka8+3h61C<{3^K}QaXp5gU^IMOu~Dj*gJhU{FQ#WvyXx)t>x=ub;Y+H# z_lf;`b+hU3J+oMD9I2I5dH>UwpMU20r>~UuKJmo!2lqaG;Hl^K9xRuh+xyHx^~`}Q zPn4f|_TY2-y#-zJ7p^Uq)c%|P-sYmWzli}Z%K%=bzEmsuO})QXxvWe2*1pAC!TO>u zd(GO7a*>!ad$tj?@hBWpwqs3OSu17fO~_)k*`wD=jjLsSWXZdATTL6J(?4ISmzzs|Fn+Gq^pDpU{jlpg<_CLb&zBau_!18Ye8F_+f zIsps=XJERsWpAklCg7~3F7E`{g;Jwvn5?L{&|u3Aun0J_TwLhPrs(TZ!>e$NV76N= z&7-U8Twq@nng4Wi;q`Dm?}7K3l||(3VC+N*6Fb?cmKQMnV10<)rj~L+ImLzK&fi)r zqsxe7jy4-tDp&Q>*t-RM2mZ)zFOk)DAW@DLEEOx*5c5_!u55_iIK;Vwbb@KHlp7?9 z#7q#cKvE<P>Cs~G`ckPz7C>&I zSnLyf#iBll=?FjcfjO2v3-B_&9vLqK+RR!~wq|7x>R%Ee0`byWnX(G;w4IKpopgd9 zwV;$*ucqEgV4Y>PK}|!&B-9l(tLBh$)m61cZAB`nuBmM*hg3>cpp14Rl~z^thgs@c66NEnp`KRHAT^RI(1QdzaA4yqTBTBok7 zL+Zznnp9eSO1*^CdgZAj>L^lENF7thk=meq^=WkyscE&OKBGR1)JAnfol>Wfn!yU@ z)fuE_k$Oe_1*GQGO?6&fKx&hELtRt_q&BNt>htOgNNrIqbxC~@sjceA!S=5qwGDks z>M~L}mB5a-gGo|h$UR3&3$K?OsF`b1N>6YwhuUZ%iDc)G>8|>D1DUqfjum6R)y94zWw#TH zV}6{92f0KWYN_b@$>o&f(@13`HKHtKzZLUGRUGOlzA<*!J{FS^#*|}f+VQ(qST-fJ)xgR z;~_2zGKZ+d7n=3O7mKKVpUmXOGM4+GwSVny?<*E74XEq&itX=)vPg5Vhd&`Yx@d+cdS0tk=6 z9S_p}HJC68nmtIIJq0R+(89!D4;;yi6&H;9GKC3kX zaj8!Cd@^vna_x$l_K&RFA?Yi00ptx+rW zz>T{CN3GVTkJ*j^tW%Pw@I6YI*G8fptH$0wVDT45(@N}E>{8-3PQ>V&0Z zw_(D!?PAI6+U5QySf2=x=+(a?^Dhz)IpSy7yz2{;FV%~E% z;STU~5nqJz6cPw(1%MvposMXiyvWe;`r%T&R+~h3^fRn^=?+AFU+4l zb7B7c@nZqR2-T#i{Jiv9sFh1v?_%0HdF<42{RJtOCNQ!tNd*N;kdTpww;%yyr|e%t zGmmm91?=++oXZ}JT04Hwnu)->bPH0UYV%gpw32BPUs@o0cK)b&x-#TRWZJRinBWi+ zqzY4&vf*AF`;qwq%+=d*)I&!ZT`@%a-_<}Pi>1wZfTDuxN-t;(Xo=U7Rp z9(kgkYu2w52)hrdPa(XFS1v0<e6lHBhg zgXP8Eo;M`dwq|2kTD%J4g8e@2#}#DNH{dhA3&HoMw;Z94! z2zm!VFil}21}1bD37P__tb&Aah9+Ta5~JQju#;VqC+CajPMtX)Y$|rm$w~umO9ft< z7Y>kj&EEI6opcQy+SGNJs4YBVd{LKgRGLd($V7tzNjK4+noeYq;Q-qDS$xPBqu`5i z@WqZlYvm${WPlLYutAqPZr}kz9sByI1D={tM{F&1u%_zVMoa;M$VSh^vfD|ZjA@_+%Y!5ebA^&t%#~k2 zCoc=&#%$OAz_A>l)wvJD<%tNif?bTW*JrtOhZ;aH1vt_B@vDntg@|w6bIj`U)_abo z9yerhnJER3Pv}RPj;QJ^v%>npW$HU9`zF5LBmmZzv{$)4`F!X;05Kjx1tJTyl>;|4 zk;gjLjM0A~ZIgbCVRDhF*9b)MQdjD$0Pk5watux)NOy7evuNba0FZ7Ssz=vZbyKON z^XG2b`O{qwTAM!(<*QSo9(&Ou#I7z~8bJC0O}~gI2~AJJh7ju+xzKXCYei7(%o>V~ zuccTjV%bw}h*0bVZRRTE2J{FiR&tm57XRnV*r-3u(ARI;-m)`RG7h*iYIPcS=( zIoZBnN4W$|uzfbnx{qADbsz#uL`{+Yk{3mpcwX0bGF$gkYoA9)sx!#n>^zo&2JxDY zusKmGz1L%Pd~jpd>`MkiiY@tZ7Rj+bHh{)1>pk6o|*Q9Eh5MiPGewO)G3fc_6N53bw0- zV5X-FQ1 zIZX59y@VL^>9=2p#gXAiXq%$ZT^(h6#5Us9vEn$6aWdd?8C`6ScyR(pI;ITI3CprL zMJ8DCrmr zjuQ-$;s6`jsP9@BFCimrwAt8CDkfVXwvp35rXtoWq48f#(wqZBe16@Hj zufK{BQNjqC1VoKDu>;xA`CWK0^Y9AK&V zSj37%AT%^wOhOL&uQ4^G(J7<8i*MH;8(k%G#_@@il3iniZhuN^nV^oJ5dr zCm;DXLBwjf#`{@MJTZaBkE}T8K?9$rVauG2q5^XEU`8-^&2cr!Am`M_T=Y| z8;8ja;tMrT{~W90x^y$ZNq~fmjbR^U_I?9x{~f;GrvYYcTY@kQi6JEOK`LomlO`mV zNI{4~jKcaHw^JXDI~i-rvhUD2W$?^-VBg8aGk@%C&wj_*9%-9)@equ7%_Y51mT-un zoID;r5Z$0|CjLPpmUjh$6@Di~D2*SA)waEY4^1MvS5~a`O{5aOTea~`;+tBDLsX`h zGcb}yRuCY=@95iY`5g{yCL=OJkvQ>< z*t-Zxze%w>fw)KvqH#TipTD6xjZ;w9-`IAMo?(Q!Pl@X5Rks4xSLfSa#4f7{>(!i6n zVD!|P!^esz4xc=A;p}mQa!kD0FKHa~7OyllVlg;|A1}h!EbC5qI2Z|^a-~$M$?01# zarD!N=U+a4mM6)_&s;bkj0xv-vkuOq%k=NcAkw=`?IjR-_y|&g%at5IW<=fs>l6WF9mLgD9O^<(G_WR+eN2-nur9<~8Qw5a zb68yxZCqgkF*yg39*iw4X$@3Fhnr}xe1NaVu5@j|xgVt*dYIwo{%FF4p%^*I;ONz=sAEBPkAiQhreO;gp;GUS@XkPo`&E8zYP=*_@GZx9HV9oXs0+*nGtrVDi8I zBQg1xeh8Dl@K7dyxyR&Tn3UGgdVHtu+Frrm5DJF`ORCdrguuq-8L;lOKUA+QNT zVAim`zl9_DTdTN)(h&m8K^Uy}bJgwmoeG5ldO!$lN{Y#`IvbzN8v_`5m2e#%#nB3)Z&YZl1({EnmEsgU*s#DV8Yf@nTQT;8B z@yh_g_=Wiao}F3Cvt|tatL*b0!P5lvr1h^6(3RJ}K_Et*aH24wuw#!Cb8PYnf}Yv2 zomt_@9*YWh3Qz7~t?;RECmBa%qO#<>oIIe zP>JmXQG4f+Ab){#K9<`U^rgmia0u1mI^)Ec_$7I6)Uxkqzn|GO`9G#NwPqtY&Y(IbBI^RMW?P?rY2nPPgOSvJ57U~kd{*i>QD3s z+u&eqd84pMf*MiuB-jvoGW||GQYVJ%fz$DmYnVjpLY*jS(TOKo8#;OkC+<**)GAAw zw>(^9%LJ|wafXS6oybY&g9O_vD967?0{LA6(o25U&^{e!*hANvOEs#%j;OZ2#(eLu5)@=vGNN9qrvPvW@)es`P?#1~Vyyi-Vk zjAL)l3K7EJhEZqxJm7%IRt0fY;ayi<1jeIc@7wx`fLpKcVL%|HCv+Y~LC?kMi5kRN zL>r4#5P>kumICIG3evPuBvf1{o+eR7jP&W-4|jbOX)SjQlSiRG0bwl^a z?;Owl!z&{L2(URQzOws=kdWP9XLg^$j8`%!&?4fS8`tc15d(A<+~bxVC^fM(+bsAX z3x?LZp0n;+z{cdd!)4RX;wc^?O^-&1=!kBs8QkfLT5N(t;mgIoPrYSsoKz*KRJGsE z%d*3Rr%Q8idE*kcF=w^A;d>+yWw&3q_3n1uhBq*blCWHNQ7(l7_&#ml#f;i>*xz}@>Z=$C=um=xnPd*8kTr}Tm2 zWIuGA)?*Raj2dZxUiwj}{~4zhy%;etK4viwW%CU(1>y6))qEAJ(SLx)wRS&j*27(l zL$Dit2K2Xsf({LMe+L;Mc!*wVZ{y8E+{G#GL_69(b{$_A-{iIU4a8gg6zbERQ+qkD zj~CO8%_zmC2l&3vc3X^b473=7c1<@%y5-|x`8dmYWunN-6BE?E5fpSTn$fa-Vs#%@ z(pxdEgqm9?#j|Jn(ys=IV<*p^zc}nnIY%y>yJ#HC636*X0_row;d(+@9IPnGyaDkP zBbjk6vU|tn=gGXAF$ksfC?Xt;Z!j8yug&8f9L2vMxt)>24N&30CjK8hLpBjk4>vKq z4YZ9HsH-??hhemvDQGW?tP4}8b4Q5REUZHjNSAv>sc@vnBw}nvxmsJFu_|iUv$$Q})O|AhTikHE1axfa z?83aZvL7I@6;jJ+Q#s0TF1_Fq>L@+$^0!HrsG2LZgx;nD=V_-A6KM;As zx;`3n=5&&Fr$*Sspy#JB;;?SYQpuERwT^7{{w|kl6jO?dDQXeJeby`@jwgxQeIZ=L zvgLzJ;X91@j)K}nOiYd?6=UU=H;qz>l=;bO3N`6sh8iZHm~UI@f?;8uI;tdRbLU`4Mvl_4kPhA~;xo_RNtp=ZmM# z9Q|zZ=$ZKwr%oO{ugTi_RaPfU3)hz_x@>N;xv;nK3Ps-$R0K-JV*fp2kw-rC$jrC3 zJwm(>hNITtktU4bk=l-LRsc6(a6o4{L5V7cXsD)kp^XU{_oBsS5xR`%kXMd85TnEJ zZO)d0hcYaBo&({fz?JBlCny}4G$%~rflq6B)arih%dsnRo9-<*T|8O3%i|K>e8JsD zyj;ZlFH*h&x9?rV-`~VxFmJK1Jx}HgpW?vWpD%$qE~Rf^TNZ?8MYD+!r*A0Fl~Qjk7$wcG$>>KFf@g9!W1TrBcZKwrGD@ASCSzctsT*6uqT z@B$H|d?DAe_vF^#ium%lr0LF*TywZQv>5#}sMr6IV9>bK{~2ZfqBrd+zH$^VHE!vx zsKB{Hv(i8WMpWG&cDoDNa_!wq;v89yY!-rPZSw_f(V($#Xu@AxmMHg5C~y1s>qf7Mwp!kQp>22vt#34BZhdSIob%dXgN zBg3`4y}zTfNt;x&qOP}6$%cws)a|gH-LD*Z+e*%{D zA0g5D#E*oFF!)C9@BA@Mb^%r*Bxl$5lLJ;GQ+aog$}(~i0uf$+N+Nw{E^4`i}^nm#6(a%q?z%m7b3XvleL*F5-j3Z&+ad5Q?(gatl zocpaqQ8l$(|Bc{Hg8xo1IIHh+Z@!Ciyd)O)mg?rH<~25y zef~C6(nUJUYJ8Oa5&FrAJHk`8`_ZUF=zJJW?UAb)m^zICMtYBOf1kzJ6}%Q{grMA_ z*cje3?_9>-ipvFZHM{Cu<=qQ}Tp9j+*Jfb#dJ;X|WvkoM;k9dAIU=uoQhFlS?P>F} zcGzJkh#+~WF6cvd>>^P==gp^t?_QQVCzGL`~iz|7jot820bjNhtsVXmxC*MWb6$T*tOyHZnX2WCnI@+f&hp(?` zpEm8U7sM{e%(vunu42ER`ci+H&1Mg;Ub%u?0{ zZfOxObt*$>zsQoz=lGU|yb0KMG`N#JZV<~v{>Z+bcq_!(<6&hIriK1_q|GddTf7sn!UB1o?46`!A5%RqZ~E7j?^oqLO|)zivMwB=|5#@c!@_wP$)15V*Pj;8mk}w#5tb!hwS$>L9d^y z#b*0umi{h*?0J8EZ@oRtNWJXDn)QaIL$l$$^V{_ce8MLQ4guhWvZ87(;GNblvZzY1 zNYEspMW??^@Rtd`1`v$M+x~UDq106R8!V>RqbcAuZ5sU#3H}kmKOy*if`3l%uL=Gw z!M`W?Rf683&9|8S6#|+SntDj0RegLq%QDeYq@&aLdL;G~*cBWJ3*LN2|2#938ObCv z6M!?Bl)ZV>%IwMP&ODyUWipv`#>u4c$!69emBiP@Cz(l(k7%J!M_z*y^Z(+Nw1ltI51ls{3y5S|}GVVC-V3VB$y9gd3 zAS1}T8-{erZE|`NF^Xsm&zv0ZVLLet-iK7MsW;QEtG6!n(UoUDI(AXUFgh*&~>HXh+YyeVL(OWNwNWO8+g6@+<-OHj0>n)FJcU_lp)< uczoDKY73a diff --git a/openblade/simulator/__pycache__/ltfs_volume.cpython-310.pyc b/openblade/simulator/__pycache__/ltfs_volume.cpython-310.pyc deleted file mode 100644 index a0eb08bd6ba7ee0e1c1e4d17aea76c11898a0047..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 15715 zcmb7rd5|1enpb94R#sJ3b@icE>$Fv}B-`z=4tssgj>cnIQjg@3TBg;qag9CYQf0PO zr8?R#vu&%XhQ-p%&cJH57X-2Ef$BdXLb1WlF1sx3EJOek1S|vu3j;wUL@exZ%n(FC zggJKG9*f`ads%mDi0O{{>RtIV^L^j<`;PajH8J5D_H;$zl@UiMrEB6;q9LF)icmdZv*nW*TnM zm3p$CZRCo%#zb*K>P|i1m@H1lb59kgFh{CB-IyuPGz`>nS$xto{He}S|JbHkJnpByXn2#} z)NR9?x@FxmizlRI+M7YkjI=x@Ejzqfw9HD&)3|1*H;4M%Evxtwo^{!n-}M8o_`Dfq z%gtshEQi%rGgzFrqKsD#%awXL2%_Xl)vtSKOQX#Xs|`O&ldiX#K3eTp%i;BC;?*_f zzuF3_@3V>t6Q`Ot=abRg>9Pt{)w|{|w!($hTGKnLR7*t%7Fw!N4ww9Q)~d=6PPdvX zRW{)Mdce-h)y7&KH}o#zs&JuPt>X?+zxj1l4gGHO#BzDnUs$WxBtT;?EFqpUo_xpLF1`%y+J7dee9MSaEO zrBz?a^Dp_qT0O)w%;y7H@ymZIr!Jppf}O#8%5JIiq;iqlArLblh6`;+xTWZ ziEjtrsgs6pducBNIdSip(55U}bLibE5@#13KjqDNGv1EdHY6|Y%~Fn{JfDXvw&Yh@ z%8L>~sG>{=wv<*WAc<--jNG8Q?w79K4E-RQtX%giH-fcBDY#yK`q}5Aq;5}DTFuaJ zhGTD&z)PGa9^_C6jgA>wwZx`**|=ve&f6-9)+o_zy{8;0t%a3)hMGrN$kuzu!^Y}y zi1eE0A3bsWsA{#sqhMO|<$Brkk3Vzug=b%QW~F@e#EBO^d-R#7e&YF~pY_YnAAR<- z-m_1wobaE0?z7Jy4=P}B<@##bJN{mHw6z)>Z()L~GC`o_wWW^N!WQaP#afxn!l&J=-6;W@zjUD*)l|CEP6C9fg zTcujiYQBJqu+z3|(>BxicRpU98(EupaFkjt->kRFUWIGe{bQMSL|$_Ng*IfPZF=S& z-Y)QeZ#2dArCkB=5|9F6ZIrsIjmUpb@Oj_=U@oiK9px+K)pDg8-qZl6sDP2X z7C>bM+0>J0Sl`oE*@MMGQngN4Yx+G^FKj8-8SKsCpi006LqCD^Rc@3VJI<UJ&ZK__mr-RYz@jWf^!FY%#+ceWRYrsE4zYA*wHKiZ2z6&JmgdDGf3 zH>|eVwmQb9@r%|wa3qB9_e_zBdx^+8b|SI`_JLG12Lo~;0N|eFGE`cP)fZ`-KB5-y zHP)XTUGihG?gJlNm2y3J@mT-Dc}JbWWr|9#zDV*4$(KmZfy}2L*dHa|E!Wq485Sks zM>GQU46o~puShP1{0cszej1}}(=lgEXG^Q!pm=njRrGBPqiD2*FEtb%fKfoJj^)87 zZ^EbZlAd$h2outlsv2w~WRqWEQd%M7Yz-aa7p`o?edwXwry>i1Du|>L}|6kFXD>Pg2KG)|neejJE*ZeTDDP@WrROd-Vkup~nFJ8K`xP11ET4v)Ht7f_)gz`O0Ye9#g|TD{M%mHd)PaG@oDd>cgQ<@+lKwBc#nFIq2zj= zchozEQr5HZfBZws^Sz(+K6l%JUF){2cqhHjqh-Ro=Dp~B0i*Mn_qun+JBxWIy{fn1 zeG#owUd=o2y@JxTcfv(|gT(9i^TAoY(T+@QU)h zyCjBOjV7rJjF{p;M4iPtEJmri4o}oWsA>@x&gQgx3%_!udTncFAh;>C3yX*n2~t%0e6GPmLMmdtiyH55XWpUXoM|Igdgp)#yoAFHU#n+@<|hHJlj)v#1rM@>M&NjU%q2kxbV z|C~8%+V{1M8({3Q?qzyXadolG#*np3pn0cPclB6z44iE!f`mRu69l*q<> zLUW6piQFbbr8)xRF)u0WHv=R+L=yfy#s<4VT(Bt(Hs$ee;omlMpQi1@f3&`DL<^4% zTrHu%kkL0WRgX(F;aE%K?q1=FgoxCUo~MWysplyswps1M*lPa>UsBOBY3#j-(!GSb z#0FUnn^Hf3%L2)?M&INRcqqpDXGY)iNBM{et&G%$C@C5RRS^y?a6-0=)2bOJp2CA1 z!M}vD60D|88zIb$Y5(x>eI3XTs!04a;stKa_Pu=u=s+$6>lJE9_m60m!AG>^w^4w$ zSfP2#QZ8|dj#m@z*G8fZEeh>+q9gvh_WL?1sUr)rX?pgZ#0C(pmcDx&60g0-jP!TC zM_2Q(ZI`k@x169`ej|x%vYm-`GRy- zFsC(rI&!N)HBSBMh|XpzB1$r~5a`JLPjWZ`Tzgm|FXEGlfQW8Wt&z|2!#G>h*KW0n zE=bwuczuP00eF;9<@XpoN~&%igdAF5hh6m&pqUWSH@ z=h;~`ZzUk7?&A8>dLcp>-GYh}(1mid;J?3GuOfgij4zY~^UGsm%w^iaC3*w}x(#nl z#8Hz~<02S05=tWFf!GOA@)?YV4It`88SVYT&yA8|;?aK-9V3$eduR%1MHo!wO#4&E zT7P1U^OAJiE|gl&_p%8ELD<{8#VaK8FmtNSGVP1Xf<$Qq$W`Y1)q89c4LHY+WTozx z)j+&G5}Z)@b;L!{k9Mw9oA7YOmA{J2NSuJ*3B`$92htjmQ*T|n=BvoAHu1caO8t0t z-(IQ@*!mU;VMF~i2>~K<^eg@j%A!GTQq1FhH{puGq+dhBnYP?d_kWyo)9~l>fN`B~ z9E5N^k@^a!X7C|#%8-5#v3vUA!Ip+#xA-`~>Db3fP#)u2vK`l$a|zS6%qG%ap&M(G z0sPH<-GE#JzaiJi+=Ua=oL@nX{+kKWHlvf55lKq?-Brw-tW5=H!fDTzaY;Q+bj$Hl zqFXba9a^_GoVKH`!>-KMcHXdpx5GJ|;cq+4CfqXBbv%on$BpmDvAb?qYBk(l+tai_ z_im)xPVi@N|7>R;Jm-|WU#guf?GGOc9}W+63Y*ML$V#B!3J-eOTjtk|eFy~aBsbFG zBkeSv`VjlPi97JUYcuV1?J)ef^fo@luMXx;tmu8Ab+@W)+C&zE`L=3*~}{;8t=O?nPmJccHNsgoUeqp|2L$e+t4O4Y$DLo`&y5HP9p z^>gUdPbyj^3Y9r|&2It})rx^)2YKHmVy4yf} zb5!0CyA5CEjn9yXPV?y^j9{vh2}ER3-W(q>f28lPE4;m-LS8o^d7aYo`HN>SU0IH% z&n%sP?QH4F;;Gk8oxgDE9-&>SVo&6`n17WdNPg*SB8$&GVPrKL;QH`Mv3($edSe!WiX zFc}g%Bx`Fl2c*xa6q}iL(5uAtLZVBeJy|y4=ez8e1*GkWCgYe9(k;P5lncv>fuqhI z%Zdu0Z}U1bTTx!Wi%;+>h>k88XXNmiB4`3Ud5wLzwkF39|D?a7FfB2K=*k7Na@LAJe=x>eHm+7x@$y z1y6&dB_zNG1irian1epfK<{kosb&4t{{JyEL*275!v5iSgc{EoIYDbpReV7hq{@AS zrqn$(4N{>1b^lm)@HtIacLoJ`HO#6(S32gVu?fFgc*&y*Iu)z8@?8SI-uxPTd-ZFW zLVb%w1g$4+k*lxfJ{Zo{=*@CCXp(CmbkO?4U~cJq>#geP*p~+C#g69IwCK$_$uR5a zduzwgQ5D^&cW@7LCQi5I;#jlK1v)4;VNY|DDRfC4jM z64OsY?+SzU37WbO5nh8Yq@WVin>w>SCh;vrfqWVo-4?Ee$m(G{Zmh|h3)^#@* zqn+JWKP>a?PQ2$xG43V@e~VMdvL9iow{9a`P`{4eeO!K8BsZghTnm)*@m$I_$5TxM zTpwDANz6c*l9l*A=A`7%GvU5g4ZE#GzPK$p(gx%P))T+QV@4=-sP8d%vc1vfMha!D z+eSL}o}Wj)7G5**CiX@KD?V}e7Z{1Qol+h(#E{{1Z3Z6rW!MR{g*!ZWvo#y`BleM6)1&12o$#C;?T^>Tu@>*8fpnB{u4@c0PQ( zyYi6N`F0xgiP~qvCp$;OW3}U(+;`;piQMA)!P9bhpdi9g2wR0R!tJtmwAHM~-r(|r zx(n&pnxZ*tt%ZfwN+G}|x#Mtd{*j*lzls!YwOo&9kNXSd6@;vXb*G@W+#zws zJzEvhxKamp1)OT_5&s4xV7goK{Y*P)U|2hK6$-Ax2P;;ESv- z@PgF!a&R4cH!u+TWL5^}SG?*q8i4twZ$_6+a1Mr{b5QCSLEDWpJ%>mAT$ng9wX^1kMr+BkVw%Mp!#c33?{sdjd6qxY*;m`x36QYi=jo zP6DS=?Udlu(?F9Pa7wVYHqlN4rySr^26$uF^1!L{;UtjKu1x{&;@m|zjl8lGA{!wX zHM*s$K4-@!H|sALCVI|Astf$p9o5B@-v8q5cluR{dR&`JH{;RDYkXQzXkI z|A0h-H}UBHA*;P?)^_4A+N7~Sg#Gt;#XlkWry%q0Ks-j-S zEw=V<U3AaJh9%i29O%$y=wMdoLI(pl1B=$ zbGlphBe0)GzGI&OC$o{WJ%Joh#zJ0VH0Sfq?m>GL%fQ$^Uia>ytM3eXmdr2ObGilh zV(e$%-6JxL`=cHHLGKw1b=@eS+vE2%&(d11?ybm0m>)>}ppgM+4u z2mWG1Qfl3ZY%UstN%BkmE07_~`L}HCM~vTLwVyERNkg=^cixyG#L$U&eTdf4cg9x0 z4IFI|Ph1TH5WI?R`l?=CEp<&7&Izmx)+Kq46L7tdb0FyIOW5<>w;7Q&Mx=<`=pX$o zC5}Kyn3{sbbwB$*nMyKG#3&SLfw66<-VAC7t{%#s4Q8P?0*7j3`k&a7WFLZur$>FFpEU>XMery1<;N1@A{4Qxwu zu@S)H;Qemn7Jop302TM=zCjPjxQ;Le#@&Muq8}@X*tWJ2jmO4BBeAIdD(<2F8c6qS z!73XB|G&yoiNs}peDFhP}6m&P&W0{9P*~S6bF1UzXIi^wyaWF-j zDo5LvA^qb5CGim!KaBzx2)5Z!@bZ|CsPV`rSiNbfUC<-ef%L31_T?Cz@3GYlTpi`Q zh{TglYA>6nnQiv@Q=d`;=kdzd{O-GdAKmH)BqQ%0pzYgycdit(S=^k%A*8YM9zWuI zj+_r15@!lVPn=JnqTf#cc0}QXk-V?ob(`XOHuUos0Zt8Xyh#yHEG{qLI3Z&I^~)p@ zf=!=3clPv`E?>D=x_JKb#Z$|t&&gV-|Hcs^2#ke*?{=?B<^Ox#SIvv_Zv#lTVRCn6 zhp?SEFqHAdRQ;EH9F;b7s4cAI4z`$Ry!*tmo?5O->pdxH{WW%_NWMiv+|eT$2IE}WZ?WNTl1Tq=vvh>yFv%k%e~aW0 z$=@cCSNI*2aJ2x1+~>!-TJ9Ymf0wnBtEJSxx7bC zrOYXBE;GgJ*?(f0slOgHbL0G^42-Yxm)Yv5@8e5!N#jLd3goHn(bn^NF0qpRu`D6) z|03u50?A7xisU_#=SW^AIZ5)fBwr(u!$yCDrC%rcHp%aj&=FVviR3?%{8y6ikPyMd zumcfcR;!ixmoeXCdmrtjMEj>Gs5I{ZZ=E(QThaj^yVLH3l}7ZpFZV-tpZkQn7bl6$ z{2yC;N!`!oX5A_LPrJ68avdv$bIF`({u%z1XJrjP4Z}})l!YekJZ`X5C)osvvc2DS z`hiNY*(OPnI2-`~H)H}x%QaaTIj|_)pUAM1K_Y0a65_B%w5#_6xMN<6n;6Fg^aH(V zn&p(To61a-j;tqH`U6fgel5><>N4C|&C_F4O_Tg`kf{&|svP6LNV-N=!FakLaKwQiLBvKfn7*ZI688n%yctcW)OB4zcld}`k zQ}z5b8E>)2$EV~c$H%W^C}IMt1rxs%^^5ZJOZ5Erz3OMu#n_2c6+^D;}~ afgM7`B}>N!#>;F+dY1I8i~GJ3?rz^ngI!#0EmSL}~498A%*$cWhO6+OGTp zK-!M{C0{x57dY`c-NG!{&inm7@8j@Jr-fks_|Q8V5rlpl;r6mY*n}+?a8N{Xh9X?u zB)15{D7T{qURj*T0Sn38+>5+xT{CKe&dOSOJ8I{C*U?2ixF#H_O1|ZFmInw zhq@Q`Iga|QdDNpGZC)Ud?ohmo2CY9(A58{W+dGLm91z`|36>}p&v>EOiP9axBvUsc zIQF(nR%~SnWg}kfrw7_oc^Q`wJ;Ns1^|IhLC`Yj66F3y0Tmp`V=sWsJsPzq>VTG6Y z9G?+ImgF2`v?S1zJz1>FFIlSC(63dPN5TrPUE@A29{_*eqkyK zQxX&oT-pH)ra|`_%P%AhD}Nt_f^!wtI6}pwlA-Z7U&HcLx0ABG!E9or2gf?`u!n6d zd=O@^>t%q~VDE#V&=I~Q6jO3V&VZ;TO3k?KiGHxXGoEa{d-Zx8?~dPpd@~xyd)sfv zlf|CGGmZmG6E3yO zisMx9LR$w+!LS-tmTCa2qYZ89dvX;TSJwED!El9K1Cum(x`0?1X|Sff={(I;T1f40 zmZxzoZN%kiHMT@TGL`aTDA-p4^yy};((a9}1;NHWPK4;17F_dmvUDooKLO5GR(-dg z#&8N`NhOl;i%Qvd(} diff --git a/tests/e2e/__pycache__/__init__.cpython-310.pyc b/tests/e2e/__pycache__/__init__.cpython-310.pyc deleted file mode 100644 index 2079408a3a83594bf0d7f3dbc63d0e33bacc774e..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 130 zcmd1j<>g`kf{&|svOx4>5P=LBfgA@QE@lA|DGb33nv8xc8Hzx{2;!Hjeo=mYiN1e9 zYMxV0VoIuhNosLPv3{yis(yTYW?p7Ve7s&k28g}(ekSa0LUt1ZNSly^4_Vo?+EZ)JPEAR*<3l$(Vje(!b9xX0Q}B5Y0d ztFKqDUe$Z=_sT1m3mX0=e=7B?s;2#mD#PflTer_|zblNrh`OSR4uvy@`;THR)%@WfYx7@F6R+!GZ)qZWW)~|2YQO`NK?UXP@ z{(WP!K{Zhj`nDm8qJ)-aqNOYw??2Qxikvg7UaB(e7&gb@W%2G128KplU=7 zbo3LQo#?`xjr<9Z#7goO_kzMmoYi5vrHxFSR8g5-9dtgZV?;ycC)mNya90d%l!Y`T ze^Zo%F-hqfOTM7eD$`qx>&Cbx3{i?}OXFoxp6H_jMlOw4RH1hY`{bD_y`93DR&E{@ z{5x(paGqORTRSLpqQhAgTMzUwE4HkmbR{YNfqs;_dbO<|tUmel-u_10FT5KQXRe+h z>TKIz4Q#pN1clL=t5MtewjUaMu005ITej?Y!fB_&G{#b~LB;u_itYQ344B77j7Q_e zk7>m)ySM8Fp5Hz#GpNcm6=X?a+7D!ybGuv8miuA8JGko%0uN-ycfG)uH5yde_GI4< zEHJ;@?Kojh$bAdfEX+nyhJ_dQ_5;TcEL(i?; z*7Y#L50@=}*BiQmhgv(Xw`IH5!0tO?-tz5jhd6|lZ}XPA-ryxEJsCEwSQh6UQidI% zCp28o7GVyiv)z^bFcWyU9SAoZ#O={Rgw5E*?t1R9?^rw18}5atZ6T~!qgLQqbDgQK zXcuLkbdY84JG=@RC~HQ-Lus;2{YBbir2q9~TnaNR!UIiS1c#%GYtr+A zwV9c(kw*PBXTxF2>U*8rP;h6rd)I-^d>oUry0;%zt$Ei$;`+-d7IYK;Exo26>J^j~ zy@b-#pGBCBQ%E1Ac%Jgpq*>Mi0Z^8j^J39a~fZJdPa1H$h zm4R7b2UR4B%$jL90RHCS2qH7jvzh1F%nLyYy=7HMnmJ6}9OZ(FN_4HUGr!1VvE zB~|0|Y^X-BNlyD7bO#6C95q!F^nx+DB>p$nptGmobeej2Pmd*}`z1*iV0B3M zYm$g)?CCM=D!#$sGoeu`+W8-76w$MtBZ!D3&kqwk|2)Stsh7Vn&qyzSW1c7UBEH1> zSQcLvkE> zdGh406F5SaQAg-{<}*T<9{|W4N3IKqM8&D!0wQUrWgSdpgE%8ZT{uX_txHkdx`d!Z zs3t-(T`b(f@CW)qgBD(I`-!;tAVqA}Ws|rrP_am?bFq&+s;@=SZ*8{8uo(9dLc(s| zBc_P3Q6L5p2jWM>W*{|TQ4lDMo^Bs+bRBU!;y_f);_|c&MBIeXyn2E~XvPAG(DYn|bqLqx=x z@j!@*fzpINr{dfo-9m_S^7*N%<@Z1x%}IlmizkmjUZ9l9|Pk72b zKT~KBti{c@O%uy=CK*<@uy zo+G~JiA!ZFWQ3#Scc|x;8UN^bFC}L`o2*=jkX0nc`ubdL4tG4u2qP*k5&Op(t!)nh z)TXB}6{WM^aUFsWJ}JD!7{b&GKo7sr7xW(2i8l3*^iTDQCLIi$LlG4;%us~O3B{vP zgaCE`%+4I9BP21=ZfLiPKpemh1jkQn<1|5kUAwc4*o~9}9y$WvNYD^hkygZU6*z=v zt9Z8F%VM;tGL*)s9E0^7aU>+t%Z>6XkGNJ)1(ih%03z`JNM2mzRS^+@+c^RhgTlDT z=%k1K5;*D7163vpejP|+HsW0#LHOl89ueV}64OXP_zQ$0%Ci~2;ThPW{4Mu_0#78A zhZ)4Tl0YJLu1PcY#>d$ z-?!W8gN#j1{vfjzeLD1EA1`0Rqyuy6DyM$_O>!6T0+C+>5qpvo&XS!4IlVPbKH#6* zCtcjsVZ+p069lH6L~vwFUQ`FJ&U|aZR|~I>`mmWG(Q7R()Z)RRQ!mFFX%qnm19Q`>X!U5=7wf0l`p%*sz(J0EpJgl9zfbuSX2lqwx}Q+pRlrL z;+!ag?EN9~A4E|r3T(lyGnm47%dtDVCY@GJz?o+~V1UW)V!1e~E_DGheG!kK^^&V1sq8agX zA`c6G0yaoVHqELM{s;2T_3B)25SLzaVlcim$3T|7CJ=gdl7>yvj7614KiT|suD1km z;;0~i_uto4;RbS&QBf3k^ifF_RVl{qlYo-+5@Yu=wKI07WW5N>qc1Q#>?*?UWLHz{ zehR&(RT8@wVN+#TAK|3fqU5)rYk7qV0!5PEsSXO#;lvhMzAU2QC>LAM+$p8dLpk>w z_K>O)a8prQQmiqpxTceTiZ!9 zquoSr2?kLdeZL7REK@$+v1K5;V#kqhQd`j(_(O@X4>7_FUw?UV z+CN)iyr!S<74z7lT|0ITQO3{i0nn09Jv4Yrc`Er~P5QjwR)@;?cFgS81 z>{e#D;N*dS`Lj=!+`a#G6p`boz-cVQX_WZCp=b4!Zs5P98~<%&Yxv&P<aH+F!8=635J@m%L%E6(Xk`hf}20={jpEx=hplg$63{f0h<%8TF`f(88H(QIYqd&Wz zQFwT$wNmc&P(!Ue+EBAJhs~2P9X<8=0U_Mu=y>ABNAFz`E_P`$-}n?Db&ht}b$t0n z?```FZg`kf{&|svOx4>5P=LBfgA@QE@lA|DGb33nv8xc8Hzx{2;!Hzeo=mYiN1e9 zYMxV0VoIuhNosLPv3^=&X-fSVT!;AX*{6sN$=;f~Yd47ho4XT>>i3qrjSDZllQG*DLjjL~e`uHn|~ zrn6F3{n$0b-Sl(UTGG5`JGHyi#I$@>nucrYRlDh$cU|-pu5OvlFIk2#7wzVHZKIs> zCl@!(>Mf_$Sgtt@!>w*ENoh;J^krGQZC$bKm^XpG*AzGyuJb*bX!zV(lo} zzv<8*wD zSV0cp_}M^!EU^tgb?wsCxw-i(=UC+r-=4LR5-3DH82O#Z@2XIY(pFu?Q|~M7sH=L>dbAVUiPYmFb|6wupcjHi zUs9ll6_uiPS5;nAPotggXkN_Cc(G7jbVpXi0@czuu3Rt^T%Nf?rzlc8N*hY97dm5J zyk7G78?d_c!ALvdB|76fIETE1&<;jA6C$&Nlgf+26cTI-*_C?}&W9FY>J_7A2i$A3 zv$OZKs^J=zy)h%!03->qrnjWUx|c^G0{JqLS3uy}nvXME#x|*GX06tI zM6wIprthcClmmJCX|#i1(Pk| zKM%9jV8BK6rMM;E!4yA6K+}T?^gykVND3F(<)aE1H{vQC)s5C;JBmd^!PkIjtX3Ckq2<5vp&a%tj zTzN`*EGW55b5@A3@LnhGDv=R;JHnz1++`2pUBYb+L7@suOFYR&dm>y#=a2}a=;&IT z^EEU^t?e5SV4ILI-buJg2n!Lcr*u-D>Za@39^yLsl(^ItSOT4=kPD$Vo7R9_a?4k5 zuB`TqsxOQ*Cc|kdK$8XH)7O_^eOFhmykCy`iY^&QIGPv{)BzS0-#H?u8+eYB1yRW- zpJZc2GB5JAq2P&dtt*>H_t4lyHjoS-*T8BwQBUruI2HH|0eZ(fDflRS&c!L<0;4}G zL3U~<3!IL-IYDOvymJ?GJ2NUjc&faK709zh&VfA6&JG}QIlsG9-@AT!b*U%S_vmI8 zqQ*kxQMpLHeSv4tgwHcPuGcJ6Qc&?A5kTboG`A;lKe1^vg=PB5a4Dm<8mJ(A){vin z0}pyc0Y#vKd^AO>P$x(c0^LK3$f1Mm04r6TO8`hx1VysnSLm3AiFVb1W14|7P(uud zC8;5fl~2^{64YG|eam#2o5kifC4)-&y-m!#OMo$^xkY&-2Gy!36Q zt$CXBLpRsSi=?NmD*`JneFZN8;7+^6&e%@mcEl0rD|zaU0uj?FxF(Gg4+j%k zZ{{}7gbB%yc@!LGYRm)j`4jjhGM^`=BJGUGikz3JDd4m2I7TPD?2d9@`9z_kpq;>_ z;Y#94wbOI}l=CZ+qR|J6Kh@WgZmhee3}e4xtCJsKc3F9xK)yCzYnGLJ^4RgM?fJ4Z z9rR3dbFS=YA9cCuH@Wj%w+~F%b-LRHChR&{c4o|`gFH-kkv3`V9?|+libca+=5)BIo=V2Id|Y3~Qx2f|foA^}Isr*xYXJAiqTe zo^D1 zY6oGus}U^2W25l&1Uwvmem?>~kAsJogU@(rFLO(6XT7ZR8}PC`LjEj#K1u!L^CIOU zglAuWALC=>|MHvQO2ol~Q_Ke*T=K|+)4e++-vPXmcX$@K5FJXM#y(&2aFhs*eLg4P z^9vzgn24>jC+$TgQ-vk_|6{g6|naSTt$)r zt_-d*T&4CnW8lQ-(aIoqFB>KvobfzR9;1vxy=@IW23nQ_2J6Eg~awU=h%%aEcFsGxvth2AoRVeXe|9;B4pz z?qiFWmKNW=adTN;zI~9r167zK0nwvDckgoAJg1{bKOZT$Km;Uz4}(KT$Y3-tzm8E!G3Z`_^U78I;`OB^ z{l>Kmi%Yv|x*su9c-jvUq(RM*T;v#U5J%v;G*QYxbq$9gr^~)NlBvStr3)+XEM3=EFE1}$y}8Oc`ap=!2_|LdyC#?i zLuDAv4Mc%qJ|3?22Xxa{iTpmuo-1_7^@3y^0lLa5c(1^TDuvQbwZT)N6L}LAIy*5J z$6Z*-K>0K1-z7hq9SY5DyH_K?#~Gsk&j2b#8ZSm!USPR)V($LAk{FAs+N74}t_d_*c1-@rUU8e-&Orpe(sZ za{iR?-{Cf@KIZr_>coSTnIK+ZU=CLaNgv;|Ml*ymS zf%y~hu}E>0tBJHgl*$oa#w}nJvFbpC7eh1`$3+<~Ww&XpQK^sd`b$&+;37Wa_4;@_ zSBtdsUe3#72Z-&RpW|t?gA{g@*R}l{}bO4G~lZMX_PU9l@7XL)kV=!v`|+URaAlfLx-l zO~_xR1$~#uuMz15V;p784=3ix`nmfgLelTja?f|eP%vS@dA8dHChYp+JD}H2SpGor z9!8^xSO+Q#y?tQ+T=)knj_iXXOA52k5;@Ot^>84GMaf^sr1At;_=7@cA4M`Gj5p{Z zNwfK3%WRqQEoy0f(SFewF{p!lNDq`C<*^_lB>4w#TljUwZ&LHOh|p=o@!n9hHWx&H zT$%`@wc(V1s7`QZFiborf14gjVd7AHcX~AZ2_SH|EFf$y3b=a1khj>Tx-U}1kMTf; zz$pM=5dip;=+6}Scjz60KsuIsWx%#`cNdzq>H8+z==M93oK-x3))TwmWKa|lY9-8XZ? zwifn^pWu01e^M1nBeY*+<790A7?ppI5k!&vR$-F(MDRv?8a%(nkjx#X<`EE#FAyIk za-2w!2#=oxhi5T=iTK#S+bg`h(NA?Ng`6tlHF;2GoC%UH`tgG_ zxy0tDdtH7q>76To*sLTgWxhGCz70x zamphpbmzd8Lox;>HkSdzWD`Avd4B2*?5|~+Zz*UwuYqJEM}TN4HLGTo<4RU5a$2B8 d)fC><{il}KN?Jg`kf{&|svOx4>5P=LBfgA@QE@lA|DGb33nv8xc8Hzx{2;!Hneo=mYiN1e9 zYMxV0VoIuhNosLPv3_P=NosmgVo7Fxo_>6MW?p7Ve7s&k5ksee(Sf#fKi$TWgD?NWEoAu_gH#n@(!yQ6}zs$vdQ0T5{KShO$J@ zHjd>y>cp**0BKUo3Dr1mg%oL#A8nAhXab~d9(|-JP$bhNKUyGw5uinp{&3VL==YtO zUCvUZEz2^30wytM?%aFl-gD16_uO;OJ+tMOmTU}v|B-Jy{VyA1v47`7^e>5r+i^LQ zMl7ac$|%Jq47r=LaWiJbX03^Y!4k8SoJ~!nByE(^6KUkdOPSg1L{{>wQqyenM6;w5 zrIy*&iPqV+iMHAHiFV0LmO5rTCpwLoonGv^98)PfrP6kG+EN*necHNUOmwTws@YzH z+#Y*vUkoKJs`cr_L@#pIPMfMtwLfi5^r@_!MNYrHmbp@|Lv^BLV1<${)s2#MQnJQg zD=j5ek6Me|^O%05w^n=a1yX?M1gub_QDscm;&&rN~5^*EUj~+_($=0DEhEA?rL=D!TUJk&%&wwyAG;Xm1xy1k_8|xu1|A>3rV)o9P1_yoaD)vY4#iGUnR|j&3OgRKC_+0g58Yv2}%Lm8BAccXbP1}>pn6<5|JQzf1?g>y#LRLN@GGrjm3vub&kbFXVI z##PF*rVW*5{`P zRFi7<5=8^)q?<-*#!FrX0iJ@WS+S~#%R>Lg>Fi>YYPoDwQz}+XdnqqH58OP{c^2=) zRjX$LpEl+TKJ9gUG|H1+3iv!OeIo^Y+9jO^9vzoV^qo{m=9*Gc-L==x|T6L>6 zUW&S)$a&v0zR*5Wm@SRWJGOR4Fy)JLexjgeaZ6VUj`Mg~E6A%gSD3Xm)|uE9D7e#3 zdG6(J%fnUk=+P_2!k%qAE9b{@PWFi~F?e*4sogne7=rK4J5X}U&Q=GH1_k?aj-Q6Q ztCZ&)`%2u8qb|W1!M>}l=-SqKxEzGkY4^1{(TSmLyM<%V&xHSB3r8+1DpBv?2%6Y) zK(^87!`|q_p5P&9cz^WapdMY;%2>2`Z)C=XlU1Z7Uzm4K&NnOwC5pcY`QiG5>qrz zx1sOj3ta$HR6(`B2DS6vy4vxK=djQ|AIAd0H6{+_gQBKz?$|vC^ocrDQ zqNR|+WKS|*CVR?FEG9+y453`qj2=S@>PaShO43j|adDQF+R+B`CabyFABjO~nEZ5s(hcSPy)LwbG=+9(wYpWXug zXfFJEI|2EqX$0wA1VaFRLd4^0f3(f8LC6tSpIK-M@>@R$fb?dj$N4I)8dk`)=>5#3 zNa+Iv2MG=l945Gp;0VD{f@1`4BapTzQMG9+9SNGpSU~IM>e9f?cDVlH`I5q zxviNPb?eVfD8w>ctz7FS0&6NQFoQq@o2^ z8>nHZwF9wgr`O5#&WL%Psu#~)Ue{ua>bnf5jjY%GZmZWxD^m?%qs!UQ?RA5qmReU` z14^$6Y)y>iC%}zzLu+$AON62uBBT8)8Ev=Fy0yzk2P|TXZE7=Csa~&FFl%kV%ye#D5wk6I%uJ_QFzW?oa9Id8NCUGiZxFNI z#rDgwYM7X~LMuj?K%c%3;A*MHzI)iEt`|`R@cS1_mz>3)5Z8+Uk-301y3i%QbcTP!}yRAU^`D?-5h(-!593)dUY?T?KK^rHWYKRy4c(SgI zqj*iAMh&n0s(8IIZ5%t@rtb7ib(gvutNcA!<#%{HR;1%Uh4?nk!h0ih{EFbaL+JPy zf-lmb6&znvm`_d)))skdKBfVut7o@#~U7i#^w{!AehVV?Br4ohyCP zAr~$6py=s$up~Mgq~F)Q(#t;co#XIToh_Hcm1laSLC` zE+dwn;gINN)TaoF0798DfqMlWXe1uD_axpSKk1a`^^|=zQ5bg5xeJ})rg{*foe}yM9lwj|yrk<5>)&Lx=Ahc7E-gQ$Zt!xwVZW(Pq=K7EK)M;MV3C6Q_lV!82%Z6`4M_A{AX_}OYYiK$6wa5* z1*OMWe^pnqH={2|Y%4BgF6#o;DS z`=?mV;whk?3dJFp3w*#C*6bn>SG_p$#fP8gJCQTF1JQ>=fnO~=ts8528g1~o?QMA4 z-%xW;!&75fqv!5mw3V^ldY%Y>h`=MD>n(EVP{pI43K8iGgD{K2Q*W!$9#{#$6j#al~J4457PTg`{j#oH&)ve+P={nsJaW8gOH=>V? zp=-o+z8UF#7fYg>K{Zt0o|?|*P&05Zn)(Z}O$^+NbZ1=aUWBt%4U2nmP>rbF-k`Wl zHia$|I3HJWFK!GR)S-K8Ijw#f!S?3TYHS7G#O3Iu=1#l0{fX;WHI6Ut%)V*k+biSS z7s?)81Opf2&KIS>{o)EZaMNV(pvYaH!d>md1$QwHZMNfBp=+@N?vhS*SR5qL1Bkfu zvztG5aZB7bt|w_#4H8Td&<~*%frQT3OtleA6PzI^5zG>Zp}WE+UeB?V*0e4YI0P<% zSej>%TIg2==hS2w8=gU0(oSS+s~>08a|BNiNc@giraSq#i+~$z#46Ss8IeYU&^H+6 zWnGxD{j40%-^HW)AmsG7`RWGI_8sEQu+_t|wd#<(6G1pZkkunuOrI!J3R6Y*e9#k2Qfzsy8pp8j>8Nd0k4dm1 zwlIFUg_r&u{V`Tf$W2&|5z@cU(SL?u+5YFV%=-i4 zwD~%AJ^DFTzeMmn!RH7*Pq0XEncxozzCiE-!50Z$eN(c7RbC|cBZ9jL{+Qqjfowd! z#MGk%e?qWq=df9J4t4p-i~QspHV+r@n)4HY|3{k#qhh{fRKoua*c{OC=jMQ$g6LJ- z5Olwyzm6d`YzNjOqX}0Wm$Mxy#&0Y0S;Q70n$g60WgIaW(HO>ehtWLrH%q>FaF?Qa zwlkW?WsK3SjM!lc-rIyq)}xRPum(>T^bytwI zK~zS4;jBeD#!hDu5rPN{hGZkI!eRu7hYwERJ7>&66CyTRtIb}M*NkPq#cOfChMwXo z?zLbfZOoUEEQMZw4CP6QtGFmQ1#uPi&}+#_C44vB(bR79)WrutO>(=oSQ z4O}qzJ;wz|m&8S6)jFK7cOWhT6jSRJKHIs32pjMo@e|;ES`A{+M|=->k62gG=1U=M zGT{CumI(JZ*Q69RJ}%VZNwT3oA=C*>z0HzlEC@NxQ-UoL9nr0}s%>7kaCl9~VVrH& zB~>^KEMkkD3MX0B9sf}`D`=NoYs73v1hXHaM$*HH8!-EcV1YDd*$zqHIA%Tk zjH|lVTdQ`$c@tI;}jOy?A8 zCc|inSz;BQvyQ5}n>_coS9u|J%di07%#OyMEq zQS@&$i!r}#Mf!e4`o5|34YUDzyq)nD>W=5J0&-ijf@ZKBM(SRx8IISbI_~@wHL!_> zRLAwtgbGMQs*g+h#x=u$Xohv_ggS|D>ef;Bf=|P0!wmK5l8&&PS`6|{ly`L}wSL%Z z+U@qz=$%^Jaz4Nl^7{BKXPV=88PR6xGaNJ182{Z3AQE; z9>2P*$InN4eBllBc;Bi${@1b|e-P>M$2Z>NM`F<8Z%41`6)2l#h+SGpgwm4(+a*#=KV3kJq`Ka2^qyu*M=nWKwWK8r&XVvNkU!oFKlO z(JToaM3^IuT7kt$E%CZEy=Ys z$gw96O`pQsetHsT82Cue!55r4HuogK9)cNwh3GeNIHe9wnOEJ-U++eqyRL=QWN~JA zvUu&+#Ihkk4W}32g9tr+B*l*oRIDfVZUm=3hHqdIRByhFbZlI9TsV>JL0vyloKyC> z<(sg~kr{gqpJnM|+=Dn4fca!kY~ikkV;wmbb@KL{s+94WFpiOeaPr`|bQWi%{0cVm z@oTNoV(fG;y#(Q5(1^TXE5CUXAL5x+e-SR4AUQ6!>;PqYGQlUA2Wo2JnIfK9R8?%2ZJZ%oSSpIBdmb8VXZ zd>uNHX_Beotc){eI92%FQe2p%UqZcLzUc2Exg7CtAn%8`ejZGa{@B8<;HSbl4U|ggO0>u=Y@aMz6>I(XC`I+ZWzBB_yar}I>-_`3Yqt1pR4T?|?XT71zpmOM z2|y`#1_6Y!O*P8GS~B`#%c%PuR1=Adi~%1BGG+nd7|^+ub-n@cBV3^l+Hmb)>Vin7 zo}%u{*1BfL{J8y-!8iYL{Vb{kle>#4aXqcTSU+<#OE{)_%1n@k0an zCb%c~!2T)Xagg8;!C`{i2#ydOB{)VPpC;~S>HxvR1at)YO+h?_gxC0q@TLoMGxl=# z;6L)6e+I~{kGKc_lDXohV<P0W z`9Ev?jiEW+VMj3PDz_E3 zK==u{FwXDG2Opge2U-B1h!1O9_~U2lS$-7#l`{kiAwIR6ui&fhcd=H1;L8MG zCm^jNDpO{gurz%HOZR-qKE`#{*$S{dmNuJ>X0sa{NXEO3-eha0-|A2ISpAvKR8Oks Ij{c$l1HV-Y(EtDd diff --git a/tests/integration/__pycache__/test_jobs.cpython-310-pytest-8.3.2.pyc b/tests/integration/__pycache__/test_jobs.cpython-310-pytest-8.3.2.pyc deleted file mode 100644 index 8be30c1a62d7c3cd61ff03bba525209889f5b202..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 7591 zcmcgxO^h7ZaqjN<@A?1P|0OvjMT%ovyIe`6M8&i%(Nb*KTsb6V*<%Ic;k=&RUUpBf zUcaHZ>z=^&8b%Pnv2MBWFnck;qjL-b+7e&L$h_cZ8NwFXnA^FmrWJxST^2(fKMXW-yF(+9Q>yT`UbK;CR3+}4W z?`o~)XEf*AEj>(rr>k~bMws8RRL8e`-}VM!#va(JBkrTVdaJ9tj<<8i+V^~0dGer@ z2pjKoWzQ4VZQFOds<->Dl%5P1Zgr)Sw%D=W@An;hU`@okR_}p7bWdob%i4RM#N)TT zLEk2Zdd}bN_APaA&wF4Epi+8U_IB-u)*b8Pq2;TvEQf;*`X*Ut&m#j%B-$(6zuAXM%*v;+Gqn zA`v8xjU$6=^l@J3BI31iK_rj$Q8LiR#UQy?B0W5<1$tlviGC7uliHSW720P+!gRC~ z!u0;Gr#!z^X4~Zg)g^o-!<;>Mhy}v*g(+U6FmpfJ%wf{;xb%p4}4Rw@vveK|cD-~wqsSPtuw4>0W>ie=u z-K4mg+3Lw;6*k~Hq`XJcSeAxf&{y#@^%AK3B0Z5+?tW(31$wjwR2NUCIjg%l}~ zevIWAWmHO~p_>w!Ul_C;Sd`3O<|sAFLO(0ASh!q}-OG(kksIX$Gsr(MMuni@D`jGl z&7jcN14VVY9G`04>NK@5e?f;M_aj+HvVxocGXx>Ue`dI%|muYSf zJhYaXucEz`$0?dau`1Sr5|61CkEu34CZ12s*J(G=%{1@3z>?t5{r?>hP}X9SdZZ4EyO!g4%{WMenV6Aube_Q3N7{Tw!ovI+eqzWi=!DgIVQ0{F zt*6D!$enC*7jl)F(w`bPZ?^Qq>zA(VAH3S~%^$@~^JbH{t1W+{Z#$MhiC4lLHE%|B z*IFD3e1x4Ri7)|WBCis;_O#UXeM>5C=`~`HeN7+k-kE0V`x#gVB&7KFt_lB z%9z2!$!xh20I_jmoo=--Tbd9$?`dCHXgK-e-gKaq&49c4XIp&5I2L_~Q8jmQF# zJdw-PW|=S&t!YPEG}B?}M2bLFeFAx`NBd595P_%isilk7 z0}&5%HRxGBA|6dzYe{~I#`H3gD@0l}I=v%bp~8DqNYEU-NyT@lm?z7leYrJXczwR` zT2zQ4@b&q^oAYX~&exo1;CeJeQ3G37=N~#zxHjLy8}o%5rz7ZEyc|=+*y#_kD!qfA zW6{w4mq1E-mcORHs2hf<8~jb`$={gSztSJ+|Dyj)m-|pW5mXxxOfirV2i1`tV;}@p zU?4*!#z~dhgQFGtFN{%AWq^FLz(qM>02i4E2`NbOA`v5=AgDX5iSyG1nx|b88*u6PT zbf4(-=w7qK*pwqige)lx@i{vL?i5kPnjG6^ziT^klQgapc@5;SJc&KAy8J3ty-kFU zrX)-!-z7r6U49cJ%mDKXd$1#~ljvJSzD?wPBHsaNy*$Ikk~Xreg1{NtYPZAYOS*~R z_V5+3pnncbMISHKQ1(IDPLPtzqB0^qN{uz-w zMDBtBsk6(()u?OZ{wWEk&N<$DRQxWH4~cw_$oGk~iTr@bM@0UdNQcM|L0XN|F0KnH zZU?k72a(ZVAmypYllQ5}GrCJ$kBCjAPsAbO5)niOL_8wX`&NY5x1RGilf~`0@nyN= zSlx$!1@Riua{AKa?yoCWfkgzTm)LWBjC#Hh$cNwnDaFv(sHt#1DwvK(VX{EIC87@dyRu zqES1{fylQIVbF4sql) z=Mmc*>>p1cx~MYRMBxC{&kG>D{#CTeS%s85qSMe^AvaUgjOAi-bFBSH2gjX6&eX1pjN{30JR z!!u!SnkhYnYQik8Ii5rsOdeqE*s-vaIV?<$Cm|{MBUH9lKkr!!i*%ox_4#>G#t+d; zn3?3t)41KawhAkeiF54@$<&8LGDM~)yH$GO$!wC+43GSTihoJuMR??=RB;M>Tw?4o zIZg4D&+O?$0PDwi(kH0+htHLnd888}G4+a0$fLw}q6m3hcqacE4Uj9CeiGoOn1cq* zDMn+Y(Txb%TLX8V1m}X2h^_`CNHRvbV3i(c6mBQK#5)(2!<`BFU^)r7{UFVF1l>SC zO1^O4z5d+3i?hLuxWnv+ci!Lr)2mmv7@W3>oCp5>2zIt$X$(LXiO`rh6U=}X8TTtB zU{~5C?$jE*#A`5n9ntwjdV`&5W;j}cV?5|Dfd!3eUVq%kNo(uCzTr+N1QwB}}Q|zGFrE zm9qPsgJ8JNv5gNQZg#lY~Y#HMRb@A!NE9jwq$(*$B`%Xnr3e*@3dxUDS1MEmS0(Ni%GvGwU|Opw2^Kai$*iEnDNWAi&>P%8o6eE zG2a|n9BCF73(e8R(PnY6*c@9NYmP6DOWSy3qB*%Z*_>LOl6;~u-Q2RcrMY!+tK^f7 zndY{|Z9=CS+nYNUcL<$s>}<|1&I)Zbb~Se|?r!c`+|%5c6ld;8p<{`6a zj)B^59yTY;Nl*tsO_|f64uaZZZUt2WHDhiAbqA>J<_=JInseqZb2q3%DBEN11$7u@ z`^^2I=0F`V4}v;^+LCz(sH3L-Sfq61Rcf}RxtU6<)wU~kt=+QDmUK7vNX5S7<{nv9 z)koS^?HOiENq7966}8khs}I+#X2o8*^ng-r3+2X_PrOOZ1E~tvS z`p(J{+L$OEk<#;L9_toG-CUJimm6MFcYnK8Et}Lp*}hV3p~iwFR(Y8vOBH3Snt8Em zxuf=#w%@M%X!3%mPB&`V%1x-My{cL&k0DutZ3llyCve?It1p7EBTmHD)^$^JA{+WO zj--CpO)QJbOHt`Tji8~Mx>#*h6;yIlQ zorh)zlzC_ta({3M4+Vetb;ogutoMMhBiAAy&^wWiWj*R+m~T+Gx=eM8qG zjn^?TV?ku$p3&qHeM~ zPO*l)Y)LE8o|jN=cSm_eRaYviTISPLv+RulEVOLHT6dw1#p%q@3R)JIp{ad*E_N1H z(clSVowyUX9<-xSgYLwaH8aZcWql)N z#_afdVk7c,da>F67pngf+0)zVN;I=Z3BJ+`5l@f*>}NGD+?%%qd3X`qvK8nuR# z+|aIRpU})mEYgYMisSuyJ=00qSto^oh?%KQ+KD-7aAY`!^+h~Ijtp=l<-(B>aHLS5 zw6oyo3g1o&92p^=ud#*j{3*8R<2lviS?bc@J#!-telup)%sCn1J?rz1z60J<;5~-R z0JnNQ7veu3=Ks28CDC(Nf)PqKAi=+oKat?y$iol`USSK7;8nKx`Ac9H%+XIqI1cwm zFa}XO@9?95MS%YY=t#QyM*fiSo8Qo`YieFN1`VB$32$91h(nX(Vg)&0d%2MIqMbOL zlK0Nskazytz&n>i@4WoezY`z!$VSAR+!E>JYp^{h@5EteDJKQF{u2FioKmo~Db{eD zL?*~5cgi$Ps1y;)zzC(H;T4Q z7MPrPEnl&$sY94He?=d>19uue8 zaRVMNuI}X2gO|)RIi?Q#+0$zyAx7pQjT=LwqiWV;<)NRB74eDIATNEMjc$XNMb9=3 z;^XjaA53;MN2^D&k@q2j`-sjVxv8CnGfuVr7y}MopF8TtDDB{soW6@b@0k_#j(PSR z<13z>jI4PMt>DNL+Oi)e6xs9J@M=n;REx4#=v01yu{Ud?}~Vj&8?- z{S-N9u!ycLf^qc4#_Db)eYSVfv%Qn0f*NCk5)#+wS+r~1vvQRYLb;;cQ3N%OMj37o zp8rV58EKP{fUkRV$9CoAAs^y`7n)-7eRyb_(_Q2=^JXuuhDtCu?we(0!|@oG*D45_&xJBlBuZ zVv2YWQzW_`#kv#qo_9Mcv#b|WB!upI6UG!tKc+|_4*1D2MFcSgqZ__44qt{jzMk!* zF`pVv+A+Xe#>rS;@OjH%&d;!1d~-J7?W*t=L3Hsk;Rh5vUAB)mcm>IKV+q6%k~hF} z?nVOMB5UT&5hp9&VyHJ{97K?==A9V$A8|(N`A)$eaSF&8W}!3c z7|tlXSJ5fLd+oqGW9Fz+gd|1Qh$MmEw*CY4NgF;&eUIFcqR)mb-PrDb*aAxcO?{s& zNCmQXJs`$G>hQGO3q4sSKPp~71)0aV=yb+J=5gOcjt_W9#!n--3b+hhMaUFcFPOSH zwV`(=oC%3g`>Y9)+ISOU(gAy#?#h7{kE2D>nUDx&QTPH4S*C^lS!^l?n_{HgvnfP- zCb;j}6yE>OK8LibEt{t@Tk zu3I@uLyNkut0JY@Y+YzI%T}~1wA#m3zZ?tmV=@UxWEpq!!ALgkJ6z5t+5R zv{bbafXvn`39%TCWw=g*N3yW=S9f53RC7#@AUOlC*<^o+n7pV9gzxO(A-y6He$g%7 z+Feu{9SPIuc8$(lH*o1OGY{V^POFsY1r6ew)xG5CI1&lN=3ncJ&JgZN@i8~`$b;t) zoj!D09Y9+s;{|c~=+|SFZy1PG-qx)H;a4H<^qPb4i*6BGQZjMAzGzSMiHQ8xpmx2X>!EAkxg7n30ggV2+&a{27$R$HNg`?(ENSR8+( z0&%)lLp9=spZ>)wmr>4}(`{+g!!dSKH%Z^n6}z@GC9ZO5hlON|539iV29! z>MaKtep};63@xiauf44O5cliaKWMLOiHKT5MPIDNB}{1_eosR5`K$Nmh=F>67+x-FUvRea96koc`AXYMx?hS zJalaEDbA~S^ZUck$y^&htLl(K=31F`U!{bllo*|x@YVvrD=RgtYAH&tXo-p-rAjkt zA;EHE59v0OfpJ)nai}#K)r(jZbXOYf6=Iddpv%&C*}l30B&|x6*`i@vjA}V1d2QPp z*=Sbt?s;wZ8|q`I>KhfryuDGOJTfX#LWzibwRHf5wqB1SZi(8l^|(!Kd83r!HJuG^nrWtFP}U6(38qRUUir-To=u` z$<>xLF2zFY8p`e_F2Qaa)j@sEiavu$vR+*R0xezmNF={hi^wu`rhED`q6JM|Ls6gB zIDCOE9|UL`wq@952!QR=nubULOZ7n;6Wi|7nmmQb`^5tECQsRE(}!(0>HD4med8TA z&^v;>9iW|c1Lz}=DPVMQtPc`S!uqtGS&#MnWnd(DG z2<{IFIZ9(aJldOQ;04pjn+1 zv+`D|2%5aToIdow%xP5if@-ac*d&ncAxi8|pMbsnkF|(Twh;Y{rhWspZpv=1lvl8; zq@u|7;UtU@MsYzz&H8KDqW|6EtHqe>_->>AcN=qJ^;jp)9L7HGBwzzcCux0%D+_Fn zBw+(U#$gM@#G`g{Jta0Uhk78>uz_Qs3{tRxsL&kySkUr5wzP?!Kge2AXi0S4a4=s0 zHIHHCloHfDC03HQGpNlvSW^a8!j)nY9O~;iz6*BWl_C#224?O{k?)h@>*%K|MJkXY z9q{&9)MNb%-aaqb# zkavEYa>uTO+<{S>qcRG4BG67=Mq$Js1*~xTM?t>V9P~f#7(Tx`*60)KlTK!RY6EMX z^=a(2j5>w-7D^!Nb=Va6eh%*=-?XZ6)(GDdUB1!cDYQsCSYdkL|?(IZ#E z?-iawLyp}-hwL#Vy>3lR!F@^5T_Y)uer~e5vjNNQg0g~bpldv zfeh32?dE>$ifnhbi;eDZc3`Bxfj3c_9k9^@tf4gW_U(Pvr(0hU{%C!=`YUPCwZ4P0 zU$T=21)QD2&#ceSY>1!jogLVH+9BTvcb;}RyXre(g|x-p&Ti;=4{UJ{YW6xgXD{?U z=8RpBTGRzO90P}UvPSfMD5P)i`^QXsJ?ek|-mH5_xa3!KIA|TU_-&=>#nF1JwD3Io zVWD@0W{I8!N3ElLN-Q|4PO#+=3yxYc`#T)Ns5k$U4lH=L-}fH>z89R$-7sLm`^+Qe zQD>j%ZojX){UP1aAMC;<7L5CDTzjA=y*?Y#C$$Tm$-W*mTHn>#BicRS9DsHYLc0f{ z-IBA%DNRQ@cR0J8JD_FE1J`5LyC4&_oP(C#Js6YOmmc8-6ot9}=T`hl$`tswZ zK+9%t1f$k>rM1`hE7CgLH@^735&S`@@BiD>_wJsS&E5#ctWiECBN$VYd}_!D#;m>4 zH%Bn0O44_D1P}OqAN21faJFzGj}g4nJZ2tu?vxQcw!`rZV$IvJnsV(>gh^jl{)lyUb&l^Tdp;#Skt3PYpJ?fN5uU4 zbOcaLEmNJ^+9`5z3mq+8sxDo&R-4$~sw|v1*?TFN#g-%(a!0Xyg(Vo)Ufss(E)LK= zObGAc70=z~6;^GnTE4(7w_fkWp*8_~c%{IMZw}Z4VmGI%z(4)24+6i#KYwG*YOgBj zDLH6M0b(c05yVbaD98mQYgsI8u)AHAQHNNccx&S~$WC_vxeM(}3(B{F;lS)Bk=#_5 zr<=FNR@Dh`-;X(ByajVya8uo3gFu5DsX#}&8>s{|`z#0w7Jw6veT#Y+0xcK4~Js z0M9hH2|o*gL6hp%1{OUJs#}Nw z95x1}QvNNeUX*-720;A|MuGVS9GXD7&^ICr;8BWCly(bLrGAYg#~EBbD-Y5^y;;aX zeVmUN;;zI=xS1=eX8ZG*Yj_#X&58&eHupJ&yK$Mp)u&mMyi~C+0Tgr--W2N^y=lTt zU#dQ1)^MWIc1OCOsXrCFMsMQuCbhIY$d|cEvuamr4Qv60M%FcYACp_$Y+G)=f3Qo3 z)o+5sK~UvWEE7;=g_Ho~i=^JiO=$;Xq7@Qm5RBb z0XD4|<`SZte}+?*OjEj{6#?@W^rH4fLGDwa{$2Yd=tm>Ert8_)y(@Lhdn?dHG*N9pA%iSl@};@X~#*d!P3EIr_T#DzveLEBM3jEWmXC2^ky5=CLCX zeE|n+f$5T#-2q$2YC3KvP|m})*nu#UM0A1E%9&CV=NkNcf+$%%1^fz#tjTkr(Y2!v z&WoBrzOZWn6xPNX|HV;xI`8N}?M6(4_|TBOi}!?a-X|jHr?3%Gi@1+&U_XShG%vrr z+AY5lxe_U5&Z>_>CN+zMN5WeS&3FXcHAs5rh1}Gc#~*s=DE3QVr8Y{M`V&3_lZmX2 z(cCPFQV%cVxVgly#9QYVjvYU-mOo;*+lLz!b+O8tQc)3|baNFG;a&H9w<@wb*#!xX z5QnDG-(@Y4PW5|CdYHe#>^SkL1TYcG;4P9O7c0XrrsO~)6K)y4tH>d&UIi;{iA(DXOd*U_*Ki5)`$85AV; z9=#ul#c^!S#%_SAzX0qM76UYP9hkh0)5*S-U{?kCL>)GSbuE5m0h|UH;Cjl$DjqhH z#;OhLF|m;tFjFA4cFs)P!bbY{0%Gep@42B7;i``SflcBJZ^}sn0p=Ox=j_7zsG0K5 zAfsH?zlL309}@()j{0$)Y*uYh6Qr=NH-rYpCU@8|P8Rc2bpOfXNz_gepA3*io&w^j zVy!+6PV(RYZ^jpawK!13yuGzPgVR8HX9V08oPzbg=z-i7usWA=;jR#H_chdyJ2>dB zUJ_0~fxCjxUG9MPN}f0xCBS(fb2R9|McKFwbV+@ewWQF)sI&woY8C})8^vj&ac5Lk zxkcNPt2ihZWFt<&^Ra~vPR?S?fJ@1~qPD&r=cFbE_k8%J&PRZUZV7+YvL2p1tNsQC zg|nQKejEUoT&^tJRs6z$>mAf&m-)L4ZckFMFQAg0V%eqQoTWJ7sh64j zJ(F8*T1m_$fzY3`okT-_O6nOV+kzx?0>G>({7|QBBEnz!0?w^ow)`^zREj@yB2vJd znRvYr=bY~ZQbBbMTfhVI9Z8W=85(vaAFeYCV>xfHp+4?1kH1j z8DN9hm$5#_olxwLW-<4~SuS%=!XBZ=#qUKHQ9sJPEma0pB!!uS`=_$=DR<6{I6dY+ zVaXGmt)7uuZ?;Mb%~^_1VwN{kf^+9goapq=;pgo!)Q)rR>YGb>qB4ui0Ov`}Z9sJ8 zH02~@)9VwILr_u7rS(bd)uaRgPdjPMIqZ@0r6FI2H6mX&;P#uSA9XMXs&5NtpdcS& z6;D1h7qtAYw8V(0SESa%Fmpm<_BZoE%kN7|aH)PIwH^YU7aIF0atFF@j>xQAosWs8m-5L#@NdpyC!m<_V3&SVQ<_6nGoE38~47PZQ_r%`iuYQ}H-og|0R+t4+qxzAsGYzJpiKk zkXO%z55>WTV{~&}aMuHq0+kP7Wy2uKVc5-F09tr9qkfS4ZDLWL43c&#^cxJ3OC z+x;;TH%^GfQh&$-5pA8+29rNPf@q}Y<;Ay)m+v|Fe?+5@9m(%4#or5*BEDVRy105G z9ZDShH%ScxN&@7=Lt7*+AiSZz%;YbaoMZA8Bz;)K{9wE=Ckrco(x_r1p{x+o;OCvH zwu0D_zj`r4CkNTT^#vr13>YmG1&t_Z-vJu&LqrmKH1%UW8uPCr?$fyPxb%T5_T!ke zWqCsCV)7jGx2qfMXE4wAvFy-~)wjXu60YD6kD&2_1>mQ~-4&nw{8n%Rto9h~3yt4u zcr?!Z>o{YNvpM3Gc`65Ivak;1uQ-4cFfMVDh(8dSSRb>wGZ~lP>|rg4(-b_LlkD#3 zTJJ!8+K$1Oa`)9B1y6!=SV4<(Y$3mTIL{XTne>#<@X>OI2bMdYmOK2B+!4*l9qTX*ph5UIb(79urIU;L7!77v^A&bfI996t(UaZ~4?2%SV%Cd%aWx97yF%wA%` zy#T*4j>na9YJ#P^nKYTmq`))CZ*~SMWxQ9k`3!+h@2h^CRBu|45C00wWI;eobNE-j z=@r-2;0pu1Eaeb!YXg_5=MjO??Mtz-xLkO!GZ?mg-9AogsR^*IH)9 zC>u%(DV=wXGC&;sUjWr|S>YRm&bmLYlat-c{)93bytd1^?^7oInAVDzKNCVGaDF!yY8Jb7%|v zcE9{|yRq7=ii>l%4{K#DRa+HRYg^tgAjip;SSMFJ>(A6(P|s?k%KyD2V$C7RMt183pw)ZzLYn_e0RN6_M$(8GDI;g( P@i&WWhcRkQZKrm6&?#^|(eT zsT-9G=(emJD$s{6qW1#4#OKg zegl2_!#E)UX;Vf!;L$+aB;VFrw55f#9qWL9S?mYeD7i@<-D|}{t*wW#AP$ewPB4Hw z+NRdgfG^47n-r+7wgCH-FeB8TlIO8@szX}wAl=S{@eXZgVP*t5mQXawP-m&9I!iwX z-c|yn=RqQEC4;`85GFQh&>zA|XoPW=`H>zbYXcGf3JH@ssSQT!mxxBp!PR4{1KV0` zX^$$c6j(3>`=!CU151Que*qS@GGT`0Ry6Sx;@9^f3DdRVn*?^xvO;%0qUO0UD}Em4 z;GyjzN(i$Zs@}N{MT_0vfuw$r^hY+Zmy-?dl?+l|fV3Ce+^syw91ZgwS{tj4vmuc2 z0x$=|{PsRd{zoAH0IdC!4eycH$42(d3t^x5EG&r66n7uk*sq`hKm%caIIt3fb=sal zSV`urxJTo`aBz!?M~Z6@xQ4*XL$yip3P!ve6Y&1j7WI$pp98Q`g3LHnJKWJ)2SUO| z!xUsf`gv@ew1&gs_7OIQnCBX1h03-Z8}DX9swWdhHu50rXm}tT4o5mMN*)rJ9}5p4 zk8AA0k4bQRleCIp9Yh)<`RIGC(Qvdj$+RW~oBoP$YYgle18c^^@vyi?TL;5~V9B3? z6YXgR@FHasxU#5qbT%23+b26@jR3_*KyfMnKZ2y60(=@_m$FGV)kF15qzXqB)vu8X zVW65)@EBwb;>J6am3p{-tGJ4a>-UNRVBjjTL(k&Fq&2|~vm@bzO#z+^&H%+sIN70_ z^!t=)I%$o;GY*y%+p}ODSh=ar?-pIbzM&}pA8)AK#asbz z{2jajbqsuQ_Y2uP*qUO;ASb56@s)V16qdF$|2MGrG&>%aASX`Xipq)U9{ZjtO_NH~ zj?xrhuy0zyQ$5ixKZ`-^4zZK$RCq|maiSZ?iJmx4K^#l1@hxpttS+lQa(d(5{M=lr zYKcJDY>E4)OXo{W*lP&E3al+c7bpqU1^_24mwEM)*8te(oN-oKw3<#(vfUc5Vk?=k zFV7Kj=Tj{D(pXfj#{L_0iC6T@hB1pwv#;o^jLn^A8~V(G?Ur?!3akcK{Y0jHdxM(` z>jC#=Uv-&RSNvw(^p~x(7cR;~)pG;x2Ed{_p2eimbk$q0duv?A9}7F+Vi>mG(}XlY zV4r#bP3UD2fF%-=N68iqX^jYIKQ;PeZ50~p$K+#8CcSF4*|0g23E%Mo(`JCwX6B@h zQ!x(wYIybbtnj>G_L~jve$}xUp9P%*f7S*SmxSu83S;_qz3EtiCrm#O-1lc1>oR@C zvHhUx)f?ANfxwrjVv%f|K@sMjUc1e7#qp|^<6oQE<$TRZyuF~2{y_kgk?F`tMw)vK zrFX!n#DXQNa4C>U6?0{&C;_XCE5xp?0{gf)0>f!|UgACm8KzZn3Gl5C6?~Y<@N7R=} z{>rxG{UxC{{|g?>&};;55~cxor&%GXh1D^HiHIDvxp^Ysl_^f3{~$VMVc)iWE3 z=!hu!~@fE9J6byFY8$q8r)T%$G-L={C61< z%PcIL3r*VzY!@_+s-P@FOYoWwQ>5mS<1JW@=~{IzQ>G8!GAKdj-%;rSUFnt(o{)oP zq_oMOG#tx~K7C>qW97p_^{d48_l@oZi+*Hi8dWxJm+>c3XI5az_Wf1Iab%mSvM8ES^}?Va*^feyh=xNlK2v^-G$8DRZ` zT__7&KcU*^`X9s-P_?Y9JW(ILB7@G&B6dusO>?mcpTOKS1y)d*+|F3>5T~PNl;bl| z2=JaHxH=IswS$sSYoi;?w@~zT9NtE5h-!cZA=+1!03(toM?AQFEa2^91}g`kf{&|svOx4>5P=LBfgA@QE@lA|DGb33nv8xc8Hzx{2;!HPeo=mYiN1e9 zYMxV0VoIuhNosLPv3@~OenDzcNu_>#d}dx|NqoFsLFFwDo80`A(wtN~kQv2HK!Sw< E05=UC2><{9 diff --git a/tests/property/__pycache__/test_robot_transitions.cpython-310-pytest-8.3.2.pyc b/tests/property/__pycache__/test_robot_transitions.cpython-310-pytest-8.3.2.pyc deleted file mode 100644 index b4c15c0bedecfa51c874db25f1354a88817c0aa6..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 6193 zcmbtYO^n;d73NTsL{UHc!rD7DIsvTA41t~4eTpYtbMQ;n(CbYr@;qp?Hv z<^7$lU5#CeH~rnMnZ`_OwlS;P1%FR#Z(}dh+>$%d+=pjBw|J44KGC5|nO8o^tTDF? zZsHTht9&QlbF{K+tUa%}N z(~^;JB6rzyWi69TUlfpdcX6@P_FR5Rh)^VzD}ESVYPF+o+Mc5JHW4gScqi-x(K{h` zvCZ@kL(y`gH^N}aBL-R}#VgK|8+EUS*WCc}Q_;=P=CqmZdkex5-Ax8lHg}RedQxb4 zfxYVZ9T%)~D`hv2rh_I8O0VZ8CXEq?6-&gsx13hncV%LQZ5MXYd3;syJsJJP3!-aD zPPkD=1c@;p2JRy5IQ*5t6!E-4S8IPp(;{t6`&jR@z8-5AwfpSaT0hgf9%ne)jf-|& z7thCRo$<^DYo>gL>);J;bTl=V<)2pc z4zoU1d*#$##AnpL>0Z>%@*H|-*6d(b9^>7&;`{li66e=g-;B-J zSY`b}T#)~aCe|l8xFyCd>R-|~3|@?;)~7eLRZadL<2$0t1`~e(x0ASlk=u+UJ|&45 zR*TO_f;dQ&6c4M^KP=+q(Tva43ZLIxYY=^{2 z(v)n*8NN!oj_jWrS_7%KNGdKVsdO-?z0<( z8+mR3c&2;x3)MwOM8f0CZq<`j$G_=xWz`F+@WI8%?^fYqRX@Z`?irff z?w(1q3b1N>Lp*h|oDe zGdD^KUI1{8LeWjKw5$}*(3o|;-Hlus*^Wr2>_x}-ZQ-_IWvlH7Hz{G*Ug&r}09__I zI&fbmv$k9bJ3d#U_Oc%?IKCY?Ex@TQ0mQUIQn{dP1V*O7zQbNpxD6C-`%aMJcE;gQ zUYR6oWidl4VeulC zd8mmW;qR-{heQ}ghu&?w!Rx-m-9vOH@=#mQ{!v#oZ4oYnk&S>BNKavZz1>X?s?!@? zRy)0^Hev{Fs6D))^h)Jb5X-Qb&PqDyHg zuEhS#CYgSRxX-~Ecl3W7e-m$E;mz|z_JAb$w$oL&A-h2Dc3U7K_n@1h0?}!B$W|bV z7@%0^0Hr1}ihc$ygR?Zq#QJ(R&O|vp`E`?LVuXV=6@D;MK(Ey-Zs>g@Dsi}TKim5v z&aRhZjQ|c3mDq^0(FD)J)ic!7oFtf!X`g(YZ%*}1t^*p)xNtq&w_;2FHkw}F!E><% zc+XS6f_F38xxP!myMyuFF+!qP0yje(fXh@ofSeablB-G*fmGa9qu?M}P&`2#%?8Y? z{{m)O4{`+H6e{AP0`pQdi_tv#vbg++EuUgMwdFIV#kbq?S0zcd{9Q>t$(Ax~m{0+!!YBDuTp0wU)P`X|+Gaz) z{DFq|wQmE7h?oGke*;09m#?U2G_r3va#OoWApQJRr|nid(&e_gXw|?4_*9oeS5_BX zKfGD(xgxA9NLJvSkb<&Q3OoTNw{DVc zQPwReZ%wTCFI~83fAIcWS1$<)sY&rBg1?)-_{2&ZOP=ov3dKor?8p;q>zKs}jEa*) zP7ygx-vh?dSLelE}&4+}lWv4yBn< zb+NnXyTd$ExU%C%DxZ`{TdF~)wE*M_m$tG_Ifh%C6g9VnQA#c8kyOv^1#=gNdd=)3 zgf=yS!u-bo&>IM*P()92(i#&lL-3)V==H-Yhk5kY6wZ?C-D-=u4{2#~B>m2;);p(e z6Ge18HO6kk09UNK5=b$~lI=xu*815rMVqUS)_P2Da$lwQQ(%Xd#NlKh4 zw_QjRc$p&KVYP(pw9>rtIGSRP$ah4kS|J6W^EBrMk&sB6$m<~EfH6Ri6fjPVqsOU7 zjxu@iyT5OOJy`)E)F|PZ)o0lsndqTw8*)%dFhY*MpfSk%fE+*rHij1X=zts$rF)7e zeSLr#dNV_~(9aC8LT?(X4U}kRseF$q^COJlRQyM|^}IraS0a-#M0oRS;H&{6n2N`T z31nhJ$#<0`A;VQQnj%9%@s!h94-E4EsED3YjNDE6d~x9OMO@cX@9=!M<-=8{-F6+J z{2ZzKO4X@S=~G330IP1-n3szKbZAsW4u9m~z!8bJiu40PTTiT*1BV^J**<+ zn0pTw`BsI+ZQ3^`(kId)vRxPPQ}nsi|1*%@+hd8$cTcW7$vxQ&@d0dGwvv?Ts7DT$ zRMc^g+BM@z8O%ItK)TJTd#Yk@?)&d|RBWv4AhUDqB*( wr_O%tZK_MJe9t@!y*j=-M_#USu#)zoJ_+31iHEYW{iw4mZ1QEI)^w%(AA&1{6#xJL diff --git a/tests/safety/__pycache__/__init__.cpython-310.pyc b/tests/safety/__pycache__/__init__.cpython-310.pyc deleted file mode 100644 index 7dbd6d78d1be595b78e043cb9da1a856b166fad9..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 133 zcmd1j<>g`kf{&|svOx4>5P=LBfgA@QE@lA|DGb33nv8xc8Hzx{2;!H9eo=mYiN1e9 zYMxV0VoIuhNosLPv3_x4T53t9etdjpUS>&ryk0@&Ee@O9{FKt1R6CFf#Y{kgg#iHW C5gZNx diff --git a/tests/unit/__pycache__/__init__.cpython-310.pyc b/tests/unit/__pycache__/__init__.cpython-310.pyc deleted file mode 100644 index cc4c115c1479d313753042713508b2be1967ffbf..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 131 zcmd1j<>g`kf{&|svOx4>5P=LBfgA@QE@lA|DGb33nv8xc8Hzx{2;!HTeo=mYiN1e9 yYMxV0VoIuhNosLPv3_Y@W{G}$d?t{JkJl@xyv1RYo1apelWGUjU(5s~SQr4Uk{dt( diff --git a/tests/unit/__pycache__/test_domain_models.cpython-310-pytest-8.3.2.pyc b/tests/unit/__pycache__/test_domain_models.cpython-310-pytest-8.3.2.pyc deleted file mode 100644 index 26d0626cbf19388adb9430b614f0442c15d13f33..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 5239 zcmbtYTW=f372a7ccb7|2SKpH~PFkmC0PEtboWzN5N}WQc4a;t8qA0u4tR%|3VRmU( z3i(p!p>SceXkYyx}kp?n}{sFt2^e59o^#ApOpnB}K_rV4x)S?Ae*wnX|L= zeczdF6^ns|-@pG_SpSn}S^uGn;a?UPmvO{@hmn>Pb*m!yY&SBtC9H;1$qK5m>u$rV zc zekayr3$-N`s*TT@52I$%QX4vdv(;%PH^Qc@M>@wBH>tybUDL(rt#+isq}FQQiQ-N@ z(YeKXD=~Ghijt0ME>mRvp9=?X!McVc{s&Bg8GkH#RwB9<&O+KfyDPf(1G|^$X5xE^ zy_J#hR)ln@e8t{&WHxcOvfI`}D?WjGHxb)Hy${bLuPJDj2QAK1i>`$hA5shQ(86UO zt@&(k3+X*`4p}`%`ZCvbY65;X@ll)WX15XHudyZ^t0!<|dbYGM=RsNf?X}j-%xry5 z`?qi0g10dDT)eO^T`GyqnG+}58)r&!@OkPSzjU0unNoa`&agrObR!?fpSiAKWTEzNDHC}6V>XMsQSL>~%uwHG34ZQbi9Iiyv zgC4oUL*re~E7ekZvbxezjWDT3ziQXRCa;2$t0pOmJPl()Jw1GJZusJ?xiGz47{0jp z+*0qMpBFRJs?|!SvC5iP>!FOM>CxipPP3M=JzB;D z)S5giJO%L)+!7Ni3w9vh{Z`Gu-G$gZ3I-4LOFSYWHUUl_3z?D5(@f9mTI)8D3tSLF zJ3t7Kdn3tg0jZ2!cYs_k$!@uf+!E>;jcN*>pW@PEpKwWKgiHTfhH(jA9N;pC+Tc~V zL?rp1n-sb(xWopRcwM{eflK_ZAAg0r9GUO>;F1EBGnW(x5h>JrNdZjph(;$5Oi`3O zF+Zjj%q72}7Q140cVgx`53!Ft%Odt1adtPyoDw7^_F#xpTyTnw!|!?g^2f_x(@IjK zFn6D4v=@U?t2Jqm<+qlXJMCH|p9^i*;y%%w?h_519io})HwMwvVkV7qAPt9le4bGU zFaj~iX)@yhY}FWZouooD4zT z44PLL;1AQ#tor_*k~70xbbk0^PQA0IZOU6F-t+1NRWr#NTAQ8Si_asBPkyH{Npf{r zMa)nXGDC4fJQ4pC>Qh|rf+8g{IULuhp&Wu@V#^HR2)I4XtO7dHl~&@cXA^hblipJY zW&V0j`cEAOYXGnU9z!yFKgn+uAbcdV!2qoJq_{Q0VEq8~qrC7xfj5>e{5<;vRw^T~ z7E-VxT^WFN9JLesfOS&3N3EXQb=ME{Jc<}V^}0SN#O~VhpK)hFjzaVW0EBV^ERRKC zqcI6&9%q1qe+#Wc31j!x}JjndP z@@BSPTRJtXW-;#umFhIPgi`RdFDa!O05oqd`Q7S#gL0@UU!ZA#jws1OW1XcX014*i zu`Vu~G|jZ_!|pV}ZXay#^tF>l5V~_9c$2~@5+U^=nUBbvqkzR`nL6DEutUM`a0bZ0 z;2utjdki!+fAjX;`$11p3SDyc;+RIkISh$JF+9VZT6)Z@?f$DwF>S8G^gIOy&xit1!cSFvA#E*Qt0I z$uoPA{5B(b@EZEFJ^=I&&_hg=WD8qN2-~K!L;r8=Zyn$L?~~$JLfu1ymta0n+%<+d z^cFd&v=e(hlQ;(KDVVb_<;)9-yXCz?dHt;%gZUKd0|xVHc=>^fkxyWz^M1-(Ky7g! zFpqE!fXtiK*8`1^@=`AF7u?m42xFwm>m~h&9-`hv^f`}ia&(V)up6W<(2jl89)tp%x4#4NI|J`@1~oOsnMvSb!X(2ghuD?%wM@)y;WYg&%G#`h1Uk(l z-KTr@!<~AUYDv&oIn_LQl=&OO3GFe4nR#|IYXgLxN6VNp6kE8AEmyeTJaHkOKS$v% zR8byFv_oUXU#8+IA)HTb>^?0LPspZML&k}m7Q4}P${+>Suiec+kesd)m$4tr(EgBR z66KshVil&l(P`A1z2;foSDJO4XCH{-bB5M4%Vd^z^{EiK>0T3Sc(B)`oY2Aef+Hbs zCz3y+ua_f%_@$8~s7KZfDcs;u;Xw*V+5dkc30Sa^RA4^5u9Ifg|KeyMvY3}~Drcdl z-8QhuV?v4{zcy#XB)|P^*zB-}vtg26ANz14y?3?UdQh^F19z}}i(~CIV8hkuQ|&Zc z?FgHrD2!W8?a(4AWz;Y5dQ}xhdooJGT0QQo-t>0>EA1Vq+R2hlSGyHddn@R&qoO|W zC#X{-y8UfDpjWbu5>^nzF8u=>8Fi(+qQz+Xe_bV`wTv72@g&{7FQOKy_8pkvy!D(S zG1`l)6O1fOqxpn`-9LT9siw10O`nxSGA2bXK*XNl-xg{Gmw3=3PUX^D+V6i+;51wx zt(vXtl*x+m#b8QBk5mocpK^mD?WJW*DB2@FB`qboQm0__VMNwkqHX__+2&7~ZN4EW zhuTrKU}7ZyKWcL)_W%F@ diff --git a/tests/unit/__pycache__/test_fuse_filesystem.cpython-310-pytest-8.3.2.pyc b/tests/unit/__pycache__/test_fuse_filesystem.cpython-310-pytest-8.3.2.pyc deleted file mode 100644 index 0197880dcf32336f09a36fa7b5b6dfb07b170fd7..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 4997 zcmb_gOLH5?5#HG+umC8Eq^O7Ggz+Ob;shkXw<9Tv6+cqtqLe7*#7HGu3(SIqym)wK zVT;V-68YpR$K|^Z#8H=2a>zeORgSsOHM#hln=Z-MvwSQeQRPE`+Qv-J>`YHj_t)JR zR4TrK&!>MaKlle)|DsO*S`OI!EPH|`W1#mA8IRc5NR2~_n-ex> zXxZH2Hg{6?z?s-%6C5`+A2{6Q*2H3%i$i)l2Xpe{xSK?8E-x>?*^7FR{!Qf$_Xf#8 zKB+8cjpeYGJWRfEu<5}_GJ?G+fmh0wt#Y#lf4Y> z&~bx6*N3N`WAiw7lK}K574>Ejj@%QsWoC_JGJG_ zgdLjPOzjTin%^-eHn$V&(4H8d81f~IJ1K5UyoMI;X2SXmWVMMq2n8*-VkciFVDgpB7X9BNMAx@(b)@^10N4wguMhWEs&XmgEKzs0D#bia zG?j%xBCAhkLMbDW=n3mpNzCfp;{& zO_Pp4biHJ$_wmR%kX>Br?;JQ0dtiKJer(>)3=OVo_rqQUC@^a)%8B}k*w-eQkwv_! z{2OssChb9Qcto6 zM|&VNXWqP7We0PlraH8C^QKnFT2(?SsH9G0?I5qLjZ9e^)fqKQ2zJH~{oBKREm|nl zMaUD!5N;?){!m1!jNza)>c$CtQMs9T>PjH@2BVnksljeMXoYb=no-3-hMkC3P%}T( z_XV8O$fxFm&Or3SB#0gk-o;+^!>i_Z7z~5L8FS#9s*MqfGiK0cpKa^ zSW*Jx1WW!5lT2WE3CxZ&?YAcU)=W+KBJf~r0blIYN^RhalRENULK1GL4!qPMzQz|f zaSmOLFF(Y%M@S;-Xcedhd~tQ#hW~one_cY3)Ybm$B}I(-sh9I#hxU9>0uK4(Lh7Ui zU_>!3%Fi*==B2cV-OI$E+WjMpXS)yej``gkpa@A+fE1OlEg+jmn3Q^23jat7V9W{1 zIY5yj_O~9CA<1ke$p}11@=qtPGbH)Ti6npf|4E|pY<@?)j4Kqc5TVNySBbm; zqHKh&L=Xgs?-E%eLV-rS0Rk6pkPA1e(*MIzV<~fGU|~&m52Bz&j_Hq&T^yKKwZw~f znOwY~7d+$Vo0*?)z5|>xyAoT}p0h<)5+GuE;4V$0%YDxN#KaD|XWWjkp5R7@@Ox-H%k2{L&`RJYUm~A{3p$icO-yF| z8kw<;?<1s8WCV3Xe(zHxVf6hFk2VZv+At*LL>wtc)jicQ(kzX=$o9f6@=A5#UL<;5 zonK~{>3k}Xnmx9rRY_@2F6O+XRiftx*;2vqWThaY#Pc=0A?eTytjuQEOaB$WLg#$7 zkkX#3mYHLrP$L-$?HPg#fE||L;M0nxjc%aBogMQa=biUQ zc<rmoyeZ1{gq;gdv&X|z13Lz4;49kQ5+ABL4D^4N;6OgrXGR%IXI6z5P9Dxkw1Q&UPRtoJq7$N z;CTTVZ2KcQ>Qx;L{o-BhD855v34|iS*1?5bW?V=3P&V?wm1RWKx@=<_X@Y!@XmYzRdXtTjO?^(8?WL$U%ZdnS~b#^VM zvu;0$b`c}?18pd`v5K4l$rR>8X&U?XX!RFW;W(4m;apt}viMWW><5{8O+;E$RXkxS z?}(>OMtLMr+-(V}YxG&v!T^PLiwf`BQ9V=REnWKSe*)C?r^l(7&;BXV{}ZUor&IW# z7k2wK9kis-K$zQrDtYXJ!)hI?tEVsOq0=HiD=559_aREHC<~yJfa`QX2R|JVG$v}Z y5~QS@u65ASC0*f0Z@@=!bc>2cxdifk{9Q0>l5*+mTC;4ih;L!hU3BN%Mel!7Bjug| diff --git a/tests/unit/__pycache__/test_hardware_parsers.cpython-310-pytest-8.3.2.pyc b/tests/unit/__pycache__/test_hardware_parsers.cpython-310-pytest-8.3.2.pyc deleted file mode 100644 index 7724fed1f69c484c505f824d5ea350f3595782e0..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 13824 zcmeHONpKw3d7hrNX9WoE6m7$jC2?#C1PQK^$Z!Lf2n1PECs*a9Gjj<~`M&>k&vXxGAj%;fr^;aU zKd;~R|NY*-zsGj5n2X`>Uq36{`uSKa_MdzS|0VEp3Xk*80Cvna8nKEYzwu@w9y4Oi zWF=*=CEiFkGnGu!teDMgCELtZa?N}t-z-!L&0?h}eG-jQbF4Df9IuSad$O^mIZ>Ht zZmn!>ZmVo-Zm(=_?x^gLwp3$hb5~_ob9ZI85xWqx({|?Ln4P&Fuk68_X=m}4#oG&b z%h`FnkojIDPr*+^jXXuyA(a zl6C(2+?=;7S~q{iI(zZ-{Ds+fdc7AeSeH*<)x9T{>Mh%Ht1C5Y@n*HPTvLuW-mkM& z{Z7sC3M*CR)U1Z%EIRez%~`gZ)s=GG+ZkZIc7DM+H~a3Tv!LkG!ZmM)Kg?ZnEW386 zzKAL6j$2N7d;B4nuYK1#bNZdLSI*5^moF_WT$;b&Z4I@Zotr&9&&u&o|#cMUkz3kq-%4Jku zTGiZD)tX;4LdeHaHu0Rr<4k}OH|84mjWwfd+={QoZDTFbjdv68#n+PEq;u9ytS9Yw zH@RfkQoj^`n6#5_YCZihb|>b11MQiv@z79ZluVXd2HIJUKQ!bw@zAhSpC)$1*6>w! zrkko8D5u>lT65j>Lty_1l9G(A#qlJ8-?)`u%eaMZ26QECbIt4~x+W;fcC*eyj3Px@ zP?Tl8P?YOY)I>YRf+9uiEPKP*eL_~QWN)>%bxVRh=VQ->uor+?C;%qoRw=Z4+rz8(Q^T1R?0WrnJh^`8F@_$p zf{AS*iNuOfQ&`EM<0#~?|2=O3HMh z93YrcFZi#=9*>16Ie_o*lIS$8NXz*R!;9A)kR;zd0&k;TwQDwz)*bnrBV}hJpX!H2gw3j53jhH%!!H;M5t9IMne`TfCI@7>6?{}eT zo&Bq=x+_KNW>wkmS5?gt+n}7Om3vs#)c6UXc+)FBw0PzoZ#X83$YI=%O>0wr_ zx#He~@w%a^>fY2_jrL--;hdc6&C_`;v}1+ zTrYv1?LY+Uba>EV{i6H%)9IJP!blp-@GEwW+V>r3bqWBc@xb_{@qzJg#$BWH%F{7T zLZM_SNE*6$LHPKDN!lt?@=X-8)Jp~J!5}S{D-{z;+Gh*}iq1|+0 z)z#}L!HR2}Z>QX#$8q)$3wx41e9MrMGBq*lFiL)@k<(&f;TzayH!U_UUJ!eb zqc7##_q2~a?34S5}pmXvZ$_Ze)i*iKYXnr?>h=DFGjqqvMXgRsY&t_Q zNpOtdIKdkPZxXykaDw0@!S4Zh86RUgugInP3ofxVOYr*yZxhTBTqc+&xI%E1fciNk zR=Y!Dr4>n_Pq`2d#laM*`pD?KI#Q;hBO^k^88Uc8sD3;kRJH-Oh-a5*f^3ZYmN1_N=(gf(1?|g}Niyvw}TG zs-6S(y#MXQUeMp}WB;e;i(N#bI4_QiXP%v#pPoK2cR&lrb!^oD2VYaKqOxpsW;T(8 zx??RiYSmV~wH)zpM4rxpLmya}Se*l94)*fvxA_GTrw)qsC?eFeynzIokl7l0ht>ub=;xp!K;jOIZ ztsdh{Z^Af%8yV%As1611J)IWr`_%!*(WYyT+rJsdDdERIWS0RZ+7nWczVfx2b;kjt zlSn5GU}`4XH&QC?Dm6vmJR2r>%V-w9_F#N$}Y`u1MTu2yMZi%NxKGC`?AI2%OKh-*O1i@>0c07L>w`X%2&Ww{V?KTfgh zBtc&hZ|Z(v^q2BF&U5Z>6AW`bwudCE|Jf8%U#Na*bd1hRPq9>yP5jah$N!O9HY4uN zZ^GdS+?^a8j(h}r*x@KVC!^py9NH+TGPxqE<%O?BWWT`IaN4U1No&Tt5zpIq+Yihg zQ*>kCBaab_xZ0;!I!(~$^+Bg2H1BA8@PcrymVV#uT~h6)P;-d$e;=Sz9>TT5$3SGa zV#Ito>zS6M11!tSSyr=cuQqBZ7cA@cYPG>cfn}+8h~m2h-y`@Vf|~^I5%g`BlxA2r z0l>J&;X0VHoRKqgY5Yr-Qqs1F>d0RLZ>R7$3_E)1_F^yHUhJjYi@kJvv6pTy_R{T4 zvX{JZuEd-iQRqCo{@l?bI}IHf1m14E%Vhgtn*BPGl$)j5UzX%1f>!lSNo}IUG&@4> zAQ69rJtX9QlRbJNH_Hrm`O8Fn`qQK&;u&f`MIv5;>`XUo8s&WTzy=AB~qO?A{?j4>tq+Vg&bpgm9Oh3vPz@{O%%fkB4z1)m#c;-}3CS zi@~3mS2Wt9JzyF~Ih;t^N?sUNGnD(Kah$|xFQe0Y&PGxv0mYWS#Cd5uLoq#LV=vRJ z+4a@tUEtd+GSYKe?!A>^d4g@23F_{^CLS_Q^ac~=q?mbyAn)!X@7_7I$s9M3gM4q; zH06?Fbgm$DU(Ofs@}Up&vOOZbLKs&NyeUpUGH#QouCKeOw!9spXC*ysEv+KK+Fo^6 zR$V4VoE(5@e1a^!x{%HZ@(7d$?P}j9kRaVmc!=)7dqtVeL`}TQLUsT=Yh#wFPaL!KQoK`zHI=5C- z&8fBAv*_!4mY*3P8XXDOo@ARR2ae{OH+;Tvkr;-Zg54NuufF4lIfKW-K?3+ywCfb{ zYbXs%5g%g@u~#S9qi3&JrrqH0D0`(trEqd~xXp;#tDj*`CWT3J-(C?9?bRUOVfN}t zyidnFo*CSEP(v8#aaJESHW#PW``mruqM}N%{OhSQ)xp@ZpJw_Y=lN2aO7mO)c1A$` zMa(4pR)DG+&2LW*jPM&jiH~sa$0*P@3i!=7>^P5n^?@%vGz8dTuj37|y(!s6WKMft zN6PndQo=PE39wVV{L=Gs(ioW;cHKavJ{P?uLF+*YUcjR)0?K6Bkd;-{ph8RP3||)k zB!y?A(0OCfNz@v=zcibK#CI@3&kp)o%ofb&&?;7iI zUu318E~eZrNy&U;Zm>dm*4|} z9}(Oq_;Z53VUv;w&I%fMJfx328O~*LDFky##TJTH>C{UNFaS z&Gg3j6${Ikt1Ba;wTMRg6{ZcMwXUJhb8wOqX3H5jzMjB!C|Z(aPY1=}YfCj@^60R14f{SDYuKd1*B*cUrWoEH{*y>fC+@E2$svC3NJ zbzd#>sX>|lgoyq+imfl?e}#tYc=U!s$dRjI-AM6Pe~DIzZ%LF8ZhVfEUM@$^pZEvg ze-+>Lce$3$ay%j56?(4d=R25dDMfTpIM;#;cs-Bn8RdCIAC1q4jyJwI=2Caa9GhQF z8GGw`z$JUaPf4EAp_FlPW){N<>agmq(R+(-qf=!cih`9TFFANPa; zlG>;J&y5!4-UN)j}yqKmYIVf$O_nVM_izL`g4<4^i@2kCK0Urj#%dTnMiT62V(Tv=oP| z%767}`R`{(izv*}yqDzvrVwW#P+pr}P9DhD@`iE-|2JW^uI{N_;A*v(fN_bJfN~79 z>xen;)LM31^>rbp>*S;QR4zD-7`~!#7_=BV>>@eIi!%#{fLC(eBL*t`_YNnbg)tH- z-z_A5oZv3OQz|Dft-r&|_yl=ofFOvOf5GV_x|(H4LaYlcT_hOlSoNG(EH4kS{Os-n z+~$0M+wy^n)l(B;Wepb#qqH(h4!T%>A4S>MV?Rg3En=c?ZpKQ6VomvLo*CTPm0Nh; zZcE?tozjDJ4p;l=hTz6Z!uE-5BtH diff --git a/tests/unit/__pycache__/test_hardware_safety.cpython-310-pytest-8.3.2.pyc b/tests/unit/__pycache__/test_hardware_safety.cpython-310-pytest-8.3.2.pyc deleted file mode 100644 index efdc4e37976500c01adb8fa10f28b0277f98d309..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 5762 zcmb_gTW=f372a7ccgZD1OR^$IvD>I=;+nbW=K6+KX?$&f$ck$VXu_u0tT-!amAx=C zOW9JXm$*QI0B-u;7LXnd6nXAX=xg7KeeF~Kg3%)V&dib`WjV-0NbvCN?Ci|hv)}p7 zIh&4+xduM}{JOsV_nKk+n+}FQB^y#QRN8|BCYeMOEG#Q_6osOqkQ|j1> zUW%t%)6mcIDtCWxwa!4R@jA4Zp^fndv>9lp_&BszpiS^eXlJ3F=2OtlL3@c$L;C@= zGyG*}KZG{JUxD^2XAg|#x&PqqjMXN~t$Tj9)im>|8@_0#Jh&Um*w5Nqw}nVWK65|t zqxXHmANnG=6-s|S3b;DeEaj7TQh~WQ(_|wgB}|iiV)%kzc5!81d*E*b+3s5US&(4E z$(6M`4{oM$>?iz&-^L)v_7e}T-@UhT+q=8=k+<^U^;@@Zn_pWHS)`c`SEn zvHRU52}Ev-AnS^xO}EnjFenxLZsI3zL1cJ@+1@SDa#!$c0JKe_NL?P(d;`Lv-3RInU|XjvoiCu zu56YC5M|_cXEzIE2K)R_1~RX%?REm~&k`<&4{ZG54xSof3Ku+EnipxB&41Vlk{glF zgL%4K&UceAQ>wRBJONKCSI%{I^H*sOS%Y;v5o`zTOnNdCVY2D@VpG=10ptmn)$x7p z8!?T85nnkNpr7*90z7x*EBJ@;61O(ZL$=R)Y}?#7d3nFoGkc}a%zdk8$q%v;d}#Nq z4aOZB-!Km?UdhS__Mx$3$hUFc>9Ip57NJ#0OBl|Rj#bq^o&c}@r8I5qm$}Ppy>iH) z+gTN7-JX5O9-OV1j@VVO8f=GACVecy}22R|J? zxTp{Gl9z`MuBs5z;w4(|!f@ae5e>_UNu(FVoeVLq+)vIaBB;T>WPfC02_qvBG{QEk zIIAOwBIik|&p4Td$Y5WA{IfDI_Y7`7F;C>3+2?r2S3*u)6MZPhE1{!J3&sk$o%S&(FIdoT}%~U;#->n@oT0 z|BaU$#8Ar^9Pw`z0|SQo8xH#ZeXP98t)7vY9L)3td;-5(#8n5@A^4(aZkM?;V}L12 z+x9+~X}{dFdS$Se-LvIN<{VUbwP%C9TpB0#!dSHjPE!9?@m~f<)f7iLyw1mZj^e0F z=3-W@R~h1{5^)rMHrR}^Mgvce9?V)^g8#)8*ss{4X^E?(Un4<4&!?mxccK9NO?kva zxOTdk;*q>^@BI%K78X~o3i2C5eV#)7VxLf;6oj$}LRr^2{3zBhHziO5nph-pk;LV_ zVHUZnS>!4hK|ObRx|$ov_jxr;b^w@E>;fku+(9;TicQ4pH1!PvL44)FK7%8EGNMH}LY%{@VwS{@ zAW%n{A*v~z`~d&iFzH{Qvn{SHf&*bPm!T{QkO!M3af8;lN#a#ns4nP1=ckK|)CUml znKwqw8bbYfy8b65E|3@vg51{IH7jC`&b~u}kS*p(ERa|v@d1ey5>o@A^3>ZZPn}>A zcwTx*>dBA?Ua+x2l^-oCDS+!bQ+*6|1j!>zdS4-#aLh{ulbXm_sPw?5WrZ7?mk-Uy zsQL&e988M3%0LZAHDGDm0iT+hPl0>l(@(O>L6rkrKpPON`INhvdr(u=;4;qFiCe`D zXk(-S_pZ{3X=&_5OxwWOQzJ}Uuq-SYhb4^>WuA0y6DjARaKyh+q}ggyt6P{hlLmEPQQm0tBZ>=c=Q37NkITNIQOgqrS2 zk4yZ9CLNV+bTjI~iBF(`^9I$1@aWVrf4-&IX4Id$r;Lji0s%)or(O{d$cNlnXH!(X ztz&e=ogf*5Xh*q|U_a{4q8Z0~kIXtKbBjvj%;q+Cp0GpfM7O>6oZBj^;v!bo^|A8K zUUfcBvU$1_-J7AAIH97|m~O;rH$fL2_&iObU0wM63=1B2&kv8>)4yc6q`iYn3Xjy8 z#Tv~11|H!U9q|Z(YXGo+;D9{x7_TN&yNzUf%lPd_{gML31*rBhN=iV<+F>BMviw+) zGtn!R7Z@*+I&_uP?E^>A?UG96XmrIDXedMmWky*QxHJaZMVVo#hP$N5oBi4NE3H#K zW1w)bRoUU7&@G|R1-u7k2Q}kmReD&TR~7ZuR`({+Y$WUI^+o@6#h&#hURCV5lKSH1 zHPH>4?3taU88S6EaD|Ei%G-LR-_`2ejrO+4-N@o6rE_{{UKTt;M!(Q>_faCyi!AF{ zZdaAU-qeu=;L2{osZ`h-ci$cAr?2Ts_!_DK6OSe@3+iO^>R_KCu2Z>wH2&wD*Y*)Gz!~cAa(k& zVUEs0)WH^u^+7d9p{>Jr`h>IS1O%~;>QnQfod(zIcV;@)>7AHr>s9Zm8UR7HllYVb zQJRWk<+-89pvY5IX_cRAZ@)`yCubqf8x?choWeV|as1qpYrBr?x^?_l-AbeKf5maU Ab^rhX diff --git a/tests/unit/__pycache__/test_safety_gates.cpython-310-pytest-8.3.2.pyc b/tests/unit/__pycache__/test_safety_gates.cpython-310-pytest-8.3.2.pyc deleted file mode 100644 index 411ff5b65ae68896a3d47b116c7023139454f942..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 3092 zcma)8Npl-T6rP?MXj^7TB}jGzz} z>qTWVgNd+IFNKr!$*^26E1MNeh12!vaHc*J&ems@%?{?mO1%Q|DVnE+=VpB#q#~UF zX#u1XodjtSq%xfXX^G|NG@YTdFLXM`mT8WXrb#Pw{<+yF^%F2|fiA+hlShtQqRVRB zDHwNxo`i9yDS4ndr~iQSYBh&QtL-I?Z4jpJwOMo{@F=?(M@_%w7_xH194RYIG_?)>M7Ie3>6{XEt(&149PcZ)ELAeReT2Pd*%Rq`X%_&gkRr4k37)UHL<)y2=ox`D|pf7m^EpJapUnrX?Tst zFD+@ecM~QOSl;sm6CA0-aZQ5iff)Sk~a9jd`mN|r=Lo&em zBJ@2}J9BgeEuRI;ksGWYyFt-uG#I18-PvZ5%OXY3Wn5Z^rb0N%2lhw6FeHz-3ks44 zcR0{{TB7e6iJ1|}qIy^B>w7je20)l2h;-&CBH7?MV7<)WzjJSGZT;@mOxn16)yeS! ztSIv>7BLP~;M1V;8EgyI(dT4ohjkxj^#{lf-QxV8aJN-3RYJC`0E` zlUgs#9!bcao&a1b5>0A!2>@sv3*gy90NOQ4p7IkgH9rZ>@9@I&*T*kSeg*_th}(!9 z=9F^c@fJ&l$CX9)v<(SFxL)Ge3g|I-F>*%m0^`o10jGQ&;WRS;8{u>%>M9q#36m&z zr}Ye82P}t3w;vQM(0h<{8PNV2(eP`hV#jjq|zdREsGH4Gn`>sn1h z^Js7C1CtgKbI%%RPc-o^^xIuBAbbs^97+*^ew5+l>L0kH{vily@dtB3>)CXImb$i2 zK+Yw3=q+?}1JWnoLhUxSo(``8uhlaZa+5WV9vQ*M-$L~~G`E1?x-T5^?DqK!?cGa` zD1MU(D<7_)v@tY+keSvTQS}AJgHMaS3r?}O;tMtSiX#+*98((Y-8F1Bu(^cIm1kv7 z2*wjN^qM2kc+Dwh8DKc0EN-Ty#`UI*V-neo(RnmE{rkhQb^|jZj=_qPubx;M43n2KB0yfLvDuC%KxJprUTHhml z6SM=t1o|imy${NM`G<>G0{OeI^^eZi3y0DTsX5kImQZCLFOl59W)xgX8cV=OURH)F z*pFPwnqDVJ(hS|Y43mkpc9*brUk4ph&;V0}sLJrVC(<&c zV8H##v>K4`Fwe zG^u>4bMGJM!8!)$FWPH?B0c9;ppTiQr6{rrlmds}&VHPondQtX9~r4=a5a8)k78TX ze#1fkRffSGxcToOs76Vm`9wYS)X+7erKWF@V#fAun9-9`>iCY*8A&;<_?5KkS5wz_ zmCj5?(wbjONBz;X?$^^Xe+={&s7*^RO@ADaL(70B099xe&?F$2jsThhRHLJSrUBLI z7@(QRKAJt9qvKKa=tATk&C`iP;|#w?Cwq7S@e`w?)2XORrz4lnL>H+=$)Q7M>D){6 zg!q>z+19-IKOk0Z!y|HjJ$o7^G2IrSh;~?*@mR!pw#Hb_JVVy*hfJ`TK8X}TR#sX3 zw2L3+txPDrXUh6Pm<6pY$wL}YB*_J37|aF-T_b_$tsv)m(~3TPD#a!Gr)(C97RwIM z|NhH>!F{;jjDrxGAV)gf#)*zQvaP)$8=lEZFvpN7OY0lK_JhqG=>S;WTK{_OmBvPK z8pWU2msy^R<;`Z4-Ah6mEsKZ?zTC=UprRH;a30Z*(YiP(LVET;?Vw|us zc$^Eu0i_D!HUOcW5}|j<&vl{+LzqVv+%^XE7(z1PcVZO5t!RAq86ohg5^WD|44AoK z!YfI;`5e8HHVb1OafZvk%>%2!P&#YhJlfpaSzCRZfdbWW_Ogm9EUVHdaNUyfYaF%uEo{T6_3tXHVJ>Ht(+fha>ZS=^gxJrqK{?J(~w{ zZt-qn*#uIG)UZj!Qz%qYdcLqZ&`I;&?)Ki$ge<(1kU>u_put%b3n0#UGBgd?f|vzD zmUlNEZtiZtUdYNB2(}&%b)!EWgJp6F4XSiB0G)H<|D@xsPuA-C*3KSNHhU$~_kp~d zk3m=7M^{unt^k6TQY|v3Do9ofdNFXKADYpDi0ICjOuhC2zPpGKVA{GDR&h51{u%~C zYin(Cq8}S=qpi2JW3z4YOpsHZlD2tBsE+zWy<<{C7^h}Od#dpU%v(b15Vi=&MwAh1 zP_xHaK}H)HH;{n{7$&uPjC+bffbo@L00M@ku+bsZ?lB)LCR8)~R?PweCL9RASkv0E zg$F|0iV0v_lz`#1?G8C1Kfys{YRAU0i8bw2uw&~n_sEM|i&vY^uY0`m;~8jt(LnUE z$FJgZ#p53;#Wjz6mE%Ut)#!%D8CL2*emzSX@GgoK6xUE(M{(mtE#y36LcM;|4AY^h3dP9-~ENZfYJZ*+7k}gaK`>i+;amJ;cglZc0`6;9+(Za>lVuye_j_@bGko|0*`ThIG0CvSrZ7xu8hNQsD-;xx}Jp zEEW)6PbEudI`$E~qdJkC$>Frw$J1ts=4lvbOA#KzMav#Co8~l1xH`^M3oRPRc`K^6 z&URfLfLJ?Bl{YqS)vZ<6R1J$YP=}=|%`{}+%PQ<7><#=0co8wx+%6(Rb%V=_wx8#T zN=jcURi9A#bUp`1TS;`A0GVS&U(sBBlFX8dQ!y)6aod(oT-_xu(f`u_ZaDt}Vl!&q From ea6457ee546693cc33fbd4c634e499d6e97bd85e Mon Sep 17 00:00:00 2001 From: Amantux Date: Sat, 26 Sep 2026 05:54:04 +0000 Subject: [PATCH 02/54] ci(mhvtl): load st/ch, fix the sg_inq permission trap, fail loudly MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Evidence from run 35688121678 (and identically 34930111187). All three documented assumptions HELD — headers 6.17.0-1022-azure installed, "SecureBoot disabled" + sig_enforce N, PID 1 is systemd, module built and loaded, mhvtl registered 1 mediumx + 3 tape and mtx status listed all 12 slots. setup.sh printed "=== Rig is up ===". The real chain was three separate things, none of them the ones we expected: 1. No /dev/nst nodes. lsscsi showed "-" in the block column for every tape row and the evidence step reported "ls: cannot access '/dev/nst*': No such file or directory". mhvtl only registers SCSI targets; /dev/nstN comes from the kernel's `st` upper-level driver, which the Azure kernel keeps in linux-modules-extra-$(uname -r) (not installed on the runner image) rather than in linux-modules-* as the generic kernel does. A fourth assumption nobody had written down. setup.sh now modprobes st (apt-installing linux-modules-extra as a fallback) and hard-fails if it cannot, tries `ch` best-effort, waits for the scsi_tape class entries, and asserts an nst node per drive before claiming success. 2. env.sh reported the WRONG cause. "found medium changer(s) /dev/sg1 but none reporting unit serial 'OBLADE_L10'" on a rig whose device.conf says exactly OBLADE_L10. /dev/sg* is 0660 root:tape; the runner user is not in `tape`, so sg_inq returned EACCES, whose stderr the serial lookup swallowed — an empty serial is indistinguishable from someone else's library. rig_sg_inq now escalates via `sudo -n`, and rig_changer reports a permission problem as a permission problem. 3. LTFS build failed on a missing dependency, not on the documented icu or submodule blockers: "configure: error: Package requirements (net-snmp >= 5.3) were not met" after "net-snmp-config: command not found". Added libsnmp-dev. Runtime libs are now installed one per apt-get call because noble renamed libfuse2 -> libfuse2t64 and one unknown name in a batched install silently installs nothing behind `|| true`. Loudness: the suite step's `eval "$(scripts/mhvtl/env.sh)"` swallowed the failure — a failing command substitution does not fail the enclosing command under set -e, so env.sh's refusal became `eval ""` and the first red line was `OPENBLADE_BACKEND: unbound variable` well after the real diagnosis. Resolution is now its own step that checks the exit status and ::error::s. setup.sh identifies the changer with rig_changer (serial-keyed) so whatever makes env.sh refuse fails at bring-up instead. --- .github/workflows/mhvtl-weekly.yml | 74 ++++++++++++++++++++++-- scripts/mhvtl/_rig.sh | 73 +++++++++++++++++++++++- scripts/mhvtl/env.sh | 21 ++----- scripts/mhvtl/setup.sh | 92 ++++++++++++++++++++++++++++-- 4 files changed, 234 insertions(+), 26 deletions(-) diff --git a/.github/workflows/mhvtl-weekly.yml b/.github/workflows/mhvtl-weekly.yml index b6d4e6c..571e088 100644 --- a/.github/workflows/mhvtl-weekly.yml +++ b/.github/workflows/mhvtl-weekly.yml @@ -126,6 +126,18 @@ jobs: echo "--- systemd (assumption 3):" systemctl is-system-running 2>&1 || true ps -p 1 -o comm= || true + # Assumption 4, added after run 35688121678: mhvtl only registers SCSI + # targets; /dev/nstN comes from the kernel's `st` upper-level driver, + # which an Azure kernel keeps in linux-modules-extra rather than the + # base linux-modules package the generic kernel uses. + echo "--- upper-level SCSI drivers st/ch (assumption 4):" + modinfo st 2>&1 | head -3 || echo "st: NOT in this kernel's module set" + modinfo ch 2>&1 | head -3 || echo "ch: NOT in this kernel's module set" + apt-cache policy "linux-modules-extra-$(uname -r)" || true + # Assumption 5: /dev/sgN is 0660 root:tape, and env.sh runs sg_inq as + # the unprivileged runner user. + echo "--- identity of the user that will run env.sh (assumption 5):" + id || true echo "--- disk / cpu budget:" df -h / || true nproc || true @@ -160,9 +172,17 @@ jobs: timeout-minutes: 20 run: | set -euxo pipefail + # libsnmp-dev added after run 35688121678: ltfs configure looks for + # net-snmp >= 5.3 and there is no --disable-snmp, so without it the + # build dies at + # ./configure: line 14786: net-snmp-config: command not found + # configure: error: Package requirements (net-snmp >= 5.3) were not met + # libsnmp-dev is what provides net-snmp-config and the pkg-config file. + # The rehearsal host already had it installed for unrelated reasons, + # which is why docs/runbooks/mhvtl-rehearsal.md §1 never mentions it. sudo apt-get install -y --no-install-recommends \ build-essential autoconf automake libtool pkg-config git \ - libfuse-dev libicu-dev libxml2-dev uuid-dev + libfuse-dev libicu-dev libxml2-dev uuid-dev libsnmp-dev # ./configure probes `icu-config`, removed in ICU 62+. The documented # ICU_MODULE_CFLAGS / ICU_MODULE_LIBS escape hatch does NOT work — @@ -203,9 +223,24 @@ jobs: continue-on-error: true run: | set -euxo pipefail + # No tarball means the build step failed (or was skipped). Say so and + # stop, rather than emitting a confusing `tar: Cannot open` on top of + # the real error further up the log. + if [ ! -f /tmp/ltfs-stage.tar ]; then + echo "No /tmp/ltfs-stage.tar — the LTFS build did not produce one. See the build step above." + exit 1 + fi # The cached tarball only carries the build products; the shared # libraries it links against still have to be present at runtime. - sudo apt-get install -y --no-install-recommends libfuse2 libicu-dev libxml2 uuid-runtime || true + # Installed ONE AT A TIME on purpose: these names are release-specific + # (noble renamed libfuse2 -> libfuse2t64 and bumps libsnmpNN), and a + # single apt-get with one unknown name installs NOTHING while the + # trailing `|| true` hides it. Per-package means a wrong guess costs + # only that guess. + for pkg in libfuse2t64 libfuse2 libicu-dev libxml2 uuid-runtime \ + libsnmp40 libsnmp-dev; do + sudo apt-get install -y --no-install-recommends "$pkg" || true + done sudo tar -C / -xf /tmp/ltfs-stage.tar sudo ldconfig command -v mkltfs @@ -246,6 +281,31 @@ jobs: sudo dmesg | tail -80 || true cat /etc/mhvtl/device.conf || true + # Resolving the rig is its own step so that "the rig cannot be addressed" + # fails HERE, loudly, instead of leaking into the pytest step. + # + # Run 35688121678 is exactly why. The old code was + # `eval "$(scripts/mhvtl/env.sh)"` inside the pytest step: a failing + # command substitution does NOT fail the enclosing command even under + # `set -e`, so env.sh's refusal became `eval ""`, the step carried on, and + # the first visible symptom was `OPENBLADE_BACKEND: unbound variable` 20 + # lines later — with the real diagnosis scrolled off as mere stderr. + - name: Resolve the rig's device paths + id: rig-env + run: | + set -euo pipefail + # env.sh DISCOVERS the device paths (they move between boots) and + # identifies the changer by unit serial, never by "first mediumx". + if ! block=$(scripts/mhvtl/env.sh); then + echo "::error::scripts/mhvtl/env.sh could not resolve the rig; the hardware suite cannot run. The diagnosis is in this step's stderr above." + exit 1 + fi + printf '%s\n' "$block" + # Only the `export K=V` lines — env.sh also emits comments, which + # $GITHUB_ENV would reject. + printf '%s\n' "$block" \ + | sed -n 's/^export \([A-Z_][A-Z0-9_]*\)=\(.*\)$/\1=\2/p' >> "$GITHUB_ENV" + - name: Run the hardware suite against the rig timeout-minutes: 20 env: @@ -255,9 +315,13 @@ jobs: OPENBLADE_FAULT_TESTS: ${{ inputs.fault_tests == true && 'enabled' || '' }} run: | set -euxo pipefail - # env.sh DISCOVERS the device paths (they move between boots) and - # identifies the changer by unit serial, never by "first mediumx". - eval "$(scripts/mhvtl/env.sh)" + # Belt and braces: the previous step is what guarantees these exist, + # and if it somehow did not, say which one is missing. + : "${OPENBLADE_BACKEND:?not set — the rig-env step did not run}" + : "${OPENBLADE_REAL_HARDWARE_ENABLED:?not set — the rig-env step did not run}" + : "${OPENBLADE_CHANGER_DEVICE:?not set — the rig-env step did not run}" + : "${OPENBLADE_DRIVE_DEVICES:?not set — the rig-env step did not run}" + : "${OPENBLADE_SCRATCH_BARCODES:?not set — the rig-env step did not run}" env | grep '^OPENBLADE_' | sort if [ "${{ steps.ltfs.outputs.available }}" = "true" ]; then diff --git a/scripts/mhvtl/_rig.sh b/scripts/mhvtl/_rig.sh index 5fd4fe9..62c8d1f 100755 --- a/scripts/mhvtl/_rig.sh +++ b/scripts/mhvtl/_rig.sh @@ -26,11 +26,74 @@ rig_die() { printf 'ERROR: %s\n' "$*" >&2; exit 1; } # resorting to a substring test - a loose match here could select a real # library, which is the whole thing this is guarding against. rig_serial_of() { - sg_inq "$1" 2>/dev/null \ + rig_sg_inq "$1" \ | awk -F': *' '/Unit serial number/{print $2; exit}' \ | awk '{print $NF}' } +# `sg_inq` on a device, escalating to `sudo -n` when the calling user cannot +# open the node. +# +# mhvtl's /dev/sgN nodes are mode 0660 root:tape. On the bare-metal rehearsal +# host the operator was in the `tape` group, so a plain `sg_inq` worked. A +# GitHub-hosted runner's `runner` user is NOT in `tape`, so every `sg_inq` +# failed with EACCES; because the serial lookup swallows stderr, that surfaced +# as an EMPTY serial and rig_changer reported "found medium changer(s) /dev/sg1 +# but none reporting unit serial 'OBLADE_L10'" on a perfectly healthy rig +# (mhvtl weekly run 35688121678). A permission problem must not be +# indistinguishable from "that is somebody else's library". +rig_sg_inq() { + local dev="$1" + if [ -r "$dev" ] && sg_inq "$dev" 2>/dev/null; then + return 0 + fi + [ "$(id -u)" -eq 0 ] && return 1 + command -v sudo >/dev/null || return 1 + sudo -n sg_inq "$dev" 2>/dev/null +} + +# True when this user cannot read the sg node directly and cannot escalate. +# Used to turn "no serial" into an accurate diagnosis instead of a wrong one. +rig_sg_unreadable() { + local dev="$1" + [ -r "$dev" ] && return 1 + [ "$(id -u)" -eq 0 ] && return 1 + sudo -n true 2>/dev/null && return 1 + return 0 +} + +# Echo the no-rewind tape node (/dev/nstN) for an sg node, using the same sysfs +# relationship openblade.hardware.discovery.resolve_sg_device() reads forwards. +# st and sg numbers are allocated independently (st0 -> sg1 and st2 -> sg4 are +# both real pairings on this rig), so this must be looked up, never derived. +# +# Failure means the kernel's `st` upper-level driver has not claimed the device +# — see the "upper-level SCSI drivers" section of setup.sh. +rig_nst_for_sg() { + local want="${1#/dev/}" tape + for tape in /sys/class/scsi_tape/nst*; do + [ -e "$tape/device/scsi_generic/$want" ] || continue + printf '/dev/%s' "$(basename "$tape")" + return 0 + done + return 1 +} + +# Why there is no /dev/nst node. Printed by both setup.sh and env.sh so the +# operator gets the same diagnosis whichever one they hit first. +rig_no_nst_hint() { + cat <<'EOF' +The kernel's SCSI tape upper-level driver ('st') has not attached to the rig's +drives: no /dev/nstN nodes exist and lsscsi prints '-' in the block column. +On Ubuntu's generic kernel st/ch ship inside linux-modules-$(uname -r) and udev +autoloads them, which is why the bare-metal rehearsal host never needed this. +On an Azure-tuned kernel (GitHub-hosted runners) they live in +linux-modules-extra-$(uname -r), which is NOT installed on the runner image. +Fix: sudo apt-get install -y "linux-modules-extra-$(uname -r)" + sudo modprobe st && sudo scripts/mhvtl/setup.sh +EOF +} + # Echo the /dev/sg node of THIS rig's medium changer. rig_changer() { local scsi dev serial candidates="" @@ -52,6 +115,14 @@ EOF rig_die "no medium changer with an sg node found - is the rig up? Run: sudo scripts/mhvtl/setup.sh" fi + for dev in $candidates; do + if rig_sg_unreadable "$dev"; then + rig_die "cannot read $dev to check its unit serial (it is mode 0660 + root:tape and this user is neither root, in the 'tape' group, nor able + to 'sudo -n'). This is a PERMISSION problem, not a wrong library. + Re-run as root, or: sudo usermod -aG tape \$USER (then re-login)." + fi + done rig_die "found medium changer(s)$candidates but none reporting unit serial '$RIG_CHANGER_SERIAL'. Refusing to guess: a real library attached to this host must not be driven by the rehearsal scripts. Check diff --git a/scripts/mhvtl/env.sh b/scripts/mhvtl/env.sh index 9451c0f..4ff06cb 100755 --- a/scripts/mhvtl/env.sh +++ b/scripts/mhvtl/env.sh @@ -29,25 +29,16 @@ changer_sg=$(rig_changer) # issuing SCSI commands on a rewinding node makes it rewind on close, which # fails on an empty drive and is a corruption footgun under LTFS. See Phase 0.3 # of docs/runbooks/real-i3-bringup-plan.md. -# Map an sg node back to its no-rewind tape node using the same sysfs -# relationship openblade.hardware.discovery.resolve_sg_device() reads forwards. -# st and sg numbers are allocated independently (st0 -> sg1 and st2 -> sg4 are -# both real pairings here), so this must be looked up, never derived. -nst_for_sg() { - local want="${1#/dev/}" tape - for tape in /sys/class/scsi_tape/nst*; do - [ -e "$tape/device/scsi_generic/$want" ] || continue - printf '/dev/%s' "$(basename "$tape")" - return 0 - done - return 1 -} +# The sg -> nst lookup lives in _rig.sh (rig_nst_for_sg) so setup.sh can verify +# it at bring-up time rather than leaving the test suite to discover it. drive_nst="" while read -r sg; do [ -n "$sg" ] || continue - node=$(nst_for_sg "$sg") \ - || rig_die "could not map $sg back to a /dev/nst node via sysfs" + if ! node=$(rig_nst_for_sg "$sg"); then + rig_no_nst_hint >&2 + rig_die "could not map $sg back to a /dev/nst node via sysfs (see above)" + fi drive_nst="${drive_nst:+$drive_nst,}${node}" done <&2; exit 1; } +# rig_changer / rig_drive_sgs / rig_nst_for_sg — the SAME helpers env.sh uses. +# Verifying the rig with them at the end of this script is deliberate: whatever +# would make env.sh refuse must fail HERE, where the diagnosis is in reach, not +# three steps later as an unbound-variable error inside the test runner. +# shellcheck source=scripts/mhvtl/_rig.sh +. "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/_rig.sh" + [ "$(id -u)" -eq 0 ] || fail "must run as root (needs modprobe + /etc/mhvtl)" # ---------------------------------------------------------------- packages @@ -115,6 +122,48 @@ else fi module_loaded || fail "mhvtl module is not loaded after modprobe" +# --------------------------------------------- upper-level SCSI drivers (st/ch) +# mhvtl registers SCSI *targets*. The /dev/nstN (tape) and /dev/schN (changer) +# nodes are created by the kernel's UPPER-LEVEL drivers `st` and `ch`, which are +# a separate matter from mhvtl entirely. +# +# Ubuntu's generic kernel ships st/ch inside linux-modules-$(uname -r) and udev +# autoloads them on device attach, which is why the bare-metal rehearsal host +# (5.15.0-174-generic) got /dev/st0-2 and /dev/sch0 for free and this script +# never mentioned them. An Azure-tuned kernel — i.e. every GitHub-hosted runner +# — puts them in linux-modules-extra-$(uname -r), which is NOT installed on the +# runner image. Evidence, mhvtl weekly run 35688121678: lsscsi printed all three +# tape rows with "-" in the block column, /sys/class/scsi_tape was empty, and +# env.sh could not map a single sg node to an nst node. +# +# `st` is REQUIRED: OPENBLADE_DRIVE_DEVICES is a list of /dev/nstN nodes and the +# hardware backend has nothing to talk to without them. `ch` is optional — the +# changer is driven through its sg node via mtx — so it is best-effort. +ensure_scsi_driver() { + local mod="$1" + # A built-in driver has no /proc/modules row; modprobe still exits 0 for it. + modprobe "$mod" 2>/dev/null && return 0 + [ "${SKIP_APT:-0}" = "1" ] && return 1 + log "'$mod' is not in this kernel's module set — trying linux-modules-extra-$(uname -r)" + DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \ + "linux-modules-extra-$(uname -r)" || true + depmod -a || true + modprobe "$mod" 2>/dev/null +} + +log "Ensuring the SCSI tape ('st') and changer ('ch') drivers are loaded" +ensure_scsi_driver st || fail "the kernel's SCSI tape driver 'st' is unavailable + on this host, so no /dev/nstN nodes can ever appear and the OpenBlade + hardware backend has no drive to open. + Running kernel: $(uname -r) + Tried: modprobe st, then apt-get install linux-modules-extra-\$(uname -r). + Check 'apt-cache policy linux-modules-extra-$(uname -r)' and + 'modinfo st'." +ensure_scsi_driver ch \ + || printf "NOTE: the 'ch' changer driver is unavailable; no /dev/schN node.\n\ +This rig drives the changer through its /dev/sg node with mtx, so this is not\n\ +fatal — it only means lsscsi shows '-' in the changer's block column.\n" + # ---------------------------------------------------------------- userspace log "Building and installing mhvtl userspace daemons" make -C "$MHVTL_SRC" @@ -229,12 +278,18 @@ fi # /dev/sg nodes are created by udev a moment after that. Waiting only for the # mediumx ROW is not enough: lsscsi will happily print it with a "-" in the sg # column, and the rig then looks broken. Wait for the sg node itself. +# +# The /dev/nstN nodes are a THIRD, independent event: `st` binds the device and +# then udev creates the node. Waiting only for the sg nodes let setup.sh declare +# "Rig is up" on a rig whose drives had no tape node at all — so wait for the +# scsi_tape class entries too. log "Waiting for SCSI devices and udev to settle" for _ in $(seq 1 30); do sleep 1 changers_sg=$(lsscsi -g 2>/dev/null | awk '/ mediumx /{print $NF}' | grep -c '^/dev/sg' || true) drives_sg=$(lsscsi -g 2>/dev/null | awk '/ tape /{print $NF}' | grep -c '^/dev/sg' || true) - if [ "$changers_sg" -ge 1 ] && [ "$drives_sg" -ge 3 ]; then break; fi + tape_nodes=$(ls -d /sys/class/scsi_tape/nst* 2>/dev/null | grep -c . || true) + if [ "$changers_sg" -ge 1 ] && [ "$drives_sg" -ge 3 ] && [ "$tape_nodes" -ge 3 ]; then break; fi done command -v udevadm >/dev/null && udevadm settle --timeout=30 || true @@ -250,7 +305,14 @@ printf '\nmediumx devices: %s (want >= 1)\ntape devices: %s (want >= 3)\n' \ [ "$changers" -ge 1 ] || fail "no medium changer appeared — check 'ps ax | grep vtl' and dmesg" [ "$drives" -ge 3 ] || fail "expected 3 tape devices, found $drives — check vtltape daemons" -changer_sg=$(lsscsi -g | awk '/ mediumx /{print $NF}' | head -1) +# Identify the changer the way env.sh and every other rig script does: by unit +# serial, never "first mediumx". Two reasons this is the right call HERE as well +# as there. It keeps a real i3 cabled to the same host from being selected +# (Phase 3 of the bring-up plan), and — the reason it moved into this script — +# it makes setup.sh fail on exactly the condition that made env.sh refuse. Run +# 35688121678 printed "Rig is up" and then died two steps later because the +# serial lookup returned nothing. +changer_sg=$(rig_changer) case "$changer_sg" in /dev/sg*) ;; *) fail "the changer has no /dev/sg node yet (lsscsi shows '$changer_sg'). @@ -265,12 +327,32 @@ mtx -f "$changer_sg" status # the "wrote to the wrong drive" trap from Phase 2 of the bring-up plan, so # print the correlation every time rather than letting anyone assume it. log "Drive correlation (SCSI address -> devices -> serial)" -printf '%-14s %-10s %-10s %s\n' "SCSI_ADDR" "BLOCK" "SG" "SERIAL" +printf '%-14s %-10s %-10s %-10s %s\n' "SCSI_ADDR" "BLOCK" "SG" "NST" "SERIAL" lsscsi -g | awk '/ tape /{print $1, $(NF-1), $NF}' | tr -d '[]' | while read -r addr blk sg; do - serial=$(sg_inq "$sg" 2>/dev/null | awk -F': *' '/Unit serial number/{print $2}') - printf '%-14s %-10s %-10s %s\n' "$addr" "$blk" "$sg" "${serial:-}" + serial=$(rig_serial_of "$sg") + nst=$(rig_nst_for_sg "$sg" || true) + printf '%-14s %-10s %-10s %-10s %s\n' \ + "$addr" "$blk" "$sg" "${nst:-}" "${serial:-}" done +# The nst nodes ARE the rig's usable surface: OPENBLADE_DRIVE_DEVICES is a list +# of them, so a rig without them cannot run a single drive-touching test. Assert +# it here, using the same lookup env.sh uses, so "the rig cannot work" fails at +# bring-up with a diagnosis instead of surfacing as an unbound variable in the +# CI step that runs pytest. +missing_nst="" +while read -r sg; do + [ -n "$sg" ] || continue + rig_nst_for_sg "$sg" >/dev/null || missing_nst="${missing_nst} $sg" +done <&2 + fail "no /dev/nst node for:${missing_nst} (see the diagnosis above). + The rig's drives are unusable in this state; refusing to report success." +fi + # ---------------------------------------------------------------- scratch media # The archive/restore jobs mount scratch media without formatting it first, so # blank cartridges fail the mount. Supply formatted scratch media here. From 4256e9fcb3f72d63793ec5e4f258fd7c4b1656e4 Mon Sep 17 00:00:00 2001 From: Amantux Date: Sat, 26 Sep 2026 05:55:58 +0000 Subject: [PATCH 03/54] fix: replace mailslot export --force with typed --confirm-barcode --force let a scripted caller wave a data-carrying export through with a bare boolean, never naming the cartridge it was letting out the door. `openblade mailslot export` now requires --confirm-barcode matching the cartridge exactly (extras: confirmBarcode on POST /tape-ops/execute); a mismatch still refuses. Blank/scratch-tape exports are unaffected -- they were never gated by --force. Mutation-checked: relaxing the equality check to a truthiness check makes test_confirm_barcode_must_match_exactly and test_export_refuses_a_mistyped_confirm_barcode fail; restored after confirming. Regenerated docs/wiki/reference/cli.md (tools/gen_wiki_reference.py) for the flag rename, and updated docs/runbooks/mailslot-and-bulk-restore.md which had documented this as open tech debt. --- docs/runbooks/mailslot-and-bulk-restore.md | 26 ++++++++------ docs/wiki/reference/cli.md | 2 +- openblade/cli/main.py | 14 +++++--- openblade/nas/mailslot.py | 4 +-- openblade/nas/tape_orchestrator.py | 12 ++++--- tests/unit/test_cli_new_commands.py | 35 ++++++++++++++++-- tests/unit/test_mailslot_service.py | 41 ++++++++++++++++------ 7 files changed, 99 insertions(+), 35 deletions(-) diff --git a/docs/runbooks/mailslot-and-bulk-restore.md b/docs/runbooks/mailslot-and-bulk-restore.md index 70b7b05..1459eda 100644 --- a/docs/runbooks/mailslot-and-bulk-restore.md +++ b/docs/runbooks/mailslot-and-bulk-restore.md @@ -22,7 +22,7 @@ export OPENBLADE_DRIVE_SERIAL_MAP="OBLADE_D01:0,OBLADE_D02:1,OBLADE_D03:2" |---|---| | `openblade mailslot list` | Every I/E element and what is in it. | | `openblade mailslot import [--to-slot N]` | I/E → storage. Without `--to-slot` it picks the first empty storage slot **and names it** in the output. | -| `openblade mailslot export [--ie-slot N] [--force]` | Storage → first empty I/E. **Refuses** when the cartridge itself carries archived data, or a write to it is still in flight. | +| `openblade mailslot export [--ie-slot N] [--confirm-barcode BARCODE]` | Storage → first empty I/E. **Refuses** when the cartridge itself carries archived data, or a write to it is still in flight. | | `openblade restore tree --dest DIR [--dry-run]` | Every archived file under a prefix, spanning tapes. | | `openblade restore file --dest PATH [--into-dir]` | One file, using the sharded reassembly path when the catalog says it is sharded. `--dest` is a file path unless it is an existing directory or `--into-dir` is given. | | `openblade archive sharded --volume-group G --mode stripe\|block-stripe [--lanes N \| --lane-barcode B …] [--block-size-mb N]` | The CLI half of `POST /archive/sharded`. | @@ -126,8 +126,8 @@ stdout bytes = 0 Export refused: Cartridge OB0001L8 still carries archived data: 4 file instance(s), 250028 bytes; volume group rigtree; e.g. /rigtree/alpha/same.txt, /rigtree/beta/deep/n.txt, /rigtree/beta/same.txt, /rigtree/blob.bin. Exporting -makes these unrestorable until the cartridge is imported again. Pass --force if -that is what you mean. +makes these unrestorable until the cartridge is imported again. Pass +--confirm-barcode OB0001L8 if that is what you mean. $ mtx -f /dev/sg2 status | grep -E "Element 1:|IMPORT" Storage Element 1:Full :VolumeTag=OB0001L8 @@ -151,7 +151,8 @@ exit=0 The sibling situation is reported as context, not used to refuse. An earlier draft *did* refuse here, with the sentence "still carries archived data" — which -was false, and would have taught operators that `--force` is how you export. +was false, and would have taught operators that `--confirm-barcode` is how you +export. ### 2.4 sharded archive + tree restore across three tapes @@ -256,12 +257,17 @@ nothing can load or unload to it by accident. failed archive leaves exactly that shape; the summary carries `filesSkipped` and `skippedPaths` and the CLI says so on stderr, because a tree that silently comes back short is the failure this command exists to prevent. -- **`--force` is still a plain flag, not a two-phase token.** `openblade format` - requires a dry-run plan plus a one-time `SafetyToken`; export does not, and an - API caller can pass `extras: {"force": true}` as a bare JSON boolean. Export - destroys *availability* rather than data, and the operator is shown exactly - what is on the cartridge first — but the asymmetry with FORMAT is deliberate - only in the sense that nobody has decided otherwise. Worth a decision. +- **Resolved:** export used to accept a bare `--force` flag (and an API caller + could pass `extras: {"force": true}` as a plain JSON boolean) to override the + data-carrying refusal. That has been replaced with typed confirmation: + `--confirm-barcode ` on the CLI, `extras: {"confirmBarcode": + ""}` on `POST /tape-ops/execute`, and it must equal the cartridge's + own barcode exactly — a mismatch (or a merely truthy value) still refuses. + This does not make export a full two-phase token like `openblade format` + (no dry-run plan, no one-time `SafetyToken`); the asymmetry is unchanged, + it's just that overriding the refusal now requires naming the cartridge + instead of flipping a boolean. Blank/scratch-tape exports are unaffected — + they were never gated by `--force` and stay friction-free. - **`docs/wiki/reference/cli.md` is stale on this branch** and `tests/unit/test_wiki_reference_generated.py` is red because of it. The page is a build artifact of `tools/gen_wiki_reference.py`; regenerating it is owned diff --git a/docs/wiki/reference/cli.md b/docs/wiki/reference/cli.md index 35e4ef9..e9a0e4d 100644 --- a/docs/wiki/reference/cli.md +++ b/docs/wiki/reference/cli.md @@ -137,7 +137,7 @@ Move a cartridge out of storage into the I/E station. | --- | --- | --- | --- | --- | --- | | `BARCODE` | argument | `STR` | yes | - | Barcode of the cartridge to export | | `--ie-slot` | option | `INT` | no | none | Import/export element to use (default: first empty) | -| `--force` | option | `BOOLEAN` | no | false | Export even though the cartridge carries archived data | +| `--confirm-barcode` | option | `STR` | no | none | Export even though the cartridge carries archived data. Must equal the cartridge's own barcode exactly. | ### `openblade mailslot import` diff --git a/openblade/cli/main.py b/openblade/cli/main.py index d13e39b..939ffd6 100644 --- a/openblade/cli/main.py +++ b/openblade/cli/main.py @@ -880,8 +880,13 @@ def mailslot_export( ie_slot: int | None = typer.Option( None, "--ie-slot", help="Import/export element to use (default: first empty)" ), - force: bool = typer.Option( - False, "--force", help="Export even though the cartridge carries archived data" + confirm_barcode: str | None = typer.Option( + None, + "--confirm-barcode", + help=( + "Export even though the cartridge carries archived data. Must equal " + "the cartridge's own barcode exactly." + ), ), ) -> None: """Move a cartridge out of storage into the I/E station. @@ -889,11 +894,12 @@ def mailslot_export( This is how data walks out of the library: once exported, every file on that cartridge is unrestorable until it is imported back. It refuses by default when the cartridge (or its volume group) still holds archived files, and - names what is on it. + names what is on it. Overriding that refusal requires typing the cartridge's + own barcode via ``--confirm-barcode``, not a bare flag. """ context, service = _mailslot_service() try: - result = service.export_cartridge(barcode, ie_slot=ie_slot, force=force) + result = service.export_cartridge(barcode, ie_slot=ie_slot, confirm_barcode=confirm_barcode) except ExportRefusedError as exc: err_console.print(f"[red]Export refused:[/red] {exc}") raise typer.Exit(code=1) from None diff --git a/openblade/nas/mailslot.py b/openblade/nas/mailslot.py index 5657fc8..014491c 100644 --- a/openblade/nas/mailslot.py +++ b/openblade/nas/mailslot.py @@ -160,7 +160,7 @@ def import_cartridge(self, ie_slot: int, to_slot: int | None = None) -> Mailslot ) def export_cartridge( - self, barcode: str, *, ie_slot: int | None = None, force: bool = False + self, barcode: str, *, ie_slot: int | None = None, confirm_barcode: str | None = None ) -> MailslotMoveResult: """Move a cartridge from storage into the first empty I/E element.""" assessment = self.preview_export(barcode) @@ -189,7 +189,7 @@ def export_cartridge( barcode=barcode, slot_id=source_slot, requested_by="mailslot", - extras={"ie_slot": target_ie, "force": force}, + extras={"ie_slot": target_ie, "confirmBarcode": confirm_barcode}, ) ) self._raise_if_failed(record, "export") diff --git a/openblade/nas/tape_orchestrator.py b/openblade/nas/tape_orchestrator.py index e3de034..0b9c906 100644 --- a/openblade/nas/tape_orchestrator.py +++ b/openblade/nas/tape_orchestrator.py @@ -431,8 +431,10 @@ def _export(self, request: TapeOpRequest) -> dict[str, Any]: """Move media from a storage slot into an import/export element. The data guard lives HERE rather than only in the service, because this - is the choke point every surface goes through. ``extras["force"]`` is - the single documented override. + is the choke point every surface goes through. ``extras["confirmBarcode"]`` + is the single documented override, and it must equal ``request.barcode`` + exactly -- a boolean flag let a scripted caller wave every export + through without ever naming the cartridge it was letting out the door. The source slot is resolved FROM THE BARCODE, never taken from ``request.slot_id``. Trusting that field meant the guard assessed one @@ -488,7 +490,8 @@ def _record_cartridge_state(self, barcode: str, state: str) -> None: setter(barcode, state) def _guard_export(self, request: TapeOpRequest) -> None: - if request.extras.get("force") is True: + confirm_barcode = request.extras.get("confirmBarcode") + if confirm_barcode is not None and str(confirm_barcode) == request.barcode: return assess = getattr(self.repo, "list_instances_for_barcode", None) if assess is None: @@ -496,7 +499,8 @@ def _guard_export(self, request: TapeOpRequest) -> None: # cartridge. Fail closed: an unknown payload is not an empty one. raise ExportRefusedError( f"Cannot determine what is on cartridge {request.barcode} without a " - "catalog; refusing to export. Re-run against the catalog, or force." + "catalog; refusing to export. Re-run against the catalog, or pass " + "--confirm-barcode matching the cartridge exactly." ) from openblade.catalog.export_policy import assess_export diff --git a/tests/unit/test_cli_new_commands.py b/tests/unit/test_cli_new_commands.py index 0b20a6e..1264829 100644 --- a/tests/unit/test_cli_new_commands.py +++ b/tests/unit/test_cli_new_commands.py @@ -176,7 +176,7 @@ def test_a_failed_file_exits_non_zero_with_the_curated_error(self, cli_home, tmp seed_tree(source) invoke("archive", "--volume-group", "photos", "--path", str(source)) # Take the tape out of the library, forcibly. - invoke("mailslot", "export", "MCK00001", "--force") + invoke("mailslot", "export", "MCK00001", "--confirm-barcode", "MCK00001") result = invoke("restore", "tree", "/photos", "--dest", str(tmp_path / "out")) @@ -393,13 +393,15 @@ def test_export_refuses_a_cartridge_with_data_and_names_it(self, cli_home, tmp_p assert result.stdout.strip() == "" assert stdout_json(invoke("mailslot", "list"))["occupiedCount"] == 0 - def test_export_with_force_reports_what_went_out_of_the_door(self, cli_home, tmp_path) -> None: + def test_export_with_matching_confirm_barcode_reports_what_went_out_of_the_door( + self, cli_home, tmp_path + ) -> None: bootstrap(cli_home) source = tmp_path / "src" seed_tree(source) invoke("archive", "--volume-group", "photos", "--path", str(source)) - result = invoke("mailslot", "export", "MCK00001", "--force") + result = invoke("mailslot", "export", "MCK00001", "--confirm-barcode", "MCK00001") assert result.exit_code == 0, result.output payload = stdout_json(result) @@ -407,6 +409,33 @@ def test_export_with_force_reports_what_went_out_of_the_door(self, cli_home, tmp assert payload["exported"]["archivedFilesOnCartridge"] == 3 assert payload["exported"]["volumeGroup"] == "photos" + def test_export_refuses_a_mistyped_confirm_barcode(self, cli_home, tmp_path) -> None: + """``--confirm-barcode`` must equal the cartridge exactly -- a typo, or + the barcode of a different cartridge, must still refuse.""" + bootstrap(cli_home) + source = tmp_path / "src" + seed_tree(source) + invoke("archive", "--volume-group", "photos", "--path", str(source)) + + result = invoke("mailslot", "export", "MCK00001", "--confirm-barcode", "MCK00002") + + assert result.exit_code == 1 + assert "Export refused" in result.stderr + assert result.stdout.strip() == "" + + def test_export_no_longer_accepts_a_force_flag(self, cli_home, tmp_path) -> None: + """``--force`` was replaced by typed ``--confirm-barcode``; it must be gone.""" + bootstrap(cli_home) + source = tmp_path / "src" + seed_tree(source) + invoke("archive", "--volume-group", "photos", "--path", str(source)) + + result = invoke("mailslot", "export", "MCK00001", "--force") + + assert result.exit_code != 0 + combined = (result.stderr + result.output).lower() + assert "no such option" in combined + def test_import_from_an_empty_element_exits_one_with_a_typed_error(self, cli_home) -> None: bootstrap(cli_home) result = invoke("mailslot", "import", "8") diff --git a/tests/unit/test_mailslot_service.py b/tests/unit/test_mailslot_service.py index 8f1137b..ce1a840 100644 --- a/tests/unit/test_mailslot_service.py +++ b/tests/unit/test_mailslot_service.py @@ -110,7 +110,7 @@ def test_honours_an_explicit_destination_slot(self) -> None: def test_clears_the_exported_flag_so_restores_work_again(self) -> None: repo, library, service = make_service() archive_one_file(repo, library, "OB0001L8", "/photos/a.txt") - service.export_cartridge("OB0001L8", force=True) + service.export_cartridge("OB0001L8", confirm_barcode="OB0001L8") assert repo.get_cartridge("OB0001L8").state == "exported" service.import_cartridge(7) @@ -165,7 +165,7 @@ def test_export_refuses_a_cartridge_carrying_data(self) -> None: Verified by deleting the ``if assessment.carries_data: raise`` block in ``TapeOperationOrchestrator._guard_export``: this test then fails while - ``test_export_with_force_moves_the_cartridge_anyway`` still passes. + ``test_export_with_matching_confirm_barcode_moves_the_cartridge_anyway`` still passes. """ repo, library, service = make_service() archive_one_file(repo, library, "OB0001L8", "/photos/holiday.jpg") @@ -190,8 +190,8 @@ def test_a_blank_tape_in_an_active_group_exports_without_a_fight(self) -> None: cartridge holding part of a striped file has instances of its own and the real check catches it. A cartridge with zero instances carries nothing -- and refusing it with the sentence "still carries archived - data" was both false and a good way to teach operators that `--force` - is the normal way to export. (Adversarial review finding.) + data" was both false and a good way to teach operators that + `--confirm-barcode` is the normal way to export. (Adversarial review finding.) """ repo, library, service = make_service() group = repo.create_volume_group("shards") @@ -220,11 +220,11 @@ def test_export_refuses_while_a_write_to_the_cartridge_is_in_flight(self) -> Non assert library.find_slot_by_barcode("OB0001L8") == 1 - def test_export_with_force_moves_the_cartridge_anyway(self) -> None: + def test_export_with_matching_confirm_barcode_moves_the_cartridge_anyway(self) -> None: repo, library, service = make_service() archive_one_file(repo, library, "OB0001L8", "/photos/holiday.jpg") - result = service.export_cartridge("OB0001L8", force=True) + result = service.export_cartridge("OB0001L8", confirm_barcode="OB0001L8") assert result.destination_slot == 7 assert library.find_slot_by_barcode("OB0001L8") is None @@ -232,11 +232,30 @@ def test_export_with_force_moves_the_cartridge_anyway(self) -> None: assert result.assessment is not None assert result.assessment.archived_files_on_cartridge == 1 + def test_confirm_barcode_must_match_exactly(self) -> None: + """MUTATION ANCHOR. A mismatched (or merely truthy) confirm_barcode must + not bypass the guard -- that would just be ``--force`` wearing a string. + + Verified by changing the equality check in + ``TapeOperationOrchestrator._guard_export`` to accept any non-empty + value (e.g. ``if confirm_barcode:``): this test then fails while + ``test_export_with_matching_confirm_barcode_moves_the_cartridge_anyway`` + still passes. + """ + repo, library, service = make_service() + archive_one_file(repo, library, "OB0001L8", "/photos/holiday.jpg") + + with pytest.raises(ExportRefusedError): + service.export_cartridge("OB0001L8", confirm_barcode="OB0002L8") + + assert library.find_slot_by_barcode("OB0001L8") == 1 + assert repo.get_cartridge("OB0001L8").state != "exported" + def test_export_marks_the_catalog_cartridge_offline(self) -> None: repo, library, service = make_service() archive_one_file(repo, library, "OB0001L8", "/photos/holiday.jpg") - service.export_cartridge("OB0001L8", force=True) + service.export_cartridge("OB0001L8", confirm_barcode="OB0001L8") # This is the flag jobs/restore.py and jobs/sharded_restore.py read. assert repo.get_cartridge("OB0001L8").state == "exported" @@ -255,7 +274,7 @@ def test_export_marks_a_cartridge_the_catalog_has_never_seen(self) -> None: repo.mark_instance_archived(instance.id) assert repo.get_cartridge("OB0001L8") is None - service.export_cartridge("OB0001L8", force=True) + service.export_cartridge("OB0001L8", confirm_barcode="OB0001L8") assert repo.get_cartridge("OB0001L8").state == "exported" @@ -269,7 +288,7 @@ def test_exported_data_actually_becomes_unrestorable(self) -> None: service = MailslotService(repo, library, ltfs) archive_one_file(repo, library, "OB0001L8", "/photos/holiday.jpg") - service.export_cartridge("OB0001L8", force=True) + service.export_cartridge("OB0001L8", confirm_barcode="OB0001L8") job = repo.create_job("restore", {}) with pytest.raises(CartridgeOfflineError): @@ -409,7 +428,7 @@ def test_export_to_a_non_ie_element_is_refused(self) -> None: TapeOpRequest( op_type=TapeOpType.EXPORT, barcode="OB0002L8", - extras={"ie_slot": 3, "force": True}, # 3 is a STORAGE slot + extras={"ie_slot": 3, "confirmBarcode": "OB0002L8"}, # 3 is a STORAGE slot ), ) assert record.status is TapeOpStatus.FAILED @@ -500,7 +519,7 @@ def test_the_catalog_flag_is_written_by_the_orchestrator_not_the_service( TapeOpRequest( op_type=TapeOpType.EXPORT, barcode="OB0001L8", - extras={"ie_slot": 5, "force": True}, + extras={"ie_slot": 5, "confirmBarcode": "OB0001L8"}, ), ) assert record.status is TapeOpStatus.COMPLETED From c3a1615ff91431b421ceac45776bcd22aa453276 Mon Sep 17 00:00:00 2001 From: Amantux Date: Sat, 26 Sep 2026 06:00:41 +0000 Subject: [PATCH 04/54] feat(frontend): own the native API bearer token in one module `src/lib/apiToken.ts` is the browser half of `openblade/api/api_auth.py`: it stores the token, mirrors the backend's native-vs-AML path classification (including the `/aml/proxy` exception), and decides which 401s belong to the token gate rather than the AML session. Two invariants keep auth-disabled deployments identical: no stored token means no header is ever added, and nothing prompts until a real 401 arrives. Session storage is the default slot so a token is forgotten when the tab closes; "remember" is the explicit localStorage opt-in. Tests mutation-checked: forcing `isNativeApiUrl` to true fails the "never attaches the native token to an AML request" case. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_013S1qbQs31YSox1azQChqs2 --- frontend/src/lib/apiToken.test.ts | 189 +++++++++++++++++++++++++ frontend/src/lib/apiToken.ts | 223 ++++++++++++++++++++++++++++++ 2 files changed, 412 insertions(+) create mode 100644 frontend/src/lib/apiToken.test.ts create mode 100644 frontend/src/lib/apiToken.ts diff --git a/frontend/src/lib/apiToken.test.ts b/frontend/src/lib/apiToken.test.ts new file mode 100644 index 0000000..915e073 --- /dev/null +++ b/frontend/src/lib/apiToken.test.ts @@ -0,0 +1,189 @@ +import { beforeEach, describe, expect, it, vi } from 'vitest'; +import { + API_TOKEN_REQUIRED_EVENT, + apiTokenHeaderValue, + attachApiTokenHeader, + clearApiToken, + getApiToken, + handleUnauthorized, + isNativeApiUrl, + resetApiTokenCache, + setApiToken, +} from './apiToken'; + +function tokenGateResponse() { + return { headers: new Headers({ 'WWW-Authenticate': 'Bearer' }) }; +} + +function amlSessionResponse() { + return { headers: new Headers() }; +} + +describe('apiToken', () => { + beforeEach(() => { + window.localStorage.clear(); + window.sessionStorage.clear(); + resetApiTokenCache(); + }); + + describe('surface classification', () => { + it('treats the OpenBlade-native surface as native', () => { + expect(isNativeApiUrl('/api/nas/shares')).toBe(true); + expect(isNativeApiUrl('/api/pools/p1/files')).toBe(true); + expect(isNativeApiUrl('/inventory')).toBe(true); + expect(isNativeApiUrl('http://localhost:8000/jobs/')).toBe(true); + }); + + it('excludes the AML emulator surface, which keeps its own session auth', () => { + expect(isNativeApiUrl('/aml/users/me')).toBe(false); + expect(isNativeApiUrl('/aml')).toBe(false); + expect(isNativeApiUrl('/iblade/system')).toBe(false); + expect(isNativeApiUrl('http://localhost:8000/aml/partitions?x=1')).toBe(false); + }); + + it('does not mistake a lookalike prefix for the AML surface', () => { + expect(isNativeApiUrl('/amlfoo/bar')).toBe(true); + }); + + it('mirrors the backend exception for native routes mounted under /aml', () => { + expect(isNativeApiUrl('/aml/proxy')).toBe(true); + expect(isNativeApiUrl('/aml/proxy/request')).toBe(true); + }); + }); + + describe('header attachment', () => { + it('omits the Authorization header when no token is configured', () => { + const headers = new Headers(); + attachApiTokenHeader(headers, '/api/nas/shares'); + + expect(headers.has('Authorization')).toBe(false); + expect(apiTokenHeaderValue('/api/nas/shares')).toBeNull(); + }); + + it('attaches the bearer token on native requests once a token is set', () => { + setApiToken('tok-native'); + + const headers = new Headers(); + attachApiTokenHeader(headers, '/api/nas/shares'); + + expect(headers.get('Authorization')).toBe('Bearer tok-native'); + }); + + it('never attaches the native token to an AML request', () => { + setApiToken('tok-native'); + + const headers = new Headers(); + attachApiTokenHeader(headers, '/aml/users/me'); + + expect(headers.has('Authorization')).toBe(false); + expect(apiTokenHeaderValue('/aml/users/me')).toBeNull(); + }); + + it('leaves a caller-supplied Authorization header alone', () => { + setApiToken('tok-native'); + + const headers = new Headers({ Authorization: 'Basic abc' }); + attachApiTokenHeader(headers, '/api/nas/shares'); + + expect(headers.get('Authorization')).toBe('Basic abc'); + }); + }); + + describe('storage', () => { + it('keeps the token in sessionStorage by default', () => { + setApiToken('tok-session'); + + expect(window.sessionStorage.getItem('openblade.api-token')).toBe('tok-session'); + expect(window.localStorage.getItem('openblade.api-token')).toBeNull(); + }); + + it('remembers the token on the device when asked', () => { + setApiToken('tok-remembered', true); + + expect(window.localStorage.getItem('openblade.api-token')).toBe('tok-remembered'); + expect(window.sessionStorage.getItem('openblade.api-token')).toBeNull(); + }); + + it('reads a remembered token back on a fresh load', () => { + window.localStorage.setItem('openblade.api-token', 'tok-remembered'); + resetApiTokenCache(); + + expect(getApiToken()).toBe('tok-remembered'); + }); + + it('clears both slots', () => { + window.localStorage.setItem('openblade.api-token', 'tok-remembered'); + window.sessionStorage.setItem('openblade.api-token', 'tok-session'); + resetApiTokenCache(); + + clearApiToken(); + + expect(getApiToken()).toBeNull(); + expect(window.localStorage.getItem('openblade.api-token')).toBeNull(); + expect(window.sessionStorage.getItem('openblade.api-token')).toBeNull(); + }); + }); + + describe('handleUnauthorized', () => { + it('claims a native 401 from the token gate, clears the token and prompts', () => { + setApiToken('tok-stale'); + const listener = vi.fn(); + window.addEventListener(API_TOKEN_REQUIRED_EVENT, listener); + + const handled = handleUnauthorized('/api/nas/shares', tokenGateResponse(), { + error: 'Unauthorized', + }); + + expect(handled).toBe(true); + expect(getApiToken()).toBeNull(); + expect(listener).toHaveBeenCalledTimes(1); + expect((listener.mock.calls[0][0] as CustomEvent).detail).toBe('rejected'); + + window.removeEventListener(API_TOKEN_REQUIRED_EVENT, listener); + }); + + it('reports "missing" when there was no token to reject', () => { + const listener = vi.fn(); + window.addEventListener(API_TOKEN_REQUIRED_EVENT, listener); + + const handled = handleUnauthorized('/api/nas/shares', amlSessionResponse(), { + error: 'Unauthorized', + }); + + expect(handled).toBe(true); + expect((listener.mock.calls[0][0] as CustomEvent).detail).toBe('missing'); + + window.removeEventListener(API_TOKEN_REQUIRED_EVENT, listener); + }); + + it('leaves an AML-session 401 on a native path to the sign-in redirect', () => { + setApiToken('tok-valid'); + const listener = vi.fn(); + window.addEventListener(API_TOKEN_REQUIRED_EVENT, listener); + + const handled = handleUnauthorized('/api/libraries', amlSessionResponse(), { + code: 'AML_AUTH_REQUIRED', + }); + + expect(handled).toBe(false); + expect(getApiToken()).toBe('tok-valid'); + expect(listener).not.toHaveBeenCalled(); + + window.removeEventListener(API_TOKEN_REQUIRED_EVENT, listener); + }); + + it('never claims a 401 from the AML surface', () => { + const listener = vi.fn(); + window.addEventListener(API_TOKEN_REQUIRED_EVENT, listener); + + const handled = handleUnauthorized('/aml/users/me', tokenGateResponse(), { + error: 'Unauthorized', + }); + + expect(handled).toBe(false); + expect(listener).not.toHaveBeenCalled(); + + window.removeEventListener(API_TOKEN_REQUIRED_EVENT, listener); + }); + }); +}); diff --git a/frontend/src/lib/apiToken.ts b/frontend/src/lib/apiToken.ts new file mode 100644 index 0000000..49bb1af --- /dev/null +++ b/frontend/src/lib/apiToken.ts @@ -0,0 +1,223 @@ +/** + * Native-API bearer token: the browser half of `openblade/api/api_auth.py`. + * + * The backend gates the OpenBlade-**native** surface on + * `Authorization: Bearer ` and leaves the Quantum AML + * emulator surface (`/aml/*`, `/iblade/*`) on its own session cookie. This + * module mirrors that classification and owns the token, so exactly one place + * decides "does this request carry the bearer header?". + * + * Two rules keep auth-disabled deployments untouched: + * - no token stored => no header is ever added, so the request is byte-for-byte + * what it was before this module existed; + * - the token-entry surface only appears after a *real* 401 from the token + * gate. Nothing probes, nothing pre-empts, there is no login wall. + * + * The token is a credential: it is never logged, never put in a URL, and never + * rendered back to the operator. + */ + +const STORAGE_KEY = 'openblade.api-token'; + +/** Dispatched when the token gate rejected a request; carries {@link ApiTokenPromptReason}. */ +export const API_TOKEN_REQUIRED_EVENT = 'openblade:api-token-required'; + +/** Why the token screen is being shown. Drives the curated message. */ +export type ApiTokenPromptReason = 'missing' | 'rejected'; + +/** + * Path prefixes owned by the Quantum AML emulator. Mirrors + * `AML_SURFACE_PREFIXES` in `api_auth.py`: these keep their cookie session and + * must never receive the native bearer token. + */ +const AML_SURFACE_PREFIXES = ['/aml', '/iblade'] as const; + +/** + * Native routes that merely happen to be mounted under an AML prefix. Mirrors + * `NATIVE_PATHS_UNDER_AML_PREFIX` in `api_auth.py` — the prefix is a mount + * point, not a statement about ownership, and these ARE token-gated. + */ +const NATIVE_PATHS_UNDER_AML_PREFIX = ['/aml/proxy'] as const; + +let cachedToken: string | null | undefined; + +function hasWindow(): boolean { + return typeof window !== 'undefined'; +} + +/** Segment-aware so `/amlfoo` is not mistaken for the `/aml` surface. */ +function hasPrefix(path: string, prefix: string): boolean { + return path === prefix || path.startsWith(`${prefix}/`); +} + +function pathOf(url: string): string { + if (/^https?:\/\//i.test(url)) { + try { + return new URL(url).pathname; + } catch { + return url; + } + } + + const withoutQuery = url.split(/[?#]/, 1)[0] ?? url; + return withoutQuery.startsWith('/') ? withoutQuery : `/${withoutQuery}`; +} + +/** + * True when `url` targets the OpenBlade-native surface, i.e. the surface the + * bearer token authenticates. AML/iBlade URLs return false. + */ +export function isNativeApiUrl(url: string): boolean { + const path = pathOf(url); + if (NATIVE_PATHS_UNDER_AML_PREFIX.some((prefix) => hasPrefix(path, prefix))) { + return true; + } + return !AML_SURFACE_PREFIXES.some((prefix) => hasPrefix(path, prefix)); +} + +/** + * The stored token, or null when auth is off / no token has been entered. + * Session storage wins over local storage so a "just this tab" entry can shadow + * a stale remembered one. + */ +export function getApiToken(): string | null { + if (cachedToken !== undefined) { + return cachedToken; + } + if (!hasWindow()) { + return null; + } + + const stored = + window.sessionStorage.getItem(STORAGE_KEY) ?? window.localStorage.getItem(STORAGE_KEY); + cachedToken = stored && stored.trim() ? stored.trim() : null; + return cachedToken; +} + +/** + * Store the token. `remember` picks the device-persistent slot (localStorage); + * the default is session-only, so closing the tab forgets the credential. + */ +export function setApiToken(token: string, remember = false): void { + const trimmed = token.trim(); + if (!trimmed) { + clearApiToken(); + return; + } + + cachedToken = trimmed; + if (!hasWindow()) { + return; + } + + // Written to exactly one slot, and the other is cleared, so there is never a + // second copy to revoke later. + if (remember) { + window.localStorage.setItem(STORAGE_KEY, trimmed); + window.sessionStorage.removeItem(STORAGE_KEY); + } else { + window.sessionStorage.setItem(STORAGE_KEY, trimmed); + window.localStorage.removeItem(STORAGE_KEY); + } +} + +/** Forget the token everywhere. Called before every re-prompt. */ +export function clearApiToken(): void { + cachedToken = null; + if (!hasWindow()) { + return; + } + window.sessionStorage.removeItem(STORAGE_KEY); + window.localStorage.removeItem(STORAGE_KEY); +} + +/** Test seam: drop the in-memory cache so the next read re-reads storage. */ +export function resetApiTokenCache(): void { + cachedToken = undefined; +} + +/** + * The `Authorization` value for a native request, or null when none applies + * (AML URL, or no token configured). For transports that are not `Headers` + * based, e.g. `XMLHttpRequest.setRequestHeader`. + */ +export function apiTokenHeaderValue(url: string): string | null { + if (!isNativeApiUrl(url)) { + return null; + } + + const token = getApiToken(); + return token ? `Bearer ${token}` : null; +} + +/** + * Attach `Authorization: Bearer ` when this is a native request and a + * token is configured. No token, or an AML URL, leaves `headers` untouched. + * An Authorization header the caller set itself is never overwritten. + */ +export function attachApiTokenHeader(headers: Headers, url: string): void { + if (headers.has('Authorization')) { + return; + } + + const value = apiTokenHeaderValue(url); + if (value) { + headers.set('Authorization', value); + } +} + +/** Ask the UI to show the token-entry surface. */ +export function notifyApiTokenRequired(reason: ApiTokenPromptReason): void { + if (!hasWindow()) { + return; + } + window.dispatchEvent(new CustomEvent(API_TOKEN_REQUIRED_EVENT, { detail: reason })); +} + +/** + * Decide whether a 401 came from the native token gate, and if so clear the bad + * token and raise the prompt. + * + * A native path can 401 from *either* gate: `api_auth.py` answers + * `{"error": "Unauthorized"}` with `WWW-Authenticate: Bearer`, while the AML + * session layer answers `{"code": "AML_AUTH_REQUIRED"}`. Only the first is ours; + * returning false leaves the existing AML sign-in redirect in charge. + */ +export function handleUnauthorized( + url: string, + response: Pick, + payload: unknown, +): boolean { + if (!isNativeApiUrl(url)) { + return false; + } + + const challenge = response.headers.get('WWW-Authenticate') ?? ''; + const isTokenGate = + challenge.toLowerCase().includes('bearer') || + (typeof payload === 'object' && + payload !== null && + 'error' in payload && + (payload as { error?: unknown }).error === 'Unauthorized'); + + if (!isTokenGate) { + return false; + } + + const hadToken = getApiToken() !== null; + clearApiToken(); + notifyApiTokenRequired(hadToken ? 'rejected' : 'missing'); + return true; +} + +/** + * Re-run whatever the operator was doing when the gate fired. A reload replays + * the current URL with the new token in place, so the failed navigation retries + * itself and no per-page retry plumbing is needed. Separate export so tests can + * substitute it. + */ +export function reloadForApiToken(): void { + if (hasWindow()) { + window.location.reload(); + } +} From ad4a2476ce9c56f8113a11db341904d6e378e0f9 Mon Sep 17 00:00:00 2001 From: Amantux Date: Sat, 26 Sep 2026 06:00:46 +0000 Subject: [PATCH 05/54] docs(mhvtl): record the GitHub-runner bring-up, with the evidence MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Appends §7 to the rehearsal runbook from runs 35688121678 (red) and 36222156436 / 36222322745 (green, cold-build and cache-hit respectively — both 51 passed, 11 skipped, 0 failed, LTFS included). States plainly that all three predicted assumptions HELD, quotes the evidence for the fourth nobody had written down (st/ch live in linux-modules-extra on an Azure kernel, "modinfo: ERROR: Module st not found", "Installed: (none)"), the fifth (runner user is not in the `tape` group, so sg_inq EACCES masqueraded as a wrong unit serial), and the LTFS dependency §1 omitted because this host already had it (net-snmp >= 5.3). Also records the runner's non-monotonic sg/st mapping (target 2 -> st2/sg4, target 3 -> st1/sg3) as independent confirmation of §3.4's "must be read, not derived", the loudness rule the red run bought, what stays degraded (only the LTFS soft cut) and why a scheduled run cannot reuse a branch cache. --- docs/runbooks/mhvtl-rehearsal.md | 145 +++++++++++++++++++++++++++++++ 1 file changed, 145 insertions(+) diff --git a/docs/runbooks/mhvtl-rehearsal.md b/docs/runbooks/mhvtl-rehearsal.md index 3199435..df5b738 100644 --- a/docs/runbooks/mhvtl-rehearsal.md +++ b/docs/runbooks/mhvtl-rehearsal.md @@ -458,3 +458,148 @@ still unrehearsed: Phases 3 and 4 stand unchanged. What this phase removes is the class of failure where the *parsers and device plumbing* are wrong — and it found seven of those. + +--- + +## 7. GitHub runner — the weekly canary's own bring-up + +`.github/workflows/mhvtl-weekly.yml` reproduces all of §1 on a hosted +`ubuntu-latest` runner every Tuesday. Its first two real runs (2026-09-15 +`34930111187`, 2026-09-22 `35688121678`) failed identically; run +`36222156436` is the first green one — **51 passed, 11 skipped, 0 failed in +22.7 s**, LTFS included. Nothing is degraded and nothing is skipped that is not +also skipped on this host (§2). + +### The three documented assumptions all HELD + +Worth stating plainly, because the workflow header predicted these as the likely +blockers and every one of them was fine: + +| Assumption | Runner reality | +|---|---| +| headers for the running kernel exist | `linux-headers-6.17.0-1022-azure Installed: 6.17.0-1022.22` | +| unsigned out-of-tree module can load | `SecureBoot disabled`, `sig_enforce` = `N`, module built + loaded | +| systemd is PID 1 | `systemctl is-system-running` → `running`, `ps -p 1` → `systemd` | + +`mhvtl` registered 1 `mediumx` + 3 `tape`, `mtx status` listed all 12 elements, +and `setup.sh` printed `=== Rig is up ===`. The failure was elsewhere. + +### 7.1 `st` and `ch` are not in an Azure kernel's module set (the real blocker) + +A **fourth** assumption nobody had written down. mhvtl registers SCSI *targets*; +`/dev/nstN` and `/dev/schN` are created by the kernel's **upper-level** drivers +`st` and `ch`, which is a separate matter from mhvtl entirely. Ubuntu's generic +kernel ships them inside `linux-modules-$(uname -r)` and udev autoloads them — +which is why this host got `/dev/st0-2` and `/dev/sch0` for free and `setup.sh` +never mentioned them. An Azure-tuned kernel puts them in +**`linux-modules-extra-$(uname -r)`, which is not installed on the runner image**: + +``` ++ modinfo st +modinfo: ERROR: Module st not found. ++ apt-cache policy linux-modules-extra-6.17.0-1022-azure + Installed: (none) + Candidate: 6.17.0-1022.22 +``` + +The rig therefore came up with every tape row showing `-` in `lsscsi`'s block +column, an empty `/sys/class/scsi_tape`, and `ls: cannot access '/dev/nst*': No +such file or directory`. `OPENBLADE_DRIVE_DEVICES` is a list of `/dev/nstN` +nodes, so the rig was unusable while looking healthy. + +`setup.sh` now `modprobe`s `st` before starting the daemons, apt-installs +`linux-modules-extra-$(uname -r)` and retries if that fails, and **hard-fails** +if it still cannot — no `/dev/nst` means no drive to open, so there is nothing to +degrade to. `ch` is attempted the same way but is only a `NOTE:` when absent: +this rig drives the changer through its `/dev/sg` node with `mtx`. + +Confirmed side benefit — the runner's mapping is **non-monotonic**, which is the +independent confirmation §3.4's "must be read, not derived" rule wanted: + +``` +SCSI_ADDR BLOCK SG NST SERIAL +2:0:1:0 /dev/st0 /dev/sg2 /dev/nst0 OBLADE_D01 +2:0:2:0 /dev/st2 /dev/sg4 /dev/nst2 OBLADE_D02 +2:0:3:0 /dev/st1 /dev/sg3 /dev/nst1 OBLADE_D03 +``` + +### 7.2 `sg_inq` needs the `tape` group — and reported the wrong cause + +`env.sh` refused with + +``` +ERROR: found medium changer(s) /dev/sg1 but none reporting unit serial + 'OBLADE_L10'. +``` + +on a rig whose `device.conf` says exactly `OBLADE_L10`. The `/dev/sgN` nodes are +`0660 root:tape`; the runner user is +`uid=1001(runner) groups=1001(runner),4(adm),100(users),118(docker),999(systemd-journal)` +— **not in `tape`**. `sg_inq` failed with EACCES, whose stderr the serial lookup +swallowed, and an empty serial is indistinguishable from "that is somebody +else's library". On this host the operator was in `tape`, so it never showed. + +`rig_sg_inq()` now escalates via `sudo -n` when the node is unreadable, and +`rig_changer` checks for exactly that condition and reports a **permission** +problem as a permission problem. The refusal-to-guess guard is unchanged. + +### 7.3 LTFS needed `libsnmp-dev`, not the documented blockers + +The `icu-config` shim and `--recurse-submodules` from §1 were both correct and +both worked. The build died earlier than either, on a dependency §1 does not +mention because this host already had it: + +``` +./configure: line 14786: net-snmp-config: command not found +configure: error: Package requirements (net-snmp >= 5.3) were not met +``` + +There is no `--disable-snmp`. `libsnmp-dev` provides `net-snmp-config` and the +pkg-config file; with it, LTFS builds, `mkltfs` formats `OB0007L8`/`OB0008L8`, +and the full suite runs. Runtime libraries are now apt-installed **one package +per call**, because noble renamed `libfuse2` → `libfuse2t64` and a single batched +`apt-get ... || true` containing one unknown name installs *nothing* and hides it. + +### 7.4 Loudness: the failure that reported itself three steps late + +The canary's first red run named the failing step as the pytest one, with +`OPENBLADE_BACKEND: unbound variable`. The real error was `env.sh`'s, 20 lines +earlier, as mere stderr. Cause: `eval "$(scripts/mhvtl/env.sh)"` — **a failing +command substitution does not fail the enclosing command even under `set -e`**, +so the refusal became `eval ""` and the step carried on. + +Three changes, and the rule behind them: *if a step's failure means the rig +cannot work, that step must be the red one.* + +- Resolution is its own workflow step that checks `env.sh`'s exit status, + `::error::`s, and exports via `$GITHUB_ENV`. The pytest step additionally + `${VAR:?}`-guards each variable. +- `setup.sh` identifies the changer with `rig_changer` (serial-keyed) and asserts + an `/dev/nst` node per drive **before** printing `Rig is up`, so anything that + would make `env.sh` refuse fails at bring-up instead. +- The `st` driver is a hard failure; only LTFS remains a documented soft cut. + +### 7.5 What the canary covers, and what stays degraded + +Nothing is degraded on a green run: LTFS builds, so all suites execute. The one +remaining soft cut is unchanged — if the LTFS build ever breaks, the job emits a +`::warning::` and runs the four non-LTFS suites (discovery, changer, drive +health, catalog integrity) rather than going red on a third-party build. The +11 skips are the same legitimate set as §2: 5 TapeAlert/diagnostic pages mhvtl +does not emulate, 4 opt-in fault-injection tests, 1 opt-in dirty unmount, 1 +absent higher-level locking backend. + +Cold runs build LTFS from source inside the 20-minute step budget; the +`actions/cache` entry is keyed on `LTFS_REF` + runner image identity, and +**Actions cache scoping means a branch's cache is not visible to the default +branch**, so the first scheduled run after any image bump pays the cold cost. + +### 7.6 Where this would block on a different runner image + +Add to the §5 table, for hosted runners specifically: + +| Requirement | Failure if missing | +|---|---| +| `st` in `linux-modules*-$(uname -r)` | no `/dev/nstN`; `setup.sh` fails with the apt hint | +| caller in `tape`, root, or able to `sudo -n` | `sg_inq` EACCES; `rig_changer` says so explicitly | +| `libsnmp-dev` | LTFS `./configure` fails on `net-snmp >= 5.3`; job degrades with a `::warning::` | From a1561deb02ead7237ed42ea1fd1a672e12434b52 Mon Sep 17 00:00:00 2001 From: Amantux Date: Sat, 26 Sep 2026 06:00:51 +0000 Subject: [PATCH 06/54] feat(frontend): send the bearer token from every native request path MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `apiRequest` in `api/client.ts` is the chokepoint nearly every api module already goes through, so it gains the header attachment and the 401 handling. Three transports bypass it and get the smallest possible insertion instead: - `api/nas.ts` — `nasRequest`'s raw fetch to `/api/nas/*` - `api/filestation.ts` — the upload `XMLHttpRequest` and the download fetch A native 401 is routed by which gate answered it: the token gate (`WWW-Authenticate: Bearer` / `{"error": "Unauthorized"}`) clears the bad token and raises the token prompt, while an AML-session 401 keeps the existing sign-in redirect. AML/iBlade requests are untouched and never carry the token. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_013S1qbQs31YSox1azQChqs2 --- frontend/src/api/client.test.ts | 128 ++++++++++++++++++++++++++++++++ frontend/src/api/client.ts | 24 ++++-- frontend/src/api/filestation.ts | 18 ++++- frontend/src/api/nas.ts | 4 +- 4 files changed, 167 insertions(+), 7 deletions(-) create mode 100644 frontend/src/api/client.test.ts diff --git a/frontend/src/api/client.test.ts b/frontend/src/api/client.test.ts new file mode 100644 index 0000000..f2e16c7 --- /dev/null +++ b/frontend/src/api/client.test.ts @@ -0,0 +1,128 @@ +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; +import { apiRequest, rootApiRequest } from './client'; +import { + API_TOKEN_REQUIRED_EVENT, + clearApiToken, + getApiToken, + resetApiTokenCache, + setApiToken, +} from '../lib/apiToken'; + +interface FakeResponseInit { + status?: number; + body?: unknown; + headers?: Record; +} + +function fakeResponse({ status = 200, body = {}, headers = {} }: FakeResponseInit = {}) { + const text = typeof body === 'string' ? body : JSON.stringify(body); + return { + ok: status >= 200 && status < 300, + status, + statusText: String(status), + url: '', + headers: new Headers(headers), + text: () => Promise.resolve(text), + }; +} + +const fetchMock = vi.fn(); + +function lastRequestHeaders(): Headers { + const init = fetchMock.mock.calls.at(-1)?.[1] as { headers: Headers }; + return init.headers; +} + +describe('apiRequest native bearer token', () => { + beforeEach(() => { + fetchMock.mockReset(); + vi.stubGlobal('fetch', fetchMock); + window.localStorage.clear(); + window.sessionStorage.clear(); + resetApiTokenCache(); + }); + + afterEach(() => { + vi.unstubAllGlobals(); + }); + + it('sends no Authorization header when no token is configured', async () => { + fetchMock.mockResolvedValue(fakeResponse({ body: { ok: true } })); + + await rootApiRequest('/jobs/'); + + expect(fetchMock).toHaveBeenCalledTimes(1); + expect(lastRequestHeaders().has('Authorization')).toBe(false); + }); + + it('attaches the bearer token to native requests when one is configured', async () => { + setApiToken('tok-abc'); + fetchMock.mockResolvedValue(fakeResponse({ body: { ok: true } })); + + await rootApiRequest('/jobs/'); + + expect(fetchMock.mock.calls[0][0]).toBe('/api/jobs/'); + expect(lastRequestHeaders().get('Authorization')).toBe('Bearer tok-abc'); + }); + + it('leaves AML requests on their own session auth', async () => { + setApiToken('tok-abc'); + fetchMock.mockResolvedValue(fakeResponse({ body: { ok: true } })); + + await apiRequest('/users/me'); + + expect(fetchMock.mock.calls[0][0]).toBe('/aml/users/me'); + expect(lastRequestHeaders().has('Authorization')).toBe(false); + }); + + it('clears the token and asks for a new one on a native 401 from the token gate', async () => { + setApiToken('tok-stale'); + const listener = vi.fn(); + window.addEventListener(API_TOKEN_REQUIRED_EVENT, listener); + fetchMock.mockResolvedValue( + fakeResponse({ + status: 401, + body: { error: 'Unauthorized', detail: 'This endpoint requires a bearer token.' }, + headers: { 'WWW-Authenticate': 'Bearer' }, + }), + ); + + await expect(rootApiRequest('/jobs/')).rejects.toMatchObject({ status: 401 }); + + expect(listener).toHaveBeenCalledTimes(1); + expect(getApiToken()).toBeNull(); + + window.removeEventListener(API_TOKEN_REQUIRED_EVENT, listener); + }); + + it('never puts the token in the thrown error or in a log line', async () => { + setApiToken('super-secret-token'); + const logSpy = vi.spyOn(console, 'log').mockImplementation(() => {}); + const warnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {}); + const errorSpy = vi.spyOn(console, 'error').mockImplementation(() => {}); + fetchMock.mockResolvedValue( + fakeResponse({ + status: 401, + body: { error: 'Unauthorized', detail: 'This endpoint requires a bearer token.' }, + headers: { 'WWW-Authenticate': 'Bearer' }, + }), + ); + + const thrown = await rootApiRequest('/jobs/').catch((error: unknown) => error); + + const serialized = JSON.stringify({ + message: (thrown as Error).message, + ...(thrown as Record), + }); + expect(serialized).not.toContain('super-secret-token'); + + for (const spy of [logSpy, warnSpy, errorSpy]) { + for (const call of spy.mock.calls) { + expect(JSON.stringify(call)).not.toContain('super-secret-token'); + } + spy.mockRestore(); + } + + clearApiToken(); + }); +}); diff --git a/frontend/src/api/client.ts b/frontend/src/api/client.ts index 5a1e8c3..541855c 100644 --- a/frontend/src/api/client.ts +++ b/frontend/src/api/client.ts @@ -1,3 +1,4 @@ +import { attachApiTokenHeader, handleUnauthorized } from '../lib/apiToken'; import { clearStoredUsername, notifyAuthRedirect } from '../lib/auth'; type BodyInitLike = BodyInit | FormData | URLSearchParams | Blob; @@ -100,7 +101,12 @@ export async function apiRequest( headers.set('X-OpenBlade-Library-Id', resolvedLibraryId); } - const response = await fetch(buildUrl(path, namespace), { + const url = buildUrl(path, namespace); + // The single chokepoint for the native bearer token: no-op when no token is + // stored (auth-disabled deployments are unchanged) and on AML URLs. + attachApiTokenHeader(headers, url); + + const response = await fetch(url, { ...init, headers, body, @@ -110,8 +116,14 @@ export async function apiRequest( const text = await response.text(); const payload = text ? safeJsonParse(text) : null; + let nativeTokenRequired = false; if (response.status === 401 && !init.skipAuthRedirect) { - redirectToLogin(); + // The native token gate and the AML session gate both answer 401 on native + // paths; only the AML one leads to the sign-in page. + nativeTokenRequired = handleUnauthorized(url, response, payload); + if (!nativeTokenRequired) { + redirectToLogin(); + } } if (!response.ok) { @@ -129,9 +141,11 @@ export async function apiRequest( message, response.status, `The backend could not complete ${init.method ?? 'GET'} ${path}.`, - response.status === 401 - ? 'Sign in again to continue using the AML console.' - : 'Check the appliance state, then retry the request.', + nativeTokenRequired + ? 'Enter the API token for this appliance to continue.' + : response.status === 401 + ? 'Sign in again to continue using the AML console.' + : 'Check the appliance state, then retry the request.', details, ); } diff --git a/frontend/src/api/filestation.ts b/frontend/src/api/filestation.ts index 34824a4..e887dac 100644 --- a/frontend/src/api/filestation.ts +++ b/frontend/src/api/filestation.ts @@ -1,4 +1,5 @@ import { rootApiRequest } from './client'; +import { apiTokenHeaderValue, attachApiTokenHeader, handleUnauthorized } from '../lib/apiToken'; const API = '/api'; @@ -54,6 +55,10 @@ export async function uploadToPool(poolId: string, file: File, onProgress?: (pct const xhr = new XMLHttpRequest(); xhr.open('POST', `${API}/pools/${encodeURIComponent(poolId)}/upload`); xhr.withCredentials = true; + const bearer = apiTokenHeaderValue(API); + if (bearer) { + xhr.setRequestHeader('Authorization', bearer); + } xhr.upload.onprogress = (event) => { if (event.lengthComputable && onProgress) { @@ -74,6 +79,10 @@ export async function uploadToPool(poolId: string, file: File, onProgress?: (pct return; } + if (xhr.status === 401) { + handleUnauthorized(API, { headers: new Headers() }, payload); + } + const detail = typeof payload === 'object' && payload && 'detail' in payload && payload.detail ? String(payload.detail) @@ -98,10 +107,17 @@ export function listFiles(poolId: string): Promise { } export async function downloadFile(fileId: string, filename?: string): Promise { - const response = await fetch(`${API}/files/${encodeURIComponent(fileId)}/download`, { + const downloadUrl = `${API}/files/${encodeURIComponent(fileId)}/download`; + const headers = new Headers(); + attachApiTokenHeader(headers, downloadUrl); + const response = await fetch(downloadUrl, { credentials: 'include', + headers, }); if (!response.ok) { + if (response.status === 401) { + handleUnauthorized(downloadUrl, response, null); + } throw new Error(`HTTP ${response.status}`); } diff --git a/frontend/src/api/nas.ts b/frontend/src/api/nas.ts index f55dd24..2c6535d 100644 --- a/frontend/src/api/nas.ts +++ b/frontend/src/api/nas.ts @@ -1,4 +1,5 @@ import { ApiError } from './client'; +import { attachApiTokenHeader, handleUnauthorized } from '../lib/apiToken'; import { clearStoredUsername, notifyAuthRedirect } from '../lib/auth'; export type PolicyType = 'critical_sequential' | 'noncritical_sharded' | 'balanced'; @@ -695,7 +696,7 @@ async function parseResponse(response: Response): Promise { const text = await response.text(); const payload = text ? safeJsonParse(text) : null; - if (response.status === 401) { + if (response.status === 401 && !handleUnauthorized(response.url || '/api/nas', response, payload)) { redirectToLogin(); } @@ -723,6 +724,7 @@ async function nasRequest(path: string, init: RequestInit = {}): Promise { headers.set('Content-Type', 'application/json'); } + attachApiTokenHeader(headers, '/api/nas'); const response = await fetch(`/api/nas${path}`, { ...init, headers, From 8911c3455379ce346b277b4dd0cac4b4cf06bf0d Mon Sep 17 00:00:00 2001 From: Amantux Date: Sat, 26 Sep 2026 06:01:00 +0000 Subject: [PATCH 07/54] feat(frontend): token-entry surface shown only after a real 401 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `ApiTokenGate` renders nothing until the API client reports a 401 from the token gate, so a deployment with `OPENBLADE_API_TOKEN` unset sees no login wall and no extra request. On submit the token is stored (session-only by default, localStorage when "remember on this device" is checked) and the page reloads, which replays the navigation that failed. The input is masked and the token is never rendered back, logged, or put in a message — the curated copy distinguishes "no token yet" from "the stored token was rejected and has been discarded". Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_013S1qbQs31YSox1azQChqs2 --- frontend/src/App.tsx | 3 + .../src/components/auth/ApiTokenGate.test.tsx | 100 +++++++++++++++ frontend/src/components/auth/ApiTokenGate.tsx | 119 ++++++++++++++++++ 3 files changed, 222 insertions(+) create mode 100644 frontend/src/components/auth/ApiTokenGate.test.tsx create mode 100644 frontend/src/components/auth/ApiTokenGate.tsx diff --git a/frontend/src/App.tsx b/frontend/src/App.tsx index a0c98a1..e8384cf 100644 --- a/frontend/src/App.tsx +++ b/frontend/src/App.tsx @@ -1,5 +1,6 @@ import { useEffect } from 'react'; import { BrowserRouter, Navigate, Outlet, Route, Routes, useLocation, useParams } from 'react-router-dom'; +import ApiTokenGate from './components/auth/ApiTokenGate'; import Layout from './components/layout/Layout'; import Spinner from './components/ui/Spinner'; import { getActiveLibraryId, setActiveLibraryId } from './lib/activeLibrary'; @@ -152,6 +153,8 @@ export default function App() { + {/* Renders nothing until the native token gate answers 401. */} + ); diff --git a/frontend/src/components/auth/ApiTokenGate.test.tsx b/frontend/src/components/auth/ApiTokenGate.test.tsx new file mode 100644 index 0000000..2b8dbba --- /dev/null +++ b/frontend/src/components/auth/ApiTokenGate.test.tsx @@ -0,0 +1,100 @@ +import { act, fireEvent, render, screen } from '@testing-library/react'; +import { beforeEach, describe, expect, it, vi } from 'vitest'; +import ApiTokenGate from './ApiTokenGate'; +import { + API_TOKEN_REQUIRED_EVENT, + getApiToken, + resetApiTokenCache, + type ApiTokenPromptReason, +} from '../../lib/apiToken'; + +function requireToken(reason: ApiTokenPromptReason = 'missing') { + act(() => { + window.dispatchEvent(new CustomEvent(API_TOKEN_REQUIRED_EVENT, { detail: reason })); + }); +} + +describe('ApiTokenGate', () => { + beforeEach(() => { + window.localStorage.clear(); + window.sessionStorage.clear(); + resetApiTokenCache(); + }); + + it('renders nothing until a 401 from the token gate — auth-disabled sees no login wall', () => { + render(); + + expect(screen.queryByRole('dialog')).toBeNull(); + expect(document.body.textContent).toBe(''); + }); + + it('shows the token entry surface after a real 401', () => { + render(); + + requireToken('missing'); + + expect(screen.getByRole('dialog')).toBeTruthy(); + expect(screen.getByText('API token required')).toBeTruthy(); + expect(document.body.textContent).toContain('This appliance requires an API token'); + }); + + it('uses a curated message for a rejected token and never echoes it', () => { + window.sessionStorage.setItem('openblade.api-token', 'rejected-token-value'); + resetApiTokenCache(); + render(); + + requireToken('rejected'); + + expect(document.body.textContent).toContain('The stored API token was rejected'); + expect(document.body.innerHTML).not.toContain('rejected-token-value'); + }); + + it('stores a session-only token by default and retries the failed navigation', () => { + const onAccepted = vi.fn(); + render(); + requireToken('missing'); + + fireEvent.change(screen.getByLabelText('API token'), { target: { value: 'entered-token' } }); + fireEvent.click(screen.getByRole('button', { name: 'Use this token' })); + + expect(window.sessionStorage.getItem('openblade.api-token')).toBe('entered-token'); + expect(window.localStorage.getItem('openblade.api-token')).toBeNull(); + expect(getApiToken()).toBe('entered-token'); + expect(onAccepted).toHaveBeenCalledTimes(1); + expect(screen.queryByRole('dialog')).toBeNull(); + }); + + it('remembers the token on this device when asked', () => { + render(); + requireToken('missing'); + + fireEvent.change(screen.getByLabelText('API token'), { target: { value: 'entered-token' } }); + fireEvent.click(screen.getByLabelText('Remember on this device')); + fireEvent.click(screen.getByRole('button', { name: 'Use this token' })); + + expect(window.localStorage.getItem('openblade.api-token')).toBe('entered-token'); + expect(window.sessionStorage.getItem('openblade.api-token')).toBeNull(); + }); + + it('masks the token and never renders or logs it', () => { + const logSpy = vi.spyOn(console, 'log').mockImplementation(() => {}); + const warnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {}); + const errorSpy = vi.spyOn(console, 'error').mockImplementation(() => {}); + render(); + requireToken('missing'); + + const input = screen.getByLabelText('API token') as HTMLInputElement; + expect(input.type).toBe('password'); + + fireEvent.change(input, { target: { value: 'super-secret-token' } }); + fireEvent.click(screen.getByRole('button', { name: 'Use this token' })); + + expect(document.body.innerHTML).not.toContain('super-secret-token'); + for (const spy of [logSpy, warnSpy, errorSpy]) { + for (const call of spy.mock.calls) { + expect(JSON.stringify(call)).not.toContain('super-secret-token'); + } + spy.mockRestore(); + } + }); +}); diff --git a/frontend/src/components/auth/ApiTokenGate.tsx b/frontend/src/components/auth/ApiTokenGate.tsx new file mode 100644 index 0000000..9951dcb --- /dev/null +++ b/frontend/src/components/auth/ApiTokenGate.tsx @@ -0,0 +1,119 @@ +import { useEffect, useState } from 'react'; +import type { FormEvent } from 'react'; +import Button from '../ui/Button'; +import Card from '../ui/Card'; +import { + API_TOKEN_REQUIRED_EVENT, + reloadForApiToken, + setApiToken, + type ApiTokenPromptReason, +} from '../../lib/apiToken'; + +const MESSAGES: Record = { + missing: + 'This appliance requires an API token. Paste the token from OPENBLADE_API_TOKEN (or the file named by OPENBLADE_API_TOKEN_FILE) to continue.', + rejected: + 'The stored API token was rejected and has been discarded. Paste the token currently configured on this appliance to continue.', +}; + +interface ApiTokenGateProps { + /** Injected in tests; production replays the current URL with the new token. */ + onAccepted?: () => void; +} + +/** + * The token-entry surface for the native REST API. + * + * It renders nothing until the API client reports a 401 from the bearer-token + * gate, so a deployment with `OPENBLADE_API_TOKEN` unset never sees it: no login + * wall, no extra request, no change in behaviour. On submit the token is stored + * and the app reloads, which re-runs the navigation that failed. + * + * The token is write-only here — it is masked on input and never echoed back, + * logged, or included in any message. + */ +export default function ApiTokenGate({ onAccepted = reloadForApiToken }: ApiTokenGateProps) { + const [reason, setReason] = useState(null); + const [token, setToken] = useState(''); + const [remember, setRemember] = useState(false); + + useEffect(() => { + function handleTokenRequired(event: Event) { + const detail = (event as CustomEvent).detail; + setReason(detail === 'rejected' ? 'rejected' : 'missing'); + } + + window.addEventListener(API_TOKEN_REQUIRED_EVENT, handleTokenRequired); + return () => window.removeEventListener(API_TOKEN_REQUIRED_EVENT, handleTokenRequired); + }, []); + + if (!reason) { + return null; + } + + function handleSubmit(event: FormEvent) { + event.preventDefault(); + if (!token.trim()) { + return; + } + + setApiToken(token, remember); + setToken(''); + setReason(null); + onAccepted(); + } + + return ( +

+ ); +} From e86aad27d90da4e066aa783be06ab4d3dc51b7e1 Mon Sep 17 00:00:00 2001 From: Amantux Date: Sat, 26 Sep 2026 06:01:01 +0000 Subject: [PATCH 08/54] docs: describe the web UI half of native API authentication Adds a "Web UI" section to the API authentication guide: what the operator sees when auth is on, where the token is stored and for how long, and that nothing changes when auth is off. Corrects the rollout note and the troubleshooting entry that said the frontend has nowhere to put a token. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_013S1qbQs31YSox1azQChqs2 --- docs/wiki/guides/api-authentication.md | 46 ++++++++++++++++++++++---- 1 file changed, 40 insertions(+), 6 deletions(-) diff --git a/docs/wiki/guides/api-authentication.md b/docs/wiki/guides/api-authentication.md index 1a889ac..fc6c4cb 100644 --- a/docs/wiki/guides/api-authentication.md +++ b/docs/wiki/guides/api-authentication.md @@ -204,10 +204,9 @@ dry-run → token → confirm dance to format a tape. Every client of the native API needs the header before you flip the switch. In a default deployment that is at least: -1. **The React frontend**, if it talks to the API from a browser. It has no - place to put a bearer token that a browser user cannot also read — if you - serve the UI publicly, front it with a reverse proxy that injects the header - and authenticates users itself, rather than shipping the token to the client. +1. **The React frontend**, if it talks to the API from a browser. It prompts the + operator for the token the first time a request is refused — see + [Web UI](#web-ui), including what it means for a publicly served UI. 2. **Your own scripts and cron jobs.** 3. **Monitoring**, unless it only polls `/health`. @@ -260,6 +259,40 @@ this works in both modes and is what the i3 compliance suite does. --- +## Web UI + +The React console sends the token too, and it costs nothing when auth is off. + +**With the token unset, nothing changes.** The UI only adds an +`Authorization` header when a token is stored in the browser, and it only asks +for one after a request actually comes back 401 from this gate. There is no login +wall, no extra probe request, and no new screen in the default deployment. + +**When auth is on**, the first refused request replaces the view with an +*API token required* panel. Paste the value of `OPENBLADE_API_TOKEN` (or the +contents of the file named by `OPENBLADE_API_TOKEN_FILE`) and the console reloads +the page you were on, so the navigation that failed simply retries. + +- **Remember on this device** stores the token in `localStorage` — it survives + closing the browser. Leave it unchecked and the token lives in `sessionStorage` + and is forgotten when the tab closes. That is the default. +- A token the server rejects is **discarded**, not retried, and the panel says so + without echoing what you typed. The field is masked and the token never appears + in a URL, a log line, or an error message. +- The AML console pages are unaffected: `/aml/*` and `/iblade/*` requests keep + using the session cookie from `POST /aml/users/login` and never carry the + bearer token. If you are prompted for a *username and password*, that is the + AML session; a token prompt is this gate. +- To forget a stored token, clear site data for the console's origin, or just + wait for the next 401 — a rejected token is cleared automatically. + +**This is not a way to hide the token from the person using the browser.** They +typed it, and anything running in that page can read it. If you serve the UI to +people who should not hold an API token, front it with a reverse proxy that +authenticates users itself and injects the header server-side. + +--- + ## Troubleshooting **Everything returns 401, including with the token.** Check the server actually @@ -283,8 +316,9 @@ surface does not accept it. Log in with `POST /aml/users/login`. A 401 with `"error": "Unauthorized"` came from this one. The body tells you which gate you hit. -**The frontend stopped working.** It has no token. See -[rolling it out](#rolling-it-out-without-an-outage). +**The frontend stopped working.** It has no token. It should be asking you for +one — see [Web UI](#web-ui). If it is not, the 401 came from the AML session +layer instead; the body tells you which gate you hit. --- From bb5377973f4a0246b228ce661f1360853f24ed14 Mon Sep 17 00:00:00 2001 From: Amantux Date: Sat, 26 Sep 2026 06:01:47 +0000 Subject: [PATCH 09/54] ci(mhvtl): count scsi_tape nodes with a glob, not ls|grep MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit shellcheck SC2010 on the nst wait-loop counter added in ea6457e — the only new finding the change introduced. `if` rather than `&&` because a failing test as the last command in a loop body aborts the script under set -e. --- scripts/mhvtl/setup.sh | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/scripts/mhvtl/setup.sh b/scripts/mhvtl/setup.sh index 0e6adb9..4029fab 100755 --- a/scripts/mhvtl/setup.sh +++ b/scripts/mhvtl/setup.sh @@ -288,7 +288,14 @@ for _ in $(seq 1 30); do sleep 1 changers_sg=$(lsscsi -g 2>/dev/null | awk '/ mediumx /{print $NF}' | grep -c '^/dev/sg' || true) drives_sg=$(lsscsi -g 2>/dev/null | awk '/ tape /{print $NF}' | grep -c '^/dev/sg' || true) - tape_nodes=$(ls -d /sys/class/scsi_tape/nst* 2>/dev/null | grep -c . || true) + # Counted with a glob rather than `ls | grep -c`: an unmatched glob stays + # literal, so the -e test is what distinguishes "no nodes" from "one node". + # The `if` rather than `&&` matters — a failing test as the last command in a + # loop body aborts the script under `set -e`. + tape_nodes=0 + for _t in /sys/class/scsi_tape/nst*; do + if [ -e "$_t" ]; then tape_nodes=$((tape_nodes + 1)); fi + done if [ "$changers_sg" -ge 1 ] && [ "$drives_sg" -ge 3 ] && [ "$tape_nodes" -ge 3 ]; then break; fi done command -v udevadm >/dev/null && udevadm settle --timeout=30 || true From 2d56ee5c5343645b8268c494f8bb71170e2b620b Mon Sep 17 00:00:00 2001 From: Amantux Date: Sat, 26 Sep 2026 06:03:10 +0000 Subject: [PATCH 10/54] fix: make connect-i3 transport-aware for the webservices robotics backend openblade hardware connect-i3 hardcoded RealLibraryBackend regardless of OPENBLADE_ROBOTICS_TRANSPORT, so it could not diagnose a Scalar reached only over Web Services -- and library.correlation / library.changer, which ScalarHttpLibraryBackend does not have, would raise AttributeError. connect_quantum_i3 now builds the backend via the same openblade.bootstrap._create_scalar_http_library the app uses when robotics_transport == "webservices", and degrades genuinely transport-absent capabilities (a local changer device, the up-front correlation summary) to a named "not applicable" report instead of crashing. Per-drive host devices (drive_devices) already worked on both backends via the shared drive_device() method and are unchanged. Mutation-checked: reverting the getattr(...) guards back to direct attribute access makes both new webservices tests fail with AttributeError; restored after confirming. SCSI-transport behavior is pinned unchanged by test_scsi_transport_is_unaffected. --- openblade/hardware/validation.py | 107 ++++++++++++++++-- tests/unit/test_hardware_validation.py | 146 +++++++++++++++++++++++++ 2 files changed, 245 insertions(+), 8 deletions(-) diff --git a/openblade/hardware/validation.py b/openblade/hardware/validation.py index ace0ef1..966853b 100644 --- a/openblade/hardware/validation.py +++ b/openblade/hardware/validation.py @@ -4,7 +4,9 @@ from pathlib import Path from openblade.config import OpenBladeConfig +from openblade.domain.backends import LibraryBackend from openblade.domain.errors import DriveCorrelationError +from openblade.domain.models import LibraryInventory from openblade.domain.policies import DryRunPlan from openblade.hardware.discovery import LibraryDiscovery, discover_library, resolve_sg_device from openblade.hardware.library import RealLibraryBackend @@ -59,14 +61,39 @@ def connect_quantum_i3( *, runner: SafeRunner | None = None, ) -> QuantumI3ConnectionReport: + """Diagnose the real robotics backend ``config`` selects. + + Transport-aware: builds the SCSI (``mtx``) or Web Services backend the same + way ``openblade.bootstrap`` does for a running app, so this command + diagnoses whichever transport is actually configured -- previously it + always built ``RealLibraryBackend`` regardless of + ``OPENBLADE_ROBOTICS_TRANSPORT``, which crashed (or silently probed the + wrong thing) against a Scalar reached only over webservices. Fields that + genuinely do not exist on the Web Services backend (a local changer + device, an up-front correlation summary -- see + ``ScalarHttpLibraryBackend``'s docstring) are reported as not applicable + rather than raising ``AttributeError``. + """ guard = require_real_hardware(config) active_runner = runner or SafeRunner(dry_run=config.hardware_dry_run) discovery = discover_library(active_runner, guard) - library = RealLibraryBackend(config=config, runner=active_runner, discovery=discovery) + library: LibraryBackend + if config.robotics_transport == "webservices": + from openblade.bootstrap import _create_scalar_http_library + + library = _create_scalar_http_library(config, active_runner, guard) + else: + library = RealLibraryBackend(config=config, runner=active_runner, discovery=discovery) inventory = library.inventory() + ( + drive_correlation, + drive_correlation_source, + drive_correlation_serials_verified, + drive_correlation_warnings, + ) = _drive_correlation_report(library, inventory) return QuantumI3ConnectionReport( library_id=inventory.library_id, - changer_device=library.changer.device, + changer_device=_changer_device_or_not_applicable(library), discovered_changers=_changer_devices(discovery), discovered_drives=_drive_devices(discovery), slot_count=len(inventory.slots), @@ -79,10 +106,10 @@ def connect_quantum_i3( _drive_device_or_blank(library, drive.drive_id) for drive in inventory.drives ], sg_inquiry=_inquiry_payloads(discovery, active_runner, guard), - drive_correlation=library.correlation.to_payload(), - drive_correlation_source=library.correlation.source, - drive_correlation_serials_verified=library.correlation.serials_verified, - drive_correlation_warnings=list(library.correlation.warnings), + drive_correlation=drive_correlation, + drive_correlation_source=drive_correlation_source, + drive_correlation_serials_verified=drive_correlation_serials_verified, + drive_correlation_warnings=drive_correlation_warnings, ) @@ -132,13 +159,77 @@ def validate_ltfs_capabilities( ) -def _drive_device_or_blank(library: RealLibraryBackend, drive_id: int) -> str: +def _drive_device_or_blank(library: LibraryBackend, drive_id: int) -> str: + """``library.drive_device`` if this backend has one, else "". + + Both real backends (``RealLibraryBackend``, ``ScalarHttpLibraryBackend``) + implement ``drive_device``; the ``LibraryBackend`` protocol does not + declare it (the simulator has no host devices to correlate), so this is a + duck-typed capability check rather than an isinstance branch on a + specific backend class. + """ + drive_device = getattr(library, "drive_device", None) + if drive_device is None: + return "" try: - return library.drive_device(drive_id) + return str(drive_device(drive_id)) except DriveCorrelationError: return "" +def _changer_device_or_not_applicable(library: LibraryBackend) -> str: + """``library.changer.device`` if this backend has a local changer. + + The Web Services backend (``OPENBLADE_ROBOTICS_TRANSPORT=webservices``) + drives the changer over HTTP and has no local device to report -- see + ``ScalarHttpLibraryBackend``, which has no ``changer`` attribute at all. + """ + changer = getattr(library, "changer", None) + if changer is not None: + return str(changer.device) + return ( + "not applicable on webservices transport (the changer is driven over " + "AML Web Services; there is no local changer device)" + ) + + +def _drive_correlation_report( + library: LibraryBackend, inventory: LibraryInventory +) -> tuple[list[dict[str, str | int]], str, bool, list[str]]: + """(drive_correlation, source, serials_verified, warnings) for the report. + + ``RealLibraryBackend`` builds and caches a ``DriveCorrelation`` up front + (``library.correlation``); ``ScalarHttpLibraryBackend`` has no such + aggregate object by design (its class docstring: correlation is resolved + and verified lazily, per drive, the first time ``drive_device()`` is + called) -- so that summary is genuinely not applicable for that + transport, not merely unimplemented. Per-drive devices are still reported + via ``drive_devices``/``_drive_device_or_blank``, which already exercises + -- and reports the refusal from -- that per-drive verification. + """ + correlation = getattr(library, "correlation", None) + if correlation is not None: + return ( + correlation.to_payload(), + correlation.source, + correlation.serials_verified, + list(correlation.warnings), + ) + payload: list[dict[str, str | int]] = [ + {"driveId": drive.drive_id, "device": device, "serial": ""} + for drive in inventory.drives + if (device := _drive_device_or_blank(library, drive.drive_id)) + ] + warnings = [ + "drive correlation summary (source/serials_verified) is not applicable " + "on the webservices transport; each drive_devices entry already went " + "through ScalarHttpLibraryBackend.drive_device()'s own " + "OPENBLADE_DRIVE_SERIAL_MAP verification, and refuses per-drive on a " + "mismatch rather than reporting one here." + ] + return payload, "not_applicable", False, warnings + + def _changer_devices(discovery: LibraryDiscovery) -> list[str]: devices: list[str] = [] for changer in discovery.changers: diff --git a/tests/unit/test_hardware_validation.py b/tests/unit/test_hardware_validation.py index e92506d..5d01193 100644 --- a/tests/unit/test_hardware_validation.py +++ b/tests/unit/test_hardware_validation.py @@ -38,6 +38,152 @@ def test_connect_quantum_i3_returns_inventory_report_in_dry_run(tmp_path: Path) assert report.drive_correlation[0] == {"driveId": 0, "device": "/dev/st0", "serial": ""} +class TestConnectQuantumI3TransportAwareness: + """``connect-i3`` used to hardcode ``RealLibraryBackend`` regardless of + ``OPENBLADE_ROBOTICS_TRANSPORT``, so it could not diagnose a Scalar reached + only over Web Services -- it built the SCSI (``mtx``) backend and then + crashed reading ``library.correlation``/``library.changer``, neither of + which ``ScalarHttpLibraryBackend`` has (see its class docstring). These + pin: (1) the SCSI path is unchanged, (2) the webservices path builds the + real Web Services backend via the same bootstrap builder the app uses, and + (3) fields that are genuinely transport-absent degrade to a named + "not applicable" report instead of raising ``AttributeError``. + """ + + @staticmethod + def _webservices_config(tmp_path: Path) -> OpenBladeConfig: + return OpenBladeConfig( + backend=BackendMode.REAL, + real_hardware_enabled=True, + hardware_dry_run=True, + ltfs_mount_root=str(tmp_path / "ltfs"), + robotics_transport="webservices", + scalar_url="https://library.example/", + scalar_user="admin", + scalar_password="password", + ) + + def test_scsi_transport_is_unaffected(self, tmp_path: Path) -> None: + """MUTATION ANCHOR (partial): the default/explicit ``scsi`` transport + must still build ``RealLibraryBackend`` and report a real correlation + summary, not the webservices "not applicable" degrade path.""" + config = OpenBladeConfig( + backend=BackendMode.REAL, + real_hardware_enabled=True, + hardware_dry_run=True, + ltfs_mount_root=str(tmp_path / "ltfs"), + robotics_transport="scsi", + ) + + report = connect_quantum_i3(config, runner=SafeRunner(dry_run=True)) + + assert report.changer_device == "/dev/sg0" + assert report.drive_correlation_source == "dry_run" + assert report.drive_correlation[0] == {"driveId": 0, "device": "/dev/st0", "serial": ""} + + def test_webservices_transport_builds_the_scalar_http_backend(self, tmp_path: Path) -> None: + """Verified by reading ``connect_quantum_i3``'s call to the same + ``openblade.bootstrap._create_scalar_http_library`` bootstrap uses for + a running app -- stubbed here (per ``tests/i3/test_scalar_http_*``'s + pattern) with a fake session so no real network call happens.""" + import openblade.bootstrap as bootstrap_module + from openblade.hardware.scalar_http import ScalarHttpLibraryBackend + + built: dict[str, object] = {} + + class _StubSession: + def get_json(self, path: str) -> dict[str, object]: + if path == "/aml/physicalLibrary/elements": + return { + "elementList": { + "element": [ + {"type": "slot", "address": 1, "barcode": "OB0001L8"}, + {"type": "slot", "address": 2, "barcode": None}, + {"type": "drive", "address": 1, "barcode": None, "state": "Empty"}, + ] + } + } + raise AssertionError(f"unexpected GET {path}") + + def _fake_create_scalar_http_library( + config: OpenBladeConfig, runner: SafeRunner, guard: object + ) -> ScalarHttpLibraryBackend: + library = ScalarHttpLibraryBackend( + _StubSession(), # type: ignore[arg-type] # structural stand-in for ScalarHttpSession + library_id="stub-i3", + ) + built["library"] = library + return library + + original = bootstrap_module._create_scalar_http_library + bootstrap_module._create_scalar_http_library = _fake_create_scalar_http_library + try: + report = connect_quantum_i3( + self._webservices_config(tmp_path), runner=SafeRunner(dry_run=True) + ) + finally: + bootstrap_module._create_scalar_http_library = original + + assert isinstance(built["library"], ScalarHttpLibraryBackend) + assert report.library_id == "stub-i3" + assert report.slot_count == 2 + assert report.drive_count == 1 + assert report.occupied_slot_count == 1 + + def test_webservices_transport_degrades_absent_capabilities_instead_of_crashing( + self, tmp_path: Path + ) -> None: + """MUTATION ANCHOR: remove the ``getattr(library, "correlation", None)`` + / ``getattr(library, "changer", None)`` guards (go back to + ``library.correlation``/``library.changer`` directly) and this test + fails with ``AttributeError`` instead of asserting the "not + applicable" report.""" + import openblade.bootstrap as bootstrap_module + from openblade.hardware.scalar_http import ScalarHttpLibraryBackend + + class _StubSession: + def get_json(self, path: str) -> dict[str, object]: + return { + "elementList": { + "element": [ + {"type": "drive", "address": 1, "barcode": None, "state": "Empty"}, + ] + } + } + + def _fake_create_scalar_http_library( + config: OpenBladeConfig, runner: SafeRunner, guard: object + ) -> ScalarHttpLibraryBackend: + # No correlation_factory: drive_device() refuses (typed error) -- + # exactly the "no OPENBLADE_DRIVE_SERIAL_MAP declared" case, which + # must be caught, not propagated, same as the SCSI path. + return ScalarHttpLibraryBackend( + _StubSession(), # type: ignore[arg-type] # structural stand-in for ScalarHttpSession + library_id="stub-i3", + ) + + original = bootstrap_module._create_scalar_http_library + bootstrap_module._create_scalar_http_library = _fake_create_scalar_http_library + try: + report = connect_quantum_i3( + self._webservices_config(tmp_path), runner=SafeRunner(dry_run=True) + ) + finally: + bootstrap_module._create_scalar_http_library = original + + assert report.changer_device == ( + "not applicable on webservices transport (the changer is driven over " + "AML Web Services; there is no local changer device)" + ) + assert report.drive_correlation_source == "not_applicable" + assert report.drive_correlation_serials_verified is False + assert report.drive_correlation_warnings + assert report.drive_correlation == [] + # No OPENBLADE_DRIVE_SERIAL_MAP -> drive_device() refuses; the refusal + # is swallowed into "" the same way the SCSI backend's is. + assert report.drive_devices == [""] + + def test_validate_ltfs_capabilities_reports_device_list_and_plan(tmp_path: Path) -> None: report = validate_ltfs_capabilities( _config(tmp_path), From ba83e82b81c9c4f45922f0e0d156b33f93907e83 Mon Sep 17 00:00:00 2001 From: Amantux Date: Sat, 26 Sep 2026 06:05:43 +0000 Subject: [PATCH 11/54] docs(mhvtl): replace the cost estimate with measured step timings MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three green dispatches: 2m41s cold, 2m14s cached, 45-minute timeout. The cold LTFS build is 40s, so the workflow header's premise that LTFS is 'the slowest thing here by far' is wrong — setup.sh is, at 55s, now that it pulls a 51MB linux-modules-extra. --- docs/runbooks/mhvtl-rehearsal.md | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/docs/runbooks/mhvtl-rehearsal.md b/docs/runbooks/mhvtl-rehearsal.md index df5b738..a064712 100644 --- a/docs/runbooks/mhvtl-rehearsal.md +++ b/docs/runbooks/mhvtl-rehearsal.md @@ -589,10 +589,15 @@ health, catalog integrity) rather than going red on a third-party build. The does not emulate, 4 opt-in fault-injection tests, 1 opt-in dirty unmount, 1 absent higher-level locking backend. -Cold runs build LTFS from source inside the 20-minute step budget; the +Measured over three consecutive dispatches: whole job **2m41s cold / 2m14s +cached**, against a 45-minute timeout. The cold LTFS build is **40 s** — not the +"slowest thing here by far" the workflow header assumed, so its 20-minute step +budget is enormously generous. `setup.sh` is now the longest step at 55 s (it +downloads a 51 MB `linux-modules-extra`), and the suite itself takes ~22 s. The `actions/cache` entry is keyed on `LTFS_REF` + runner image identity, and **Actions cache scoping means a branch's cache is not visible to the default -branch**, so the first scheduled run after any image bump pays the cold cost. +branch**, so the first scheduled run after any image bump pays the cold cost — +which, at 40 s, does not matter. ### 7.6 Where this would block on a different runner image From a0d79f968aee047ce923f320da7ccd5c43939360 Mon Sep 17 00:00:00 2001 From: Hermes Agent Date: Sat, 26 Sep 2026 06:12:08 +0000 Subject: [PATCH 12/54] feat(web_flask): land the Flask NAS operator UI Resurrects openblade/web_flask/ from the 2026-09-11 parked state (see docs/decisions/2026-09-11-parked-flask-nas-ci-readme.md). The parked note could not land it because the NAS/AML endpoints it consumes were untracked; all 46 of them now exist on master with matching methods, verified by diffing the client's paths against app.openapi(). The package needed no functional port -- 31/31 of its tests passed unchanged, mypy --strict is clean, and ruff check passed; only `ruff format` drift (the repo reformatted since) had to be applied. A separate WSGI app, not a second implementation of anything: it owns no domain logic and reaches the FastAPI control plane over HTTP via BackendClient. It sits alongside the React SPA rather than replacing it, per CLAUDE.md's "React frontend + Flask NAS UI". Two auth fixes, both found by probing the real backend rather than reading: 1. Native-surface auth was entirely broken. api_auth gates /api, /jobs, /nas, /catalog, /status, /system, /volume-groups, /archive and /restore on the static OPENBLADE_API_TOKEN bearer, while /aml/* authenticates per-user via routes_aml_auth.require_auth. The client sent its AML *session* token as the only bearer, so with native auth enabled all 21 native endpoints -- the whole Storage/Jobs/Reports/System surface -- returned 401. Fixed in one place: require_auth checks the sessionID cookie BEFORE the bearer, so the session token now travels as that cookie and Authorization carries the native token. Falls back to the old behaviour when no native token is configured. 2. The login lockout was bypassable. _client_ip() honoured the caller-controlled X-Forwarded-For, so rotating the header minted a fresh rate-limit bucket per attempt. Now peer-address only, matching the rule already documented on openblade.api.routes_assist.client_key. flask moves into the runtime dependencies because Dockerfile.web serves this under gunicorn and `pip install -e .` must be enough to import it. Both fixes carry mutation-checked regression tests: removing either guard makes the new test fail (verified), while the pre-existing lockout test passed against the vulnerable code -- it was vacuous for the forged-header attack. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_013S1qbQs31YSox1azQChqs2 --- openblade/web_flask/__init__.py | 5 + openblade/web_flask/app.py | 2146 +++++++++++++++++ openblade/web_flask/client.py | 898 +++++++ openblade/web_flask/models.py | 29 + openblade/web_flask/static/css/main.css | 383 +++ .../web_flask/static/js/confirm-actions.js | 21 + openblade/web_flask/static/js/copy-actions.js | 24 + openblade/web_flask/static/js/devices-sse.js | 95 + openblade/web_flask/static/js/jobs-sse.js | 59 + .../web_flask/static/js/operations-search.js | 27 + .../web_flask/static/js/register-form.js | 55 + openblade/web_flask/templates/base.html | 50 + .../web_flask/templates/devices/index.html | 51 + .../web_flask/templates/devices/manage.html | 24 + .../templates/devices/manage_section.html | 723 ++++++ .../web_flask/templates/devices/register.html | 59 + openblade/web_flask/templates/jobs/index.html | 89 + openblade/web_flask/templates/login.html | 23 + .../web_flask/templates/nas_placeholder.html | 13 + .../web_flask/templates/reports/index.html | 81 + .../web_flask/templates/storage/index.html | 677 ++++++ .../web_flask/templates/system/index.html | 121 + pyproject.toml | 5 + tests/unit/test_web_flask_app.py | 1273 ++++++++++ 24 files changed, 6931 insertions(+) create mode 100644 openblade/web_flask/__init__.py create mode 100644 openblade/web_flask/app.py create mode 100644 openblade/web_flask/client.py create mode 100644 openblade/web_flask/models.py create mode 100644 openblade/web_flask/static/css/main.css create mode 100644 openblade/web_flask/static/js/confirm-actions.js create mode 100644 openblade/web_flask/static/js/copy-actions.js create mode 100644 openblade/web_flask/static/js/devices-sse.js create mode 100644 openblade/web_flask/static/js/jobs-sse.js create mode 100644 openblade/web_flask/static/js/operations-search.js create mode 100644 openblade/web_flask/static/js/register-form.js create mode 100644 openblade/web_flask/templates/base.html create mode 100644 openblade/web_flask/templates/devices/index.html create mode 100644 openblade/web_flask/templates/devices/manage.html create mode 100644 openblade/web_flask/templates/devices/manage_section.html create mode 100644 openblade/web_flask/templates/devices/register.html create mode 100644 openblade/web_flask/templates/jobs/index.html create mode 100644 openblade/web_flask/templates/login.html create mode 100644 openblade/web_flask/templates/nas_placeholder.html create mode 100644 openblade/web_flask/templates/reports/index.html create mode 100644 openblade/web_flask/templates/storage/index.html create mode 100644 openblade/web_flask/templates/system/index.html create mode 100644 tests/unit/test_web_flask_app.py diff --git a/openblade/web_flask/__init__.py b/openblade/web_flask/__init__.py new file mode 100644 index 0000000..fe8266b --- /dev/null +++ b/openblade/web_flask/__init__.py @@ -0,0 +1,5 @@ +"""Flask frontend package for NAS-first OpenBlade UX.""" + +from .app import create_app + +__all__ = ["create_app"] diff --git a/openblade/web_flask/app.py b/openblade/web_flask/app.py new file mode 100644 index 0000000..2df70c2 --- /dev/null +++ b/openblade/web_flask/app.py @@ -0,0 +1,2146 @@ +from __future__ import annotations + +import json +import os +import re +import secrets +import time +from collections.abc import Callable, Iterator +from dataclasses import asdict +from datetime import timedelta +from ipaddress import ip_address +from pathlib import PurePosixPath +from urllib.parse import urlparse + +from flask import ( + Flask, + Response, + abort, + flash, + g, + jsonify, + redirect, + render_template, + request, + session, + stream_with_context, + url_for, +) +from flask.typing import ResponseReturnValue + +from .client import BackendClient, BackendError +from .models import Device + +MODEL_OPTIONS: tuple[str, ...] = ("Scalar i3", "Scalar i6", "Scalar i6H") +ROLE_OPTIONS: tuple[tuple[str, str], ...] = ( + ("primary", "Primary Archive"), + ("archive", "Deep Archive"), + ("cold_storage", "Cold Storage (Offline)"), +) +MANAGE_SECTIONS: tuple[tuple[str, str], ...] = ( + ("overview", "Overview"), + ("inventory", "Inventory"), + ("drives", "Drives"), + ("partitions", "Partitions"), + ("jobs", "Jobs"), + ("operations", "Operations"), + ("reports", "Reports"), + ("settings", "Settings"), +) +STORAGE_SECTIONS: tuple[tuple[str, str], ...] = ( + ("overview", "Overview"), + ("access", "Client Access"), + ("write-path", "Write Path"), + ("archive", "Archive"), + ("catalog", "Catalog"), + ("restore", "Restore"), +) +NAS_NAV: tuple[tuple[str, str], ...] = ( + ("Storage", "/storage"), + ("Jobs", "/jobs"), + ("Reports", "/reports"), + ("System", "/system"), +) + +_ASCII_PRINTABLE = re.compile(r"^[\x20-\x7E]{1,64}$") +_SERIAL_ALLOWED = re.compile(r"^[A-Za-z0-9-]{1,32}$") +_DEVICE_USERNAME_ALLOWED = re.compile(r"^[A-Za-z0-9._-]{1,64}$") +_OPERATION_TOKEN = re.compile(r"^[A-Za-z0-9,._:-]{1,64}$") +_BARCODE_TOKEN = re.compile(r"^[A-Za-z0-9._-]{1,64}$") +_POOL_ID_ALLOWED = re.compile(r"^[A-Za-z0-9._:-]{1,64}$") +_SHARE_NAME_ALLOWED = re.compile(r"^[A-Za-z0-9._-]{1,64}$") +_JOB_TYPES = ("archive", "restore", "format", "verify", "inventory", "import", "export") +_JOB_STATES = ("pending", "running", "completed", "failed", "failed_recoverable", "cancelled") +_AML_ROLE_OPTIONS: tuple[tuple[int, str], ...] = ( + (0, "Administrator"), + (1, "Operator"), + (2, "Service"), +) +_POLICY_TYPE_OPTIONS = ("balanced", "critical_sequential", "noncritical_sharded") +_INGEST_MODE_OPTIONS = ("cache_drive", "source_stream") +_SHARD_STRATEGY_OPTIONS = ( + "round_robin", + "capacity_weighted", + "directory_batch", + "hash_prefix", + "restore_parallelism_optimized", +) +_CACHE_EVICTION_OPTIONS = ("never", "after_verified", "after_days", "lru", "manual") +_SOURCE_CHECKSUM_MODES = ("precompute", "streaming", "post_verify", "precompute_and_post_verify") +_SHARE_TYPE_OPTIONS = ("inbox", "restore", "catalog", "virtual", "pool") +_SHARE_ACCESS_OPTIONS = ("read_only", "read_write") +_MAX_PATH_LENGTH = 512 +_LOGIN_WINDOW_SECONDS = 300 +_LOGIN_MAX_ATTEMPTS = 8 +_LOGIN_LOCK_SECONDS = 120 +_DANGEROUS_DEVICE_HOSTS = { + "localhost", + "127.0.0.1", + "::1", + "0.0.0.0", + "169.254.169.254", + "metadata.google.internal", + "metadata.aws.internal", +} + + +def _bool_env(name: str, *, default: bool = False) -> bool: + value = os.environ.get(name) + if value is None: + return default + return value.strip().lower() in {"1", "true", "yes", "on"} + + +def _secret_key() -> str: + configured = os.environ.get("FLASK_SECRET_KEY") + if configured: + return configured + if os.environ.get("OPENBLADE_ENV", "development").lower() == "production": + raise RuntimeError("FLASK_SECRET_KEY is required in production") + return secrets.token_hex(32) + + +def _backend_client_factory() -> BackendClient: + return BackendClient( + base_url=os.environ.get("OPENBLADE_WEB_BACKEND_URL", "http://127.0.0.1:8000"), + timeout_seconds=float(os.environ.get("OPENBLADE_WEB_TIMEOUT_SECONDS", "8")), + service_token=os.environ.get("OPENBLADE_SERVICE_TOKEN"), + # Must equal the backend's OPENBLADE_API_TOKEN. Without it every + # OpenBlade-native endpoint 401s the moment an operator enables native + # auth -- see BackendClient._headers for why the session token cannot + # serve double duty here. + api_token=os.environ.get("OPENBLADE_API_TOKEN"), + ) + + +def _ensure_csrf_token() -> str: + token = session.get("_csrf_token") + if isinstance(token, str) and token: + return token + token = secrets.token_urlsafe(32) + session["_csrf_token"] = token + return token + + +def _validate_csrf() -> bool: + expected = session.get("_csrf_token") + supplied = request.form.get("csrf_token") or request.headers.get("X-CSRF-Token") + if not isinstance(expected, str) or not isinstance(supplied, str): + return False + return secrets.compare_digest(expected, supplied) + + +def _is_login_required(endpoint: str | None) -> bool: + if endpoint is None: + return True + return endpoint not in {"login", "static"} + + +def _parse_int(value: str | None) -> int | None: + if value is None or not value.strip(): + return None + try: + return int(value) + except ValueError: + return None + + +def _client_ip() -> str: + """Identify the caller for login rate limiting and security logging. + + The peer address only. ``X-Forwarded-For`` is deliberately ignored, matching + ``openblade.api.routes_assist.client_key``: it is caller-controlled, so + honouring it would let one client mint unlimited identities and defeat the + login lockout in ``_is_login_blocked`` entirely. Deploy behind a reverse proxy + that rate-limits on its own if you need per-real-client accounting. + """ + return request.remote_addr or "unknown" + + +def _is_safe_redirect_target(target: str) -> bool: + if not target or not target.startswith("/") or target.startswith("//"): + return False + if any(ord(char) < 0x20 for char in target): + return False + parsed = urlparse(target) + return not (parsed.scheme or parsed.netloc) + + +def _validate_device_url(connection_url: str) -> str | None: + parsed_url = urlparse(connection_url) + if parsed_url.scheme not in {"http", "https"} or not parsed_url.netloc: + return "Connection URL must start with http:// or https://" + if parsed_url.username or parsed_url.password: + return "Connection URL must not include embedded credentials." + hostname = (parsed_url.hostname or "").strip().lower() + if not hostname: + return "Connection URL must include a hostname." + allow_unsafe_targets = _bool_env("OPENBLADE_WEB_ALLOW_UNSAFE_DEVICE_TARGETS", default=False) + if not allow_unsafe_targets and hostname in _DANGEROUS_DEVICE_HOSTS: + return "Connection URL host is not allowed by web security policy." + try: + ip_value = ip_address(hostname) + except ValueError: + ip_value = None + if ( + ip_value is not None + and not allow_unsafe_targets + and ( + ip_value.is_loopback + or ip_value.is_link_local + or ip_value.is_unspecified + or ip_value.is_multicast + ) + ): + return "Connection URL host is not allowed by web security policy." + return None + + +def _is_safe_storage_path(path: str) -> bool: + if not path or len(path) > _MAX_PATH_LENGTH or "\x00" in path: + return False + if not path.startswith("/"): + return False + return ".." not in PurePosixPath(path).parts + + +def _attempt_bucket(app: Flask) -> dict[str, dict[str, object]]: + bucket = app.extensions.setdefault("login_attempts", {}) + if isinstance(bucket, dict): + return bucket + raise RuntimeError("login attempt bucket is misconfigured") + + +def _record_login_failure(app: Flask, key: str) -> None: + now = time.monotonic() + bucket = _attempt_bucket(app) + entry = bucket.get(key) + if not isinstance(entry, dict): + entry = {"attempts": [], "blocked_until": 0.0} + bucket[key] = entry + attempts = entry.get("attempts") + if not isinstance(attempts, list): + attempts = [] + entry["attempts"] = attempts + attempts.append(now) + window_start = now - _LOGIN_WINDOW_SECONDS + attempts[:] = [item for item in attempts if isinstance(item, float) and item >= window_start] + if len(attempts) >= _LOGIN_MAX_ATTEMPTS: + entry["blocked_until"] = now + _LOGIN_LOCK_SECONDS + + +def _is_login_blocked(app: Flask, key: str) -> bool: + now = time.monotonic() + bucket = _attempt_bucket(app) + stale_cutoff = now - (_LOGIN_WINDOW_SECONDS * 2) + stale_keys: list[str] = [] + for bucket_key, entry in bucket.items(): + if not isinstance(entry, dict): + stale_keys.append(bucket_key) + continue + blocked_until = entry.get("blocked_until") + attempts = entry.get("attempts") + if not isinstance(blocked_until, float) or not isinstance(attempts, list): + stale_keys.append(bucket_key) + continue + filtered = [item for item in attempts if isinstance(item, float) and item >= stale_cutoff] + entry["attempts"] = filtered + if not filtered and blocked_until < now: + stale_keys.append(bucket_key) + for stale in stale_keys: + bucket.pop(stale, None) + + current_entry = bucket.get(key) + if not isinstance(current_entry, dict): + return False + blocked_until = current_entry.get("blocked_until") + return isinstance(blocked_until, float) and blocked_until > now + + +def _clear_login_failures(app: Flask, key: str) -> None: + _attempt_bucket(app).pop(key, None) + + +def _role_label(role: str) -> str: + for key, label in ROLE_OPTIONS: + if key == role: + return label + return "Unknown Role" + + +def _aml_role_label(role: int | str) -> str: + try: + normalized = int(role) + except (TypeError, ValueError): + return "Unknown Role" + for key, label in _AML_ROLE_OPTIONS: + if key == normalized: + return label + return "Unknown Role" + + +def _status_class(status: str) -> str: + normalized = status.lower() + if normalized in {"online", "good", "healthy"}: + return "status-online" + if normalized in {"degraded", "warning"}: + return "status-warning" + if normalized in {"running", "active"}: + return "status-running" + if normalized in {"offline", "error", "failed", "critical", "unhealthy"}: + return "status-offline" + return "status-unknown" + + +def _status_text(status: str) -> str: + normalized = status.replace("_", " ").strip() + if normalized: + return normalized.capitalize() + return "Unknown" + + +def _job_state_class(state: str) -> str: + return { + "pending": "status-unknown", + "running": "status-running", + "completed": "status-online", + "failed": "status-offline", + "failed_recoverable": "status-warning", + "cancelled": "status-unknown", + }.get(state.lower(), "status-unknown") + + +def _job_type_label(job_type: str) -> str: + return { + "archive": "Archive", + "restore": "Restore", + "format": "Format", + "verify": "Verify", + "inventory": "Inventory", + "import": "Import", + "export": "Export", + }.get(job_type.lower(), _status_text(job_type)) + + +def _share_type_label(share_type: str) -> str: + return { + "inbox": "Ingest Inbox", + "restore": "Restore Output", + "catalog": "Catalog Browser", + "virtual": "Virtual Pool", + "pool": "Pool Share", + }.get(share_type.lower(), _status_text(share_type)) + + +def _connect_host() -> str: + override = os.environ.get("OPENBLADE_CONNECT_HOST", "").strip() + if override: + return override + host = request.host.split(":", 1)[0].strip().lower() + if not host or host in {"0.0.0.0", "::"}: + return "127.0.0.1" + return host + + +def _parse_csv_tokens(raw_value: str) -> list[str]: + tokens = [item.strip() for item in raw_value.split(",")] + return [item for item in tokens if item] + + +def _drive_media_index( + *, + media: list[dict[str, object]], + drives: list[dict[str, object]], + catalog_files: list[dict[str, object]], +) -> list[dict[str, object]]: + drive_by_barcode: dict[str, str] = {} + for drive in drives: + barcode = str(drive.get("loadedMedia") or "").strip() + if not barcode: + continue + drive_id = str(drive.get("serialNumber") or drive.get("id") or "—") + drive_by_barcode[barcode] = drive_id + + files_by_barcode: dict[str, list[str]] = {} + for item in catalog_files: + barcode = str(item.get("primary_barcode") or "").strip() + path = str(item.get("path") or "").strip() + if not barcode or not path: + continue + bucket = files_by_barcode.setdefault(barcode, []) + if len(bucket) < 4: + bucket.append(path) + + index: list[dict[str, object]] = [] + seen_barcodes: set[str] = set() + for item in media: + barcode = str(item.get("barcode") or "").strip() + if not barcode: + continue + seen_barcodes.add(barcode) + linked_files = files_by_barcode.get(barcode, []) + index.append( + { + "barcode": barcode, + "slot": str(item.get("slot") or item.get("location") or "—"), + "partition": str(item.get("partition") or "—"), + "state": str(item.get("state") or "unknown"), + "drive": drive_by_barcode.get(barcode, "—"), + "file_count": len(linked_files), + "sample_paths": linked_files, + } + ) + for barcode, drive_name in drive_by_barcode.items(): + if barcode in seen_barcodes: + continue + linked_files = files_by_barcode.get(barcode, []) + index.append( + { + "barcode": barcode, + "slot": "—", + "partition": "—", + "state": "mounted", + "drive": drive_name, + "file_count": len(linked_files), + "sample_paths": linked_files, + } + ) + return index + + +def _write_path_recommendations( + *, + pools: list[dict[str, object]], + policies: list[dict[str, object]], + source_stream: dict[str, object], +) -> list[str]: + recommendations: list[str] = [] + + def _int_or_default(value: object, default: int) -> int: + try: + if isinstance(value, bool): + return int(value) + if isinstance(value, int): + return value + if isinstance(value, float): + return int(value) + if isinstance(value, str): + return int(value.strip()) + return default + except (TypeError, ValueError): + return default + + if not any(_int_or_default(pool.get("replication_factor"), 1) >= 2 for pool in pools): + recommendations.append( + "Set replication factor to at least 2 on critical pools for tape copy resiliency." + ) + if not any(bool(policy.get("allow_sharding")) for policy in policies): + recommendations.append( + "Enable sharding on at least one policy to parallelize multi-drive writes." + ) + if not any(_int_or_default(policy.get("shard_size_bytes"), 0) > 0 for policy in policies): + recommendations.append( + "Set shard_size_bytes on sharded policies to control per-drive shard wave sizing." + ) + if not any(bool(policy.get("auto_clean_before_archive", True)) for policy in policies): + recommendations.append( + "Keep auto-clean enabled for active policies to avoid ingest failures from overdue drives." + ) + if not any( + str(policy.get("default_ingest_mode") or "").lower() == "source_stream" + for policy in policies + ): + recommendations.append( + "Add a source-stream policy for low-latency direct-to-tape ingest workflows." + ) + + checksum_mode = str(source_stream.get("checksum_mode") or "").lower() + if checksum_mode not in {"streaming", "precompute_and_post_verify"}: + recommendations.append( + "Use streaming or precompute+post-verify checksum mode to reduce integrity blind spots." + ) + if not bool(source_stream.get("preflight_read_check", True)): + recommendations.append( + "Enable source preflight read checks to fail fast before expensive tape movement." + ) + if _int_or_default(source_stream.get("max_retries"), 0) < 2: + recommendations.append( + "Set source-stream retries to at least 2 to absorb transient network/read errors." + ) + + if not recommendations: + recommendations.append( + "Current write-path profile is balanced for replication, integrity checks, and throughput." + ) + return recommendations + + +def _write_path_metrics( + *, + pools: list[dict[str, object]], + policies: list[dict[str, object]], + shares: list[dict[str, object]], + source_stream: dict[str, object], +) -> dict[str, int]: + def _int_or_default(value: object, default: int) -> int: + try: + if isinstance(value, bool): + return int(value) + if isinstance(value, int): + return value + if isinstance(value, float): + return int(value) + if isinstance(value, str): + return int(value.strip()) + return default + except (TypeError, ValueError): + return default + + replicated_pools = sum( + 1 for pool in pools if _int_or_default(pool.get("replication_factor"), 1) >= 2 + ) + sharded_policies = sum(1 for policy in policies if bool(policy.get("allow_sharding"))) + auto_clean_policies = sum( + 1 for policy in policies if bool(policy.get("auto_clean_before_archive", True)) + ) + folder_rules = 0 + for share in shares: + mappings = share.get("folder_mappings") + if isinstance(mappings, list): + folder_rules += len(mappings) + writable_shares = sum(1 for share in shares if bool(share.get("writable"))) + source_stream_enabled = 1 if bool(source_stream.get("enabled")) else 0 + return { + "replicated_pools": replicated_pools, + "sharded_policies": sharded_policies, + "auto_clean_policies": auto_clean_policies, + "folder_rules": folder_rules, + "writable_shares": writable_shares, + "source_stream_enabled": source_stream_enabled, + } + + +def _summary(devices: list[Device]) -> dict[str, int]: + online = sum(1 for item in devices if item.status == "online") + warning = sum(1 for item in devices if item.status in {"degraded", "warning"}) + offline = sum(1 for item in devices if item.status in {"offline", "error", "failed"}) + unknown = max(len(devices) - online - warning - offline, 0) + return { + "total": len(devices), + "online": online, + "warning": warning, + "offline": offline, + "unknown": unknown, + } + + +def _pick_device(devices: list[Device], requested_id: int | None) -> Device | None: + if requested_id is not None: + for device in devices: + if device.id == requested_id: + return device + return devices[0] if devices else None + + +def _backend(app: Flask) -> BackendClient: + client = app.extensions.get("backend_client") + if not isinstance(client, BackendClient): + raise RuntimeError("Backend client is not configured") + return client + + +def create_app(client_factory: Callable[[], BackendClient] | None = None) -> Flask: + app = Flask(__name__, template_folder="templates", static_folder="static") + app.config.update( + SECRET_KEY=_secret_key(), + SESSION_COOKIE_HTTPONLY=True, + SESSION_COOKIE_SAMESITE="Lax", + SESSION_COOKIE_SECURE=_bool_env( + "OPENBLADE_WEB_SECURE_COOKIES", + default=os.environ.get("OPENBLADE_ENV", "development").lower() == "production", + ), + PERMANENT_SESSION_LIFETIME=timedelta(hours=8), + SESSION_REFRESH_EACH_REQUEST=False, + MAX_CONTENT_LENGTH=1_000_000, + ) + app.extensions["backend_client"] = (client_factory or _backend_client_factory)() + + @app.before_request + def _load_context() -> ResponseReturnValue | None: + g.nas_nav = NAS_NAV + g.current_user = session.get("username") + g.csrf_token = _ensure_csrf_token() + if _is_login_required(request.endpoint) and "api_token" not in session: + return redirect(url_for("login", next=request.path)) + if request.method in {"POST", "PUT", "PATCH", "DELETE"} and not _validate_csrf(): + app.logger.warning( + "CSRF validation failed", extra={"endpoint": request.endpoint, "ip": _client_ip()} + ) + abort(400, description="Invalid CSRF token") + return None + + @app.after_request + def _security_headers(response: Response) -> Response: + response.headers["X-Content-Type-Options"] = "nosniff" + response.headers["X-Frame-Options"] = "DENY" + response.headers["Referrer-Policy"] = "strict-origin-when-cross-origin" + response.headers["Permissions-Policy"] = "camera=(), microphone=(), geolocation=()" + csp_directives = [ + "default-src 'self'", + "script-src 'self'", + "style-src 'self'", + "img-src 'self' data:", + "connect-src 'self'", + "frame-ancestors 'none'", + "base-uri 'self'", + "form-action 'self'", + ] + if app.config.get("SESSION_COOKIE_SECURE"): + csp_directives.append("upgrade-insecure-requests") + response.headers["Strict-Transport-Security"] = "max-age=31536000; includeSubDomains" + response.headers["Content-Security-Policy"] = "; ".join(csp_directives) + if "api_token" in session: + response.headers["Cache-Control"] = "no-store, no-cache, must-revalidate" + response.headers["Pragma"] = "no-cache" + response.headers["Expires"] = "0" + return response + + @app.context_processor + def _inject_template_context() -> dict[str, object]: + return { + "nas_nav": NAS_NAV, + "csrf_token": _ensure_csrf_token(), + "current_user": session.get("username"), + "status_class": _status_class, + "status_text": _status_text, + "role_label": _role_label, + "job_state_class": _job_state_class, + "job_type_label": _job_type_label, + "share_type_label": _share_type_label, + "aml_role_label": _aml_role_label, + "job_types": _JOB_TYPES, + "job_states": _JOB_STATES, + "manage_sections": MANAGE_SECTIONS, + "storage_sections": STORAGE_SECTIONS, + "role_options": ROLE_OPTIONS, + "aml_role_options": _AML_ROLE_OPTIONS, + "policy_type_options": _POLICY_TYPE_OPTIONS, + "ingest_mode_options": _INGEST_MODE_OPTIONS, + "shard_strategy_options": _SHARD_STRATEGY_OPTIONS, + "cache_eviction_options": _CACHE_EVICTION_OPTIONS, + "source_checksum_modes": _SOURCE_CHECKSUM_MODES, + "share_type_options": _SHARE_TYPE_OPTIONS, + "share_access_options": _SHARE_ACCESS_OPTIONS, + } + + def _token() -> str: + token = session.get("api_token") + if not isinstance(token, str) or not token: + abort(401) + return token + + def _handle_backend_error(error: BackendError) -> ResponseReturnValue: + if error.status_code == 401: + session.clear() + return redirect(url_for("login")) + flash(error.detail, "error") + return redirect(url_for("devices_index")) + + @app.get("/") + def home() -> ResponseReturnValue: + if "api_token" not in session: + return redirect(url_for("login")) + return redirect(url_for("devices_index")) + + @app.route("/login", methods=["GET", "POST"]) + def login() -> ResponseReturnValue: + next_path = request.values.get("next", "").strip() + if request.method == "GET": + return Response(render_template("login.html", next_path=next_path), status=200) + + username = request.form.get("username", "").strip() + password = request.form.get("password", "") + login_key = f"{_client_ip()}:{username.lower()[:64]}" + if _is_login_blocked(app, login_key): + flash("Too many login attempts. Try again in a few minutes.", "error") + return Response(render_template("login.html", next_path=next_path), status=429) + if not username or not password: + _record_login_failure(app, login_key) + flash("Username and password are required.", "error") + return Response(render_template("login.html", next_path=next_path), status=400) + + try: + token = _backend(app).login(username=username, password=password) + except BackendError as error: + _record_login_failure(app, login_key) + app.logger.warning("Login failed", extra={"username": username, "ip": _client_ip()}) + flash(error.detail, "error") + return Response(render_template("login.html", next_path=next_path), status=401) + + session.clear() + session["username"] = username + session["api_token"] = token + session.permanent = True + _ensure_csrf_token() + _clear_login_failures(app, login_key) + if _is_safe_redirect_target(next_path): + return redirect(next_path) + return redirect(url_for("devices_index")) + + @app.post("/logout") + def logout() -> ResponseReturnValue: + session.clear() + return redirect(url_for("login")) + + @app.get("/devices") + def devices_index() -> ResponseReturnValue: + try: + devices = _backend(app).list_devices(_token()) + except BackendError as error: + return _handle_backend_error(error) + return Response( + render_template( + "devices/index.html", + devices=devices, + summary=_summary(devices), + page_id="devices-index", + ), + status=200, + ) + + @app.route("/devices/register", methods=["GET", "POST"]) + def devices_register() -> ResponseReturnValue: + values = { + "name": request.form.get("name", "").strip(), + "connection_url": request.form.get("connection_url", "").strip(), + "serial_number": request.form.get("serial_number", "").strip(), + "model": request.form.get("model", MODEL_OPTIONS[0]), + "device_username": request.form.get("device_username", "").strip(), + } + device_password = request.form.get("device_password", "") + errors: dict[str, str] = {} + + if request.method == "POST": + if not _ASCII_PRINTABLE.fullmatch(values["name"]): + errors["name"] = "Device name must be 1-64 printable ASCII characters." + connection_error = _validate_device_url(values["connection_url"]) + if connection_error: + errors["connection_url"] = connection_error + if values["serial_number"] and not _SERIAL_ALLOWED.fullmatch(values["serial_number"]): + errors["serial_number"] = "Serial number accepts letters, numbers, and dashes only." + if values["model"] not in MODEL_OPTIONS: + errors["model"] = "Unsupported device model." + if values["device_username"] and not _DEVICE_USERNAME_ALLOWED.fullmatch( + values["device_username"] + ): + errors["device_username"] = ( + "Device username accepts letters, numbers, dot, underscore, and dash." + ) + if device_password and not values["device_username"]: + errors["device_username"] = ( + "Provide a device username when testing authenticated connection." + ) + + if not errors: + try: + _backend(app).probe_device_endpoint( + connection_url=values["connection_url"], + username=values["device_username"] or None, + password=device_password or None, + ) + except BackendError as error: + errors["connection_url"] = error.detail + + if not errors: + try: + devices = _backend(app).list_devices(_token()) + next_order = max((item.sort_order for item in devices), default=0) + 1 + _backend(app).create_device( + _token(), + { + "name": values["name"], + "emulator_url": values["connection_url"], + "serial_number": values["serial_number"] or None, + "model": values["model"], + "role": "primary", + "enabled": True, + "sort_order": next_order, + }, + ) + except BackendError as error: + return _handle_backend_error(error) + flash("Device registered successfully.", "success") + return redirect(url_for("devices_index")) + + status = 400 if request.method == "POST" and errors else 200 + return Response( + render_template( + "devices/register.html", + model_options=MODEL_OPTIONS, + values=values, + errors=errors, + page_id="devices-register", + ), + status=status, + ) + + @app.get("/devices//manage") + def devices_manage(device_id: int) -> ResponseReturnValue: + return redirect(url_for("devices_manage_section", device_id=device_id, section="overview")) + + @app.post("/devices//settings") + def devices_update_settings(device_id: int) -> ResponseReturnValue: + name = request.form.get("name", "").strip() + connection_url = request.form.get("connection_url", "").strip() + enabled = request.form.get("enabled", "").strip().lower() == "true" + if not _ASCII_PRINTABLE.fullmatch(name): + flash("Device name must be 1-64 printable ASCII characters.", "error") + return redirect( + url_for("devices_manage_section", device_id=device_id, section="settings") + ) + connection_error = _validate_device_url(connection_url) + if connection_error: + flash(connection_error, "error") + return redirect( + url_for("devices_manage_section", device_id=device_id, section="settings") + ) + + try: + current = _backend(app).get_device(_token(), device_id) + if current is None: + abort(404, description="Device not found") + _backend(app).probe_device_endpoint(connection_url=connection_url) + _backend(app).update_device( + _token(), + device_id, + { + "name": name, + "emulator_url": connection_url, + "model": current.model, + "role": current.role or "primary", + "enabled": enabled, + "sort_order": current.sort_order, + }, + ) + except BackendError as error: + return _handle_backend_error(error) + flash("Device settings updated.", "success") + return redirect(url_for("devices_manage_section", device_id=device_id, section="settings")) + + @app.post("/devices//operations/inventory") + def devices_trigger_inventory(device_id: int) -> ResponseReturnValue: + try: + _backend(app).trigger_inventory(_token(), library_id=device_id) + except BackendError as error: + return _handle_backend_error(error) + flash("Inventory scan started.", "success") + return redirect( + url_for("devices_manage_section", device_id=device_id, section="operations") + ) + + @app.post("/devices//operations/move") + def devices_move_media(device_id: int) -> ResponseReturnValue: + source = request.form.get("source", "").strip() + destination = request.form.get("destination", "").strip() + barcode = request.form.get("barcode", "").strip() + if not source or not destination: + flash("Source and destination are required.", "error") + return redirect( + url_for("devices_manage_section", device_id=device_id, section="operations") + ) + if not _OPERATION_TOKEN.fullmatch(source) or not _OPERATION_TOKEN.fullmatch(destination): + flash( + "Source and destination accept letters, numbers, comma, dot, underscore, dash, and colon.", + "error", + ) + return redirect( + url_for("devices_manage_section", device_id=device_id, section="operations") + ) + if barcode and not _BARCODE_TOKEN.fullmatch(barcode): + flash("Barcode accepts letters, numbers, dot, underscore, and dash only.", "error") + return redirect( + url_for("devices_manage_section", device_id=device_id, section="operations") + ) + try: + _backend(app).move_media_operation( + _token(), + library_id=device_id, + source=source, + destination=destination, + barcode=barcode or None, + ) + except BackendError as error: + return _handle_backend_error(error) + flash("Move operation queued.", "success") + return redirect( + url_for("devices_manage_section", device_id=device_id, section="operations") + ) + + @app.post("/devices//operations/mount") + def devices_mount_media(device_id: int) -> ResponseReturnValue: + barcode = request.form.get("barcode", "").strip() + drive = request.form.get("drive", "").strip() + partition = request.form.get("partition", "").strip() + if not barcode or not drive: + flash("Barcode and drive are required.", "error") + return redirect( + url_for("devices_manage_section", device_id=device_id, section="operations") + ) + if not _BARCODE_TOKEN.fullmatch(barcode): + flash("Barcode accepts letters, numbers, dot, underscore, and dash only.", "error") + return redirect( + url_for("devices_manage_section", device_id=device_id, section="operations") + ) + if not _OPERATION_TOKEN.fullmatch(drive): + flash( + "Drive accepts letters, numbers, comma, dot, underscore, dash, and colon.", "error" + ) + return redirect( + url_for("devices_manage_section", device_id=device_id, section="operations") + ) + if partition and not _OPERATION_TOKEN.fullmatch(partition): + flash( + "Partition accepts letters, numbers, comma, dot, underscore, dash, and colon.", + "error", + ) + return redirect( + url_for("devices_manage_section", device_id=device_id, section="operations") + ) + try: + _backend(app).mount_media_operation( + _token(), + library_id=device_id, + barcode=barcode, + drive=drive, + partition=partition or None, + ) + except BackendError as error: + return _handle_backend_error(error) + flash("Mount operation queued.", "success") + return redirect( + url_for("devices_manage_section", device_id=device_id, section="operations") + ) + + @app.post("/devices//operations/unmount") + def devices_unmount_media(device_id: int) -> ResponseReturnValue: + barcode = request.form.get("barcode", "").strip() + drive = request.form.get("drive", "").strip() + if not barcode or not drive: + flash("Barcode and drive are required.", "error") + return redirect( + url_for("devices_manage_section", device_id=device_id, section="operations") + ) + if not _BARCODE_TOKEN.fullmatch(barcode): + flash("Barcode accepts letters, numbers, dot, underscore, and dash only.", "error") + return redirect( + url_for("devices_manage_section", device_id=device_id, section="operations") + ) + if not _OPERATION_TOKEN.fullmatch(drive): + flash( + "Drive accepts letters, numbers, comma, dot, underscore, dash, and colon.", "error" + ) + return redirect( + url_for("devices_manage_section", device_id=device_id, section="operations") + ) + try: + _backend(app).unmount_media_operation( + _token(), + library_id=device_id, + barcode=barcode, + drive=drive, + ) + except BackendError as error: + return _handle_backend_error(error) + flash("Unmount operation queued.", "success") + return redirect( + url_for("devices_manage_section", device_id=device_id, section="operations") + ) + + @app.post("/devices//operations/import") + def devices_import_media(device_id: int) -> ResponseReturnValue: + partition = request.form.get("partition", "").strip() + ie_station = request.form.get("ie_station", "").strip() + if not partition or not ie_station: + flash("Partition and I/E station are required.", "error") + return redirect( + url_for("devices_manage_section", device_id=device_id, section="operations") + ) + if not _OPERATION_TOKEN.fullmatch(partition) or not _OPERATION_TOKEN.fullmatch(ie_station): + flash( + "Partition and I/E station accept letters, numbers, comma, dot, underscore, dash, and colon.", + "error", + ) + return redirect( + url_for("devices_manage_section", device_id=device_id, section="operations") + ) + try: + _backend(app).start_import_operation( + _token(), + library_id=device_id, + partition=partition, + ie_station=ie_station, + ) + except BackendError as error: + return _handle_backend_error(error) + flash("Import operation queued.", "success") + return redirect( + url_for("devices_manage_section", device_id=device_id, section="operations") + ) + + @app.post("/devices//operations/export") + def devices_export_media(device_id: int) -> ResponseReturnValue: + raw_barcodes = request.form.get("barcodes", "").strip() + ie_station = request.form.get("ie_station", "").strip() + barcodes = [item for item in re.split(r"[\s,]+", raw_barcodes) if item] + if not barcodes or not ie_station: + flash("At least one barcode and an I/E station are required.", "error") + return redirect( + url_for("devices_manage_section", device_id=device_id, section="operations") + ) + if not _OPERATION_TOKEN.fullmatch(ie_station): + flash( + "I/E station accepts letters, numbers, comma, dot, underscore, dash, and colon.", + "error", + ) + return redirect( + url_for("devices_manage_section", device_id=device_id, section="operations") + ) + if any(_BARCODE_TOKEN.fullmatch(item) is None for item in barcodes): + flash("Each barcode must use letters, numbers, dot, underscore, or dash.", "error") + return redirect( + url_for("devices_manage_section", device_id=device_id, section="operations") + ) + try: + _backend(app).start_export_operation( + _token(), + library_id=device_id, + barcodes=barcodes, + ie_station=ie_station, + ) + except BackendError as error: + return _handle_backend_error(error) + flash("Export operation queued.", "success") + return redirect( + url_for("devices_manage_section", device_id=device_id, section="operations") + ) + + @app.post("/devices//operations/restore") + def devices_restore_from_tape(device_id: int) -> ResponseReturnValue: + catalog_path = request.form.get("catalog_path", "").strip() + dest_path = request.form.get("dest_path", "").strip() + if not catalog_path or not dest_path: + flash("Catalog path and destination path are required.", "error") + return redirect( + url_for("devices_manage_section", device_id=device_id, section="operations") + ) + if not _is_safe_storage_path(catalog_path) or not _is_safe_storage_path(dest_path): + flash( + "Catalog and destination paths must be absolute normalized paths within policy limits.", + "error", + ) + return redirect( + url_for("devices_manage_section", device_id=device_id, section="operations") + ) + try: + _backend(app).create_restore_job( + _token(), catalog_path=catalog_path, dest_path=dest_path + ) + except BackendError as error: + return _handle_backend_error(error) + flash( + "Restore hydration job submitted. Tape loading will be handled by job workflow.", + "success", + ) + return redirect( + url_for("devices_manage_section", device_id=device_id, section="operations") + ) + + @app.post("/devices//operations/pool") + def devices_create_pool(device_id: int) -> ResponseReturnValue: + pool_id = request.form.get("pool_id", "").strip() + pool_name = request.form.get("pool_name", "").strip() + description = request.form.get("description", "").strip() + mount_path = request.form.get("mount_path", "").strip() + restore_target_path = ( + request.form.get("restore_target_path", "").strip() or "/openblade/restore" + ) + volume_group_ids_raw = request.form.get("volume_group_ids", "").strip() + access_mode = request.form.get("access_mode", "read_only").strip().lower() + backup_order_mode = request.form.get("backup_order_mode", "sequential").strip().lower() + replication_factor_raw = request.form.get("replication_factor", "1").strip() + virtual_mount_enabled = request.form.get("virtual_mount_enabled", "").strip().lower() in { + "1", + "true", + "yes", + "on", + } + + if not _POOL_ID_ALLOWED.fullmatch(pool_id): + flash("Pool ID accepts letters, numbers, dot, underscore, dash, and colon.", "error") + return redirect( + url_for("devices_manage_section", device_id=device_id, section="operations") + ) + if not _ASCII_PRINTABLE.fullmatch(pool_name): + flash("Pool name must be 1-64 printable ASCII characters.", "error") + return redirect( + url_for("devices_manage_section", device_id=device_id, section="operations") + ) + if mount_path and not _is_safe_storage_path(mount_path): + flash("Pool mount path must be an absolute normalized path.", "error") + return redirect( + url_for("devices_manage_section", device_id=device_id, section="operations") + ) + if not _is_safe_storage_path(restore_target_path): + flash("Restore target path must be an absolute normalized path.", "error") + return redirect( + url_for("devices_manage_section", device_id=device_id, section="operations") + ) + if access_mode not in {"read_only", "read_write"}: + flash("Pool access mode must be read_only or read_write.", "error") + return redirect( + url_for("devices_manage_section", device_id=device_id, section="operations") + ) + if backup_order_mode not in {"sequential", "parallel"}: + flash("Backup order mode must be sequential or parallel.", "error") + return redirect( + url_for("devices_manage_section", device_id=device_id, section="operations") + ) + try: + replication_factor = int(replication_factor_raw) + except ValueError: + flash("Replication factor must be a number between 1 and 4.", "error") + return redirect( + url_for("devices_manage_section", device_id=device_id, section="operations") + ) + if replication_factor < 1 or replication_factor > 4: + flash("Replication factor must be between 1 and 4.", "error") + return redirect( + url_for("devices_manage_section", device_id=device_id, section="operations") + ) + + volume_group_ids = _parse_csv_tokens(volume_group_ids_raw) + if any(_POOL_ID_ALLOWED.fullmatch(item) is None for item in volume_group_ids): + flash( + "Volume group IDs accept letters, numbers, dot, underscore, dash, and colon.", + "error", + ) + return redirect( + url_for("devices_manage_section", device_id=device_id, section="operations") + ) + + payload: dict[str, object] = { + "id": pool_id, + "name": pool_name, + "description": description or None, + "volume_group_ids": volume_group_ids, + "mount_path": mount_path or None, + "virtual_mount_enabled": virtual_mount_enabled, + "restore_target_path": restore_target_path, + "access_mode": access_mode, + "replication_factor": replication_factor, + "backup_order_mode": backup_order_mode, + } + try: + _backend(app).create_nas_pool(_token(), payload) + except BackendError as error: + return _handle_backend_error(error) + flash("Pool created or updated.", "success") + return redirect( + url_for("devices_manage_section", device_id=device_id, section="operations") + ) + + @app.post("/devices//operations/share") + def devices_create_pool_share(device_id: int) -> ResponseReturnValue: + pool_id = request.form.get("pool_id", "").strip() + share_name = request.form.get("share_name", "").strip() + writable = request.form.get("writable", "").strip().lower() in {"1", "true", "yes", "on"} + if not _POOL_ID_ALLOWED.fullmatch(pool_id): + flash("Pool ID accepts letters, numbers, dot, underscore, dash, and colon.", "error") + return redirect( + url_for("devices_manage_section", device_id=device_id, section="operations") + ) + if not _SHARE_NAME_ALLOWED.fullmatch(share_name): + flash("Share name accepts letters, numbers, dot, underscore, and dash.", "error") + return redirect( + url_for("devices_manage_section", device_id=device_id, section="operations") + ) + share_path = f"/pools/{pool_id}" + payload = { + "path": share_path, + "name": share_name, + "share_type": "pool", + "writable": writable, + "description": f"Pool mapped share for {pool_id}", + } + try: + _backend(app).create_nas_share(_token(), payload) + except BackendError as error: + return _handle_backend_error(error) + flash("Pool share created or updated.", "success") + return redirect( + url_for("devices_manage_section", device_id=device_id, section="operations") + ) + + @app.post("/devices//operations/magazine//eject") + def devices_eject_magazine(device_id: int, magazine_id: str) -> ResponseReturnValue: + if not _OPERATION_TOKEN.fullmatch(magazine_id): + flash("Magazine ID is invalid.", "error") + return redirect( + url_for("devices_manage_section", device_id=device_id, section="operations") + ) + try: + _backend(app).eject_magazine(_token(), library_id=device_id, magazine_id=magazine_id) + except BackendError as error: + return _handle_backend_error(error) + flash(f"Magazine {magazine_id} eject requested.", "success") + return redirect( + url_for("devices_manage_section", device_id=device_id, section="operations") + ) + + @app.post("/devices//operations/magazine//insert") + def devices_insert_magazine(device_id: int, magazine_id: str) -> ResponseReturnValue: + if not _OPERATION_TOKEN.fullmatch(magazine_id): + flash("Magazine ID is invalid.", "error") + return redirect( + url_for("devices_manage_section", device_id=device_id, section="operations") + ) + try: + _backend(app).insert_magazine(_token(), library_id=device_id, magazine_id=magazine_id) + except BackendError as error: + return _handle_backend_error(error) + flash(f"Magazine {magazine_id} insert requested.", "success") + return redirect( + url_for("devices_manage_section", device_id=device_id, section="operations") + ) + + @app.get("/devices//manage/
") + def devices_manage_section(device_id: int, section: str) -> ResponseReturnValue: + section_keys = {key for key, _ in MANAGE_SECTIONS} + if section not in section_keys: + abort(404) + try: + device = _backend(app).get_device(_token(), device_id) + if device is None: + abort(404, description="Device not found") + section_data: dict[str, object] = {} + if section == "overview": + section_data["library_status"] = _backend(app).get_library_status( + _token(), library_id=device_id + ) + section_data["drives"] = _backend(app).list_drives(_token(), library_id=device_id) + section_data["media_count"] = len( + _backend(app).list_media(_token(), library_id=device_id, limit=500) + ) + elif section == "inventory": + section_data["media"] = _backend(app).list_media( + _token(), library_id=device_id, limit=100 + ) + elif section == "drives": + section_data["drives"] = _backend(app).list_drives(_token(), library_id=device_id) + elif section == "partitions": + section_data["partitions"] = _backend(app).list_partitions( + _token(), library_id=device_id + ) + elif section == "jobs": + section_data["jobs"] = _backend(app).list_jobs(_token(), library_id=device_id) + elif section == "operations": + jobs = _backend(app).list_jobs(_token(), library_id=device_id) + drives = _backend(app).list_drives(_token(), library_id=device_id) + media = _backend(app).list_media(_token(), library_id=device_id, limit=200) + catalog_files = ( + _backend(app).list_catalog_files(_token(), limit=200).get("files") or [] + ) + if not isinstance(catalog_files, list): + catalog_files = [] + section_data["drives"] = drives + section_data["media"] = media + section_data["partitions"] = _backend(app).list_partitions( + _token(), library_id=device_id + ) + section_data["ie_stations"] = _backend(app).list_ie_stations( + _token(), library_id=device_id + ) + section_data["mounts"] = _backend(app).list_mounts(_token(), library_id=device_id) + section_data["pools"] = _backend(app).list_nas_pools(_token()) + section_data["shares"] = _backend(app).list_nas_shares(_token()) + section_data["volume_groups"] = _backend(app).list_volume_groups(_token()) + magazines = _backend(app).list_magazines(_token(), library_id=device_id) + magazines_with_mapping: list[dict[str, object]] = [] + for magazine in magazines: + magazine_id = str(magazine.get("id") or "").strip() + if not magazine_id: + continue + slots = _backend(app).list_magazine_slots( + _token(), library_id=device_id, magazine_id=magazine_id + ) + entry = dict(magazine) + entry["mappedSlotCount"] = len(slots) + entry["slotMapAligned"] = int(magazine.get("slotCount") or 0) == len(slots) + magazines_with_mapping.append(entry) + section_data["magazines"] = magazines_with_mapping + section_data["catalog_files"] = [ + item for item in catalog_files if isinstance(item, dict) + ] + section_data["drive_media_index"] = _drive_media_index( + media=[item for item in media if isinstance(item, dict)], + drives=[item for item in drives if isinstance(item, dict)], + catalog_files=[item for item in catalog_files if isinstance(item, dict)], + ) + section_data["inventory_status"] = _backend(app).get_inventory_status( + _token(), library_id=device_id + ) + section_data["import_status"] = _backend(app).get_import_status( + _token(), library_id=device_id + ) + section_data["export_status"] = _backend(app).get_export_status( + _token(), library_id=device_id + ) + section_data["active_ops"] = [ + job + for job in jobs + if str(job.get("state", "")).lower() in {"pending", "running"} + ] + elif section == "reports": + section_data["event_summary"] = _backend(app).get_event_summary( + _token(), library_id=device_id + ) + section_data["alert_summary"] = _backend(app).get_alert_summary( + _token(), library_id=device_id + ) + section_data["events"] = _backend(app).list_events( + _token(), library_id=device_id, limit=30 + ) + elif section == "settings": + section_data["device"] = device + except BackendError as error: + return _handle_backend_error(error) + return Response( + render_template( + "devices/manage_section.html", + device=device, + section=section, + sections=MANAGE_SECTIONS, + section_data=section_data, + page_id=f"devices-manage-section-{section}", + ), + status=200, + ) + + @app.get("/storage") + @app.get("/storage/
") + def storage(section: str = "overview") -> ResponseReturnValue: + section_keys = {key for key, _ in STORAGE_SECTIONS} + if section not in section_keys: + abort(404) + selected_id = _parse_int(request.args.get("library_id")) + try: + devices = _backend(app).list_devices(_token()) + active_device = _pick_device(devices, selected_id) + section_data: dict[str, object] = {} + if section == "overview": + if active_device is not None: + section_data["library_status"] = _backend(app).get_library_status( + _token(), library_id=active_device.id + ) + section_data["media_pools"] = _backend(app).list_media_pools( + _token(), library_id=active_device.id + ) + section_data["catalog_status"] = _backend(app).get_catalog_status(_token()) + elif section == "access": + shares = _backend(app).list_nas_shares(_token()) + pools = _backend(app).list_nas_pools(_token()) + gateway_config = _backend(app).get_gateway_config(_token()) + gateway_status = _backend(app).get_gateway_status(_token()) + gateway_credentials = _backend(app).list_gateway_credentials(_token()) + connect_host = _connect_host() + enabled_credentials = [ + item + for item in gateway_credentials + if bool(item.get("enabled", True)) and item.get("username") + ] + example_username = ( + str(enabled_credentials[0]["username"]) + if enabled_credentials + else str(session.get("username") or "admin") + ) + share_access: list[dict[str, object]] = [] + for share in shares: + share_name = str(share.get("name") or "").strip() + share_path = str(share.get("path") or "").strip() + if not share_name or not share_path: + continue + mount_point = f"/mnt/openblade/{share_name}" + windows_unc = f"\\\\{connect_host}\\{share_name}" + share_access.append( + { + "name": share_name, + "path": share_path, + "share_type": str(share.get("share_type") or "share"), + "writable": bool(share.get("writable", False)), + "windows_unc": windows_unc, + "windows_command": ( + f"net use Z: {windows_unc} /user:{example_username} /persistent:yes" + ), + "linux_cifs_command": ( + f"sudo mount -t cifs //{connect_host}/{share_name} {mount_point} " + f"-o username={example_username},uid=$(id -u),gid=$(id -g),vers=3.0" + ), + "linux_nfs_command": f"sudo mount -t nfs {connect_host}:{share_path} {mount_point}", + } + ) + section_data["connect_host"] = connect_host + section_data["gateway_config"] = gateway_config + section_data["gateway_status"] = gateway_status + section_data["gateway_credentials"] = gateway_credentials + section_data["nas_shares"] = shares + section_data["nas_pools"] = pools + section_data["share_access"] = share_access + section_data["example_username"] = example_username + elif section == "write-path": + pools = _backend(app).list_nas_pools(_token()) + shares = _backend(app).list_nas_shares(_token()) + policies = _backend(app).list_nas_policies(_token()) + cache_drives = _backend(app).list_cache_drives(_token()) + source_stream = _backend(app).get_source_stream_config(_token()) + section_data["nas_pools"] = pools + section_data["nas_shares"] = shares + section_data["policies"] = policies + section_data["cache_drives"] = cache_drives + section_data["source_stream"] = source_stream + section_data["recommendations"] = _write_path_recommendations( + pools=[item for item in pools if isinstance(item, dict)], + policies=[item for item in policies if isinstance(item, dict)], + source_stream=source_stream if isinstance(source_stream, dict) else {}, + ) + section_data["metrics"] = _write_path_metrics( + pools=[item for item in pools if isinstance(item, dict)], + policies=[item for item in policies if isinstance(item, dict)], + shares=[item for item in shares if isinstance(item, dict)], + source_stream=source_stream if isinstance(source_stream, dict) else {}, + ) + elif section == "archive": + section_data["archive_jobs"] = _backend(app).list_jobs(_token(), job_type="archive") + elif section == "catalog": + section_data["catalog_status"] = _backend(app).get_catalog_status(_token()) + section_data["catalog"] = _backend(app).list_catalog_files(_token(), limit=50) + section_data["datasets"] = _backend(app).list_nas_datasets(_token(), limit=100) + verify_snapshot = session.get("nas_verify_snapshot") + section_data["verify_snapshot"] = ( + verify_snapshot if isinstance(verify_snapshot, dict) else None + ) + elif section == "restore": + section_data["restore_jobs"] = _backend(app).list_jobs(_token(), job_type="restore") + except BackendError as error: + return _handle_backend_error(error) + + return Response( + render_template( + "storage/index.html", + section=section, + devices=devices, + active_device=active_device, + section_data=section_data, + page_id="nas-storage", + ), + status=200, + ) + + @app.post("/storage/archive") + def storage_archive_submit() -> ResponseReturnValue: + source_path = request.form.get("source_path", "").strip() + volume_group = request.form.get("volume_group", "").strip() + if not source_path or not volume_group: + flash("Source path and volume group are required.", "error") + return redirect(url_for("storage", section="archive")) + if not _is_safe_storage_path(source_path): + flash("Source path must be an absolute normalized path within policy limits.", "error") + return redirect(url_for("storage", section="archive")) + try: + _backend(app).create_archive_job( + _token(), source_path=source_path, volume_group=volume_group + ) + except BackendError as error: + return _handle_backend_error(error) + flash("Archive job submitted.", "success") + return redirect(url_for("jobs")) + + @app.post("/storage/restore") + def storage_restore_submit() -> ResponseReturnValue: + catalog_path = request.form.get("catalog_path", "").strip() + dest_path = request.form.get("dest_path", "").strip() + if not catalog_path or not dest_path: + flash("Catalog path and destination path are required.", "error") + return redirect(url_for("storage", section="restore")) + if not _is_safe_storage_path(catalog_path) or not _is_safe_storage_path(dest_path): + flash( + "Catalog and destination paths must be absolute normalized paths within policy limits.", + "error", + ) + return redirect(url_for("storage", section="restore")) + try: + _backend(app).create_restore_job( + _token(), catalog_path=catalog_path, dest_path=dest_path + ) + except BackendError as error: + return _handle_backend_error(error) + flash("Restore job submitted.", "success") + return redirect(url_for("jobs")) + + @app.post("/storage/datasets//verify") + def storage_dataset_verify(dataset_id: str) -> ResponseReturnValue: + normalized_dataset_id = dataset_id.strip() + if not _POOL_ID_ALLOWED.fullmatch(normalized_dataset_id): + flash("Dataset ID is invalid.", "error") + return redirect(url_for("storage", section="catalog")) + try: + result = _backend(app).verify_nas_dataset(_token(), dataset_id=normalized_dataset_id) + except BackendError as error: + flash(error.detail, "error") + return redirect(url_for("storage", section="catalog")) + files_verified = int(result.get("files_verified", 0)) + files_corrupt = int(result.get("files_corrupt", 0)) + files_updated = int(result.get("files_updated", 0)) + session["nas_verify_snapshot"] = { + "dataset_id": normalized_dataset_id, + "files_verified": files_verified, + "files_corrupt": files_corrupt, + "files_updated": files_updated, + "updated_at": int(time.time()), + } + if files_corrupt > 0: + flash( + f"Verify completed for {normalized_dataset_id}: {files_verified} verified, {files_corrupt} corrupt.", + "warning", + ) + else: + flash( + f"Verify completed for {normalized_dataset_id}: {files_verified} verified, {files_updated} checksums updated.", + "success", + ) + return redirect(url_for("storage", section="catalog")) + + @app.post("/storage/policies") + def storage_policy_submit() -> ResponseReturnValue: + policy_id = request.form.get("policy_id", "").strip() + name = request.form.get("name", "").strip() + policy_type = request.form.get("policy_type", "balanced").strip().lower() + ingest_mode = request.form.get("default_ingest_mode", "cache_drive").strip().lower() + shard_strategy = request.form.get("shard_strategy", "").strip().lower() or None + shard_size_bytes_raw = request.form.get("shard_size_bytes", "").strip() + copies_required_raw = request.form.get("copies_required", "1").strip() + max_parallelism_raw = request.form.get("max_parallelism", "1").strip() + allow_sharding = request.form.get("allow_sharding", "").strip().lower() in { + "1", + "true", + "yes", + "on", + } + auto_clean_before_archive = request.form.get( + "auto_clean_before_archive", "" + ).strip().lower() in { + "1", + "true", + "yes", + "on", + } + verify_before_archive = request.form.get("verify_before_archive", "").strip().lower() in { + "1", + "true", + "yes", + "on", + } + verify_after_archive = request.form.get("verify_after_archive", "").strip().lower() in { + "1", + "true", + "yes", + "on", + } + if not _POOL_ID_ALLOWED.fullmatch(policy_id): + flash("Policy ID accepts letters, numbers, dot, underscore, dash, and colon.", "error") + return redirect(url_for("storage", section="write-path")) + if not _ASCII_PRINTABLE.fullmatch(name): + flash("Policy name must be 1-64 printable ASCII characters.", "error") + return redirect(url_for("storage", section="write-path")) + if policy_type not in _POLICY_TYPE_OPTIONS: + flash("Policy type is invalid.", "error") + return redirect(url_for("storage", section="write-path")) + if ingest_mode not in _INGEST_MODE_OPTIONS: + flash("Default ingest mode must be cache_drive or source_stream.", "error") + return redirect(url_for("storage", section="write-path")) + if shard_strategy is not None and shard_strategy not in _SHARD_STRATEGY_OPTIONS: + flash("Shard strategy is invalid.", "error") + return redirect(url_for("storage", section="write-path")) + try: + copies_required = int(copies_required_raw) + max_parallelism = int(max_parallelism_raw) + except ValueError: + flash("Copies required and max parallelism must be numbers.", "error") + return redirect(url_for("storage", section="write-path")) + shard_size_bytes: int | None = None + if shard_size_bytes_raw: + try: + shard_size_bytes = int(shard_size_bytes_raw) + except ValueError: + flash("Shard size bytes must be a whole number when provided.", "error") + return redirect(url_for("storage", section="write-path")) + if shard_size_bytes <= 0: + flash("Shard size bytes must be greater than zero.", "error") + return redirect(url_for("storage", section="write-path")) + if copies_required < 1 or copies_required > 4: + flash("Copies required must be between 1 and 4.", "error") + return redirect(url_for("storage", section="write-path")) + if max_parallelism < 1 or max_parallelism > 16: + flash("Max parallelism must be between 1 and 16.", "error") + return redirect(url_for("storage", section="write-path")) + payload: dict[str, object] = { + "id": policy_id, + "name": name, + "policy_type": policy_type, + "default_ingest_mode": ingest_mode, + "copies_required": copies_required, + "verify_before_archive": verify_before_archive, + "verify_after_archive": verify_after_archive, + "allow_spillover": True, + "allow_sharding": allow_sharding, + "shard_size_bytes": shard_size_bytes, + "max_parallelism": max_parallelism, + "shard_strategy": shard_strategy, + "auto_clean_before_archive": auto_clean_before_archive, + "manifest_strategy": "per_tape", + "cache_retention": "after_verified", + "allow_source_delete": False, + } + try: + _backend(app).create_or_update_nas_policy(_token(), payload) + except BackendError as error: + flash(error.detail, "error") + return redirect(url_for("storage", section="write-path")) + flash("Write policy saved.", "success") + return redirect(url_for("storage", section="write-path")) + + @app.post("/storage/cache-drives") + def storage_cache_drive_submit() -> ResponseReturnValue: + drive_id = request.form.get("drive_id", "").strip() + name = request.form.get("name", "").strip() + root_path = request.form.get("root_path", "").strip() + max_bytes_raw = request.form.get("max_bytes", "0").strip() + min_free_bytes_raw = request.form.get("min_free_bytes", "0").strip() + retention_days_raw = request.form.get("retention_days", "30").strip() + stabilization_seconds_raw = request.form.get("stabilization_seconds", "5").strip() + eviction_policy = request.form.get("eviction_policy", "after_verified").strip().lower() + enabled = request.form.get("enabled", "").strip().lower() in {"1", "true", "yes", "on"} + verify_before_archive = request.form.get("verify_before_archive", "").strip().lower() in { + "1", + "true", + "yes", + "on", + } + verify_after_archive = request.form.get("verify_after_archive", "").strip().lower() in { + "1", + "true", + "yes", + "on", + } + if not _POOL_ID_ALLOWED.fullmatch(drive_id): + flash( + "Cache drive ID accepts letters, numbers, dot, underscore, dash, and colon.", + "error", + ) + return redirect(url_for("storage", section="write-path")) + if not _ASCII_PRINTABLE.fullmatch(name): + flash("Cache drive name must be 1-64 printable ASCII characters.", "error") + return redirect(url_for("storage", section="write-path")) + if not _is_safe_storage_path(root_path): + flash("Cache drive root path must be an absolute normalized path.", "error") + return redirect(url_for("storage", section="write-path")) + if eviction_policy not in _CACHE_EVICTION_OPTIONS: + flash("Cache eviction policy is invalid.", "error") + return redirect(url_for("storage", section="write-path")) + try: + max_bytes = int(max_bytes_raw) + min_free_bytes = int(min_free_bytes_raw) + retention_days = int(retention_days_raw) + stabilization_seconds = int(stabilization_seconds_raw) + except ValueError: + flash("Cache sizing and timing fields must be numeric.", "error") + return redirect(url_for("storage", section="write-path")) + if max_bytes <= 0: + flash("Cache max bytes must be greater than zero.", "error") + return redirect(url_for("storage", section="write-path")) + if min_free_bytes < 0 or retention_days < 0 or stabilization_seconds < 0: + flash( + "Cache minimum free bytes, retention days, and stabilization seconds must be non-negative.", + "error", + ) + return redirect(url_for("storage", section="write-path")) + payload: dict[str, object] = { + "id": drive_id, + "name": name, + "root_path": root_path, + "max_bytes": max_bytes, + "min_free_bytes": min_free_bytes, + "eviction_policy": eviction_policy, + "retention_days": retention_days, + "verify_before_archive": verify_before_archive, + "verify_after_archive": verify_after_archive, + "allow_source_delete_after_verify": False, + "stabilization_seconds": stabilization_seconds, + "support_reflink_or_hardlink": False, + "quarantine_failed_files": True, + "quarantine_path": f"{root_path.rstrip('/')}/quarantine", + "enabled": enabled, + } + try: + _backend(app).create_or_update_cache_drive(_token(), payload) + except BackendError as error: + flash(error.detail, "error") + return redirect(url_for("storage", section="write-path")) + flash("Cache drive saved.", "success") + return redirect(url_for("storage", section="write-path")) + + @app.post("/storage/source-stream") + def storage_source_stream_submit() -> ResponseReturnValue: + checksum_mode = ( + request.form.get("checksum_mode", "precompute_and_post_verify").strip().lower() + ) + max_retries_raw = request.form.get("max_retries", "3").strip() + enabled = request.form.get("enabled", "").strip().lower() in {"1", "true", "yes", "on"} + require_source_online = request.form.get( + "require_source_online_for_entire_job", "" + ).strip().lower() in { + "1", + "true", + "yes", + "on", + } + preflight_read_check = request.form.get("preflight_read_check", "").strip().lower() in { + "1", + "true", + "yes", + "on", + } + fail_on_source_change = request.form.get("fail_on_source_change", "").strip().lower() in { + "1", + "true", + "yes", + "on", + } + allow_partial = request.form.get("allow_partial_dataset_success", "").strip().lower() in { + "1", + "true", + "yes", + "on", + } + if checksum_mode not in _SOURCE_CHECKSUM_MODES: + flash("Source-stream checksum mode is invalid.", "error") + return redirect(url_for("storage", section="write-path")) + try: + max_retries = int(max_retries_raw) + except ValueError: + flash("Source-stream max retries must be numeric.", "error") + return redirect(url_for("storage", section="write-path")) + if max_retries < 0 or max_retries > 10: + flash("Source-stream max retries must be between 0 and 10.", "error") + return redirect(url_for("storage", section="write-path")) + payload = { + "enabled": enabled, + "require_source_online_for_entire_job": require_source_online, + "preflight_read_check": preflight_read_check, + "checksum_mode": checksum_mode, + "retry_policy": "linear", + "max_retries": max_retries, + "fail_on_source_change": fail_on_source_change, + "snapshot_required": False, + "source_change_detection": "size_mtime_checksum", + "allow_partial_dataset_success": allow_partial, + } + try: + _backend(app).update_source_stream_config(_token(), payload) + except BackendError as error: + flash(error.detail, "error") + return redirect(url_for("storage", section="write-path")) + flash("Source-stream configuration saved.", "success") + return redirect(url_for("storage", section="write-path")) + + @app.post("/storage/shares") + def storage_share_submit() -> ResponseReturnValue: + share_path = request.form.get("share_path", "").strip() + share_name = request.form.get("share_name", "").strip() + share_type = request.form.get("share_type", "pool").strip().lower() + default_policy_id = request.form.get("default_policy_id", "").strip() + pool_ids_raw = request.form.get("pool_ids", "").strip() + folder_mappings_raw = request.form.get("folder_mappings", "").strip() + writable = request.form.get("writable", "").strip().lower() in {"1", "true", "yes", "on"} + + if not _is_safe_storage_path(share_path): + flash("Share path must be an absolute normalized path.", "error") + return redirect(url_for("storage", section="write-path")) + if not _SHARE_NAME_ALLOWED.fullmatch(share_name): + flash("Share name accepts letters, numbers, dot, underscore, and dash only.", "error") + return redirect(url_for("storage", section="write-path")) + if share_type not in _SHARE_TYPE_OPTIONS: + flash("Share type is invalid.", "error") + return redirect(url_for("storage", section="write-path")) + if default_policy_id and not _POOL_ID_ALLOWED.fullmatch(default_policy_id): + flash("Default policy ID is invalid.", "error") + return redirect(url_for("storage", section="write-path")) + + pool_ids = _parse_csv_tokens(pool_ids_raw) + for pool_id in pool_ids: + if not _POOL_ID_ALLOWED.fullmatch(pool_id): + flash( + "Pool IDs must contain only letters, numbers, dot, underscore, dash, and colon.", + "error", + ) + return redirect(url_for("storage", section="write-path")) + + folder_mappings: list[dict[str, str]] = [] + for line_number, line in enumerate(folder_mappings_raw.splitlines(), start=1): + rule = line.strip() + if not rule: + continue + parts = [part.strip() for part in rule.split("|")] + if len(parts) != 3: + flash( + f"Folder mapping line {line_number} must use 'folder_path|pool_id|access_mode'.", + "error", + ) + return redirect(url_for("storage", section="write-path")) + folder_path, pool_id, access_mode = parts + if not _is_safe_storage_path(folder_path): + flash(f"Folder mapping line {line_number} has an invalid folder path.", "error") + return redirect(url_for("storage", section="write-path")) + if not _POOL_ID_ALLOWED.fullmatch(pool_id): + flash(f"Folder mapping line {line_number} has an invalid pool ID.", "error") + return redirect(url_for("storage", section="write-path")) + if access_mode not in _SHARE_ACCESS_OPTIONS: + flash( + f"Folder mapping line {line_number} access mode must be read_only or read_write.", + "error", + ) + return redirect(url_for("storage", section="write-path")) + folder_mappings.append( + { + "folder_path": folder_path, + "pool_id": pool_id, + "access_mode": access_mode, + } + ) + + if not pool_ids and folder_mappings: + pool_ids = list(dict.fromkeys(item["pool_id"] for item in folder_mappings)) + known_pool_ids = set(pool_ids) + for mapping in folder_mappings: + if mapping["pool_id"] not in known_pool_ids: + flash("Every folder mapping pool must be listed in pool_ids.", "error") + return redirect(url_for("storage", section="write-path")) + + payload: dict[str, object] = { + "path": share_path, + "name": share_name, + "share_type": share_type, + "default_policy_id": default_policy_id or None, + "pool_ids": pool_ids, + "folder_mappings": folder_mappings, + "writable": writable, + "description": f"Share mapping for {share_name}", + } + try: + _backend(app).create_nas_share(_token(), payload) + except BackendError as error: + flash(error.detail, "error") + return redirect(url_for("storage", section="write-path")) + flash("Share mapping saved.", "success") + return redirect(url_for("storage", section="write-path")) + + @app.get("/jobs") + def jobs() -> ResponseReturnValue: + selected_id = _parse_int(request.args.get("library_id")) + state = request.args.get("state", "").strip().lower() or None + job_type = request.args.get("job_type", "").strip().lower() or None + if state and state not in _JOB_STATES: + state = None + if job_type and job_type not in _JOB_TYPES: + job_type = None + try: + devices = _backend(app).list_devices(_token()) + jobs_list = _backend(app).list_jobs( + _token(), library_id=selected_id, state=state, job_type=job_type + ) + except BackendError as error: + return _handle_backend_error(error) + has_active = any( + str(item.get("state", "")).lower() in {"pending", "running"} for item in jobs_list + ) + jobs_summary = { + "total": len(jobs_list), + "running": sum( + 1 for job in jobs_list if str(job.get("state", "")).lower() == "running" + ), + "pending": sum( + 1 for job in jobs_list if str(job.get("state", "")).lower() == "pending" + ), + "failed": sum( + 1 for job in jobs_list if str(job.get("state", "")).lower().startswith("failed") + ), + } + return Response( + render_template( + "jobs/index.html", + jobs=jobs_list, + jobs_summary=jobs_summary, + devices=devices, + selected_library_id=selected_id, + selected_state=state or "all", + selected_job_type=job_type or "all", + has_active=has_active, + page_id="nas-jobs", + ), + status=200, + ) + + @app.get("/reports") + def reports() -> ResponseReturnValue: + selected_id = _parse_int(request.args.get("library_id")) + severity = request.args.get("severity", "all").strip().lower() or "all" + if severity not in {"all", "critical", "warning", "info"}: + severity = "all" + try: + devices = _backend(app).list_devices(_token()) + active_device = _pick_device(devices, selected_id) + events: list[dict[str, object]] = [] + event_summary: dict[str, object] = {} + alert_summary: dict[str, object] = {} + if active_device is not None: + event_summary = _backend(app).get_event_summary( + _token(), library_id=active_device.id + ) + alert_summary = _backend(app).get_alert_summary( + _token(), library_id=active_device.id + ) + events = _backend(app).list_events( + _token(), + library_id=active_device.id, + severity=severity, + limit=50, + ) + except BackendError as error: + return _handle_backend_error(error) + return Response( + render_template( + "reports/index.html", + devices=devices, + active_device=active_device, + events=events, + event_summary=event_summary, + alert_summary=alert_summary, + severity=severity, + page_id="nas-reports", + ), + status=200, + ) + + @app.get("/system") + def system() -> ResponseReturnValue: + try: + health = _backend(app).get_health(_token()) + config = _backend(app).get_system_config(_token()) + catalog_status = _backend(app).get_catalog_status(_token()) + system_status = _backend(app).get_system_status(_token()) + users = _backend(app).list_aml_users(_token()) + except BackendError as error: + return _handle_backend_error(error) + return Response( + render_template( + "system/index.html", + health=health, + config=config, + catalog_status=catalog_status, + system_status=system_status, + users=users, + page_id="nas-system", + ), + status=200, + ) + + @app.post("/system/users") + def system_create_user() -> ResponseReturnValue: + username = request.form.get("username", "").strip() + password = request.form.get("password", "") + role_raw = request.form.get("role", "").strip() + if not _DEVICE_USERNAME_ALLOWED.fullmatch(username): + flash("User name accepts letters, numbers, dot, underscore, and dash.", "error") + return redirect(url_for("system")) + if len(password) < 8 or len(password) > 64: + flash("Password must be between 8 and 64 characters.", "error") + return redirect(url_for("system")) + try: + role = int(role_raw) + except ValueError: + flash("User role is invalid.", "error") + return redirect(url_for("system")) + if role not in {0, 1, 2}: + flash("User role is invalid.", "error") + return redirect(url_for("system")) + try: + _backend(app).create_aml_user( + _token(), + { + "name": username, + "password": password, + "role": role, + }, + ) + except BackendError as error: + flash(error.detail, "error") + return redirect(url_for("system")) + flash(f"User {username} created.", "success") + return redirect(url_for("system")) + + @app.get("/events/devices") + def devices_events() -> ResponseReturnValue: + token = _token() + once = request.args.get("once", "").lower() in {"1", "true", "yes"} + + def _stream() -> Iterator[str]: + last_payload = "" + while True: + try: + devices = _backend(app).list_devices(token) + payload = json.dumps( + { + "summary": _summary(devices), + "devices": [asdict(item) for item in devices], + "updated_at": int(time.time()), + } + ) + if payload != last_payload: + yield f"event: devices.snapshot\ndata: {payload}\n\n" + last_payload = payload + else: + yield ": keepalive\n\n" + except BackendError as error: + yield f"event: devices.error\ndata: {json.dumps({'detail': error.detail})}\n\n" + if once: + break + time.sleep(10) + + headers = { + "Cache-Control": "no-cache", + "Connection": "keep-alive", + "X-Accel-Buffering": "no", + } + return Response( + stream_with_context(_stream()), mimetype="text/event-stream", headers=headers + ) + + @app.get("/events/jobs") + def jobs_events() -> ResponseReturnValue: + token = _token() + once = request.args.get("once", "").lower() in {"1", "true", "yes"} + + def _stream() -> Iterator[str]: + last_payload = "" + while True: + try: + jobs_list = _backend(app).list_jobs(token) + running = sum( + 1 for job in jobs_list if str(job.get("state", "")).lower() == "running" + ) + pending = sum( + 1 for job in jobs_list if str(job.get("state", "")).lower() == "pending" + ) + failed = sum( + 1 + for job in jobs_list + if str(job.get("state", "")).lower().startswith("failed") + ) + payload = json.dumps( + { + "total": len(jobs_list), + "running": running, + "pending": pending, + "failed": failed, + "updated_at": int(time.time()), + } + ) + if payload != last_payload: + yield f"event: jobs.snapshot\ndata: {payload}\n\n" + last_payload = payload + else: + yield ": keepalive\n\n" + except BackendError as error: + yield f"event: jobs.error\ndata: {json.dumps({'detail': error.detail})}\n\n" + if once: + break + time.sleep(8) + + headers = { + "Cache-Control": "no-cache", + "Connection": "keep-alive", + "X-Accel-Buffering": "no", + } + return Response( + stream_with_context(_stream()), mimetype="text/event-stream", headers=headers + ) + + @app.post("/devices/probe") + def devices_probe() -> ResponseReturnValue: + connection_url = request.form.get("connection_url", "").strip() + device_username = request.form.get("device_username", "").strip() + device_password = request.form.get("device_password", "") + connection_error = _validate_device_url(connection_url) + if connection_error: + response = jsonify({"ok": False, "detail": connection_error}) + response.status_code = 400 + return response + if device_username and not _DEVICE_USERNAME_ALLOWED.fullmatch(device_username): + response = jsonify({"ok": False, "detail": "Invalid device username format"}) + response.status_code = 400 + return response + if device_password and not device_username: + response = jsonify( + {"ok": False, "detail": "Username is required when password is provided"} + ) + response.status_code = 400 + return response + try: + probe = _backend(app).probe_device_endpoint( + connection_url=connection_url, + username=device_username or None, + password=device_password or None, + ) + except BackendError as error: + response = jsonify({"ok": False, "detail": error.detail}) + response.status_code = 400 + return response + response = jsonify( + {"ok": True, "detail": "Connection and auth probe passed", "probe": probe} + ) + response.status_code = 200 + return response + + return app + + +app = create_app() diff --git a/openblade/web_flask/client.py b/openblade/web_flask/client.py new file mode 100644 index 0000000..9fd814a --- /dev/null +++ b/openblade/web_flask/client.py @@ -0,0 +1,898 @@ +from __future__ import annotations + +from dataclasses import asdict +from http import HTTPStatus +from typing import Any +from urllib.parse import urlparse + +import httpx + +from .models import Device, parse_device + + +class BackendError(Exception): + def __init__(self, status_code: int, detail: str) -> None: + super().__init__(detail) + self.status_code = status_code + self.detail = detail + + +class BackendClient: + def __init__( + self, + *, + base_url: str, + timeout_seconds: float = 8.0, + service_token: str | None = None, + api_token: str | None = None, + ) -> None: + self.base_url = base_url.rstrip("/") + self.timeout_seconds = timeout_seconds + self.service_token = service_token + self.api_token = api_token + + def _headers( + self, token: str | None = None, *, library_id: int | None = None + ) -> dict[str, str]: + """Build headers carrying BOTH credentials the backend may demand. + + The backend authenticates its two surfaces independently + (openblade.api.api_auth): the OpenBlade-native surface (/api, /jobs, + /nas, /catalog, /status, /system, /volume-groups, /archive, /restore) is + gated by the static ``OPENBLADE_API_TOKEN`` bearer, while /aml/* keeps its + own per-user session from ``openblade.api.routes_aml_auth.require_auth``. + ``token`` here is the AML *session* token minted by our /login -- it is NOT + the native API token, so sending it as the only bearer made every native + call 401 as soon as an operator enabled OPENBLADE_API_TOKEN (21 of the 46 + endpoints this client uses, i.e. all of Storage/Jobs/Reports/System). + + So: the session token travels as the ``sessionID`` cookie, which + ``require_auth`` checks FIRST, leaving ``Authorization`` free for the + native token. When no native token is configured we keep putting the + session token in ``Authorization`` as before, so behaviour is unchanged + for deployments that leave native auth disabled. + """ + headers = {"Accept": "application/json"} + bearer = self.api_token or token + if bearer: + headers["Authorization"] = f"Bearer {bearer}" + if token: + headers["Cookie"] = f"sessionID={token}" + if self.service_token: + headers["X-Openblade-Service-Token"] = self.service_token + if library_id is not None: + headers["X-OpenBlade-Library-Id"] = str(library_id) + return headers + + def _raise_for_error(self, response: httpx.Response) -> None: + if response.status_code < HTTPStatus.BAD_REQUEST: + return + detail = "Backend request failed" + try: + payload = response.json() + if isinstance(payload, dict): + detail = str(payload.get("detail") or payload.get("message") or detail) + except ValueError: + detail = response.text or detail + raise BackendError(response.status_code, detail) + + def login(self, *, username: str, password: str) -> str: + payload = {"name": username, "password": password} + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.post( + f"{self.base_url}/aml/users/login", + json=payload, + headers=self._headers(), + ) + self._raise_for_error(response) + body = response.json() + token = body.get("token") + if not isinstance(token, str) or not token: + raise BackendError( + HTTPStatus.BAD_GATEWAY, "Backend login response did not include a token" + ) + return token + + def list_devices(self, token: str) -> list[Device]: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.get( + f"{self.base_url}/api/libraries", + headers=self._headers(token), + ) + self._raise_for_error(response) + body = response.json() + if not isinstance(body, list): + return [] + return [parse_device(item) for item in body if isinstance(item, dict)] + + def get_device(self, token: str, device_id: int) -> Device | None: + for item in self.list_devices(token): + if item.id == device_id: + return item + return None + + def create_device(self, token: str, payload: dict[str, Any]) -> Device: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.post( + f"{self.base_url}/api/libraries", + json=payload, + headers=self._headers(token), + ) + self._raise_for_error(response) + body = response.json() + if not isinstance(body, dict): + raise BackendError(HTTPStatus.BAD_GATEWAY, "Backend create-device response was invalid") + return parse_device(body) + + def update_device(self, token: str, device_id: int, payload: dict[str, Any]) -> Device: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.put( + f"{self.base_url}/api/libraries/{device_id}", + json=payload, + headers=self._headers(token), + ) + self._raise_for_error(response) + body = response.json() + if not isinstance(body, dict): + raise BackendError(HTTPStatus.BAD_GATEWAY, "Backend update-device response was invalid") + return parse_device(body) + + def list_jobs( + self, + token: str, + *, + library_id: int | None = None, + state: str | None = None, + job_type: str | None = None, + ) -> list[dict[str, Any]]: + params: dict[str, Any] = {} + if library_id is not None: + params["library_id"] = library_id + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.get( + f"{self.base_url}/jobs/", + params=params, + headers=self._headers(token), + ) + self._raise_for_error(response) + body = response.json() + if not isinstance(body, list): + return [] + jobs = [item for item in body if isinstance(item, dict)] + if state: + jobs = [item for item in jobs if str(item.get("state", "")).lower() == state.lower()] + if job_type: + jobs = [ + item for item in jobs if str(item.get("job_type", "")).lower() == job_type.lower() + ] + return jobs + + def list_media(self, token: str, *, library_id: int, limit: int = 100) -> list[dict[str, Any]]: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.get( + f"{self.base_url}/aml/media", + headers=self._headers(token, library_id=library_id), + ) + self._raise_for_error(response) + body = response.json() + items = body.get("mediaList", {}).get("media", []) if isinstance(body, dict) else [] + if not isinstance(items, list): + return [] + return [item for item in items[:limit] if isinstance(item, dict)] + + def list_drives(self, token: str, *, library_id: int) -> list[dict[str, Any]]: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.get( + f"{self.base_url}/aml/drives", + headers=self._headers(token, library_id=library_id), + ) + self._raise_for_error(response) + body = response.json() + items = body.get("driveList", {}).get("drive", []) if isinstance(body, dict) else [] + if not isinstance(items, list): + return [] + return [item for item in items if isinstance(item, dict)] + + def list_partitions(self, token: str, *, library_id: int) -> list[dict[str, Any]]: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.get( + f"{self.base_url}/aml/partitions", + headers=self._headers(token, library_id=library_id), + ) + self._raise_for_error(response) + body = response.json() + items = body.get("partitionList", {}).get("partition", []) if isinstance(body, dict) else [] + if not isinstance(items, list): + return [] + return [item for item in items if isinstance(item, dict)] + + def trigger_inventory(self, token: str, *, library_id: int) -> dict[str, Any]: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.post( + f"{self.base_url}/aml/inventory", + headers=self._headers(token, library_id=library_id), + ) + self._raise_for_error(response) + body = response.json() + if isinstance(body, dict): + return body + return {"message": "Inventory started"} + + def move_media_operation( + self, + token: str, + *, + library_id: int, + source: str, + destination: str, + barcode: str | None = None, + ) -> dict[str, Any]: + move_payload: dict[str, Any] = {"source": source, "destination": destination} + if barcode: + move_payload["barcode"] = barcode + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.post( + f"{self.base_url}/aml/operations/move", + json={"move": move_payload}, + headers=self._headers(token, library_id=library_id), + ) + self._raise_for_error(response) + body = response.json() + if isinstance(body, dict): + return body + return {} + + def mount_media_operation( + self, + token: str, + *, + library_id: int, + barcode: str, + drive: str, + partition: str | None = None, + ) -> dict[str, Any]: + mount_payload: dict[str, Any] = {"barcode": barcode, "drive": drive} + if partition: + mount_payload["partition"] = partition + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.post( + f"{self.base_url}/aml/mount", + json={"mount": mount_payload}, + headers=self._headers(token, library_id=library_id), + ) + self._raise_for_error(response) + body = response.json() + if isinstance(body, dict): + return body + return {} + + def unmount_media_operation( + self, + token: str, + *, + library_id: int, + barcode: str, + drive: str, + ) -> dict[str, Any]: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.post( + f"{self.base_url}/aml/unmount", + json={"unmount": {"barcode": barcode, "drive": drive}}, + headers=self._headers(token, library_id=library_id), + ) + self._raise_for_error(response) + body = response.json() + if isinstance(body, dict): + return body + return {} + + def start_import_operation( + self, + token: str, + *, + library_id: int, + partition: str, + ie_station: str, + ) -> dict[str, Any]: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.post( + f"{self.base_url}/aml/import", + json={"import": {"partition": partition, "ieStation": ie_station}}, + headers=self._headers(token, library_id=library_id), + ) + self._raise_for_error(response) + body = response.json() + if isinstance(body, dict): + return body + return {} + + def start_export_operation( + self, + token: str, + *, + library_id: int, + barcodes: list[str], + ie_station: str, + ) -> dict[str, Any]: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.post( + f"{self.base_url}/aml/export", + json={"export": {"barcodes": barcodes, "ieStation": ie_station}}, + headers=self._headers(token, library_id=library_id), + ) + self._raise_for_error(response) + body = response.json() + if isinstance(body, dict): + return body + return {} + + def list_mounts(self, token: str, *, library_id: int) -> list[dict[str, Any]]: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.get( + f"{self.base_url}/aml/mounts", + headers=self._headers(token, library_id=library_id), + ) + self._raise_for_error(response) + body = response.json() + items = body.get("mountList", {}).get("mount", []) if isinstance(body, dict) else [] + if not isinstance(items, list): + return [] + return [item for item in items if isinstance(item, dict)] + + def list_ie_stations(self, token: str, *, library_id: int) -> list[dict[str, Any]]: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.get( + f"{self.base_url}/aml/ieStations", + headers=self._headers(token, library_id=library_id), + ) + self._raise_for_error(response) + body = response.json() + items = body.get("ieStationList", {}).get("ieStation", []) if isinstance(body, dict) else [] + if not isinstance(items, list): + return [] + return [item for item in items if isinstance(item, dict)] + + def get_inventory_status(self, token: str, *, library_id: int) -> dict[str, Any]: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.get( + f"{self.base_url}/aml/operations/inventory/status", + headers=self._headers(token, library_id=library_id), + ) + self._raise_for_error(response) + body = response.json() + if not isinstance(body, dict): + return {} + status = body.get("inventoryStatus") + if isinstance(status, dict): + return status + return body + + def get_import_status(self, token: str, *, library_id: int) -> dict[str, Any]: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.get( + f"{self.base_url}/aml/operations/import/status", + headers=self._headers(token, library_id=library_id), + ) + self._raise_for_error(response) + body = response.json() + if not isinstance(body, dict): + return {} + status = body.get("importStatus") + if isinstance(status, dict): + return status + return body + + def get_export_status(self, token: str, *, library_id: int) -> dict[str, Any]: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.get( + f"{self.base_url}/aml/operations/export/status", + headers=self._headers(token, library_id=library_id), + ) + self._raise_for_error(response) + body = response.json() + if not isinstance(body, dict): + return {} + status = body.get("exportStatus") + if isinstance(status, dict): + return status + return body + + def list_media_pools(self, token: str, *, library_id: int) -> list[dict[str, Any]]: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.get( + f"{self.base_url}/aml/media/pools", + headers=self._headers(token, library_id=library_id), + ) + self._raise_for_error(response) + body = response.json() + items = body.get("poolList", {}).get("pool", []) if isinstance(body, dict) else [] + if not isinstance(items, list): + return [] + return [item for item in items if isinstance(item, dict)] + + def list_nas_shares(self, token: str) -> list[dict[str, Any]]: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.get( + f"{self.base_url}/nas/shares", + headers=self._headers(token), + ) + self._raise_for_error(response) + body = response.json() + if not isinstance(body, list): + return [] + return [item for item in body if isinstance(item, dict)] + + def list_nas_pools(self, token: str) -> list[dict[str, Any]]: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.get( + f"{self.base_url}/nas/pools", + headers=self._headers(token), + ) + self._raise_for_error(response) + body = response.json() + if not isinstance(body, list): + return [] + return [item for item in body if isinstance(item, dict)] + + def list_aml_users(self, token: str) -> list[dict[str, Any]]: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.get( + f"{self.base_url}/aml/users", + headers=self._headers(token), + ) + self._raise_for_error(response) + body = response.json() + items = body.get("user", []) if isinstance(body, dict) else [] + if not isinstance(items, list): + return [] + return [item for item in items if isinstance(item, dict)] + + def create_aml_user(self, token: str, payload: dict[str, Any]) -> dict[str, Any]: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.post( + f"{self.base_url}/aml/users", + json=payload, + headers=self._headers(token), + ) + self._raise_for_error(response) + body = response.json() + if isinstance(body, dict): + return body + return {} + + def list_nas_policies(self, token: str) -> list[dict[str, Any]]: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.get( + f"{self.base_url}/nas/policies", + headers=self._headers(token), + ) + self._raise_for_error(response) + body = response.json() + if not isinstance(body, list): + return [] + return [item for item in body if isinstance(item, dict)] + + def create_or_update_nas_policy(self, token: str, payload: dict[str, Any]) -> dict[str, Any]: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.post( + f"{self.base_url}/nas/policies", + json=payload, + headers=self._headers(token), + ) + self._raise_for_error(response) + body = response.json() + if isinstance(body, dict): + return body + return {} + + def list_cache_drives(self, token: str) -> list[dict[str, Any]]: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.get( + f"{self.base_url}/nas/cache-drives", + headers=self._headers(token), + ) + self._raise_for_error(response) + body = response.json() + if not isinstance(body, list): + return [] + return [item for item in body if isinstance(item, dict)] + + def create_or_update_cache_drive(self, token: str, payload: dict[str, Any]) -> dict[str, Any]: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.post( + f"{self.base_url}/nas/cache-drives", + json=payload, + headers=self._headers(token), + ) + self._raise_for_error(response) + body = response.json() + if isinstance(body, dict): + return body + return {} + + def get_source_stream_config(self, token: str) -> dict[str, Any]: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.get( + f"{self.base_url}/nas/source-stream", + headers=self._headers(token), + ) + self._raise_for_error(response) + body = response.json() + if isinstance(body, dict): + return body + return {} + + def update_source_stream_config(self, token: str, payload: dict[str, Any]) -> dict[str, Any]: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.put( + f"{self.base_url}/nas/source-stream", + json=payload, + headers=self._headers(token), + ) + self._raise_for_error(response) + body = response.json() + if isinstance(body, dict): + return body + return {} + + def get_gateway_config(self, token: str) -> dict[str, Any]: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.get( + f"{self.base_url}/api/gateway/config", + headers=self._headers(token), + ) + self._raise_for_error(response) + body = response.json() + if isinstance(body, dict): + return body + return {} + + def get_gateway_status(self, token: str) -> dict[str, Any]: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.get( + f"{self.base_url}/api/gateway/status", + headers=self._headers(token), + ) + self._raise_for_error(response) + body = response.json() + if isinstance(body, dict): + return body + return {} + + def list_gateway_credentials(self, token: str) -> list[dict[str, Any]]: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.get( + f"{self.base_url}/api/gateway/credentials", + headers=self._headers(token), + ) + self._raise_for_error(response) + body = response.json() + if not isinstance(body, list): + return [] + return [item for item in body if isinstance(item, dict)] + + def get_event_summary(self, token: str, *, library_id: int) -> dict[str, Any]: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.get( + f"{self.base_url}/aml/events/summary", + headers=self._headers(token, library_id=library_id), + ) + self._raise_for_error(response) + body = response.json() + if not isinstance(body, dict): + return {} + summary = body.get("eventSummary") + if isinstance(summary, dict): + return summary + return body + + def get_alert_summary(self, token: str, *, library_id: int) -> dict[str, Any]: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.get( + f"{self.base_url}/aml/alerts/summary", + headers=self._headers(token, library_id=library_id), + ) + self._raise_for_error(response) + body = response.json() + if not isinstance(body, dict): + return {} + summary = body.get("alertSummary") + if isinstance(summary, dict): + return summary + return body + + def list_events( + self, + token: str, + *, + library_id: int, + severity: str | None = None, + limit: int = 50, + ) -> list[dict[str, Any]]: + params: dict[str, Any] = {"limit": max(1, min(limit, 200))} + if severity and severity != "all": + params["severity"] = severity + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.get( + f"{self.base_url}/aml/events", + params=params, + headers=self._headers(token, library_id=library_id), + ) + self._raise_for_error(response) + body = response.json() + items = body.get("eventList", {}).get("event", []) if isinstance(body, dict) else [] + if not isinstance(items, list): + return [] + return [item for item in items if isinstance(item, dict)] + + def get_system_status(self, token: str, *, library_id: int | None = None) -> dict[str, Any]: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.get( + f"{self.base_url}/aml/system/status", + headers=self._headers(token, library_id=library_id), + ) + self._raise_for_error(response) + body = response.json() + if not isinstance(body, dict): + return {} + status = body.get("systemStatus") + if isinstance(status, dict): + return status + return body + + def get_health(self, token: str) -> dict[str, Any]: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.get( + f"{self.base_url}/healthz", + headers=self._headers(token), + ) + self._raise_for_error(response) + body = response.json() + if isinstance(body, dict): + return body + return {} + + def get_system_config(self, token: str) -> dict[str, Any]: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.get( + f"{self.base_url}/system/config-summary", + headers=self._headers(token), + ) + self._raise_for_error(response) + body = response.json() + if isinstance(body, dict): + return body + return {} + + def get_catalog_status(self, token: str) -> dict[str, Any]: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.get( + f"{self.base_url}/status/catalog", + headers=self._headers(token), + ) + self._raise_for_error(response) + body = response.json() + if isinstance(body, dict): + return body + return {} + + def get_library_status(self, token: str, *, library_id: int) -> dict[str, Any]: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.get( + f"{self.base_url}/status/library", + headers=self._headers(token, library_id=library_id), + ) + self._raise_for_error(response) + body = response.json() + if isinstance(body, dict): + return body + return {} + + def list_catalog_files(self, token: str, *, limit: int = 25) -> dict[str, Any]: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.get( + f"{self.base_url}/catalog/", + params={"limit": max(1, min(limit, 100))}, + headers=self._headers(token), + ) + self._raise_for_error(response) + body = response.json() + if isinstance(body, dict): + return body + return {"files": [], "total": 0} + + def list_nas_datasets( + self, + token: str, + *, + status: str | None = None, + limit: int = 100, + ) -> list[dict[str, Any]]: + params: dict[str, Any] = {"limit": max(1, min(limit, 500))} + if status: + params["status"] = status + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.get( + f"{self.base_url}/nas/datasets", + params=params, + headers=self._headers(token), + ) + self._raise_for_error(response) + body = response.json() + if not isinstance(body, list): + return [] + return [item for item in body if isinstance(item, dict)] + + def verify_nas_dataset(self, token: str, *, dataset_id: str) -> dict[str, Any]: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.post( + f"{self.base_url}/nas/datasets/{dataset_id}/verify", + headers=self._headers(token), + ) + self._raise_for_error(response) + body = response.json() + if isinstance(body, dict): + return body + return {} + + def create_archive_job( + self, token: str, *, source_path: str, volume_group: str + ) -> dict[str, Any]: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.post( + f"{self.base_url}/archive/", + json={"source_path": source_path, "volume_group": volume_group}, + headers=self._headers(token), + ) + self._raise_for_error(response) + body = response.json() + if isinstance(body, dict): + return body + return {} + + def create_restore_job( + self, token: str, *, catalog_path: str, dest_path: str + ) -> dict[str, Any]: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.post( + f"{self.base_url}/restore/", + json={"catalog_path": catalog_path, "dest_path": dest_path}, + headers=self._headers(token), + ) + self._raise_for_error(response) + body = response.json() + if isinstance(body, dict): + return body + return {} + + def probe_device_endpoint( + self, + *, + connection_url: str, + username: str | None = None, + password: str | None = None, + ) -> dict[str, Any]: + parsed = urlparse(connection_url) + if parsed.scheme not in {"http", "https"} or not parsed.netloc: + raise BackendError(HTTPStatus.BAD_REQUEST, "Invalid device URL") + base_url = connection_url.rstrip("/") + timeout = min(self.timeout_seconds, 5.0) + with httpx.Client(timeout=timeout, follow_redirects=True) as client: + reachable = False + for path in ("/healthz", "/health", "/aml/system/status"): + try: + response = client.get( + f"{base_url}{path}", headers={"Accept": "application/json"} + ) + except httpx.HTTPError: + continue + if response.status_code < HTTPStatus.BAD_REQUEST: + reachable = True + break + if not reachable: + raise BackendError(HTTPStatus.BAD_GATEWAY, "Unable to reach device health endpoint") + authenticated = False + if username: + login_response = client.post( + f"{base_url}/aml/users/login", + json={"name": username, "password": password or ""}, + headers={"Accept": "application/json"}, + ) + if login_response.status_code >= HTTPStatus.BAD_REQUEST: + raise BackendError(HTTPStatus.BAD_REQUEST, "Device authentication check failed") + authenticated = True + return {"reachable": True, "authenticated": authenticated} + + def list_volume_groups(self, token: str) -> list[dict[str, Any]]: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.get( + f"{self.base_url}/volume-groups/", + headers=self._headers(token), + ) + self._raise_for_error(response) + body = response.json() + if not isinstance(body, list): + return [] + return [item for item in body if isinstance(item, dict)] + + def create_nas_pool(self, token: str, payload: dict[str, Any]) -> dict[str, Any]: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.post( + f"{self.base_url}/nas/pools", + json=payload, + headers=self._headers(token), + ) + self._raise_for_error(response) + body = response.json() + if isinstance(body, dict): + return body + return {} + + def create_nas_share(self, token: str, payload: dict[str, Any]) -> dict[str, Any]: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.post( + f"{self.base_url}/nas/shares", + json=payload, + headers=self._headers(token), + ) + self._raise_for_error(response) + body = response.json() + if isinstance(body, dict): + return body + return {} + + def list_magazines(self, token: str, *, library_id: int) -> list[dict[str, Any]]: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.get( + f"{self.base_url}/aml/magazines", + headers=self._headers(token, library_id=library_id), + ) + self._raise_for_error(response) + body = response.json() + items = body.get("magazineList", {}).get("magazine", []) if isinstance(body, dict) else [] + if not isinstance(items, list): + return [] + return [item for item in items if isinstance(item, dict)] + + def list_magazine_slots( + self, token: str, *, library_id: int, magazine_id: str + ) -> list[dict[str, Any]]: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.get( + f"{self.base_url}/aml/magazine/{magazine_id}/slots", + headers=self._headers(token, library_id=library_id), + ) + self._raise_for_error(response) + body = response.json() + items = body.get("slotList", {}).get("slot", []) if isinstance(body, dict) else [] + if not isinstance(items, list): + return [] + return [item for item in items if isinstance(item, dict)] + + def eject_magazine(self, token: str, *, library_id: int, magazine_id: str) -> dict[str, Any]: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.post( + f"{self.base_url}/aml/magazine/{magazine_id}/eject", + headers=self._headers(token, library_id=library_id), + ) + self._raise_for_error(response) + body = response.json() + if isinstance(body, dict): + return body + return {} + + def insert_magazine(self, token: str, *, library_id: int, magazine_id: str) -> dict[str, Any]: + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.post( + f"{self.base_url}/aml/magazine/{magazine_id}/insert", + headers=self._headers(token, library_id=library_id), + ) + self._raise_for_error(response) + body = response.json() + if isinstance(body, dict): + return body + return {} + + @staticmethod + def device_payload(device: Device) -> dict[str, Any]: + return asdict(device) diff --git a/openblade/web_flask/models.py b/openblade/web_flask/models.py new file mode 100644 index 0000000..54ebae9 --- /dev/null +++ b/openblade/web_flask/models.py @@ -0,0 +1,29 @@ +from __future__ import annotations + +from dataclasses import dataclass +from typing import Any + + +@dataclass(frozen=True, slots=True) +class Device: + id: int + name: str + emulator_url: str + model: str + role: str + status: str + enabled: bool + sort_order: int + + +def parse_device(payload: dict[str, Any]) -> Device: + return Device( + id=int(payload.get("id", 0)), + name=str(payload.get("name", "Unknown Device")), + emulator_url=str(payload.get("emulator_url", "")), + model=str(payload.get("model", "Scalar i3")), + role=str(payload.get("role", "primary")), + status=str(payload.get("status", "unknown")).lower(), + enabled=bool(payload.get("enabled", True)), + sort_order=int(payload.get("sort_order", 0)), + ) diff --git a/openblade/web_flask/static/css/main.css b/openblade/web_flask/static/css/main.css new file mode 100644 index 0000000..9657621 --- /dev/null +++ b/openblade/web_flask/static/css/main.css @@ -0,0 +1,383 @@ +* { + box-sizing: border-box; +} + +body { + margin: 0; + font-family: Inter, "Segoe UI", Arial, sans-serif; + background: #0b1220; + color: #e2e8f0; +} + +a { + color: #93c5fd; + text-decoration: none; +} + +a:hover { + text-decoration: underline; +} + +.topbar { + display: flex; + align-items: center; + justify-content: space-between; + gap: 1rem; + padding: 0.9rem 1.2rem; + border-bottom: 1px solid #1e293b; + background: #111827; +} + +.brand { + font-weight: 700; + letter-spacing: 0.02em; +} + +.nav { + display: flex; + gap: 0.75rem; + flex-wrap: wrap; +} + +.nav a { + padding: 0.35rem 0.55rem; + border-radius: 0.35rem; + color: #cbd5e1; +} + +.nav a.active { + background: #1d4ed8; + color: #f8fafc; +} + +.identity { + display: flex; + align-items: center; + gap: 0.75rem; + color: #cbd5e1; +} + +.inline-form { + margin: 0; +} + +.link-button { + border: 0; + background: transparent; + color: #93c5fd; + cursor: pointer; + padding: 0; + font: inherit; +} + +.link-button:hover { + text-decoration: underline; +} + +.container { + max-width: 1200px; + margin: 1.25rem auto; + padding: 0 1rem 2rem; +} + +.page-header { + display: flex; + align-items: flex-start; + justify-content: space-between; + gap: 1rem; + margin-bottom: 1rem; +} + +.panel { + border: 1px solid #1e293b; + border-radius: 0.5rem; + padding: 1rem; + background: #0f172a; +} + +.panel-narrow { + max-width: 560px; +} + +.summary { + display: flex; + flex-wrap: wrap; + gap: 0.5rem; + margin-bottom: 1rem; +} + +.chip { + border: 1px solid #1e293b; + border-radius: 999px; + padding: 0.25rem 0.65rem; + font-size: 0.86rem; + background: #0f172a; +} + +.grid { + display: grid; + grid-template-columns: repeat(auto-fit, minmax(280px, 1fr)); + gap: 1rem; +} + +.card { + border: 1px solid #1e293b; + border-radius: 0.6rem; + padding: 0.9rem; + background: #0f172a; +} + +.card h2 { + margin: 0.6rem 0 0.6rem; + font-size: 1.1rem; +} + +.card-top { + display: flex; + justify-content: space-between; + gap: 1rem; + align-items: center; +} + +.meta { + display: grid; + gap: 0.4rem; + font-size: 0.9rem; + color: #cbd5e1; +} + +.status { + border-radius: 999px; + padding: 0.18rem 0.55rem; + font-size: 0.8rem; + font-weight: 700; +} + +.status-online { + border: 1px solid #16a34a; + color: #4ade80; +} + +.status-warning { + border: 1px solid #f59e0b; + color: #fbbf24; +} + +.status-running { + border: 1px solid #3b82f6; + color: #60a5fa; +} + +.status-offline { + border: 1px solid #dc2626; + color: #fca5a5; +} + +.status-unknown { + border: 1px solid #64748b; + color: #94a3b8; +} + +.button, +button { + background: #2563eb; + color: #f8fafc; + border: 1px solid #2563eb; + border-radius: 0.4rem; + padding: 0.45rem 0.75rem; + font-weight: 600; + cursor: pointer; + text-decoration: none; + display: inline-flex; + align-items: center; +} + +.button:hover, +button:hover { + background: #1d4ed8; + border-color: #1d4ed8; + text-decoration: none; +} + +.button-secondary { + background: transparent; + border-color: #334155; + color: #e2e8f0; +} + +.button-danger { + background: #7f1d1d; + border-color: #dc2626; + color: #fca5a5; +} + +.button-danger:hover { + background: #991b1b; + border-color: #ef4444; + text-decoration: none; +} + +.filter-form { + display: flex; + align-items: end; + gap: 0.6rem; + flex-wrap: wrap; +} + +.filter-form label { + display: grid; + gap: 0.3rem; + font-size: 0.85rem; +} + +.form-grid { + display: grid; + gap: 0.8rem; +} + +.form-grid label { + display: grid; + gap: 0.35rem; + font-weight: 600; +} + +.form-grid input, +.form-grid select, +.form-grid textarea { + border: 1px solid #334155; + border-radius: 0.35rem; + padding: 0.5rem; + background: #020617; + color: #e2e8f0; +} + +.error { + color: #fca5a5; +} + +.actions { + display: flex; + gap: 0.5rem; +} + +.alerts { + display: grid; + gap: 0.5rem; + margin-bottom: 0.85rem; +} + +.alert { + border-radius: 0.4rem; + padding: 0.45rem 0.6rem; + border: 1px solid #334155; + background: #111827; +} + +.alert.success { + border-color: #16a34a; +} + +.alert.error { + border-color: #dc2626; +} + +.alert.warning { + border-color: #f59e0b; +} + +.alert.info { + border-color: #3b82f6; +} + +.hidden { + display: none; +} + +.tab-bar { + display: flex; + flex-wrap: wrap; + gap: 0.4rem; + margin-bottom: 1rem; +} + +.tab { + padding: 0.35rem 0.65rem; + border-radius: 0.35rem; + border: 1px solid #334155; + color: #cbd5e1; +} + +.tab:hover { + text-decoration: none; + border-color: #475569; +} + +.tab-active { + border-color: #1d4ed8; + background: #1d4ed8; + color: #f8fafc; +} + +.data-table { + width: 100%; + border-collapse: collapse; +} + +.data-table th, +.data-table td { + border-bottom: 1px solid #1e293b; + text-align: left; + padding: 0.45rem; + vertical-align: top; +} + +.data-table th { + font-size: 0.8rem; + text-transform: uppercase; + letter-spacing: 0.05em; + color: #94a3b8; +} + +.mono { + font-family: ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, "Liberation Mono", "Courier New", monospace; +} + +.empty-state { + border: 1px dashed #334155; + border-radius: 0.5rem; + padding: 0.8rem; +} + +.plain-list { + margin: 0; + padding-left: 1rem; +} + +.hint { + color: #94a3b8; +} + +.command-block { + margin: 0.5rem 0; + border: 1px solid #334155; + border-radius: 0.4rem; + background: #020617; + padding: 0.6rem; + overflow-x: auto; + white-space: pre-wrap; + word-break: break-word; +} + +.summary.vertical { + display: grid; +} + +.grid-70-30 { + grid-template-columns: minmax(0, 2fr) minmax(220px, 1fr); +} + +@media (max-width: 900px) { + .grid-70-30 { + grid-template-columns: 1fr; + } +} diff --git a/openblade/web_flask/static/js/confirm-actions.js b/openblade/web_flask/static/js/confirm-actions.js new file mode 100644 index 0000000..8510b30 --- /dev/null +++ b/openblade/web_flask/static/js/confirm-actions.js @@ -0,0 +1,21 @@ +(() => { + document.querySelectorAll("[data-confirm]").forEach((button) => { + const form = button.closest("form"); + if (!form) { + return; + } + form.addEventListener("submit", (event) => { + if (form.dataset.confirmed === "true") { + form.dataset.confirmed = ""; + return; + } + const message = button.getAttribute("data-confirm") || "Confirm this action?"; + if (!window.confirm(message)) { + event.preventDefault(); + return; + } + form.dataset.confirmed = "true"; + }); + }); +})(); + diff --git a/openblade/web_flask/static/js/copy-actions.js b/openblade/web_flask/static/js/copy-actions.js new file mode 100644 index 0000000..4d1b512 --- /dev/null +++ b/openblade/web_flask/static/js/copy-actions.js @@ -0,0 +1,24 @@ +(() => { + const buttons = document.querySelectorAll("[data-copy-text]"); + if (!buttons.length) { + return; + } + buttons.forEach((button) => { + button.addEventListener("click", async () => { + const text = button.getAttribute("data-copy-text") || ""; + if (!text) { + return; + } + try { + await navigator.clipboard.writeText(text); + } catch { + const field = document.createElement("textarea"); + field.value = text; + document.body.appendChild(field); + field.select(); + document.execCommand("copy"); + document.body.removeChild(field); + } + }); + }); +})(); diff --git a/openblade/web_flask/static/js/devices-sse.js b/openblade/web_flask/static/js/devices-sse.js new file mode 100644 index 0000000..27667a7 --- /dev/null +++ b/openblade/web_flask/static/js/devices-sse.js @@ -0,0 +1,95 @@ +(() => { + const page = document.body.dataset.page; + if (page !== "devices-index") { + return; + } + + const summaryNode = document.getElementById("device-summary"); + const warningNode = document.getElementById("sse-warning"); + const updateNode = document.getElementById("devices-last-update"); + if (!summaryNode || !warningNode || !updateNode) { + return; + } + + let retryDelayMs = 1000; + let currentSource = null; + + function setWarning(visible) { + warningNode.classList.toggle("hidden", !visible); + } + + function updateSummary(summary) { + ["online", "warning", "offline", "unknown", "total"].forEach((key) => { + const node = summaryNode.querySelector(`[data-summary="${key}"]`); + if (node && typeof summary[key] === "number") { + node.textContent = String(summary[key]); + } + }); + } + + function updateDeviceCards(devices) { + devices.forEach((device) => { + const card = document.querySelector(`[data-device-id="${device.id}"]`); + if (!card) { + return; + } + const statusNode = card.querySelector('[data-device-field="status"]'); + const nameNode = card.querySelector('[data-device-field="name"]'); + if (statusNode) { + statusNode.textContent = String(device.status || "unknown"); + statusNode.className = `status ${statusClass(device.status)}`; + } + if (nameNode) { + nameNode.textContent = String(device.name || "Unknown"); + } + }); + } + + function statusClass(status) { + const normalized = String(status || "").toLowerCase(); + if (normalized === "online") { + return "status-online"; + } + if (normalized === "degraded" || normalized === "warning") { + return "status-warning"; + } + if (normalized === "offline" || normalized === "failed" || normalized === "error") { + return "status-offline"; + } + return "status-unknown"; + } + + function connect() { + currentSource = new EventSource("/events/devices"); + + currentSource.addEventListener("devices.snapshot", (event) => { + try { + const payload = JSON.parse(event.data); + updateSummary(payload.summary || {}); + updateDeviceCards(payload.devices || []); + setWarning(false); + const timestamp = Number(payload.updated_at || 0); + updateNode.textContent = timestamp > 0 ? new Date(timestamp * 1000).toLocaleTimeString() : "just now"; + retryDelayMs = 1000; + } catch (_err) { + setWarning(true); + } + }); + + currentSource.addEventListener("devices.error", () => { + setWarning(true); + }); + + currentSource.onerror = () => { + setWarning(true); + if (currentSource) { + currentSource.close(); + } + setTimeout(connect, retryDelayMs); + retryDelayMs = Math.min(retryDelayMs * 2, 30000); + }; + } + + connect(); +})(); + diff --git a/openblade/web_flask/static/js/jobs-sse.js b/openblade/web_flask/static/js/jobs-sse.js new file mode 100644 index 0000000..0a9a031 --- /dev/null +++ b/openblade/web_flask/static/js/jobs-sse.js @@ -0,0 +1,59 @@ +(() => { + const page = document.body.dataset.page; + if (page !== "nas-jobs") { + return; + } + + const summaryNode = document.getElementById("jobs-summary"); + const warningNode = document.getElementById("jobs-sse-warning"); + const updateNode = document.getElementById("jobs-last-update"); + if (!summaryNode || !warningNode || !updateNode) { + return; + } + + let retryDelayMs = 1000; + let source = null; + + function showWarning(visible) { + warningNode.classList.toggle("hidden", !visible); + } + + function connect() { + source = new EventSource("/events/jobs"); + + source.addEventListener("jobs.snapshot", (event) => { + try { + const payload = JSON.parse(event.data); + ["total", "running", "pending", "failed"].forEach((key) => { + const node = summaryNode.querySelector(`[data-jobs-summary="${key}"]`); + if (node && typeof payload[key] === "number") { + node.textContent = String(payload[key]); + } + }); + updateNode.textContent = payload.updated_at + ? new Date(Number(payload.updated_at) * 1000).toLocaleTimeString() + : "just now"; + showWarning(false); + retryDelayMs = 1000; + } catch (_err) { + showWarning(true); + } + }); + + source.addEventListener("jobs.error", () => { + showWarning(true); + }); + + source.onerror = () => { + showWarning(true); + if (source) { + source.close(); + } + setTimeout(connect, retryDelayMs); + retryDelayMs = Math.min(retryDelayMs * 2, 30000); + }; + } + + connect(); +})(); + diff --git a/openblade/web_flask/static/js/operations-search.js b/openblade/web_flask/static/js/operations-search.js new file mode 100644 index 0000000..5aad1ed --- /dev/null +++ b/openblade/web_flask/static/js/operations-search.js @@ -0,0 +1,27 @@ +(() => { + const searchInputs = document.querySelectorAll("[data-table-search]"); + if (!searchInputs.length) { + return; + } + + const normalize = (value) => value.trim().toLowerCase(); + + searchInputs.forEach((input) => { + const tableId = input.getAttribute("data-table-search"); + if (!tableId) { + return; + } + const table = document.getElementById(tableId); + if (!table) { + return; + } + const rows = Array.from(table.querySelectorAll("tbody tr[data-search-row]")); + input.addEventListener("input", () => { + const term = normalize(input.value || ""); + rows.forEach((row) => { + const text = normalize(row.textContent || ""); + row.classList.toggle("hidden", Boolean(term) && !text.includes(term)); + }); + }); + }); +})(); diff --git a/openblade/web_flask/static/js/register-form.js b/openblade/web_flask/static/js/register-form.js new file mode 100644 index 0000000..9461bdb --- /dev/null +++ b/openblade/web_flask/static/js/register-form.js @@ -0,0 +1,55 @@ +(() => { + const form = document.getElementById("register-device-form"); + if (!form) { + return; + } + + const connectionField = form.querySelector('input[name="connection_url"]'); + const usernameField = form.querySelector('input[name="device_username"]'); + const passwordField = form.querySelector('input[name="device_password"]'); + const statusNode = document.getElementById("register-probe-status"); + if (!connectionField) { + return; + } + + const runProbe = async () => { + const csrfNode = document.querySelector('meta[name="csrf-token"]'); + const token = csrfNode ? csrfNode.getAttribute("content") : ""; + const formData = new FormData(); + formData.set("connection_url", connectionField.value); + if (usernameField) { + formData.set("device_username", usernameField.value); + } + if (passwordField) { + formData.set("device_password", passwordField.value); + } + formData.set("csrf_token", token || ""); + try { + const response = await fetch("/devices/probe", { + method: "POST", + body: formData, + credentials: "same-origin", + }); + if (!statusNode) { + return; + } + if (response.ok) { + statusNode.textContent = "Connection probe passed."; + statusNode.classList.remove("error"); + } else { + const payload = await response.json().catch(() => ({})); + statusNode.textContent = payload.detail || "Connection probe failed."; + statusNode.classList.add("error"); + } + } catch (_err) { + if (statusNode) { + statusNode.textContent = "Connection probe unavailable."; + statusNode.classList.add("error"); + } + } + }; + connectionField.addEventListener("blur", runProbe); + if (usernameField) { + usernameField.addEventListener("blur", runProbe); + } +})(); diff --git a/openblade/web_flask/templates/base.html b/openblade/web_flask/templates/base.html new file mode 100644 index 0000000..cfc3bda --- /dev/null +++ b/openblade/web_flask/templates/base.html @@ -0,0 +1,50 @@ + + + + + + + {% block title %}OpenBlade NAS{% endblock %} + + + +
+
OpenBlade NAS
+ +
+ {% if current_user %} + {{ current_user }} +
+ + +
+ {% endif %} +
+
+ +
+ {% with messages = get_flashed_messages(with_categories=true) %} + {% if messages %} +
+ {% for category, message in messages %} +
{{ message }}
+ {% endfor %} +
+ {% endif %} + {% endwith %} + {% block content %}{% endblock %} +
+ + + + + + + + + diff --git a/openblade/web_flask/templates/devices/index.html b/openblade/web_flask/templates/devices/index.html new file mode 100644 index 0000000..2799537 --- /dev/null +++ b/openblade/web_flask/templates/devices/index.html @@ -0,0 +1,51 @@ +{% extends "base.html" %} +{% block title %}Connected Devices · OpenBlade NAS{% endblock %} +{% block content %} +
+
+

Connected Devices

+

Register and monitor each Quantum device from a single NAS-focused dashboard.

+
+ Register Device +
+ +
+ Online: {{ summary.online }} + Degraded: {{ summary.warning }} + Offline: {{ summary.offline }} + Unknown: {{ summary.unknown }} + Total: {{ summary.total }} + Last update: initial +
+ + {% if devices %} +
+ {% for device in devices %} +
+
+ {{ status_text(device.status) }} +
+ Client Access + Manage +
+
+

{{ device.name }}

+
+ Model: {{ device.model }} + URL: {{ device.emulator_url }} +
+
+ {% endfor %} +
+ {% else %} +
+

No devices registered yet

+

Register your first device to unlock library management pages.

+ Register your first device +
+ {% endif %} + + +{% endblock %} diff --git a/openblade/web_flask/templates/devices/manage.html b/openblade/web_flask/templates/devices/manage.html new file mode 100644 index 0000000..cd357b2 --- /dev/null +++ b/openblade/web_flask/templates/devices/manage.html @@ -0,0 +1,24 @@ +{% extends "base.html" %} +{% block title %}Manage {{ device.name }} · OpenBlade NAS{% endblock %} +{% block content %} +
+
+

Devices / Manage

+

{{ device.name }}

+

+ {{ status_text(device.status) }} + · {{ device.model }} +

+
+
+ +
+ {% for section_key, section_label in sections %} +
+

{{ section_label }}

+

Device-scoped {{ section_label | lower }} controls for {{ device.name }}.

+ Open {{ section_label }} +
+ {% endfor %} +
+{% endblock %} diff --git a/openblade/web_flask/templates/devices/manage_section.html b/openblade/web_flask/templates/devices/manage_section.html new file mode 100644 index 0000000..2d229fc --- /dev/null +++ b/openblade/web_flask/templates/devices/manage_section.html @@ -0,0 +1,723 @@ +{% extends "base.html" %} +{% block title %}{{ section|capitalize }} · {{ device.name }} · OpenBlade NAS{% endblock %} +{% block content %} +
+
+

Devices / {{ device.name }}

+

{{ device.name }}

+

+ {{ status_text(device.status) }} + · {{ device.model }} +

+
+
+ + + + {% if section == "overview" %} +
+
+

Device Health

+

Status: {{ status_text(device.status) }}

+

Connection: {{ device.emulator_url }}

+
+
+

Library Snapshot

+ {% set status = section_data.library_status or {} %} +

Overall: {{ status_text(status.overall or status.get('overall', 'unknown')) }}

+

Drives: {{ section_data.drives|length }}

+

Media: {{ section_data.media_count or 0 }}

+
+
+ {% elif section == "inventory" %} +
+

Slot Inventory

+ {% set media = section_data.media or [] %} + {% if media %} + + + + + + + + + + + + {% for item in media %} + + + + + + + + {% endfor %} + +
BarcodeTypeStatePartitionSlot
{{ item.barcode or "—" }}{{ item.type or "—" }}{{ status_text(item.state or "unknown") }}{{ item.partition or "—" }}{{ item.slot or item.location or "—" }}
+ {% else %} +
+

No inventory records available

+

Run an inventory operation to refresh the media map for this device.

+ Go to Operations +
+ {% endif %} +
+ {% elif section == "drives" %} +
+

Drive Status

+ {% set drives = section_data.drives or [] %} + {% if drives %} + + + + + + + + + + + {% for drive in drives %} + + + + + + + {% endfor %} + +
SerialTypeStateLoaded Media
{{ drive.serialNumber or "—" }}{{ drive.type or "—" }}{{ status_text(drive.state or drive.status or "unknown") }}{{ drive.loadedMedia or "—" }}
+ {% else %} +
+

No drives reported

+

This library has not reported drives yet.

+
+ {% endif %} +
+ {% elif section == "partitions" %} +
+

Partitions

+ {% set partitions = section_data.partitions or [] %} + {% if partitions %} + + + + + + + + + + + {% for partition in partitions %} + + + + + + + {% endfor %} + +
NameTypeStateSlots
{{ partition.name or partition.partitionName or "—" }}{{ partition.type or "—" }}{{ status_text(partition.state or "unknown") }}{{ partition.slotCount or partition.totalSlots or "—" }}
+ {% else %} +
+

No partitions configured

+

This device currently has no partition records.

+
+ {% endif %} +
+ {% elif section == "jobs" %} +
+

Device Jobs

+ {% set jobs = section_data.jobs or [] %} + {% if jobs %} + + + + + + + + + + + + {% for job in jobs %} + + + + + + + + {% endfor %} + +
Job IDTypeStatusUpdatedError
{{ (job.id or "—")[:8] }}{% if job.id %}…{% endif %}{{ job_type_label(job.job_type or "unknown") }}{{ status_text(job.state or "unknown") }}{{ job.updated_at or "—" }}{{ job.error or "" }}
+ {% else %} +
+

No jobs for {{ device.name }}

+

Run an operation to create archive, restore, or inventory jobs for this device.

+ Go to Operations +
+ {% endif %} +
+ {% elif section == "operations" %} + {% set active_ops = section_data.active_ops or [] %} + {% set drives = section_data.drives or [] %} + {% set media = section_data.media or [] %} + {% set partitions = section_data.partitions or [] %} + {% set ie_stations = section_data.ie_stations or [] %} + {% set mounts = section_data.mounts or [] %} + {% set pools = section_data.pools or [] %} + {% set shares = section_data.shares or [] %} + {% set volume_groups = section_data.volume_groups or [] %} + {% set magazines = section_data.magazines or [] %} + {% set catalog_files = section_data.catalog_files or [] %} + {% set drive_media_index = section_data.drive_media_index or [] %} + {% set inventory_status = section_data.inventory_status or {} %} + {% set import_status = section_data.import_status or {} %} + {% set export_status = section_data.export_status or {} %} + {% if active_ops %} + + {% endif %} + + {% for item in media %} + {% if item.barcode %} + + {% endif %} + {% endfor %} + {% for mount in mounts %} + {% if mount.barcode %} + + {% endif %} + {% endfor %} + + + {% for drive in drives %} + + {% endfor %} + + + {% for item in media %} + {% if item.slot %} + + {% endif %} + {% endfor %} + + + {% for partition in partitions %} + {% set partition_name = partition.name or partition.partitionName %} + {% if partition_name %} + + {% endif %} + {% endfor %} + + + {% for station in ie_stations %} + + {% endfor %} + + + {% for pool in pools %} + + {% endfor %} + + + {% for file in catalog_files %} + {% if file.path %} + + {% endif %} + {% endfor %} + +
+

Operations

+
Operations directly control hardware behavior. Confirm each action before execution.
+
+
+

Operation Status

+

Inventory: {{ status_text(inventory_status.state or "idle") }}

+

Import: {{ status_text(import_status.state or "idle") }}

+

Export: {{ status_text(export_status.state or "idle") }}

+
+ + +
+
+ +
+

Move Media

+
+ + + + + +
+
+ +
+

Mount Media

+
+ + + + + +
+
+ +
+

Unmount Media

+
+ + + + +
+
+ +
+

Import from I/E Station

+
+ + + + +
+
+ +
+

Export to I/E Station

+
+ + + + +
+
+ +
+

Pool Controls

+
+ + + + + + + + + + + +
+
+ +
+

Map Share to Pool

+
+ + + + + +
+ {% if volume_groups %} +

Available Volume Groups: {% for group in volume_groups %}{{ group.id }}{% if not loop.last %}, {% endif %}{% endfor %}

+ {% endif %} +
+ +
+

Hydrate File from Tape

+
+ + + + +
+
+
+ +

Pools

+ {% if pools %} + + + + + + + + + + + + + {% for pool in pools %} + + + + + + + + + {% endfor %} + +
IDNameReplicationBackup OrderAccessVolume Groups
{{ pool.id }}{{ pool.name }}{{ pool.replication_factor or 1 }}{{ status_text(pool.backup_order_mode or "sequential") }}{{ status_text(pool.access_mode or "read_only") }}{{ (pool.volume_group_ids or [])|join(", ") or "—" }}
+ {% else %} +
+

No pools configured

+

Create a pool to control tape sequencing and replication behavior.

+
+ {% endif %} + +

Pool Shares

+ {% if shares %} + + + + + + + + + + + {% for share in shares %} + + + + + + + {% endfor %} + +
NamePathTypeWritable
{{ share.name }}{{ share.path }}{{ share_type_label(share.share_type or "pool") }}{{ "Yes" if share.writable else "No" }}
+ {% endif %} + +

Magazine Control

+ {% if magazines %} + + + + + + + + + + + + + + {% for magazine in magazines %} + + + + + + + + + + {% endfor %} + +
IDLocationStatusSlotsMapped SlotsTapesActions
{{ magazine.id }}{{ magazine.location or "—" }}{{ status_text(magazine.status or "unknown") }}{{ magazine.slotCount or 0 }} + {{ magazine.mappedSlotCount or 0 }} + {% if not magazine.slotMapAligned %} + Mismatch + {% endif %} + {{ magazine.occupiedSlots or 0 }} +
+
+ + +
+
+ + +
+
+
+ {% else %} +
+

No magazines reported

+

Magazine inventory will appear after physical sync and inventory scan.

+
+ {% endif %} + +

Tape and Content Search

+ + {% if drive_media_index %} + + + + + + + + + + + + + {% for item in drive_media_index %} + + + + + + + + + {% endfor %} + +
BarcodeDriveSlotPartitionStateCatalog Paths
{{ item.barcode }}{{ item.drive }}{{ item.slot }}{{ item.partition }}{{ status_text(item.state or "unknown") }} + {% if item.sample_paths %} +
    + {% for path in item.sample_paths %} +
  • {{ path }}
    • + {% endfor %} +
+ {% else %} + — + {% endif %} +
+ {% else %} +
+

No tape content indexed

+

Run inventory and archive jobs to populate searchable tape content.

+
+ {% endif %} + + {% if mounts %} +

Current Mounts

+ + + + + + + + + + + {% for mount in mounts %} + + + + + + + {% endfor %} + +
BarcodeDriveStateMounted
{{ mount.barcode or "—" }}{{ mount.drive or "—" }}{{ status_text(mount.state or "unknown") }}{{ mount.mountTime or "—" }}
+ {% endif %} + + {% if active_ops %} +

Active Operations

+ + + + + + + + + + + {% for job in active_ops %} + + + + + + + {% endfor %} + +
Job IDTypeStatusUpdated
{{ (job.id or "—")[:8] }}{% if job.id %}…{% endif %}{{ job_type_label(job.job_type or "unknown") }}{{ status_text(job.state or "unknown") }}{{ job.updated_at or "—" }}
+ {% endif %} +
+ {% elif section == "reports" %} +
+
+

Event Summary

+ {% set event_summary = section_data.event_summary or {} %} +

Total: {{ event_summary.total or 0 }}

+

Critical: {{ event_summary.critical or 0 }}

+

Warning: {{ event_summary.warning or 0 }}

+

Info: {{ event_summary.info or 0 }}

+
+
+

Alert Summary

+ {% set alert_summary = section_data.alert_summary or {} %} +

Critical: {{ alert_summary.critical or 0 }}

+

Warning: {{ alert_summary.warning or 0 }}

+

Info: {{ alert_summary.info or 0 }}

+
+
+
+

Recent Events

+ {% set events = section_data.events or [] %} + {% if events %} + + + + + + + + + + + {% for event in events %} + + + + + + + {% endfor %} + +
TimestampSeverityComponentMessage
{{ event.timestamp or "—" }}{{ status_text(event.severity or "unknown") }}{{ event.component or "—" }}{{ event.message or "—" }}
+ {% else %} +
+

No events recorded

+

This device has no reportable events yet.

+
+ {% endif %} +
+ {% elif section == "settings" %} +
+

Device Settings

+
+ + + + +
+ +
+
+
+ {% endif %} +{% endblock %} diff --git a/openblade/web_flask/templates/devices/register.html b/openblade/web_flask/templates/devices/register.html new file mode 100644 index 0000000..6e95538 --- /dev/null +++ b/openblade/web_flask/templates/devices/register.html @@ -0,0 +1,59 @@ +{% extends "base.html" %} +{% block title %}Register Device · OpenBlade NAS{% endblock %} +{% block content %} +
+

Register Device

+

Add a connected Quantum library endpoint to the dashboard.

+ +
+ + + + + + + + + + + + + + +
+ + Cancel +
+ +
+
+
+{% endblock %} diff --git a/openblade/web_flask/templates/jobs/index.html b/openblade/web_flask/templates/jobs/index.html new file mode 100644 index 0000000..99b4cb1 --- /dev/null +++ b/openblade/web_flask/templates/jobs/index.html @@ -0,0 +1,89 @@ +{% extends "base.html" %} +{% block title %}Jobs · OpenBlade NAS{% endblock %} +{% block content %} +
+
+

Jobs

+

Track archive, restore, inventory, and system jobs across connected devices.

+
+
+ + + + +
+
+ +
+ Total: {{ jobs_summary.total }} + Running: {{ jobs_summary.running }} + Pending: {{ jobs_summary.pending }} + Failed: {{ jobs_summary.failed }} + Last update: initial +
+ + {% if jobs %} +
+ + + + + + + + + + + + + + {% for job in jobs %} + + + + + + + + + + {% endfor %} + +
Job IDTypeStatusLibraryCreatedUpdatedError
{{ (job.id or "—")[:8] }}{% if job.id %}…{% endif %}{{ job_type_label(job.job_type) }}{{ status_text(job.state) }}{{ job.library_id or "—" }}{{ job.created_at }}{{ job.updated_at }}{{ job.error or "" }}
+
+ {% else %} +
+

No jobs yet

+

Jobs are created when archive, restore, or inventory operations are triggered.

+ Go to Devices +
+ {% endif %} + + +{% endblock %} diff --git a/openblade/web_flask/templates/login.html b/openblade/web_flask/templates/login.html new file mode 100644 index 0000000..f3db2f8 --- /dev/null +++ b/openblade/web_flask/templates/login.html @@ -0,0 +1,23 @@ +{% extends "base.html" %} +{% block title %}Login · OpenBlade NAS{% endblock %} +{% block content %} +
+

Sign in

+

Use your Quantum-compatible credentials to access NAS operations.

+
+ + {% if next_path %} + + {% endif %} + + + +
+
+{% endblock %} diff --git a/openblade/web_flask/templates/nas_placeholder.html b/openblade/web_flask/templates/nas_placeholder.html new file mode 100644 index 0000000..a618794 --- /dev/null +++ b/openblade/web_flask/templates/nas_placeholder.html @@ -0,0 +1,13 @@ +{% extends "base.html" %} +{% block title %}{{ heading }} · OpenBlade NAS{% endblock %} +{% block content %} +
+

{{ heading }}

+

+ NAS workflows stay globally available in this top bar. Library-specific actions remain inside each device's + Manage page. +

+

Current section: {{ section }}

+
+{% endblock %} + diff --git a/openblade/web_flask/templates/reports/index.html b/openblade/web_flask/templates/reports/index.html new file mode 100644 index 0000000..70c37d8 --- /dev/null +++ b/openblade/web_flask/templates/reports/index.html @@ -0,0 +1,81 @@ +{% extends "base.html" %} +{% block title %}Reports · OpenBlade NAS{% endblock %} +{% block content %} +
+
+

Reports

+

Event and alert reporting scoped to a selected connected device.

+
+
+ + + +
+
+ + {% if active_device %} +
+
+

Event Log

+ {% if events %} + + + + + + + + + + + {% for event in events %} + + + + + + + {% endfor %} + +
TimestampSeverityComponentMessage
{{ event.timestamp or "—" }}{{ status_text(event.severity or "unknown") }}{{ event.component or "—" }}{{ event.message or "—" }}
+ {% else %} +
+

No events found

+

No events were recorded for this filter on {{ active_device.name }}.

+
+ {% endif %} +
+
+

Alert Summary

+
+ Critical: {{ (event_summary.critical or 0) + (alert_summary.critical or 0) }} + Warnings: {{ (event_summary.warning or 0) + (alert_summary.warning or 0) }} + Info: {{ (event_summary.info or 0) + (alert_summary.info or 0) }} +
+

Total events: {{ event_summary.total or 0 }}

+
+
+ {% else %} +
+

No devices connected

+

Register a device first to unlock report and alert views.

+ Register Device +
+ {% endif %} +{% endblock %} + diff --git a/openblade/web_flask/templates/storage/index.html b/openblade/web_flask/templates/storage/index.html new file mode 100644 index 0000000..05c42b7 --- /dev/null +++ b/openblade/web_flask/templates/storage/index.html @@ -0,0 +1,677 @@ +{% extends "base.html" %} +{% block title %}Storage · OpenBlade NAS{% endblock %} +{% block content %} +
+
+

Storage

+

NAS-level archive and restore workflows with device-scoped execution controls.

+
+
+ + +
+
+ + + + {% if section == "overview" %} +
+ {% set catalog = section_data.catalog_status or {} %} + Catalog files: {{ catalog.total_file_records or 0 }} + Datasets: {{ catalog.total_datasets or 0 }} + Cartridges: {{ catalog.total_cartridges or 0 }} + + Library: {{ status_text((section_data.library_status or {}).overall or 'unknown') }} + +
+
+
+

Catalog Health

+

DB reachable: {{ "Yes" if (catalog.db_reachable or false) else "No" }}

+

Path mappings: {{ catalog.total_path_mappings or 0 }}

+

Last rebuild: {{ catalog.last_rebuild_status or "unknown" }}

+
+
+

Media Pools

+ {% set pools = section_data.media_pools or [] %} + {% if pools %} +
    + {% for pool in pools[:6] %} +
  • {{ pool.name or "Unnamed" }} ({{ pool.type or "pool" }})
    • + {% endfor %} +
+ {% else %} +

No media pools reported for this device.

+ {% endif %} +
+
+ {% elif section == "access" %} + {% set access = section_data.share_access or [] %} + {% set pools = section_data.nas_pools or [] %} + {% set gateway_status = section_data.gateway_status or {} %} + {% set gateway_config = section_data.gateway_config or {} %} +
+ Connect host: {{ section_data.connect_host or "unknown" }} + NAS shares: {{ access|length }} + Virtual mounts enabled: {{ pools|selectattr("virtual_mount_enabled")|list|length }} + + SFTP gateway: {{ status_text(gateway_status.status or gateway_config.status or "unknown") }} + +
+ +
+
+

Windows mapped drive (SMB)

+

Use a share UNC path in File Explorer or map a drive letter via command line.

+
    +
  1. Open File Explorer and choose Map network drive.
    2. +
  2. Enter UNC path \\{{ section_data.connect_host }}\{{ (access[0].name if access else "inbox") }}.
    3. +
  3. Authenticate with your OpenBlade account (or configured gateway credential).
    4. +
+ {% if access %} + 
{{ access[0].windows_command }}
+ + {% endif %} +
+
+

Linux mount (CIFS/NFS)

+

Create mountpoint then run CIFS (Windows-compatible) or NFS command.

+ {% if access %} + 
{{ access[0].linux_cifs_command }}
+ + 
{{ access[0].linux_nfs_command }}
+ + {% else %} +

No NAS shares are currently configured.

+ {% endif %} +
+
+ +
+

SFTP fallback connection

+ {% set username = section_data.example_username or current_user or "admin" %} + {% set gateway_port = gateway_config.bind_port or 2222 %} + {% set sftp_command = "sftp -P " ~ gateway_port ~ " " ~ username ~ "@" ~ (section_data.connect_host or "127.0.0.1") %} +

If SMB/NFS is unavailable, use SFTP for direct transfer into ingest/restore paths.

+ 
{{ sftp_command }}
+ +
+ +
+

Per-share quick commands

+ {% if access %} + + + + {% for item in access %} + + + + + + + + {% endfor %} + +
SharePurposeModeWindows UNCLinux CIFS
{{ item.name }}{{ share_type_label(item.share_type) }}{{ "Read/Write" if item.writable else "Read-only" }} +
{{ item.windows_unc }}
+ + 		+
{{ item.linux_cifs_command }}
+ +
+ {% else %} +
+

No NAS shares configured

+

Create share definitions in NAS configuration before mounting from clients.

+
+ {% endif %} +
+ {% elif section == "write-path" %} + {% set policies = section_data.policies or [] %} + {% set cache_drives = section_data.cache_drives or [] %} + {% set pools = section_data.nas_pools or [] %} + {% set shares = section_data.nas_shares or [] %} + {% set source_stream = section_data.source_stream or {} %} + {% set recommendations = section_data.recommendations or [] %} + {% set metrics = section_data.metrics or {} %} +
+ Pools: {{ pools|length }} + Policies: {{ policies|length }} + Cache drives: {{ cache_drives|length }} + + Source stream: {{ "Enabled" if source_stream.enabled else "Disabled" }} + + Replicated pools: {{ metrics.replicated_pools or 0 }} + Sharded policies: {{ metrics.sharded_policies or 0 }} + Folder rules: {{ metrics.folder_rules or 0 }} +
+ + + {% for policy in policies %} + + {% endfor %} + + + {% for pool in pools %} + + {% endfor %} + + +
+
+

Policy for multi-tape writes

+

Tune shard size and parallelism here, then map folders to pools below.

+
+ + + + + + + + + + + + + +
+ +
+
+
+ +
+

Cache drive staging

+
+ + + + + + + + + + + + +
+ +
+
+
+ +
+

Streaming direct-to-tape

+
+ + + + + + + + +
+ +
+
+
+ +
+

Folder-to-pool access mapping

+
+ + + + + + + + +
+ +
+
+
+
+ +
+

Optimization guidance

+
    + {% for item in recommendations %} +
  • {{ item }}
    • + {% endfor %} +
+
+ +
+

Current policy and tape write profile

+ {% if policies %} + + + + {% for policy in policies %} + + + + + + + + + + + {% endfor %} + +
PolicyTypeIngestCopiesShardingShard SizeParallelismAuto Clean
{{ policy.name or policy.id }}{{ status_text(policy.policy_type or "balanced") }}{{ status_text(policy.default_ingest_mode or "cache_drive") }}{{ policy.copies_required or 1 }}{{ "Enabled" if policy.allow_sharding else "Disabled" }}{{ policy.shard_size_bytes or "—" }}{{ policy.max_parallelism or 1 }}{{ "Enabled" if policy.auto_clean_before_archive else "Disabled" }}
+ {% else %} +

No storage policies configured yet.

+ {% endif %} +
+ +
+

Pool and cache alignment

+ + + + {% for pool in pools %} + + + + + + + + {% endfor %} + {% if not pools %} + + {% endif %} + +
PoolReplicationBackup OrderCache TargetRestore Path
{{ pool.name or pool.id }}{{ pool.replication_factor or 1 }}{{ status_text(pool.backup_order_mode or "sequential") }}{{ pool.cache_target_id or "—" }}{{ pool.restore_target_path or "—" }}
No pools defined.
+ {% if cache_drives %} + + + + {% for drive in cache_drives %} + + + + + + + + + {% endfor %} + +
Cache DriveRootCapacityMin FreeEvictionStatus
{{ drive.name or drive.id }}{{ drive.root_path }}{{ drive.max_bytes }}{{ drive.min_free_bytes }}{{ status_text(drive.eviction_policy or "after_verified") }}{{ "Enabled" if drive.enabled else "Disabled" }}
+ {% else %} +

No cache drives configured yet.

+ {% endif %} +
+ +
+

Current folder access mappings

+ {% if shares %} + + + + {% for share in shares %} + {% set pool_ids = share.pool_ids or [] %} + {% set folder_rules = share.folder_mappings or [] %} + + + + + + + + {% endfor %} + +
SharePathPoolsFolder RulesAccess
{{ share.name or "—" }}{{ share.path or "—" }}{{ (pool_ids|join(", ")) if pool_ids else "—" }} + {% if folder_rules %} +
    + {% for rule in folder_rules %} +
  • {{ rule.folder_path }} → {{ rule.pool_id }} ({{ status_text(rule.access_mode or "read_only") }})
    • + {% endfor %} +
+ {% else %} + — + {% endif %} + 		{{ "Read/Write" if share.writable else "Read-only" }}
+ {% else %} +

No share mappings configured yet.

+ {% endif %} +
+ {% elif section == "archive" %} +
+

Archive Dataset

+

Submit archive jobs from NAS scope; execution is routed through the selected device context.

+
+ + + +
+ +
+
+
+
+

Recent Archive Jobs

+ {% set jobs = section_data.archive_jobs or [] %} + {% if jobs %} + + + + {% for job in jobs %} + + + + + + + {% endfor %} + +
Job IDStatusCreatedError
{{ (job.id or "—")[:8] }}{% if job.id %}…{% endif %}{{ status_text(job.state) }}{{ job.created_at }}{{ job.error or "" }}
+ {% else %} +
+

No archive jobs

+

No archive jobs have been submitted yet.

+
+ {% endif %} +
+ {% elif section == "catalog" %} + {% set catalog = section_data.catalog_status or {} %} + {% set datasets = section_data.datasets or [] %} + {% set verify_snapshot = section_data.verify_snapshot or {} %} +
+ Total files: {{ catalog.total_file_records or 0 }} + Datasets: {{ catalog.total_datasets or 0 }} + Tape records: {{ catalog.total_cartridges or 0 }} + Mapped paths: {{ catalog.total_path_mappings or 0 }} +
+ {% if verify_snapshot %} +
+

Last verify run

+

+ Dataset {{ verify_snapshot.dataset_id }} verified + {{ verify_snapshot.files_verified or 0 }} files, + corrupt {{ verify_snapshot.files_corrupt or 0 }}, + checksums updated {{ verify_snapshot.files_updated or 0 }}. +

+
+ {% endif %} +
+
+

Dataset verification workbench

+ +
+

Run tape checksum verification per dataset after archive or import operations.

+ {% if datasets %} + + + + {% for item in datasets %} + + + + + + + + + + {% endfor %} + +
DatasetStatusPoolFilesBytesSourceAction
+ {{ item.name or item.id }} +
{{ item.id }}
+ 		{{ status_text(item.status or "pending") }}{{ item.pool_id or "—" }}{{ item.file_count or 0 }}{{ item.total_bytes or 0 }}{{ item.source_path or "—" }} + {% if item.id %} +
+ + +
+ {% else %} + — + {% endif %} +
+ {% else %} +
+

No datasets available for verification

+

Archive files first, then verify checksums from this workbench.

+
+ {% endif %} +
+
+

Catalog Files

+ {% set files = (section_data.catalog or {}).files or [] %} + {% if files %} + + + + {% for item in files %} + + + + + + + {% endfor %} + +
PathSizeBarcodeShard
{{ item.path }}{{ item.size_bytes }}{{ item.primary_barcode or "—" }}{{ item.shard_count or 1 }}
+ {% else %} +
+

Catalog is empty

+

Archive datasets to populate catalog records.

+
+ {% endif %} +
+ {% elif section == "restore" %} +
+

Restore Dataset

+
+ + + +
+ +
+
+
+
+

Recent Restore Jobs

+ {% set jobs = section_data.restore_jobs or [] %} + {% if jobs %} + + + + {% for job in jobs %} + + + + + + + {% endfor %} + +
Job IDStatusCreatedError
{{ (job.id or "—")[:8] }}{% if job.id %}…{% endif %}{{ status_text(job.state) }}{{ job.created_at }}{{ job.error or "" }}
+ {% else %} +
+

No restore jobs

+

Submit a restore request to repopulate this queue.

+
+ {% endif %} +
+ {% endif %} +{% endblock %} diff --git a/openblade/web_flask/templates/system/index.html b/openblade/web_flask/templates/system/index.html new file mode 100644 index 0000000..ae1de78 --- /dev/null +++ b/openblade/web_flask/templates/system/index.html @@ -0,0 +1,121 @@ +{% extends "base.html" %} +{% block title %}System · OpenBlade NAS{% endblock %} +{% block content %} + +
+
+

System Health

+

Control plane health, component readiness, and configuration posture.

+
+
+ +
+
+

Overall

+

+ {% set overall = system_status.overall or health.status or 'unknown' %} + {{ status_text(overall) }} +

+

Version: {{ config.version or "unknown" }}

+
+
+

Build & Runtime

+

Backend: {{ config.backend or "unknown" }}

+

Libraries: {{ config.library_count or 0 }}

+

Gateway: {{ "Enabled" if config.gateway_enabled else "Disabled" }}

+
+
+

Catalog

+

DB reachable: {{ "Yes" if catalog_status.db_reachable else "No" }}

+

Total files: {{ catalog_status.total_file_records or 0 }}

+

Total datasets: {{ catalog_status.total_datasets or 0 }}

+
+
+ +
+

Component Status

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ComponentStatus
CPU{{ status_text(system_status.cpu or "unknown") }}
Memory{{ status_text(system_status.memory or "unknown") }}
Disk{{ status_text(system_status.disk or "unknown") }}
Network{{ status_text(system_status.network or "unknown") }}
Services{{ status_text(system_status.services or "unknown") }}
+
+ +
+
+

Multi-user accounts

+ {% if users %} + + + + + + + + + + {% for user in users %} + + + + + + {% endfor %} + +
UserRolePassword Change
{{ user.name }}{{ aml_role_label(user.role) }}{{ "Required" if user.requirePasswordChange else "No" }}
+ {% else %} +

No user records are visible for this session.

+ {% endif %} +
+
+

Create user

+
+ + + + +
+ +
+
+
+
+{% endblock %} diff --git a/pyproject.toml b/pyproject.toml index d964fa5..e4e7172 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -28,6 +28,11 @@ dependencies = [ "httpx>=0.27", "pyotp>=2.9", "python-multipart>=0.0.9", + # openblade.web_flask: the NAS-first operator UI. It is a WSGI app SEPARATE from + # the FastAPI surface and reaches it over HTTP (web_flask.client.BackendClient), + # so Flask is a runtime dependency rather than a dev one: Dockerfile.web serves + # it under gunicorn, and a plain `pip install -e .` must be enough to import it. + "flask>=3.0", ] [project.optional-dependencies] diff --git a/tests/unit/test_web_flask_app.py b/tests/unit/test_web_flask_app.py new file mode 100644 index 0000000..82f1254 --- /dev/null +++ b/tests/unit/test_web_flask_app.py @@ -0,0 +1,1273 @@ +from __future__ import annotations + +from dataclasses import replace + +import pytest + +from openblade.web_flask.app import create_app +from openblade.web_flask.client import BackendClient, BackendError +from openblade.web_flask.models import Device + + +class FakeBackendClient(BackendClient): + def __init__(self) -> None: + super().__init__(base_url="http://backend") + self.devices = [ + Device( + id=1, + name="Quantum i3 A", + emulator_url="http://device-a:8010", + model="Scalar i3", + role="primary", + status="online", + enabled=True, + sort_order=1, + ) + ] + self.jobs = [ + { + "id": "job-1", + "state": "running", + "job_type": "archive", + "error": None, + "created_at": "2026-05-30T00:00:00Z", + "updated_at": "2026-05-30T00:01:00Z", + "library_id": 1, + } + ] + self.mounts = [ + { + "id": "mount-1", + "barcode": "ABC123L7", + "drive": "DRV001", + "state": "mounted", + "mountTime": "2026-05-30T00:01:30Z", + } + ] + self.inventory_status = {"state": "idle"} + self.import_status = {"state": "idle"} + self.export_status = {"state": "idle"} + self.last_move: dict[str, object] | None = None + self.last_export: dict[str, object] | None = None + self.last_restore: dict[str, object] | None = None + self.probe_calls: list[dict[str, object]] = [] + self.last_pool: dict[str, object] | None = None + self.last_share: dict[str, object] | None = None + self.last_magazine_action: tuple[str, str] | None = None + self.last_policy: dict[str, object] | None = None + self.last_cache_drive: dict[str, object] | None = None + self.last_source_stream: dict[str, object] | None = None + self.last_user: dict[str, object] | None = None + self.users = [ + {"name": "admin", "role": 0, "requirePasswordChange": False}, + {"name": "operator1", "role": 1, "requirePasswordChange": False}, + ] + self.policies = [ + { + "id": "balanced", + "name": "Balanced", + "policy_type": "balanced", + "default_ingest_mode": "cache_drive", + "copies_required": 1, + "allow_sharding": False, + "shard_size_bytes": None, + "max_parallelism": 1, + "auto_clean_before_archive": True, + } + ] + self.cache_drives = [ + { + "id": "cache-primary", + "name": "Primary Cache", + "root_path": "/openblade/cache", + "max_bytes": 107374182400, + "min_free_bytes": 5368709120, + "eviction_policy": "after_verified", + "enabled": True, + } + ] + self.source_stream = { + "enabled": True, + "require_source_online_for_entire_job": True, + "preflight_read_check": True, + "checksum_mode": "precompute_and_post_verify", + "max_retries": 3, + "fail_on_source_change": True, + "allow_partial_dataset_success": False, + } + self.volume_groups = [{"id": "vg-default", "name": "Default", "barcodes": ["ABC123L7"]}] + self.pools = [ + { + "id": "pool-1", + "name": "Primary Pool", + "replication_factor": 1, + "backup_order_mode": "sequential", + "access_mode": "read_only", + "volume_group_ids": ["vg-default"], + } + ] + self.shares = [ + { + "path": "/openblade/inbox", + "name": "inbox", + "share_type": "inbox", + "pool_ids": ["pool-1"], + "folder_mappings": [ + {"folder_path": "/inbox", "pool_id": "pool-1", "access_mode": "read_write"} + ], + "writable": True, + } + ] + self.magazines = [ + { + "id": "MAG-1", + "location": "left", + "status": "inserted", + "slotCount": 10, + "occupiedSlots": 2, + "tapes": ["ABC123L7"], + }, + ] + self.magazine_slots = { + "MAG-1": [ + { + "id": "slot-1", + "address": "1,1,1", + "state": "occupied", + "barcode": "ABC123L7", + "type": "magazine", + } + ] + } + self.datasets = [ + { + "id": "dataset-1", + "name": "finance-q1", + "pool_id": "pool-1", + "source_path": "/openblade/inbox/finance-q1", + "file_count": 2, + "total_bytes": 4096, + "status": "archived", + } + ] + self.last_dataset_verify_id: str | None = None + + def login(self, *, username: str, password: str) -> str: + if username == "admin" and password == "admin": + return "session-token" + raise BackendError(status_code=401, detail="invalid credentials") + + def list_devices(self, token: str) -> list[Device]: + assert token == "session-token" + return self.devices + + def get_device(self, token: str, device_id: int) -> Device | None: + assert token == "session-token" + for device in self.devices: + if device.id == device_id: + return device + return None + + def create_device(self, token: str, payload: dict[str, object]) -> Device: + assert token == "session-token" + next_device = replace( + self.devices[0], + id=2, + name=str(payload["name"]), + emulator_url=str(payload["emulator_url"]), + model=str(payload["model"]), + role=str(payload["role"]), + sort_order=int(payload["sort_order"]), + ) + self.devices.append(next_device) + return next_device + + def probe_device_endpoint( + self, + *, + connection_url: str, + username: str | None = None, + password: str | None = None, + ) -> dict[str, object]: + self.probe_calls.append( + {"connection_url": connection_url, "username": username, "password": password} + ) + if "invalid" in connection_url: + raise BackendError(status_code=400, detail="Unable to reach device health endpoint") + return {"reachable": True, "authenticated": bool(username)} + + def update_device(self, token: str, device_id: int, payload: dict[str, object]) -> Device: + assert token == "session-token" + for index, device in enumerate(self.devices): + if device.id == device_id: + updated = replace( + device, + name=str(payload["name"]), + emulator_url=str(payload["emulator_url"]), + role=str(payload["role"]), + enabled=bool(payload["enabled"]), + ) + self.devices[index] = updated + return updated + raise AssertionError("device not found") + + def list_jobs( + self, + token: str, + *, + library_id: int | None = None, + state: str | None = None, + job_type: str | None = None, + ) -> list[dict[str, object]]: + assert token == "session-token" + jobs: list[dict[str, object]] = self.jobs + if library_id is not None: + jobs = [item for item in jobs if int(item["library_id"]) == library_id] + if state is not None: + jobs = [item for item in jobs if str(item["state"]) == state] + if job_type is not None: + jobs = [item for item in jobs if str(item["job_type"]) == job_type] + return jobs + + def list_media( + self, token: str, *, library_id: int, limit: int = 100 + ) -> list[dict[str, object]]: + assert token == "session-token" + assert library_id == 1 + return [ + { + "barcode": "ABC123L7", + "type": "LTO7", + "state": "online", + "partition": "p1", + "slot": "1,1,1", + } + ] + + def get_file_versions(self, token: str, *, file_path: str) -> list[dict[str, object]]: + assert token == "session-token" + assert file_path + return [ + { + "file_path": file_path, + "tape_barcode": "ABC123L7", + "tape_slot": "1,1,1", + "checksum": "abc123", + } + ] + + def list_drives(self, token: str, *, library_id: int) -> list[dict[str, object]]: + assert token == "session-token" + assert library_id == 1 + return [ + {"serialNumber": "DRV001", "type": "LTO7", "state": "online", "loadedMedia": "ABC123L7"} + ] + + def list_partitions(self, token: str, *, library_id: int) -> list[dict[str, object]]: + assert token == "session-token" + assert library_id == 1 + return [{"name": "p1", "type": "classic", "state": "online", "slotCount": 24}] + + def trigger_inventory(self, token: str, *, library_id: int) -> dict[str, object]: + assert token == "session-token" + assert library_id == 1 + return {"message": "Inventory started"} + + def move_media_operation( + self, + token: str, + *, + library_id: int, + source: str, + destination: str, + barcode: str | None = None, + ) -> dict[str, object]: + assert token == "session-token" + assert library_id == 1 + self.last_move = {"source": source, "destination": destination, "barcode": barcode} + return {"resultCode": "0", "resultMsg": "Move queued"} + + def mount_media_operation( + self, + token: str, + *, + library_id: int, + barcode: str, + drive: str, + partition: str | None = None, + ) -> dict[str, object]: + assert token == "session-token" + assert library_id == 1 + assert barcode + assert drive + return {"resultCode": "0", "resultMsg": "Mount queued"} + + def unmount_media_operation( + self, + token: str, + *, + library_id: int, + barcode: str, + drive: str, + ) -> dict[str, object]: + assert token == "session-token" + assert library_id == 1 + assert barcode + assert drive + return {"resultCode": "0", "resultMsg": "Unmount queued"} + + def start_import_operation( + self, + token: str, + *, + library_id: int, + partition: str, + ie_station: str, + ) -> dict[str, object]: + assert token == "session-token" + assert library_id == 1 + assert partition + assert ie_station + return {"resultCode": "0", "resultMsg": "Import queued"} + + def start_export_operation( + self, + token: str, + *, + library_id: int, + barcodes: list[str], + ie_station: str, + ) -> dict[str, object]: + assert token == "session-token" + assert library_id == 1 + assert barcodes + assert ie_station + self.last_export = {"barcodes": barcodes, "ie_station": ie_station} + return {"resultCode": "0", "resultMsg": "Export queued"} + + def list_mounts(self, token: str, *, library_id: int) -> list[dict[str, object]]: + assert token == "session-token" + assert library_id == 1 + return self.mounts + + def list_ie_stations(self, token: str, *, library_id: int) -> list[dict[str, object]]: + assert token == "session-token" + assert library_id == 1 + return [{"id": "IE-1", "status": "closed"}] + + def get_inventory_status(self, token: str, *, library_id: int) -> dict[str, object]: + assert token == "session-token" + assert library_id == 1 + return self.inventory_status + + def get_import_status(self, token: str, *, library_id: int) -> dict[str, object]: + assert token == "session-token" + assert library_id == 1 + return self.import_status + + def get_export_status(self, token: str, *, library_id: int) -> dict[str, object]: + assert token == "session-token" + assert library_id == 1 + return self.export_status + + def list_media_pools(self, token: str, *, library_id: int) -> list[dict[str, object]]: + assert token == "session-token" + assert library_id == 1 + return [{"name": "default", "type": "archive"}] + + def list_nas_shares(self, token: str) -> list[dict[str, object]]: + assert token == "session-token" + return self.shares + + def list_nas_pools(self, token: str) -> list[dict[str, object]]: + assert token == "session-token" + return self.pools + + def list_aml_users(self, token: str) -> list[dict[str, object]]: + assert token == "session-token" + return self.users + + def create_aml_user(self, token: str, payload: dict[str, object]) -> dict[str, object]: + assert token == "session-token" + self.last_user = payload + record = { + "name": str(payload["name"]), + "role": int(payload["role"]), + "requirePasswordChange": False, + } + self.users.append(record) + return record + + def list_nas_policies(self, token: str) -> list[dict[str, object]]: + assert token == "session-token" + return self.policies + + def create_or_update_nas_policy( + self, token: str, payload: dict[str, object] + ) -> dict[str, object]: + assert token == "session-token" + self.last_policy = payload + self.policies = [payload] + return payload + + def list_cache_drives(self, token: str) -> list[dict[str, object]]: + assert token == "session-token" + return self.cache_drives + + def create_or_update_cache_drive( + self, token: str, payload: dict[str, object] + ) -> dict[str, object]: + assert token == "session-token" + self.last_cache_drive = payload + self.cache_drives = [payload] + return payload + + def get_source_stream_config(self, token: str) -> dict[str, object]: + assert token == "session-token" + return self.source_stream + + def update_source_stream_config( + self, token: str, payload: dict[str, object] + ) -> dict[str, object]: + assert token == "session-token" + self.last_source_stream = payload + self.source_stream = payload + return payload + + def list_volume_groups(self, token: str) -> list[dict[str, object]]: + assert token == "session-token" + return self.volume_groups + + def create_nas_pool(self, token: str, payload: dict[str, object]) -> dict[str, object]: + assert token == "session-token" + self.last_pool = payload + self.pools = [payload] + return payload + + def create_nas_share(self, token: str, payload: dict[str, object]) -> dict[str, object]: + assert token == "session-token" + self.last_share = payload + self.shares.append(payload) + return payload + + def list_magazines(self, token: str, *, library_id: int) -> list[dict[str, object]]: + assert token == "session-token" + assert library_id == 1 + return self.magazines + + def list_magazine_slots( + self, token: str, *, library_id: int, magazine_id: str + ) -> list[dict[str, object]]: + assert token == "session-token" + assert library_id == 1 + return self.magazine_slots.get(magazine_id, []) + + def eject_magazine(self, token: str, *, library_id: int, magazine_id: str) -> dict[str, object]: + assert token == "session-token" + assert library_id == 1 + self.last_magazine_action = ("eject", magazine_id) + return {"resultCode": "0"} + + def insert_magazine( + self, token: str, *, library_id: int, magazine_id: str + ) -> dict[str, object]: + assert token == "session-token" + assert library_id == 1 + self.last_magazine_action = ("insert", magazine_id) + return {"resultCode": "0"} + + def get_gateway_config(self, token: str) -> dict[str, object]: + assert token == "session-token" + return {"bind_host": "0.0.0.0", "bind_port": 2222, "status": "running"} + + def get_gateway_status(self, token: str) -> dict[str, object]: + assert token == "session-token" + return {"status": "running", "active_sessions": 1, "total_sessions": 3} + + def list_gateway_credentials(self, token: str) -> list[dict[str, object]]: + assert token == "session-token" + return [{"username": "ops-user", "enabled": True}] + + def get_event_summary(self, token: str, *, library_id: int) -> dict[str, object]: + assert token == "session-token" + assert library_id == 1 + return {"total": 3, "critical": 1, "warning": 1, "info": 1} + + def get_alert_summary(self, token: str, *, library_id: int) -> dict[str, object]: + assert token == "session-token" + assert library_id == 1 + return {"critical": 1, "warning": 0, "info": 1} + + def list_events( + self, + token: str, + *, + library_id: int, + severity: str | None = None, + limit: int = 50, + ) -> list[dict[str, object]]: + assert token == "session-token" + assert library_id == 1 + events = [ + { + "timestamp": "2026-05-30T00:00:00Z", + "severity": "critical", + "component": "drive", + "message": "Drive alert", + } + ] + if severity and severity != "all": + return [item for item in events if item["severity"] == severity] + return events[:limit] + + def get_system_status(self, token: str, *, library_id: int | None = None) -> dict[str, object]: + assert token == "session-token" + return { + "overall": "good", + "cpu": "good", + "memory": "warning", + "disk": "good", + "network": "good", + "services": "good", + } + + def get_health(self, token: str) -> dict[str, object]: + assert token == "session-token" + return {"status": "healthy"} + + def get_system_config(self, token: str) -> dict[str, object]: + assert token == "session-token" + return { + "version": "0.1.0", + "backend": "simulator", + "library_count": 1, + "gateway_enabled": True, + } + + def get_catalog_status(self, token: str) -> dict[str, object]: + assert token == "session-token" + return { + "db_reachable": True, + "total_file_records": 12, + "total_datasets": 2, + "total_cartridges": 4, + "total_path_mappings": 12, + "last_rebuild_status": "completed", + } + + def get_library_status(self, token: str, *, library_id: int) -> dict[str, object]: + assert token == "session-token" + assert library_id == 1 + return {"overall": "good"} + + def list_catalog_files(self, token: str, *, limit: int = 25) -> dict[str, object]: + assert token == "session-token" + return { + "files": [ + { + "path": "/vg/file1", + "size_bytes": 10, + "primary_barcode": "ABC123L7", + "shard_count": 1, + } + ], + "total": 1, + } + + def list_nas_datasets( + self, + token: str, + *, + status: str | None = None, + limit: int = 100, + ) -> list[dict[str, object]]: + assert token == "session-token" + datasets = self.datasets[:limit] + if status is not None: + datasets = [item for item in datasets if str(item.get("status")) == status] + return datasets + + def verify_nas_dataset(self, token: str, *, dataset_id: str) -> dict[str, object]: + assert token == "session-token" + self.last_dataset_verify_id = dataset_id + for dataset in self.datasets: + if str(dataset.get("id")) == dataset_id: + dataset["status"] = "verified" + return { + "dataset_id": dataset_id, + "files_verified": 2, + "files_corrupt": 0, + "files_updated": 1, + "checksums": {"sample.bin": "abc123"}, + } + + def create_archive_job( + self, token: str, *, source_path: str, volume_group: str + ) -> dict[str, object]: + assert token == "session-token" + assert source_path + assert volume_group + return {"job_id": "job-archive", "status": "pending"} + + def create_restore_job( + self, token: str, *, catalog_path: str, dest_path: str + ) -> dict[str, object]: + assert token == "session-token" + assert catalog_path + assert dest_path + self.last_restore = {"catalog_path": catalog_path, "destination_path": dest_path} + return {"job_id": "job-restore", "status": "pending"} + + +@pytest.fixture +def app_client(monkeypatch: pytest.MonkeyPatch): + monkeypatch.setenv("FLASK_SECRET_KEY", "test-secret-key") + app = create_app(client_factory=FakeBackendClient) + app.config["TESTING"] = True + return app.test_client() + + +def _backend_from_client(client) -> FakeBackendClient: + app = client.application + backend = app.extensions["backend_client"] + assert isinstance(backend, FakeBackendClient) + return backend + + +def _csrf(client) -> str: + with client.session_transaction() as sess: + return str(sess["_csrf_token"]) + + +def _login(client) -> None: + client.get("/login") + client.post( + "/login", + data={"username": "admin", "password": "admin", "csrf_token": _csrf(client)}, + follow_redirects=False, + ) + + +def test_requires_auth_for_devices_page(app_client) -> None: + response = app_client.get("/devices") + assert response.status_code == 302 + assert "/login" in response.headers["Location"] + + +def test_login_and_devices_page(app_client) -> None: + _login(app_client) + response = app_client.get("/devices") + assert response.status_code == 200 + body = response.get_data(as_text=True) + assert "Connected Devices" in body + assert "Fleet" not in body + assert "Manage" in body + assert "Reports" in body + assert "System" in body + + +def test_login_rejects_protocol_relative_next_redirect(app_client) -> None: + app_client.get("/login?next=//evil.example/path") + response = app_client.post( + "/login", + data={ + "username": "admin", + "password": "admin", + "next": "//evil.example/path", + "csrf_token": _csrf(app_client), + }, + follow_redirects=False, + ) + assert response.status_code == 302 + assert response.headers["Location"].endswith("/devices") + + +def test_login_rejects_control_chars_in_next_redirect(app_client) -> None: + app_client.get("/login") + response = app_client.post( + "/login", + data={ + "username": "admin", + "password": "admin", + "next": "/devices\x00//evil.example", + "csrf_token": _csrf(app_client), + }, + follow_redirects=False, + ) + assert response.status_code == 302 + assert response.headers["Location"].endswith("/devices") + + +def test_login_session_is_permanent(app_client) -> None: + _login(app_client) + with app_client.session_transaction() as sess: + assert sess.permanent is True + + +def test_logout_requires_post_and_csrf(app_client) -> None: + _login(app_client) + get_response = app_client.get("/logout") + assert get_response.status_code == 405 + post_response = app_client.post( + "/logout", data={"csrf_token": _csrf(app_client)}, follow_redirects=False + ) + assert post_response.status_code == 302 + assert post_response.headers["Location"].endswith("/login") + + +def test_login_rate_limit_blocks_after_repeated_failures(app_client) -> None: + app_client.get("/login") + for _ in range(8): + response = app_client.post( + "/login", + data={"username": "admin", "password": "bad-password", "csrf_token": _csrf(app_client)}, + follow_redirects=False, + ) + assert response.status_code == 401 + blocked = app_client.post( + "/login", + data={"username": "admin", "password": "bad-password", "csrf_token": _csrf(app_client)}, + follow_redirects=False, + ) + assert blocked.status_code == 429 + + +def test_login_rate_limit_ignores_forged_forwarded_for(app_client) -> None: + """A rotating ``X-Forwarded-For`` must not mint fresh rate-limit identities. + + The header is caller-controlled, so if ``_client_ip`` honoured it an attacker + would defeat the login lockout entirely by sending a new value each attempt. + Mirrors the rule documented on ``openblade.api.routes_assist.client_key``. + + Mutation check: restore the ``X-Forwarded-For`` branch in ``_client_ip`` and + this test fails — every attempt lands in a distinct bucket, so the ninth POST + returns 401 instead of 429. + """ + app_client.get("/login") + for attempt in range(8): + response = app_client.post( + "/login", + data={ + "username": "admin", + "password": "bad-password", + "csrf_token": _csrf(app_client), + }, + headers={"X-Forwarded-For": f"10.9.9.{attempt}"}, + follow_redirects=False, + ) + assert response.status_code == 401 + blocked = app_client.post( + "/login", + data={"username": "admin", "password": "bad-password", "csrf_token": _csrf(app_client)}, + headers={"X-Forwarded-For": "10.9.9.200"}, + follow_redirects=False, + ) + assert blocked.status_code == 429 + + +def test_register_device_requires_csrf(app_client) -> None: + _login(app_client) + response = app_client.post( + "/devices/register", + data={ + "name": "Quantum i3 B", + "connection_url": "http://device-b:8010", + "serial_number": "SN-2", + "model": "Scalar i3", + }, + ) + assert response.status_code == 400 + + +def test_register_device_flow(app_client) -> None: + _login(app_client) + app_client.get("/devices/register") + response = app_client.post( + "/devices/register", + data={ + "name": "Quantum i3 B", + "connection_url": "http://device-b:8010", + "serial_number": "SN-2", + "model": "Scalar i3", + "csrf_token": _csrf(app_client), + }, + follow_redirects=False, + ) + assert response.status_code == 302 + assert response.headers["Location"].endswith("/devices") + backend = _backend_from_client(app_client) + assert backend.probe_calls[-1]["connection_url"] == "http://device-b:8010" + + +def test_register_page_hides_storage_role_and_supports_probe_auth(app_client) -> None: + _login(app_client) + response = app_client.get("/devices/register") + assert response.status_code == 200 + body = response.get_data(as_text=True) + assert "Storage role" not in body + assert "Device Username (optional)" in body + assert "Device Password (optional)" in body + + +def test_devices_index_hides_role_metadata(app_client) -> None: + _login(app_client) + response = app_client.get("/devices") + assert response.status_code == 200 + body = response.get_data(as_text=True) + assert "Role:" not in body + + +def test_probe_endpoint_validates_auth_connectivity(app_client) -> None: + _login(app_client) + app_client.get("/devices/register") + response = app_client.post( + "/devices/probe", + data={ + "connection_url": "http://device-b:8010", + "device_username": "operator", + "device_password": "secret", + "csrf_token": _csrf(app_client), + }, + ) + assert response.status_code == 200 + payload = response.get_json() + assert payload is not None + assert payload["ok"] is True + assert payload["probe"]["reachable"] is True + assert payload["probe"]["authenticated"] is True + backend = _backend_from_client(app_client) + assert backend.probe_calls[-1]["username"] == "operator" + + +def test_manage_page_is_device_scoped(app_client) -> None: + _login(app_client) + response = app_client.get("/devices/1/manage", follow_redirects=True) + assert response.status_code == 200 + body = response.get_data(as_text=True) + assert "Inventory" in body + assert "Drives" in body + assert "Quantum i3 A" in body + + +def test_jobs_reports_and_system_pages_render(app_client) -> None: + _login(app_client) + jobs_response = app_client.get("/jobs") + reports_response = app_client.get("/reports") + system_response = app_client.get("/system") + assert jobs_response.status_code == 200 + assert reports_response.status_code == 200 + assert system_response.status_code == 200 + assert "Jobs" in jobs_response.get_data(as_text=True) + assert "Reports" in reports_response.get_data(as_text=True) + assert "System Health" in system_response.get_data(as_text=True) + + +def test_system_page_lists_users_and_creation_controls(app_client) -> None: + _login(app_client) + response = app_client.get("/system") + assert response.status_code == 200 + body = response.get_data(as_text=True) + assert "Multi-user accounts" in body + assert "admin" in body + assert "Create user" in body + + +def test_system_create_user_flow(app_client) -> None: + _login(app_client) + app_client.get("/system") + response = app_client.post( + "/system/users", + data={ + "username": "operator2", + "password": "operator-password", + "role": "1", + "csrf_token": _csrf(app_client), + }, + follow_redirects=False, + ) + assert response.status_code == 302 + assert response.headers["Location"].endswith("/system") + backend = _backend_from_client(app_client) + assert backend.last_user is not None + assert backend.last_user["name"] == "operator2" + assert backend.last_user["role"] == 1 + + +def test_storage_paths_reject_parent_traversal(app_client) -> None: + _login(app_client) + app_client.get("/storage/archive") + response = app_client.post( + "/storage/archive", + data={ + "source_path": "/data/../etc", + "volume_group": "default", + "csrf_token": _csrf(app_client), + }, + follow_redirects=False, + ) + assert response.status_code == 302 + assert response.headers["Location"].endswith("/storage/archive") + + +def test_storage_access_page_shows_windows_and_linux_mount_commands(app_client) -> None: + _login(app_client) + response = app_client.get("/storage/access") + assert response.status_code == 200 + body = response.get_data(as_text=True) + assert "Windows mapped drive" in body + assert "net use Z:" in body + assert "mount -t cifs" in body + assert "sftp -P 2222 ops-user@" in body + + +def test_storage_write_path_page_renders_controls(app_client) -> None: + _login(app_client) + response = app_client.get("/storage/write-path") + assert response.status_code == 200 + body = response.get_data(as_text=True) + assert "Policy for multi-tape writes" in body + assert "Cache drive staging" in body + assert "Streaming direct-to-tape" in body + assert "Folder-to-pool access mapping" in body + assert "Optimization guidance" in body + + +def test_storage_catalog_page_renders_dataset_verification_workbench(app_client) -> None: + _login(app_client) + response = app_client.get("/storage/catalog") + assert response.status_code == 200 + body = response.get_data(as_text=True) + assert "Dataset verification workbench" in body + assert "finance-q1" in body + assert "Verify now" in body + + +def test_storage_catalog_verify_dataset_flow(app_client) -> None: + _login(app_client) + app_client.get("/storage/catalog") + response = app_client.post( + "/storage/datasets/dataset-1/verify", + data={"csrf_token": _csrf(app_client)}, + follow_redirects=False, + ) + assert response.status_code == 302 + assert response.headers["Location"].endswith("/storage/catalog") + backend = _backend_from_client(app_client) + assert backend.last_dataset_verify_id == "dataset-1" + + catalog = app_client.get("/storage/catalog") + body = catalog.get_data(as_text=True) + assert "Last verify run" in body + assert "dataset-1" in body + + +def test_storage_write_path_actions_flow(app_client) -> None: + _login(app_client) + app_client.get("/storage/write-path") + policy_response = app_client.post( + "/storage/policies", + data={ + "policy_id": "critical-fast", + "name": "Critical Fast", + "policy_type": "critical_sequential", + "default_ingest_mode": "source_stream", + "copies_required": "2", + "max_parallelism": "4", + "shard_size_bytes": "1048576", + "shard_strategy": "round_robin", + "allow_sharding": "on", + "verify_before_archive": "on", + "verify_after_archive": "on", + "auto_clean_before_archive": "on", + "csrf_token": _csrf(app_client), + }, + follow_redirects=False, + ) + assert policy_response.status_code == 302 + backend = _backend_from_client(app_client) + assert backend.last_policy is not None + assert backend.last_policy["id"] == "critical-fast" + assert backend.last_policy["copies_required"] == 2 + assert backend.last_policy["shard_size_bytes"] == 1048576 + assert backend.last_policy["auto_clean_before_archive"] is True + + app_client.get("/storage/write-path") + cache_response = app_client.post( + "/storage/cache-drives", + data={ + "drive_id": "cache-b", + "name": "Cache B", + "root_path": "/openblade/cache/b", + "max_bytes": "1000000", + "min_free_bytes": "1000", + "retention_days": "7", + "stabilization_seconds": "3", + "eviction_policy": "after_verified", + "enabled": "on", + "verify_before_archive": "on", + "verify_after_archive": "on", + "csrf_token": _csrf(app_client), + }, + follow_redirects=False, + ) + assert cache_response.status_code == 302 + assert backend.last_cache_drive is not None + assert backend.last_cache_drive["id"] == "cache-b" + assert backend.last_cache_drive["root_path"] == "/openblade/cache/b" + + app_client.get("/storage/write-path") + source_response = app_client.post( + "/storage/source-stream", + data={ + "checksum_mode": "streaming", + "max_retries": "2", + "enabled": "on", + "require_source_online_for_entire_job": "on", + "preflight_read_check": "on", + "fail_on_source_change": "on", + "csrf_token": _csrf(app_client), + }, + follow_redirects=False, + ) + assert source_response.status_code == 302 + assert backend.last_source_stream is not None + assert backend.last_source_stream["checksum_mode"] == "streaming" + assert backend.last_source_stream["max_retries"] == 2 + + app_client.get("/storage/write-path") + share_response = app_client.post( + "/storage/shares", + data={ + "share_path": "/openblade/finance", + "share_name": "finance", + "share_type": "pool", + "default_policy_id": "critical-fast", + "pool_ids": "pool-1,pool-2", + "folder_mappings": "/finance/reports|pool-1|read_only\n/finance/ops|pool-2|read_write", + "writable": "on", + "csrf_token": _csrf(app_client), + }, + follow_redirects=False, + ) + assert share_response.status_code == 302 + assert backend.last_share is not None + assert backend.last_share["path"] == "/openblade/finance" + assert backend.last_share["pool_ids"] == ["pool-1", "pool-2"] + assert len(backend.last_share["folder_mappings"]) == 2 + + +def test_device_operations_actions_flow(app_client) -> None: + _login(app_client) + app_client.get("/devices/1/manage/operations") + response = app_client.post( + "/devices/1/operations/move", + data={ + "source": "1,1,1", + "destination": "DRV-1", + "barcode": "ABC123L7", + "csrf_token": _csrf(app_client), + }, + follow_redirects=False, + ) + assert response.status_code == 302 + assert response.headers["Location"].endswith("/devices/1/manage/operations") + backend = _backend_from_client(app_client) + assert backend.last_move == {"source": "1,1,1", "destination": "DRV-1", "barcode": "ABC123L7"} + + +def test_operations_section_includes_pool_magazine_and_search_ui(app_client) -> None: + _login(app_client) + response = app_client.get("/devices/1/manage/operations") + assert response.status_code == 200 + body = response.get_data(as_text=True) + assert "Pool Controls" in body + assert "Tape and Content Search" in body + assert "Magazine Control" in body + + +def test_create_pool_and_map_share_actions(app_client) -> None: + _login(app_client) + app_client.get("/devices/1/manage/operations") + pool_response = app_client.post( + "/devices/1/operations/pool", + data={ + "pool_id": "pool-2", + "pool_name": "Primary Pool", + "volume_group_ids": "vg-default", + "restore_target_path": "/openblade/restore", + "replication_factor": "2", + "backup_order_mode": "sequential", + "access_mode": "read_write", + "csrf_token": _csrf(app_client), + }, + follow_redirects=False, + ) + assert pool_response.status_code == 302 + backend = _backend_from_client(app_client) + assert backend.last_pool is not None + assert backend.last_pool["name"] == "Primary Pool" + assert backend.last_pool["replication_factor"] == 2 + + app_client.get("/devices/1/manage/operations") + share_response = app_client.post( + "/devices/1/operations/share", + data={ + "share_name": "poolshare", + "pool_id": "pool-1", + "writable": "on", + "csrf_token": _csrf(app_client), + }, + follow_redirects=False, + ) + assert share_response.status_code == 302 + assert backend.last_share is not None + assert backend.last_share["path"] == "/pools/pool-1" + assert backend.last_share["share_type"] == "pool" + + +def test_hydrate_restore_action_uses_latest_tape_copy(app_client) -> None: + _login(app_client) + app_client.get("/devices/1/manage/operations") + response = app_client.post( + "/devices/1/operations/restore", + data={ + "catalog_path": "/archive/file-a.bin", + "dest_path": "/restore", + "csrf_token": _csrf(app_client), + }, + follow_redirects=False, + ) + assert response.status_code == 302 + backend = _backend_from_client(app_client) + assert backend.last_restore is not None + assert backend.last_restore["destination_path"] == "/restore" + + +def test_magazine_eject_action_calls_backend(app_client) -> None: + _login(app_client) + app_client.get("/devices/1/manage/operations") + response = app_client.post( + "/devices/1/operations/magazine/MAG-1/eject", + data={"csrf_token": _csrf(app_client)}, + follow_redirects=False, + ) + assert response.status_code == 302 + backend = _backend_from_client(app_client) + assert backend.last_magazine_action == ("eject", "MAG-1") + + +def test_export_operation_validates_barcodes(app_client) -> None: + _login(app_client) + app_client.get("/devices/1/manage/operations") + response = app_client.post( + "/devices/1/operations/export", + data={ + "barcodes": "GOOD123 BAD*CODE", + "ie_station": "IE-1", + "csrf_token": _csrf(app_client), + }, + follow_redirects=False, + ) + assert response.status_code == 302 + backend = _backend_from_client(app_client) + assert backend.last_export is None + + +def test_jobs_sse_stream_returns_snapshot(app_client) -> None: + _login(app_client) + response = app_client.get("/events/jobs?once=1") + assert response.status_code == 200 + assert response.content_type.startswith("text/event-stream") + body = response.get_data(as_text=True) + assert "event: jobs.snapshot" in body + assert '"running": 1' in body + + +def test_devices_sse_stream_returns_snapshot(app_client) -> None: + _login(app_client) + response = app_client.get("/events/devices?once=1") + assert response.status_code == 200 + assert response.content_type.startswith("text/event-stream") + body = response.get_data(as_text=True) + assert "event: devices.snapshot" in body + assert "Quantum i3 A" in body + + +def test_security_headers_present(app_client) -> None: + _login(app_client) + response = app_client.get("/devices") + assert response.headers["X-Content-Type-Options"] == "nosniff" + assert "default-src 'self'" in response.headers["Content-Security-Policy"] + + +def test_headers_send_native_api_token_as_bearer_and_session_as_cookie() -> None: + """With native auth on, Authorization must carry the NATIVE token, not the session. + + openblade.api.api_auth gates the native surface on the static + OPENBLADE_API_TOKEN, while /aml/* authenticates per-user via + routes_aml_auth.require_auth, which checks the ``sessionID`` cookie before the + bearer. Sending the AML session token as the only bearer 401s every native + endpoint (verified against the live app: /jobs, /nas/policies, /api/libraries + and /volume-groups all returned 401). + + Mutation check: drop ``self.api_token or`` from BackendClient._headers and the + bearer assertion below fails. + """ + client = BackendClient(base_url="http://backend", api_token="native-tok") + headers = client._headers("aml-session-token") + assert headers["Authorization"] == "Bearer native-tok" + assert headers["Cookie"] == "sessionID=aml-session-token" + + +def test_headers_fall_back_to_session_bearer_when_no_native_token() -> None: + """No OPENBLADE_API_TOKEN configured => unchanged pre-existing behaviour. + + Native auth is disabled in that case, so the session token stays in + Authorization and nothing about the previous wiring changes. + """ + client = BackendClient(base_url="http://backend") + headers = client._headers("aml-session-token") + assert headers["Authorization"] == "Bearer aml-session-token" + assert headers["Cookie"] == "sessionID=aml-session-token" + + +def test_headers_omit_credentials_when_unauthenticated() -> None: + """The login call itself must not present a session cookie.""" + client = BackendClient(base_url="http://backend") + headers = client._headers() + assert "Authorization" not in headers + assert "Cookie" not in headers + + +def test_compose_service_token_default_matches_backend_default() -> None: + """docker-compose.yml repeats service_auth's dev token; keep them in step. + + web-flask must send the same service token `api` falls back to, or /aml/mount + and /aml/unmount 403 in the default dev stack. The literal is duplicated into + compose out of necessity (compose cannot import Python), so this test is what + stops the two copies drifting. + """ + from pathlib import Path + + from openblade.api.service_auth import _DEFAULT_SERVICE_TOKEN + + compose = Path(__file__).resolve().parents[2] / "docker-compose.yml" + text = compose.read_text(encoding="utf-8") + assert ( + f"OPENBLADE_SERVICE_TOKEN: ${{OPENBLADE_SERVICE_TOKEN:-{_DEFAULT_SERVICE_TOKEN}}}" in text + ) + + +def test_compose_keeps_react_web_service_alongside_flask() -> None: + """The Flask UI is additive. `web` (React) must not have been repurposed. + + The parked change rewired `web` itself to Dockerfile.web, which would have + retired the React SPA as a side effect while frontend-build-test still gated + it. Both surfaces are named in CLAUDE.md, so both stay. + """ + from pathlib import Path + + compose = Path(__file__).resolve().parents[2] / "docker-compose.yml" + text = compose.read_text(encoding="utf-8") + assert "context: ./frontend" in text + assert "dockerfile: Dockerfile.web" in text From e8cdbad06cf66b0fdca02986e5d2a80bb931ad4c Mon Sep 17 00:00:00 2001 From: Hermes Agent Date: Sat, 26 Sep 2026 06:12:23 +0000 Subject: [PATCH 13/54] build(web_flask): package the Flask NAS UI as an additive, hardened service Dockerfile.web and the compose wiring, from the same parked state. Three corrections to what was parked, each verified by running the built image rather than reasoning about it: - `-k gevent` was dead. gunicorn.workers.ggevent imports `packaging`, which neither gunicorn nor gevent declares, so the container died at start with ModuleNotFoundError while `docker compose build` stayed green. Installed explicitly. This is exactly the bug the parked CI job would have shipped: its only image check was a build. - The image baked OPENBLADE_ENV=development, silently disabling both hardening checks the code offers -- _secret_key()'s refusal to start without FLASK_SECRET_KEY, and SESSION_COOKIE_SECURE. Now production, with compose overriding it back to development for the local plain-HTTP stack. Verified: the image now exits with "FLASK_SECRET_KEY is required in production". - gunicorn ran as root. Now drops to the non-root openblade user, mirroring Dockerfile. Verified uid=999 for every process in the running container, which then serves /login 200 with X-Frame-Options: DENY. The compose change is additive where the parked one was destructive: it rewired `web` itself to Dockerfile.web, which would have retired the React SPA as a side effect while frontend-build-test still gated it. `web` stays React; the Flask UI is a new `web-flask` service behind a profile, following the `emulator` precedent, so `docker compose up` is byte-for-byte unchanged. Host port 5175: 5173 is React and 5174 is the standalone emulator UI, so both would have clashed under `make fleet-up`. Both services now declare OPENBLADE_API_TOKEN and OPENBLADE_SERVICE_TOKEN from one variable each so they cannot drift apart on either credential; empty is the documented "unset" for both, keeping default behaviour. The service token's dev default has to be repeated as a literal (compose cannot import Python) because /aml/mount and /aml/unmount require it, so a test asserts the two copies agree. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_013S1qbQs31YSox1azQChqs2 --- Dockerfile.web | 34 ++++++++++++++++++++++++++++++++++ docker-compose.yml | 41 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 75 insertions(+) create mode 100644 Dockerfile.web diff --git a/Dockerfile.web b/Dockerfile.web new file mode 100644 index 0000000..eb80963 --- /dev/null +++ b/Dockerfile.web @@ -0,0 +1,34 @@ +FROM python:3.12-slim + +WORKDIR /app + +COPY pyproject.toml README.md ./ +COPY openblade/ ./openblade/ + +RUN pip install --no-cache-dir -e . \ + && pip install --no-cache-dir gunicorn gevent packaging + +# packaging is explicit on purpose: gunicorn.workers.ggevent imports it, but +# neither gunicorn nor gevent declares it, so `-k gevent` died at container start +# with ModuleNotFoundError while `docker compose build` stayed green. Verified by +# running the image, which is the only way this class of bug shows up. + +# Non-root, mirroring Dockerfile. The Flask UI holds no state, so it needs no +# /data volume and therefore no chown entrypoint -- a plain USER switch suffices. +RUN groupadd -r openblade && useradd -r -g openblade -d /app openblade + +# production, NOT development. This image is the deployable artifact, and +# OPENBLADE_ENV gates two hardening checks in openblade.web_flask.app: _secret_key() +# refuses to start without FLASK_SECRET_KEY, and SESSION_COOKIE_SECURE defaults on. +# Baking "development" here silently disabled both for anyone deploying the image +# as-is. docker-compose.yml overrides it back to development for the local dev +# stack, where a per-boot random key and plain-HTTP cookies are what you want. +ENV OPENBLADE_ENV=production +ENV OPENBLADE_WEB_BACKEND_URL=http://api:8000 +ENV PYTHONUNBUFFERED=1 + +USER openblade + +EXPOSE 8080 + +CMD ["gunicorn", "-k", "gevent", "-w", "1", "-b", "0.0.0.0:8080", "openblade.web_flask.app:app"] diff --git a/docker-compose.yml b/docker-compose.yml index 00bc9eb..eb9f3b1 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -7,6 +7,12 @@ services: OPENBLADE_BACKEND: mock OPENBLADE_DB_URL: sqlite:////data/openblade.db OPENBLADE_EMULATOR_URLS: ${OPENBLADE_EMULATOR_URLS:-http://host.docker.internal:8010,http://host.docker.internal:8011,http://host.docker.internal:8012} + # Declared so `api` and `web-flask` cannot drift apart on either credential. + # Empty is the documented "unset" for both (api_auth: empty => native auth + # disabled; service_auth: empty => dev fallback token), so the default stack + # behaves exactly as before. Export either var and BOTH services pick it up. + OPENBLADE_API_TOKEN: ${OPENBLADE_API_TOKEN:-} + OPENBLADE_SERVICE_TOKEN: ${OPENBLADE_SERVICE_TOKEN:-} volumes: - openblade-data:/data ports: @@ -40,5 +46,40 @@ services: api: condition: service_healthy + # The Flask NAS operator UI (openblade/web_flask). ADDITIVE, not a replacement + # for `web` above: CLAUDE.md describes "React frontend + Flask NAS UI" as two + # surfaces, and frontend-build-test still gates the React one. Off by default + # via a profile, following `emulator`: `docker compose --profile web-flask up` + # runs it, and the default stack is byte-for-byte unchanged. Host port 5175 + # deliberately: 5173 is the React service and 5174 is the standalone emulator UI + # (deploy/emulator/docker-compose.standalone.yml), so both would clash under + # `make fleet-up`. + web-flask: + build: + context: . + dockerfile: Dockerfile.web + profiles: ["web-flask"] + environment: + # Relaxes Dockerfile.web's production default: the local stack is plain + # HTTP, so a per-boot random SECRET_KEY and non-Secure cookies are correct + # here. A real deployment keeps production and supplies FLASK_SECRET_KEY. + OPENBLADE_ENV: development + OPENBLADE_WEB_BACKEND_URL: http://api:8000 + # Both halves of the backend's auth model; both MUST match `api`. + # OPENBLADE_API_TOKEN is the native-surface bearer (empty => api has native + # auth off, so an empty bearer here is equally correct). + OPENBLADE_API_TOKEN: ${OPENBLADE_API_TOKEN:-} + # The controller header, required by /aml/mount and /aml/unmount, which this + # UI calls. Unlike the bearer it has no "off" state: when the var is unset + # api falls back to service_auth._DEFAULT_SERVICE_TOKEN, so the UI has to + # send that exact value or mount/unmount 403s in the dev stack. Naming the + # literal here keeps the two in step; exporting the var overrides both. + OPENBLADE_SERVICE_TOKEN: ${OPENBLADE_SERVICE_TOKEN:-openblade-controller-dev-token-do-not-expose} + ports: + - "5175:8080" + depends_on: + api: + condition: service_healthy + volumes: openblade-data: From 9e70476346ddf1773bf81e36b9d00b830df33b12 Mon Sep 17 00:00:00 2001 From: Hermes Agent Date: Sat, 26 Sep 2026 06:12:53 +0000 Subject: [PATCH 14/54] ci: add web-flask-smoke, and document the Flask NAS UI Resurrects the parked ci.yml job with its path filter corrected. The parked filter included 'pyproject.toml', so it fired on every dependency bump; the `backend` filter already carries that file, so a pyproject change still gets ruff + mypy + the whole tests/unit run (which imports this package) and adding it here only tacked a docker build onto Dependabot traffic. 'docker-compose.yml' IS kept, because it defines the service this job builds. The job's unique value, given the existing layering: backend-lint already runs ruff tree-wide and backend-tests already runs tests/unit, but `mypy openblade` in backend-typecheck must stay advisory because of ~300 pre-existing errors in the god-file modules. openblade/web_flask is strict-clean today, so this job type-checks it BLOCKING and keeps it out of that backlog. It is also the only thing in CI that builds Dockerfile.web -- named explicitly, since a bare `docker compose build` skips profiled services. Registered in ci-gate's needs and its check list, without which a failure would not have blocked anything. README: adds the Flask UI section, its configuration table (including that OPENBLADE_API_TOKEN must match the backend), and the web-flask-smoke row in the layered-CI table. Deliberately NOT taking two of the parked README hunks: the one deleting the `cd frontend && npm ...` quickstart line, since the React frontend is not being retired and the CI table still lists frontend-build-test; and the reword of goal 4, since landing a UI is no reason to restate the project's goals or to drop "without out-of-scope feature leakage". Verified: workflow YAML parses, `yamllint -c .yamllint` passes, and `gen_wiki_reference.py --check` reports the generated pages up to date. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_013S1qbQs31YSox1azQChqs2 --- .github/workflows/ci.yml | 40 ++++++++++++++++++++++++++++++++++++++++ README.md | 36 +++++++++++++++++++++++++++++++++++- 2 files changed, 75 insertions(+), 1 deletion(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index a4c1130..ea74395 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -22,6 +22,7 @@ jobs: api: ${{ steps.filter.outputs.api }} simulator: ${{ steps.filter.outputs.simulator }} frontend: ${{ steps.filter.outputs.frontend }} + webflask: ${{ steps.filter.outputs.webflask }} cicd: ${{ steps.filter.outputs.cicd }} wiki: ${{ steps.filter.outputs.wiki }} steps: @@ -65,6 +66,17 @@ jobs: - 'tools/emulator_spec/**' frontend: - 'frontend/**' + # The Flask NAS UI. Deliberately NOT including 'pyproject.toml' here: + # the `backend` filter already carries it, so a dependency bump + # (including every Dependabot PR) still gets ruff + mypy + the whole + # tests/unit run, which imports this package. Adding it here would + # only tack a docker build onto that traffic. 'docker-compose.yml' IS + # included because it defines the web-flask service this job builds. + webflask: + - 'openblade/web_flask/**' + - 'tests/unit/test_web_flask_app.py' + - 'Dockerfile.web' + - 'docker-compose.yml' cicd: - '.github/workflows/**' # docs/wiki/reference/{cli,api}.md are GENERATED from the Typer CLI @@ -213,6 +225,32 @@ jobs: - run: npm run build - run: npm run test -- --run + web-flask-smoke: + name: web-flask-smoke + needs: changes + if: ${{ github.event_name == 'workflow_dispatch' || needs.changes.outputs.webflask == 'true' }} + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v5 + - uses: actions/setup-python@v5 + with: + python-version: "3.12" + cache: "pip" + - run: pip install -e '.[dev]' + # BLOCKING mypy, scoped to this package alone. `mypy openblade` in + # backend-typecheck has to stay advisory because of ~300 pre-existing errors + # in the god-file modules; openblade/web_flask is clean under strict mode + # today, and this job is what keeps it that way instead of letting it join + # the backlog. + - run: mypy openblade/web_flask + # Redundant with backend-tests (tests/unit covers this file) but kept so the + # job is a complete gate on its own for a web_flask-only change. Seconds. + - run: pytest tests/unit/test_web_flask_app.py -q --tb=short + # The only thing in CI that builds this image. web-flask sits behind a + # compose profile, so name it explicitly — a bare `docker compose build` + # skips profiled services. + - run: docker compose build web-flask + wiki-reference: name: wiki-reference needs: changes @@ -270,6 +308,7 @@ jobs: - api-aml-integration - i3-smoke - frontend-build-test + - web-flask-smoke - cicd-workflow-validate - wiki-reference if: ${{ always() }} @@ -288,6 +327,7 @@ jobs: check api-aml-integration "${{ needs.api-aml-integration.result }}" check i3-smoke "${{ needs.i3-smoke.result }}" check frontend-build-test "${{ needs.frontend-build-test.result }}" + check web-flask-smoke "${{ needs.web-flask-smoke.result }}" check cicd-workflow-validate "${{ needs.cicd-workflow-validate.result }}" # Blocking on purpose: the fix is one command # (`python3 tools/gen_wiki_reference.py`) and a stale generated page diff --git a/README.md b/README.md index 1f09304..fc871b0 100644 --- a/README.md +++ b/README.md @@ -25,7 +25,7 @@ OpenBlade CI/CD is intentionally split by layer so each change runs only relevan | --- | --- | --- | | API + backend domain | `CI`: `backend-lint`, `backend-typecheck`, `backend-tests`, `api-aml-integration` | `openblade/**/*.py`, AML integration tests, backend config | | Simulator/emulator parity | `CI`: `i3-smoke`; `i3-emulator-compliance`; `emulator-change-gates` | simulator, AML routes, emulator contract/tools, i3 tests, compose/runtime wiring | -| Frontend/UI | `CI`: `frontend-build-test` | `frontend/**` | +| Frontend/UI | `CI`: `frontend-build-test` (React SPA), `web-flask-smoke` (Flask NAS UI) | `frontend/**`; `openblade/web_flask/**`, `Dockerfile.web`, `docker-compose.yml` | | CI/CD policy layer | `CI`: `cicd-workflow-validate` | `.github/workflows/**` | This keeps checks up to date and targeted while preserving full coverage on workflow dispatch and on emulator-specific workflows. @@ -39,6 +39,8 @@ openblade inventory uvicorn openblade.api.main:app --reload # Flask-style WSGI deployment option (same API behavior): gunicorn openblade.api.wsgi:application +# Flask NAS operator UI (a separate app that talks to the API over HTTP): +FLASK_SECRET_KEY=dev-only gunicorn -k gevent -w 1 -b 0.0.0.0:5175 openblade.web_flask.app:app ``` ## Multi-Library Setup @@ -60,6 +62,38 @@ gunicorn openblade.api.wsgi:application - Override UI proxy targets with `EMULATOR_UI_TARGET_LIBRARY{1,2,3}_URL` in the standalone env file - Override OpenBlade controller routing with `OPENBLADE_EMULATOR_URLS` when targeting the standalone emulator endpoints +## Flask NAS operator UI +`openblade/web_flask/` is a NAS-first operator console. It is a **separate WSGI +app**, not a second implementation: it owns no domain logic and reaches the +FastAPI control plane over HTTP through `web_flask.client.BackendClient`. It sits +alongside the React SPA in `frontend/` rather than replacing it — both surfaces +are named in `CLAUDE.md`, and each has its own CI job. + +Run it with `docker compose --profile web-flask up` (http://localhost:5175); it is +profile-gated so the default stack is unchanged. + +Surfaces, each backed by existing API endpoints: +- **Storage → Write Path** — replication/sharding/cache-drive/ingest policy + (`/nas/policies`, `/nas/cache-drives`, `/nas/source-stream`), and folder-to-pool + share mappings with per-folder access modes (`/nas/shares`) +- **Storage → Catalog** — dataset table with per-dataset checksum verification + (`/nas/datasets`, `/nas/datasets/{dataset_id}/verify`) +- **Storage → Archive / Restore** — archive and restore submission (`/archive/`, + `/restore/`) +- **Devices** — register/probe libraries and drive inventory, mount/unmount, move, + import/export, and magazine eject/insert against `/api/libraries` and `/aml/*` +- **System** — AML users and role assignments (`/aml/users`) + +### Configuration +| Variable | Purpose | +| --- | --- | +| `FLASK_SECRET_KEY` | Session signing key. **Required** when `OPENBLADE_ENV=production` (the `Dockerfile.web` default) — the app refuses to start without it rather than minting a throwaway key that would invalidate every session on restart. | +| `OPENBLADE_WEB_BACKEND_URL` | Base URL of the FastAPI control plane. | +| `OPENBLADE_API_TOKEN` | Must match the backend's value. The backend gates its native surface (`/api`, `/jobs`, `/nas`, `/catalog`, `/status`, `/system`, `/volume-groups`, `/archive`, `/restore`) on this bearer, while `/aml/*` uses the per-user session this UI obtains at login. Omit it only where the backend also leaves native auth off. | +| `OPENBLADE_SERVICE_TOKEN` | Must match the backend's value; `/aml/mount` and `/aml/unmount` require it. | +| `OPENBLADE_WEB_SECURE_COOKIES` | Forces `Secure` session cookies; defaults on under `OPENBLADE_ENV=production`. | +| `OPENBLADE_WEB_ALLOW_UNSAFE_DEVICE_TARGETS` | Development escape hatch: permits loopback/link-local/metadata hosts as device connection URLs, which are refused by default. Leave unset in production. | + ## Safety defaults - Mock backend is the default - Real hardware requires `OPENBLADE_BACKEND=real` and `OPENBLADE_REAL_HARDWARE_ENABLED=true` From be7f715cdb431e3a5cc3b3e143805725b75aac37 Mon Sep 17 00:00:00 2001 From: Hermes Agent Date: Sat, 26 Sep 2026 06:16:19 +0000 Subject: [PATCH 15/54] docs: record the final disposition of the parked Flask NAS / parity work Ends the parked state opened on 2026-09-11 with a per-component decision, and marks the old note superseded so its now-obsolete "How to recover" section cannot mislead. The headline finding is that two months moved the ground under the parked note: the NAS/AML endpoints the Flask UI needed have all landed (46 present, 0 missing against app.openapi()), and the parity generator plus its artifacts landed independently in ce89749 -- master's generator is the same file, only ruff formatted. So the parity half retires at zero loss rather than needing the T3 emulator_contract review the old note anticipated. Includes the exact cleanup commands for the integrator, deliberately NOT run here: dropping a stash is irreversible and belongs to whoever owns that working tree. They carry two cautions -- the parity files look untracked in the main checkout only because it sits 36 commits behind origin/master, and ~45 unrelated files from the same abandoned period are explicitly out of scope. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_013S1qbQs31YSox1azQChqs2 --- .../2026-09-11-parked-flask-nas-ci-readme.md | 8 +- .../2026-09-26-flask-nas-disposition.md | 280 ++++++++++++++++++ 2 files changed, 287 insertions(+), 1 deletion(-) create mode 100644 docs/decisions/2026-09-26-flask-nas-disposition.md diff --git a/docs/decisions/2026-09-11-parked-flask-nas-ci-readme.md b/docs/decisions/2026-09-11-parked-flask-nas-ci-readme.md index 0db7393..77cfae8 100644 --- a/docs/decisions/2026-09-11-parked-flask-nas-ci-readme.md +++ b/docs/decisions/2026-09-11-parked-flask-nas-ci-readme.md @@ -1,7 +1,13 @@ # Parked: CI + README tail of the Flask NAS UI and iBlade parity-coverage work **Date:** 2026-09-11 -**Status:** parked in `git stash` (not lost, not landed) +**Status:** ⚠️ SUPERSEDED 2026-09-26 — the parked state has been resolved. See +[`2026-09-26-flask-nas-disposition.md`](2026-09-26-flask-nas-disposition.md) for +the final per-component disposition and the stash cleanup commands. The "How to +recover" section below is **obsolete**: the Flask UI, `Dockerfile.web` and the +`web-flask-smoke` job were landed (with corrections), and the parity generator +plus its artifacts turned out to have already landed on master in `ce89749`. +Read the rest of this file as the historical record of *why* it was parked. **Origin:** abandoned session of 2026-08-22, checkpoint `caca9241` (see `.claude/RESUME.md`) ## Why this note exists diff --git a/docs/decisions/2026-09-26-flask-nas-disposition.md b/docs/decisions/2026-09-26-flask-nas-disposition.md new file mode 100644 index 0000000..4ac3174 --- /dev/null +++ b/docs/decisions/2026-09-26-flask-nas-disposition.md @@ -0,0 +1,280 @@ +# Disposition: the parked Flask NAS UI and iBlade parity-coverage work + +**Date:** 2026-09-26 +**Status:** decided — the parked state is ENDED +**Supersedes:** [`2026-09-11-parked-flask-nas-ci-readme.md`](2026-09-11-parked-flask-nas-ci-readme.md) +**Branch:** `feat/flask-nas-disposition` + +The 2026-09-11 note parked four files in `git stash` plus a set of untracked +prerequisites, because they "could be neither safely landed nor honestly +discarded". This note closes that out. Each component was evaluated against +current `master` and decided on its own. + +## Summary + +| Component | Disposition | Why | +| --- | --- | --- | +| `openblade/web_flask/` (+ `tests/unit/test_web_flask_app.py`) | **LANDED** | All 46 endpoints it consumes now exist on master; it needed no functional port | +| `Dockerfile.web` | **LANDED, corrected** | Shipped a dead gevent worker, a root process, and dev-mode hardening | +| `docker-compose.yml` `web` rewire | **RETIRED**, replaced | The parked version retired the React SPA as a side effect | +| `ci.yml` `web-flask-smoke` job | **LANDED, filter corrected** | Its blocker (untracked inputs) is gone; the filter was over-broad | +| `tools/emulator_spec/generate_iblade_parity_coverage.py` | **RETIRED — already on master** | Landed independently in `ce89749`; master's copy is the formatted one | +| `emulator_contract/openblade_iblade_*parity*.{json,md}` | **RETIRED — already on master** | Byte-identical to the tracked files | +| `emulator-change-gates.yml` / `i3-emulator-compliance.yml` parity steps | **RETIRED (deferred)** | Out of scope here; see "Deferred" below | +| README parity/NAS hunks | **PARTIALLY LANDED** | Two hunks were wrong and were dropped | + +## What changed since the work was parked + +The parked note's central blocker was that the Flask UI called NAS/AML endpoints +that existed only in the abandoned session's dirty tree. **That is no longer +true.** Extracting every path from `BackendClient` and diffing it against +`app.openapi()` (862 paths on master) gives **46 present, 0 missing**, with +matching methods. The backend surface landed independently in the intervening +commits, chiefly `bd4f3b1`. + +Consequently the package needed no functional port at all: + +- `pytest tests/unit/test_web_flask_app.py` — **31/31 passed unchanged** +- `mypy openblade/web_flask` (strict) — clean, 4 files +- `ruff check` — clean + +The only mechanical work was `ruff format` drift, because the repo's formatter +baseline moved after the work was parked (`2026-09-11-pin-ruff-toolchain.md`). +The parity generator had drifted the *same* way, which is the tell that master's +copy is the descendant of the parked one and not a divergent reimplementation. + +## Per-component rationale + +### LANDED: `openblade/web_flask/` + +Kept because it is not a duplicate of the React SPA. It is a separate WSGI app +that owns **no** domain logic and reaches the FastAPI control plane over HTTP +through `web_flask.client.BackendClient`, which is exactly the surface +relationship `CLAUDE.md` describes with "React frontend + Flask NAS UI". Its +content is NAS-appliance-shaped (write-path policy, cache-drive staging, +source-stream profiles, folder-to-pool share mappings, dataset checksum +verification) rather than a second rendering of the SPA's pages. + +Two real auth defects were found and fixed, both by probing the running backend +rather than by reading it: + +1. **The native surface was entirely broken under native auth.** + `openblade/api/api_auth.py` gates the OpenBlade-native surface on the static + `OPENBLADE_API_TOKEN` bearer, while `/aml/*` authenticates per-user through + `routes_aml_auth.require_auth`. The client sent its AML *session* token as the + only bearer, so with `OPENBLADE_API_TOKEN` set, **all 21 native endpoints it + uses returned 401** — the whole Storage/Jobs/Reports/System surface. Verified + against a live uvicorn instance: `/jobs/`, `/nas/policies`, `/api/libraries` + and `/volume-groups/` all 401, while `/aml/*` worked. + + Fixed in one place. `require_auth` checks the `sessionID` cookie *before* the + bearer, so the session token now travels as that cookie and `Authorization` + is free for the native token. With no native token configured it falls back to + the previous behaviour, so nothing changes for deployments that leave native + auth off. + +2. **The login lockout was bypassable.** `_client_ip()` honoured the + caller-controlled `X-Forwarded-For`, so rotating the header minted a fresh + rate-limit bucket per attempt and defeated `_is_login_blocked` entirely. Now + peer-address only, matching the rule already documented on + `openblade.api.routes_assist.client_key`. + +Both fixes carry **mutation-checked** regression tests: restoring either +vulnerable line makes the new test fail (verified by doing it). Note that the +pre-existing `test_login_rate_limit_blocks_after_repeated_failures` passed +against the vulnerable code — it was vacuous for the forged-header attack, which +is precisely the failure mode the mutation rule exists to catch. + +`flask` moved into the runtime dependencies: `Dockerfile.web` serves this under +gunicorn, so `pip install -e .` has to be enough to import it. + +### LANDED, corrected: `Dockerfile.web` + +Three defects, all found by *running* the built image. The parked CI job's only +image check was `docker compose build`, which passes on all three: + +- **`-k gevent` was dead.** `gunicorn.workers.ggevent` imports `packaging`, which + neither gunicorn nor gevent declares. The container died at start with + `ModuleNotFoundError: No module named 'packaging'`. +- **`ENV OPENBLADE_ENV=development` was baked into the deployable artifact**, + silently disabling both hardening checks the code offers: `_secret_key()`'s + refusal to start without `FLASK_SECRET_KEY`, and `SESSION_COOKIE_SECURE`. Now + `production`, with compose overriding it back for the local plain-HTTP stack. +- **gunicorn ran as root**, against the repo's own convention in `Dockerfile`. + +Verified on the rebuilt image: it exits with "FLASK_SECRET_KEY is required in +production" when the key is absent; with the key it serves `/login` 200 with +`X-Frame-Options: DENY`, and every process reports `uid=999`. + +### RETIRED and replaced: the `docker-compose.yml` `web` rewire + +The parked hunk pointed the existing `web` service at `Dockerfile.web`, i.e. it +**replaced the React SPA with the Flask UI**. That is a bigger decision than +"land the parked tail", it contradicts `CLAUDE.md`'s two-surface description, and +it would have left `frontend-build-test` gating a service nothing deployed. The +parked note had already flagged the matching README deletion as suspect. + +Replaced with an **additive** `web-flask` service behind a compose profile, +following the `emulator` precedent in the same file, so `docker compose up` is +byte-for-byte unchanged. Host port **5175**: 5173 is the React service and 5174 +is the standalone emulator UI, so the parked choice of 5173 *and* an obvious +alternative of 5174 would both have clashed under `make fleet-up`. + +Both services now take `OPENBLADE_API_TOKEN` and `OPENBLADE_SERVICE_TOKEN` from +one variable each so they cannot drift apart on either credential. Empty is the +documented "unset" for both, so default behaviour is unchanged. The service +token's dev default must be repeated as a literal in compose (it cannot import +Python) because `/aml/mount` and `/aml/unmount` require it — a test asserts the +two copies agree. + +### LANDED, filter corrected: `web-flask-smoke` + +The parked filter included `pyproject.toml`, so it fired on every dependency +bump. The `backend` filter already carries that file, so a pyproject change still +gets ruff + mypy + the whole `tests/unit` run (which imports this package); +adding it here only tacked a docker build onto Dependabot traffic. +`docker-compose.yml` is kept, because it defines the service the job builds. + +Given the existing layering, the job's unique value is narrow but real: +`backend-lint` already runs ruff tree-wide and `backend-tests` already runs +`tests/unit`, but `mypy openblade` in `backend-typecheck` must stay +`continue-on-error` because of ~300 pre-existing errors in the god-file modules. +`openblade/web_flask` is strict-clean today, so this job type-checks it +**blocking** and keeps it out of that backlog. It is also the only thing in CI +that builds `Dockerfile.web`. Registered in `ci-gate`'s `needs` *and* its `check` +list — without the latter a failure would not have blocked anything. + +### RETIRED (zero loss): the parity generator and artifacts + +**These already landed on master** in `ce89749` ("Add iBlade Rev A parity +tracker"). The untracked copies are redundant: + +- `openblade_iblade_rev_a_parity.json`, `openblade_iblade_parity_coverage.json` + and `.md` are **byte-identical** to the tracked files. +- `generate_iblade_parity_coverage.py` differs only in line wrapping; master has + the `ruff format`ed version, i.e. master's copy is strictly the better one. + +Confirmed live: `python3 tools/emulator_spec/generate_iblade_parity_coverage.py +--allow-partial` exits 0 (`implemented=12, partial=4, missing=1`) and regenerates +the tracked artifacts with **no** diff. The parked note's recorded +`FileNotFoundError` is therefore stale, and the T3 "commits new +`emulator_contract/**` artifacts" concern no longer applies — that already +happened through the normal path. + +### Deferred, not landed: the parity steps in the two emulator workflows + +The generator step and artifact uploads for `emulator-change-gates.yml` and +`i3-emulator-compliance.yml` are **not** present on master — only the generator +itself landed. They are deliberately out of scope for this change, which was +scoped to touch `.github/workflows/` only for `web-flask-smoke`. + +Their original blocker is gone (the generator runs green and its inputs are +tracked), so this is now an ordinary small change rather than a parked one: add +`generate_iblade_parity_coverage.py --allow-partial` plus the two artifact paths +to those workflows. Worth doing; it just is not this change. **Nothing needs to +be recovered from the stash to do it** — the hunks are reproduced in the parked +note's own diff table and are four lines of YAML. + +### Partially landed: README + +Taken: the `web-flask-smoke` row in the layered-CI table, and a Flask NAS UI +section — rewritten rather than copied, with a configuration table that documents +the `OPENBLADE_API_TOKEN` lockstep requirement the parked version did not know +about. Every endpoint and Flask route named in it was verified to exist. + +**Not taken**, both deliberately: + +- The deletion of the `cd frontend && npm install && npm run test && npm run + build` quickstart line. The React frontend is not being retired and CI still + gates it. (The parked note flagged this itself.) +- The reword of goal 4 from "fleet and library workflows without out-of-scope + feature leakage" to "NAS-first operations with device-scoped deep controls". + Landing a UI is no reason to restate the project's goals, and the reword drops + an explicit scope-discipline clause. + +## Verification + +| Gate | Result | +| --- | --- | +| `pytest tests/unit/test_web_flask_app.py` | 37 passed (31 inherited + 6 added) | +| `mypy openblade/web_flask` (strict) | clean, 4 files | +| `ruff check .` | clean, tree-wide | +| `ruff format --check .` | clean, 352 files | +| `python3 tools/gen_wiki_reference.py --check` | up to date | +| Workflow YAML parse + `yamllint -c .yamllint` | pass | +| `docker compose build web-flask` | image built | +| Built image, no `FLASK_SECRET_KEY` | fails closed as designed | +| Built image, with key | `/login` 200, `XFO: DENY`, all processes `uid=999` | +| Live-backend auth probe, native auth ON | 7/7 native + AML endpoints OK (4/4 native were 401 before) | +| Mutation check, both new guards | each new test fails when its guard is removed | + +Not run: the full `pytest -m 'not real_hardware'` suite and the i3/emulator +gates. Nothing here touches `/aml/*` routes, `aml_state.py`, `emulator_contract/**`, +the safety gates, or the catalog schema; the backend is unmodified. + +## Cleanup for the integrator + +Everything worth keeping is now committed on this branch, and the rest is +verified-redundant. Once this branch is merged, the main checkout's parked state +can be dropped. **These commands were deliberately NOT run as part of this +change** — dropping a stash is irreversible and belongs to whoever owns that +working tree. + +Run from the main checkout (`/root/OpenBlade`), **after** confirming +`git stash list` still shows the same entry at index 0: + +```bash +# 1. Confirm you are dropping the right thing (message must start "PARKED 2026-09-11") +git -C /root/OpenBlade stash list | head -1 + +# 2. Drop the parked CI+README tail. Its landable content is in this branch; +# the parity-workflow hunks are reproduced in the 2026-09-11 note's diff table. +git -C /root/OpenBlade stash drop stash@{0} + +# 3. Remove the untracked prerequisites, now redundant. The parity files are +# byte-identical to the tracked ones; web_flask/ and Dockerfile.web are +# superseded by the corrected versions on this branch. +rm -rf /root/OpenBlade/openblade/web_flask +rm -f /root/OpenBlade/Dockerfile.web +rm -f /root/OpenBlade/tests/unit/test_web_flask_app.py +rm -f /root/OpenBlade/tools/emulator_spec/generate_iblade_parity_coverage.py +rm -f /root/OpenBlade/openblade/emulator_contract/openblade_iblade_parity_coverage.json +rm -f /root/OpenBlade/openblade/emulator_contract/openblade_iblade_parity_coverage.md +rm -f /root/OpenBlade/openblade/emulator_contract/openblade_iblade_rev_a_parity.json +``` + +Two cautions on step 3: + +- `generate_iblade_parity_coverage.py` and the three `emulator_contract` files are + **tracked on master**. They appear untracked in the main checkout only because + that checkout sits at `f8a4ed1`, **36 commits behind `origin/master`** — + `ce89749`, which added them, is in that gap. **The simplest and safest route is + to skip the last four `rm`s entirely and let `git pull` reconcile them**, since + the working copies are byte-identical to the tracked ones. If they are deleted + first, restore with: + ```bash + git -C /root/OpenBlade checkout -- \ + tools/emulator_spec/generate_iblade_parity_coverage.py \ + openblade/emulator_contract/ + ``` +- The main checkout has **~45 other modified/untracked files** from the same + abandoned period that are outside this disposition's scope (`openblade/api/*` + modifications, `openblade/hardware/scalar_http/`, `scripts/mvp_e2e_demo.py`, + `tests/e2e/`, `deploy/emulator/observability/`, and stale `.pyc` files). Do not + read the commands above as clearance to clean those — several have since landed + on master in different form and need their own comparison. + +## Noticed, not fixed + +- `.github/workflows/ci.yml`: the comment above `ci-gate` says "backend-lint + blocks", but the `check` list treats it as advisory and only echoes it + alongside `backend-typecheck`. One of the two is wrong. Pre-existing. +- `openblade/web_flask/app.py` is 1,885 lines with a ~1,350-line `create_app`. + Under `CLAUDE.md`'s god-file rule it is a future split candidate (Variant B — + the routes close over `app` and shared helpers). +- `client.py` opens a fresh `httpx.Client` per call (≈60 methods), so no + connection pooling. Fine at operator-UI request rates; worth a shared client if + a page ever fans out. +- PyYAML is used by `tests/integration/test_operability_alerts.py` but is not a + declared dependency — it is present only transitively. The compose assertions + added here deliberately use plain text matching to avoid relying on it. From c4caeaf181ecd603fb6c69b9ccb8c49b09802fc7 Mon Sep 17 00:00:00 2001 From: Hermes Agent Date: Sat, 26 Sep 2026 06:18:58 +0000 Subject: [PATCH 16/54] refactor: narrow six blind except clauses to their real failure modes Adopting BLE001 means judging each `except Exception` individually. These six handlers can name what they are catching, so the blind catch was costing us the ability to see a defect in the same block: - `_decode_password_hash`: b64decode raises binascii.Error/UnicodeEncodeError, both ValueError subclasses; TypeError/AttributeError cover a NULL password column. A bug in the KDF constants now surfaces instead of reading as "password does not verify". - AML move fast-path `int(...)` x2 -> (TypeError, ValueError); the 422 text and `from None` are unchanged. - AML move `result.data.get("barcode")` x2 -> (AttributeError, TypeError): a non-mapping backend payload means "no barcode to name". - `_parse_json_report` -> (OSError, ValueError): a vanished or truncated pytest report means "no counts", nothing else. - `real_library_backend` fixture -> (ImportError, AttributeError): the optional hardware module. A constructor that raises for any other reason must not be silently swapped for a different backend. Behaviour is preserved at every site (same fall-through value, same status codes); the narrowing only changes which exceptions escape. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_013S1qbQs31YSox1azQChqs2 --- openblade/api/aml_state.py | 9 ++++++++- openblade/api/routes_aml_operations.py | 12 ++++++++---- openblade/api/routes_test_runner.py | 6 +++++- tests/hardware/conftest.py | 6 +++++- 4 files changed, 26 insertions(+), 7 deletions(-) diff --git a/openblade/api/aml_state.py b/openblade/api/aml_state.py index a4fc941..c5e9ab0 100644 --- a/openblade/api/aml_state.py +++ b/openblade/api/aml_state.py @@ -2054,7 +2054,14 @@ def hash_password(password: str) -> str: def _decode_password_hash(password_hash: str) -> tuple[bytes, bytes] | None: try: decoded = base64.b64decode(password_hash.encode("ascii"), validate=True) - except Exception: + except (ValueError, TypeError, AttributeError): + # ValueError covers both real failure modes of the expression above: + # binascii.Error (non-base64 / bad padding) and UnicodeEncodeError + # (non-ASCII hash) are both ValueError subclasses. TypeError/ + # AttributeError cover a stored hash that is not a str at all (a NULL + # password column read back as None). Anything else here is a defect in + # this function, not a malformed hash, and must not be reported as + # "password does not verify". return None expected_length = _PASSWORD_SALT_BYTES + _PASSWORD_KEY_BYTES if len(decoded) != expected_length: diff --git a/openblade/api/routes_aml_operations.py b/openblade/api/routes_aml_operations.py index 81b53ba..585029b 100644 --- a/openblade/api/routes_aml_operations.py +++ b/openblade/api/routes_aml_operations.py @@ -625,7 +625,7 @@ async def create_move( ) s = int(s_raw) d = int(d_raw) - except Exception: + except (TypeError, ValueError): # `from None`, not `from exc`: the ValueError/TypeError text can echo # the caller's raw payload, and this detail string is client-facing. raise HTTPException(status_code=422, detail="Invalid slot/drive identifiers") from None @@ -637,7 +637,9 @@ async def create_move( barcode_val = ( result.data.get("barcode") if getattr(result, "data", None) else None ) - except Exception: + except (AttributeError, TypeError): + # `result.data` is backend-shaped and only sometimes a mapping; + # a non-mapping payload means "no barcode to name", not an error. barcode_val = None return _ws_result(f"Queued move for {barcode_val or str(s)}") else: @@ -674,7 +676,7 @@ def _first_present(d: dict, keys: list[str]): try: s = int(source_raw) d = int(dest_raw) - except Exception: + except (TypeError, ValueError): # `from None`, not `from exc`: the ValueError/TypeError text can echo # the caller's raw payload, and this detail string is client-facing. raise HTTPException(status_code=422, detail="Invalid slot/drive identifiers") from None @@ -688,7 +690,9 @@ def _first_present(d: dict, keys: list[str]): barcode_val = ( result.data.get("barcode") if getattr(result, "data", None) else None ) - except Exception: + except (AttributeError, TypeError): + # `result.data` is backend-shaped and only sometimes a mapping; + # a non-mapping payload means "no barcode to name", not an error. barcode_val = None return _ws_result(f"Queued move for {barcode_val or str(s)}") else: diff --git a/openblade/api/routes_test_runner.py b/openblade/api/routes_test_runner.py index be0e034..c395f6b 100644 --- a/openblade/api/routes_test_runner.py +++ b/openblade/api/routes_test_runner.py @@ -116,7 +116,11 @@ def _parse_json_report(run_id: str) -> dict: return {} try: return json.loads(path.read_text()) - except Exception: + except (OSError, ValueError): + # OSError: the report file vanished or is unreadable between exists() and + # read. ValueError: json.JSONDecodeError for a truncated report (pytest + # killed mid-write). Both mean "no counts available"; anything else is a + # defect here and should surface. return {} diff --git a/tests/hardware/conftest.py b/tests/hardware/conftest.py index 1616709..fc6118a 100644 --- a/tests/hardware/conftest.py +++ b/tests/hardware/conftest.py @@ -81,7 +81,11 @@ def real_library_backend(real_hardware_guard): return backend_cls(changer_device=changer) except TypeError: return backend_cls(changer) - except Exception: + except (ImportError, AttributeError): + # The real hardware module is optional (ImportError) and its class name + # has moved before (AttributeError); either way fall back to mtx. A + # backend constructor that raises for any other reason is a real failure + # and must not be papered over with a different backend. return MtxChangerBackend( device=changer, guard=_real_hardware_guard(), From d0ed7d258ac85938459d810914fd8233db6d2755 Mon Sep 17 00:00:00 2001 From: Hermes Agent Date: Sat, 26 Sep 2026 06:19:31 +0000 Subject: [PATCH 17/54] fix: stop raw exception text reaching unauthenticated NAS and proxy error fields MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three blind handlers put `str(exc)` into a field that is served to clients. This is the leak class docs/ and test_job_error_sanitization.py already guard on jobs.error ("CommandError carries argv + tool stderr"), reached through routes nobody had audited: - `HydrationExecutor._hydrate_record` -> NasRestoreJob.error_message, served by GET /nas/restore-jobs/{id} (nas_config.router has no auth dependency). - `_BaseIngest.run` and its per-file handler -> IngestJob.errors, served by GET /nas/ingest/{job_id}. - `probe_remote_library` -> {"status": "offline", "error": str(exc)}; httpx exception text carries the probe URL and wraps OS-level detail. All three stay broad — per-file isolation and "a probe reports, it does not 500" are the reason they exist — and are now logged with exc_info, with safe_job_error()/a class-name string on the wire. The failing file's relative path stays: it names what failed and the dataset listing already shows it. Not degrading error quality to satisfy the sanitizer: ingest's curated preflight refusals ("cache drive cannot reserve N bytes", "source_path is required", "Source changed during source-stream ingest") were bare RuntimeErrors, which safe_job_error reduces to a class name. They are now IngestRefusedError (OpenBladeError, RuntimeError) so the message passes through, following the ShardedRestoreFailed precedent in jobs/tree_restore.py; RuntimeError stays in the bases so existing `except RuntimeError` callers are unaffected. The two existing tests asserting that text pass unchanged. Each leak has a regression test, mutation-checked: restoring `str(exc)` / `{exc}` fails all four. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_013S1qbQs31YSox1azQChqs2 --- openblade/api/routes_proxy.py | 16 ++++- openblade/nas/hydration.py | 17 ++++- openblade/nas/ingest.py | 51 ++++++++++----- tests/unit/test_job_error_sanitization.py | 58 +++++++++++++++++ tests/unit/test_nas_hydration.py | 26 ++++++++ tests/unit/test_nas_ingest.py | 77 +++++++++++++++++++++++ 6 files changed, 226 insertions(+), 19 deletions(-) diff --git a/openblade/api/routes_proxy.py b/openblade/api/routes_proxy.py index e456389..8ddab2b 100644 --- a/openblade/api/routes_proxy.py +++ b/openblade/api/routes_proxy.py @@ -1,6 +1,7 @@ from __future__ import annotations import asyncio +import logging from typing import Any from urllib.parse import quote @@ -12,6 +13,7 @@ from openblade.catalog.models import AmlUser router = APIRouter(prefix="/aml/proxy", tags=["proxy"]) +logger = logging.getLogger(__name__) class RemoteLibraryProbeRequest(BaseModel): @@ -223,5 +225,15 @@ async def probe_remote_library( "activeJobs": _count_active_jobs(jobs), }, } - except Exception as exc: - return {"status": "offline", "error": str(exc)} + # Probing an operator-supplied host: every failure mode (DNS, TLS, timeout, + # a non-JSON body) is the same answer — "offline" — so this stays broad. + # str(exc) did NOT stay: httpx exception text carries the probe URL and + # httpx.ConnectError wraps OS-level text, and this dict is returned to the + # client. Full detail goes to the log; the wire gets the class name, the same + # contract as safe_job_error() on jobs.error. + except Exception as exc: # noqa: BLE001 - see above + logger.warning("remote library probe failed", exc_info=True) + return { + "status": "offline", + "error": f"Remote library unreachable ({type(exc).__name__}); see server logs", + } diff --git a/openblade/nas/hydration.py b/openblade/nas/hydration.py index aad9d0a..afa959c 100644 --- a/openblade/nas/hydration.py +++ b/openblade/nas/hydration.py @@ -9,7 +9,7 @@ from dataclasses import dataclass, field from pathlib import Path, PurePosixPath -from openblade.domain.errors import ChecksumMismatchError +from openblade.domain.errors import ChecksumMismatchError, safe_job_error from openblade.nas.restore_planner import RestorePlan from openblade.nas.service import NasService from openblade.nas.tape_paths import dataset_tape_path @@ -204,10 +204,21 @@ def _hydrate_record(self, hydration_job: HydrationJob, record: NasFileRecord) -> except _HydrationControlSignal: self._set_file_status(record, original_status) raise - except Exception as exc: + # Per-file isolation: one unreadable file must not abandon the rest of the + # restore, so this stays broad. safe_job_error() replaced the raw `{exc}`: + # these strings are joined into NasRestoreJob.error_message, which + # GET /nas/restore-jobs/{id} serves with no auth dependency — the same + # surface docs/ and test_job_error_sanitization.py guard on jobs.error. + # Typed OpenBlade errors (ChecksumMismatchError) still pass through. + except Exception as exc: # noqa: BLE001 - see above + logger.warning( + "hydration file restore failed", + extra={"relative_path": record.relative_path}, + exc_info=True, + ) self._persist_file_record(record.model_copy(update={"status": NasFileState.FAILED})) hydration_job.files_failed += 1 - hydration_job.errors.append(f"{record.relative_path}: {exc}") + hydration_job.errors.append(f"{record.relative_path}: {safe_job_error(exc)}") self._update_status( hydration_job.job_id, RestoreJobStatus.RUNNING, diff --git a/openblade/nas/ingest.py b/openblade/nas/ingest.py index 7d7d00a..c2681c1 100644 --- a/openblade/nas/ingest.py +++ b/openblade/nas/ingest.py @@ -12,6 +12,7 @@ from pydantic import BaseModel, Field +from openblade.domain.errors import OpenBladeError, safe_job_error from openblade.domain.models import MountMode from openblade.domain.policies import FormatConfirmation, SafetyToken from openblade.nas.archive_lifecycle import ( @@ -94,6 +95,20 @@ class IngestCancelledError(RuntimeError): pass +class IngestRefusedError(OpenBladeError, RuntimeError): + """An ingest refusal whose message is curated operator text. + + Preflight budget/availability refusals, the source-changed-under-us guard and + the post-write verification failure all say something the operator has to act + on. Typed so ``safe_job_error`` passes the message through: "cache drive + cannot reserve N bytes" reduced to "Job failed (RuntimeError)" on + ``GET /nas/ingest/{job_id}`` would be a worse answer than the leak the + sanitizer exists to stop. Genuinely unexpected exceptions stay untyped and + still reduce to a class name. ``RuntimeError`` is kept in the bases so the + existing ``except RuntimeError`` callers are unaffected. + """ + + def _reserve_cache_drive_bytes(drive_id: str, job_id: str, bytes_required: int) -> None: with _STORE_LOCK: reservations = _CACHE_DRIVE_RESERVATIONS.setdefault(drive_id, {}) @@ -161,8 +176,14 @@ def run(self) -> IngestJob: self._finalize_job() except IngestCancelledError as exc: self._mark_dataset_cancelled(str(exc)) - except Exception as exc: - self._mark_dataset_failed(str(exc)) + # Job-runner boundary: run() owns the dataset's terminal state, so it has + # to catch everything or a defect leaves the dataset stuck in ARCHIVING. + # safe_job_error() replaced the raw str(exc): job.errors is served by + # GET /nas/ingest/{job_id}, which has no auth dependency, and a + # CommandError there would publish mkltfs/mtx argv and stderr. + except Exception as exc: # noqa: BLE001 - see above + logger.warning("ingest run failed", exc_info=True) + self._mark_dataset_failed(safe_job_error(exc)) finally: self._cleanup() self.job.current_tape = None @@ -231,7 +252,7 @@ def _write_assignment(self, assignment: TapeAssignment) -> None: self.ltfs, ) if not lifecycle_result.success: - raise RuntimeError( + raise IngestRefusedError( "; ".join(lifecycle_result.errors) or f"archive lifecycle failed for {prepared.relative_path}" ) @@ -240,7 +261,9 @@ def _write_assignment(self, assignment: TapeAssignment) -> None: self.job.bytes_written += prepared.size_bytes except Exception as exc: self.job.files_failed += 1 - self.job.errors.append(f"{relative_path}: {exc}") + # safe_job_error: job.errors is served by GET /nas/ingest/{job_id} + # with no auth dependency, same contract as jobs.error. + self.job.errors.append(f"{relative_path}: {safe_job_error(exc)}") self._upsert_file_record( prepared, tape_barcode=assignment.barcode, @@ -373,7 +396,7 @@ def _verify_tape_copy( stat.checksum_sha256 != prepared.checksum_sha256 or stat.size_bytes != prepared.size_bytes ): - raise RuntimeError(f"verification failed for {prepared.relative_path}") + raise IngestRefusedError(f"verification failed for {prepared.relative_path}") def _upsert_file_record( self, @@ -515,7 +538,7 @@ def _record_cache_path(self, relative_path: str) -> str | None: def _preflight(self, job: IngestJob) -> list[str]: cache_root = Path(self.cache_drive.root_path) if not cache_root.exists() or not cache_root.is_dir(): - raise RuntimeError(f"Cache drive root_path {cache_root} is not available") + raise IngestRefusedError(f"Cache drive root_path {cache_root} is not available") total_bytes = 0 seen_paths: set[str] = set() @@ -525,13 +548,13 @@ def _preflight(self, job: IngestJob) -> list[str]: seen_paths.add(prepared.source_path) source_path = Path(prepared.source_path).resolve() if not source_path.exists(): - raise RuntimeError( + raise IngestRefusedError( f"Cache-drive source file {prepared.source_path} is not available" ) try: source_path.relative_to(cache_root.resolve()) except ValueError as exc: - raise RuntimeError( + raise IngestRefusedError( f"Cache-drive source file {prepared.source_path} is outside cache root {cache_root}" ) from exc total_bytes += prepared.size_bytes @@ -541,7 +564,7 @@ def _preflight(self, job: IngestJob) -> list[str]: self.cache_drive.id, exclude_job_id=self.job.job_id ) if total_bytes + reserved_bytes > available_bytes: - raise RuntimeError( + raise IngestRefusedError( f"Cache drive {self.cache_drive.id} cannot reserve {total_bytes} bytes; " f"{available_bytes - reserved_bytes} bytes remain within configured budget" ) @@ -581,7 +604,7 @@ def _prepare_files(self) -> dict[str, _PreparedFile]: def _preflight(self, job: IngestJob) -> list[str]: if not self.config.enabled: - raise RuntimeError("Source-stream ingest is disabled") + raise IngestRefusedError("Source-stream ingest is disabled") if not job.plan.source_path: if any( @@ -592,7 +615,7 @@ def _preflight(self, job: IngestJob) -> list[str]: self.config.snapshot_required, ) ): - raise RuntimeError("source_path is required for robust source-stream ingest") + raise IngestRefusedError("source_path is required for robust source-stream ingest") warnings = [] for assignment in job.plan.tape_assignments: for filepath in assignment.files: @@ -603,7 +626,7 @@ def _preflight(self, job: IngestJob) -> list[str]: source_root = Path(job.plan.source_path) if not source_root.exists() or not source_root.is_dir(): - raise RuntimeError(f"Source path {source_root} is not available") + raise IngestRefusedError(f"Source path {source_root} is not available") for assignment in job.plan.tape_assignments: for relative_path in assignment.files: @@ -632,7 +655,7 @@ def _before_write_file(self, prepared: _PreparedFile) -> None: self._capture_snapshot(prepared.relative_path) != self._source_snapshots[prepared.relative_path] ): - raise RuntimeError( + raise IngestRefusedError( f"Source changed during source-stream ingest: {prepared.relative_path}" ) @@ -643,7 +666,7 @@ def _abort_on_file_error(self, prepared: _PreparedFile, error: Exception) -> boo def _capture_snapshot(self, relative_path: str) -> _SourceSnapshot: source_path = Path(self._resolve_source_path(relative_path)) if not source_path.exists() or not source_path.is_file(): - raise RuntimeError(f"Source file {source_path} is not available") + raise IngestRefusedError(f"Source file {source_path} is not available") stat = source_path.stat() checksum = None if "checksum" in self.config.source_change_detection or self.config.checksum_mode in { diff --git a/tests/unit/test_job_error_sanitization.py b/tests/unit/test_job_error_sanitization.py index 6ae2b22..7b9a572 100644 --- a/tests/unit/test_job_error_sanitization.py +++ b/tests/unit/test_job_error_sanitization.py @@ -4,6 +4,8 @@ import contextlib +import pytest + from openblade.domain.errors import OpenBladeError, safe_job_error from openblade.hardware.runner import CommandError @@ -63,3 +65,59 @@ def test_aml_move_error_detail_is_sanitized() -> None: src = inspect.getsource(mod) assert "detail=str(exc)" not in src assert src.count("detail=safe_job_error(exc)") >= 2 + + +def test_remote_library_probe_error_is_curated() -> None: + """The remote-library probe reports "offline" for every failure mode, but the + text it returns used to be str(exc) on a bare `except Exception`: httpx carries + the probe URL, and ConnectError wraps OS-level text. The class name is the whole + payload the client gets.""" + import asyncio + + import httpx + + from openblade.api.routes_proxy import RemoteLibraryProbeRequest, probe_remote_library + + class _ExplodingClient: + def __init__(self, *args, **kwargs) -> None: + pass + + async def __aenter__(self): + return self + + async def __aexit__(self, *exc_info) -> None: + return None + + async def post(self, *args, **kwargs): + raise RuntimeError("connect to http://remote:8000 failed: SECRET-DETAIL") + + monkeypatch = pytest.MonkeyPatch() + monkeypatch.setattr(httpx, "AsyncClient", _ExplodingClient) + try: + result = asyncio.run( + probe_remote_library( + "lib-1", + RemoteLibraryProbeRequest(host="remote", port=8000, username="u", password="p"), + None, + ) + ) + finally: + monkeypatch.undo() + + assert result["status"] == "offline" + assert "SECRET-DETAIL" not in result["error"] + assert "remote:8000" not in result["error"] + assert "RuntimeError" in result["error"] + + +def test_decode_password_hash_rejects_malformed_hashes_without_swallowing_defects() -> None: + """_decode_password_hash() narrowed from `except Exception` to the three ways a + STORED hash can be malformed (non-base64, non-ASCII, NULL column). All three + still answer "not a hash" rather than raising into the login path.""" + from openblade.api.aml_state import is_password_hash + + assert is_password_hash("not base64!!!") is False + assert is_password_hash("\u00e9\u00e9\u00e9") is False + assert is_password_hash("") is False + assert is_password_hash(None) is False # type: ignore[arg-type] + assert is_password_hash("aGVsbG8=") is False # valid base64, wrong length diff --git a/tests/unit/test_nas_hydration.py b/tests/unit/test_nas_hydration.py index 9e58e72..cd15a57 100644 --- a/tests/unit/test_nas_hydration.py +++ b/tests/unit/test_nas_hydration.py @@ -432,3 +432,29 @@ def get_dataset(self, dataset_id): ) assert "hydration tape read failed" in joined assert any(r.exc_info is not None for r in caplog.records), "no traceback captured" + + +def test_per_file_error_message_is_sanitized(monkeypatch) -> None: + """NasRestoreJob.error_message is served by GET /nas/restore-jobs/{id}, a route + with no auth dependency. A CommandError there would publish mtx/mkltfs argv and + raw tool stderr, which is exactly what safe_job_error() exists to prevent on + jobs.error. The relative path stays — it names which file failed and is already + readable through the dataset listing.""" + from openblade.hardware.runner import CommandError + + service, job = setup_restore_fixture(paths=["photos/a.jpg"], tapes=["VOL001L9"]) + executor = hydration_executor(service) + + def leaky(record: NasFileRecord) -> bytes: + raise CommandError(["mtx", "-f", "/dev/sg0", "status"], 1, "raw stderr LEAK") + + monkeypatch.setattr(executor, "_simulate_content", leaky) + + restored = executor.run(job.id) + + assert restored.error_message is not None + assert "LEAK" not in restored.error_message + assert "/dev/sg0" not in restored.error_message + assert "mtx" not in restored.error_message + assert "CommandError" in restored.error_message + assert "photos/a.jpg" in restored.error_message diff --git a/tests/unit/test_nas_ingest.py b/tests/unit/test_nas_ingest.py index a0cb7ae..6c23451 100644 --- a/tests/unit/test_nas_ingest.py +++ b/tests/unit/test_nas_ingest.py @@ -437,3 +437,80 @@ def test_ingest_with_no_plan_returns_400(tmp_path: Path) -> None: ) assert response.status_code == 400 + + +def test_ingest_run_failure_error_is_sanitized(tmp_path: Path, monkeypatch) -> None: + """IngestJob.errors is served by GET /nas/ingest/{job_id}, a route with no auth + dependency. run()'s catch-all owns the dataset's terminal state, so it must stay + broad — but the text it records goes through safe_job_error(), or a CommandError + publishes mkltfs/mtx argv and stderr on an unauthenticated surface.""" + from openblade.hardware.runner import CommandError + from openblade.nas.ingest import CacheDriveIngest + + service, cache_root = _setup_service(tmp_path) + plan = register_archive_plan(_make_plan(cache_root)) + job = start_ingest_job( + plan=plan, + dataset_name="dataset-a", + pool_id="pool-1", + nas_service=service, + cache_drive_id="cache-1", + ) + + def leaky(self): + raise CommandError(["mkltfs", "-d", "/dev/nst0"], 1, "raw stderr LEAK") + + monkeypatch.setattr(CacheDriveIngest, "_prepare_files", leaky) + + result = _run_job(service, job.job_id) + + assert result.status is DatasetStatus.FAILED + joined = " ".join(result.errors) + assert "LEAK" not in joined + assert "/dev/nst0" not in joined + assert "mkltfs" not in joined + assert "CommandError" in joined + + +def test_ingest_per_file_error_is_sanitized(tmp_path: Path, monkeypatch) -> None: + """The per-file handler records into the same unauthenticated IngestJob.errors, + so it goes through safe_job_error() too. The relative path stays: it names which + file failed, which the dataset listing already shows.""" + from openblade.hardware.runner import CommandError + + service, cache_root = _setup_service(tmp_path) + plan = register_archive_plan(_make_plan(cache_root)) + job = start_ingest_job( + plan=plan, + dataset_name="dataset-a", + pool_id="pool-1", + nas_service=service, + cache_drive_id="cache-1", + ) + + context = get_context() + + def leaky_write(handle, dest, content, **kwargs): + raise CommandError(["mkltfs", "-d", "/dev/nst0"], 1, "raw stderr LEAK") + + monkeypatch.setattr(context.ltfs, "write_bytes", leaky_write) + + result = _run_job(service, job.job_id) + + joined = " ".join(result.errors) + assert result.files_failed == 2 + assert "LEAK" not in joined + assert "/dev/nst0" not in joined + assert "a.txt" in joined + assert "CommandError" in joined + + +def test_preflight_refusals_keep_their_curated_text(tmp_path: Path) -> None: + """IngestRefusedError is typed so safe_job_error passes it through: sanitizing + the run() boundary must not degrade the messages an operator acts on.""" + from openblade.domain.errors import safe_job_error + from openblade.nas.ingest import IngestRefusedError + + exc = IngestRefusedError("Cache drive cache-1 cannot reserve 10 bytes") + assert safe_job_error(exc) == "Cache drive cache-1 cannot reserve 10 bytes" + assert isinstance(exc, RuntimeError) From 51f75c6ca2bb9b9541f0a7ddb2c36c7b0a7d02da Mon Sep 17 00:00:00 2001 From: Hermes Agent Date: Sat, 26 Sep 2026 06:20:00 +0000 Subject: [PATCH 18/54] fix: log the swallowed cause in seven silent exception handlers MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `except Exception: pass|continue|` with no log is the category BLE001 is actually worth enabling for: the handler is defensible, but the evidence is gone. Each of these keeps its behaviour (and its broad catch, with a reason) and gains a warning with exc_info: - CatalogRebuildPlanner: disaster recovery discarded the only evidence of why a cartridge could not be recovered; the operator saw "failed to recover catalog data for VOL001L9" and nothing else. - POST /nas/datasets/{id}/verify: a per-file read failure is counted as CORRUPT. Logged, and the misclassification (an unavailable drive or failed mount is not corruption) is now flagged at the site — narrowing it needs a new NasFileState, so it is deliberately not changed here. - ArchiveLifecycleManager step dispatcher: which step failed was reported, why was not. - GET /ltfs/status: "unknown" with no trace of why the backend went dark. - AML move barcode inference: the handler body was `barcode_raw = barcode_raw`, a no-op standing in for a comment. Inner aml_state scan narrowed to (AttributeError, TypeError, ValueError). - bootstrap demo LTFS seeding (x2): a half-seeded demo environment looked like an empty one. Import guard narrowed to ImportError. - restore_and_verify campaign script: silently skipped paths with no resolvable archived instance, so a short campaign run looked complete. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_013S1qbQs31YSox1azQChqs2 --- openblade/api/nas_config.py | 15 ++++++++++++++- openblade/api/routes_aml_operations.py | 19 ++++++++++++++----- openblade/api/routes_ltfs.py | 6 +++++- openblade/bootstrap.py | 12 +++++++++--- openblade/nas/archive_lifecycle.py | 12 +++++++++++- openblade/nas/catalog_rebuild.py | 15 ++++++++++++++- scripts/campaign/restore_and_verify.py | 5 ++++- 7 files changed, 71 insertions(+), 13 deletions(-) diff --git a/openblade/api/nas_config.py b/openblade/api/nas_config.py index 1de6050..41940cc 100644 --- a/openblade/api/nas_config.py +++ b/openblade/api/nas_config.py @@ -2,6 +2,7 @@ from __future__ import annotations +import logging from datetime import datetime, timezone from pathlib import PurePosixPath from uuid import uuid4 @@ -68,6 +69,7 @@ ) router = APIRouter(prefix="/nas", tags=["NAS Config"]) +logger = logging.getLogger(__name__) _FUSE_HOOKS: dict[int, FuseHook] = {} @@ -404,7 +406,18 @@ async def verify_dataset( handle = mounts[record.tape_barcode] tape_path = PurePosixPath("/") / dataset.name / record.relative_path observed_checksum = ltfs.stat(handle[0], tape_path).checksum_sha256 - except Exception: + # Per-file isolation: verify has to report on every file, so a load, + # mount or stat failure on one record cannot abort the sweep. NOTE the + # classification is coarser than it looks — an unavailable drive or a + # failed mount is counted as CORRUPT, not "unverifiable". Narrowing + # that needs a new NasFileState, so it is logged and flagged rather + # than silently reclassified here. + except Exception: # noqa: BLE001 - see above + logger.warning( + "dataset verify could not read file", + extra={"barcode": record.tape_barcode, "path": record.relative_path}, + exc_info=True, + ) checksums[record.relative_path] = "" files_corrupt += 1 service.upsert_file_record( diff --git a/openblade/api/routes_aml_operations.py b/openblade/api/routes_aml_operations.py index 585029b..c8c987e 100644 --- a/openblade/api/routes_aml_operations.py +++ b/openblade/api/routes_aml_operations.py @@ -2,6 +2,7 @@ from __future__ import annotations +import logging from datetime import datetime, timedelta, timezone from typing import Any from uuid import uuid4 @@ -17,6 +18,7 @@ from openblade.domain.errors import safe_job_error router = APIRouter() +logger = logging.getLogger(__name__) def _normalize_move_address(address: str) -> str: @@ -743,11 +745,18 @@ def _first_present(d: dict, keys: list[str]): ): barcode_raw = str(media.get("barcode")) break - except Exception: - pass - except Exception: - # If inventory lookup fails, continue and let validation handle it - barcode_raw = barcode_raw + except (AttributeError, TypeError, ValueError) as exc: + # A malformed aml_state media entry only costs us the + # barcode hint; validation below still rejects the move. + logger.warning("aml media slot scan failed: %s", type(exc).__name__) + # Best-effort inventory probe: any failure degrades to "no barcode hint" + # and the _is_missing() validation below answers 422. Logged (class name + # only, never the client-facing detail) because the original swallow — + # `barcode_raw = barcode_raw` — lost the cause entirely. + except Exception as exc: # noqa: BLE001 - see above + logger.warning( + "move-medium barcode inference failed: %s", type(exc).__name__, exc_info=True + ) # If any core move fields are missing, return 422 (validation error) so clients see a clear validation response def _is_missing(val: object) -> bool: diff --git a/openblade/api/routes_ltfs.py b/openblade/api/routes_ltfs.py index 49c45b5..9ed7fb2 100644 --- a/openblade/api/routes_ltfs.py +++ b/openblade/api/routes_ltfs.py @@ -140,5 +140,9 @@ async def ltfs_status(context: AppContext = Depends(get_context)) -> dict: # Return LTFS backend status if available try: return context.ltfs.to_json() - except Exception: + # Status endpoints report, they do not fail: any backend shape that cannot + # serialise itself becomes "unknown" rather than a 500. Logged because the + # original swallow left no trace of why the backend went dark. + except Exception: # noqa: BLE001 - see above + logger.warning("ltfs status unavailable", exc_info=True) return {"status": "unknown"} diff --git a/openblade/bootstrap.py b/openblade/bootstrap.py index ff88b34..f6b1e95 100644 --- a/openblade/bootstrap.py +++ b/openblade/bootstrap.py @@ -499,9 +499,10 @@ def _seed_demo_ltfs(catalog: CatalogRepository, ltfs) -> None: This helps emulator-backed tests that expect LTFS content to exist. """ + logger = structlog.get_logger(__name__) try: from openblade.simulator.ltfs_volume import MockFileRecord - except Exception: + except ImportError: return datasets = catalog.list_nas_datasets() @@ -509,7 +510,11 @@ def _seed_demo_ltfs(catalog: CatalogRepository, ltfs) -> None: dataset_id = ds.get("id") try: files = catalog.list_nas_file_records(dataset_id) - except Exception: + # Demo seeding is best-effort and must never break startup, so this stays + # broad; the log replaces a silent empty list that made a half-seeded + # demo environment look like an empty one. + except Exception: # noqa: BLE001 - see above + logger.warning("demo ltfs seed: file records unavailable", dataset_id=dataset_id) files = [] for f in files: barcode = f.get("tape_barcode") @@ -518,7 +523,8 @@ def _seed_demo_ltfs(catalog: CatalogRepository, ltfs) -> None: continue try: tape = ltfs.ensure_tape(barcode) - except Exception: + except Exception: # noqa: BLE001 - best-effort demo seeding, see above + logger.warning("demo ltfs seed: tape unavailable", barcode=barcode) continue # create small deterministic content to avoid large memory usage content = (f"{dataset_id}:{path}").encode()[:1024] diff --git a/openblade/nas/archive_lifecycle.py b/openblade/nas/archive_lifecycle.py index 13cdd92..27f68c9 100644 --- a/openblade/nas/archive_lifecycle.py +++ b/openblade/nas/archive_lifecycle.py @@ -1,6 +1,7 @@ from __future__ import annotations import hashlib +import logging from datetime import datetime, timezone from pydantic import BaseModel, Field @@ -28,6 +29,8 @@ PathMappingRecord, ) +logger = logging.getLogger(__name__) + class ArchiveLifecycleResult(BaseModel): """Result of completing archive lifecycle for one file.""" @@ -142,7 +145,14 @@ def complete_file_archive( ) return result result.steps_completed.append(step_name) - except Exception: # pragma: no cover - guarded by failure-path tests + # Step dispatcher: every step is a different subsystem, and the + # contract is that the lifecycle reports which step failed instead of + # propagating. Curated text ("unexpected error") goes in the result; + # the exception itself now goes to the log instead of nowhere. + except Exception: # noqa: BLE001 - see above + logger.warning( + "archive lifecycle step failed", extra={"step": step_name}, exc_info=True + ) result.steps_failed.append(step_name) result.errors.append(f"{step_name} encountered an unexpected error") result.final_file_state = self._current_file_state( diff --git a/openblade/nas/catalog_rebuild.py b/openblade/nas/catalog_rebuild.py index e70900b..9abf80a 100644 --- a/openblade/nas/catalog_rebuild.py +++ b/openblade/nas/catalog_rebuild.py @@ -2,6 +2,7 @@ from __future__ import annotations +import logging from datetime import datetime from uuid import uuid4 @@ -26,6 +27,8 @@ RebuildRunStatus, ) +logger = logging.getLogger(__name__) + def _utcnow_iso() -> str: return datetime.utcnow().isoformat() + "Z" @@ -191,7 +194,17 @@ def execute_rebuild_run(self, run_id: str) -> CatalogRebuildRunRecord: path_mappings_recovered += self._recover_path_mappings(shard) self._recover_manifest_versions(barcode) completed.append(barcode) - except Exception: + # Per-barcode isolation: one unreadable tape must not abandon the + # rebuild of the others, so this stays broad and the barcode is named + # in error_summary (curated text — the summary is operator-facing). + # The log is the point of this change: disaster recovery previously + # discarded the only evidence of why a tape could not be recovered. + except Exception: # noqa: BLE001 - see above + logger.warning( + "catalog rebuild failed for cartridge", + extra={"barcode": barcode}, + exc_info=True, + ) failed.append(barcode) error_summary.append(f"failed to recover catalog data for {barcode}") diff --git a/scripts/campaign/restore_and_verify.py b/scripts/campaign/restore_and_verify.py index 68d8a1a..7d652bc 100644 --- a/scripts/campaign/restore_and_verify.py +++ b/scripts/campaign/restore_and_verify.py @@ -71,7 +71,10 @@ def main(argv: list[str] | None = None) -> int: path = record.path try: _, instance = catalog.get_latest_instance_for_path(path) - except Exception: + # A path with no resolvable archived instance is not a campaign target; + # skipping it is correct, but the skip is now visible in the run log. + except Exception as exc: # noqa: BLE001 - see above + print(f"skipping {path}: no archived instance ({type(exc).__name__})", file=sys.stderr) continue targets.append((path, instance.barcode)) From 2933bf4c9371e32b23b66bb301f8dd37830ac073 Mon Sep 17 00:00:00 2001 From: Hermes Agent Date: Sat, 26 Sep 2026 06:20:12 +0000 Subject: [PATCH 19/54] docs: record why 22 blind handlers stay broad (# noqa: BLE001 with reasons) These are boundaries whose job is to catch everything. Each noqa names where the error is logged and what curated form escapes, so the next reader does not have to re-derive the judgement: - health_service (x10): readiness, library status, four catalog counts, the rebuild lookup, the per-probe DB loop, library and LTFS component checks. All already log with exc_info; the escape is a degraded HealthStatus, never an exception to the caller. A health endpoint that 500s is a broken health endpoint. - cli/main restore_file / restore_tree / archive_sharded (x3): CLI top level, safe_job_error() into the job record and stderr, then exit 1. - routes_iblade CSV and XML negotiation (x2): the escape is the original JSON body plus X-OpenBlade-Content-Negotiation: json-fallback; reason=... A parity-gated endpoint must not 500 because a converter tripped. - routes_test_runner background task: nothing can catch this task's exception; the run record (status=failed, exit_code=-1) is the escape. - check_emulators probe script (x2): an unreachable emulator is the result. - tests/hardware teardown (x7): a failed unmount in `finally` must not mask the assertion that failed first, and the unload still has to run. No behaviour change in this commit. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_013S1qbQs31YSox1azQChqs2 --- openblade/api/routes_iblade.py | 4 ++-- openblade/api/routes_test_runner.py | 2 +- openblade/cli/main.py | 6 +++--- openblade/nas/health_service.py | 20 ++++++++++---------- scripts/security_agents/check_emulators.py | 4 ++-- tests/hardware/test_ltfs_operations.py | 10 +++++----- tests/hardware/test_performance.py | 4 ++-- 7 files changed, 25 insertions(+), 25 deletions(-) diff --git a/openblade/api/routes_iblade.py b/openblade/api/routes_iblade.py index af26588..9b5294c 100644 --- a/openblade/api/routes_iblade.py +++ b/openblade/api/routes_iblade.py @@ -308,7 +308,7 @@ def _negotiate_iblade_response(request: Request, response: Response) -> Response try: payload = json.loads(bytes(response.body)) csv_payload = _report_payload_to_csv(payload) - except Exception: + except Exception: # noqa: BLE001 - content negotiation must never 500 a parity-gated endpoint: the escape is the original JSON body plus X-OpenBlade-Content-Negotiation: json-fallback; reason=csv-serialization-failed response.headers["X-OpenBlade-Content-Negotiation"] = ( "json-fallback; reason=csv-serialization-failed" ) @@ -337,7 +337,7 @@ def _negotiate_iblade_response(request: Request, response: Response) -> Response try: payload = json.loads(bytes(response.body)) xml_payload = _json_payload_to_xml(payload) - except Exception: + except Exception: # noqa: BLE001 - content negotiation must never 500 a parity-gated endpoint: the escape is the original JSON body plus X-OpenBlade-Content-Negotiation: json-fallback; reason=xml-serialization-failed response.headers["X-OpenBlade-Content-Negotiation"] = ( "json-fallback; reason=xml-serialization-failed" ) diff --git a/openblade/api/routes_test_runner.py b/openblade/api/routes_test_runner.py index c395f6b..a4827de 100644 --- a/openblade/api/routes_test_runner.py +++ b/openblade/api/routes_test_runner.py @@ -277,7 +277,7 @@ async def _run_tests(run_id: str, req: TestRunRequest) -> None: await proc.wait() run["exit_code"] = proc.returncode - except Exception as exc: + except Exception as exc: # noqa: BLE001 - background task boundary: logged via log.error, and the escape is the run record itself (status=failed, exit_code=-1) since nothing can catch this task's exception log.error("test_run_error", error=str(exc)) run["output_lines"].append(f"ERROR: {exc}") run["exit_code"] = -1 diff --git a/openblade/cli/main.py b/openblade/cli/main.py index d13e39b..25779b4 100644 --- a/openblade/cli/main.py +++ b/openblade/cli/main.py @@ -612,7 +612,7 @@ def restore_file( "status": "completed", } failed = False - except Exception as exc: + except Exception as exc: # noqa: BLE001 - CLI top level: safe_job_error() curates the text into the job record and stderr, then exit 1 (`from None` keeps the traceback off the console) message = safe_job_error(exc) context.catalog.update_job_state(job.id, "failed", error=message) _save_state(context) @@ -660,7 +660,7 @@ def _progress(tick: TreeRestoreProgress) -> None: job.id, progress=_progress, ) - except Exception as exc: + except Exception as exc: # noqa: BLE001 - CLI top level: safe_job_error() curates the text into the job record and stderr, then exit 1 message = safe_job_error(exc) context.catalog.update_job_state(job.id, "failed", error=message) _save_state(context) @@ -797,7 +797,7 @@ def archive_sharded( scheduler, job.id, ) - except Exception as exc: + except Exception as exc: # noqa: BLE001 - CLI top level: safe_job_error() curates the text into the job record and stderr, then exit 1 message = safe_job_error(exc) context.catalog.update_job_state(job.id, "failed", error=message) _save_state(context) diff --git a/openblade/nas/health_service.py b/openblade/nas/health_service.py index f062d55..a4e17be 100644 --- a/openblade/nas/health_service.py +++ b/openblade/nas/health_service.py @@ -72,7 +72,7 @@ def check_ready(self) -> ReadyResponse: return ReadyResponse( ready=not reasons, reason="; ".join(reasons), checked_at=checked_at ) - except Exception: + except Exception: # noqa: BLE001 - readiness must answer, not 500: logged with exc_info; the curated escape is a degraded HealthStatus, never an exception to the caller logger.warning("readiness check failed", exc_info=True) return ReadyResponse( ready=False, reason="dependency check unavailable", checked_at=checked_at @@ -101,7 +101,7 @@ def get_library_status(self) -> LibraryStatusResponse: cartridges_loaded=sum(1 for drive in inventory.drives if drive.barcode is not None), last_updated_at=checked_at, ) - except Exception: + except Exception: # noqa: BLE001 - library status must answer, not 500: logged with exc_info; the curated escape is a degraded HealthStatus, never an exception to the caller logger.warning("library status check failed", exc_info=True) return LibraryStatusResponse( library_connected=False, @@ -128,7 +128,7 @@ def get_catalog_status(self) -> CatalogStatusResponse: self.repo.session.execute(select(func.count()).select_from(NasDataset)).scalar_one() ) successful_checks += 1 - except Exception: + except Exception: # noqa: BLE001 - one failed count must not hide the others: logged with exc_info, the field stays -1 and db_reachable reflects the survivors logger.warning("catalog dataset count failed", exc_info=True) try: @@ -138,7 +138,7 @@ def get_catalog_status(self) -> CatalogStatusResponse: ).scalar_one() ) successful_checks += 1 - except Exception: + except Exception: # noqa: BLE001 - one failed count must not hide the others: logged with exc_info, the field stays -1 and db_reachable reflects the survivors logger.warning("catalog file record count failed", exc_info=True) try: @@ -148,7 +148,7 @@ def get_catalog_status(self) -> CatalogStatusResponse: ).scalar_one() ) successful_checks += 1 - except Exception: + except Exception: # noqa: BLE001 - one failed count must not hide the others: logged with exc_info, the field stays -1 and db_reachable reflects the survivors logger.warning("catalog path mapping count failed", exc_info=True) try: @@ -156,7 +156,7 @@ def get_catalog_status(self) -> CatalogStatusResponse: self.repo.session.execute(select(func.count()).select_from(Cartridge)).scalar_one() ) successful_checks += 1 - except Exception: + except Exception: # noqa: BLE001 - one failed count must not hide the others: logged with exc_info, the field stays -1 and db_reachable reflects the survivors logger.warning("catalog cartridge count failed", exc_info=True) try: @@ -169,7 +169,7 @@ def get_catalog_status(self) -> CatalogStatusResponse: str(latest_rebuild_run["status"]) if latest_rebuild_run is not None else None ) successful_checks += 1 - except Exception: + except Exception: # noqa: BLE001 - one failed count must not hide the others: logged with exc_info, the field stays -1 and db_reachable reflects the survivors logger.warning("catalog rebuild lookup failed", exc_info=True) return CatalogStatusResponse( @@ -198,7 +198,7 @@ def _check_database_component(self) -> ComponentHealth: for probe_name, probe in probes.items(): try: probe() - except Exception: + except Exception: # noqa: BLE001 - per-probe isolation: logged with exc_info and named in failed_probes, which decides OK/DEGRADED/UNHEALTHY failed_probes.append(probe_name) logger.warning("database health probe failed", probe=probe_name, exc_info=True) @@ -242,7 +242,7 @@ def _check_library_component(self) -> ComponentHealth: latency_ms=round((time.perf_counter() - started_at) * 1000, 3), last_checked_at=checked_at, ) - except Exception: + except Exception: # noqa: BLE001 - logged with exc_info; the curated escape is a degraded HealthStatus, never an exception to the caller logger.warning("library health check failed", exc_info=True) return ComponentHealth( name="library", @@ -274,7 +274,7 @@ def _check_ltfs_component(self) -> ComponentHealth: latency_ms=round((time.perf_counter() - started_at) * 1000, 3), last_checked_at=checked_at, ) - except Exception: + except Exception: # noqa: BLE001 - logged with exc_info; the curated escape is a degraded HealthStatus, never an exception to the caller logger.warning("ltfs health check failed", exc_info=True) return ComponentHealth( name="ltfs", diff --git a/scripts/security_agents/check_emulators.py b/scripts/security_agents/check_emulators.py index 879cb66..9a72865 100755 --- a/scripts/security_agents/check_emulators.py +++ b/scripts/security_agents/check_emulators.py @@ -22,10 +22,10 @@ try: h = requests.get(url + "/health", timeout=5) print(url + "/health", h.status_code) - except Exception as e: + except Exception as e: # noqa: BLE001 - operator probe script: an unreachable emulator is the result, printed, not a crash print("emulator", url, "unreachable", e) try: inv = requests.get(url + "/api/aml/inventory", timeout=5) print(url + "/api/aml/inventory", inv.status_code) - except Exception as e: + except Exception as e: # noqa: BLE001 - operator probe script: an unreachable endpoint is the result, printed, not a crash print("inventory failed for", url, e) diff --git a/tests/hardware/test_ltfs_operations.py b/tests/hardware/test_ltfs_operations.py index c8eeff9..e644561 100644 --- a/tests/hardware/test_ltfs_operations.py +++ b/tests/hardware/test_ltfs_operations.py @@ -126,7 +126,7 @@ def test_ltfs_mount_formatted( finally: try: _unmount_ltfs(runner, tmp_mount_dir) - except Exception: + except Exception: # noqa: BLE001 - real-hardware teardown: a failed unmount must not mask the assertion that failed first, and the unload below still has to run pass _unload_barcode(changer_device, runner, slot_id) @@ -154,7 +154,7 @@ def test_ltfs_write_small_file( finally: try: _unmount_ltfs(runner, tmp_mount_dir) - except Exception: + except Exception: # noqa: BLE001 - real-hardware teardown: a failed unmount must not mask the assertion that failed first, and the unload below still has to run pass _unload_barcode(changer_device, runner, slot_id) @@ -184,7 +184,7 @@ def test_ltfs_read_small_file( finally: try: _unmount_ltfs(runner, tmp_mount_dir) - except Exception: + except Exception: # noqa: BLE001 - real-hardware teardown: a failed unmount must not mask the assertion that failed first, and the unload below still has to run pass _unload_barcode(changer_device, runner, slot_id) @@ -234,7 +234,7 @@ def test_ltfs_remount_persistence( finally: try: _unmount_ltfs(runner, tmp_mount_dir) - except Exception: + except Exception: # noqa: BLE001 - real-hardware teardown: a failed unmount must not mask the assertion that failed first, and the unload below still has to run pass _unload_barcode(changer_device, runner, slot_id) @@ -261,7 +261,7 @@ def test_ltfs_capacity_reporting( finally: try: _unmount_ltfs(runner, tmp_mount_dir) - except Exception: + except Exception: # noqa: BLE001 - real-hardware teardown: a failed unmount must not mask the assertion that failed first, and the unload below still has to run pass _unload_barcode(changer_device, runner, slot_id) diff --git a/tests/hardware/test_performance.py b/tests/hardware/test_performance.py index 45ea877..5076171 100644 --- a/tests/hardware/test_performance.py +++ b/tests/hardware/test_performance.py @@ -128,7 +128,7 @@ def test_single_drive_write_throughput( finally: try: _unmount_ltfs(runner, tmp_mount_dir) - except Exception: + except Exception: # noqa: BLE001 - real-hardware teardown: a failed unmount must not mask the assertion that failed first, and the unload below still has to run pass _unload_barcode(changer_device, runner, slot_id) @@ -162,7 +162,7 @@ def test_single_drive_read_throughput( finally: try: _unmount_ltfs(runner, tmp_mount_dir) - except Exception: + except Exception: # noqa: BLE001 - real-hardware teardown: a failed unmount must not mask the assertion that failed first, and the unload below still has to run pass _unload_barcode(changer_device, runner, slot_id) From cd53169d89f048fec2f450a7f678497887a39db5 Mon Sep 17 00:00:00 2001 From: Hermes Agent Date: Sat, 26 Sep 2026 06:20:28 +0000 Subject: [PATCH 20/54] chore: select BLE in ruff, retire the five inert S noqa markers MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit BLE is now in `select`, so every `# noqa: BLE001` in the tree is live enforcement instead of documentation about a follow-up. The rule-set comment records the policy the markers encode: broad is legitimate only at a boundary whose job is to catch everything, and the reason must name where the error is logged and what curated form escapes. S stays deliberately unselected (3,748 findings, overwhelmingly S101 `assert` in tests, which is the point of tests), so the five `# noqa: S404/S603/S310` markers were enforcing nothing while looking like they were. They are now plain comments stating the safety property they stood for — argv-list/shell=False discipline in scripts/deploy.py and the two subprocess-based tests, and that the deploy postcheck URL is operator-supplied at deploy time, never per-request. The information survives without pretending a linter checks it. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_013S1qbQs31YSox1azQChqs2 --- pyproject.toml | 28 ++++++++++++++----- scripts/deploy.py | 11 ++++++-- .../test_scalar_http_drive_device_emulator.py | 3 +- tests/unit/test_hardware_safety.py | 3 +- 4 files changed, 33 insertions(+), 12 deletions(-) diff --git a/pyproject.toml b/pyproject.toml index d964fa5..2f2dfb1 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -70,13 +70,27 @@ target-version = "py312" [tool.ruff.lint] # EXPLICIT rule set — never rely on ruff's defaults, which widen between minor # releases (this is why the version above is pinned exactly as well). Anything -# not listed here is OFF, including rules referenced by `# noqa` comments in the -# tree: 23 `# noqa: BLE001` and 4 `# noqa: S3xx/S4xx` markers are currently inert -# documentation, because BLE and S are deliberately not selected yet. Enabling -# them is a scoped follow-up (BLE = 44 findings, S = 3,748, mostly S101 in -# tests) — see docs/decisions/2026-09-11-pin-ruff-toolchain.md. Do not delete those -# noqa markers to "clean up"; they record intent for that follow-up. -select = ["E", "F", "I", "B", "UP", "A", "SIM"] +# not listed here is OFF. +# +# BLE is now SELECTED, so every `# noqa: BLE001` in the tree is live and enforced +# rather than inert documentation. The policy behind those markers: a broad +# `except Exception` is legitimate only at a boundary whose job is to catch +# everything (job/step runners, CLI top level, health probes, content-negotiation +# fallbacks, best-effort probes and demo seeding, real-hardware test teardown). +# Each one carries a reason naming WHERE the error is logged and WHAT curated form +# escapes — `safe_job_error()` (openblade/domain/errors.py) for anything that +# reaches an unauthenticated field, a degraded status for health, a response +# header for content negotiation. A handler that can name its failure modes is +# narrowed instead; a handler that swallows silently is a bug, not a style issue. +# +# S stays deliberately UNSELECTED: 3,748 findings, overwhelmingly S101 (`assert` +# in tests, which is the point of tests). The `# noqa: S6xx/S3xx/S4xx` markers +# that used to sit in the tree pretending to be enforcement are now plain +# comments stating the safety property they documented (argv-list/`shell=False` +# discipline in scripts/deploy.py and the subprocess-based tests, the +# operator-supplied probe URL). Selecting S is still a separate, scoped decision +# — see docs/decisions/2026-09-11-pin-ruff-toolchain.md. +select = ["E", "F", "I", "B", "UP", "A", "SIM", "BLE"] # A002 (argument shadows a builtin) is ignored: FastAPI path/query params like # `id`, `type`, `filter` ARE the wire contract — renaming them would break the # /aml endpoints (see docs/tech-debt.md). diff --git a/scripts/deploy.py b/scripts/deploy.py index c6c0cfd..e63d355 100644 --- a/scripts/deploy.py +++ b/scripts/deploy.py @@ -16,7 +16,7 @@ import json import os import shlex -import subprocess # noqa: S404 - deploy command is operator-supplied, run without a shell +import subprocess # argv-list only, shell=False — see _deploy() import sys import urllib.error import urllib.request @@ -37,7 +37,9 @@ def _precheck() -> StageResult: def _deploy(cmd: list[str] | None) -> StageResult: if not cmd: return StageResult(Stage.DEPLOY, True, "skipped (no --deploy-cmd)") - completed = subprocess.run(cmd, check=False) # noqa: S603 - list form, no shell + # `cmd` is already an argv list (shlex.split of --deploy-cmd) and no shell=True: + # an operator string can therefore never become shell syntax. + completed = subprocess.run(cmd, check=False) ok = completed.returncode == 0 return StageResult(Stage.DEPLOY, ok, f"`{' '.join(cmd)}` exited {completed.returncode}") @@ -68,7 +70,10 @@ def _postcheck_live(base_url: str) -> StageResult: def probe(method: str, path: str) -> int: req = urllib.request.Request(base_url.rstrip("/") + path, method=method) try: - with urllib.request.urlopen(req, timeout=5) as resp: # noqa: S310 - operator-supplied URL + # The URL is operator-supplied (--postcheck-url), so the scheme is not + # constrained here; this runs as a deliberate deploy-time probe from the + # operator's own shell, never on behalf of a request. + with urllib.request.urlopen(req, timeout=5) as resp: return resp.status except urllib.error.HTTPError as exc: return exc.code diff --git a/tests/integration/test_scalar_http_drive_device_emulator.py b/tests/integration/test_scalar_http_drive_device_emulator.py index d3bdbae..a2973a2 100644 --- a/tests/integration/test_scalar_http_drive_device_emulator.py +++ b/tests/integration/test_scalar_http_drive_device_emulator.py @@ -76,7 +76,8 @@ def emulator_url(tmp_path_factory: pytest.TempPathFactory) -> Generator[str, Non "OPENBLADE_DB_URL": f"sqlite:///{db_path}", "PYTHONPATH": str(REPO_ROOT), } - process = subprocess.Popen( # noqa: S603 - fixed argv, no shell + # Fixed argv list, shell=False: nothing here is caller-controlled. + process = subprocess.Popen( [ sys.executable, "-m", diff --git a/tests/unit/test_hardware_safety.py b/tests/unit/test_hardware_safety.py index 79e890a..3054723 100644 --- a/tests/unit/test_hardware_safety.py +++ b/tests/unit/test_hardware_safety.py @@ -307,7 +307,8 @@ def test_procfs_scan_finds_a_real_process_holding_the_mount(self, tmp_path: Path # `executable=` lets us choose argv[0] independently of the binary, so # the child genuinely presents as `ltfs ` in procfs - a # shebang script would show up as /bin/sh and prove nothing. - process = subprocess.Popen( # noqa: S603 + # Fixed argv list, shell=False. + process = subprocess.Popen( ["ltfs", "-c", "import time; time.sleep(30)", mount_point], executable=sys.executable, stdout=subprocess.DEVNULL, From 08d2720e4ae18f7960753a3bbabe822b0347426e Mon Sep 17 00:00:00 2001 From: Amantux Date: Sat, 26 Sep 2026 06:21:48 +0000 Subject: [PATCH 21/54] types: type the PathMapping model, seeder specs, and coordinate parsing Bucket 1 of the mypy-to-zero effort: pure annotations, no behaviour change. - catalog/models.py: PathMapping was the one table declared with bare `Column(...)` instead of `Mapped[...] = mapped_column(...)`, so every field read/write typed as `Column[str]` rather than `str`. Same columns, same nullability, same defaults. - domain/scalar_coordinate.py: `_coerce_int` replaces six `# type: ignore` comments that mypy now reports as unused; it raises TypeError, which `from_dict` already converts to ValueError, so bad payloads behave as before. - bootstrap.py: TypedDicts for the library/demo-dataset seeder specs (they were `dict[str, object]`), enum members instead of the bare strings pydantic was coercing anyway, and explicit LibraryBackend/LTFSBackend annotations on the two-branch backend construction. 304 -> 234 mypy errors. --- openblade/bootstrap.py | 81 +++++++++++++++++++++------ openblade/catalog/models.py | 41 ++++++++------ openblade/domain/scalar_coordinate.py | 26 +++++++-- 3 files changed, 107 insertions(+), 41 deletions(-) diff --git a/openblade/bootstrap.py b/openblade/bootstrap.py index ff88b34..bfeb3ea 100644 --- a/openblade/bootstrap.py +++ b/openblade/bootstrap.py @@ -7,7 +7,7 @@ from contextlib import suppress from dataclasses import dataclass from pathlib import Path -from typing import cast +from typing import TypedDict, cast import structlog @@ -28,7 +28,19 @@ from openblade.jobs.restore import RestoreService from openblade.jobs.worker import Worker from openblade.nas.service import NasService -from openblade.nas.types import NasDataset, NasFileRecord, NasFileState, NasPool, StoragePolicy +from openblade.nas.types import ( + DatasetStatus, + HydrationBehavior, + IngestMode, + NasDataset, + NasFileRecord, + NasFileState, + NasPool, + PolicyType, + PoolAccessMode, + ShardStrategy, + StoragePolicy, +) from openblade.simulator.i3_config import scalar_i3_active_config from openblade.simulator.scenarios import scalar_i3_default @@ -45,27 +57,51 @@ _catalog: CatalogRepository | None = None +class _LibrarySpec(TypedDict): + """One configured library instance, as the seeder describes it.""" + + name: str + emulator_url: str + serial_number: str + model: str + role: str + sort_order: int + legacy_names: set[str] + + +class _DemoDatasetSpec(TypedDict): + """One demo dataset. ``files`` is (relative_path, size_bytes, barcode).""" + + dataset_id: str + name: str + volume_group: str + pool_id: str + policy_id: str + source_path: str + files: list[tuple[str, int, str]] + + def _default_policies() -> list[StoragePolicy]: return [ StoragePolicy( id="critical_sequential", name="Critical Sequential", - policy_type="critical_sequential", + policy_type=PolicyType.CRITICAL_SEQUENTIAL, allow_spillover=False, max_parallelism=1, ), StoragePolicy( id="noncritical_sharded", name="Noncritical Sharded", - policy_type="noncritical_sharded", + policy_type=PolicyType.NONCRITICAL_SHARDED, allow_sharding=True, max_parallelism=4, - shard_strategy="round_robin", + shard_strategy=ShardStrategy.ROUND_ROBIN, ), StoragePolicy( id="balanced", name="Balanced", - policy_type="balanced", + policy_type=PolicyType.BALANCED, allow_spillover=True, max_parallelism=2, ), @@ -85,13 +121,13 @@ def _seed_nas_defaults(catalog: CatalogRepository) -> None: service.upsert_share(share) -def _configured_library_specs(config: OpenBladeConfig) -> list[dict[str, object]]: +def _configured_library_specs(config: OpenBladeConfig) -> list[_LibrarySpec]: default_metadata = [ ("Primary Tape Library", "OB-SCALAR-I3-001", "primary", {"primary"}), ("Secondary Archive", "OB-SCALAR-I3-002", "archive", {"Library 2"}), ("Cold Storage Vault", "OB-SCALAR-I3-003", "cold_storage", {"Library 3"}), ] - specs: list[dict[str, object]] = [] + specs: list[_LibrarySpec] = [] for index, emulator_url in enumerate(config.emulator_urls): if index < len(default_metadata): name, serial, role, legacy_names = default_metadata[index] @@ -151,7 +187,7 @@ def _seed_library_defaults(catalog: CatalogRepository, config: OpenBladeConfig) def _seed_demo_catalog(catalog: CatalogRepository) -> None: - demo_specs = [ + demo_specs: list[_DemoDatasetSpec] = [ { "dataset_id": "demo-project-alpha", "name": "Project Alpha", @@ -243,13 +279,13 @@ def _seed_demo_catalog(catalog: CatalogRepository) -> None: pool_id=spec["pool_id"], policy_id=spec["policy_id"], source_path=spec["source_path"], - ingest_mode="cache_drive", + ingest_mode=IngestMode.CACHE_DRIVE, volume_group_id=volume_group.id, tape_set=tape_set, shard_map=shard_map, file_count=len(spec["files"]), total_bytes=total_bytes, - status="archived", + status=DatasetStatus.ARCHIVED, copies_completed=len(tape_set), manifest_path=f"{spec['source_path']}/manifest.json", created_at="2024-01-22T08:00:00Z", @@ -319,9 +355,9 @@ def _seed_nas_pools(catalog: CatalogRepository) -> None: volume_group_ids=[catalog.create_volume_group("project-alpha").id], default_policy_id="critical_sequential", mount_path="/openblade/pools/critical-projects", - hydration_behavior="queue", + hydration_behavior=HydrationBehavior.QUEUE, restore_target_path="/openblade/restore/critical-projects", - access_mode="read_only", + access_mode=PoolAccessMode.READ_ONLY, ), NasPool( id="general-archive", @@ -330,9 +366,9 @@ def _seed_nas_pools(catalog: CatalogRepository) -> None: volume_group_ids=[catalog.create_volume_group("media-archive-2024").id], default_policy_id="balanced", mount_path="/openblade/pools/general-archive", - hydration_behavior="queue", + hydration_behavior=HydrationBehavior.QUEUE, restore_target_path="/openblade/restore/general-archive", - access_mode="read_only", + access_mode=PoolAccessMode.READ_ONLY, ), NasPool( id="media-cache", @@ -341,9 +377,9 @@ def _seed_nas_pools(catalog: CatalogRepository) -> None: volume_group_ids=[catalog.create_volume_group("backup-set-a").id], default_policy_id="balanced", mount_path="/openblade/pools/media-cache", - hydration_behavior="auto", + hydration_behavior=HydrationBehavior.AUTO, restore_target_path="/openblade/restore/media-cache", - access_mode="read_write", + access_mode=PoolAccessMode.READ_WRITE, ), ] existing_pool_ids = {pool.id for pool in service.list_pools()} @@ -494,7 +530,7 @@ def get_catalog() -> CatalogRepository: return _catalog -def _seed_demo_ltfs(catalog: CatalogRepository, ltfs) -> None: +def _seed_demo_ltfs(catalog: CatalogRepository, ltfs: LTFSBackend) -> None: """Populate MockLTFSBackend with small example files based on catalog NAS records. This helps emulator-backed tests that expect LTFS content to exist. @@ -506,7 +542,12 @@ def _seed_demo_ltfs(catalog: CatalogRepository, ltfs) -> None: datasets = catalog.list_nas_datasets() for ds in datasets: + # These rows are `dict[str, object]` (serialized NAS models), so the + # id/barcode/path reads need narrowing. A non-str id previously reached + # list_nas_file_records and matched nothing, so skipping is equivalent. dataset_id = ds.get("id") + if not isinstance(dataset_id, str): + continue try: files = catalog.list_nas_file_records(dataset_id) except Exception: @@ -514,6 +555,8 @@ def _seed_demo_ltfs(catalog: CatalogRepository, ltfs) -> None: for f in files: barcode = f.get("tape_barcode") path = f.get("relative_path") + if not isinstance(barcode, str) or not isinstance(path, str): + continue if not barcode or not path: continue try: @@ -537,6 +580,8 @@ def _seed_demo_ltfs(catalog: CatalogRepository, ltfs) -> None: def create_context(config: OpenBladeConfig | None = None) -> AppContext: active_config = config or load_config() init_db(active_config.db_url) + library: LibraryBackend + ltfs: LTFSBackend if active_config.backend == BackendMode.MOCK: library, ltfs = scalar_i3_default() else: diff --git a/openblade/catalog/models.py b/openblade/catalog/models.py index da4cb24..cf51f58 100644 --- a/openblade/catalog/models.py +++ b/openblade/catalog/models.py @@ -5,11 +5,10 @@ import json import uuid from datetime import datetime -from typing import Any +from typing import Any, cast from sqlalchemy import ( Boolean, - Column, DateTime, ForeignKey, Integer, @@ -140,7 +139,7 @@ class Job(Base): @property def metadata_dict(self) -> dict[str, Any]: - return json.loads(self.metadata_json or "{}") + return cast("dict[str, Any]", json.loads(self.metadata_json or "{}")) class SafetyTokenRecord(Base): @@ -356,20 +355,28 @@ class PathMapping(Base): __tablename__ = "path_mappings" - id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4())) - logical_path = Column(String, nullable=False, index=True) - pool_id = Column(String, nullable=True, index=True) - dataset_id = Column(String, nullable=True, index=True) - primary_barcode = Column(String, nullable=True) - all_barcodes = Column(String, nullable=False, default="[]") - file_record_id = Column(String, nullable=True) - file_state = Column(String, nullable=False, default="offline_on_tape") - restore_strategy = Column(String, nullable=False, default="single_tape") - size = Column(Integer, nullable=True) - checksum = Column(String, nullable=True) - last_seen_at = Column(String, nullable=True) - created_at = Column(String, nullable=False, default=lambda: datetime.utcnow().isoformat() + "Z") - updated_at = Column(String, nullable=False, default=lambda: datetime.utcnow().isoformat() + "Z") + # Declared with Mapped[]/mapped_column like every other model here. The bare + # `Column(...)` form this table used gives the attributes type `Column[str]` + # instead of `str`, so every read/write of a PathMapping field was a type + # error at the call site. Same columns, same nullability, same defaults. + id: Mapped[str] = mapped_column(String, primary_key=True, default=lambda: str(uuid.uuid4())) + logical_path: Mapped[str] = mapped_column(String, nullable=False, index=True) + pool_id: Mapped[str | None] = mapped_column(String, nullable=True, index=True) + dataset_id: Mapped[str | None] = mapped_column(String, nullable=True, index=True) + primary_barcode: Mapped[str | None] = mapped_column(String, nullable=True) + all_barcodes: Mapped[str] = mapped_column(String, nullable=False, default="[]") + file_record_id: Mapped[str | None] = mapped_column(String, nullable=True) + file_state: Mapped[str] = mapped_column(String, nullable=False, default="offline_on_tape") + restore_strategy: Mapped[str] = mapped_column(String, nullable=False, default="single_tape") + size: Mapped[int | None] = mapped_column(Integer, nullable=True) + checksum: Mapped[str | None] = mapped_column(String, nullable=True) + last_seen_at: Mapped[str | None] = mapped_column(String, nullable=True) + created_at: Mapped[str] = mapped_column( + String, nullable=False, default=lambda: datetime.utcnow().isoformat() + "Z" + ) + updated_at: Mapped[str] = mapped_column( + String, nullable=False, default=lambda: datetime.utcnow().isoformat() + "Z" + ) __table_args__ = (UniqueConstraint("logical_path", "pool_id", name="uq_path_pool"),) diff --git a/openblade/domain/scalar_coordinate.py b/openblade/domain/scalar_coordinate.py index b541f26..d5f8fbf 100644 --- a/openblade/domain/scalar_coordinate.py +++ b/openblade/domain/scalar_coordinate.py @@ -23,6 +23,20 @@ from enum import IntFlag +def _coerce_int(value: object) -> int: + """``int(value)`` for an untyped wire value, with the same failure modes. + + ``from_dict`` parses decoded JSON, so its values are ``object`` and ``int()`` + has no overload for that. This narrows to the argument types ``int()`` + actually accepts and raises ``TypeError`` otherwise -- already one of the + exceptions ``from_dict`` converts to ``ValueError``, so a bad payload + behaves exactly as before. + """ + if isinstance(value, int | float | str): + return int(value) + raise TypeError(f"expected an int-like value, got {type(value).__name__}") + + @dataclass(frozen=True) class ScalarCoordinate: """A physical element coordinate on a Scalar i3. @@ -54,12 +68,12 @@ def from_dict(cls, data: dict[str, object]) -> ScalarCoordinate: """Parse the full {frame,rack,section,column,row,type} form.""" try: return cls( - frame=int(data["frame"]), # type: ignore[arg-type] - rack=int(data["rack"]), # type: ignore[arg-type] - section=int(data["section"]), # type: ignore[arg-type] - column=int(data["column"]), # type: ignore[arg-type] - row=int(data["row"]), # type: ignore[arg-type] - element_type=int(data.get("type", data.get("element_type", 0))), # type: ignore[arg-type] + frame=_coerce_int(data["frame"]), + rack=_coerce_int(data["rack"]), + section=_coerce_int(data["section"]), + column=_coerce_int(data["column"]), + row=_coerce_int(data["row"]), + element_type=_coerce_int(data.get("type", data.get("element_type", 0))), ) except (KeyError, TypeError, ValueError) as exc: raise ValueError(f"invalid ScalarCoordinate dict: {data!r}") from exc From ddc466543cfdcb831f0da4010a7ee3848ffd62fb Mon Sep 17 00:00:00 2001 From: Amantux Date: Sat, 26 Sep 2026 06:21:58 +0000 Subject: [PATCH 22/54] types: annotate bare generics, untyped defs, and FastAPI dependency params Bucket 2: pure annotations (no-untyped-def, type-arg). No behaviour change. - Bare `dict`/`list` annotations spelled out as `dict[str, Any]` / `list[Any]` in the protocol gateway, path-mapping stats and the LTFS/storage-compat/ gateway routes. - `repo=Depends(...)` / `context=Depends(...)` parameters given the explicit types the rest of the API package already uses. - `SidecarValidationError.field` was an implicit-Optional `str = None`; now `str | None`. `SidecarResolver.__init__` takes `NasService | None`. - repository.py: `_SupportsModelDump`/`_FileRecordLike` protocols replace `hasattr`-then-attribute-access and an `object` parameter; `create_role` takes a `Mapping` (read-only, and `dict` is invariant so a caller holding a narrower value type could not pass one). - routes_ltfs: `_active_mounts` is `dict[str, MountHandle]` as its comment said, and `GET /ltfs/status` probes a `_SupportsToJson` protocol instead of relying on AttributeError from the real backend falling into the generic handler. 234 -> 171 mypy errors. --- openblade/api/rbac_deps.py | 6 +++- openblade/api/routes_aml_library.py | 2 +- openblade/api/routes_aml_matrix_fallback.py | 5 ++- openblade/api/routes_gateway.py | 14 ++++---- openblade/api/routes_ltfs.py | 33 ++++++++++++++---- openblade/api/routes_rbac.py | 2 +- openblade/api/routes_storage_compat.py | 12 ++++--- openblade/catalog/repository.py | 37 ++++++++++++++++----- openblade/jobs/inventory.py | 3 +- openblade/nas/__init__.py | 2 +- openblade/nas/path_mapping.py | 4 ++- openblade/nas/protocol_gateway.py | 9 ++--- openblade/nas/types.py | 4 +-- 13 files changed, 96 insertions(+), 37 deletions(-) diff --git a/openblade/api/rbac_deps.py b/openblade/api/rbac_deps.py index 2d760da..df3202d 100644 --- a/openblade/api/rbac_deps.py +++ b/openblade/api/rbac_deps.py @@ -2,6 +2,8 @@ from __future__ import annotations +from collections.abc import Awaitable, Callable + from fastapi import Depends, HTTPException, Request from openblade.api.routes_aml_auth import require_auth @@ -56,7 +58,9 @@ def ensure_permission_for_user( raise HTTPException(status_code=403, detail="Permission denied") -def require_permission(permission: RbacPermission): +def require_permission( + permission: RbacPermission, +) -> Callable[..., Awaitable[AmlUser]]: """FastAPI dependency factory. Returns a dependency that checks the cookie-authed user has permission.""" async def _dep( diff --git a/openblade/api/routes_aml_library.py b/openblade/api/routes_aml_library.py index eb5fe70..a6d1dad 100644 --- a/openblade/api/routes_aml_library.py +++ b/openblade/api/routes_aml_library.py @@ -1293,7 +1293,7 @@ async def get_library_physical_alias( async def get_library_inventory_alias( user: AmlUser = Depends(require_auth), context: AppContext = Depends(get_context), -) -> dict: +) -> dict[str, Any]: """Backwards-compatible inventory response shape expected by UI/tests.""" _ensure_state(context) inventory = context.library.inventory() diff --git a/openblade/api/routes_aml_matrix_fallback.py b/openblade/api/routes_aml_matrix_fallback.py index 722ccbc..4ee691b 100644 --- a/openblade/api/routes_aml_matrix_fallback.py +++ b/openblade/api/routes_aml_matrix_fallback.py @@ -2,6 +2,7 @@ from __future__ import annotations +from collections.abc import Awaitable, Callable from typing import Any from fastapi import Depends, FastAPI, Request @@ -78,7 +79,9 @@ def _result_payload(summary: str) -> dict[str, Any]: } -def _build_endpoint(method: str, path_template: str, operation_class: str): +def _build_endpoint( + method: str, path_template: str, operation_class: str +) -> Callable[..., Awaitable[dict[str, Any]]]: async def endpoint( request: Request, current_user: AmlUser = Depends(require_auth), diff --git a/openblade/api/routes_gateway.py b/openblade/api/routes_gateway.py index 3fb0add..b88cc7d 100644 --- a/openblade/api/routes_gateway.py +++ b/openblade/api/routes_gateway.py @@ -2,6 +2,8 @@ from __future__ import annotations +from typing import Any + from fastapi import APIRouter, Depends, HTTPException from pydantic import BaseModel @@ -54,7 +56,7 @@ class InboxPathOption(BaseModel): @router.get("/config", response_model=GatewayConfigResponse, dependencies=[Depends(require_auth)]) -def get_gateway_config(): +def get_gateway_config() -> dict[str, Any]: return get_gateway().config @@ -89,12 +91,12 @@ def stop_gateway() -> GatewayCommandResponse: @router.get("/credentials", dependencies=[Depends(require_auth)]) -def list_credentials(): +def list_credentials() -> list[dict[str, Any]]: return get_gateway().list_credentials() @router.post("/credentials", dependencies=[Depends(require_auth)]) -def add_credential(data: CredentialCreate): +def add_credential(data: CredentialCreate) -> dict[str, Any]: try: cred = get_gateway().add_credential( data.username, @@ -111,7 +113,7 @@ def add_credential(data: CredentialCreate): @router.put("/credentials/{username}", dependencies=[Depends(require_auth)]) -def update_credential(username: str, data: CredentialUpdate): +def update_credential(username: str, data: CredentialUpdate) -> dict[str, Any]: try: cred = get_gateway().update_credential( username, @@ -129,14 +131,14 @@ def update_credential(username: str, data: CredentialUpdate): @router.delete("/credentials/{username}", dependencies=[Depends(require_auth)]) -def remove_credential(username: str): +def remove_credential(username: str) -> dict[str, Any]: if not get_gateway().remove_credential(username): raise HTTPException(404, f"Credential {username!r} not found") return {"deleted": username} @router.get("/sessions", dependencies=[Depends(require_auth)]) -def list_sessions(active_only: bool = False): +def list_sessions(active_only: bool = False) -> list[dict[str, Any]]: sessions = get_gateway().list_sessions(active_only=active_only) return [ { diff --git a/openblade/api/routes_ltfs.py b/openblade/api/routes_ltfs.py index 49c45b5..9b922fd 100644 --- a/openblade/api/routes_ltfs.py +++ b/openblade/api/routes_ltfs.py @@ -4,19 +4,29 @@ import logging from datetime import datetime +from typing import Any, Protocol, runtime_checkable from fastapi import APIRouter, Depends, HTTPException, Query from pydantic import BaseModel from openblade.bootstrap import AppContext, get_context from openblade.catalog.repository import CatalogBrowseEntry +from openblade.domain.models import MountHandle logger = logging.getLogger(__name__) + +@runtime_checkable +class _SupportsToJson(Protocol): + """Optional LTFS-backend introspection used by GET /ltfs/status.""" + + def to_json(self) -> dict[str, Any]: ... + + router = APIRouter() # Simple in-memory mount registry for LTFS mounts (only for simulator/testing) -_active_mounts: dict[str, object] = {} # barcode -> MountHandle +_active_mounts: dict[str, MountHandle] = {} class LtfsBrowseEntryResponse(BaseModel): @@ -59,7 +69,9 @@ async def list_ltfs_catalog_tapes( # LTFS operational endpoints expected by i3 tests @router.post("/format") -async def ltfs_format(payload: dict, context: AppContext = Depends(get_context)) -> dict: +async def ltfs_format( + payload: dict[str, Any], context: AppContext = Depends(get_context) +) -> dict[str, Any]: barcode = payload.get("barcode") if not barcode: raise HTTPException(status_code=422, detail="barcode is required") @@ -89,7 +101,9 @@ async def ltfs_format(payload: dict, context: AppContext = Depends(get_context)) @router.post("/mount") -async def ltfs_mount(payload: dict, context: AppContext = Depends(get_context)) -> dict: +async def ltfs_mount( + payload: dict[str, Any], context: AppContext = Depends(get_context) +) -> dict[str, Any]: barcode = payload.get("barcode") if not barcode: raise HTTPException(status_code=422, detail="barcode is required") @@ -119,7 +133,9 @@ async def ltfs_mount(payload: dict, context: AppContext = Depends(get_context)) @router.post("/unmount") -async def ltfs_unmount(payload: dict, context: AppContext = Depends(get_context)) -> dict: +async def ltfs_unmount( + payload: dict[str, Any], context: AppContext = Depends(get_context) +) -> dict[str, Any]: barcode = payload.get("barcode") if not barcode: raise HTTPException(status_code=422, detail="barcode is required") @@ -136,9 +152,14 @@ async def ltfs_unmount(payload: dict, context: AppContext = Depends(get_context) @router.get("/status") -async def ltfs_status(context: AppContext = Depends(get_context)) -> dict: +async def ltfs_status(context: AppContext = Depends(get_context)) -> dict[str, Any]: # Return LTFS backend status if available try: - return context.ltfs.to_json() + # Only the simulator LTFS backend exposes to_json(); the real backend does + # not, and previously raised AttributeError into this same fallback. + backend = context.ltfs + if isinstance(backend, _SupportsToJson): + return backend.to_json() + return {"status": "unknown"} except Exception: return {"status": "unknown"} diff --git a/openblade/api/routes_rbac.py b/openblade/api/routes_rbac.py index ab55e04..c990701 100644 --- a/openblade/api/routes_rbac.py +++ b/openblade/api/routes_rbac.py @@ -216,7 +216,7 @@ async def create_token( return service.create_token(rbac_user.id, payload) -@router.get("/tokens", response_model=list[dict]) +@router.get("/tokens", response_model=list[dict[str, object]]) async def list_tokens( current_user: AmlUser = Depends(require_permission(RbacPermission.TOKEN_MANAGE)), service: RbacService = Depends(get_rbac_service), diff --git a/openblade/api/routes_storage_compat.py b/openblade/api/routes_storage_compat.py index f27fda1..dee9821 100644 --- a/openblade/api/routes_storage_compat.py +++ b/openblade/api/routes_storage_compat.py @@ -1,9 +1,11 @@ from __future__ import annotations +from typing import Any + from fastapi import APIRouter, Depends, HTTPException from pydantic import BaseModel -from openblade.bootstrap import get_context +from openblade.bootstrap import AppContext, get_context router = APIRouter() @@ -16,7 +18,7 @@ class ArchivePlanningResponse(BaseModel): @router.post("/storage/archive-planning") async def storage_archive_planning( - payload: dict, context=Depends(get_context) + payload: dict[str, Any], context: AppContext = Depends(get_context) ) -> ArchivePlanningResponse: # Lightweight compatibility shim for the UI tests. # Return an empty plan when no actionable input provided. @@ -25,13 +27,15 @@ async def storage_archive_planning( @router.get("/storage/restore-queue") -async def storage_restore_queue(context=Depends(get_context)) -> list: +async def storage_restore_queue(context: AppContext = Depends(get_context)) -> list[Any]: # Compatibility endpoint: surface an empty restore queue if backend not configured. return [] @router.post("/restore/plan") -async def restore_plan_compat(payload: dict, context=Depends(get_context)) -> dict: +async def restore_plan_compat( + payload: dict[str, Any], context: AppContext = Depends(get_context) +) -> dict[str, Any]: # Tests expect this endpoint to exist and return either 200/202/422. # If pool_id missing, return 422 to indicate validation. if not payload or ( diff --git a/openblade/catalog/repository.py b/openblade/catalog/repository.py index c26521c..3126075 100644 --- a/openblade/catalog/repository.py +++ b/openblade/catalog/repository.py @@ -4,9 +4,11 @@ import json import threading +from collections.abc import Mapping, Sequence from dataclasses import dataclass from datetime import datetime from pathlib import PurePosixPath +from typing import Any, Protocol, cast, runtime_checkable from uuid import uuid4 from sqlalchemy import func, select @@ -75,10 +77,29 @@ } +@runtime_checkable +class _SupportsModelDump(Protocol): + """Pydantic v2 model surface this module relies on.""" + + def model_dump(self, *, mode: str = ...) -> dict[str, Any]: ... + + +class _FileRecordLike(Protocol): + """The attributes ``save_file_record`` reads off a domain file record.""" + + path: str + size_bytes: int + checksum_sha256: str + volume_group_id: str + + def _model_to_json_dict(model: object) -> dict[str, object]: - if hasattr(model, "model_dump"): + if isinstance(model, _SupportsModelDump): return model.model_dump(mode="json") - return model.dict() + # pydantic v1 models (and test doubles) still expose .dict(); an object with + # neither raises AttributeError here exactly as it did before. + legacy_dict = getattr(model, "dict") # noqa: B009 + return cast("dict[str, object]", legacy_dict()) def _load_json_value(value: str | None, default: object) -> object: @@ -327,12 +348,12 @@ def _tape_op_to_dict(self, row: TapeOpLog) -> dict[str, object]: "completed_at": row.completed_at, } - def __getattribute__(self, name: str): + def __getattribute__(self, name: str) -> Any: attr = object.__getattribute__(self, name) if name.startswith("_") or name == "session" or not callable(attr): return attr - def _locked(*args, **kwargs): + def _locked(*args: Any, **kwargs: Any) -> Any: with object.__getattribute__(self, "_lock"): return attr(*args, **kwargs) @@ -374,7 +395,7 @@ def list_library_instances(self) -> list[LibraryInstance]: stmt = select(LibraryInstance).order_by(LibraryInstance.sort_order, LibraryInstance.name) return list(self.session.execute(stmt).scalars().all()) - def update_library_instance(self, library_id: int, **kwargs) -> LibraryInstance | None: + def update_library_instance(self, library_id: int, **kwargs: object) -> LibraryInstance | None: library = self.get_library_instance(library_id) if library is None: return None @@ -669,7 +690,7 @@ def add_barcode_to_volume_group(self, volume_group_id: str, barcode: str) -> Car def save_file_record( self, - record: object, + record: _FileRecordLike, barcode: str, tape_path: PurePosixPath, state: FileInstanceState, @@ -1288,7 +1309,7 @@ def count_path_mappings(self, pool_id: str = "", dataset_id: str = "") -> int: stmt = stmt.where(PathMapping.dataset_id == dataset_id) return int(self.session.execute(stmt).scalar_one()) - def create_role(self, role: dict[str, object]) -> dict[str, object]: + def create_role(self, role: Mapping[str, object]) -> dict[str, object]: """Create or replace an RBAC role record.""" now = _utcnow_iso() parsed = RbacRoleRecord.model_validate( @@ -1686,7 +1707,7 @@ def list_tape_ops( def seed_default_roles(self) -> None: """Create built-in roles if they don't exist: admin, operator, readonly.""" all_permissions = [permission.value for permission in RbacPermission] - defaults = [ + defaults: list[dict[str, Sequence[str] | str]] = [ { "id": "admin", "name": "admin", diff --git a/openblade/jobs/inventory.py b/openblade/jobs/inventory.py index 546924e..fd90d1a 100644 --- a/openblade/jobs/inventory.py +++ b/openblade/jobs/inventory.py @@ -4,13 +4,14 @@ from openblade.catalog.repository import CatalogRepository from openblade.domain.backends import LibraryBackend +from openblade.domain.models import LibraryInventory class InventoryService: def __init__(self, library: LibraryBackend) -> None: self.library = library - def snapshot(self): + def snapshot(self) -> LibraryInventory: return self.library.inventory() diff --git a/openblade/nas/__init__.py b/openblade/nas/__init__.py index 97559c6..6bd7422 100644 --- a/openblade/nas/__init__.py +++ b/openblade/nas/__init__.py @@ -16,7 +16,7 @@ ] -def __getattr__(name: str): +def __getattr__(name: str) -> object: if name == "NasService": from openblade.nas.service import NasService diff --git a/openblade/nas/path_mapping.py b/openblade/nas/path_mapping.py index 97383aa..ba174e4 100644 --- a/openblade/nas/path_mapping.py +++ b/openblade/nas/path_mapping.py @@ -2,6 +2,8 @@ from __future__ import annotations +from typing import Any + from openblade.catalog.repository import CatalogRepository from openblade.nas.types import ( NasFileState, @@ -94,7 +96,7 @@ def list_tapes_for_pool(self, pool_id: str) -> list[str]: tapes.update(barcode for barcode in record.all_barcodes if barcode) return sorted(tapes) - def get_stats(self, pool_id: str = "", dataset_id: str = "") -> dict: + def get_stats(self, pool_id: str = "", dataset_id: str = "") -> dict[str, Any]: """ Return stats dict: total_files, total_bytes, by_state (dict[state, count]), tape_count (distinct barcodes), pool_id, dataset_id. diff --git a/openblade/nas/protocol_gateway.py b/openblade/nas/protocol_gateway.py index 0ea6aec..515bb4d 100644 --- a/openblade/nas/protocol_gateway.py +++ b/openblade/nas/protocol_gateway.py @@ -17,6 +17,7 @@ from dataclasses import dataclass, field from datetime import datetime from enum import Enum +from typing import Any class GatewayStatus(str, Enum): @@ -145,7 +146,7 @@ class GatewaySession: class ProtocolGateway: """In-process protocol gateway manager for emulated SFTP/SCP ingest.""" - def __init__(self): + def __init__(self) -> None: self._credentials: dict[str, GatewayCredential] = {} self._sessions: list[GatewaySession] = [] self._last_error: str | None = None @@ -164,7 +165,7 @@ def status(self) -> GatewayStatus: return self._status @property - def config(self) -> dict: + def config(self) -> dict[str, Any]: return { "bind_host": self._bind_host, "bind_port": self._bind_port, @@ -293,7 +294,7 @@ def list_sessions(self, active_only: bool = False) -> list[GatewaySession]: return [s for s in self._sessions if s.disconnected_at is None] return list(self._sessions) - def list_credentials(self) -> list[dict]: + def list_credentials(self) -> list[dict[str, Any]]: return [ { "username": c.username, @@ -347,7 +348,7 @@ def set_error(self, message: str) -> None: self._last_error = message self._status = GatewayStatus.ERROR - def get_stats(self) -> dict: + def get_stats(self) -> dict[str, Any]: sessions = self._sessions active = [s for s in sessions if s.disconnected_at is None] return { diff --git a/openblade/nas/types.py b/openblade/nas/types.py index b39a4b7..97d5225 100644 --- a/openblade/nas/types.py +++ b/openblade/nas/types.py @@ -263,7 +263,7 @@ class RbacAuditEventRecord(BaseModel): resource: str = "" action: str = "" outcome: str = "" - details: dict = Field(default_factory=dict) + details: dict[str, Any] = Field(default_factory=dict) created_at: str ip_address: str | None = None @@ -335,7 +335,7 @@ def validate_name(cls, value: object) -> str: class SidecarValidationError(Exception): - def __init__(self, message: str, field: str = None, raw_value=None): + def __init__(self, message: str, field: str | None = None, raw_value: object = None) -> None: super().__init__(message) self.field = field self.raw_value = raw_value From dc934ba17fa5f939acb7a36e1c7c4fdc4b9bfc78 Mon Sep 17 00:00:00 2001 From: Amantux Date: Sat, 26 Sep 2026 06:22:25 +0000 Subject: [PATCH 23/54] types: narrow the backend protocols, mount handles and catalog-row reads Bucket 3: annotations plus narrowings the runtime already guaranteed. No behaviour change; every narrowing is either a `cast` (no runtime effect) or an isinstance guard on a branch that previously reached the same outcome. - domain/backends.py: LTFSBackend gains `write_bytes`, which both backends already implement and callers already reached through this protocol. Positional-only with the wider `FileInstance | None` return so the real and simulator signatures both conform. - nas/ingest.py, nas/hydration.py: parameters typed `MockLibraryBackend` / `MockLTFSBackend` now take the LibraryBackend/LTFSBackend protocols. They only ever used protocol methods, and nas_config already passed `context.ltfs` -- which is the REAL backend in a real-hardware deployment, so the Mock annotations were simply wrong. - Mount registries typed `dict[str, MountHandle]` instead of `object` (routes_ltfs, nas_config verify, sharded_archive) so `unmount`/`stat`/ `write_file` type-check without a cast. - nas_config: `_as_int` for decoded-JSON drive fields (raises TypeError exactly where `int()` did); loop variables renamed where one name held two types. - nas/catalog_rebuild.py: `_text_column`/`_int_column` are cast-only helpers for columns read back out of serialized `dict[str, object]` rows -- validation stays in the pydantic model one line down, so no new failure mode. - api/main.py: middleware `call_next` typed `RequestResponseEndpoint` from starlette, removing eleven `# type: ignore[operator]` comments. - api/aml_state.py: bound `.get()` results before their isinstance checks so they narrow, widened `_next_media_slot_address` to accept the `None` partition that imported media legitimately carry, and renamed a `partition` variable that shadowed the partition *dict*. 171 -> 24 mypy errors (remaining ones are in files handled separately). One `# type: ignore[method-assign]` added, on `app.openapi = custom_openapi` -- the documented FastAPI "Extending OpenAPI" idiom, with no subclass-based alternative that keeps `app` a module-level instance. --- openblade/api/aml_state.py | 33 +++++++--------- openblade/api/main.py | 57 ++++++++++++++++++---------- openblade/api/nas_config.py | 62 ++++++++++++++++++++----------- openblade/domain/backends.py | 16 ++++++++ openblade/jobs/sharded_archive.py | 14 +++---- openblade/nas/catalog_rebuild.py | 45 ++++++++++++++-------- openblade/nas/hydration.py | 4 +- openblade/nas/ingest.py | 37 +++++++++--------- 8 files changed, 165 insertions(+), 103 deletions(-) diff --git a/openblade/api/aml_state.py b/openblade/api/aml_state.py index a4fc941..5ea78a1 100644 --- a/openblade/api/aml_state.py +++ b/openblade/api/aml_state.py @@ -13,7 +13,7 @@ from copy import deepcopy from dataclasses import dataclass, field from datetime import datetime, timedelta, timezone -from typing import Any +from typing import Any, cast from uuid import uuid4 import pyotp @@ -2678,7 +2678,7 @@ def update_eth_port(blade_id: str, port_id: str, updates: dict[str, Any]) -> dic for port in blade.get("ports", []): if port.get("id") == port_id: port.update(deepcopy(updates)) - return deepcopy(port) + return cast("dict[str, Any]", deepcopy(port)) return None @@ -2720,7 +2720,7 @@ def update_fc_port(blade_id: str, port_id: str, updates: dict[str, Any]) -> dict for port in blade.get("ports", []): if port.get("id") == port_id: port.update(deepcopy(updates)) - return deepcopy(port) + return cast("dict[str, Any]", deepcopy(port)) return None @@ -3468,8 +3468,8 @@ def remove_aml_partition_access_value(name: str, field: str, value: str) -> bool def _sync_aml_media_counts() -> None: partition_counts: dict[str, int] = {} for media in _STATE.aml_media.values(): - partition = str(media.get("partition", "partition1")) - partition_counts[partition] = partition_counts.get(partition, 0) + 1 + partition_name = str(media.get("partition", "partition1")) + partition_counts[partition_name] = partition_counts.get(partition_name, 0) + 1 for name, partition in _STATE.aml_partitions.items(): media_count = partition_counts.get(name, 0) partition["mediaCount"] = media_count @@ -3575,7 +3575,7 @@ def delete_aml_media(barcode: str) -> bool: return True -def _next_media_slot_address(partition: str) -> str: +def _next_media_slot_address(partition: str | None) -> str: slot_numbers: list[int] = [] for media in _STATE.aml_media.values(): if media.get("partition") != partition: @@ -3721,18 +3721,14 @@ def assign_aml_media_to_pool(pool_id: str, barcodes: list[str]) -> dict[str, Any return get_aml_media_pool(normalized_pool_id) for candidate in _STATE.aml_media_pools.values(): - existing = ( - candidate.get("assignedBarcodes") - if isinstance(candidate.get("assignedBarcodes"), list) - else [] - ) + candidate_barcodes = candidate.get("assignedBarcodes") + existing = candidate_barcodes if isinstance(candidate_barcodes, list) else [] candidate["assignedBarcodes"] = [ barcode for barcode in existing if barcode not in normalized_barcodes ] - assigned = ( - pool.get("assignedBarcodes") if isinstance(pool.get("assignedBarcodes"), list) else [] - ) + pool_barcodes = pool.get("assignedBarcodes") + assigned = pool_barcodes if isinstance(pool_barcodes, list) else [] for barcode in normalized_barcodes: if barcode not in assigned: assigned.append(barcode) @@ -3748,9 +3744,8 @@ def unassign_aml_media_from_pool(pool_id: str, barcodes: list[str]) -> dict[str, return None normalized_barcodes = set(barcodes) - existing = ( - pool.get("assignedBarcodes") if isinstance(pool.get("assignedBarcodes"), list) else [] - ) + pool_barcodes = pool.get("assignedBarcodes") + existing = pool_barcodes if isinstance(pool_barcodes, list) else [] pool["assignedBarcodes"] = [ barcode for barcode in existing if barcode not in normalized_barcodes ] @@ -4572,7 +4567,7 @@ def get_fc_port_by_number(serial_number: str, port_number: int) -> dict[str, Any return None for port in blade.get("ports", []): if int(port.get("portNumber", 0)) == port_number: - return port + return cast("dict[str, Any]", port) return None @@ -4585,7 +4580,7 @@ def update_fc_port_by_number( for port in blade.get("ports", []): if int(port.get("portNumber", 0)) == port_number: port.update(deepcopy(updates)) - return deepcopy(port) + return cast("dict[str, Any]", deepcopy(port)) return None return None diff --git a/openblade/api/main.py b/openblade/api/main.py index 4b4f91e..a9419c2 100644 --- a/openblade/api/main.py +++ b/openblade/api/main.py @@ -13,6 +13,7 @@ from fastapi.openapi.utils import get_openapi from fastapi.responses import JSONResponse from pydantic import BaseModel +from starlette.middleware.base import RequestResponseEndpoint from openblade.api import ( aml_scope, @@ -112,16 +113,18 @@ class ErrorResponse(BaseModel): @app.middleware("http") -async def apply_forwarded_root_path(request: Request, call_next: object) -> Response: +async def apply_forwarded_root_path( + request: Request, call_next: RequestResponseEndpoint +) -> Response: forwarded_prefix = request.headers.get("x-forwarded-prefix", "").strip() if forwarded_prefix.startswith("/"): request.scope["root_path"] = forwarded_prefix.rstrip("/") - return await call_next(request) # type: ignore[operator] + return await call_next(request) @app.middleware("http") -async def add_security_headers(request: Request, call_next: object) -> Response: - response: Response = await call_next(request) # type: ignore[operator] +async def add_security_headers(request: Request, call_next: RequestResponseEndpoint) -> Response: + response: Response = await call_next(request) response.headers["X-Content-Type-Options"] = "nosniff" if request.url.path.startswith("/docs") or request.url.path.startswith("/redoc"): response.headers["X-Frame-Options"] = "SAMEORIGIN" @@ -135,16 +138,20 @@ async def add_security_headers(request: Request, call_next: object) -> Response: @app.middleware("http") -async def bind_active_library_context(request: Request, call_next: object) -> Response: +async def bind_active_library_context( + request: Request, call_next: RequestResponseEndpoint +) -> Response: token = library_context.set_active_library_id(request.headers.get("X-OpenBlade-Library-Id", "")) try: - return await call_next(request) # type: ignore[operator] + return await call_next(request) finally: library_context.reset_active_library_id(token) @app.middleware("http") -async def apply_iblade_strict_uri_gateway(request: Request, call_next: object) -> Response: +async def apply_iblade_strict_uri_gateway( + request: Request, call_next: RequestResponseEndpoint +) -> Response: context = get_context() compat_mode = getattr(context.config, "iblade_compat_mode", "extended") strict_interface = str(getattr(compat_mode, "value", compat_mode)).lower() == "strict" @@ -160,26 +167,28 @@ async def apply_iblade_strict_uri_gateway(request: Request, call_next: object) - content=_aml_error_payload(exc.status_code, exc.detail), ) if alias is None: - return await call_next(request) # type: ignore[operator] + return await call_next(request) blade_type, section_number, strict_alias_path = alias request.scope["path"] = strict_alias_path request.scope["raw_path"] = strict_alias_path.encode("utf-8") request.scope["openblade_blade_type"] = blade_type request.scope["openblade_blade_section_number"] = section_number - return await call_next(request) # type: ignore[operator] + return await call_next(request) @app.middleware("http") -async def apply_aml_emulator_latency(request: Request, call_next: object) -> Response: +async def apply_aml_emulator_latency( + request: Request, call_next: RequestResponseEndpoint +) -> Response: if not should_capture_latency_metrics(request.url.path): - return await call_next(request) # type: ignore[operator] + return await call_next(request) start = time.perf_counter() simulated_delay = await apply_request_latency(request) status_code = 500 try: - response: Response = await call_next(request) # type: ignore[operator] + response: Response = await call_next(request) status_code = response.status_code return response finally: @@ -290,21 +299,23 @@ def _is_aml_compatible_path(path: str) -> bool: @app.middleware("http") -async def enforce_scalar_api_scope(request: Request, call_next: object) -> Response: +async def enforce_scalar_api_scope( + request: Request, call_next: RequestResponseEndpoint +) -> Response: if request.method.upper() == "OPTIONS": - return await call_next(request) # type: ignore[operator] + return await call_next(request) context = get_context() if not context.config.scalar_api_only: - return await call_next(request) # type: ignore[operator] + return await call_next(request) path = aml_scope.normalize_aml_path(request.url.path) if path in _STRICT_NON_MATRIX_ALLOWED_PATHS: - return await call_next(request) # type: ignore[operator] + return await call_next(request) if path.startswith("/aml") or path.startswith("/iblade"): if aml_scope.is_matrix_endpoint(request.method, path): - return await call_next(request) # type: ignore[operator] + return await call_next(request) return JSONResponse( status_code=404, content=_aml_error_payload(404, "Endpoint not available in matrix scope"), @@ -351,7 +362,7 @@ def _aml_error_payload(status_code: int, detail: object) -> dict[str, object]: @app.exception_handler(HTTPException) -async def handle_http_exception(request: Request, exc: HTTPException): +async def handle_http_exception(request: Request, exc: HTTPException) -> Response: if not _is_aml_compatible_path(request.url.path): return await http_exception_handler(request, exc) return JSONResponse( @@ -360,7 +371,9 @@ async def handle_http_exception(request: Request, exc: HTTPException): @app.exception_handler(RequestValidationError) -async def handle_request_validation_error(request: Request, exc: RequestValidationError): +async def handle_request_validation_error( + request: Request, exc: RequestValidationError +) -> Response: if not _is_aml_compatible_path(request.url.path): return await request_validation_exception_handler(request, exc) description = ( @@ -496,4 +509,8 @@ def custom_openapi() -> dict[str, object]: return schema -app.openapi = custom_openapi +# The documented FastAPI idiom for overriding the schema builder is to replace +# the bound method on the instance (see fastapi.applications.FastAPI.openapi and +# the "Extending OpenAPI" guide). mypy flags any method assignment; there is no +# subclass-based alternative that keeps `app` a module-level FastAPI instance. +app.openapi = custom_openapi # type: ignore[method-assign] # documented override idiom diff --git a/openblade/api/nas_config.py b/openblade/api/nas_config.py index 1de6050..9263245 100644 --- a/openblade/api/nas_config.py +++ b/openblade/api/nas_config.py @@ -11,10 +11,12 @@ from pydantic import BaseModel, ValidationError from openblade.api import aml_state -from openblade.api.routes_aml_auth import AmlUser, require_auth +from openblade.api.routes_aml_auth import require_auth from openblade.bootstrap import get_context from openblade.catalog.db import get_catalog_repository -from openblade.domain.models import MountMode +from openblade.catalog.models import AmlUser +from openblade.catalog.repository import CatalogRepository +from openblade.domain.models import MountHandle, MountMode from openblade.nas.catalog_rebuild import CatalogRebuildPlanner from openblade.nas.catalog_rebuild_worker import SAFE_REBUILD_PREFLIGHT_ERROR, CatalogRebuildWorker from openblade.nas.catalog_shard import CatalogShardWriter @@ -47,6 +49,7 @@ EffectivePolicy, IngestMode, ManifestVersionRecord, + NasDataset, NasFileRecord, NasFileState, NasPool, @@ -71,16 +74,18 @@ _FUSE_HOOKS: dict[int, FuseHook] = {} -def get_nas_service(repo=Depends(get_catalog_repository)) -> NasService: +def get_nas_service(repo: CatalogRepository = Depends(get_catalog_repository)) -> NasService: return NasService(repo) -def get_path_mapping_service(repo=Depends(get_catalog_repository)) -> PathMappingService: +def get_path_mapping_service( + repo: CatalogRepository = Depends(get_catalog_repository), +) -> PathMappingService: return PathMappingService(repo) def get_catalog_rebuild_planner( - repo=Depends(get_catalog_repository), + repo: CatalogRepository = Depends(get_catalog_repository), ) -> CatalogRebuildPlanner: metadata_writer = TapeMetadataWriter(get_context().ltfs) shard_writer = CatalogShardWriter(metadata_writer) @@ -95,13 +100,13 @@ def get_catalog_rebuild_planner( def get_catalog_rebuild_worker( - repo=Depends(get_catalog_repository), + repo: CatalogRepository = Depends(get_catalog_repository), planner: CatalogRebuildPlanner = Depends(get_catalog_rebuild_planner), ) -> CatalogRebuildWorker: return CatalogRebuildWorker(repo=repo, planner=planner) -def _loaded_tape_barcodes(repo) -> list[str]: +def _loaded_tape_barcodes(repo: CatalogRepository) -> list[str]: seen: set[str] = set() barcodes: list[str] = [] for cartridge in repo.list_cartridges(): @@ -162,7 +167,7 @@ def _require_restore_job(service: NasService, job_id: str) -> NasRestoreJob: return job -def _require_dataset(service: NasService, dataset_id: str): +def _require_dataset(service: NasService, dataset_id: str) -> NasDataset: dataset = service.get_dataset(dataset_id) if dataset is None: raise HTTPException( @@ -237,15 +242,28 @@ def _utc_timestamp() -> str: return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z") +def _as_int(value: object) -> int: + """``int(value)`` over a decoded-JSON drive field, with identical failures. + + The drive dicts come from ``aml_state`` as ``dict[str, object]`` and ``int()`` + has no overload for ``object``. This narrows to the types ``int()`` accepts + and raises TypeError for the rest -- exactly what ``int()`` itself did -- so + every success and failure path is unchanged. + """ + if isinstance(value, int | float | str | bytes | bytearray): + return int(value) + raise TypeError(f"expected an int-like value, got {type(value).__name__}") + + def _drive_needs_cleaning(drive: dict[str, object]) -> bool: if bool(drive.get("cleaningRequired", False)): return True state = str(drive.get("state", "")).lower() if state in {"cleaning_required", "cleaning-required", "needs_cleaning"}: return True - threshold = int(drive.get("cleaningThreshold", 100)) - load_count = int(drive.get("loadCount", 0)) - cleaning_count = int(drive.get("cleaningCount", 0)) + threshold = _as_int(drive.get("cleaningThreshold", 100)) + load_count = _as_int(drive.get("loadCount", 0)) + cleaning_count = _as_int(drive.get("cleaningCount", 0)) return load_count > 0 and (load_count - cleaning_count * 50) >= threshold @@ -379,7 +397,7 @@ async def verify_dataset( dataset = _require_dataset(service, dataset_id) context = get_context() ltfs = context.ltfs - mounts: dict[str, tuple[object, int, int | None]] = {} + mounts: dict[str, tuple[MountHandle, int, int | None]] = {} files_verified = 0 files_corrupt = 0 files_updated = 0 @@ -423,10 +441,12 @@ async def verify_dataset( files_corrupt += 1 service.upsert_file_record(record.model_copy(update={"status": NasFileState.CORRUPT})) finally: - for handle, drive_id, slot_id in mounts.values(): - ltfs.unmount(handle) - if slot_id is not None: - context.library.unload(drive_id, slot_id) + # Distinct names: `handle` above holds the whole (handle, drive, slot) + # tuple, so reusing it here gave the unpacked mount handle two types. + for mount_handle, mount_drive_id, mount_slot_id in mounts.values(): + ltfs.unmount(mount_handle) + if mount_slot_id is not None: + context.library.unload(mount_drive_id, mount_slot_id) return { "dataset_id": dataset_id, @@ -546,7 +566,7 @@ async def plan_catalog_rebuild( @router.post("/catalog/rebuild/activate", response_model=RebuildActivationResult) async def activate_catalog_rebuild( request: RebuildActivationRequest, - repo=Depends(get_catalog_repository), + repo: CatalogRepository = Depends(get_catalog_repository), planner: CatalogRebuildPlanner = Depends(get_catalog_rebuild_planner), worker: CatalogRebuildWorker = Depends(get_catalog_rebuild_worker), _: AmlUser = Depends(require_auth), @@ -598,7 +618,7 @@ async def activate_catalog_rebuild( @router.get("/catalog/rebuild/loaded-tapes", response_model=list[str]) async def list_catalog_rebuild_loaded_tapes( - repo=Depends(get_catalog_repository), + repo: CatalogRepository = Depends(get_catalog_repository), _: AmlUser = Depends(require_auth), ) -> list[str]: return _loaded_tape_barcodes(repo) @@ -623,7 +643,7 @@ async def execute_catalog_rebuild( @router.get("/catalog/rebuild/runs", response_model=list[CatalogRebuildRunRecord]) async def list_catalog_rebuild_runs( limit: int = Query(default=50, ge=1, le=500), - repo=Depends(get_catalog_repository), + repo: CatalogRepository = Depends(get_catalog_repository), _: AmlUser = Depends(require_auth), ) -> list[CatalogRebuildRunRecord]: return [CatalogRebuildRunRecord.model_validate(run) for run in repo.list_rebuild_runs(limit)] @@ -632,7 +652,7 @@ async def list_catalog_rebuild_runs( @router.get("/catalog/rebuild/{run_id}", response_model=CatalogRebuildRunRecord) async def get_catalog_rebuild_run( run_id: str, - repo=Depends(get_catalog_repository), + repo: CatalogRepository = Depends(get_catalog_repository), _: AmlUser = Depends(require_auth), ) -> CatalogRebuildRunRecord: run = repo.get_rebuild_run(run_id) @@ -644,7 +664,7 @@ async def get_catalog_rebuild_run( @router.get("/catalog/manifest-versions/{barcode}", response_model=list[ManifestVersionRecord]) async def list_catalog_manifest_versions( barcode: str, - repo=Depends(get_catalog_repository), + repo: CatalogRepository = Depends(get_catalog_repository), _: AmlUser = Depends(require_auth), ) -> list[ManifestVersionRecord]: return [ diff --git a/openblade/domain/backends.py b/openblade/domain/backends.py index bad7c3d..f645382 100644 --- a/openblade/domain/backends.py +++ b/openblade/domain/backends.py @@ -68,3 +68,19 @@ def remaining_capacity(self, barcode: str) -> int: ... def read_bytes( self, barcode_or_path: str, path: PurePosixPath | str | None = None ) -> bytes | None: ... + # Positional-only, returning the wider `FileInstance | None`, so both + # implementations conform: the real backend names the first parameter + # `handle` and always returns a FileInstance; the simulator names it + # `handle_or_barcode` (it also accepts a bare barcode for metadata writes) + # and returns None for that form. Declared here because callers already + # reach it through this protocol. + def write_bytes( + self, + handle: MountHandle, + dest: PurePosixPath | str, + content: bytes, + /, + *, + size_bytes: int | None = None, + checksum_sha256: str | None = None, + ) -> FileInstance | None: ... diff --git a/openblade/jobs/sharded_archive.py b/openblade/jobs/sharded_archive.py index 807576d..9ee07b8 100644 --- a/openblade/jobs/sharded_archive.py +++ b/openblade/jobs/sharded_archive.py @@ -15,7 +15,7 @@ from openblade.domain.backends import LibraryBackend, LTFSBackend from openblade.domain.capacity import has_room_for from openblade.domain.errors import TapeFullError, safe_job_error -from openblade.domain.models import MountMode +from openblade.domain.models import MountHandle, MountMode from openblade.jobs.scheduler import DriveHandle, DriveScheduler from openblade.jobs.shard import ( DEFAULT_BLOCK_SIZE, @@ -261,7 +261,7 @@ def _clean_unmount_and_unload( catalog: CatalogRepository, library: LibraryBackend, ltfs: LTFSBackend, - mounts: dict[str, object], + mounts: dict[str, MountHandle], handles: list[DriveHandle], loaded_slots: dict[int, int | None], job_id: str, @@ -342,7 +342,7 @@ def _archive_stripe( for batch in batches: batch_barcodes = list(dict.fromkeys(barcode for _, barcode in batch)) handles = scheduler.acquire_drives(batch_barcodes) - mounts: dict[str, object] = {} + mounts: dict[str, MountHandle] = {} loaded_slots: dict[int, int | None] = {} # (main_instance_id, shard_instance_id, size_bytes) for shards written+verified # this batch. Nothing here is marked archived until the WHOLE batch has verified @@ -357,7 +357,7 @@ def _archive_stripe( def _write_one( source_file: Path, barcode: str, - mount: object, + mount: MountHandle, ) -> tuple[Path, str, str, str, int]: tape_path = _stripe_tape_path(source_file, request.source_path) _require_lane_room(ltfs, barcode, source_file.stat().st_size) @@ -479,7 +479,7 @@ def _archive_block_stripe( ) shard_group_ids.append(plan.shard_group_id) handles = scheduler.acquire_drives(request.lane_barcodes) - mounts: dict[str, object] = {} + mounts: dict[str, MountHandle] = {} loaded_slots: dict[int, int | None] = {} shard_dir = scratch_dir / plan.shard_group_id shard_dir.mkdir(parents=True, exist_ok=True) @@ -524,8 +524,8 @@ def _write_shard(spec: ShardSpec, shard_tmp: Path) -> tuple[int, str, int]: pool.submit(_write_shard, spec, shard_tmp_files[spec.shard_index]) for spec in plan.shards ] - for future in concurrent.futures.as_completed(futures): - shard_index, checksum, shard_size = future.result() + for shard_future in concurrent.futures.as_completed(futures): + shard_index, checksum, shard_size = shard_future.result() shard_checksums[shard_index] = checksum shard_sizes[shard_index] = shard_size diff --git a/openblade/nas/catalog_rebuild.py b/openblade/nas/catalog_rebuild.py index e70900b..b8a18b8 100644 --- a/openblade/nas/catalog_rebuild.py +++ b/openblade/nas/catalog_rebuild.py @@ -3,6 +3,7 @@ from __future__ import annotations from datetime import datetime +from typing import cast from uuid import uuid4 from openblade.catalog.repository import CatalogRepository @@ -31,6 +32,18 @@ def _utcnow_iso() -> str: return datetime.utcnow().isoformat() + "Z" +# The catalog rows below come back as `dict[str, object]` (serialized from the +# same pydantic models that are rebuilt from them), so every column read needs a +# type. These are `cast` only -- no runtime narrowing, no new failure mode: +# validation stays exactly where it was, in the model construction one line down. +def _text_column(value: object) -> str | None: + return cast("str | None", value) + + +def _int_column(value: object) -> int | None: + return cast("int | None", value) + + class CatalogRebuildPlanner: """ Plans and executes a catalog rebuild from on-tape metadata. @@ -222,7 +235,7 @@ def _recover_datasets(self, barcode: str, shard: CatalogShard) -> int: for entry in shard.datasets: existing = self.repo.get_nas_dataset(entry.dataset_id) or {} is_new = not existing - existing_tape_set: list[str] = list(existing.get("tape_set") or []) + existing_tape_set: list[str] = list(cast("list[str]", existing.get("tape_set") or [])) tape_set = list(entry.tape_set or [barcode]) # Merge: add this barcode and any from existing without duplicates. merged = list(existing_tape_set) @@ -231,22 +244,22 @@ def _recover_datasets(self, barcode: str, shard: CatalogShard) -> int: merged.append(bc) dataset = NasDataset( id=entry.dataset_id, - pool_id=entry.pool_id or existing.get("pool_id"), + pool_id=entry.pool_id or _text_column(existing.get("pool_id")), name=str(existing.get("name") or entry.dataset_id), - source_path=existing.get("source_path"), - source_host=existing.get("source_host"), - policy_id=entry.policy or existing.get("policy_id"), + source_path=_text_column(existing.get("source_path")), + source_host=_text_column(existing.get("source_host")), + policy_id=entry.policy or _text_column(existing.get("policy_id")), ingest_mode=self._parse_ingest_mode(entry.ingest_mode) - or existing.get("ingest_mode"), - volume_group_id=entry.volume_group or existing.get("volume_group_id"), + or cast("IngestMode | None", existing.get("ingest_mode")), + volume_group_id=entry.volume_group or _text_column(existing.get("volume_group_id")), tape_set=merged, - shard_map=dict(existing.get("shard_map") or {}), + shard_map=dict(cast("dict[str, list[str]]", existing.get("shard_map") or {})), file_count=entry.file_count, total_bytes=entry.total_bytes, status=DatasetStatus.ARCHIVED, copies_completed=len(merged), manifest_path="/.openblade/manifest.json", - created_at=existing.get("created_at"), + created_at=_text_column(existing.get("created_at")), updated_at=_utcnow_iso(), ) self.repo.upsert_nas_dataset(dataset.model_dump(mode="json")) @@ -261,21 +274,21 @@ def _recover_files(self, barcode: str, shard: CatalogShard) -> int: existing = self.repo.get_nas_file_record(entry.file_record_id) or {} is_new = not existing # Preserve existing tape_barcode when file already catalogued elsewhere. - tape_barcode = existing.get("tape_barcode") or barcode + tape_barcode = _text_column(existing.get("tape_barcode")) or barcode file_record = NasFileRecord( id=entry.file_record_id, dataset_id=entry.dataset_id, - pool_id=entry.pool_id or existing.get("pool_id"), + pool_id=entry.pool_id or _text_column(existing.get("pool_id")), relative_path=entry.logical_path, - source_path=existing.get("source_path"), + source_path=_text_column(existing.get("source_path")), size_bytes=entry.size, - mtime=entry.mtime or existing.get("mtime"), - checksum_sha256=entry.checksum or existing.get("checksum_sha256"), + mtime=entry.mtime or _text_column(existing.get("mtime")), + checksum_sha256=entry.checksum or _text_column(existing.get("checksum_sha256")), tape_barcode=tape_barcode, - tape_offset=existing.get("tape_offset"), + tape_offset=_int_column(existing.get("tape_offset")), status=NasFileState.OFFLINE_ON_TAPE, cache_path=None, - created_at=existing.get("created_at"), + created_at=_text_column(existing.get("created_at")), updated_at=_utcnow_iso(), ) self.repo.upsert_nas_file_record(file_record.model_dump(mode="json")) diff --git a/openblade/nas/hydration.py b/openblade/nas/hydration.py index aad9d0a..041e7be 100644 --- a/openblade/nas/hydration.py +++ b/openblade/nas/hydration.py @@ -9,12 +9,12 @@ from dataclasses import dataclass, field from pathlib import Path, PurePosixPath +from openblade.domain.backends import LTFSBackend from openblade.domain.errors import ChecksumMismatchError from openblade.nas.restore_planner import RestorePlan from openblade.nas.service import NasService from openblade.nas.tape_paths import dataset_tape_path from openblade.nas.types import NasFileRecord, NasFileState, NasRestoreJob, RestoreJobStatus -from openblade.simulator.ltfs_volume import MockLTFSBackend logger = logging.getLogger(__name__) @@ -42,7 +42,7 @@ class HydrationJob: class HydrationExecutor: """Execute NAS restore jobs against the simulator without real filesystem I/O.""" - def __init__(self, service: NasService, ltfs: MockLTFSBackend) -> None: + def __init__(self, service: NasService, ltfs: LTFSBackend) -> None: self.service = service self.ltfs = ltfs diff --git a/openblade/nas/ingest.py b/openblade/nas/ingest.py index 7d7d00a..c95febb 100644 --- a/openblade/nas/ingest.py +++ b/openblade/nas/ingest.py @@ -12,7 +12,9 @@ from pydantic import BaseModel, Field -from openblade.domain.models import MountMode +from openblade.catalog.repository import CatalogRepository +from openblade.domain.backends import LibraryBackend, LTFSBackend +from openblade.domain.models import MountHandle, MountMode from openblade.domain.policies import FormatConfirmation, SafetyToken from openblade.nas.archive_lifecycle import ( ArchiveLifecycleManager, @@ -36,8 +38,6 @@ SourceStreamConfig, TapeAssignment, ) -from openblade.simulator.library import MockLibraryBackend -from openblade.simulator.ltfs_volume import MockLTFSBackend logger = logging.getLogger(__name__) @@ -127,8 +127,8 @@ def __init__( job: IngestJob, dataset: NasDataset, service: NasService, - library: MockLibraryBackend, - ltfs: MockLTFSBackend, + library: LibraryBackend, + ltfs: LTFSBackend, ) -> None: self.job = job self.dataset = dataset @@ -324,7 +324,7 @@ def _ensure_formatted(self, barcode: str) -> None: def _simulate_tape_write( self, *, - handle, + handle: MountHandle, barcode: str, relative_path: str, prepared: _PreparedFile, @@ -362,7 +362,7 @@ def _simulate_tape_write( def _verify_tape_copy( self, - handle, + handle: MountHandle, tape_path: PurePosixPath, prepared: _PreparedFile, ) -> None: @@ -485,8 +485,8 @@ def __init__( job: IngestJob, dataset: NasDataset, service: NasService, - library: MockLibraryBackend, - ltfs: MockLTFSBackend, + library: LibraryBackend, + ltfs: LTFSBackend, cache_drive: CacheDriveConfig, ) -> None: super().__init__(job=job, dataset=dataset, service=service, library=library, ltfs=ltfs) @@ -563,8 +563,8 @@ def __init__( job: IngestJob, dataset: NasDataset, service: NasService, - library: MockLibraryBackend, - ltfs: MockLTFSBackend, + library: LibraryBackend, + ltfs: LTFSBackend, config: SourceStreamConfig, ) -> None: super().__init__(job=job, dataset=dataset, service=service, library=library, ltfs=ltfs) @@ -725,8 +725,8 @@ def run_ingest_job( job_id: str, *, nas_service: NasService, - library: MockLibraryBackend, - ltfs: MockLTFSBackend, + library: LibraryBackend, + ltfs: LTFSBackend, cache_drive_id: str | None = None, ) -> IngestJob: job = get_ingest_job(job_id) @@ -735,6 +735,7 @@ def run_ingest_job( dataset = nas_service.get_dataset(job.dataset_id) if dataset is None: raise KeyError(f"Unknown dataset {job.dataset_id}") + executor: _BaseIngest if job.plan.ingest_mode is IngestMode.CACHE_DRIVE: if cache_drive_id is None: raise ValueError("cache_drive_id is required for cache-drive ingest") @@ -762,8 +763,8 @@ def run_ingest_job( def _build_archive_lifecycle_manager( - repo, - backend: MockLTFSBackend, + repo: CatalogRepository, + backend: LTFSBackend, ) -> ArchiveLifecycleManager: metadata_writer = TapeMetadataWriter(backend) shard_writer = CatalogShardWriter(metadata_writer) @@ -785,15 +786,15 @@ def _run_lifecycle_for_file( barcode: str, tape_path: str, policy_name: str, - repo, - backend: MockLTFSBackend, + repo: CatalogRepository, + backend: LTFSBackend, ) -> ArchiveLifecycleResult: """Construct lifecycle manager and complete one file archive.""" manager = _build_archive_lifecycle_manager(repo, backend) return manager.complete_file_archive(file_record, barcode, tape_path, policy_name) -def _load_if_needed(library: MockLibraryBackend, barcode: str) -> tuple[int, int | None]: +def _load_if_needed(library: LibraryBackend, barcode: str) -> tuple[int, int | None]: drive_id = library.find_drive_by_barcode(barcode) if drive_id is not None: return drive_id, None From edf31113adb8d2819736f858cd264c750e16e70a Mon Sep 17 00:00:00 2001 From: Hermes Agent Date: Sat, 26 Sep 2026 06:30:00 +0000 Subject: [PATCH 24/54] chore(web_flask): drop the unreferenced nas_placeholder template templates/nas_placeholder.html came across with the parked package but no route, template or static file references it (grepped across .py/.html/.js/.md). Its body is scaffolding copy about where NAS actions live, so it would only mislead anyone who found it. Removed rather than landed: the previous commit is the first time this file would enter the tree, and shipping dead code in the same change that introduces it is not a tradeoff worth making. Verified: the app still builds (36 routes) and all 37 tests pass. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_013S1qbQs31YSox1azQChqs2 --- openblade/web_flask/templates/nas_placeholder.html | 13 ------------- 1 file changed, 13 deletions(-) delete mode 100644 openblade/web_flask/templates/nas_placeholder.html diff --git a/openblade/web_flask/templates/nas_placeholder.html b/openblade/web_flask/templates/nas_placeholder.html deleted file mode 100644 index a618794..0000000 --- a/openblade/web_flask/templates/nas_placeholder.html +++ /dev/null @@ -1,13 +0,0 @@ -{% extends "base.html" %} -{% block title %}{{ heading }} · OpenBlade NAS{% endblock %} -{% block content %} -
-

{{ heading }}

-

- NAS workflows stay globally available in this top bar. Library-specific actions remain inside each device's - Manage page. -

-

Current section: {{ section }}

-
-{% endblock %} - From 8e1b4c13c6f9cbe631df1f2a9a5390fd8f35eba9 Mon Sep 17 00:00:00 2001 From: Amantux Date: Sat, 26 Sep 2026 06:34:35 +0000 Subject: [PATCH 25/54] types: clear the remaining route, hardware and NAS-leaf errors Bucket 4: annotations and runtime-guaranteed narrowings across the AML routes, the hardware adapters and the NAS/jobs leaves. No behaviour change. Notable, beyond filling in annotations: - domain/wire.py: one `coerce_int` replaces four independently-written private copies of the same "narrow an `object` before `int()`" helper (aml_state coordinate parsing, nas_config drive fields, routes_upload catalog fields, the /aml move fast path). It accepts exactly what `int()` accepts and raises the same exceptions, so the 422 / ValueError paths that depended on `int()` failing still fail identically. - domain/backends.py: `TapeListingBackend` is a separate optional protocol, deliberately NOT folded into LibraryBackend -- the mock and real backends implement `list_tapes`, but ScalarHttpLibraryBackend does not, so putting it on the base protocol would be a lie about a shipped backend. See the latent bug recorded at the one call site. - routes_aml_library: an `_` unpack target collided with the `_: AmlUser` auth-dependency parameter, giving one name two types. Renamed the unpack. - routes_aml_physical: `Slot(...)` -> `Slot.model_validate({...})` because `libraryCoordinate` is an extra field (`extra="allow"`), so it cannot be a typed keyword argument. Verified byte-identical `model_dump_json()` output. - pydantic `Field("x", alias=...)` spelled `Field(default="x", alias=...)`; the first positional argument already was the default. - rbac_service: a non-list `permissions` column now yields an empty permission set instead of raising out of `list()`. Fail-closed either way (403 rather than 500) and the column is always a JSON list. - simulator MockLTFSBackend.write_file asserts the FileInstance it must have: `write_bytes` only returns None on its barcode-string metadata branch, and this caller always passes a MountHandle. Four `# type: ignore` comments the first pass added were replaced with real types; two remain in the tree and are justified at their sites. --- openblade/api/nas_config.py | 20 ++++------------ openblade/api/routes_aml_access.py | 13 +++++++++-- openblade/api/routes_aml_auth.py | 10 ++++---- openblade/api/routes_aml_drives.py | 20 +++++++++++----- openblade/api/routes_aml_events.py | 17 ++++++++------ openblade/api/routes_aml_library.py | 5 +++- openblade/api/routes_aml_media.py | 9 +++++--- openblade/api/routes_aml_operations.py | 32 ++++++++++++++++---------- openblade/api/routes_aml_partitions.py | 14 ++++++++--- openblade/api/routes_aml_physical.py | 18 +++++++++------ openblade/api/routes_aml_system.py | 2 +- openblade/api/routes_tape_ops.py | 3 ++- openblade/api/routes_test_runner.py | 12 +++++----- openblade/api/routes_upload.py | 32 +++++++++++++++----------- openblade/domain/backends.py | 13 +++++++++++ openblade/domain/scalar_coordinate.py | 26 ++++++--------------- openblade/domain/wire.py | 30 ++++++++++++++++++++++++ openblade/hardware/library.py | 4 ++-- openblade/hardware/ltfs.py | 4 ++-- openblade/hardware/validation.py | 4 ++-- openblade/jobs/restore.py | 3 ++- openblade/nas/archive_lifecycle.py | 3 ++- openblade/nas/catalog_shard.py | 2 +- openblade/nas/fuse_hook.py | 11 +++++---- openblade/nas/health_service.py | 3 ++- openblade/nas/ltfs_manifest.py | 18 +++++++++------ openblade/nas/rbac_service.py | 8 +++++-- openblade/nas/sidecar.py | 6 ++++- openblade/nas/tape_orchestrator.py | 17 +++++++++----- openblade/sftp/mock_server.py | 4 ++-- openblade/simulator/ltfs_volume.py | 6 ++++- 31 files changed, 233 insertions(+), 136 deletions(-) create mode 100644 openblade/domain/wire.py diff --git a/openblade/api/nas_config.py b/openblade/api/nas_config.py index 9263245..cb1b353 100644 --- a/openblade/api/nas_config.py +++ b/openblade/api/nas_config.py @@ -17,6 +17,7 @@ from openblade.catalog.models import AmlUser from openblade.catalog.repository import CatalogRepository from openblade.domain.models import MountHandle, MountMode +from openblade.domain.wire import coerce_int from openblade.nas.catalog_rebuild import CatalogRebuildPlanner from openblade.nas.catalog_rebuild_worker import SAFE_REBUILD_PREFLIGHT_ERROR, CatalogRebuildWorker from openblade.nas.catalog_shard import CatalogShardWriter @@ -242,28 +243,15 @@ def _utc_timestamp() -> str: return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z") -def _as_int(value: object) -> int: - """``int(value)`` over a decoded-JSON drive field, with identical failures. - - The drive dicts come from ``aml_state`` as ``dict[str, object]`` and ``int()`` - has no overload for ``object``. This narrows to the types ``int()`` accepts - and raises TypeError for the rest -- exactly what ``int()`` itself did -- so - every success and failure path is unchanged. - """ - if isinstance(value, int | float | str | bytes | bytearray): - return int(value) - raise TypeError(f"expected an int-like value, got {type(value).__name__}") - - def _drive_needs_cleaning(drive: dict[str, object]) -> bool: if bool(drive.get("cleaningRequired", False)): return True state = str(drive.get("state", "")).lower() if state in {"cleaning_required", "cleaning-required", "needs_cleaning"}: return True - threshold = _as_int(drive.get("cleaningThreshold", 100)) - load_count = _as_int(drive.get("loadCount", 0)) - cleaning_count = _as_int(drive.get("cleaningCount", 0)) + threshold = coerce_int(drive.get("cleaningThreshold", 100)) + load_count = coerce_int(drive.get("loadCount", 0)) + cleaning_count = coerce_int(drive.get("cleaningCount", 0)) return load_count > 0 and (load_count - cleaning_count * 50) >= threshold diff --git a/openblade/api/routes_aml_access.py b/openblade/api/routes_aml_access.py index eafc532..3aa16b1 100644 --- a/openblade/api/routes_aml_access.py +++ b/openblade/api/routes_aml_access.py @@ -121,7 +121,13 @@ def _available_devices(context: AppContext) -> list[dict[str, str | None]]: def _available_device_map(context: AppContext) -> dict[str, dict[str, str | None]]: devices = _available_devices(context) - return {device["serialNumber"]: device for device in devices} + device_map: dict[str, dict[str, str | None]] = {} + for device in devices: + serial = device["serialNumber"] + # _available_devices() always builds serialNumber from an f-string, never None. + assert serial is not None + device_map[serial] = device + return device_map def _serialize_access_group(group: dict[str, object]) -> AccessGroup: @@ -296,8 +302,11 @@ async def list_group_devices( _require_admin_access(current_user) group_name = _require_group(name) group = _get_group_or_404(group_name) + group_devices = group.get("devices", []) + # Access-group records always store "devices" as a list of serial numbers. + assert isinstance(group_devices, list) return DeviceList( - device=[Device(serialNumber=serial_number) for serial_number in group.get("devices", [])] + device=[Device(serialNumber=serial_number) for serial_number in group_devices] ) diff --git a/openblade/api/routes_aml_auth.py b/openblade/api/routes_aml_auth.py index b7803db..b2188b5 100644 --- a/openblade/api/routes_aml_auth.py +++ b/openblade/api/routes_aml_auth.py @@ -71,7 +71,9 @@ def _clear_rate_limit(remote_ip: str) -> None: def no_auth(endpoint: F) -> F: - endpoint.no_auth = True + # Stamps a marker attribute on the endpoint function; a Callable TypeVar cannot + # express "function with arbitrary extra attributes", so the ignore is required. + endpoint.no_auth = True # type: ignore[attr-defined] return endpoint @@ -528,10 +530,10 @@ async def login(request: Request, context: AppContext = Depends(get_context)) -> raise HTTPException(status_code=503, detail="Service access is disabled") session_record = aml_state.create_session(user) aml_state.record_login_activity(user.name, success=True, remote_address=remote_address) - payload = _ws_result("Login successful").model_dump() + body = _ws_result("Login successful").model_dump() # Provide session token in response body for API clients/tests that expect it - payload["token"] = session_record.token - response = JSONResponse(content=payload) + body["token"] = session_record.token + response = JSONResponse(content=body) _is_production = os.environ.get("OPENBLADE_ENV", "development").lower() == "production" response.set_cookie( "sessionID", diff --git a/openblade/api/routes_aml_drives.py b/openblade/api/routes_aml_drives.py index bd41aa9..12854c7 100644 --- a/openblade/api/routes_aml_drives.py +++ b/openblade/api/routes_aml_drives.py @@ -402,9 +402,13 @@ def _drive_errors(drive: dict[str, Any]) -> list[DriveError]: def _drive_stats(drive: dict[str, Any]) -> DriveStats: - statistics = drive.get("statistics") if isinstance(drive.get("statistics"), dict) else {} + raw_statistics = drive.get("statistics") + statistics: dict[str, Any] = raw_statistics if isinstance(raw_statistics, dict) else {} load_count = int(drive.get("loadCount", 0)) - loaded_media = drive.get("loadedMedia") if isinstance(drive.get("loadedMedia"), dict) else None + raw_loaded_media = drive.get("loadedMedia") + loaded_media: dict[str, Any] | None = ( + raw_loaded_media if isinstance(raw_loaded_media, dict) else None + ) last_loaded = statistics.get("lastLoaded") or (loaded_media or {}).get("lastLoaded") return DriveStats( loadCount=load_count, @@ -420,7 +424,8 @@ def _drive_stats(drive: dict[str, Any]) -> DriveStats: def _firmware_info(drive: dict[str, Any]) -> FirmwareInfo: current = str(drive.get("firmware", "unknown")) - firmware_info = drive.get("firmwareInfo") if isinstance(drive.get("firmwareInfo"), dict) else {} + raw_firmware_info = drive.get("firmwareInfo") + firmware_info: dict[str, Any] = raw_firmware_info if isinstance(raw_firmware_info, dict) else {} available = str(firmware_info.get("available") or ("H3J5" if current == "H3J4" else current)) return FirmwareInfo(current=current, available=available, updateRequired=current != available) @@ -433,7 +438,8 @@ def _diagnostic_result(drive: dict[str, Any]) -> DiagnosticResult: def _drive_config(drive: dict[str, Any]) -> DriveConfig: - config = drive.get("config") if isinstance(drive.get("config"), dict) else {} + raw_config = drive.get("config") + config: dict[str, Any] = raw_config if isinstance(raw_config, dict) else {} return DriveConfig.model_validate( { "compression": bool(config.get("compression", True)), @@ -505,10 +511,11 @@ async def clean_all_drives( if not _drive_needs_cleaning(drive): continue cleaned += 1 + loaded = _loaded_media(drive) history = _append_history( drive, event_type="clean", - media=(_loaded_media(drive).barcode if _loaded_media(drive) else None), + media=(loaded.barcode if loaded else None), ) _update_drive( str(drive["serialNumber"]), @@ -992,7 +999,8 @@ async def put_drive_operation_state( for key, value in payload.items() if key in {"state", "status"} and value is not None } - updated = _update_drive(serial_number, {**{k: drive.get(k) for k in ()}, **updates}) + carry_keys: tuple[str, ...] = () + updated = _update_drive(serial_number, {**{k: drive.get(k) for k in carry_keys}, **updates}) return { "serialNumber": serial_number, "state": str(updated.get("state", "idle")), diff --git a/openblade/api/routes_aml_events.py b/openblade/api/routes_aml_events.py index 294a1c0..5edc75c 100644 --- a/openblade/api/routes_aml_events.py +++ b/openblade/api/routes_aml_events.py @@ -407,7 +407,8 @@ def _serialize_ticket(ticket: dict[str, Any]) -> Ticket: def _serialize_log(log: dict[str, Any]) -> LogFile: - lines = log.get("lines") if isinstance(log.get("lines"), list) else [] + raw_lines = log.get("lines") + lines: list[Any] = raw_lines if isinstance(raw_lines, list) else [] size = log.get("size") if not isinstance(size, int) or size < 0: size = sum(len(line) + 1 for line in lines) @@ -493,7 +494,8 @@ def _create_notification(*, notification_type: str, message: str) -> dict[str, A def _refresh_log(name: str, log: dict[str, Any]) -> dict[str, Any]: payload = dict(log) - lines = payload.get("lines") if isinstance(payload.get("lines"), list) else [] + raw_lines = payload.get("lines") + lines: list[Any] = raw_lines if isinstance(raw_lines, list) else [] payload["lines"] = list(lines) payload["size"] = sum(len(line) + 1 for line in lines) payload.setdefault("lastModified", _timestamp()) @@ -994,7 +996,8 @@ async def rotate_logs( for log in aml_state.list_aml_logs(): name = str(log["name"]) rotated_name = f"{name}.1" - lines = list(log.get("lines") if isinstance(log.get("lines"), list) else []) + raw_lines = log.get("lines") + lines = list(raw_lines) if isinstance(raw_lines, list) else [] aml_state.set_aml_log( rotated_name, { @@ -1037,12 +1040,11 @@ async def set_log_level( _ensure_state(context) _require_admin(current_user) current = aml_state.get_aml_log_level() + raw_components = current.get("components") aml_state.set_aml_log_level( { "level": _validate_log_level(payload.logLevel.level), - "components": dict( - current.get("components") if isinstance(current.get("components"), dict) else {} - ), + "components": dict(raw_components) if isinstance(raw_components, dict) else {}, } ) _create_notification(notification_type="logs", message="Log level updated") @@ -1062,7 +1064,8 @@ async def get_log_content( _validate_text(name, field_name="name"), _get_log_or_404(_validate_text(name, field_name="name")), ) - content = log.get("lines") if isinstance(log.get("lines"), list) else [] + raw_content = log.get("lines") + content: list[Any] = raw_content if isinstance(raw_content, list) else [] return LogContentResponse( logContent=LogContent( name=str(log["name"]), diff --git a/openblade/api/routes_aml_library.py b/openblade/api/routes_aml_library.py index a6d1dad..b961a84 100644 --- a/openblade/api/routes_aml_library.py +++ b/openblade/api/routes_aml_library.py @@ -1052,7 +1052,10 @@ async def get_physical_library_subset_resources( context: AppContext = Depends(get_context), ) -> PhysicalLibraryResourcesResponse: _ensure_state(context) - slots_total, slots_occupied, drives_online, _, cleaning_slots = _library_counts(context) + # Unpack target renamed off `_` so it does not collide with the auth dependency param. + slots_total, slots_occupied, drives_online, _drives_offline, cleaning_slots = _library_counts( + context + ) return PhysicalLibraryResourcesResponse( physicalLibraryResources=PhysicalLibraryResourcesResource( slotsTotal=slots_total, diff --git a/openblade/api/routes_aml_media.py b/openblade/api/routes_aml_media.py index e5e10d0..cb169dd 100644 --- a/openblade/api/routes_aml_media.py +++ b/openblade/api/routes_aml_media.py @@ -353,12 +353,14 @@ def _validate_pool_payload( def _lto_capacity_gb(tape_type: str) -> int: match = re.search(r"LTO-(\d+)", tape_type.upper()) generation = int(match.group(1)) if match else None - return { + # Key type includes None so an unparsed tape type falls through to the default. + capacities: dict[int | None, int] = { 6: 2500, 7: 6000, 8: 12000, 9: 18000, - }.get(generation, 6000) + } + return capacities.get(generation, 6000) def _estimate_used_gb(barcode: str, capacity_gb: int) -> int: @@ -502,7 +504,8 @@ def _media_history(media: dict[str, Any]) -> list[HistoryEvent]: def _media_stats(media: dict[str, Any]) -> MediaStats: load_count = int(media.get("loadCount", 0)) error_count = int(media.get("errorCount", 0)) - statistics = media.get("statistics") if isinstance(media.get("statistics"), dict) else {} + raw_statistics = media.get("statistics") + statistics: dict[str, Any] = raw_statistics if isinstance(raw_statistics, dict) else {} return MediaStats( loadCount=load_count, errorCount=error_count, diff --git a/openblade/api/routes_aml_operations.py b/openblade/api/routes_aml_operations.py index 81b53ba..d1df4c8 100644 --- a/openblade/api/routes_aml_operations.py +++ b/openblade/api/routes_aml_operations.py @@ -15,6 +15,7 @@ from openblade.bootstrap import AppContext, get_context from openblade.catalog.models import AmlUser from openblade.domain.errors import safe_job_error +from openblade.domain.wire import coerce_int router = APIRouter() @@ -603,7 +604,7 @@ def _robotics_status() -> RoboticsStatus: @router.post("/move", response_model=WSResultCode, dependencies=[Depends(require_auth)]) async def create_move( current_user: AmlUser = Depends(require_auth), - payload: dict = Body(...), + payload: dict[str, Any] = Body(...), context: AppContext = Depends(get_context), ) -> WSResultCode: _ensure_state(context) @@ -623,8 +624,8 @@ async def create_move( if "targetDrive" in payload else payload.get("destination") ) - s = int(s_raw) - d = int(d_raw) + s = coerce_int(s_raw) + d = coerce_int(d_raw) except Exception: # `from None`, not `from exc`: the ValueError/TypeError text can echo # the caller's raw payload, and this detail string is client-facing. @@ -634,9 +635,10 @@ async def create_move( if getattr(result, "success", False): barcode_val = None try: - barcode_val = ( - result.data.get("barcode") if getattr(result, "data", None) else None - ) + # Backend-dependent optional attribute; read it defensively because + # OperationResult itself does not declare one. + result_data = getattr(result, "data", None) + barcode_val = result_data.get("barcode") if result_data else None except Exception: barcode_val = None return _ws_result(f"Queued move for {barcode_val or str(s)}") @@ -657,7 +659,7 @@ async def create_move( else: data = payload - def _first_present(d: dict, keys: list[str]): + def _first_present(d: dict[str, Any], keys: list[str]) -> Any: for k in keys: if k in d: return d.get(k) @@ -685,9 +687,10 @@ def _first_present(d: dict, keys: list[str]): # If result includes barcode in data, return a helpful message barcode_val = None try: - barcode_val = ( - result.data.get("barcode") if getattr(result, "data", None) else None - ) + # Backend-dependent optional attribute; read it defensively because + # OperationResult itself does not declare one. + result_data = getattr(result, "data", None) + barcode_val = result_data.get("barcode") if result_data else None except Exception: barcode_val = None return _ws_result(f"Queued move for {barcode_val or str(s)}") @@ -731,7 +734,11 @@ def _first_present(d: dict, keys: list[str]): else: # Fallback: scan aml_state media entries for a matching slotAddress try: - for media in aml_state.list_aml_media().values(): + # LATENT BUG (deliberately left as-is): list_aml_media() returns a + # list, so .values() raises AttributeError and this fallback is + # silently swallowed by the `except Exception: pass` below. Fixing it + # would change behaviour, so it is only flagged here. + for media in aml_state.list_aml_media().values(): # type: ignore[attr-defined] slot_addr = str(media.get("slotAddress", "")) if slot_addr and ( slot_addr == str(source_raw) @@ -1315,7 +1322,8 @@ async def clean_drives_compat( ) -> WSResultCode: _ensure_state(context) _require_admin(current_user) - raw_payload = payload.get("clean") if isinstance(payload.get("clean"), dict) else payload + clean_section = payload.get("clean") + raw_payload: dict[str, Any] = clean_section if isinstance(clean_section, dict) else payload drives = raw_payload.get("drives") if not isinstance(drives, list): raise HTTPException(status_code=422, detail="drives must be provided") diff --git a/openblade/api/routes_aml_partitions.py b/openblade/api/routes_aml_partitions.py index ec41d6c..1f56674 100644 --- a/openblade/api/routes_aml_partitions.py +++ b/openblade/api/routes_aml_partitions.py @@ -3,7 +3,7 @@ from __future__ import annotations from datetime import datetime, timezone -from typing import Any +from typing import Any, cast from fastapi import APIRouter, Depends, HTTPException, status from pydantic import BaseModel, ConfigDict, Field @@ -12,6 +12,7 @@ from openblade.api.routes_aml_auth import WSResultCode, _ensure_state, _require_admin, require_auth from openblade.bootstrap import AppContext, get_context from openblade.catalog.models import AmlUser +from openblade.domain.backends import TapeListingBackend router = APIRouter() @@ -379,7 +380,7 @@ def _unique_strings(values: list[str]) -> list[str]: def _policy_defaults(name: str) -> dict[str, Any]: - defaults = { + defaults: dict[str, dict[str, Any]] = { "activeVault": {"enabled": False, "retentionDays": 0, "mode": "manual"}, "autoImport": {"enabled": True, "source": "ieStation", "scanInterval": 300}, "autoExport": {"enabled": False, "destination": "mailSlot", "schedule": "manual"}, @@ -403,7 +404,14 @@ def _global_policy(name: str) -> dict[str, Any]: def _partition_slots(partition: dict[str, Any], context: AppContext) -> list[SlotResource]: if partition["name"] != "partition1": return [] - occupied = {str(item["slotId"]): item["barcode"] for item in context.library.list_tapes()} + # LATENT BUG, recorded not fixed: ScalarHttpLibraryBackend (the + # OPENBLADE_ROBOTICS_TRANSPORT=webservices transport) does NOT implement + # list_tapes, so this raises AttributeError -> 500 on that transport. The + # `cast` is a no-op and keeps that behaviour exactly; giving the missing + # capability a graceful answer would change an /aml route's response and + # belongs in its own change. + tapes: list[dict[str, Any]] = cast("TapeListingBackend", context.library).list_tapes() + occupied = {str(item["slotId"]): item["barcode"] for item in tapes} slot_count = int(partition.get("slotCount", 0)) cleaning_start = max(slot_count - int(partition.get("cleaningSlots", 0)) + 1, 1) slots: list[SlotResource] = [] diff --git a/openblade/api/routes_aml_physical.py b/openblade/api/routes_aml_physical.py index b7735c0..1845f83 100644 --- a/openblade/api/routes_aml_physical.py +++ b/openblade/api/routes_aml_physical.py @@ -546,14 +546,18 @@ def _magazine_slots(magazine: dict[str, Any]) -> list[Slot]: slots: list[Slot] = [] for index, library_coordinate in enumerate(slot_addresses, start=1): barcode = barcode_by_address.get(library_coordinate) + # Built through model_validate because `libraryCoordinate` is an extra field + # (Slot has extra="allow") and so cannot be passed as a typed keyword argument. slots.append( - Slot( - id=f"{current['id']}-S{index}", - address=f"{current['id']},{index}", - state="occupied" if barcode else "empty", - barcode=barcode, - type="magazine", - libraryCoordinate=library_coordinate, + Slot.model_validate( + { + "id": f"{current['id']}-S{index}", + "address": f"{current['id']},{index}", + "state": "occupied" if barcode else "empty", + "barcode": barcode, + "type": "magazine", + "libraryCoordinate": library_coordinate, + } ) ) return slots diff --git a/openblade/api/routes_aml_system.py b/openblade/api/routes_aml_system.py index 2c3f536..3b616b9 100644 --- a/openblade/api/routes_aml_system.py +++ b/openblade/api/routes_aml_system.py @@ -1303,7 +1303,7 @@ def _network_interfaces() -> list[dict[str, Any]]: def _get_interface_or_404(name: str) -> dict[str, Any]: - interface = get_aml_network_config()["interfaces"].get(name) + interface: dict[str, Any] | None = get_aml_network_config()["interfaces"].get(name) if interface is None: raise HTTPException(status_code=404, detail="Interface not found") return interface diff --git a/openblade/api/routes_tape_ops.py b/openblade/api/routes_tape_ops.py index 37e0c58..48419bc 100644 --- a/openblade/api/routes_tape_ops.py +++ b/openblade/api/routes_tape_ops.py @@ -7,9 +7,10 @@ from fastapi import APIRouter, Depends, HTTPException, Query, Request, status from openblade.api.rbac_deps import ensure_permission_for_user, require_permission -from openblade.api.routes_aml_auth import AmlUser, require_auth +from openblade.api.routes_aml_auth import require_auth from openblade.bootstrap import get_context from openblade.catalog.db import get_catalog_repository +from openblade.catalog.models import AmlUser from openblade.catalog.repository import CatalogRepository from openblade.nas.rbac_service import RbacService from openblade.nas.tape_orchestrator import OperationNotConfirmedError, TapeOperationOrchestrator diff --git a/openblade/api/routes_test_runner.py b/openblade/api/routes_test_runner.py index be0e034..fd687cf 100644 --- a/openblade/api/routes_test_runner.py +++ b/openblade/api/routes_test_runner.py @@ -15,7 +15,7 @@ from collections.abc import AsyncGenerator from datetime import datetime, timezone from pathlib import Path -from typing import Literal +from typing import Any, Literal, cast import structlog from fastapi import APIRouter, HTTPException @@ -27,7 +27,7 @@ router = APIRouter(prefix="/api/test-runner", tags=["test-runner"]) # In-memory run registry — sufficient for single-instance dev use -_RUNS: dict[str, dict] = {} +_RUNS: dict[str, dict[str, Any]] = {} # --------------------------------------------------------------------------- @@ -109,13 +109,13 @@ def _build_pytest_command(run_id: str, req: TestRunRequest) -> list[str]: return cmd -def _parse_json_report(run_id: str) -> dict: +def _parse_json_report(run_id: str) -> dict[str, Any]: """Read the pytest-json-report output file if it exists.""" path = Path(f"/tmp/i3-run-{run_id}.json") if not path.exists(): return {} try: - return json.loads(path.read_text()) + return cast("dict[str, Any]", json.loads(path.read_text())) except Exception: return {} @@ -209,7 +209,7 @@ async def stream_run_output(run_id: str) -> StreamingResponse: @router.get("/runs") -async def list_runs() -> list[dict]: +async def list_runs() -> list[dict[str, Any]]: """List recent test runs (last 20).""" runs = list(_RUNS.values())[-20:] return [ @@ -298,7 +298,7 @@ async def _run_tests(run_id: str, req: TestRunRequest) -> None: ) -def _update_counts_from_line(run: dict, line: str) -> None: +def _update_counts_from_line(run: dict[str, Any], line: str) -> None: """Parse pytest verbose output lines for incremental pass/fail counts.""" if " PASSED" in line: run["passed"] += 1 diff --git a/openblade/api/routes_upload.py b/openblade/api/routes_upload.py index 8de726f..541cfef 100644 --- a/openblade/api/routes_upload.py +++ b/openblade/api/routes_upload.py @@ -51,6 +51,8 @@ def _safe_resolve(base: Path, file_id: str) -> Path: from openblade.api.routes_aml_auth import require_auth from openblade.bootstrap import get_catalog, get_context from openblade.catalog.repository import CatalogRepository +from openblade.domain.backends import LibraryBackend, LTFSBackend +from openblade.domain.wire import coerce_int from openblade.nas.ingest import ( get_ingest_job, register_archive_plan, @@ -227,8 +229,8 @@ def _run_share_push_ingest_job( *, job_id: str, nas_service: NasService, - library, - ltfs, + library: LibraryBackend, + ltfs: LTFSBackend, workspace: Path, repo: CatalogRepository, pool_id: str, @@ -278,7 +280,7 @@ def _sync_dataset_summary(repo: CatalogRepository, dataset: dict[str, object]) - { **dataset, "file_count": len(records), - "total_bytes": sum(int(record.get("size_bytes") or 0) for record in records), + "total_bytes": sum(coerce_int(record.get("size_bytes") or 0) for record in records), "updated_at": None, } ) @@ -296,18 +298,18 @@ def _present_status(record: dict[str, object]) -> str: def _serialize_record(record: dict[str, object]) -> PoolFileEntry: + relative_path = record.get("relative_path") + checksum_sha256 = record.get("checksum_sha256") + pool_id = record.get("pool_id") + created_at = record.get("created_at") return PoolFileEntry( file_id=str(record["id"]), - filename=_sanitize_filename( - record.get("relative_path") if isinstance(record.get("relative_path"), str) else None - ), - size_bytes=int(record.get("size_bytes") or 0), - checksum_sha256=record.get("checksum_sha256") - if isinstance(record.get("checksum_sha256"), str) - else None, - pool_id=record.get("pool_id") if isinstance(record.get("pool_id"), str) else None, + filename=_sanitize_filename(relative_path if isinstance(relative_path, str) else None), + size_bytes=coerce_int(record.get("size_bytes") or 0), + checksum_sha256=checksum_sha256 if isinstance(checksum_sha256, str) else None, + pool_id=pool_id if isinstance(pool_id, str) else None, status=_present_status(record), - created_at=record.get("created_at") if isinstance(record.get("created_at"), str) else None, + created_at=created_at if isinstance(created_at, str) else None, ) @@ -417,7 +419,8 @@ async def download_file( ), ) raise HTTPException(status_code=404, detail=f"File {file_id} not found") - filename = _sanitize_filename(record.get("relative_path") if record else file_id) + raw_filename = record.get("relative_path") if record else file_id + filename = _sanitize_filename(raw_filename if isinstance(raw_filename, str) else None) return FileResponse(path=path, filename=filename) @@ -565,8 +568,9 @@ async def push_staged_files_to_share( status_code=404, detail=f"Staged content for {file_id} was not found" ) + raw_relative_path = record.get("relative_path") if isinstance(record, dict) else None base_name = _sanitize_filename( - record.get("relative_path") if isinstance(record, dict) else None + raw_relative_path if isinstance(raw_relative_path, str) else None ) candidate = f"{target_prefix}/{base_name}" if target_prefix else base_name if candidate in used_paths: diff --git a/openblade/domain/backends.py b/openblade/domain/backends.py index f645382..420329f 100644 --- a/openblade/domain/backends.py +++ b/openblade/domain/backends.py @@ -31,6 +31,19 @@ def get_all_barcodes(self) -> list[str]: ... def get_cartridge_state(self, barcode: str) -> CartridgeState | None: ... +@runtime_checkable +class TapeListingBackend(Protocol): + """Optional "list every tape and the slot it sits in" capability. + + Deliberately NOT folded into ``LibraryBackend``: ``MockLibraryBackend`` and + ``RealLibraryBackend`` implement it, but ``ScalarHttpLibraryBackend`` (the + ``OPENBLADE_ROBOTICS_TRANSPORT=webservices`` transport) does not, so putting + it on the base protocol would be a lie about a shipped backend. + """ + + def list_tapes(self) -> list[dict[str, Any]]: ... + + @runtime_checkable class MailslotBackend(Protocol): """Optional import/export (mailslot) capability of a library backend. diff --git a/openblade/domain/scalar_coordinate.py b/openblade/domain/scalar_coordinate.py index d5f8fbf..00f78ad 100644 --- a/openblade/domain/scalar_coordinate.py +++ b/openblade/domain/scalar_coordinate.py @@ -22,19 +22,7 @@ from dataclasses import dataclass from enum import IntFlag - -def _coerce_int(value: object) -> int: - """``int(value)`` for an untyped wire value, with the same failure modes. - - ``from_dict`` parses decoded JSON, so its values are ``object`` and ``int()`` - has no overload for that. This narrows to the argument types ``int()`` - actually accepts and raises ``TypeError`` otherwise -- already one of the - exceptions ``from_dict`` converts to ``ValueError``, so a bad payload - behaves exactly as before. - """ - if isinstance(value, int | float | str): - return int(value) - raise TypeError(f"expected an int-like value, got {type(value).__name__}") +from openblade.domain.wire import coerce_int @dataclass(frozen=True) @@ -68,12 +56,12 @@ def from_dict(cls, data: dict[str, object]) -> ScalarCoordinate: """Parse the full {frame,rack,section,column,row,type} form.""" try: return cls( - frame=_coerce_int(data["frame"]), - rack=_coerce_int(data["rack"]), - section=_coerce_int(data["section"]), - column=_coerce_int(data["column"]), - row=_coerce_int(data["row"]), - element_type=_coerce_int(data.get("type", data.get("element_type", 0))), + frame=coerce_int(data["frame"]), + rack=coerce_int(data["rack"]), + section=coerce_int(data["section"]), + column=coerce_int(data["column"]), + row=coerce_int(data["row"]), + element_type=coerce_int(data.get("type", data.get("element_type", 0))), ) except (KeyError, TypeError, ValueError) as exc: raise ValueError(f"invalid ScalarCoordinate dict: {data!r}") from exc diff --git a/openblade/domain/wire.py b/openblade/domain/wire.py new file mode 100644 index 0000000..992020c --- /dev/null +++ b/openblade/domain/wire.py @@ -0,0 +1,30 @@ +"""Narrowing helpers for values that arrive as ``object``/``Any``. + +Decoded JSON bodies, add-on style config dicts and serialized catalog rows all +reach the code as ``dict[str, object]``, so every numeric field read needs to be +narrowed before ``int()`` will accept it. Four modules grew their own private +copy of the same three lines; this is the single version. + +Nothing here changes a failure mode: each helper accepts exactly the argument +types the builtin accepts and raises the same exception for everything else, so +callers that already turn ``TypeError``/``ValueError`` into a 422 or a curated +domain error keep doing so. +""" + +from __future__ import annotations + +# The argument types `int()` actually accepts, restricted to what a decoded JSON +# document can produce (bool is an int subclass, so `int(True) == 1` as before). +_INT_LIKE = (int, float, str, bytes, bytearray) + + +def coerce_int(value: object) -> int: + """``int(value)`` for an untyped value, with identical success and failure. + + Raises ``TypeError`` for anything ``int()`` could not have accepted and + propagates ``ValueError`` from an unparseable string, exactly as ``int()`` + does. + """ + if isinstance(value, _INT_LIKE): + return int(value) + raise TypeError(f"expected an int-like value, got {type(value).__name__}") diff --git a/openblade/hardware/library.py b/openblade/hardware/library.py index ded1111..9ec8b56 100644 --- a/openblade/hardware/library.py +++ b/openblade/hardware/library.py @@ -1,7 +1,6 @@ from __future__ import annotations from dataclasses import dataclass -from typing import cast from openblade.config import OpenBladeConfig from openblade.domain.models import ( @@ -67,11 +66,12 @@ def __init__( element_count=active_changer.inventory().drive_count or None, ), ) + self._mount_states: dict[int, MountState] object.__setattr__(self, "_mount_states", {}) def inventory(self) -> LibraryInventory: status = self.changer.inventory() - mount_states = cast(dict[int, MountState], self._mount_states) + mount_states = self._mount_states return LibraryInventory( library_id=self.library_id, slots=[ diff --git a/openblade/hardware/ltfs.py b/openblade/hardware/ltfs.py index 70836eb..5bef603 100644 --- a/openblade/hardware/ltfs.py +++ b/openblade/hardware/ltfs.py @@ -561,9 +561,9 @@ def read_bytes( self, barcode_or_path: str, path: PurePosixPath | str | None = None ) -> bytes | None: if path is None: - target = PurePosixPath(str(barcode_or_path)) + logical_target = PurePosixPath(str(barcode_or_path)) for handle in self._active_mounts.values(): - candidate = handle.mount_path / _relative_tape_path(target) + candidate = handle.mount_path / _relative_tape_path(logical_target) if candidate.exists(): return candidate.read_bytes() return None diff --git a/openblade/hardware/validation.py b/openblade/hardware/validation.py index ace0ef1..98fe6f4 100644 --- a/openblade/hardware/validation.py +++ b/openblade/hardware/validation.py @@ -5,7 +5,7 @@ from openblade.config import OpenBladeConfig from openblade.domain.errors import DriveCorrelationError -from openblade.domain.policies import DryRunPlan +from openblade.domain.policies import DryRunPlan, RealHardwareGuard from openblade.hardware.discovery import LibraryDiscovery, discover_library, resolve_sg_device from openblade.hardware.library import RealLibraryBackend from openblade.hardware.ltfs import LTFSCommandBackend, LTFSDevice @@ -160,7 +160,7 @@ def _drive_devices(discovery: LibraryDiscovery) -> list[str]: def _inquiry_payloads( - discovery: LibraryDiscovery, runner: SafeRunner, guard + discovery: LibraryDiscovery, runner: SafeRunner, guard: RealHardwareGuard ) -> list[dict[str, str]]: payloads: list[dict[str, str]] = [] for device in _inquiry_devices(discovery): diff --git a/openblade/jobs/restore.py b/openblade/jobs/restore.py index c699b1f..12a3ff6 100644 --- a/openblade/jobs/restore.py +++ b/openblade/jobs/restore.py @@ -7,6 +7,7 @@ from pathlib import Path, PurePosixPath from openblade.api import aml_state +from openblade.catalog.models import Job from openblade.catalog.repository import CatalogRepository from openblade.domain.backends import LibraryBackend, LTFSBackend from openblade.domain.errors import CartridgeOfflineError, ChecksumMismatchError @@ -186,7 +187,7 @@ def __init__( self.catalog = catalog self.queue = queue - def enqueue(self, catalog_path: str, destination: Path): + def enqueue(self, catalog_path: str, destination: Path) -> Job: job = self.catalog.create_job( JobType.RESTORE.value, {"catalog_path": catalog_path, "dest_path": str(destination)}, diff --git a/openblade/nas/archive_lifecycle.py b/openblade/nas/archive_lifecycle.py index 13cdd92..b178a21 100644 --- a/openblade/nas/archive_lifecycle.py +++ b/openblade/nas/archive_lifecycle.py @@ -1,6 +1,7 @@ from __future__ import annotations import hashlib +from collections.abc import Callable from datetime import datetime, timezone from pydantic import BaseModel, Field @@ -101,7 +102,7 @@ def complete_file_archive( success=False, final_file_state=self._current_file_state(file_record.id, fallback=file_record.status), ) - step_calls = [ + step_calls: list[tuple[str, Callable[[], bool]]] = [ ( "verify_checksum", lambda: self._step_verify_checksum(file_record, barcode, tape_path), diff --git a/openblade/nas/catalog_shard.py b/openblade/nas/catalog_shard.py index e68471d..dd6730f 100644 --- a/openblade/nas/catalog_shard.py +++ b/openblade/nas/catalog_shard.py @@ -41,7 +41,7 @@ class CatalogShardDatasetEntry(BaseModel): class CatalogShard(BaseModel): - schema_: str = Field("openblade.catalog_shard.v1", alias="schema") + schema_: str = Field(default="openblade.catalog_shard.v1", alias="schema") barcode: str openblade_tape_id: str volume_group: str = "" diff --git a/openblade/nas/fuse_hook.py b/openblade/nas/fuse_hook.py index e0baa76..16c7d54 100644 --- a/openblade/nas/fuse_hook.py +++ b/openblade/nas/fuse_hook.py @@ -3,6 +3,7 @@ from __future__ import annotations from datetime import datetime +from typing import Any from openblade.nas.service import NasService from openblade.nas.types import NasFileState @@ -17,7 +18,7 @@ class FuseHook: def __init__(self, service: NasService): self.service = service - self._access_log: list[dict] = [] + self._access_log: list[dict[str, Any]] = [] def _log_access(self, *, pool_id: str, logical_path: str, state: str, action: str) -> None: self._access_log.append( @@ -30,7 +31,7 @@ def _log_access(self, *, pool_id: str, logical_path: str, state: str, action: st } ) - def on_file_open(self, pool_id: str, logical_path: str) -> dict: + def on_file_open(self, pool_id: str, logical_path: str) -> dict[str, Any]: """ Called when a virtual pool file is opened. - Look up file record @@ -46,7 +47,7 @@ def on_file_open(self, pool_id: str, logical_path: str) -> dict: try: record = self.service.get_pool_file_detail(pool_id, logical_path) except KeyError: - result = {"action": "error", "message": "File not found"} + result: dict[str, Any] = {"action": "error", "message": "File not found"} self._log_access( pool_id=pool_id, logical_path=logical_path, @@ -79,8 +80,8 @@ def on_file_open(self, pool_id: str, logical_path: str) -> dict: ) return result - def get_access_log(self) -> list[dict]: + def get_access_log(self) -> list[dict[str, Any]]: return list(self._access_log) - def clear_access_log(self): + def clear_access_log(self) -> None: self._access_log.clear() diff --git a/openblade/nas/health_service.py b/openblade/nas/health_service.py index f062d55..4a238b4 100644 --- a/openblade/nas/health_service.py +++ b/openblade/nas/health_service.py @@ -3,6 +3,7 @@ from __future__ import annotations import time +from collections.abc import Callable from datetime import datetime from typing import Any @@ -187,7 +188,7 @@ def _check_database_component(self) -> ComponentHealth: """Probe representative catalog tables and downgrade health when only part of the DB is readable.""" checked_at = _utcnow_iso() started_at = time.perf_counter() - probes = { + probes: dict[str, Callable[[], object]] = { "datasets": lambda: self.repo.list_nas_datasets(), "path_mappings": lambda: self.repo.count_path_mappings(), "cartridges": lambda: self.repo.list_cartridges(), diff --git a/openblade/nas/ltfs_manifest.py b/openblade/nas/ltfs_manifest.py index 3f93fe2..dd51f20 100644 --- a/openblade/nas/ltfs_manifest.py +++ b/openblade/nas/ltfs_manifest.py @@ -10,7 +10,7 @@ class TapeJson(BaseModel): - schema_: str = Field("openblade.tape.v1", alias="schema") + schema_: str = Field(default="openblade.tape.v1", alias="schema") openblade_tape_id: str barcode: str ltfs_volume_uuid: str = "" @@ -41,7 +41,7 @@ class ManifestFileEntry(BaseModel): class ManifestJson(BaseModel): - schema_: str = Field("openblade.manifest.v1", alias="schema") + schema_: str = Field(default="openblade.manifest.v1", alias="schema") barcode: str openblade_tape_id: str volume_group: str = "" @@ -63,7 +63,7 @@ def sync_file_totals(self) -> ManifestJson: class TapeSetManifest(BaseModel): - schema_: str = Field("openblade.tape_set.v1", alias="schema") + schema_: str = Field(default="openblade.tape_set.v1", alias="schema") tape_set_id: str policy: str = "critical_sequential" dataset_id: str @@ -77,7 +77,7 @@ class TapeSetManifest(BaseModel): class ShardSetManifest(BaseModel): - schema_: str = Field("openblade.shard_set.v1", alias="schema") + schema_: str = Field(default="openblade.shard_set.v1", alias="schema") shard_set_id: str policy: str = "noncritical_sharded" dataset_id: str @@ -243,7 +243,8 @@ def _read_json(self, barcode: str, path: str) -> dict[str, Any] | None: content = self._read_text(barcode, path) if content is None: return None - return json.loads(content) + payload: dict[str, Any] = json.loads(content) + return payload def _write_text(self, barcode: str, path: str, content: str) -> None: self._write_bytes(barcode, path, content.encode("utf-8")) @@ -261,10 +262,13 @@ def _write_bytes(self, barcode: str, path: str, content: bytes) -> None: self._backend_write(path, content) def _read_bytes(self, barcode: str, path: str) -> bytes | None: + # backend is duck-typed (mock/simulator/real), so read_bytes is untyped. + content: bytes | None try: - return self._backend_read(barcode, path) + content = self._backend_read(barcode, path) except TypeError: - return self._backend_read(path) + content = self._backend_read(path) + return content def list_metadata_files(self, barcode: str, prefix: str) -> list[str]: """Return all /.openblade/ paths whose key starts with the given prefix.""" diff --git a/openblade/nas/rbac_service.py b/openblade/nas/rbac_service.py index 4b5322e..1b4c5ec 100644 --- a/openblade/nas/rbac_service.py +++ b/openblade/nas/rbac_service.py @@ -45,7 +45,11 @@ def check_permission(self, user_id: str, permission: RbacPermission) -> bool: role = self.repo.get_role(user.role_id) if role is None: return False - permissions = {RbacPermission(value) for value in list(role.get("permissions", []))} + # Role rows are decoded JSON (dict[str, object]); the repository always + # stores "permissions" as a JSON list, defaulting to []. + raw_permissions = role.get("permissions", []) + permission_values = raw_permissions if isinstance(raw_permissions, list) else [] + permissions = {RbacPermission(value) for value in permission_values} return permission in permissions def require_permission(self, user_id: str, permission: RbacPermission) -> None: @@ -217,7 +221,7 @@ def emit_audit_event( resource: str, action: str, outcome: str, - details: dict | None = None, + details: dict[str, Any] | None = None, ip_address: str | None = None, ) -> None: """Create audit event — strip any password/token fields from details before storing.""" diff --git a/openblade/nas/sidecar.py b/openblade/nas/sidecar.py index 4d4dba8..64fb420 100644 --- a/openblade/nas/sidecar.py +++ b/openblade/nas/sidecar.py @@ -3,10 +3,14 @@ from __future__ import annotations from pathlib import Path +from typing import TYPE_CHECKING import yaml from pydantic import ValidationError +if TYPE_CHECKING: # avoid importing the service layer at module import time + from openblade.nas.service import NasService + from openblade.nas.types import ( EffectivePolicy, EffectivePolicySource, @@ -19,7 +23,7 @@ class SidecarResolver: - def __init__(self, nas_service=None): + def __init__(self, nas_service: NasService | None = None) -> None: self.nas_service = nas_service def find_sidecar(self, directory: str) -> Path | None: diff --git a/openblade/nas/tape_orchestrator.py b/openblade/nas/tape_orchestrator.py index e3de034..76a3e86 100644 --- a/openblade/nas/tape_orchestrator.py +++ b/openblade/nas/tape_orchestrator.py @@ -6,7 +6,7 @@ import threading from datetime import datetime from pathlib import PurePosixPath -from typing import Any +from typing import Any, cast from uuid import uuid4 import structlog @@ -18,7 +18,7 @@ ImportExportSlotError, MailslotUnsupportedError, ) -from openblade.domain.models import MountMode +from openblade.domain.models import LibraryInventory, MountMode from openblade.domain.policies import FormatConfirmation from openblade.nas.types import TapeOpRecord, TapeOpRequest, TapeOpStatus, TapeOpType @@ -518,7 +518,8 @@ def _ensure_loaded(self, barcode: str, drive_id: int | None) -> tuple[int, int | def _resolve_drive_for_write(self, request: TapeOpRequest) -> int: if request.drive_id is not None: return request.drive_id - loaded_drive_id = self.library.find_drive_by_barcode(request.barcode) + # library is duck-typed (simulator or real backend), so its helpers are untyped. + loaded_drive_id: int | None = self.library.find_drive_by_barcode(request.barcode) if loaded_drive_id is not None: return loaded_drive_id return 0 @@ -528,7 +529,7 @@ def _source_slot(self, request: TapeOpRequest) -> int: "source_slot_id", request.extras.get("source_slot", request.slot_id) ) if source_slot is None: - slot_id = self.library.find_slot_by_barcode(request.barcode) + slot_id: int | None = self.library.find_slot_by_barcode(request.barcode) if slot_id is None: raise ValueError(f"Barcode {request.barcode} is not present in a slot") return slot_id @@ -563,7 +564,7 @@ def _dest_slot(self, request: TapeOpRequest) -> int: return destination_id def _find_empty_slot(self) -> int: - inventory = self.library.inventory() + inventory: LibraryInventory = self.library.inventory() for slot in inventory.slots: if slot.barcode is None: return slot.slot_id @@ -642,7 +643,11 @@ def execute_tape_request( active_repo: CatalogRepository | _TransientTapeOpRepository = ( repo or _TransientTapeOpRepository() ) - record = TapeOperationOrchestrator(active_repo, library, ltfs).execute(request) + # _TransientTapeOpRepository implements the create_tape_op/update_tape_op + # subset that execute() uses; the orchestrator's read helpers are not reached + # from this entry point. + orchestrator = TapeOperationOrchestrator(cast(CatalogRepository, active_repo), library, ltfs) + record = orchestrator.execute(request) if raise_on_failed and record.status is TapeOpStatus.FAILED: raise TapeOperationFailedError( record.error or f"Tape {request.op_type.value} operation failed" diff --git a/openblade/sftp/mock_server.py b/openblade/sftp/mock_server.py index 817dabe..875499f 100644 --- a/openblade/sftp/mock_server.py +++ b/openblade/sftp/mock_server.py @@ -13,7 +13,7 @@ import structlog -from openblade.nas.types import HydrationRequest, VirtualFileStatus +from openblade.nas.types import HydrationRequest, VirtualFileEntry, VirtualFileStatus from openblade.nas.virtual_fs import VirtualFilesystem logger = structlog.get_logger(__name__) @@ -99,7 +99,7 @@ def open(self, path: str, mode: str = "r") -> MockSftpFile: content = self._render_stub_content(entry.path, entry.status.value) return MockSftpFile(entry.path, content) - def _to_attributes(self, entry) -> SFTPAttributes: + def _to_attributes(self, entry: VirtualFileEntry) -> SFTPAttributes: timestamp = self._to_timestamp(entry.mtime) mode = _DIRECTORY_MODE if entry.is_directory else _FILE_MODE longname = f"{entry.status.value} {entry.size_bytes} {entry.path}" diff --git a/openblade/simulator/ltfs_volume.py b/openblade/simulator/ltfs_volume.py index 5e1f34b..7d5f19c 100644 --- a/openblade/simulator/ltfs_volume.py +++ b/openblade/simulator/ltfs_volume.py @@ -278,7 +278,11 @@ def unmount(self, handle: MountHandle) -> OperationResult: ) def write_file(self, handle: MountHandle, source: Path, dest: PurePosixPath) -> FileInstance: - return self.write_bytes(handle, dest, source.read_bytes()) + instance = self.write_bytes(handle, dest, source.read_bytes()) + # write_bytes only returns None on its barcode-string (metadata) branch; + # handle is a MountHandle here, so a FileInstance is always produced. + assert instance is not None + return instance def write_bytes( self, From 23c6777c9ac087c0b0c1ffc85b48346d7815e214 Mon Sep 17 00:00:00 2001 From: Amantux Date: Sat, 26 Sep 2026 06:34:46 +0000 Subject: [PATCH 26/54] fix: archive-job completion log crashed whenever INFO was enabled Latent bug mypy found (call-arg), with its regression test. `run_archive_job` ended with structlog-style keyword arguments on a *stdlib* logger: logger.info("archive job completed", job_id=job_id, files_archived=...) `logging.Logger.info` forwards unknown keywords to `Logger._log()`, which raises `TypeError: Logger._log() got an unexpected keyword argument 'job_id'`. It hid because `Logger.info` checks `isEnabledFor(INFO)` first: with the root logger at its default WARNING -- which is what the whole test suite runs at -- the call is a no-op. Any deployment that logs at INFO crashed the SUCCESS path of every archive job, *after* `catalog.update_job_state(job_id, "completed")`, so the catalog said completed while the caller got a 500. `logger` here is `logging.getLogger(__name__)`, not structlog (this is the only stdlib logger in the tree being called that way -- the other nine structured call sites all use `structlog.get_logger`), so the fields move into `extra`, which attaches them to the record instead of passing them to `_log()`. The test raises the level explicitly, because running the archive job without doing so proves nothing. Mutation-checked: reverting the one-line fix makes it fail with the TypeError. --- openblade/jobs/archive.py | 15 +++++- tests/unit/test_archive_job_logging.py | 73 ++++++++++++++++++++++++++ 2 files changed, 86 insertions(+), 2 deletions(-) create mode 100644 tests/unit/test_archive_job_logging.py diff --git a/openblade/jobs/archive.py b/openblade/jobs/archive.py index a3eb217..ecc424a 100644 --- a/openblade/jobs/archive.py +++ b/openblade/jobs/archive.py @@ -8,6 +8,7 @@ from pathlib import Path, PurePosixPath from openblade.api import aml_state +from openblade.catalog.models import Job from openblade.catalog.repository import CatalogRepository from openblade.domain.backends import LibraryBackend, LTFSBackend from openblade.domain.capacity import has_room_for @@ -348,7 +349,17 @@ def finalize_current_mount() -> None: errors=[], ) catalog.update_job_state(job_id, "completed") - logger.info("archive job completed", job_id=job_id, files_archived=files_archived) + # structlog-style kwargs on a stdlib logger: `Logger.info(msg, job_id=...)` + # raises TypeError from `Logger._log()`. It only ever fired when INFO was + # enabled -- `isEnabledFor(INFO)` short-circuits first -- so it slept through + # a test suite that never turns INFO on, and crashed the SUCCESS path of an + # archive job (after the catalog was already marked completed) in any + # deployment that logs at INFO. `logger` here is `logging.getLogger`, not + # structlog; structured fields go in `extra`. + logger.info( + "archive job completed", + extra={"job_id": job_id, "files_archived": files_archived}, + ) return result @@ -365,7 +376,7 @@ def __init__( self.catalog = catalog self.queue = queue - def enqueue(self, volume_group_name: str, source_path: Path): + def enqueue(self, volume_group_name: str, source_path: Path) -> Job: job = self.catalog.create_job( JobType.ARCHIVE.value, {"source_path": str(source_path), "volume_group": volume_group_name}, diff --git a/tests/unit/test_archive_job_logging.py b/tests/unit/test_archive_job_logging.py new file mode 100644 index 0000000..eec1105 --- /dev/null +++ b/tests/unit/test_archive_job_logging.py @@ -0,0 +1,73 @@ +"""Regression: the archive-job completion log must not crash when INFO is on. + +`run_archive_job` ended with structlog-style kwargs on a *stdlib* logger +(`logger.info("...", job_id=..., files_archived=...)`), which raises TypeError +inside `Logger._log()`. `Logger.info` checks `isEnabledFor(INFO)` first, so the +call was a no-op in a test suite that never enables INFO -- and blew up on the +SUCCESS path of a real archive job in any deployment that logs at INFO, after +the catalog had already been marked completed. + +The test therefore has to raise the level explicitly; running the archive job +without doing so proves nothing. +""" + +from __future__ import annotations + +import logging +from pathlib import Path + +import pytest + +from openblade.catalog.db import get_session, init_db +from openblade.catalog.repository import CatalogRepository +from openblade.domain.policies import FormatConfirmation, SafetyToken +from openblade.jobs.archive import ArchiveRequest, run_archive_job +from openblade.simulator.library import MockLibraryBackend +from openblade.simulator.ltfs_volume import MockLTFSBackend +from openblade.simulator.scenarios import one_drive_twenty_slots_five_cartridges + +_ARCHIVE_LOGGER = "openblade.jobs.archive" + + +def _formatted_stack() -> tuple[CatalogRepository, MockLibraryBackend, MockLTFSBackend]: + init_db("sqlite:///:memory:") + catalog = CatalogRepository(get_session()) + library, ltfs = one_drive_twenty_slots_five_cartridges() + barcode = str(library.inventory().slots[0].barcode) + library.load(1, 0) + ltfs.format(barcode, FormatConfirmation(barcode, SafetyToken.generate("format", barcode))) + library.unload(0, 1) + group = catalog.create_volume_group("photos") + catalog.add_barcode_to_volume_group(group.id, barcode) + return catalog, library, ltfs + + +def test_archive_job_completion_log_survives_info_level( + tmp_path: Path, caplog: pytest.LogCaptureFixture +) -> None: + catalog, library, ltfs = _formatted_stack() + source = tmp_path / "source" + source.mkdir() + (source / "a.txt").write_text("hello archive") + job = catalog.create_job("archive", {"source_path": str(source), "volume_group": "photos"}) + + with caplog.at_level(logging.INFO, logger=_ARCHIVE_LOGGER): + result = run_archive_job( + ArchiveRequest(source_path=source, volume_group_name="photos"), + library, + ltfs, + catalog, + job.id, + ) + + assert result.files_archived == 1 + completion = [ + record + for record in caplog.records + if record.name == _ARCHIVE_LOGGER and record.message == "archive job completed" + ] + assert len(completion) == 1 + # The structured fields survive the fix -- they moved into `extra`, which + # attaches them to the record rather than being passed to `_log()`. + assert completion[0].job_id == job.id + assert completion[0].files_archived == 1 From 53a35bc5dcf1a55985d7718d4da833f501a0a91e Mon Sep 17 00:00:00 2001 From: Amantux Date: Sat, 26 Sep 2026 06:34:51 +0000 Subject: [PATCH 27/54] simulator: honour the i3 profile's declared ieSlotCount MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The scalar-i3 profile has declared `ieSlotCount: 2` since it was written, but `scalar_i3_default()` never passed it to MockLibraryBackend, so the default simulator library reported an import/export station of ZERO elements while the /aml surface advertised two. Every mailslot operation therefore answered "this library has no import/export elements" — the opposite of what the config says. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_013S1qbQs31YSox1azQChqs2 --- openblade/simulator/scenarios.py | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/openblade/simulator/scenarios.py b/openblade/simulator/scenarios.py index 199b9a9..f6c4d9b 100644 --- a/openblade/simulator/scenarios.py +++ b/openblade/simulator/scenarios.py @@ -70,6 +70,12 @@ def scalar_i3_default() -> tuple[MockLibraryBackend, MockLTFSBackend]: library_id=str(config["library"].get("mockLibraryId", "mock-i3-001")), num_slots=int(partition["slotCount"]), num_drives=len(config["drives"]), + # The i3 profile has declared `ieSlotCount` since it was written, but this + # scenario never passed it on, so the default simulator backend reported an + # I/E station of zero elements: every mailslot operation answered "this + # library has no import/export elements" while /aml advertised two. The + # config is the single source of truth for the simulated library's shape. + num_import_export_slots=int(partition.get("ieSlotCount", 0)), ) for item in config["media"]: barcode = str(item["barcode"]) From 1e9d5b6802af5b5c8beb007950e595a724ff54ac Mon Sep 17 00:00:00 2001 From: Amantux Date: Sat, 26 Sep 2026 06:35:07 +0000 Subject: [PATCH 28/54] api: expose the mailslot (import/export) flows over HTTP MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `openblade mailslot list|import|export` had no HTTP surface, so the web UI could not drive the I/E station at all. This is the thin half: MailslotService keeps every decision (slot picking, the catalog state write) and the tape orchestrator keeps the export guard. - GET /mailslot/slots every I/E element and what is in it - GET /mailslot/export-preview/{barcode} what would leave, moving nothing - POST /mailslot/import {ie_slot, to_slot?} - POST /mailslot/export {barcode, ie_slot?, force?} An export that would take archived data out of the library answers 409 with the orchestrator's own refusal message — which names the cartridge, its volume group and sample paths — and `force: true` is the only way past it, exactly as `--force` is on the CLI. Tests cover the refusal and the forced override, and are mutation-checked: disabling `_guard_export` fails them. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_013S1qbQs31YSox1azQChqs2 --- openblade/api/main.py | 2 + openblade/api/routes_mailslot.py | 175 ++++++++++++++++++ tests/integration/test_mailslot_api.py | 234 +++++++++++++++++++++++++ 3 files changed, 411 insertions(+) create mode 100644 openblade/api/routes_mailslot.py create mode 100644 tests/integration/test_mailslot_api.py diff --git a/openblade/api/main.py b/openblade/api/main.py index 4b4f91e..cef5ca1 100644 --- a/openblade/api/main.py +++ b/openblade/api/main.py @@ -44,6 +44,7 @@ routes_jobs, routes_libraries, routes_ltfs, + routes_mailslot, routes_proxy, routes_rbac, routes_restore, @@ -226,6 +227,7 @@ async def apply_aml_emulator_latency(request: Request, call_next: object) -> Res app.include_router(routes_dashboard.router, prefix="/dashboard", tags=["dashboard"]) app.include_router(routes_ltfs.router, prefix="/ltfs", tags=["ltfs"]) app.include_router(routes_restore.router, prefix="/restore", tags=["restore"]) +app.include_router(routes_mailslot.router) app.include_router(routes_jobs.router, prefix="/jobs", tags=["jobs"]) app.include_router(routes_libraries.router) app.include_router(routes_assist.router, tags=["assist"]) diff --git a/openblade/api/routes_mailslot.py b/openblade/api/routes_mailslot.py new file mode 100644 index 0000000..aab4789 --- /dev/null +++ b/openblade/api/routes_mailslot.py @@ -0,0 +1,175 @@ +"""Import/export (mailslot) API endpoints. + +The mailslot flows have existed only as CLI commands (``openblade mailslot +list|import|export``) on top of :class:`openblade.nas.mailslot.MailslotService`. +This module is the HTTP half so the web UI can drive the same station; every +decision -- slot picking, the export refusal, the catalog state write -- stays in +the service and the tape orchestrator. Nothing here reimplements policy. + +The export refusal is the one thing worth reading twice: ``POST /mailslot/export`` +answers **409** with the orchestrator's own message, which names the cartridge, +its volume group and sample paths. ``force`` is the only way past it, exactly as +``--force`` is on the CLI, and callers are expected to have shown the operator the +``GET /mailslot/export-preview/{barcode}`` assessment first. +""" + +from __future__ import annotations + +from fastapi import APIRouter, Depends, HTTPException, status +from pydantic import BaseModel, Field + +from openblade.bootstrap import AppContext, get_context +from openblade.domain.errors import ( + CartridgeNotFoundError, + ExportRefusedError, + ImportExportSlotError, + MailslotUnsupportedError, +) +from openblade.nas.mailslot import MailslotMoveResult, MailslotService +from openblade.nas.tape_orchestrator import TapeOperationFailedError + +router = APIRouter(prefix="/mailslot", tags=["mailslot"]) + + +class MailslotSlotResponse(BaseModel): + slotId: int + occupied: bool + barcode: str | None + + +class MailslotListingResponse(BaseModel): + importExportSlots: list[MailslotSlotResponse] + slotCount: int + occupiedCount: int + #: False when the active backend has no import/export station at all. The UI + #: has to tell "this library has no mailslot" apart from "the mailslot is + #: empty" -- they look identical in the slot list and mean opposite things. + supported: bool + + +class ExportAssessmentResponse(BaseModel): + barcode: str + volumeGroup: str | None + archivedFilesOnCartridge: int + pendingFilesOnCartridge: int + bytesOnCartridge: int + samplePaths: list[str] + volumeGroupBarcodesWithData: list[str] + carriesData: bool + + +class MailslotImportRequest(BaseModel): + ie_slot: int = Field(ge=0, description="Import/export element holding the cartridge") + #: None means "pick the first empty storage slot"; the response says which. + to_slot: int | None = Field(default=None, ge=0) + + +class MailslotExportRequest(BaseModel): + barcode: str = Field(min_length=1, max_length=64) + ie_slot: int | None = Field(default=None, ge=0) + #: Export a cartridge that still carries archived data. The refusal names what + #: would leave; this is the operator saying they meant it. + force: bool = False + + +class MailslotMoveResponse(BaseModel): + opId: str + barcode: str + source: str + sourceSlot: int + destination: str + destinationSlot: int + destinationSlotChosen: bool + exported: ExportAssessmentResponse | None = None + + +def _service(context: AppContext) -> MailslotService: + return MailslotService(context.catalog, context.library, context.ltfs) + + +def _move_response(result: MailslotMoveResult) -> MailslotMoveResponse: + return MailslotMoveResponse.model_validate(result.to_dict()) + + +@router.get("/slots", response_model=MailslotListingResponse) +async def list_mailslot_slots( + context: AppContext = Depends(get_context), +) -> MailslotListingResponse: + """Every import/export element and what is in it.""" + try: + listing = _service(context).list_slots() + except MailslotUnsupportedError: + # Not an error the operator can act on: this backend simply has no I/E + # station. Reported as an empty, unsupported listing so the UI can say so. + return MailslotListingResponse( + importExportSlots=[], slotCount=0, occupiedCount=0, supported=False + ) + return MailslotListingResponse( + importExportSlots=[ + MailslotSlotResponse.model_validate(slot.to_dict()) for slot in listing.slots + ], + slotCount=listing.slot_count, + occupiedCount=len(listing.occupied), + supported=True, + ) + + +@router.get("/export-preview/{barcode}", response_model=ExportAssessmentResponse) +async def preview_mailslot_export( + barcode: str, + context: AppContext = Depends(get_context), +) -> ExportAssessmentResponse: + """What would leave with ``barcode``. Moves nothing.""" + assessment = _service(context).preview_export(barcode) + return ExportAssessmentResponse.model_validate(assessment.to_dict()) + + +@router.post("/import", response_model=MailslotMoveResponse) +async def import_from_mailslot( + payload: MailslotImportRequest, + context: AppContext = Depends(get_context), +) -> MailslotMoveResponse: + """Move a cartridge out of an I/E element into a storage slot.""" + try: + result = _service(context).import_cartridge(payload.ie_slot, payload.to_slot) + except MailslotUnsupportedError as exc: + raise HTTPException(status_code=status.HTTP_409_CONFLICT, detail=str(exc)) from None + except ImportExportSlotError as exc: + raise HTTPException(status_code=status.HTTP_409_CONFLICT, detail=str(exc)) from None + except CartridgeNotFoundError as exc: + raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)) from None + except TapeOperationFailedError as exc: + # Curated at the orchestrator's raise site (`_safe_error_message`). + raise HTTPException(status_code=status.HTTP_502_BAD_GATEWAY, detail=str(exc)) from None + return _move_response(result) + + +@router.post("/export", response_model=MailslotMoveResponse) +async def export_through_mailslot( + payload: MailslotExportRequest, + context: AppContext = Depends(get_context), +) -> MailslotMoveResponse: + """Move a cartridge out of storage into the I/E station. + + Refuses with 409 while the cartridge still carries archived or in-flight file + instances; the detail is the refusal message naming them. ``force: true`` + overrides, and is the HTTP spelling of ``--force``. + """ + try: + result = _service(context).export_cartridge( + payload.barcode, ie_slot=payload.ie_slot, force=payload.force + ) + except ExportRefusedError as exc: + # 409, not 400: the request is well-formed and the refusal is about the + # state of the cartridge. The message is operator-facing text that exists + # precisely to be read -- it is passed through verbatim. + raise HTTPException(status_code=status.HTTP_409_CONFLICT, detail=str(exc)) from None + except MailslotUnsupportedError as exc: + raise HTTPException(status_code=status.HTTP_409_CONFLICT, detail=str(exc)) from None + except ImportExportSlotError as exc: + raise HTTPException(status_code=status.HTTP_409_CONFLICT, detail=str(exc)) from None + except CartridgeNotFoundError as exc: + raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)) from None + except TapeOperationFailedError as exc: + raise HTTPException(status_code=status.HTTP_502_BAD_GATEWAY, detail=str(exc)) from None + return _move_response(result) diff --git a/tests/integration/test_mailslot_api.py b/tests/integration/test_mailslot_api.py new file mode 100644 index 0000000..78aec66 --- /dev/null +++ b/tests/integration/test_mailslot_api.py @@ -0,0 +1,234 @@ +"""``/mailslot/*`` -- the HTTP half of ``openblade mailslot``. + +The interesting assertions are the refusal ones: the API must not let archived +data walk out of the library on an unforced request, and it must say what would +have left. See ``openblade/catalog/export_policy.py``. +""" + +from __future__ import annotations + +from pathlib import Path + +import pytest +from fastapi.testclient import TestClient + +from openblade.api.main import app +from openblade.bootstrap import create_context, get_context, reset_context +from openblade.config import OpenBladeConfig + +SERVICE_TOKEN = "openblade-controller-dev-token-do-not-expose" + + +@pytest.fixture() +def client(tmp_path: Path) -> TestClient: + context = create_context(OpenBladeConfig(db_url=f"sqlite:///{tmp_path / 'mailslot.db'}")) + reset_context(context) + return TestClient(app) + + +def _data_barcodes(limit: int) -> list[str]: + context = get_context() + return [ + str(slot.barcode) + for slot in context.library.inventory().slots + if slot.barcode is not None and not str(slot.barcode).startswith("CLN") + ][:limit] + + +def _admin_auth_headers(client: TestClient) -> dict[str, str]: + response = client.post("/aml/users/login", json={"name": "admin", "password": "password"}) + assert response.status_code == 200 + session_id = response.cookies.get("sessionID") + assert session_id is not None + return {"Cookie": f"sessionID={session_id}"} + + +def _format_and_assign(client: TestClient, volume_group: str, barcode: str) -> None: + auth_headers = _admin_auth_headers(client) + assert client.post("/volume-groups/", json={"name": volume_group}).status_code == 201 + assert ( + client.post(f"/volume-groups/{volume_group}/assign", json={"barcode": barcode}).status_code + == 200 + ) + dry_run = client.post(f"/cartridges/{barcode}/format/dry-run", headers=auth_headers) + assert dry_run.status_code == 200 + assert ( + client.post( + "/cartridges/format/confirm", + json={"barcode": barcode, "token": dry_run.json()["token"]}, + headers={**auth_headers, "X-Openblade-Service-Token": SERVICE_TOKEN}, + ).status_code + == 200 + ) + + +def _archive_one_file( + client: TestClient, source_dir: Path, volume_group: str, file_name: str +) -> None: + source_dir.mkdir() + (source_dir / file_name).write_text("mailslot export policy fixture") + response = client.post( + "/archive/", + json={"source_path": str(source_dir), "volume_group": volume_group}, + ) + assert response.status_code == 202 + + +def test_slots_lists_every_import_export_element(client: TestClient) -> None: + response = client.get("/mailslot/slots") + assert response.status_code == 200 + payload = response.json() + assert payload["supported"] is True + # The i3 profile declares two I/E elements; the listing reports all of them, + # empty ones included, because "empty" is the answer to "can I export?". + assert payload["slotCount"] == len(payload["importExportSlots"]) == 2 + assert payload["occupiedCount"] == 0 + assert all( + slot["occupied"] is False and slot["barcode"] is None + for slot in payload["importExportSlots"] + ) + + +def test_export_then_import_round_trips_a_scratch_cartridge(client: TestClient) -> None: + barcode = _data_barcodes(limit=1)[0] + + exported = client.post("/mailslot/export", json={"barcode": barcode}) + assert exported.status_code == 200, exported.text + export_payload = exported.json() + assert export_payload["barcode"] == barcode + assert export_payload["source"] == "storage_slot" + assert export_payload["destination"] == "import_export_slot" + # Nobody named a destination element, so the service picked one and says so. + assert export_payload["destinationSlotChosen"] is True + ie_slot = export_payload["destinationSlot"] + + occupied = client.get("/mailslot/slots").json() + assert occupied["occupiedCount"] == 1 + assert [slot["barcode"] for slot in occupied["importExportSlots"] if slot["occupied"]] == [ + barcode + ] + + imported = client.post("/mailslot/import", json={"ie_slot": ie_slot}) + assert imported.status_code == 200, imported.text + import_payload = imported.json() + assert import_payload["barcode"] == barcode + assert import_payload["source"] == "import_export_slot" + assert import_payload["sourceSlot"] == ie_slot + assert import_payload["destinationSlotChosen"] is True + assert client.get("/mailslot/slots").json()["occupiedCount"] == 0 + + +def test_import_honours_an_explicit_target_slot(client: TestClient) -> None: + barcode = _data_barcodes(limit=1)[0] + ie_slot = client.post("/mailslot/export", json={"barcode": barcode}).json()["destinationSlot"] + + context = get_context() + empty_slot = next( + slot.slot_id for slot in context.library.inventory().slots if slot.barcode is None + ) + imported = client.post("/mailslot/import", json={"ie_slot": ie_slot, "to_slot": empty_slot}) + assert imported.status_code == 200, imported.text + assert imported.json()["destinationSlot"] == empty_slot + assert imported.json()["destinationSlotChosen"] is False + + +def test_import_from_an_empty_element_is_a_409_naming_the_element(client: TestClient) -> None: + response = client.post("/mailslot/import", json={"ie_slot": 51}) + assert response.status_code == 409 + assert "51" in response.json()["detail"] + + +def test_import_from_an_unknown_element_says_which_exist(client: TestClient) -> None: + response = client.post("/mailslot/import", json={"ie_slot": 9999}) + assert response.status_code == 409 + assert "9999" in response.json()["detail"] + assert "does not exist" in response.json()["detail"] + + +def test_export_preview_reports_what_would_leave(client: TestClient, tmp_path: Path) -> None: + barcode = _data_barcodes(limit=1)[0] + _format_and_assign(client, "photos", barcode) + _archive_one_file(client, tmp_path / "photos-source", "photos", "a.txt") + + response = client.get(f"/mailslot/export-preview/{barcode}") + assert response.status_code == 200 + payload = response.json() + assert payload["barcode"] == barcode + assert payload["carriesData"] is True + assert payload["archivedFilesOnCartridge"] >= 1 + assert payload["bytesOnCartridge"] > 0 + assert payload["volumeGroup"] == "photos" + assert any(path.endswith("a.txt") for path in payload["samplePaths"]) + # A preview moves nothing: the cartridge is still in storage afterwards. + assert client.get("/mailslot/slots").json()["occupiedCount"] == 0 + + +def test_export_preview_of_a_scratch_cartridge_carries_no_data(client: TestClient) -> None: + barcode = _data_barcodes(limit=1)[0] + payload = client.get(f"/mailslot/export-preview/{barcode}").json() + assert payload["carriesData"] is False + assert payload["archivedFilesOnCartridge"] == 0 + assert payload["samplePaths"] == [] + + +def test_export_of_a_cartridge_carrying_data_is_refused_and_names_it( + client: TestClient, tmp_path: Path +) -> None: + barcode = _data_barcodes(limit=1)[0] + _format_and_assign(client, "photos", barcode) + _archive_one_file(client, tmp_path / "photos-source", "photos", "a.txt") + + response = client.post("/mailslot/export", json={"barcode": barcode}) + assert response.status_code == 409 + detail = response.json()["detail"] + assert barcode in detail + assert "still carries archived data" in detail + assert "photos" in detail + assert "a.txt" in detail + # Refused means nothing moved: the I/E station is still empty. + assert client.get("/mailslot/slots").json()["occupiedCount"] == 0 + + +def test_force_exports_the_same_cartridge(client: TestClient, tmp_path: Path) -> None: + barcode = _data_barcodes(limit=1)[0] + _format_and_assign(client, "photos", barcode) + _archive_one_file(client, tmp_path / "photos-source", "photos", "a.txt") + assert client.post("/mailslot/export", json={"barcode": barcode}).status_code == 409 + + forced = client.post("/mailslot/export", json={"barcode": barcode, "force": True}) + assert forced.status_code == 200, forced.text + payload = forced.json() + assert payload["barcode"] == barcode + # The response carries the assessment, so an audit of a forced export still + # records what left the library. + assert payload["exported"]["carriesData"] is True + assert payload["exported"]["archivedFilesOnCartridge"] >= 1 + assert client.get("/mailslot/slots").json()["occupiedCount"] == 1 + + +def test_export_of_an_unknown_barcode_is_a_404(client: TestClient) -> None: + response = client.post("/mailslot/export", json={"barcode": "NOSUCH1L8"}) + assert response.status_code == 404 + assert "NOSUCH1L8" in response.json()["detail"] + + +def test_export_into_an_occupied_element_is_refused(client: TestClient) -> None: + first, second = _data_barcodes(limit=2) + occupied_slot = client.post("/mailslot/export", json={"barcode": first}).json()[ + "destinationSlot" + ] + + response = client.post("/mailslot/export", json={"barcode": second, "ie_slot": occupied_slot}) + assert response.status_code == 409 + assert first in response.json()["detail"] + + +def test_export_refuses_once_every_element_is_full(client: TestClient) -> None: + barcodes = _data_barcodes(limit=3) + slot_count = client.get("/mailslot/slots").json()["slotCount"] + for barcode in barcodes[:slot_count]: + assert client.post("/mailslot/export", json={"barcode": barcode}).status_code == 200 + + response = client.post("/mailslot/export", json={"barcode": barcodes[slot_count]}) + assert response.status_code == 409 + assert "empty the mailslot" in response.json()["detail"] From ef121ca81190e8f9fecf37da49b05fee68a36398 Mon Sep 17 00:00:00 2001 From: Amantux Date: Sat, 26 Sep 2026 06:35:17 +0000 Subject: [PATCH 29/54] api: add a read-only drive-health (inquiry + TapeAlert) route MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `openblade hardware drive-health` was CLI-only. GET /hardware/drive-health serves the same report so the console can show a failing drive. Read-only and guarded: it goes through require_real_hardware(), so on the simulator backend it answers 503 with the curated reason naming both hardware variables rather than going looking for a device. A drive that does not implement the TapeAlert log page is a 200 with `tapeAlertSupported: false` and a reason — that is a property of the drive, not a failure. The blocking sg_inq / sg_logs subprocesses run off the event loop so a 30s device timeout cannot stall the AML parity surface. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_013S1qbQs31YSox1azQChqs2 --- openblade/api/main.py | 2 + openblade/api/routes_drive_health.py | 131 +++++++++++++++++++ tests/integration/test_drive_health_api.py | 144 +++++++++++++++++++++ 3 files changed, 277 insertions(+) create mode 100644 openblade/api/routes_drive_health.py create mode 100644 tests/integration/test_drive_health_api.py diff --git a/openblade/api/main.py b/openblade/api/main.py index cef5ca1..32606e4 100644 --- a/openblade/api/main.py +++ b/openblade/api/main.py @@ -38,6 +38,7 @@ routes_assist, routes_catalog, routes_dashboard, + routes_drive_health, routes_health, routes_iblade, routes_inventory, @@ -228,6 +229,7 @@ async def apply_aml_emulator_latency(request: Request, call_next: object) -> Res app.include_router(routes_ltfs.router, prefix="/ltfs", tags=["ltfs"]) app.include_router(routes_restore.router, prefix="/restore", tags=["restore"]) app.include_router(routes_mailslot.router) +app.include_router(routes_drive_health.router) app.include_router(routes_jobs.router, prefix="/jobs", tags=["jobs"]) app.include_router(routes_libraries.router) app.include_router(routes_assist.router, tags=["assist"]) diff --git a/openblade/api/routes_drive_health.py b/openblade/api/routes_drive_health.py new file mode 100644 index 0000000..aa37b96 --- /dev/null +++ b/openblade/api/routes_drive_health.py @@ -0,0 +1,131 @@ +"""Read-only drive health: SCSI inquiry plus TapeAlert flags. + +The same report ``openblade hardware drive-health`` prints +(``openblade/cli/fuse_and_health.py``), served over HTTP so the console can show +it. **Read-only and guarded**: every path goes through +:func:`openblade.hardware.safety.require_real_hardware`, so with the simulator +backend -- the default -- this answers 503 with the curated reason and touches no +device. There is no write here and there must never be one: TapeAlert is a LOG +SENSE read, and this module exists so an operator can *see* a failing drive. + +A drive that does not implement the TapeAlert log page is reported with +``tapeAlertSupported: false`` and a reason. That is a fact about the drive, not an +error, and it is a 200. +""" + +from __future__ import annotations + +from anyio import to_thread +from fastapi import APIRouter, Depends, HTTPException, Query, status +from pydantic import BaseModel + +from openblade.bootstrap import AppContext, get_context +from openblade.domain.errors import RealHardwareDisabledError +from openblade.domain.policies import RealHardwareGuard +from openblade.hardware.discovery import discover_library +from openblade.hardware.runner import SafeRunner +from openblade.hardware.safety import require_real_hardware +from openblade.hardware.sg import ScsiInquiry, sg_inq +from openblade.hardware.tapealert import TapeAlertReport, read_tape_alerts + +router = APIRouter(prefix="/hardware", tags=["hardware"]) + + +class DriveInquiryResponse(BaseModel): + deviceType: str + vendor: str + product: str + revision: str + #: Empty when the drive reports no unit serial number. Never guessed. + serial: str + + +class TapeAlertFlagResponse(BaseModel): + #: None when ``sg_logs`` named a flag this build's spec table does not know. + number: int | None + name: str + severity: str + + +class DriveHealthResponse(BaseModel): + device: str + inquiry: DriveInquiryResponse + tapeAlertSupported: bool + tapeAlertReason: str | None + #: Severity of the worst flag that is SET, or None when nothing is set. + worstSeverity: str | None + flagsRead: int + activeFlags: list[TapeAlertFlagResponse] + + +class DriveHealthListResponse(BaseModel): + drives: list[DriveHealthResponse] + + +def _serialize(device: str, inquiry: ScsiInquiry, report: TapeAlertReport) -> DriveHealthResponse: + worst = report.worst_severity + return DriveHealthResponse( + device=device, + inquiry=DriveInquiryResponse( + deviceType=inquiry.device_type, + vendor=inquiry.vendor, + product=inquiry.product, + revision=inquiry.revision, + serial=inquiry.serial, + ), + tapeAlertSupported=report.supported, + tapeAlertReason=report.reason, + worstSeverity=None if worst is None else str(worst), + flagsRead=len(report.flags), + activeFlags=[ + TapeAlertFlagResponse(number=flag.number, name=flag.name, severity=str(flag.severity)) + for flag in report.active + ], + ) + + +def _collect( + devices: list[str] | None, runner: SafeRunner, guard: RealHardwareGuard +) -> list[DriveHealthResponse]: + """Blocking half: discovery plus one INQUIRY and one LOG SENSE per drive.""" + targets = devices + if targets is None: + discovery = discover_library(runner, guard) + targets = [ + node + for node in (drive.sg_device or drive.block_device for drive in discovery.drives) + if node is not None + ] + return [ + _serialize(target, sg_inq(target, runner, guard), read_tape_alerts(target, runner, guard)) + for target in targets + ] + + +@router.get("/drive-health", response_model=DriveHealthListResponse) +async def get_drive_health( + device: str | None = Query( + default=None, + min_length=1, + max_length=256, + description="Tape device to inspect, e.g. /dev/nst0. Default: every discovered drive.", + ), + context: AppContext = Depends(get_context), +) -> DriveHealthListResponse: + """Inquiry data and TapeAlert flags for one drive, or every discovered drive.""" + try: + guard = require_real_hardware(context.config) + except RealHardwareDisabledError as exc: + # 503, not 403: nothing is wrong with the request or the caller's rights -- + # this deployment has no real drives to inspect. The detail is the curated + # guard message naming the two variables that turn it on. + raise HTTPException( + status_code=status.HTTP_503_SERVICE_UNAVAILABLE, detail=str(exc) + ) from None + + runner = SafeRunner(dry_run=context.config.hardware_dry_run) + devices = None if device is None else [device] + # Off the event loop: `sg_inq`/`sg_logs` are subprocesses with a 30s timeout + # each, and this process also serves the AML emulator parity surface. + drives = await to_thread.run_sync(_collect, devices, runner, guard) + return DriveHealthListResponse(drives=drives) diff --git a/tests/integration/test_drive_health_api.py b/tests/integration/test_drive_health_api.py new file mode 100644 index 0000000..23fe487 --- /dev/null +++ b/tests/integration/test_drive_health_api.py @@ -0,0 +1,144 @@ +"""``GET /hardware/drive-health`` -- the HTTP half of ``openblade hardware drive-health``. + +Read-only and guarded. Two properties matter more than the payload shape: + +* with the simulator backend (the default) the route must refuse rather than go + looking for a device, and +* a drive without the TapeAlert log page must come back as a 200 saying so, not + as an error -- that is a property of the drive. + +The "real hardware" cases run with ``hardware_dry_run=True``, so the guard is +satisfied and ``SafeRunner`` returns the sample ``sg_inq``/``sg_logs`` output +captured from a real rig instead of executing anything. +""" + +from __future__ import annotations + +from dataclasses import replace +from pathlib import Path + +import pytest +from fastapi.testclient import TestClient + +from openblade.api.main import app +from openblade.bootstrap import create_context, get_context, reset_context +from openblade.config import BackendMode, OpenBladeConfig +from openblade.hardware import tapealert + + +@pytest.fixture() +def client(tmp_path: Path) -> TestClient: + context = create_context(OpenBladeConfig(db_url=f"sqlite:///{tmp_path / 'health.db'}")) + reset_context(context) + return TestClient(app) + + +@pytest.fixture() +def hardware_enabled() -> None: + """Claim real hardware while keeping the simulator backends. + + The route only consults ``context.config``; swapping the config alone + exercises the guarded path without building real backends in a test process. + """ + context = get_context() + reset_context( + replace( + context, + config=replace( + context.config, + backend=BackendMode.REAL, + real_hardware_enabled=True, + hardware_dry_run=True, + ), + ) + ) + + +def test_simulator_backend_refuses_with_the_curated_reason(client: TestClient) -> None: + response = client.get("/hardware/drive-health") + assert response.status_code == 503 + detail = response.json()["detail"] + # The refusal names both variables, because "unavailable" alone is unactionable. + assert "OPENBLADE_BACKEND=real" in detail + assert "OPENBLADE_REAL_HARDWARE_ENABLED=true" in detail + + +def test_one_named_device_reports_inquiry_and_tapealert( + client: TestClient, hardware_enabled: None +) -> None: + response = client.get("/hardware/drive-health", params={"device": "/dev/sg1"}) + assert response.status_code == 200 + drives = response.json()["drives"] + assert len(drives) == 1 + drive = drives[0] + assert drive["device"] == "/dev/sg1" + assert drive["inquiry"]["vendor"] + assert drive["inquiry"]["product"] + assert drive["tapeAlertSupported"] is True + # The clean sample sets no flags: every flag is read, none is active. + assert drive["flagsRead"] > 0 + assert drive["activeFlags"] == [] + assert drive["worstSeverity"] is None + assert drive["tapeAlertReason"] is None + + +def test_every_discovered_drive_is_reported_when_no_device_is_named( + client: TestClient, hardware_enabled: None +) -> None: + response = client.get("/hardware/drive-health") + assert response.status_code == 200 + drives = response.json()["drives"] + assert len(drives) >= 1 + assert len({drive["device"] for drive in drives}) == len(drives) + + +def test_active_flags_carry_their_severity( + client: TestClient, hardware_enabled: None, monkeypatch: pytest.MonkeyPatch +) -> None: + report = tapealert.parse_sg_logs_tapealert( + tapealert.SAMPLE_SG_LOGS_TAPEALERT_CLEAN, device="/dev/sg1" + ) + flags = tuple( + replace(flag, value=flag.number in {4, 3}) if flag.number in {4, 3} else flag + for flag in report.flags + ) + monkeypatch.setattr( + "openblade.api.routes_drive_health.read_tape_alerts", + lambda device, runner, guard: replace(report, flags=flags), + ) + + response = client.get("/hardware/drive-health", params={"device": "/dev/sg1"}) + assert response.status_code == 200 + drive = response.json()["drives"][0] + severities = {flag["number"]: flag["severity"] for flag in drive["activeFlags"]} + assert severities == {4: "critical", 3: "warning"} + # Worst-first, so a UI badge cannot report a critical drive as merely warned. + assert drive["worstSeverity"] == "critical" + + +def test_a_drive_without_the_log_page_is_a_200_with_a_reason( + client: TestClient, hardware_enabled: None, monkeypatch: pytest.MonkeyPatch +) -> None: + monkeypatch.setattr( + "openblade.api.routes_drive_health.read_tape_alerts", + lambda device, runner, guard: tapealert.TapeAlertReport( + device=device, + supported=False, + flags=(), + reason="no TapeAlert page in sg_logs output", + ), + ) + + response = client.get("/hardware/drive-health", params={"device": "/dev/sg1"}) + assert response.status_code == 200 + drive = response.json()["drives"][0] + assert drive["tapeAlertSupported"] is False + assert drive["tapeAlertReason"] == "no TapeAlert page in sg_logs output" + assert drive["activeFlags"] == [] + + +def test_the_route_is_read_only(client: TestClient, hardware_enabled: None) -> None: + # Not a formality: drive health is the one surface that talks SCSI to a drive + # a human is worried about, and a write here would be the worst possible bug. + for method in (client.post, client.put, client.delete, client.patch): + assert method("/hardware/drive-health").status_code == 405 From 2d468f7d23836c2ebcfff5163a598641e5c9808c Mon Sep 17 00:00:00 2001 From: Amantux Date: Sat, 26 Sep 2026 06:35:27 +0000 Subject: [PATCH 30/54] api: add POST /restore/tree (spanning tree restore, with dry run) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `openblade restore tree` was CLI-only. The route reuses run_tree_restore and answers with TreeRestoreResult.to_dict() verbatim, so the CLI and the API cannot drift: same request, same payload, same job row. `dry_run: true` plans only and is the preview a UI should call first — it is the only place the file and tape count shows up before media moves. Failures answer with safe_job_error() text, so mkltfs/mtx output never reaches the wire, and the job row is marked failed. It runs on the event loop thread deliberately, like POST /restore/: the catalog session is process-global and not thread-safe. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_013S1qbQs31YSox1azQChqs2 --- openblade/api/routes_restore.py | 91 +++++++++- tests/integration/test_tree_restore_api.py | 184 +++++++++++++++++++++ 2 files changed, 274 insertions(+), 1 deletion(-) create mode 100644 tests/integration/test_tree_restore_api.py diff --git a/openblade/api/routes_restore.py b/openblade/api/routes_restore.py index cfffdb7..982034b 100644 --- a/openblade/api/routes_restore.py +++ b/openblade/api/routes_restore.py @@ -7,14 +7,16 @@ from uuid import uuid4 from fastapi import APIRouter, Depends, HTTPException, status -from pydantic import BaseModel +from pydantic import BaseModel, Field from openblade.api import aml_state from openblade.bootstrap import AppContext, get_context +from openblade.domain.errors import safe_job_error from openblade.jobs.restore import RestoreRequest as RestoreJobRequest from openblade.jobs.restore import run_restore_job from openblade.jobs.scheduler import DriveScheduler from openblade.jobs.sharded_restore import ShardedRestoreRequest, run_sharded_restore +from openblade.jobs.tree_restore import TreeRestoreRequest, run_tree_restore router = APIRouter() @@ -91,6 +93,39 @@ class EnqueuedJobResponse(BaseModel): status: str +class TreeRestoreApiRequest(BaseModel): + catalog_prefix: str = Field(min_length=1) + dest_dir: str = Field(min_length=1) + #: Plan only: counts and per-tape totals come back, no media moves. + dry_run: bool = False + + +class TreeRestoreFailureResponse(BaseModel): + catalogPath: str + error: str + + +class TreeRestoreResponse(BaseModel): + """The shape ``openblade restore tree`` already emits (``TreeRestoreResult``).""" + + jobId: str + catalogPrefix: str + destDir: str + dryRun: bool + filesRestored: int + filesFailed: int + bytesRestored: int + filesVerified: int + perTapeCounts: dict[str, int] + tapesUsed: list[str] + failures: list[TreeRestoreFailureResponse] + #: Catalogued records with nothing archived. Not failures, but an operator who + #: asked for a tree and got fewer files back has to be told which ones. + filesSkipped: int + skippedPaths: list[str] + status: str + + @router.post("/", response_model=EnqueuedJobResponse, status_code=status.HTTP_202_ACCEPTED) async def enqueue_restore( request: RestoreRequest, @@ -153,3 +188,57 @@ async def enqueue_restore( error=refreshed.error, ) return EnqueuedJobResponse(job_id=refreshed.id, status="pending") + + +@router.post("/tree", response_model=TreeRestoreResponse) +async def restore_tree( + request: TreeRestoreApiRequest, + context: AppContext = Depends(get_context), +) -> TreeRestoreResponse: + """Restore every archived file under a catalog prefix, spanning tapes. + + The HTTP half of ``openblade restore tree``: same request, same + ``TreeRestoreResult`` payload, same job row. ``dry_run: true`` plans only and + is what a UI should call first — it is the only preview of how many files and + which tapes a tree restore will touch. + + Runs to completion before answering (like ``POST /restore/``) rather than + enqueueing, so the caller gets the authoritative per-tape result; progress + ticks are the CLI's stderr affordance and have no home in one response. + """ + job = context.catalog.create_job( + "restore", + { + "catalog_prefix": request.catalog_prefix, + "dest_dir": request.dest_dir, + "bulk": True, + "dry_run": request.dry_run, + }, + ) + scheduler = DriveScheduler(num_drives=len(context.library.inventory().drives)) + try: + # Deliberately NOT handed to a worker thread: `context.catalog` wraps one + # process-global SQLAlchemy Session which is not thread-safe, and every + # other handler in this router touches it from the event loop thread only + # (see routes_assist.py for what running it off-loop costs). + result = run_tree_restore( + TreeRestoreRequest( + catalog_prefix=request.catalog_prefix, + dest_dir=Path(request.dest_dir), + dry_run=request.dry_run, + ), + context.library, + context.ltfs, + context.catalog, + scheduler, + job.id, + ) + except Exception as exc: + # Curated text only: a tree restore wraps mkltfs/mtx output, which must + # never reach the wire (see safe_job_error). + message = safe_job_error(exc) + context.catalog.update_job_state(job.id, "failed", error=message) + raise HTTPException( + status_code=status.HTTP_500_INTERNAL_SERVER_ERROR, detail=message + ) from None + return TreeRestoreResponse.model_validate(result.to_dict()) diff --git a/tests/integration/test_tree_restore_api.py b/tests/integration/test_tree_restore_api.py new file mode 100644 index 0000000..895e5d8 --- /dev/null +++ b/tests/integration/test_tree_restore_api.py @@ -0,0 +1,184 @@ +"""``POST /restore/tree`` -- the HTTP half of ``openblade restore tree``. + +The payload is ``TreeRestoreResult.to_dict()``, so the CLI and the API cannot +drift. What is asserted here is the behaviour a UI depends on: a dry run moves +nothing, a real run writes the tree under the destination preserving the source +layout, and a short result names the files it did not restore. +""" + +from __future__ import annotations + +from pathlib import Path + +import pytest +from fastapi.testclient import TestClient + +from openblade.api.main import app +from openblade.bootstrap import create_context, get_context, reset_context +from openblade.config import OpenBladeConfig + +SERVICE_TOKEN = "openblade-controller-dev-token-do-not-expose" + + +@pytest.fixture() +def client(tmp_path: Path) -> TestClient: + context = create_context(OpenBladeConfig(db_url=f"sqlite:///{tmp_path / 'tree.db'}")) + reset_context(context) + return TestClient(app) + + +def _data_barcodes(limit: int) -> list[str]: + context = get_context() + return [ + str(slot.barcode) + for slot in context.library.inventory().slots + if slot.barcode is not None and not str(slot.barcode).startswith("CLN") + ][:limit] + + +def _admin_auth_headers(client: TestClient) -> dict[str, str]: + response = client.post("/aml/users/login", json={"name": "admin", "password": "password"}) + assert response.status_code == 200 + session_id = response.cookies.get("sessionID") + assert session_id is not None + return {"Cookie": f"sessionID={session_id}"} + + +def _format_and_assign(client: TestClient, volume_group: str, barcode: str) -> None: + auth_headers = _admin_auth_headers(client) + assert client.post("/volume-groups/", json={"name": volume_group}).status_code == 201 + assert ( + client.post(f"/volume-groups/{volume_group}/assign", json={"barcode": barcode}).status_code + == 200 + ) + dry_run = client.post(f"/cartridges/{barcode}/format/dry-run", headers=auth_headers) + assert dry_run.status_code == 200 + assert ( + client.post( + "/cartridges/format/confirm", + json={"barcode": barcode, "token": dry_run.json()["token"]}, + headers={**auth_headers, "X-Openblade-Service-Token": SERVICE_TOKEN}, + ).status_code + == 200 + ) + + +def _archive_tree(client: TestClient, source_dir: Path, volume_group: str) -> None: + (source_dir / "nested").mkdir(parents=True) + (source_dir / "top.txt").write_text("top level file") + (source_dir / "nested" / "deep.txt").write_text("nested file") + response = client.post( + "/archive/", + json={"source_path": str(source_dir), "volume_group": volume_group}, + ) + assert response.status_code == 202 + + +@pytest.fixture() +def archived(client: TestClient, tmp_path: Path) -> str: + barcode = _data_barcodes(limit=1)[0] + _format_and_assign(client, "photos", barcode) + _archive_tree(client, tmp_path / "photos-source", "photos") + return barcode + + +def test_dry_run_plans_without_writing_anything( + client: TestClient, tmp_path: Path, archived: str +) -> None: + destination = tmp_path / "planned" + response = client.post( + "/restore/tree", + json={"catalog_prefix": "/photos", "dest_dir": str(destination), "dry_run": True}, + ) + assert response.status_code == 200, response.text + payload = response.json() + assert payload["dryRun"] is True + assert payload["catalogPrefix"] == "/photos" + assert payload["filesRestored"] == 2 + assert payload["bytesRestored"] > 0 + assert payload["tapesUsed"] == [archived] + assert payload["perTapeCounts"][archived] == 2 + assert payload["failures"] == [] + assert payload["status"] == "completed" + # A preview that leaves files behind is the whole point of a preview. + assert not destination.exists() + + +def test_run_restores_the_tree_preserving_the_source_layout( + client: TestClient, tmp_path: Path, archived: str +) -> None: + destination = tmp_path / "restored" + response = client.post( + "/restore/tree", + json={"catalog_prefix": "/photos", "dest_dir": str(destination), "dry_run": False}, + ) + assert response.status_code == 200, response.text + payload = response.json() + assert payload["dryRun"] is False + assert payload["filesRestored"] == 2 + assert payload["status"] == "completed" + assert (destination / "top.txt").read_text() == "top level file" + assert (destination / "nested" / "deep.txt").read_text() == "nested file" + + +def test_the_job_row_records_the_tree_restore( + client: TestClient, tmp_path: Path, archived: str +) -> None: + payload = client.post( + "/restore/tree", + json={"catalog_prefix": "/photos", "dest_dir": str(tmp_path / "out"), "dry_run": False}, + ).json() + job = client.get(f"/jobs/{payload['jobId']}") + assert job.status_code == 200 + assert job.json()["job_type"] == "restore" + + +def test_a_prefix_with_nothing_archived_is_an_honest_empty_result( + client: TestClient, tmp_path: Path +) -> None: + response = client.post( + "/restore/tree", + json={"catalog_prefix": "/nothing-here", "dest_dir": str(tmp_path / "out")}, + ) + assert response.status_code == 200 + payload = response.json() + assert payload["filesRestored"] == 0 + assert payload["filesSkipped"] == 0 + assert payload["tapesUsed"] == [] + # dry_run defaults to False: an absent flag must not silently plan-only. + assert payload["dryRun"] is False + + +def test_a_blank_prefix_or_destination_is_rejected(client: TestClient, tmp_path: Path) -> None: + assert ( + client.post( + "/restore/tree", json={"catalog_prefix": "", "dest_dir": str(tmp_path / "out")} + ).status_code + == 422 + ) + assert ( + client.post("/restore/tree", json={"catalog_prefix": "/photos", "dest_dir": ""}).status_code + == 422 + ) + + +def test_a_failure_answers_with_curated_text_and_fails_the_job( + client: TestClient, tmp_path: Path, archived: str, monkeypatch: pytest.MonkeyPatch +) -> None: + class Boom(RuntimeError): + """Stands in for a tool failure carrying argv and device paths.""" + + def explode(*_args: object, **_kwargs: object) -> None: + raise Boom("mkltfs /dev/sg3 failed: password=hunter2") + + monkeypatch.setattr("openblade.api.routes_restore.run_tree_restore", explode) + response = client.post( + "/restore/tree", + json={"catalog_prefix": "/photos", "dest_dir": str(tmp_path / "out")}, + ) + assert response.status_code == 500 + detail = response.json()["detail"] + # Untyped failures never reach the wire verbatim (safe_job_error). + assert "hunter2" not in detail + assert "/dev/sg3" not in detail + assert "Boom" in detail From 9b7d338ffb8c98be09f17ebee6676f762fd1d764 Mon Sep 17 00:00:00 2001 From: Amantux Date: Sat, 26 Sep 2026 06:35:27 +0000 Subject: [PATCH 31/54] docs: regenerate the wiki API reference for the six new native operations tools/gen_wiki_reference.py output; guarded by tests/unit/test_wiki_reference_generated.py. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_013S1qbQs31YSox1azQChqs2 --- docs/wiki/reference/api.md | 20 ++++++++++++++++++-- 1 file changed, 18 insertions(+), 2 deletions(-) diff --git a/docs/wiki/reference/api.md b/docs/wiki/reference/api.md index cd0b478..4b75fe4 100644 --- a/docs/wiki/reference/api.md +++ b/docs/wiki/reference/api.md @@ -6,7 +6,7 @@ Introspected from the OpenAPI schema of `openblade.api.main:app`. -**1172 operations** total: 192 on the native OpenBlade control plane, 980 on the Quantum AML / iBlade emulator surface. +**1178 operations** total: 198 on the native OpenBlade control plane, 980 on the Quantum AML / iBlade emulator surface. The application serves two surfaces from one ASGI app. Setting `OPENBLADE_SCALAR_API_ONLY=true` puts it in emulator-only mode, where the @@ -76,11 +76,12 @@ Interactive docs for a running instance are at `/docs` and `/redoc`. | `POST` | `/archive/` | Enqueue Archive | - | `ArchiveRequest` | `EnqueuedJobResponse` | | `POST` | `/archive/sharded` | Enqueue Sharded Archive | - | `ShardedArchiveApiRequest` | `EnqueuedJobResponse` | -### `restore` (1 operation) +### `restore` (2 operations) | Method | Path | Summary | Parameters | Request | Response | | --- | --- | --- | --- | --- | --- | | `POST` | `/restore/` | Enqueue Restore | - | `openblade__api__routes_restore__RestoreRequest` | `EnqueuedJobResponse` | +| `POST` | `/restore/tree` | Restore Tree | - | `TreeRestoreApiRequest` | `TreeRestoreResponse` | ### `jobs` (2 operations) @@ -313,6 +314,21 @@ Interactive docs for a running instance are at `/docs` and `/redoc`. | --- | --- | --- | --- | --- | --- | | `POST` | `/assist` | Assist | - | `AssistRequest` | `AssistResponse` | +### `hardware` (1 operation) + +| Method | Path | Summary | Parameters | Request | Response | +| --- | --- | --- | --- | --- | --- | +| `GET` | `/hardware/drive-health` | Get Drive Health | `device`? (query) | - | `DriveHealthListResponse` | + +### `mailslot` (4 operations) + +| Method | Path | Summary | Parameters | Request | Response | +| --- | --- | --- | --- | --- | --- | +| `POST` | `/mailslot/export` | Export Through Mailslot | - | `MailslotExportRequest` | `MailslotMoveResponse` | +| `GET` | `/mailslot/export-preview/{barcode}` | Preview Mailslot Export | `barcode` (path) | - | `ExportAssessmentResponse` | +| `POST` | `/mailslot/import` | Import From Mailslot | - | `MailslotImportRequest` | `MailslotMoveResponse` | +| `GET` | `/mailslot/slots` | List Mailslot Slots | - | - | `MailslotListingResponse` | + ## Quantum AML / iBlade emulator surface These operations implement the Quantum Scalar i3/i6 Web Services wire From bf1a03c57142b9dbbf56ffc13cdd6db391435fd5 Mon Sep 17 00:00:00 2001 From: Amantux Date: Sat, 26 Sep 2026 06:35:38 +0000 Subject: [PATCH 32/54] =?UTF-8?q?frontend:=20Mailslot=20page=20=E2=80=94?= =?UTF-8?q?=20I/E=20listing,=20import,=20and=20the=20guarded=20export?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Lists every import/export element with occupancy and barcode (and says so when the backend has no station at all, which looks identical to "empty" but means the opposite). Import offers automatic or explicit target-slot placement and names the slot that was picked. Export surfaces the data-loss posture the CLI has: a preview names what would leave the library, the API's 409 refusal is rendered with the message that names the files, and `force` only ever travels after the operator types the barcode into ExportConfirmDialog. That gate is mutation-checked — removing the typed-barcode comparison fails "gates the forced export behind the typed barcode". Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_013S1qbQs31YSox1azQChqs2 --- frontend/src/api/mailslot.ts | 75 ++++ .../mailslot/ExportConfirmDialog.tsx | 133 +++++++ frontend/src/pages/Mailslot.test.tsx | 308 +++++++++++++++ frontend/src/pages/Mailslot.tsx | 372 ++++++++++++++++++ 4 files changed, 888 insertions(+) create mode 100644 frontend/src/api/mailslot.ts create mode 100644 frontend/src/components/mailslot/ExportConfirmDialog.tsx create mode 100644 frontend/src/pages/Mailslot.test.tsx create mode 100644 frontend/src/pages/Mailslot.tsx diff --git a/frontend/src/api/mailslot.ts b/frontend/src/api/mailslot.ts new file mode 100644 index 0000000..7e7faf0 --- /dev/null +++ b/frontend/src/api/mailslot.ts @@ -0,0 +1,75 @@ +import { rootApiRequest } from './client'; + +export interface MailslotSlot { + slotId: number; + occupied: boolean; + barcode: string | null; +} + +export interface MailslotListing { + importExportSlots: MailslotSlot[]; + slotCount: number; + occupiedCount: number; + /** False when this backend has no import/export station at all. */ + supported: boolean; +} + +/** What would leave the library with a cartridge. Returned by the preview call. */ +export interface ExportAssessment { + barcode: string; + volumeGroup: string | null; + archivedFilesOnCartridge: number; + pendingFilesOnCartridge: number; + bytesOnCartridge: number; + samplePaths: string[]; + volumeGroupBarcodesWithData: string[]; + carriesData: boolean; +} + +export interface MailslotMoveResult { + opId: string; + barcode: string; + source: string; + sourceSlot: number; + destination: string; + destinationSlot: number; + /** True when the backend picked the destination slot rather than the operator. */ + destinationSlotChosen: boolean; + exported?: ExportAssessment | null; +} + +export function listMailslotSlots(): Promise { + return rootApiRequest('/mailslot/slots'); +} + +export function previewMailslotExport(barcode: string): Promise { + return rootApiRequest(`/mailslot/export-preview/${encodeURIComponent(barcode)}`); +} + +export function importFromMailslot(ieSlot: number, toSlot?: number): Promise { + return rootApiRequest('/mailslot/import', { + method: 'POST', + body: { ie_slot: ieSlot, to_slot: toSlot ?? null }, + }); +} + +/** + * Export a cartridge out of the library. + * + * The API answers 409 with the refusal message when the cartridge still carries + * archived data; `force` is the only way past it and must never be sent without + * an explicit operator confirmation (see ExportConfirmDialog). + */ +export function exportThroughMailslot( + barcode: string, + options: { ieSlot?: number; force?: boolean } = {}, +): Promise { + return rootApiRequest('/mailslot/export', { + method: 'POST', + body: { + barcode, + ie_slot: options.ieSlot ?? null, + force: options.force ?? false, + }, + }); +} diff --git a/frontend/src/components/mailslot/ExportConfirmDialog.tsx b/frontend/src/components/mailslot/ExportConfirmDialog.tsx new file mode 100644 index 0000000..d1ea67d --- /dev/null +++ b/frontend/src/components/mailslot/ExportConfirmDialog.tsx @@ -0,0 +1,133 @@ +import { useEffect, useState } from 'react'; +import type { ExportAssessment } from '../../api/mailslot'; +import Button from '../ui/Button'; +import { formatBytes } from '../../lib/utils'; + +interface ExportConfirmDialogProps { + open: boolean; + barcode: string; + /** What the API says would leave with this cartridge. Null while unknown. */ + assessment: ExportAssessment | null; + isProcessing?: boolean; + onConfirm: () => void; + onCancel: () => void; +} + +/** + * The gate in front of a forced export. + * + * Exporting a cartridge that carries archived data makes every file on it + * unrestorable until someone physically puts it back, and the API refuses unless + * `force` is set. `force` therefore never travels on a single click: the operator + * has to type the barcode, which is the same posture as the CLI's `--force` — + * you cannot get there without naming the cartridge. + */ +export default function ExportConfirmDialog({ + open, + barcode, + assessment, + isProcessing = false, + onConfirm, + onCancel, +}: ExportConfirmDialogProps) { + const [typedBarcode, setTypedBarcode] = useState(''); + + useEffect(() => { + // A stale value from a previous cartridge must never satisfy this gate. + setTypedBarcode(''); + }, [barcode, open]); + + if (!open) { + return null; + } + + const barcodeMatches = typedBarcode.trim() === barcode; + + return ( +
+
+
Data loss
+

Force export of {barcode}?

+ +

+ Exporting moves this cartridge out of the library. Every file instance on it becomes + unrestorable until the cartridge is imported again. +

+ + {assessment ? ( +
+
+
Archived file instances
+
{assessment.archivedFilesOnCartridge}
+
+ {assessment.pendingFilesOnCartridge > 0 ? ( +
+
Writes still in flight
+
{assessment.pendingFilesOnCartridge}
+
+ ) : null} +
+
Bytes on cartridge
+
{formatBytes(assessment.bytesOnCartridge)}
+
+
+
Volume group
+
{assessment.volumeGroup ?? 'None'}
+
+ {assessment.samplePaths.length > 0 ? ( +
+
Files that leave with it
+
+
    + {assessment.samplePaths.map((path) => ( +
  • {path}
    • + ))} +
+ {assessment.archivedFilesOnCartridge > assessment.samplePaths.length ? ( +

+ +{assessment.archivedFilesOnCartridge - assessment.samplePaths.length} more +

+ ) : null} +
+
+ ) : null} + {assessment.volumeGroupBarcodesWithData.length > 0 ? ( +
+
Other cartridges in that group holding data
+
+ {assessment.volumeGroupBarcodesWithData.join(', ')} +
+
+ ) : null} +
+ ) : null} + + + setTypedBarcode(event.target.value)} + autoComplete="off" + placeholder="Barcode" + className="mt-2 w-full rounded-md border border-quantum-border bg-quantum-panel px-3 py-2 font-mono text-sm text-slate-100 outline-none focus:border-quantum-red" + /> + +
+ + +
+
+
+ ); +} diff --git a/frontend/src/pages/Mailslot.test.tsx b/frontend/src/pages/Mailslot.test.tsx new file mode 100644 index 0000000..688d834 --- /dev/null +++ b/frontend/src/pages/Mailslot.test.tsx @@ -0,0 +1,308 @@ +import { fireEvent, render, screen, waitFor } from '@testing-library/react'; +import { QueryClient, QueryClientProvider } from '@tanstack/react-query'; +import { beforeEach, describe, expect, it, vi } from 'vitest'; +import Mailslot from './Mailslot'; +import { ApiError } from '../api/client'; +import type { ExportAssessment, MailslotListing, MailslotMoveResult } from '../api/mailslot'; + +const mailslotModule = vi.hoisted(() => ({ + listMailslotSlots: vi.fn<() => Promise>(), + previewMailslotExport: vi.fn<(barcode: string) => Promise>(), + importFromMailslot: vi.fn<(ieSlot: number, toSlot?: number) => Promise>(), + exportThroughMailslot: vi.fn< + (barcode: string, options?: { ieSlot?: number; force?: boolean }) => Promise + >(), +})); + +vi.mock('../api/mailslot', () => mailslotModule); + +const OCCUPIED_LISTING: MailslotListing = { + importExportSlots: [ + { slotId: 51, occupied: true, barcode: 'PHO001L8' }, + { slotId: 52, occupied: false, barcode: null }, + ], + slotCount: 2, + occupiedCount: 1, + supported: true, +}; + +const CARRIES_DATA: ExportAssessment = { + barcode: 'ARC001L8', + volumeGroup: 'photo-archive', + archivedFilesOnCartridge: 3, + pendingFilesOnCartridge: 1, + bytesOnCartridge: 4096, + samplePaths: ['/photo-archive/a.raw', '/photo-archive/b.raw'], + volumeGroupBarcodesWithData: ['ARC002L8'], + carriesData: true, +}; + +const EMPTY_CARTRIDGE: ExportAssessment = { + barcode: 'SCR001L8', + volumeGroup: null, + archivedFilesOnCartridge: 0, + pendingFilesOnCartridge: 0, + bytesOnCartridge: 0, + samplePaths: [], + volumeGroupBarcodesWithData: [], + carriesData: false, +}; + +function moveResult(overrides: Partial = {}): MailslotMoveResult { + return { + opId: 'op-1', + barcode: 'PHO001L8', + source: 'import_export_slot', + sourceSlot: 51, + destination: 'storage_slot', + destinationSlot: 7, + destinationSlotChosen: true, + ...overrides, + }; +} + +function refusal(): ApiError { + return new ApiError( + 'Cartridge ARC001L8 still carries archived data: 3 file instance(s), 4096 bytes; ' + + 'volume group photo-archive; e.g. /photo-archive/a.raw. Exporting makes these ' + + 'unrestorable until the cartridge is imported again. Pass --force if that is what you mean.', + 409, + 'The backend could not complete POST /mailslot/export.', + 'Check the appliance state, then retry the request.', + ); +} + +function renderPage() { + const queryClient = new QueryClient({ + defaultOptions: { queries: { retry: false } }, + }); + return render( + + + , + ); +} + +describe('Mailslot', () => { + beforeEach(() => { + mailslotModule.listMailslotSlots.mockReset(); + mailslotModule.previewMailslotExport.mockReset(); + mailslotModule.importFromMailslot.mockReset(); + mailslotModule.exportThroughMailslot.mockReset(); + + mailslotModule.listMailslotSlots.mockResolvedValue(OCCUPIED_LISTING); + mailslotModule.previewMailslotExport.mockResolvedValue(CARRIES_DATA); + mailslotModule.importFromMailslot.mockResolvedValue(moveResult()); + mailslotModule.exportThroughMailslot.mockResolvedValue( + moveResult({ + barcode: 'SCR001L8', + source: 'storage_slot', + sourceSlot: 4, + destination: 'import_export_slot', + destinationSlot: 52, + }), + ); + }); + + it('lists import/export elements with their occupancy and barcodes', async () => { + renderPage(); + + await waitFor(() => { + expect(screen.getByText('I/E slot 51')).toBeTruthy(); + }); + expect(screen.getByText('PHO001L8')).toBeTruthy(); + expect(screen.getByText('I/E slot 52')).toBeTruthy(); + expect(screen.getByText('Occupied')).toBeTruthy(); + expect(screen.getByText('Empty')).toBeTruthy(); + expect(screen.getByText('1 of 2 occupied')).toBeTruthy(); + }); + + it('says so when the backend has no import/export station at all', async () => { + mailslotModule.listMailslotSlots.mockResolvedValue({ + importExportSlots: [], + slotCount: 0, + occupiedCount: 0, + supported: false, + }); + renderPage(); + + await waitFor(() => { + expect(screen.getByText(/no import\/export station/i)).toBeTruthy(); + }); + expect(screen.getByText(/No I\/E element holds a cartridge/i)).toBeTruthy(); + }); + + it('imports from a chosen I/E element, letting the backend pick the storage slot', async () => { + renderPage(); + await waitFor(() => { + expect(screen.getByLabelText('Source I/E element')).toBeTruthy(); + }); + + fireEvent.change(screen.getByLabelText('Source I/E element'), { target: { value: '51' } }); + fireEvent.click(screen.getByRole('button', { name: 'Import cartridge' })); + + await waitFor(() => { + expect(mailslotModule.importFromMailslot).toHaveBeenCalledWith(51, undefined); + }); + expect(screen.getByText(/Imported PHO001L8/)).toBeTruthy(); + expect(screen.getByText(/slot chosen automatically/)).toBeTruthy(); + }); + + it('imports into an explicit target slot when one is given', async () => { + renderPage(); + await waitFor(() => { + expect(screen.getByLabelText('Source I/E element')).toBeTruthy(); + }); + + fireEvent.change(screen.getByLabelText('Source I/E element'), { target: { value: '51' } }); + fireEvent.click(screen.getByLabelText('Target slot')); + fireEvent.change(screen.getByLabelText('Target storage slot'), { target: { value: '12' } }); + fireEvent.click(screen.getByRole('button', { name: 'Import cartridge' })); + + await waitFor(() => { + expect(mailslotModule.importFromMailslot).toHaveBeenCalledWith(51, 12); + }); + }); + + it('previews what would leave with a cartridge before anything moves', async () => { + renderPage(); + await waitFor(() => { + expect(screen.getByLabelText('Cartridge barcode')).toBeTruthy(); + }); + + fireEvent.change(screen.getByLabelText('Cartridge barcode'), { + target: { value: 'ARC001L8' }, + }); + fireEvent.click(screen.getByRole('button', { name: 'Check what would leave' })); + + await waitFor(() => { + expect(screen.getByText(/still carries archived data/)).toBeTruthy(); + }); + expect(screen.getByText('3 archived file instance(s)')).toBeTruthy(); + expect(screen.getByText('1 write(s) still in flight to it')).toBeTruthy(); + expect(screen.getByText('Volume group photo-archive')).toBeTruthy(); + expect(screen.getAllByText('/photo-archive/a.raw').length).toBeGreaterThan(0); + expect(mailslotModule.exportThroughMailslot).not.toHaveBeenCalled(); + }); + + it('exports a cartridge that carries nothing without asking for confirmation', async () => { + mailslotModule.previewMailslotExport.mockResolvedValue(EMPTY_CARTRIDGE); + renderPage(); + await waitFor(() => { + expect(screen.getByLabelText('Cartridge barcode')).toBeTruthy(); + }); + + fireEvent.change(screen.getByLabelText('Cartridge barcode'), { + target: { value: 'SCR001L8' }, + }); + fireEvent.click(screen.getByRole('button', { name: 'Export cartridge' })); + + await waitFor(() => { + expect(mailslotModule.exportThroughMailslot).toHaveBeenCalledWith('SCR001L8', { + force: false, + }); + }); + expect(screen.getByText(/Exported SCR001L8/)).toBeTruthy(); + }); + + it('surfaces the refusal naming what is on the cartridge, and never forces on its own', async () => { + mailslotModule.exportThroughMailslot.mockRejectedValue(refusal()); + renderPage(); + await waitFor(() => { + expect(screen.getByLabelText('Cartridge barcode')).toBeTruthy(); + }); + + fireEvent.change(screen.getByLabelText('Cartridge barcode'), { + target: { value: 'ARC001L8' }, + }); + fireEvent.click(screen.getByRole('button', { name: 'Export cartridge' })); + + await waitFor(() => { + expect(screen.getByText('Export refused')).toBeTruthy(); + }); + expect(screen.getByText(/still carries archived data: 3 file instance\(s\)/)).toBeTruthy(); + expect(mailslotModule.exportThroughMailslot).toHaveBeenCalledTimes(1); + expect(mailslotModule.exportThroughMailslot).toHaveBeenCalledWith('ARC001L8', { force: false }); + }); + + it('gates the forced export behind the typed barcode', async () => { + mailslotModule.exportThroughMailslot.mockRejectedValue(refusal()); + renderPage(); + await waitFor(() => { + expect(screen.getByLabelText('Cartridge barcode')).toBeTruthy(); + }); + + fireEvent.change(screen.getByLabelText('Cartridge barcode'), { + target: { value: 'ARC001L8' }, + }); + fireEvent.click(screen.getByRole('button', { name: 'Export cartridge' })); + await waitFor(() => { + expect(screen.getByRole('button', { name: 'Force export…' })).toBeTruthy(); + }); + fireEvent.click(screen.getByRole('button', { name: 'Force export…' })); + + const confirmButton = await waitFor(() => + screen.getByRole('button', { name: 'Export anyway' }), + ); + // Nothing typed: the gate is shut. + expect((confirmButton as HTMLButtonElement).disabled).toBe(true); + fireEvent.click(confirmButton); + expect(mailslotModule.exportThroughMailslot).toHaveBeenCalledTimes(1); + + // A near miss is still a miss. + const typedBarcode = screen.getByLabelText(/Type the barcode/); + fireEvent.change(typedBarcode, { target: { value: 'ARC001L' } }); + expect((screen.getByRole('button', { name: 'Export anyway' }) as HTMLButtonElement).disabled).toBe( + true, + ); + fireEvent.click(screen.getByRole('button', { name: 'Export anyway' })); + expect(mailslotModule.exportThroughMailslot).toHaveBeenCalledTimes(1); + + // The exact barcode, and only then, opens it. + fireEvent.change(typedBarcode, { target: { value: 'ARC001L8' } }); + expect((screen.getByRole('button', { name: 'Export anyway' }) as HTMLButtonElement).disabled).toBe( + false, + ); + fireEvent.click(screen.getByRole('button', { name: 'Export anyway' })); + + await waitFor(() => { + expect(mailslotModule.exportThroughMailslot).toHaveBeenCalledTimes(2); + }); + expect(mailslotModule.exportThroughMailslot).toHaveBeenLastCalledWith('ARC001L8', { + force: true, + }); + }); + + it('shows what leaves with the cartridge inside the force dialog', async () => { + mailslotModule.exportThroughMailslot.mockRejectedValue(refusal()); + renderPage(); + await waitFor(() => { + expect(screen.getByLabelText('Cartridge barcode')).toBeTruthy(); + }); + + fireEvent.change(screen.getByLabelText('Cartridge barcode'), { + target: { value: 'ARC001L8' }, + }); + fireEvent.click(screen.getByRole('button', { name: 'Export cartridge' })); + await waitFor(() => { + expect(screen.getByRole('button', { name: 'Force export…' })).toBeTruthy(); + }); + fireEvent.click(screen.getByRole('button', { name: 'Force export…' })); + + await waitFor(() => { + expect(screen.getByText('Force export of ARC001L8?')).toBeTruthy(); + }); + expect(screen.getByText('Files that leave with it')).toBeTruthy(); + expect(screen.getByText('Archived file instances')).toBeTruthy(); + expect(screen.getAllByText(/ARC002L8/).length).toBeGreaterThan(0); + }); + + it('renders a load failure with a retry instead of an empty station', async () => { + mailslotModule.listMailslotSlots.mockRejectedValue(new Error('backend down')); + renderPage(); + + await waitFor(() => { + expect(screen.getByRole('button', { name: /Retry/ })).toBeTruthy(); + }); + }); +}); diff --git a/frontend/src/pages/Mailslot.tsx b/frontend/src/pages/Mailslot.tsx new file mode 100644 index 0000000..bfbd3e7 --- /dev/null +++ b/frontend/src/pages/Mailslot.tsx @@ -0,0 +1,372 @@ +import { useState } from 'react'; +import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query'; +import { ApiError } from '../api/client'; +import { + exportThroughMailslot, + importFromMailslot, + listMailslotSlots, + previewMailslotExport, + type ExportAssessment, + type MailslotMoveResult, +} from '../api/mailslot'; +import ExportConfirmDialog from '../components/mailslot/ExportConfirmDialog'; +import Badge from '../components/ui/Badge'; +import Button from '../components/ui/Button'; +import Card from '../components/ui/Card'; +import ErrorMessage from '../components/ui/ErrorMessage'; +import Spinner from '../components/ui/Spinner'; +import { formatBytes } from '../lib/utils'; + +const SLOTS_QUERY_KEY = ['mailslot', 'slots']; + +function MoveResultPanel({ result }: { result: MailslotMoveResult }) { + return ( +
+
+ {result.source === 'import_export_slot' ? 'Imported' : 'Exported'} {result.barcode} +
+

+ {result.source === 'import_export_slot' + ? `I/E slot ${result.sourceSlot} → storage slot ${result.destinationSlot}` + : `Storage slot ${result.sourceSlot} → I/E slot ${result.destinationSlot}`} + {result.destinationSlotChosen ? ' (slot chosen automatically)' : ''} +

+

operation {result.opId}

+
+ ); +} + +function AssessmentPanel({ assessment }: { assessment: ExportAssessment }) { + if (!assessment.carriesData) { + return ( +
+
+ {assessment.barcode} carries no archived data +
+

+ Nothing becomes unrestorable by exporting it. It can leave the library as-is. +

+
+ ); + } + + return ( +
+
{assessment.barcode} still carries archived data
+
    +
  • {assessment.archivedFilesOnCartridge} archived file instance(s)
    • + {assessment.pendingFilesOnCartridge > 0 ? ( +
  • {assessment.pendingFilesOnCartridge} write(s) still in flight to it
    • + ) : null} +
  • {formatBytes(assessment.bytesOnCartridge)} on the cartridge
    • + {assessment.volumeGroup ?
  • Volume group {assessment.volumeGroup}
    • : null} +
+ {assessment.samplePaths.length > 0 ? ( +
    + {assessment.samplePaths.map((path) => ( +
  • {path}
    • + ))} +
+ ) : null} + {assessment.volumeGroupBarcodesWithData.length > 0 ? ( +

+ Other cartridges in that group hold data too:{' '} + {assessment.volumeGroupBarcodesWithData.join(', ')} +

+ ) : null} +

+ The API refuses this export. Exporting anyway needs an explicit confirmation. +

+
+ ); +} + +export default function Mailslot() { + const queryClient = useQueryClient(); + const [sourceIeSlot, setSourceIeSlot] = useState(); + const [targetSlotMode, setTargetSlotMode] = useState<'auto' | 'manual'>('auto'); + const [targetSlot, setTargetSlot] = useState(''); + const [exportBarcode, setExportBarcode] = useState(''); + const [confirmBarcode, setConfirmBarcode] = useState(); + + const slotsQuery = useQuery({ + queryKey: SLOTS_QUERY_KEY, + queryFn: listMailslotSlots, + refetchInterval: 10_000, + }); + + const previewMutation = useMutation({ + mutationFn: (barcode: string) => previewMailslotExport(barcode), + }); + + const importMutation = useMutation({ + mutationFn: () => + importFromMailslot( + sourceIeSlot!, + targetSlotMode === 'manual' && targetSlot.trim() ? Number(targetSlot) : undefined, + ), + onSuccess: async () => { + await queryClient.invalidateQueries({ queryKey: SLOTS_QUERY_KEY }); + }, + }); + + const exportMutation = useMutation({ + mutationFn: ({ barcode, force }: { barcode: string; force: boolean }) => + exportThroughMailslot(barcode, { force }), + onSuccess: async () => { + setConfirmBarcode(undefined); + await queryClient.invalidateQueries({ queryKey: SLOTS_QUERY_KEY }); + }, + }); + + if (slotsQuery.isLoading) { + return ; + } + if (slotsQuery.isError) { + return slotsQuery.refetch()} />; + } + + const listing = slotsQuery.data ?? { + importExportSlots: [], + slotCount: 0, + occupiedCount: 0, + supported: false, + }; + const occupiedSlots = listing.importExportSlots.filter((slot) => slot.occupied); + const assessment = previewMutation.data ?? null; + const exportError = exportMutation.error; + const exportRefused = exportError instanceof ApiError && exportError.status === 409; + + return ( +
+ +
+
+

Library

+

Mailslot

+

+ The import/export station: bring a cartridge in from an I/E element, or send one out. + Exports that would take archived data out of the library are refused until you confirm + them by barcode. +

+
+ +
+ + + +
+
+

Station

+

Import/export elements

+
+ + {listing.occupiedCount} of {listing.slotCount} occupied + +
+ + {!listing.supported ? ( +

+ This library backend has no import/export station, so there is nothing to import from + and nowhere to export to. +

+ ) : listing.slotCount === 0 ? ( +

+ This library reports no import/export elements. +

+ ) : ( +
+ {listing.importExportSlots.map((slot) => ( +
+
+
I/E slot {slot.slotId}
+ + {slot.occupied ? 'Occupied' : 'Empty'} + +
+
+ Barcode{' '} + + {slot.barcode ?? '—'} + +
+
+ ))} +
+ )} + + +
+ +

Import

+

Bring a cartridge in

+ + {occupiedSlots.length === 0 ? ( +

+ No I/E element holds a cartridge. Load one into the station first. +

+ ) : ( +
+
+ + +
+ +
+ Destination storage slot + + + {targetSlotMode === 'manual' ? ( + setTargetSlot(event.target.value.replace(/\D/g, ''))} + placeholder="Slot number" + className="w-full rounded-md border border-quantum-border bg-quantum-panel px-3 py-2 text-sm text-slate-100 outline-none focus:border-quantum-red" + /> + ) : null} +
+ + +
+ )} + + {importMutation.isError ? ( +
+ +
+ ) : null} + {importMutation.data ? : null} + + + +

Export

+

Send a cartridge out

+ +
+
+ + setExportBarcode(event.target.value)} + autoComplete="off" + placeholder="e.g. PHO001L8" + className="mt-2 w-full rounded-md border border-quantum-border bg-quantum-panel px-3 py-2 font-mono text-sm text-slate-100 outline-none focus:border-quantum-red" + /> +
+ +
+ + +
+
+ + {previewMutation.isError ? ( +
+ +
+ ) : null} + {assessment ? : null} + + {exportMutation.isError ? ( +
+ + {exportRefused ? ( + + ) : null} +
+ ) : null} + {exportMutation.data ? : null} + +
+ + exportMutation.mutate({ barcode: confirmBarcode!, force: true })} + onCancel={() => setConfirmBarcode(undefined)} + /> +
+ ); +} From 29eecdb79ab414dce95da66f6e4a92694ead8cfa Mon Sep 17 00:00:00 2001 From: Amantux Date: Sat, 26 Sep 2026 06:35:38 +0000 Subject: [PATCH 33/54] =?UTF-8?q?frontend:=20Drive=20Health=20page=20?= =?UTF-8?q?=E2=80=94=20inquiry=20plus=20TapeAlert=20severity=20badges?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Per-drive inquiry data and the flags that are SET, badged by severity, with the worst severity summarised on the drive. An unclassified flag is purple, never rendered as fine. The 503 from a simulator backend is rendered as setup instructions rather than an error, and a drive with no TapeAlert log page is shown as a property of the drive. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_013S1qbQs31YSox1azQChqs2 --- frontend/src/api/driveHealth.ts | 44 +++++++ frontend/src/pages/DriveHealth.test.tsx | 151 ++++++++++++++++++++++ frontend/src/pages/DriveHealth.tsx | 164 ++++++++++++++++++++++++ 3 files changed, 359 insertions(+) create mode 100644 frontend/src/api/driveHealth.ts create mode 100644 frontend/src/pages/DriveHealth.test.tsx create mode 100644 frontend/src/pages/DriveHealth.tsx diff --git a/frontend/src/api/driveHealth.ts b/frontend/src/api/driveHealth.ts new file mode 100644 index 0000000..722f73b --- /dev/null +++ b/frontend/src/api/driveHealth.ts @@ -0,0 +1,44 @@ +import { rootApiRequest } from './client'; + +export type TapeAlertSeverity = 'critical' | 'warning' | 'information' | 'unknown'; + +export interface DriveInquiry { + deviceType: string; + vendor: string; + product: string; + revision: string; + /** Empty when the drive reports no unit serial number. */ + serial: string; +} + +export interface TapeAlertFlag { + /** Null when sg_logs named a flag the backend's spec table does not classify. */ + number: number | null; + name: string; + severity: TapeAlertSeverity; +} + +export interface DriveHealth { + device: string; + inquiry: DriveInquiry; + tapeAlertSupported: boolean; + tapeAlertReason: string | null; + worstSeverity: TapeAlertSeverity | null; + flagsRead: number; + activeFlags: TapeAlertFlag[]; +} + +export interface DriveHealthListing { + drives: DriveHealth[]; +} + +/** + * Per-drive inquiry plus TapeAlert flags. + * + * Answers 503 unless the backend runs real hardware — the simulator has no SCSI + * drive to interrogate, and the detail names the two variables that enable it. + */ +export function getDriveHealth(device?: string): Promise { + const query = device ? `?device=${encodeURIComponent(device)}` : ''; + return rootApiRequest(`/hardware/drive-health${query}`); +} diff --git a/frontend/src/pages/DriveHealth.test.tsx b/frontend/src/pages/DriveHealth.test.tsx new file mode 100644 index 0000000..721652d --- /dev/null +++ b/frontend/src/pages/DriveHealth.test.tsx @@ -0,0 +1,151 @@ +import { fireEvent, render, screen, waitFor } from '@testing-library/react'; +import { QueryClient, QueryClientProvider } from '@tanstack/react-query'; +import { beforeEach, describe, expect, it, vi } from 'vitest'; +import DriveHealth from './DriveHealth'; +import { ApiError } from '../api/client'; +import type { DriveHealthListing } from '../api/driveHealth'; + +const driveHealthModule = vi.hoisted(() => ({ + getDriveHealth: vi.fn<(device?: string) => Promise>(), +})); + +vi.mock('../api/driveHealth', () => driveHealthModule); + +const LISTING: DriveHealthListing = { + drives: [ + { + device: '/dev/sg1', + inquiry: { + deviceType: 'tape', + vendor: 'IBM', + product: 'ULT3580-TD8', + revision: 'J4C1', + serial: '1068000073', + }, + tapeAlertSupported: true, + tapeAlertReason: null, + worstSeverity: 'critical', + flagsRead: 64, + activeFlags: [ + { number: 4, name: 'Media', severity: 'critical' }, + { number: 3, name: 'Hard Error', severity: 'warning' }, + { number: 56, name: 'Loading Failure', severity: 'unknown' }, + ], + }, + { + device: '/dev/sg2', + inquiry: { + deviceType: 'tape', + vendor: 'IBM', + product: 'ULT3580-TD8', + revision: 'J4C1', + serial: '', + }, + tapeAlertSupported: false, + tapeAlertReason: 'no TapeAlert page in sg_logs output', + worstSeverity: null, + flagsRead: 0, + activeFlags: [], + }, + ], +}; + +function renderPage() { + const queryClient = new QueryClient({ defaultOptions: { queries: { retry: false } } }); + return render( + + + , + ); +} + +describe('DriveHealth', () => { + beforeEach(() => { + driveHealthModule.getDriveHealth.mockReset(); + driveHealthModule.getDriveHealth.mockResolvedValue(LISTING); + }); + + it('renders inquiry data and active TapeAlert flags per drive', async () => { + renderPage(); + + await waitFor(() => { + expect(screen.getByText('/dev/sg1')).toBeTruthy(); + }); + expect(screen.getAllByText(/ULT3580-TD8 · rev J4C1/).length).toBe(2); + expect(screen.getByText(/Serial 1068000073/)).toBeTruthy(); + expect(screen.getByText('Media')).toBeTruthy(); + expect(screen.getByText('Hard Error')).toBeTruthy(); + expect(screen.getByText('TapeAlert critical')).toBeTruthy(); + // Severity badges, including the unclassified one — never rendered as fine. + expect(screen.getByText('critical')).toBeTruthy(); + expect(screen.getByText('warning')).toBeTruthy(); + expect(screen.getByText('unknown')).toBeTruthy(); + }); + + it('reports a drive without the TapeAlert log page as a fact, not a failure', async () => { + renderPage(); + + await waitFor(() => { + expect(screen.getByText('/dev/sg2')).toBeTruthy(); + }); + expect(screen.getByText(/does not implement the TapeAlert log page/)).toBeTruthy(); + expect(screen.getByText(/no TapeAlert page in sg_logs output/)).toBeTruthy(); + expect(screen.getByText('Serial (none reported) · type tape')).toBeTruthy(); + }); + + it('inspects one named device when asked', async () => { + renderPage(); + await waitFor(() => { + expect(driveHealthModule.getDriveHealth).toHaveBeenCalledWith(undefined); + }); + + fireEvent.change(screen.getByLabelText('Device (optional)'), { + target: { value: '/dev/nst0' }, + }); + fireEvent.click(screen.getByRole('button', { name: 'Inspect' })); + + await waitFor(() => { + expect(driveHealthModule.getDriveHealth).toHaveBeenCalledWith('/dev/nst0'); + }); + }); + + it('renders the 503 as setup instructions rather than an error', async () => { + driveHealthModule.getDriveHealth.mockRejectedValue( + new ApiError( + 'Real hardware operations require OPENBLADE_BACKEND=real and ' + + 'OPENBLADE_REAL_HARDWARE_ENABLED=true', + 503, + 'impact', + 'action', + ), + ); + renderPage(); + + await waitFor(() => { + expect(screen.getByText('Real hardware is not enabled')).toBeTruthy(); + }); + expect(screen.getAllByText(/OPENBLADE_REAL_HARDWARE_ENABLED=true/).length).toBeGreaterThan(0); + expect(screen.queryByRole('button', { name: /Retry/ })).toBeNull(); + }); + + it('renders any other failure as a retryable error', async () => { + driveHealthModule.getDriveHealth.mockRejectedValue( + new ApiError('sg_logs is not installed', 500, 'impact', 'action'), + ); + renderPage(); + + await waitFor(() => { + expect(screen.getByRole('button', { name: /Retry/ })).toBeTruthy(); + }); + expect(screen.getAllByText(/sg_logs is not installed/).length).toBeGreaterThan(0); + }); + + it('says so when no drive was discovered', async () => { + driveHealthModule.getDriveHealth.mockResolvedValue({ drives: [] }); + renderPage(); + + await waitFor(() => { + expect(screen.getByText('No tape drives discovered')).toBeTruthy(); + }); + }); +}); diff --git a/frontend/src/pages/DriveHealth.tsx b/frontend/src/pages/DriveHealth.tsx new file mode 100644 index 0000000..6d72833 --- /dev/null +++ b/frontend/src/pages/DriveHealth.tsx @@ -0,0 +1,164 @@ +import { useState } from 'react'; +import { useQuery } from '@tanstack/react-query'; +import { ApiError } from '../api/client'; +import { getDriveHealth, type DriveHealth as DriveHealthRecord, type TapeAlertSeverity } from '../api/driveHealth'; +import Badge from '../components/ui/Badge'; +import Button from '../components/ui/Button'; +import Card from '../components/ui/Card'; +import ErrorMessage from '../components/ui/ErrorMessage'; +import Spinner from '../components/ui/Spinner'; +import type { BadgeVariant } from '../lib/utils'; + +const SEVERITY_VARIANT: Record = { + critical: 'red', + warning: 'amber', + information: 'blue', + // Set on the wire but unclassified by the backend's spec table. Never rendered + // as "fine": an operator still needs to see that the drive raised it. + unknown: 'purple', +}; + +function severityVariant(severity: TapeAlertSeverity | null): BadgeVariant { + return severity ? SEVERITY_VARIANT[severity] : 'green'; +} + +function DriveCard({ drive }: { drive: DriveHealthRecord }) { + return ( + +
+
+
{drive.device}
+
+ {drive.inquiry.vendor} {drive.inquiry.product} · rev {drive.inquiry.revision} +
+
+ Serial {drive.inquiry.serial || '(none reported)'} · type {drive.inquiry.deviceType} +
+
+ + {drive.worstSeverity ? `TapeAlert ${drive.worstSeverity}` : 'No flags set'} + +
+ + {!drive.tapeAlertSupported ? ( +

+ This drive does not implement the TapeAlert log page + {drive.tapeAlertReason ? `: ${drive.tapeAlertReason}` : '.'} That is a property of the + drive, not a failure. +

+ ) : drive.activeFlags.length === 0 ? ( +

+ No TapeAlert flags set ({drive.flagsRead} flags read). +

+ ) : ( + + + + + + + + + + {drive.activeFlags.map((flag) => ( + + + + + + ))} + +
FlagNameSeverity
{flag.number ?? '?'}{flag.name} + {flag.severity} +
+ )} + + ); +} + +export default function DriveHealth() { + const [deviceInput, setDeviceInput] = useState(''); + const [device, setDevice] = useState(''); + + const healthQuery = useQuery({ + queryKey: ['drive-health', device], + queryFn: () => getDriveHealth(device || undefined), + retry: false, + }); + + const error = healthQuery.error; + // 503 is the documented "this deployment has no real drives" answer, not a + // fault: the guard refuses unless both hardware variables are set. + const hardwareDisabled = error instanceof ApiError && error.status === 503; + const drives = healthQuery.data?.drives ?? []; + + return ( +
+ +
+
+

Drives

+

Drive Health

+

+ SCSI inquiry data and TapeAlert flags, read straight from the drives. Read-only: this + page issues an INQUIRY and one LOG SENSE per drive and changes nothing. +

+
+ +
+ + + +
+
+ + setDeviceInput(event.target.value)} + placeholder="/dev/nst0 — leave blank for every discovered drive" + autoComplete="off" + className="mt-2 w-full rounded-md border border-quantum-border bg-quantum-panel px-3 py-2 font-mono text-sm text-slate-100 outline-none focus:border-quantum-red" + /> +
+ +
+ + + {healthQuery.isLoading ? : null} + + {hardwareDisabled ? ( + +

Real hardware is not enabled

+

+ Drive health reads a physical drive, so there is nothing to report on a simulator + backend. Enable it deliberately: +

+ 
+            OPENBLADE_BACKEND=real{'\n'}OPENBLADE_REAL_HARDWARE_ENABLED=true
+
+

Reported by the backend: {error.message}

+ + ) : healthQuery.isError ? ( + healthQuery.refetch()} /> + ) : null} + + {!healthQuery.isLoading && !healthQuery.isError && drives.length === 0 ? ( + +

No tape drives discovered

+

+ The backend found no drive to interrogate. Name a device above to inspect one directly. +

+ + ) : null} + + {drives.map((drive) => ( + + ))} +
+ ); +} From cc30de7d732e48317d7946d22f226d245bbd7909 Mon Sep 17 00:00:00 2001 From: Amantux Date: Sat, 26 Sep 2026 06:35:50 +0000 Subject: [PATCH 34/54] =?UTF-8?q?frontend:=20Tree=20Restore=20page=20?= =?UTF-8?q?=E2=80=94=20prefix,=20destination,=20dry=20run,=20then=20run?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Prefixes are folded out of the same /catalog/ listing the Catalog Records page browses, offered as chips, and still typeable when that listing fails. "Run restore" stays disabled until a dry run has been done for the prefix and destination currently in the form, so nobody moves media without seeing the file and tape count first. The result panel names skipped files (catalogued but never archived) and per-file failures — a tree that comes back short must not read as a clean run. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_013S1qbQs31YSox1azQChqs2 --- frontend/src/api/treeRestore.ts | 62 ++++++ frontend/src/pages/TreeRestore.test.tsx | 194 ++++++++++++++++++ frontend/src/pages/TreeRestore.tsx | 248 ++++++++++++++++++++++++ 3 files changed, 504 insertions(+) create mode 100644 frontend/src/api/treeRestore.ts create mode 100644 frontend/src/pages/TreeRestore.test.tsx create mode 100644 frontend/src/pages/TreeRestore.tsx diff --git a/frontend/src/api/treeRestore.ts b/frontend/src/api/treeRestore.ts new file mode 100644 index 0000000..ccf89b7 --- /dev/null +++ b/frontend/src/api/treeRestore.ts @@ -0,0 +1,62 @@ +import { rootApiRequest } from './client'; +import type { CatalogListResponse } from '../types/api'; + +export interface TreeRestoreFailure { + catalogPath: string; + error: string; +} + +export interface TreeRestoreResult { + jobId: string; + catalogPrefix: string; + destDir: string; + dryRun: boolean; + filesRestored: number; + filesFailed: number; + bytesRestored: number; + filesVerified: number; + perTapeCounts: Record; + tapesUsed: string[]; + failures: TreeRestoreFailure[]; + /** Catalogued files with nothing archived: skipped, not failed, but named. */ + filesSkipped: number; + skippedPaths: string[]; + status: string; +} + +export interface TreeRestoreRequest { + catalogPrefix: string; + destDir: string; + dryRun: boolean; +} + +export function runTreeRestore(request: TreeRestoreRequest): Promise { + return rootApiRequest('/restore/tree', { + method: 'POST', + body: { + catalog_prefix: request.catalogPrefix, + dest_dir: request.destDir, + dry_run: request.dryRun, + }, + }); +} + +/** + * Catalog prefixes offered as starting points, derived from the same + * `/catalog/` listing the Catalog Records page browses. The API has no + * prefix/tree endpoint, so the directory levels are folded out of the file + * paths here rather than invented server-side. + */ +export async function listCatalogPrefixes(limit = 500): Promise { + const params = new URLSearchParams({ limit: String(limit), offset: '0' }); + const response = await rootApiRequest(`/catalog/?${params.toString()}`); + const prefixes = new Set(); + for (const file of response.files) { + const segments = file.source_path.split('/').filter(Boolean); + // Every ancestor directory is a valid prefix; the file itself is not one. + for (let depth = 1; depth < segments.length; depth += 1) { + prefixes.add(`/${segments.slice(0, depth).join('/')}`); + } + } + return [...prefixes].sort((left, right) => left.localeCompare(right)); +} diff --git a/frontend/src/pages/TreeRestore.test.tsx b/frontend/src/pages/TreeRestore.test.tsx new file mode 100644 index 0000000..82ab78d --- /dev/null +++ b/frontend/src/pages/TreeRestore.test.tsx @@ -0,0 +1,194 @@ +import { fireEvent, render, screen, waitFor } from '@testing-library/react'; +import { QueryClient, QueryClientProvider } from '@tanstack/react-query'; +import { beforeEach, describe, expect, it, vi } from 'vitest'; +import TreeRestore from './TreeRestore'; +import { ApiError } from '../api/client'; +import type { TreeRestoreResult } from '../api/treeRestore'; + +const treeRestoreModule = vi.hoisted(() => ({ + runTreeRestore: vi.fn< + (request: { catalogPrefix: string; destDir: string; dryRun: boolean }) => Promise + >(), + listCatalogPrefixes: vi.fn<() => Promise>(), +})); + +vi.mock('../api/treeRestore', () => treeRestoreModule); + +function result(overrides: Partial = {}): TreeRestoreResult { + return { + jobId: 'job-1', + catalogPrefix: '/photo-archive', + destDir: '/restore/photos', + dryRun: true, + filesRestored: 4, + filesFailed: 0, + bytesRestored: 2048, + filesVerified: 4, + perTapeCounts: { PHO001L8: 3, PHO002L8: 1 }, + tapesUsed: ['PHO001L8', 'PHO002L8'], + failures: [], + filesSkipped: 0, + skippedPaths: [], + status: 'completed', + ...overrides, + }; +} + +function renderPage() { + const queryClient = new QueryClient({ defaultOptions: { queries: { retry: false } } }); + return render( + + + , + ); +} + +async function fillForm() { + await waitFor(() => { + expect(screen.getByLabelText('Catalog prefix')).toBeTruthy(); + }); + fireEvent.change(screen.getByLabelText('Catalog prefix'), { + target: { value: '/photo-archive' }, + }); + fireEvent.change(screen.getByLabelText('Destination directory'), { + target: { value: '/restore/photos' }, + }); +} + +describe('TreeRestore', () => { + beforeEach(() => { + treeRestoreModule.runTreeRestore.mockReset(); + treeRestoreModule.listCatalogPrefixes.mockReset(); + treeRestoreModule.listCatalogPrefixes.mockResolvedValue(['/photo-archive', '/photo-archive/2020']); + treeRestoreModule.runTreeRestore.mockResolvedValue(result()); + }); + + it('offers catalog prefixes from the catalog listing', async () => { + renderPage(); + + await waitFor(() => { + expect(screen.getByRole('button', { name: '/photo-archive/2020' })).toBeTruthy(); + }); + fireEvent.click(screen.getByRole('button', { name: '/photo-archive/2020' })); + expect((screen.getByLabelText('Catalog prefix') as HTMLInputElement).value).toBe( + '/photo-archive/2020', + ); + }); + + it('requires a dry-run before the restore can be run', async () => { + renderPage(); + await fillForm(); + + expect((screen.getByRole('button', { name: 'Run restore' }) as HTMLButtonElement).disabled).toBe( + true, + ); + expect(screen.getByText(/Run the dry-run preview first/)).toBeTruthy(); + + fireEvent.click(screen.getByRole('button', { name: 'Dry-run preview' })); + + await waitFor(() => { + expect(treeRestoreModule.runTreeRestore).toHaveBeenCalledWith({ + catalogPrefix: '/photo-archive', + destDir: '/restore/photos', + dryRun: true, + }); + }); + await waitFor(() => { + expect( + (screen.getByRole('button', { name: 'Run restore' }) as HTMLButtonElement).disabled, + ).toBe(false); + }); + }); + + it('shows the dry-run plan with per-tape counts', async () => { + renderPage(); + await fillForm(); + fireEvent.click(screen.getByRole('button', { name: 'Dry-run preview' })); + + await waitFor(() => { + expect(screen.getByText('Dry-run preview')).toBeTruthy(); + }); + expect(screen.getByText('Files planned')).toBeTruthy(); + expect(screen.getByText('Planned')).toBeTruthy(); + expect(screen.getByText('/photo-archive → /restore/photos')).toBeTruthy(); + expect(screen.getByText(/PHO001L8/)).toBeTruthy(); + expect(screen.getByText(/3 file\(s\)/)).toBeTruthy(); + }); + + it('runs the restore after the plan and reports the outcome', async () => { + renderPage(); + await fillForm(); + fireEvent.click(screen.getByRole('button', { name: 'Dry-run preview' })); + await waitFor(() => { + expect( + (screen.getByRole('button', { name: 'Run restore' }) as HTMLButtonElement).disabled, + ).toBe(false); + }); + + treeRestoreModule.runTreeRestore.mockResolvedValue( + result({ dryRun: false, filesRestored: 4, status: 'completed' }), + ); + fireEvent.click(screen.getByRole('button', { name: 'Run restore' })); + + await waitFor(() => { + expect(treeRestoreModule.runTreeRestore).toHaveBeenLastCalledWith({ + catalogPrefix: '/photo-archive', + destDir: '/restore/photos', + dryRun: false, + }); + }); + await waitFor(() => { + expect(screen.getByText('Files restored')).toBeTruthy(); + }); + expect(screen.getByText('completed')).toBeTruthy(); + }); + + it('names skipped and failed files rather than reporting a clean run', async () => { + treeRestoreModule.runTreeRestore.mockResolvedValue( + result({ + dryRun: false, + filesRestored: 1, + filesFailed: 1, + failures: [{ catalogPath: '/photo-archive/b.raw', error: 'ChecksumMismatchError' }], + filesSkipped: 1, + skippedPaths: ['/photo-archive/never-archived.raw'], + status: 'failed', + }), + ); + renderPage(); + await fillForm(); + fireEvent.click(screen.getByRole('button', { name: 'Dry-run preview' })); + + await waitFor(() => { + expect(screen.getByText(/1 catalogued file\(s\) had nothing archived/)).toBeTruthy(); + }); + expect(screen.getByText('/photo-archive/never-archived.raw')).toBeTruthy(); + expect(screen.getByText('1 file(s) failed')).toBeTruthy(); + expect(screen.getByText(/ChecksumMismatchError/)).toBeTruthy(); + }); + + it('renders a failed dry-run as an error and leaves the run gated', async () => { + treeRestoreModule.runTreeRestore.mockRejectedValue( + new ApiError('Job failed (CommandError); see server logs for detail', 500, 'impact', 'action'), + ); + renderPage(); + await fillForm(); + fireEvent.click(screen.getByRole('button', { name: 'Dry-run preview' })); + + await waitFor(() => { + expect(screen.getAllByText(/Job failed \(CommandError\)/).length).toBeGreaterThan(0); + }); + expect((screen.getByRole('button', { name: 'Run restore' }) as HTMLButtonElement).disabled).toBe( + true, + ); + }); + + it('still lets a prefix be typed when the catalog listing fails', async () => { + treeRestoreModule.listCatalogPrefixes.mockRejectedValue(new Error('catalog unavailable')); + renderPage(); + + await waitFor(() => { + expect(screen.getByText(/could not be loaded; type a prefix instead/)).toBeTruthy(); + }); + }); +}); diff --git a/frontend/src/pages/TreeRestore.tsx b/frontend/src/pages/TreeRestore.tsx new file mode 100644 index 0000000..1926556 --- /dev/null +++ b/frontend/src/pages/TreeRestore.tsx @@ -0,0 +1,248 @@ +import { useState } from 'react'; +import { useMutation, useQuery } from '@tanstack/react-query'; +import { + listCatalogPrefixes, + runTreeRestore, + type TreeRestoreResult, +} from '../api/treeRestore'; +import Badge from '../components/ui/Badge'; +import Button from '../components/ui/Button'; +import Card from '../components/ui/Card'; +import ErrorMessage from '../components/ui/ErrorMessage'; +import Spinner from '../components/ui/Spinner'; +import { formatBytes } from '../lib/utils'; + +function ResultPanel({ result }: { result: TreeRestoreResult }) { + const failed = result.failures.length > 0; + return ( + +
+
+

+ {result.dryRun ? 'Dry-run preview' : 'Tree restore'} +

+

+ {result.catalogPrefix} → {result.destDir} +

+

job {result.jobId}

+
+ + {result.dryRun ? 'Planned' : result.status} + +
+ +
+
+
+ {result.dryRun ? 'Files planned' : 'Files restored'} +
+
{result.filesRestored}
+
+
+
Bytes
+
+ {formatBytes(result.bytesRestored)} +
+
+
+
Failed
+
{result.filesFailed}
+
+
+
Tapes
+
{result.tapesUsed.length}
+
+
+ + {result.tapesUsed.length > 0 ? ( +
+
Files per tape
+
    + {result.tapesUsed.map((barcode) => ( +
  • + {barcode}{' '} + {result.perTapeCounts[barcode] ?? 0} file(s) +
    • + ))} +
+
+ ) : null} + + {result.filesSkipped > 0 ? ( +
+
+ {result.filesSkipped} catalogued file(s) had nothing archived and were skipped +
+
    + {result.skippedPaths.slice(0, 10).map((path) => ( +
  • {path}
    • + ))} +
+ {result.skippedPaths.length > 10 ? ( +

+ +{result.skippedPaths.length - 10} more +

+ ) : null} +
+ ) : null} + + {failed ? ( +
+
{result.failures.length} file(s) failed
+
    + {result.failures.map((failure) => ( +
  • + {failure.catalogPath} — {failure.error} +
    • + ))} +
+
+ ) : null} + + ); +} + +export default function TreeRestore() { + const [catalogPrefix, setCatalogPrefix] = useState(''); + const [destDir, setDestDir] = useState(''); + const [plan, setPlan] = useState(); + + const prefixesQuery = useQuery({ + queryKey: ['catalog', 'prefixes'], + queryFn: () => listCatalogPrefixes(), + }); + + const dryRunMutation = useMutation({ + mutationFn: () => + runTreeRestore({ catalogPrefix: catalogPrefix.trim(), destDir: destDir.trim(), dryRun: true }), + onSuccess: (result) => setPlan(result), + }); + + const runMutation = useMutation({ + mutationFn: () => + runTreeRestore({ + catalogPrefix: catalogPrefix.trim(), + destDir: destDir.trim(), + dryRun: false, + }), + }); + + const ready = Boolean(catalogPrefix.trim() && destDir.trim()); + // The plan is only an answer about the prefix and destination it was run for. + const planMatchesForm = + plan !== undefined && + plan.destDir === destDir.trim() && + // The API normalises the prefix ("photos" -> "/photos"), so compare loosely. + plan.catalogPrefix.replace(/^\/+|\/+$/g, '') === catalogPrefix.trim().replace(/^\/+|\/+$/g, ''); + + return ( +
+ +
+

Restore

+

Tree Restore

+

+ Restore every archived file under a catalog prefix, spanning as many tapes as it takes. + The source tree is preserved under the destination: restoring /vg writes{' '} + /vg/a/x.txt to <destination>/a/x.txt. +

+
+ + + +

Selection

+

What to restore

+ +
+
+ + setCatalogPrefix(event.target.value)} + placeholder="/photo-archive" + autoComplete="off" + className="mt-2 w-full rounded-md border border-quantum-border bg-quantum-panel px-3 py-2 font-mono text-sm text-slate-100 outline-none focus:border-quantum-red" + /> + {prefixesQuery.isLoading ? ( +

Loading catalog prefixes…

+ ) : prefixesQuery.isError ? ( +

+ Catalog prefixes could not be loaded; type a prefix instead. +

+ ) : (prefixesQuery.data ?? []).length === 0 ? ( +

+ The catalog lists no archived files yet, so there is no prefix to browse. +

+ ) : ( +
+ {(prefixesQuery.data ?? []).map((prefix) => ( + + ))} +
+ )} +
+ +
+ + setDestDir(event.target.value)} + placeholder="/restore/photo-archive" + autoComplete="off" + className="mt-2 w-full rounded-md border border-quantum-border bg-quantum-panel px-3 py-2 font-mono text-sm text-slate-100 outline-none focus:border-quantum-red" + /> +
+ +
+ + +
+ {!planMatchesForm ? ( +

+ Run the dry-run preview first — it is the only place the file and tape count is shown + before media moves. +

+ ) : null} +
+ + + {dryRunMutation.isPending || runMutation.isPending ? : null} + {dryRunMutation.isError ? : null} + {runMutation.isError ? : null} + + {runMutation.data ? ( + + ) : plan ? ( + + ) : null} +
+ ); +} From 811a091464d8a0cabf8fcf07a8e0d7ae21a88bdc Mon Sep 17 00:00:00 2001 From: Amantux Date: Sat, 26 Sep 2026 06:35:50 +0000 Subject: [PATCH 35/54] frontend: Assistant page over POST /assist (read-only, no streaming) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Message thread with the whole conversation sent on every turn — the endpoint is stateless and request/response, so there is no stream to render. Tool calls are shown dim and by name only, matching what the API returns. The empty state carries domain suggestion chips, since the assistant's capabilities are otherwise invisible. A 503 renders the documented setup instructions rather than an error, pops the orphaned user turn and hands the question back to the composer: a thread must never show a question that nothing answered. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_013S1qbQs31YSox1azQChqs2 --- frontend/src/api/assist.ts | 32 +++++ frontend/src/pages/Assistant.test.tsx | 131 +++++++++++++++++++ frontend/src/pages/Assistant.tsx | 179 ++++++++++++++++++++++++++ 3 files changed, 342 insertions(+) create mode 100644 frontend/src/api/assist.ts create mode 100644 frontend/src/pages/Assistant.test.tsx create mode 100644 frontend/src/pages/Assistant.tsx diff --git a/frontend/src/api/assist.ts b/frontend/src/api/assist.ts new file mode 100644 index 0000000..ec6f327 --- /dev/null +++ b/frontend/src/api/assist.ts @@ -0,0 +1,32 @@ +import { rootApiRequest } from './client'; + +export type AssistRole = 'user' | 'assistant'; + +export interface AssistMessage { + role: AssistRole; + content: string; +} + +export interface AssistReply { + reply: string; + /** Tool names only — a trace of what the assistant consulted, never arguments. */ + toolCalls: string[]; +} + +/** Conversation caps enforced by the API (docs/wiki/guides/assist-api.md). */ +export const ASSIST_MAX_MESSAGE_CHARS = 8000; + +/** + * Ask one question. The whole conversation travels with every request: the + * endpoint is stateless and request/response — there is no streaming and no + * session id to hold on to. + * + * The HTTP surface is read-only by construction; it cannot create, format or + * erase anything, so nothing here needs a confirmation flow. + */ +export function askAssistant(messages: AssistMessage[]): Promise { + return rootApiRequest('/assist', { + method: 'POST', + body: { messages }, + }); +} diff --git a/frontend/src/pages/Assistant.test.tsx b/frontend/src/pages/Assistant.test.tsx new file mode 100644 index 0000000..f36ffa3 --- /dev/null +++ b/frontend/src/pages/Assistant.test.tsx @@ -0,0 +1,131 @@ +import { fireEvent, render, screen, waitFor } from '@testing-library/react'; +import { QueryClient, QueryClientProvider } from '@tanstack/react-query'; +import { beforeEach, describe, expect, it, vi } from 'vitest'; +import Assistant from './Assistant'; +import { ApiError } from '../api/client'; +import type { AssistMessage, AssistReply } from '../api/assist'; + +const assistModule = vi.hoisted(() => ({ + askAssistant: vi.fn<(messages: AssistMessage[]) => Promise>(), + ASSIST_MAX_MESSAGE_CHARS: 8000, +})); + +vi.mock('../api/assist', () => assistModule); + +const NOT_CONFIGURED_DETAIL = + 'The OpenBlade assistant is not configured. Set OPENBLADE_OLLAMA_URL to an Ollama endpoint.'; + +function renderPage() { + const queryClient = new QueryClient({ defaultOptions: { queries: { retry: false } } }); + return render( + + + , + ); +} + +describe('Assistant', () => { + beforeEach(() => { + assistModule.askAssistant.mockReset(); + assistModule.askAssistant.mockResolvedValue({ + reply: 'Two: PH000001 and PH000002.', + toolCalls: ['get_inventory', 'list_volume_groups'], + }); + }); + + it('shows domain suggestions before anything has been asked', async () => { + renderPage(); + + expect(screen.getByText('Ask about the library')).toBeTruthy(); + expect(screen.getByRole('button', { name: /Why did my last restore fail\?/ })).toBeTruthy(); + }); + + it('sends the question and renders the reply with a dim tool trace', async () => { + renderPage(); + + fireEvent.change(screen.getByLabelText('Question'), { + target: { value: 'how many tapes are in photo-archive?' }, + }); + fireEvent.click(screen.getByRole('button', { name: 'Ask' })); + + await waitFor(() => { + expect(screen.getByText('Two: PH000001 and PH000002.')).toBeTruthy(); + }); + expect(assistModule.askAssistant).toHaveBeenCalledWith([ + { role: 'user', content: 'how many tapes are in photo-archive?' }, + ]); + expect(screen.getByText('consulted get_inventory, list_volume_groups')).toBeTruthy(); + expect(screen.getByText('how many tapes are in photo-archive?')).toBeTruthy(); + }); + + it('sends the whole conversation on the next turn — the endpoint is stateless', async () => { + renderPage(); + + fireEvent.change(screen.getByLabelText('Question'), { target: { value: 'first question' } }); + fireEvent.click(screen.getByRole('button', { name: 'Ask' })); + await waitFor(() => { + expect(screen.getByText('Two: PH000001 and PH000002.')).toBeTruthy(); + }); + + fireEvent.change(screen.getByLabelText('Question'), { target: { value: 'second question' } }); + fireEvent.click(screen.getByRole('button', { name: 'Ask' })); + + await waitFor(() => { + expect(assistModule.askAssistant).toHaveBeenLastCalledWith([ + { role: 'user', content: 'first question' }, + { role: 'assistant', content: 'Two: PH000001 and PH000002.' }, + { role: 'user', content: 'second question' }, + ]); + }); + }); + + it('asks a suggestion straight from the empty state', async () => { + renderPage(); + + fireEvent.click(screen.getByRole('button', { name: /Why did my last restore fail\?/ })); + + await waitFor(() => { + expect(assistModule.askAssistant).toHaveBeenCalledWith([ + { role: 'user', content: 'Why did my last restore fail?' }, + ]); + }); + }); + + it('renders a 503 as setup instructions and hands the question back', async () => { + assistModule.askAssistant.mockRejectedValue( + new ApiError(NOT_CONFIGURED_DETAIL, 503, 'impact', 'action'), + ); + renderPage(); + + fireEvent.change(screen.getByLabelText('Question'), { target: { value: 'anything' } }); + fireEvent.click(screen.getByRole('button', { name: 'Ask' })); + + await waitFor(() => { + expect(screen.getByText('The assistant is not configured')).toBeTruthy(); + }); + expect(screen.getByText(/OPENBLADE_OLLAMA_URL=http:\/\/localhost:11434/)).toBeTruthy(); + // The orphaned user turn is popped and the text returned to the composer, so + // the thread never shows a question that nothing answered. + expect(screen.queryByText('anything')).toBeNull(); + expect((screen.getByLabelText('Question') as HTMLInputElement).value).toBe('anything'); + expect(screen.getByText('Ask about the library')).toBeTruthy(); + }); + + it('renders an upstream failure as an error, keeping the thread', async () => { + assistModule.askAssistant.mockRejectedValue( + new ApiError('The model endpoint could not be reached', 502, 'impact', 'action'), + ); + renderPage(); + + fireEvent.change(screen.getByLabelText('Question'), { target: { value: 'why is it slow?' } }); + fireEvent.click(screen.getByRole('button', { name: 'Ask' })); + + await waitFor(() => { + expect(screen.getAllByText(/The model endpoint could not be reached/).length).toBeGreaterThan( + 0, + ); + }); + expect(screen.queryByText('The assistant is not configured')).toBeNull(); + expect(screen.getByText('why is it slow?')).toBeTruthy(); + }); +}); diff --git a/frontend/src/pages/Assistant.tsx b/frontend/src/pages/Assistant.tsx new file mode 100644 index 0000000..7559a5d --- /dev/null +++ b/frontend/src/pages/Assistant.tsx @@ -0,0 +1,179 @@ +import { useState, type FormEvent } from 'react'; +import { useMutation } from '@tanstack/react-query'; +import { askAssistant, ASSIST_MAX_MESSAGE_CHARS, type AssistMessage } from '../api/assist'; +import { ApiError } from '../api/client'; +import Button from '../components/ui/Button'; +import Card from '../components/ui/Card'; +import ErrorMessage from '../components/ui/ErrorMessage'; +import Spinner from '../components/ui/Spinner'; + +interface Turn extends AssistMessage { + /** Tool names the assistant consulted for this reply. Names only, never args. */ + toolCalls?: string[]; +} + +const SUGGESTIONS = [ + 'How many tapes are in the library, and which are scratch?', + 'Why did my last restore fail?', + 'Which cartridges hold data for the photo-archive volume group?', + 'Show me the drives and what is loaded in them.', +]; + +function ToolTrace({ toolCalls }: { toolCalls: string[] }) { + return ( +

+ consulted {toolCalls.join(', ')} +

+ ); +} + +function SetupInstructions({ detail }: { detail: string }) { + return ( + +

The assistant is not configured

+

+ The API answered 503: no model endpoint is set, so there is nothing to ask. Point OpenBlade + at an Ollama endpoint and restart it: +

+ 
+        {'# local\nexport OPENBLADE_OLLAMA_URL=http://localhost:11434\n\n'}
+        {'# cloud\nexport OPENBLADE_OLLAMA_URL=https://ollama.com\nexport OPENBLADE_OLLAMA_API_KEY='}
+
+

Reported by the backend: {detail}

+ + ); +} + +export default function Assistant() { + const [turns, setTurns] = useState([]); + const [draft, setDraft] = useState(''); + + const askMutation = useMutation({ + mutationFn: (messages: AssistMessage[]) => + askAssistant(messages.map(({ role, content }) => ({ role, content }))), + onSuccess: (reply) => { + setTurns((current) => [ + ...current, + { role: 'assistant', content: reply.reply, toolCalls: reply.toolCalls }, + ]); + }, + onError: (error, variables) => { + if (error instanceof ApiError && error.status === 503) { + // Nothing answered the question, so the thread must not keep a turn that + // looks answered: pop it and hand the text back to the composer. + setTurns((current) => current.slice(0, -1)); + setDraft(variables[variables.length - 1]?.content ?? ''); + } + }, + }); + + function send(question: string) { + const trimmed = question.trim(); + if (!trimmed || askMutation.isPending) { + return; + } + const next: Turn[] = [...turns, { role: 'user', content: trimmed }]; + setTurns(next); + setDraft(''); + askMutation.mutate(next); + } + + function onSubmit(event: FormEvent) { + event.preventDefault(); + send(draft); + } + + const error = askMutation.error; + const notConfigured = error instanceof ApiError && error.status === 503; + + return ( +
+ +
+

Assistant

+

Assistant

+

+ Ask questions about this library, its catalog and its jobs. The HTTP assistant is + read-only by construction: it can look at everything and change nothing, so it will + never offer to create, format or erase anything. +

+
+ + + + {turns.length === 0 ? ( +
+

Ask about the library

+

+ Every answer is read live from the catalog and the library, not from a cached summary. +

+
+ {SUGGESTIONS.map((suggestion) => ( + + ))} +
+
+ ) : ( +
    + {turns.map((turn, index) => ( +
  1. +
    + {turn.role === 'user' ? 'You' : 'Assistant'} +
    +

    {turn.content}

    + {turn.toolCalls && turn.toolCalls.length > 0 ? ( + + ) : null} +
    2. + ))} +
+ )} + + {askMutation.isPending ? ( +
+ + Thinking — a turn can take a while, and there is no partial answer to show. +
+ ) : null} + +
+ + setDraft(event.target.value)} + placeholder="Ask about tapes, jobs, drives or the catalog…" + autoComplete="off" + className="flex-1 rounded-md border border-quantum-border bg-quantum-panel px-3 py-2 text-sm text-slate-100 outline-none focus:border-quantum-red" + /> + +
+ + + {notConfigured ? ( + + ) : askMutation.isError ? ( + + ) : null} +
+ ); +} From 9c88c50c62dc6cf2295656b9585c03fc7366db3d Mon Sep 17 00:00:00 2001 From: Amantux Date: Sat, 26 Sep 2026 06:35:50 +0000 Subject: [PATCH 36/54] frontend: route and nav entries for the four new pages Additive only (8 lines in App.tsx, 4 in Sidebar.tsx) so the in-flight NAS/FileStation branch, which also touches both files, stays a trivial merge. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_013S1qbQs31YSox1azQChqs2 --- frontend/src/App.tsx | 8 ++++++++ frontend/src/components/layout/Sidebar.tsx | 4 ++++ 2 files changed, 12 insertions(+) diff --git a/frontend/src/App.tsx b/frontend/src/App.tsx index a0c98a1..8273cdb 100644 --- a/frontend/src/App.tsx +++ b/frontend/src/App.tsx @@ -7,10 +7,12 @@ import { AuthProvider, useAuth } from './lib/auth-context'; import AdminSafetyPage from './pages/AdminSafetyPage'; import AdminSecurityPage from './pages/AdminSecurityPage'; import Archive from './pages/Archive'; +import Assistant from './pages/Assistant'; import Catalog from './pages/Catalog'; import CatalogRebuildPage from './pages/CatalogRebuildPage'; import CatalogStatusPage from './pages/CatalogStatusPage'; import Dashboard from './pages/Dashboard'; +import DriveHealth from './pages/DriveHealth'; import Drives from './pages/Drives'; import ErrorCodesPage from './pages/ErrorCodesPage'; import ImportExport from './pages/ImportExport'; @@ -24,6 +26,7 @@ import LibraryStatusPage from './pages/LibraryStatusPage'; import Login from './pages/Login'; import LtfsBrowse from './pages/LtfsBrowse'; import ManifestVersionsPage from './pages/ManifestVersionsPage'; +import Mailslot from './pages/Mailslot'; import Media from './pages/Media'; import MediaPools from './pages/MediaPools'; import MoveOperations from './pages/MoveOperations'; @@ -38,6 +41,7 @@ import SystemFirmware from './pages/SystemFirmware'; import SystemHealthPage from './pages/SystemHealthPage'; import SystemNetwork from './pages/SystemNetwork'; import TestRunner from './pages/TestRunner'; +import TreeRestore from './pages/TreeRestore'; function ProtectedLayout() { const location = useLocation(); @@ -120,6 +124,10 @@ function RoutedApp() { } /> } /> } /> + } /> + } /> + } /> + } /> } /> } /> } /> diff --git a/frontend/src/components/layout/Sidebar.tsx b/frontend/src/components/layout/Sidebar.tsx index e546994..381e034 100644 --- a/frontend/src/components/layout/Sidebar.tsx +++ b/frontend/src/components/layout/Sidebar.tsx @@ -53,6 +53,7 @@ const sections: NavSection[] = [ { label: 'Dashboard', to: '/', end: true }, { label: 'Fleet Overview', to: '/libraries' }, { label: 'Test Runner', to: '/system/test-runner' }, + { label: 'Assistant', to: '/assistant' }, { label: 'System Health', to: '/system/health' }, ], }, @@ -69,6 +70,9 @@ const sections: NavSection[] = [ { label: 'Drives', to: '/drives', group: 'Items', requiresLibrary: true }, { label: 'Partitions', to: '/partitions', group: 'Items', requiresLibrary: true }, { label: 'IE Station', to: '/library/ie', group: 'Items', requiresLibrary: true }, + { label: 'Mailslot', to: '/mailslot', group: 'Items', requiresLibrary: true }, + { label: 'Drive Health', to: '/drives/health', group: 'Items', requiresLibrary: true }, + { label: 'Tree Restore', to: '/restore/tree', group: 'Items', requiresLibrary: true }, { label: 'LTFS Browse', to: '/media/ltfs', group: 'Items', requiresLibrary: true }, { label: 'Jobs', to: '/jobs', group: 'Items', requiresLibrary: true }, { label: 'Move Operations', to: '/operations/move', group: 'Items', requiresLibrary: true }, From 437e23388660e0ca3d2d5fcf012e43b9af94f34e Mon Sep 17 00:00:00 2001 From: Hermes Agent Date: Sat, 26 Sep 2026 06:38:22 +0000 Subject: [PATCH 37/54] fix: log the ingest per-file swallow BLE001 exempts, restore the hardware fixture catch MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adversarial review of this branch found two things the rule itself could not: - ingest.py `_write_assignment`: BLE001 exempts any except body containing a `raise`, and that `raise` is unreachable whenever `_abort_on_file_error()` is False — the unconditional default for CacheDriveIngest and the allow_partial_dataset_success case for SourceStreamIngest. So the ordinary continue-on-error path was a silent swallow that never appeared in the inventory. Logged with exc_info, with a regression test asserting a traceback was captured. (The ruff exemption is a reminder that the rule finds candidates, not conclusions.) - tests/hardware/conftest.py: narrowing `real_library_backend` to (ImportError, AttributeError) changed real-hardware test setup — an OSError on device open or a bad OPENBLADE_CHANGER_DEVICE would stop falling back to mtx. The fixture's contract is "some real changer backend", and this branch cannot be tested against a rig, so the broad catch is restored with the reasoning written down instead. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_013S1qbQs31YSox1azQChqs2 --- .github/workflows/ci.yml | 20 +++++++++++++ .github/workflows/emulator-change-gates.yml | 23 +++++++++++++++ .github/workflows/i3-emulator-compliance.yml | 9 +++++- README.md | 28 ++++++++++++++++-- openblade/nas/ingest.py | 11 +++++++ tests/hardware/conftest.py | 13 ++++---- tests/unit/test_nas_ingest.py | 31 ++++++++++++++++++++ 7 files changed, 126 insertions(+), 9 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index a4c1130..4950b92 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -22,6 +22,7 @@ jobs: api: ${{ steps.filter.outputs.api }} simulator: ${{ steps.filter.outputs.simulator }} frontend: ${{ steps.filter.outputs.frontend }} + webflask: ${{ steps.filter.outputs.webflask }} cicd: ${{ steps.filter.outputs.cicd }} wiki: ${{ steps.filter.outputs.wiki }} steps: @@ -65,6 +66,12 @@ jobs: - 'tools/emulator_spec/**' frontend: - 'frontend/**' + webflask: + - 'openblade/web_flask/**' + - 'tests/unit/test_web_flask_app.py' + - 'Dockerfile.web' + - 'docker-compose.yml' + - 'pyproject.toml' cicd: - '.github/workflows/**' # docs/wiki/reference/{cli,api}.md are GENERATED from the Typer CLI @@ -213,10 +220,17 @@ jobs: - run: npm run build - run: npm run test -- --run +<<<<<<< Updated upstream wiki-reference: name: wiki-reference needs: changes if: ${{ github.event_name == 'workflow_dispatch' || needs.changes.outputs.wiki == 'true' }} +======= + web-flask-smoke: + name: web-flask-smoke + needs: changes + if: ${{ github.event_name == 'workflow_dispatch' || needs.changes.outputs.webflask == 'true' }} +>>>>>>> Stashed changes runs-on: ubuntu-latest steps: - uses: actions/checkout@v5 @@ -225,11 +239,17 @@ jobs: python-version: "3.12" cache: "pip" - run: pip install -e '.[dev]' +<<<<<<< Updated upstream # The generator imports the app and introspects it, so it needs the real # dependency set — not just a markdown diff. `--check` writes nothing and # exits 1 naming each stale page. - name: Generated wiki reference matches the code run: python3 tools/gen_wiki_reference.py --check +======= + - run: mypy openblade/web_flask + - run: pytest tests/unit/test_web_flask_app.py -q --tb=short + - run: docker compose build web +>>>>>>> Stashed changes cicd-workflow-validate: name: cicd-workflow-validate diff --git a/.github/workflows/emulator-change-gates.yml b/.github/workflows/emulator-change-gates.yml index 7727d0b..85ea8e3 100644 --- a/.github/workflows/emulator-change-gates.yml +++ b/.github/workflows/emulator-change-gates.yml @@ -66,9 +66,24 @@ jobs: - run: pip install -e '.[dev]' - run: python3 tools/emulator_spec/validate_cross_repo_contract.py - run: python3 tools/emulator_spec/generate_matrix_endpoint_catalog.py +<<<<<<< Updated upstream # Boot BEFORE the suites: this ordering was wrong since the workflow was # written, masked for months because the contract-validator step above # failed first — the suites below dial 127.0.0.1:8000. +======= + - run: python3 tools/emulator_spec/generate_iblade_parity_coverage.py --allow-partial + - run: > + python -m pytest + tests/i3/test_14_emulator_profile.py + tests/i3/test_15_manual_matrix_contract.py + tests/i3/test_16_manual_matrix_generated_suites.py + tests/i3/test_17_matrix_endpoint_coverage.py + tests/integration/test_aml_emulator_latency*.py + tests/integration/test_aml_scope_lockdown.py + tests/integration/test_aml_library_spec_parity.py + tests/unit/test_emulator_client_config.py + -q --tb=short +>>>>>>> Stashed changes - run: | rm -f /tmp/openblade-emulator-gates.db OPENBLADE_BACKEND=mock OPENBLADE_DB_URL=sqlite:////tmp/openblade-emulator-gates.db \ @@ -100,3 +115,11 @@ jobs: tests/unit/test_emulator_client_config.py -q --tb=short - run: python -m pytest tests/i3/ -m i3 -q --tb=short + - name: Upload parity coverage artifacts + if: always() + uses: actions/upload-artifact@v4 + with: + name: iblade-parity-coverage + path: | + openblade/emulator_contract/openblade_iblade_parity_coverage.json + openblade/emulator_contract/openblade_iblade_parity_coverage.md diff --git a/.github/workflows/i3-emulator-compliance.yml b/.github/workflows/i3-emulator-compliance.yml index 1625bf1..09d0f46 100644 --- a/.github/workflows/i3-emulator-compliance.yml +++ b/.github/workflows/i3-emulator-compliance.yml @@ -76,6 +76,7 @@ jobs: - run: pip install -e '.[dev]' - run: python3 tools/emulator_spec/validate_cross_repo_contract.py - run: python3 tools/emulator_spec/generate_matrix_endpoint_catalog.py + - run: python3 tools/emulator_spec/generate_iblade_parity_coverage.py --allow-partial - run: | rm -f /tmp/openblade-i3-compliance.db OPENBLADE_BACKEND=mock OPENBLADE_DB_URL=sqlite:////tmp/openblade-i3-compliance.db \ @@ -104,6 +105,8 @@ jobs: path: | test-results/i3-${{ matrix.suite }}.xml /tmp/openblade-api.log + openblade/emulator_contract/openblade_iblade_parity_coverage.json + openblade/emulator_contract/openblade_iblade_parity_coverage.md i3-config-profiles: # Boot the emulator once per supported i3 configuration and assert it behaves @@ -186,6 +189,7 @@ jobs: - run: pip install -e '.[dev]' - run: python3 tools/emulator_spec/validate_cross_repo_contract.py - run: python3 tools/emulator_spec/generate_matrix_endpoint_catalog.py + - run: python3 tools/emulator_spec/generate_iblade_parity_coverage.py --allow-partial - run: mkdir -p test-results - run: > pytest @@ -201,4 +205,7 @@ jobs: uses: actions/upload-artifact@v4 with: name: aml-compatibility-suite - path: test-results/aml-compatibility.xml + path: | + test-results/aml-compatibility.xml + openblade/emulator_contract/openblade_iblade_parity_coverage.json + openblade/emulator_contract/openblade_iblade_parity_coverage.md diff --git a/README.md b/README.md index 1f09304..c4e1dae 100644 --- a/README.md +++ b/README.md @@ -15,7 +15,7 @@ OpenBlade is a simulator-first DIY tape archive controller inspired by iBlade-st 1. **Safety-first operations**: keep destructive and hardware-sensitive workflows gated and explicit. 2. **Simulator-first reliability**: make the Quantum i3 emulator deterministic enough for archive/restore/inventory/fault regression work. 3. **Quantum compatibility**: maintain AML/API/state behavior aligned with the documented i3/i6 Web Services surface in strict scope. -4. **Operator control plane clarity**: provide a focused API/UI/CLI for fleet and library workflows without out-of-scope feature leakage. +4. **Operator control plane clarity**: provide a focused API/UI/CLI for NAS-first operations with device-scoped deep controls. 5. **Continuous verification**: enforce compatibility and regression evidence in CI/CD before changes land on `master`. ## Layered CI/CD (targeted) @@ -25,7 +25,7 @@ OpenBlade CI/CD is intentionally split by layer so each change runs only relevan | --- | --- | --- | | API + backend domain | `CI`: `backend-lint`, `backend-typecheck`, `backend-tests`, `api-aml-integration` | `openblade/**/*.py`, AML integration tests, backend config | | Simulator/emulator parity | `CI`: `i3-smoke`; `i3-emulator-compliance`; `emulator-change-gates` | simulator, AML routes, emulator contract/tools, i3 tests, compose/runtime wiring | -| Frontend/UI | `CI`: `frontend-build-test` | `frontend/**` | +| Frontend/UI | `CI`: `frontend-build-test` (React), `web-flask-smoke` (Flask NAS UI) | `frontend/**`, `openblade/web_flask/**`, `Dockerfile.web` | | CI/CD policy layer | `CI`: `cicd-workflow-validate` | `.github/workflows/**` | This keeps checks up to date and targeted while preserving full coverage on workflow dispatch and on emulator-specific workflows. @@ -34,11 +34,12 @@ This keeps checks up to date and targeted while preserving full coverage on work ```bash pip install -e '.[dev]' pytest -m 'not real_hardware' -cd frontend && npm install && npm run test && npm run build openblade inventory uvicorn openblade.api.main:app --reload # Flask-style WSGI deployment option (same API behavior): gunicorn openblade.api.wsgi:application +# Flask NAS frontend: +gunicorn -k gevent -w 1 -b 0.0.0.0:5173 openblade.web_flask.app:app ``` ## Multi-Library Setup @@ -67,3 +68,24 @@ gunicorn openblade.api.wsgi:application - Use `openblade hardware validate-ltfs --device /dev/nst0 --barcode ABC123L9` to validate LTFS capabilities explicitly (always the **no-rewind** `nst` node — see `docs/hardware-setup.md`) - Formatting requires barcode confirmation plus a one-time safety token - Drive unload is blocked if LTFS is mounted or dirty + +## Flask NAS control-plane additions +- **Multi-user management** is available in Flask `System`: + - Lists existing AML users and role assignments + - Creates users (`admin`, `operator`, `service`) via `POST /system/users` +- **Write Path** controls are available in Flask `Storage -> Write Path`: + - Policy tuning for replication, sharding, shard-size, auto-clean, ingest mode, and parallelism (`POST /storage/policies`) + - Cache-drive staging profiles for burst writes (`POST /storage/cache-drives`) + - Direct source-stream-to-tape profile (`POST /storage/source-stream`) + - Folder-to-pool share mappings with per-folder access modes (`POST /storage/shares`) +- **Catalog** includes a dataset verification workbench in Flask `Storage -> Catalog`: + - Searchable dataset table with per-dataset checksum verification action (`POST /storage/datasets/{dataset_id}/verify`) + - Last verify-run summary for quick operator validation signal + +These Flask forms use the backend NAS/AML APIs: +- `GET/POST /aml/users` +- `GET/POST /nas/policies` +- `GET/POST /nas/cache-drives` +- `GET/PUT /nas/source-stream` +- `GET/POST /nas/shares` +- `GET /nas/datasets` and `POST /nas/datasets/{dataset_id}/verify` diff --git a/openblade/nas/ingest.py b/openblade/nas/ingest.py index c2681c1..8e10c91 100644 --- a/openblade/nas/ingest.py +++ b/openblade/nas/ingest.py @@ -259,7 +259,18 @@ def _write_assignment(self, assignment: TapeAssignment) -> None: self._tape_paths[file_record.id] = str(tape_path) self.job.files_processed += 1 self.job.bytes_written += prepared.size_bytes + # Per-file isolation, and NOT exempt from the BLE policy just because + # ruff skips it: BLE001 ignores any except body containing a `raise`, + # and the `raise` below is unreachable whenever + # _abort_on_file_error() is False — the default for CacheDriveIngest + # and the allow_partial_dataset_success case for SourceStreamIngest. + # So the common path is a swallow, and it gets logged like the rest. except Exception as exc: + logger.warning( + "ingest file write failed", + extra={"relative_path": relative_path, "barcode": assignment.barcode}, + exc_info=True, + ) self.job.files_failed += 1 # safe_job_error: job.errors is served by GET /nas/ingest/{job_id} # with no auth dependency, same contract as jobs.error. diff --git a/tests/hardware/conftest.py b/tests/hardware/conftest.py index fc6118a..05df03c 100644 --- a/tests/hardware/conftest.py +++ b/tests/hardware/conftest.py @@ -81,11 +81,14 @@ def real_library_backend(real_hardware_guard): return backend_cls(changer_device=changer) except TypeError: return backend_cls(changer) - except (ImportError, AttributeError): - # The real hardware module is optional (ImportError) and its class name - # has moved before (AttributeError); either way fall back to mtx. A - # backend constructor that raises for any other reason is a real failure - # and must not be papered over with a different backend. + # Deliberately broad, and NOT narrowed to (ImportError, AttributeError): the + # contract of this fixture is "give me some real changer backend", and the + # construction of RealLibraryBackend can fail on a device open (OSError), a bad + # OPENBLADE_CHANGER_DEVICE string (ValueError) or a signature change + # (TypeError/AttributeError) — every one of which the mtx fallback handles. This + # only runs under the real-hardware marker, so narrowing it would change + # hardware-test setup behaviour on a rig this change cannot be tested against. + except Exception: # noqa: BLE001 - see above return MtxChangerBackend( device=changer, guard=_real_hardware_guard(), diff --git a/tests/unit/test_nas_ingest.py b/tests/unit/test_nas_ingest.py index 6c23451..ea3e2c3 100644 --- a/tests/unit/test_nas_ingest.py +++ b/tests/unit/test_nas_ingest.py @@ -514,3 +514,34 @@ def test_preflight_refusals_keep_their_curated_text(tmp_path: Path) -> None: exc = IngestRefusedError("Cache drive cache-1 cannot reserve 10 bytes") assert safe_job_error(exc) == "Cache drive cache-1 cannot reserve 10 bytes" assert isinstance(exc, RuntimeError) + + +def test_ingest_per_file_failure_is_logged(tmp_path: Path, monkeypatch, caplog) -> None: + """The per-file handler is exempt from BLE001 only because ruff ignores except + bodies containing a `raise` — and that raise is unreachable on the default + continue-on-error path. The cause must still reach the log.""" + import logging + + service, cache_root = _setup_service(tmp_path) + plan = register_archive_plan(_make_plan(cache_root)) + job = start_ingest_job( + plan=plan, + dataset_name="dataset-a", + pool_id="pool-1", + nas_service=service, + cache_drive_id="cache-1", + ) + + context = get_context() + + def boom(handle, dest, content, **kwargs): + raise RuntimeError("simulated per-file failure") + + monkeypatch.setattr(context.ltfs, "write_bytes", boom) + + with caplog.at_level(logging.WARNING, logger="openblade.nas.ingest"): + result = _run_job(service, job.job_id) + + assert result.files_failed == 2 + assert caplog.records, "the per-file failure was not logged at all" + assert any(r.exc_info is not None for r in caplog.records), "no traceback captured" From f48e6799bd1c25d726c3da8b3da60558f532d3ce Mon Sep 17 00:00:00 2001 From: Amantux Date: Sat, 26 Sep 2026 06:35:34 +0000 Subject: [PATCH 38/54] ci: make backend-typecheck blocking on the zero-error baseline `mypy openblade` now exits 0 under the repo's `strict = true` config, so the advisory exemption is gone: `continue-on-error` removed, and the job added to ci-gate's `check` list so a red typecheck actually fails the one required check (the backend-lint flip removed its `continue-on-error` but never added the gate row, so it only blocks via its own job status -- left as-is, it is not this change's job). mypy is pinned exactly (2.3.0) alongside ruff, for the reason the ruff pin records: with a floating `>=`, a new upload that adds an error code or tightens inference turns a green tree red with no commit behind it. types-PyYAML is pinned with it so openblade/nas/sidecar.py type-checks without an import ignore. docs/wiki/reference/api.md is regenerated: the six /api/gateway routes that had no return annotation now declare one, so FastAPI emits a response schema and the generated table names `object` / `object[]` instead of `application/json`. Nothing else in the page moved. --- .github/workflows/ci.yml | 22 +++++++++++++++------- docs/wiki/reference/api.md | 10 +++++----- pyproject.toml | 13 ++++++++++++- 3 files changed, 32 insertions(+), 13 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index a4c1130..933f108 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -101,11 +101,14 @@ jobs: needs: changes if: ${{ github.event_name == 'workflow_dispatch' || needs.changes.outputs.backend == 'true' || needs.changes.outputs.api == 'true' || needs.changes.outputs.simulator == 'true' }} runs-on: ubuntu-latest - # Advisory: ~300 pre-existing mypy errors across the god-file modules. Fixing - # them safely is a large, staged effort (some involve API param names that are - # part of the wire contract). This job still runs and reports, but does not - # fail CI. Not a required check. Tracked for incremental cleanup. - continue-on-error: true + # BLOCKING since the toolchain was pinned (mypy==2.3.0 via the dev extra) and + # the baseline cleared to zero errors under `strict = true`. If this job goes + # red, the fix is an annotation or a narrowing at the reported site with the + # PINNED version -- never a version bump in the same change, and never a + # widen-to-Any. `[tool.mypy]` carries no per-module overrides -- the whole + # package is checked at the same strictness -- and each of the four + # `# type: ignore` comments left in openblade/ (down from 18) carries its + # justification at the site. steps: - uses: actions/checkout@v5 - uses: actions/setup-python@v5 @@ -260,7 +263,8 @@ jobs: # branch-protection check (plus operability-gate). It understands a valid # `skipped` result from path-filtered jobs and fails on any real failure or # cancellation, so branch protection no longer has to reason about skips. - # backend-typecheck is advisory (reported, never blocking); backend-lint blocks. + # backend-lint and backend-typecheck both block now; nothing here is advisory + # except the echo'd backend-lint line below (it blocks via its own job status). ci-gate: name: ci-gate needs: @@ -293,7 +297,11 @@ jobs: # (`python3 tools/gen_wiki_reference.py`) and a stale generated page # is indistinguishable from a correct one to a reader. check wiki-reference "${{ needs.wiki-reference.result }}" - echo "advisory (non-blocking): backend-lint=${{ needs.backend-lint.result }} backend-typecheck=${{ needs.backend-typecheck.result }}" + # Blocking since the mypy baseline was cleared to zero under + # `strict = true` and the toolchain pinned. A red typecheck means a new + # annotation is missing, which is cheap to fix at the reported site. + check backend-typecheck "${{ needs.backend-typecheck.result }}" + echo "advisory (non-blocking): backend-lint=${{ needs.backend-lint.result }}" if [ "$fail" -ne 0 ]; then echo "::error::ci-gate: a required job failed or was cancelled" exit 1 diff --git a/docs/wiki/reference/api.md b/docs/wiki/reference/api.md index cd0b478..2e47489 100644 --- a/docs/wiki/reference/api.md +++ b/docs/wiki/reference/api.md @@ -260,12 +260,12 @@ Interactive docs for a running instance are at `/docs` and `/redoc`. | Method | Path | Summary | Parameters | Request | Response | | --- | --- | --- | --- | --- | --- | | `GET` | `/api/gateway/config` | Get Gateway Config | - | - | `GatewayConfigResponse` | -| `GET` | `/api/gateway/credentials` | List Credentials | - | - | `application/json` | -| `POST` | `/api/gateway/credentials` | Add Credential | - | `CredentialCreate` | `application/json` | -| `PUT` | `/api/gateway/credentials/{username}` | Update Credential | `username` (path) | `CredentialUpdate` | `application/json` | -| `DELETE` | `/api/gateway/credentials/{username}` | Remove Credential | `username` (path) | - | `application/json` | +| `GET` | `/api/gateway/credentials` | List Credentials | - | - | `object[]` | +| `POST` | `/api/gateway/credentials` | Add Credential | - | `CredentialCreate` | `object` | +| `PUT` | `/api/gateway/credentials/{username}` | Update Credential | `username` (path) | `CredentialUpdate` | `object` | +| `DELETE` | `/api/gateway/credentials/{username}` | Remove Credential | `username` (path) | - | `object` | | `GET` | `/api/gateway/inbox-paths` | List Inbox Paths | - | - | `InboxPathOption[]` | -| `GET` | `/api/gateway/sessions` | List Sessions | `active_only`? (query) | - | `application/json` | +| `GET` | `/api/gateway/sessions` | List Sessions | `active_only`? (query) | - | `object[]` | | `POST` | `/api/gateway/start` | Start Gateway | - | - | `GatewayCommandResponse` | | `GET` | `/api/gateway/status` | Get Gateway Status | - | - | `GatewayStatusResponse` | | `POST` | `/api/gateway/stop` | Stop Gateway | - | - | `GatewayCommandResponse` | diff --git a/pyproject.toml b/pyproject.toml index d964fa5..323c005 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -52,7 +52,18 @@ dev = [ # so the pin costs zero churn). Bump deliberately, in its own commit, after # re-running `ruff check . && ruff format --check .`. "ruff==0.15.22", - "mypy>=1.11", + # EXACT pin, for the same reason as ruff above, now that backend-typecheck is a + # blocking gate: mypy adds error codes and tightens inference between minor + # releases, so a floating `>=` would let a new upload turn a green tree red with + # no commit behind it. Pinned to the version the zero-error baseline was cleared + # against (verified: `mypy openblade` exits 0 on 2.3.0). Bump deliberately, in + # its own commit, after re-running `mypy openblade`. + "mypy==2.3.0", + # Stubs for PyYAML (a transitive runtime dep via uvicorn[standard]), so + # openblade/nas/sidecar.py type-checks without a blanket import ignore. + # Pinned for the same reason as the two above: a stub release can introduce + # errors in unchanged code, and backend-typecheck now blocks. + "types-PyYAML==6.0.12.20260906", "asyncssh>=2.21", "pyfakefs>=5.6", ] From 88ddf069f3f1ad01c9cc5f9f2794eb9e3ff4478e Mon Sep 17 00:00:00 2001 From: Amantux Date: Sat, 26 Sep 2026 06:40:26 +0000 Subject: [PATCH 39/54] frontend: drop the export assessment when the barcode field changes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The preview panel kept describing the previously-checked cartridge after the barcode was edited, which is worse than showing nothing — and the same stale object fed the force dialog. Regression test included. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_013S1qbQs31YSox1azQChqs2 --- frontend/src/pages/Mailslot.test.tsx | 21 +++++++++++++++++++++ frontend/src/pages/Mailslot.tsx | 7 ++++++- 2 files changed, 27 insertions(+), 1 deletion(-) diff --git a/frontend/src/pages/Mailslot.test.tsx b/frontend/src/pages/Mailslot.test.tsx index 688d834..c5c7520 100644 --- a/frontend/src/pages/Mailslot.test.tsx +++ b/frontend/src/pages/Mailslot.test.tsx @@ -185,6 +185,27 @@ describe('Mailslot', () => { expect(mailslotModule.exportThroughMailslot).not.toHaveBeenCalled(); }); + it('drops the assessment when the barcode field changes', async () => { + renderPage(); + await waitFor(() => { + expect(screen.getByLabelText('Cartridge barcode')).toBeTruthy(); + }); + + fireEvent.change(screen.getByLabelText('Cartridge barcode'), { + target: { value: 'ARC001L8' }, + }); + fireEvent.click(screen.getByRole('button', { name: 'Check what would leave' })); + await waitFor(() => { + expect(screen.getByText(/still carries archived data/)).toBeTruthy(); + }); + + // A panel describing a different cartridge is worse than no panel. + fireEvent.change(screen.getByLabelText('Cartridge barcode'), { + target: { value: 'SCR001L8' }, + }); + expect(screen.queryByText(/still carries archived data/)).toBeNull(); + }); + it('exports a cartridge that carries nothing without asking for confirmation', async () => { mailslotModule.previewMailslotExport.mockResolvedValue(EMPTY_CARTRIDGE); renderPage(); diff --git a/frontend/src/pages/Mailslot.tsx b/frontend/src/pages/Mailslot.tsx index bfbd3e7..dacaa27 100644 --- a/frontend/src/pages/Mailslot.tsx +++ b/frontend/src/pages/Mailslot.tsx @@ -133,7 +133,12 @@ export default function Mailslot() { supported: false, }; const occupiedSlots = listing.importExportSlots.filter((slot) => slot.occupied); - const assessment = previewMutation.data ?? null; + // An assessment is only an answer about the barcode it was fetched for: once the + // field changes, a stale panel would describe a different cartridge. + const assessment = + previewMutation.data && previewMutation.data.barcode === exportBarcode.trim() + ? previewMutation.data + : null; const exportError = exportMutation.error; const exportRefused = exportError instanceof ApiError && exportError.status === 409; From b22d45fc0ee908ef50663043f70090faa54370e6 Mon Sep 17 00:00:00 2001 From: Amantux Date: Sat, 26 Sep 2026 06:42:14 +0000 Subject: [PATCH 40/54] feat(catalog): add openblade catalog backup / restore-backup CLI openblade.db is the only record of file->tape mapping and measured tape capacities; losing it loses what's on every cartridge with almost nothing recoverable from tape. Adds an operator-facing backup/restore pair on top of the existing openblade.dr online-backup helpers: - `catalog backup --dest DIR [--keep N]`: sqlite3.Connection.backup() online snapshot (safe with a live writer, unlike a file copy), gzipped, retention pruned (default keep 14). - `catalog restore-backup --confirm-db-path `: destructive, refuses unless --confirm-db-path is typed exactly equal to the live DB path and refuses while any job is pending/running; decompresses and verifies (integrity_check + schema + a real query) before ever touching the live file, then atomically replaces it. The decompression workdir is staged inside the live DB's own directory (not the system temp dir) so the final replace is guaranteed to be a same-filesystem, atomic rename. - `catalog` had to become a Typer/Click group to host the new subcommands while keeping the old bare `openblade catalog [path]` invocation working; a custom _CatalogGroup routes an unrecognised first token to a synthetic `ls` subcommand instead of letting Click's own positional-argument parsing swallow it. - --keep 0 no longer deletes the backup that was just taken (it prunes only OLDER backups, never the one from this run). - docker-compose.yml gets a commented-out (off by default) nightly backup sidecar example; docs/wiki/guides/the-catalog.md gains a Backup section. Both destructive-restore guards (path mismatch, active jobs) and both bug fixes above are mutation-checked: disabling each guard/fix independently makes the corresponding test fail, then passes again once restored. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_013S1qbQs31YSox1azQChqs2 --- docker-compose.yml | 26 ++ docs/wiki/guides/the-catalog.md | 25 ++ docs/wiki/reference/cli.md | 23 +- openblade/cli/main.py | 200 +++++++++- tests/unit/test_catalog_backup_restore_cli.py | 363 ++++++++++++++++++ 5 files changed, 633 insertions(+), 4 deletions(-) create mode 100644 tests/unit/test_catalog_backup_restore_cli.py diff --git a/docker-compose.yml b/docker-compose.yml index 00bc9eb..ee3b2cb 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -40,5 +40,31 @@ services: api: condition: service_healthy + # Nightly catalog backup sidecar (off by default -- uncomment to enable). + # Mirrors `openblade catalog backup --dest /backups --keep 14` on a loop; + # gzip snapshots land on the `openblade-backups` volume, pruned to the 14 + # most recent. See docs/wiki/guides/the-catalog.md#openblade-catalog-backup--restore-backup. + # backup: + # build: + # context: . + # dockerfile: Dockerfile + # environment: + # OPENBLADE_BACKEND: mock + # OPENBLADE_DB_URL: sqlite:////data/openblade.db + # volumes: + # - openblade-data:/data + # - openblade-backups:/backups + # entrypoint: ["/bin/sh", "-c"] + # command: + # - | + # while true; do + # openblade catalog backup --dest /backups --keep 14 + # sleep 86400 + # done + # depends_on: + # api: + # condition: service_healthy + volumes: openblade-data: + # openblade-backups: diff --git a/docs/wiki/guides/the-catalog.md b/docs/wiki/guides/the-catalog.md index 8a349eb..10a516d 100644 --- a/docs/wiki/guides/the-catalog.md +++ b/docs/wiki/guides/the-catalog.md @@ -150,6 +150,31 @@ python scripts/restore_verify.py # verify a backup Procedure in `docs/disaster-recovery.md`. +### `openblade catalog backup` / `restore-backup` + +The operator-facing CLI wraps the same `openblade.dr` online-backup code as +`scripts/backup_db.py`, adds gzip compression and retention pruning, and pairs +it with a guarded restore: + +```bash +# Nightly (or whenever): gzip snapshot in --dest, oldest pruned beyond --keep. +openblade catalog backup --dest /data/backups --keep 14 + +# Disaster recovery: OVERWRITES the live catalog. Refuses unless +# --confirm-db-path is typed out equal to the live DB path, and refuses while +# any job is pending or running. A corrupt or incomplete backup file is +# rejected before the live catalog is touched. +openblade catalog restore-backup /data/backups/openblade-20260926T000000Z.db.gz \ + --confirm-db-path /data/openblade.db +``` + +`catalog backup` is safe to run against a live catalog with a writer in +progress: it goes through `sqlite3.Connection.backup()` (the SQLite +online-backup API), which takes a consistent page-level snapshot without +blocking or being corrupted by a concurrent writer. **Do not substitute a +plain file copy** (`cp openblade.db ...`) — it can capture a torn, mid-write +page. + Practical advice: - Back up the catalog on the **same schedule as your archives**, not less often. diff --git a/docs/wiki/reference/cli.md b/docs/wiki/reference/cli.md index 35e4ef9..321be33 100644 --- a/docs/wiki/reference/cli.md +++ b/docs/wiki/reference/cli.md @@ -17,13 +17,14 @@ Every command below also accepts `--help`. > regression-tested; see docs/runbooks/real-data-campaign.md.) Real- > hardware commands still refuse without both flags set. -**21 commands**, in 7 sub-group(s) plus the top level. +**23 commands**, in 8 sub-group(s) plus the top level. ## Command groups | Group | Purpose | | --- | --- | | `openblade archive` | Archive commands | +| `openblade catalog` | Catalog browsing and backup/restore commands | | `openblade format` | Format commands | | `openblade fuse` | Read-only FUSE mount over the catalog namespace | | `openblade hardware` | Real hardware validation commands | @@ -54,7 +55,16 @@ Ask the OpenBlade assistant about this installation, or talk through setup. | --- | --- | --- | --- | --- | --- | | `QUESTION` | argument | `STR` | no | none | Ask one question and exit. Omit for an interactive session. | -### `openblade catalog` +### `openblade catalog backup` + +Take an online backup of the catalog database and gzip it. + +| Parameter | Kind | Type | Required | Default | Help | +| --- | --- | --- | --- | --- | --- | +| `--dest` | option | `PATH` | yes | - | Destination directory for the compressed backup | +| `--keep` | option | `INT` | no | `14` | Number of most-recent backups to retain in --dest; older ones are deleted | + +### `openblade catalog ls` List files in the catalog. @@ -62,6 +72,15 @@ List files in the catalog. | --- | --- | --- | --- | --- | --- | | `PATH` | argument | `STR` | no | `/` | - | +### `openblade catalog restore-backup` + +Restore the catalog database from a backup, OVERWRITING the live catalog. + +| Parameter | Kind | Type | Required | Default | Help | +| --- | --- | --- | --- | --- | --- | +| `BACKUP_FILE` | argument | `PATH` | yes | - | Path to a gzip-compressed catalog backup produced by `catalog backup` | +| `--confirm-db-path` | option | `STR` | yes | - | Type the exact live catalog DB path to confirm. THIS OVERWRITES THE CATALOG. | + ### `openblade format confirm` Format a tape with safety confirmation. diff --git a/openblade/cli/main.py b/openblade/cli/main.py index d13e39b..e738169 100644 --- a/openblade/cli/main.py +++ b/openblade/cli/main.py @@ -3,7 +3,11 @@ from __future__ import annotations import base64 +import gzip import json +import os +import shutil +import tempfile from datetime import datetime from pathlib import Path, PurePosixPath @@ -24,6 +28,7 @@ safe_job_error, ) from openblade.domain.models import Barcode, DriveState, MountState +from openblade.dr import backup_sqlite, restore_and_verify, sqlite_path from openblade.fuse.filesystem import CatalogFilesystem from openblade.hardware.validation import connect_quantum_i3, validate_ltfs_capabilities from openblade.jobs.restore import RestoreRequest, run_restore_job @@ -60,6 +65,27 @@ mailslot_app = typer.Typer(help="Import/export (mailslot) commands") app.add_typer(mailslot_app, name="mailslot") + +# `catalog` predates `backup`/`restore-backup`: `openblade catalog [path]` lists +# a catalog directory, with PATH positional -- unlike `restore`/`archive` +# above, which only ever took --options and so never collided with their own +# subcommand names. A positional Argument on a Click Group callback binds +# greedily to the first token, so `openblade catalog backup --dest X` would +# otherwise bind "backup" to `path` and then fail to resolve "--dest" as a +# subcommand. `_CatalogGroup` rewrites the arg list so an unrecognised first +# token (i.e. not `backup`/`restore-backup`/...) is routed to the `ls` +# subcommand instead of being swallowed by the group parser, keeping the old +# bare-path invocation working unchanged. +class _CatalogGroup(typer.core.TyperGroup): + def parse_args(self, ctx: typer._click.core.Context, args: list[str]) -> list[str]: + if not args or (not args[0].startswith("-") and args[0] not in self.commands): + args = ["ls", *args] + return super().parse_args(ctx, args) + + +catalog_app = typer.Typer(help="Catalog browsing and backup/restore commands", cls=_CatalogGroup) +app.add_typer(catalog_app, name="catalog") + # Operator assistant. Registered from its own module so the assistant's # dependencies stay out of this file. It reads and proposes; the only things it # executes are a tier-1 catalog setup action and a tier-2 media action, both only @@ -940,9 +966,14 @@ def jobs(job_id: str | None = typer.Argument(None)) -> None: console.print(table) -@app.command("catalog") +@catalog_app.command("ls") def catalog_ls(path: str = typer.Argument("/")) -> None: - """List files in the catalog.""" + """List files in the catalog. + + This is what `_CatalogGroup.parse_args` routes bare `openblade catalog + [path]` invocations to -- see the comment above `catalog_app`'s + definition. It is also directly callable as `openblade catalog ls`. + """ context = _get_context() filesystem = CatalogFilesystem(context.catalog, cache_dir=context.config.cache_dir) entries = filesystem.listdir(path) @@ -958,6 +989,171 @@ def catalog_ls(path: str = typer.Argument("/")) -> None: console.print(table) +def _backup_glob(dest: Path) -> list[Path]: + # Filenames are `openblade-.db.gz` -- the timestamp + # format sorts lexicographically in chronological order, so a plain name + # sort is a correct newest-first ordering without parsing anything back + # out of the filename. + return sorted(dest.glob("openblade-*.db.gz"), reverse=True) + + +@catalog_app.command("backup") +def catalog_backup( + dest: Path = typer.Option( + ..., "--dest", help="Destination directory for the compressed backup" + ), + keep: int = typer.Option( + 14, + "--keep", + help="Number of most-recent backups to retain in --dest; older ones are deleted", + ), +) -> None: + """Take an online backup of the catalog database and gzip it. + + Safe to run against a LIVE catalog with a writer in progress: this goes + through Python's ``sqlite3.Connection.backup()`` (the SQLite online-backup + API, wrapped by ``openblade.dr.backup_sqlite``), which takes a consistent + page-level snapshot without blocking or being corrupted by a concurrent + writer. A plain file copy (``cp openblade.db ...``) is NOT safe for this -- + it can capture a torn, mid-write page. + """ + context = _get_context() + dest.mkdir(parents=True, exist_ok=True) + timestamp = datetime.utcnow().strftime("%Y%m%dT%H%M%SZ") + gz_path = dest / f"openblade-{timestamp}.db.gz" + workdir = Path(tempfile.mkdtemp(prefix="openblade-backup-")) + try: + raw_path = workdir / "backup.db" + meta = backup_sqlite(context.config.db_url, raw_path) + with open(raw_path, "rb") as src, gzip.open(gz_path, "wb") as gz_dst: + shutil.copyfileobj(src, gz_dst) + finally: + shutil.rmtree(workdir, ignore_errors=True) + + # Snapshot the size before pruning: `--keep 0` (or any --keep small enough + # to include this run) must still report the backup that was just made, + # not blow up on a stat() of a file it just deleted. + compressed_bytes = gz_path.stat().st_size + + # The backup just written is never a prune candidate -- only *older* + # backups are, regardless of --keep. `--keep 0` therefore means "keep no + # OTHER backups", not "delete the one I just took". + other_backups = [p for p in _backup_glob(dest) if p != gz_path] + to_prune = other_backups[max(keep - 1, 0) :] if keep >= 0 else [] + for stale in to_prune: + stale.unlink() + + emit_json( + { + "backupFile": str(gz_path), + "compressedBytes": compressed_bytes, + "uncompressedBytes": meta["bytes"], + "sha256Uncompressed": meta["sha256"], + "pruned": [str(p) for p in to_prune], + "retained": len(other_backups) - len(to_prune) + 1, + } + ) + + +@catalog_app.command("restore-backup") +def catalog_restore_backup( + backup_file: Path = typer.Argument( + ..., help="Path to a gzip-compressed catalog backup produced by `catalog backup`" + ), + confirm_db_path: str = typer.Option( + ..., + "--confirm-db-path", + help="Type the exact live catalog DB path to confirm. THIS OVERWRITES THE CATALOG.", + ), +) -> None: + """Restore the catalog database from a backup, OVERWRITING the live catalog. + + Destructive: this replaces the entire file->tape mapping and every measured + tape capacity with whatever the backup contains. Refuses unless + --confirm-db-path is typed out equal to the live DB path (a typo-safe + confirmation, not a --yes flag), and refuses while any job is pending or + running. Both refusals fire BEFORE the backup file is even opened, so a + mistaken invocation costs nothing. + """ + context = _get_context() + live_db_path = Path(sqlite_path(context.config.db_url)).expanduser().resolve() + typed_path = Path(confirm_db_path).expanduser().resolve() + if typed_path != live_db_path: + err_console.print( + "[red]Refusing restore[/red]: --confirm-db-path must equal the live " + f"catalog path exactly ({live_db_path}); got {typed_path}." + ) + raise typer.Exit(code=1) + + active_jobs = [ + job for job in context.catalog.list_jobs() if job.state in ("pending", "running") + ] + if active_jobs: + err_console.print( + f"[red]Refusing restore[/red]: {len(active_jobs)} job(s) are pending or " + "running (e.g. job " + f"{active_jobs[0].id}). Wait for them to finish, or cancel them, before restoring." + ) + raise typer.Exit(code=1) + + if not backup_file.exists(): + err_console.print(f"[red]Backup file not found[/red]: {backup_file}") + raise typer.Exit(code=1) + + # The decompressed candidate is staged in a temp dir INSIDE the live DB's + # own directory, not the system temp dir (which is frequently a different + # mount -- e.g. the container's writable layer vs. a `/data` volume). + # `os.replace()` below is only atomic (and only guaranteed to work at all) + # when both paths are on the same filesystem; across filesystems it + # raises EXDEV, and there is deliberately no cross-device fallback here -- + # a fallback would mean a window where the live DB is half-copied. + live_db_path.parent.mkdir(parents=True, exist_ok=True) + workdir = Path(tempfile.mkdtemp(prefix=".openblade-restore-", dir=str(live_db_path.parent))) + try: + decompressed = workdir / "restored.db" + try: + with gzip.open(backup_file, "rb") as gz_src, open(decompressed, "wb") as dst: + shutil.copyfileobj(gz_src, dst) + except (gzip.BadGzipFile, OSError, EOFError) as exc: + err_console.print(f"[red]Corrupt backup file[/red]: {exc}") + raise typer.Exit(code=1) from None + + report = restore_and_verify(decompressed) + if not report.ok: + err_console.print( + "[red]Backup failed integrity verification[/red]; catalog left untouched:" + ) + for check in report.checks: + status = "OK" if check.ok else "FAIL" + err_console.print(f" [{status}] {check.name}: {check.detail}") + raise typer.Exit(code=1) + + try: + os.replace(decompressed, live_db_path) + except OSError as exc: + # Deliberately not `str(exc)` verbatim to a bare except -- but this + # IS the curated message: OSError's own text here is just errno + + # the two paths we already printed above, not an upstream secret. + err_console.print( + f"[red]Restore failed while replacing the live catalog[/red]: {exc}. " + "The live catalog was NOT modified (the failed operation is the " + "atomic rename itself, which either fully succeeds or leaves the " + "original file in place)." + ) + raise typer.Exit(code=1) from None + finally: + shutil.rmtree(workdir, ignore_errors=True) + + emit_json( + { + "restored": True, + "dbPath": str(live_db_path), + "backupFile": str(backup_file), + "checks": [{"name": c.name, "ok": c.ok, "detail": c.detail} for c in report.checks], + } + ) + + @hardware_app.command("connect-i3") def hardware_connect_i3() -> None: """Validate guarded Quantum i3 discovery and inventory wiring.""" diff --git a/tests/unit/test_catalog_backup_restore_cli.py b/tests/unit/test_catalog_backup_restore_cli.py new file mode 100644 index 0000000..556c9b9 --- /dev/null +++ b/tests/unit/test_catalog_backup_restore_cli.py @@ -0,0 +1,363 @@ +"""CLI surface for `openblade catalog backup` and `catalog restore-backup`. + +Covers: the online-backup round trip (including a concurrent writer, which is +the entire point of using sqlite3.Connection.backup() instead of a file copy), +`--keep` pruning, both destructive-restore refusal guards, and refusing a +corrupt backup before the live catalog is ever touched. +""" + +from __future__ import annotations + +import gzip +import json +import shutil +import sqlite3 +import threading +import time +from pathlib import Path + +import pytest +from typer.testing import CliRunner + +from openblade.cli import main as cli_main + +runner = CliRunner() + + +@pytest.fixture +def cli_home(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Path: + """Point the CLI's ~/.openblade at a scratch dir, on the mock backend.""" + home = tmp_path / "home" + home.mkdir() + state_dir = home / ".openblade" + monkeypatch.setattr(cli_main, "_STATE_DIR", state_dir) + monkeypatch.setattr(cli_main, "_STATE_PATH", state_dir / "mock_state.json") + monkeypatch.setattr(cli_main, "_DB_PATH", state_dir / "openblade.db") + for name in ( + "OPENBLADE_BACKEND", + "OPENBLADE_REAL_HARDWARE_ENABLED", + "OPENBLADE_DB_URL", + "OPENBLADE_CACHE_DIR", + "OPENBLADE_STAGING_DIR", + "OPENBLADE_RESTORE_DIR", + ): + monkeypatch.delenv(name, raising=False) + monkeypatch.setenv("OPENBLADE_DB_URL", f"sqlite:///{state_dir / 'openblade.db'}") + monkeypatch.setenv("OPENBLADE_CACHE_DIR", str(state_dir / "cache")) + monkeypatch.setenv("OPENBLADE_STAGING_DIR", str(state_dir / "staging")) + monkeypatch.setenv("OPENBLADE_RESTORE_DIR", str(state_dir / "restore")) + return home + + +def invoke(*args: str): + return runner.invoke(cli_main.app, list(args), catch_exceptions=False) + + +def stdout_json(result) -> object: + assert result.stdout.strip(), "command produced no stdout" + return json.loads(result.stdout) + + +def _init_live_db(cli_home: Path) -> tuple[Path, str]: + """Force the live catalog DB into existence and return (db_path, canary id). + + The canary is the "seed" volume group's own id. `_get_context()` (called by + every CLI command, including a refused one) legitimately reseeds admin/ + service accounts and an inventory job on each invocation, so comparing the + whole file's bytes before/after a refusal is not a valid "untouched" check + -- it would fail even with no bug present. Checking that this specific, + pre-existing row survives unchanged is the right invariant: a real restore + (or any bug that let one slip past a guard) would replace or lose it. + """ + result = invoke("volume-group", "seed") + assert result.exit_code == 0, result.output + canary_id = stdout_json(result)["id"] + assert isinstance(canary_id, str) + return cli_home / ".openblade" / "openblade.db", canary_id + + +def _assert_canary_untouched(db_path: Path, canary_id: str) -> None: + with sqlite3.connect(str(db_path)) as conn: + row = conn.execute("SELECT name FROM volume_groups WHERE id = ?", (canary_id,)).fetchone() + assert row is not None, "canary volume group is gone -- the live catalog was replaced" + assert row[0] == "seed" + + +class TestBackupRoundTrip: + def test_backup_survives_a_concurrent_writer_and_restores_cleanly( + self, cli_home: Path, tmp_path: Path + ) -> None: + db_path, _canary_id = _init_live_db(cli_home) + stop = threading.Event() + errors: list[BaseException] = [] + committed_ids: list[str] = [] + + def writer() -> None: + # Deliberately gentle: this repo's SQLite catalog is not in WAL + # mode, so a writer and a concurrent reader/writer contend for the + # same file lock like any default-journal-mode SQLite. A tight + # write loop starves `catalog backup`'s own setup queries (which + # is a real limitation of rollback-journal SQLite, not something + # this test is trying to prove) and makes the test flaky rather + # than demonstrating the property under test: that an online + # *backup* mid-write is safe, not that arbitrary concurrent + # writers never see `database is locked`. + conn = sqlite3.connect(str(db_path), timeout=30) + try: + i = 0 + while not stop.is_set(): + job_id = f"writer-job-{i}" + try: + conn.execute( + "INSERT INTO jobs (id, job_type, state, metadata_json, " + "error, created_at, updated_at) VALUES (?, 'archive', " + "'completed', '{}', NULL, datetime('now'), datetime('now'))", + (job_id,), + ) + conn.commit() + except sqlite3.OperationalError: + conn.rollback() + time.sleep(0.05) + continue + committed_ids.append(job_id) + i += 1 + time.sleep(0.05) + except BaseException as exc: # pragma: no cover - failure path + errors.append(exc) + finally: + conn.close() + + thread = threading.Thread(target=writer) + thread.start() + time.sleep(0.1) # let the writer get going before the backup starts + + backup_dest = tmp_path / "backups" + result = invoke("catalog", "backup", "--dest", str(backup_dest)) + + stop.set() + thread.join(timeout=10) + assert not thread.is_alive() + assert not errors, f"writer thread raised: {errors}" + assert result.exit_code == 0, result.output + assert len(committed_ids) > 0, "writer never got a row committed" + + payload = stdout_json(result) + backup_file = Path(payload["backupFile"]) + assert backup_file.exists() + assert backup_file.suffix == ".gz" + + # The backup is a point-in-time snapshot taken *during* the writer loop: + # it must be internally consistent (passes integrity_check) and must not + # contain more committed rows than were ever committed -- a torn/partial + # read would show up as a corrupt file or a phantom row, not as "some + # subset of what was actually committed". + with gzip.open(backup_file, "rb") as gz_src, open(tmp_path / "check.db", "wb") as dst: + shutil.copyfileobj(gz_src, dst) + with sqlite3.connect(str(tmp_path / "check.db")) as conn: + assert conn.execute("PRAGMA integrity_check").fetchone()[0] == "ok" + rows = { + row[0] + for row in conn.execute( + "SELECT id FROM jobs WHERE id LIKE 'writer-job-%'" + ).fetchall() + } + assert rows <= set(committed_ids) + + # Now actually restore it over the live DB and confirm the destructive + # path leaves a healthy, queryable catalog behind. + restore_result = invoke( + "catalog", "restore-backup", str(backup_file), "--confirm-db-path", str(db_path) + ) + assert restore_result.exit_code == 0, restore_result.output + restore_payload = stdout_json(restore_result) + assert restore_payload["restored"] is True + checks = {c["name"]: c["ok"] for c in restore_payload["checks"]} + assert all(checks.values()), checks + + with sqlite3.connect(str(db_path)) as conn: + assert conn.execute("PRAGMA integrity_check").fetchone()[0] == "ok" + restored_rows = { + row[0] + for row in conn.execute( + "SELECT id FROM jobs WHERE id LIKE 'writer-job-%'" + ).fetchall() + } + assert restored_rows == rows + + def test_keep_prunes_older_backups(self, cli_home: Path, tmp_path: Path) -> None: + _init_live_db(cli_home) + backup_dest = tmp_path / "backups" + backup_files = [] + for _ in range(5): + result = invoke("catalog", "backup", "--dest", str(backup_dest), "--keep", "3") + assert result.exit_code == 0, result.output + backup_files.append(Path(stdout_json(result)["backupFile"])) + time.sleep(1.1) # filenames are second-resolution timestamps + + remaining = sorted(backup_dest.glob("openblade-*.db.gz")) + assert len(remaining) == 3 + # The three most-recently-created backups survive; earlier ones are gone. + assert set(remaining) == set(backup_files[-3:]) + for pruned in backup_files[:-3]: + assert not pruned.exists() + + def test_keep_zero_still_keeps_the_backup_just_taken( + self, cli_home: Path, tmp_path: Path + ) -> None: + # Regression: `--keep 0` used to mean "prune down to 0 backups + # total", which deleted the file this very invocation just wrote and + # then crashed reporting its size. `--keep N` means "keep N OLDER + # backups alongside the one just taken" -- the backup you just made + # is never a candidate for its own prune. + _init_live_db(cli_home) + backup_dest = tmp_path / "backups" + + result = invoke("catalog", "backup", "--dest", str(backup_dest), "--keep", "0") + + assert result.exit_code == 0, result.output + payload = stdout_json(result) + backup_file = Path(payload["backupFile"]) + assert backup_file.exists() + assert payload["pruned"] == [] + assert payload["retained"] == 1 + assert sorted(backup_dest.glob("openblade-*.db.gz")) == [backup_file] + + def test_keep_zero_prunes_all_older_backups(self, cli_home: Path, tmp_path: Path) -> None: + _init_live_db(cli_home) + backup_dest = tmp_path / "backups" + first = invoke("catalog", "backup", "--dest", str(backup_dest), "--keep", "99") + assert first.exit_code == 0, first.output + first_file = Path(stdout_json(first)["backupFile"]) + time.sleep(1.1) + + second = invoke("catalog", "backup", "--dest", str(backup_dest), "--keep", "0") + + assert second.exit_code == 0, second.output + payload = stdout_json(second) + second_file = Path(payload["backupFile"]) + assert str(first_file) in payload["pruned"] + assert not first_file.exists() + assert second_file.exists() + assert sorted(backup_dest.glob("openblade-*.db.gz")) == [second_file] + + +class TestRestoreBackupStaging: + def test_stages_the_decompressed_backup_next_to_the_live_db( + self, cli_home: Path, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + # Regression: the decompression workdir used to be created in the + # system temp dir (plain `tempfile.mkdtemp()`), which is frequently a + # different filesystem/mount from the live catalog's directory (e.g. + # the container's writable layer vs. a `/data` volume). The final + # `os.replace()` is only atomic -- or even possible -- when both + # paths share a filesystem, so the workdir must be created WITH + # `dir=`. This would still pass on a dev machine + # where both happen to be under /tmp, so we assert the call's `dir` + # kwarg directly rather than relying on an actual cross-device mount. + import tempfile as tempfile_module + + db_path, _canary_id = _init_live_db(cli_home) + backup_dest = tmp_path / "backups" + backup = invoke("catalog", "backup", "--dest", str(backup_dest)) + backup_file = Path(stdout_json(backup)["backupFile"]) + + seen_dirs: list[str | None] = [] + real_mkdtemp = tempfile_module.mkdtemp + + def spy_mkdtemp(*args: object, **kwargs: object) -> str: + seen_dirs.append(kwargs.get("dir")) # type: ignore[arg-type] + return real_mkdtemp(*args, **kwargs) # type: ignore[arg-type] + + monkeypatch.setattr(tempfile_module, "mkdtemp", spy_mkdtemp) + + result = invoke( + "catalog", "restore-backup", str(backup_file), "--confirm-db-path", str(db_path) + ) + + assert result.exit_code == 0, result.output + assert str(db_path.parent) in seen_dirs + + +class TestRestoreBackupGuards: + def test_refuses_when_confirm_path_does_not_match_live_db( + self, cli_home: Path, tmp_path: Path + ) -> None: + db_path, canary_id = _init_live_db(cli_home) + backup_dest = tmp_path / "backups" + backup = invoke("catalog", "backup", "--dest", str(backup_dest)) + backup_file = Path(stdout_json(backup)["backupFile"]) + + result = invoke( + "catalog", + "restore-backup", + str(backup_file), + "--confirm-db-path", + str(tmp_path / "not-the-live-db.db"), + ) + + assert result.exit_code != 0 + assert "Refusing restore" in result.output + _assert_canary_untouched(db_path, canary_id) + + def test_refuses_while_a_job_is_pending_or_running( + self, cli_home: Path, tmp_path: Path + ) -> None: + db_path, canary_id = _init_live_db(cli_home) + with sqlite3.connect(str(db_path)) as conn: + conn.execute( + "INSERT INTO jobs (id, job_type, state, metadata_json, error, " + "created_at, updated_at) VALUES ('still-running', 'restore', " + "'running', '{}', NULL, datetime('now'), datetime('now'))" + ) + conn.commit() + + backup_dest = tmp_path / "backups" + backup = invoke("catalog", "backup", "--dest", str(backup_dest)) + backup_file = Path(stdout_json(backup)["backupFile"]) + + result = invoke( + "catalog", "restore-backup", str(backup_file), "--confirm-db-path", str(db_path) + ) + + assert result.exit_code != 0 + assert "Refusing restore" in result.output + assert "running" in result.output + _assert_canary_untouched(db_path, canary_id) + with sqlite3.connect(str(db_path)) as conn: + state = conn.execute("SELECT state FROM jobs WHERE id = 'still-running'").fetchone() + assert state == ("running",) + + def test_refuses_a_corrupt_gzip_before_touching_the_live_db( + self, cli_home: Path, tmp_path: Path + ) -> None: + db_path, canary_id = _init_live_db(cli_home) + + bad_backup = tmp_path / "corrupt.db.gz" + bad_backup.write_bytes(b"not actually gzip") + + result = invoke( + "catalog", "restore-backup", str(bad_backup), "--confirm-db-path", str(db_path) + ) + + assert result.exit_code != 0 + assert "Corrupt backup file" in result.output + _assert_canary_untouched(db_path, canary_id) + + def test_refuses_a_backup_that_fails_integrity_verification( + self, cli_home: Path, tmp_path: Path + ) -> None: + db_path, canary_id = _init_live_db(cli_home) + + # A well-formed gzip stream around bytes that are not a valid SQLite + # database at all -- distinct from the "not gzip" corruption case above. + bad_backup = tmp_path / "bad-content.db.gz" + with gzip.open(bad_backup, "wb") as gz: + gz.write(b"this is not a sqlite database" * 10) + + result = invoke( + "catalog", "restore-backup", str(bad_backup), "--confirm-db-path", str(db_path) + ) + + assert result.exit_code != 0 + assert "failed integrity verification" in result.output + _assert_canary_untouched(db_path, canary_id) From 76b54fe8cadd3b971c531bbaf1af34e402a187f1 Mon Sep 17 00:00:00 2001 From: Amantux Date: Sat, 26 Sep 2026 06:43:07 +0000 Subject: [PATCH 41/54] feat(release): expose openblade --version and add tagged-release CI - `openblade --version` reads the single-sourced pyproject [project].version via importlib.metadata (eager Typer/Click option, exits before any subcommand resolves). - .github/workflows/release.yml: on `v*` tag push, cross-checks the tag against pyproject's version (fails fast on mismatch), builds+pushes the controller image (root Dockerfile) to ghcr.io/amantux/openblade tagged {version,latest}, and cuts a GitHub Release with generated notes. Permissions are least-privilege and job-scoped (contents: read at the workflow level, contents: write + packages: write only on the release job), matching this repo's existing workflows. Actions are pinned to tags (@vN), matching ci.yml's own style. Dockerfile.web is not tracked on master, so no web image is built -- documented at the build step rather than invented. - docs/wiki/guides/releasing.md: how to cut a release, what CI does, how operators pull the image, and a first-run caveat (see report). This workflow has not been exercised by an actual tag push yet -- YAML parses, yamllint is clean, and every `run:` block passes `bash -n`, but the GHCR login/push and release-notes generation are only proven by a real `v*` tag. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_013S1qbQs31YSox1azQChqs2 --- .github/workflows/release.yml | 92 ++++++++++++++++++++++++++++++++++ docs/wiki/README.md | 1 + docs/wiki/guides/releasing.md | 71 ++++++++++++++++++++++++++ openblade/cli/main.py | 32 ++++++++++++ tests/unit/test_cli_version.py | 38 ++++++++++++++ 5 files changed, 234 insertions(+) create mode 100644 .github/workflows/release.yml create mode 100644 docs/wiki/guides/releasing.md create mode 100644 tests/unit/test_cli_version.py diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml new file mode 100644 index 0000000..da266b7 --- /dev/null +++ b/.github/workflows/release.yml @@ -0,0 +1,92 @@ +name: Release + +# Tag-triggered release: builds the controller image, pushes it to GHCR, and +# cuts a GitHub Release from the tag. Runs on `v*` tag pushes only -- it never +# runs on a branch push, so it cannot race the version-bump automation in +# other workflows. +# +# NOTE: this workflow has never been exercised by an actual tag push (no `v*` +# tag exists on this repo yet). The first real run is what proves the GHCR +# login/push and release-notes generation actually work end to end; until +# then, treat it as reviewed-but-unverified. See docs/wiki/guides/releasing.md +# for what to check if that first run fails, and keep changes to this file +# small and easy to bisect for exactly that reason. + +on: + push: + tags: + - "v*" + +concurrency: + group: release-${{ github.ref }} + cancel-in-progress: false + +# Workflow-wide default is read-only, matching every other workflow in this +# repo (ci.yml, operability.yml, emulator-change-gates.yml, ...). The one job +# that needs to push images/create a release grants exactly those extra +# scopes itself, rather than widening this default. +permissions: + contents: read + +jobs: + release: + name: build-and-release + runs-on: ubuntu-latest + permissions: + contents: write # to create the GitHub Release from the tag + packages: write # to push images to ghcr.io + steps: + - uses: actions/checkout@v5 + + - name: Derive version from the tag + id: version + run: | + tag="${GITHUB_REF#refs/tags/}" + version="${tag#v}" + echo "tag=${tag}" >> "$GITHUB_OUTPUT" + echo "version=${version}" >> "$GITHUB_OUTPUT" + + - name: Cross-check tag against pyproject [project] version + run: | + pyproject_version="$(python3 -c 'import tomllib; print(tomllib.load(open("pyproject.toml", "rb"))["project"]["version"])')" + if [ "$pyproject_version" != "${{ steps.version.outputs.version }}" ]; then + echo "::error::tag v${{ steps.version.outputs.version }} does not match pyproject version ${pyproject_version} -- bump pyproject.toml and re-tag" + exit 1 + fi + + - name: Log in to GHCR + uses: docker/login-action@v3 + with: + registry: ghcr.io + username: ${{ github.actor }} + password: ${{ secrets.GITHUB_TOKEN }} + + - name: Set up Docker Buildx + uses: docker/setup-buildx-action@v3 + + # Controller image: the root Dockerfile (FastAPI/Typer app). This is the + # only image this workflow builds -- see the Dockerfile.web note below. + - name: Build and push controller image + uses: docker/build-push-action@v6 + with: + context: . + file: Dockerfile + push: true + tags: | + ghcr.io/amantux/openblade:${{ steps.version.outputs.version }} + ghcr.io/amantux/openblade:latest + + # `Dockerfile.web` is not tracked on master as of this workflow's + # authoring (only `frontend/Dockerfile` exists, consumed by + # docker-compose's `web` service, not published standalone to GHCR). + # If/when a top-level `Dockerfile.web` is added for a standalone web + # image, add a matching build-push step here tagged + # `ghcr.io/amantux/openblade-web:{version,latest}` -- do not invent one + # now. + + - name: Create GitHub Release + uses: softprops/action-gh-release@v2 + with: + tag_name: ${{ steps.version.outputs.tag }} + name: ${{ steps.version.outputs.tag }} + generate_release_notes: true diff --git a/docs/wiki/README.md b/docs/wiki/README.md index 9d71b74..4121d6d 100644 --- a/docs/wiki/README.md +++ b/docs/wiki/README.md @@ -52,6 +52,7 @@ Something is broken → **[Troubleshooting](guides/troubleshooting.md)**. | [Safety model](guides/safety-model.md) | The eight gates, condensed — with each one marked by where it is *actually* enforced | | [API authentication](guides/api-authentication.md) | The native bearer token: why it is off by default, the token file, what it does **not** cover, and rolling it out without an outage | | [FUSE & NAS namespace](guides/fuse-and-nas.md) | What is mountable today (nothing), what the namespace APIs do, the three hydration implementations | +| [Releasing](guides/releasing.md) | Cutting a tagged release: version bump, tag, what CI builds and publishes, how operators pull the image | ### Hardware diff --git a/docs/wiki/guides/releasing.md b/docs/wiki/guides/releasing.md new file mode 100644 index 0000000..692f23a --- /dev/null +++ b/docs/wiki/guides/releasing.md @@ -0,0 +1,71 @@ +# Releasing + +Version is single-sourced in `pyproject.toml`'s `[project].version`. There is +no auto-bump automation in this repo (unlike the sibling-repo playbook some +teams use) -- bumping and tagging are both manual, deliberate steps. + +## Cut a release + +1. **Bump the version.** Edit `version = "X.Y.Z"` in `pyproject.toml`. Nothing + else needs to change in lockstep (there is no HACS `manifest.json` or + similar to keep in sync in this repo). +2. **Commit and merge to `master`** through the normal PR/CI path. `make all` + (or CI's `backend-lint`/`backend-test` jobs) must be green before tagging -- + the release workflow does not re-run the test suite. +3. **Tag and push the tag:** + ```bash + git tag vX.Y.Z # e.g. v0.2.0 -- MUST match pyproject's version exactly + git push origin vX.Y.Z + ``` + The `v` prefix is required: `.github/workflows/release.yml` only triggers + on `v*` tag pushes, and its first step fails the build if the tag's version + doesn't match `pyproject.toml`'s -- a mismatched tag is rejected rather than + silently publishing the wrong version. + +## What CI does + +`.github/workflows/release.yml` runs on the tag push: + +1. Checks out the tagged commit. +2. Cross-checks the tag against `pyproject.toml`'s version (fails fast on a + mismatch, before anything is built or pushed). +3. Builds the controller image from the root `Dockerfile` and pushes it to + `ghcr.io/amantux/openblade`, tagged both `` and `latest`. +4. Creates a GitHub Release from the tag with auto-generated release notes + (commits since the previous tag). + +It does **not** build a separate web image: this repo has `frontend/Dockerfile` +(consumed by `docker-compose.yml`'s `web` service for local dev) but no +top-level `Dockerfile.web`, so there is nothing to publish standalone yet. If +one is added later, extend the workflow to publish +`ghcr.io/amantux/openblade-web` alongside the controller image. + +Permissions are least-privilege and job-scoped: the workflow's default is +`contents: read` (matching every other workflow in this repo); only the +`release` job grants `contents: write` (to create the Release) and +`packages: write` (to push to GHCR). + +## How operators pull the image + +```bash +docker pull ghcr.io/amantux/openblade:X.Y.Z # a specific release +docker pull ghcr.io/amantux/openblade:latest # the most recent tagged release +``` + +`docker-compose.yml`'s `api` service builds the image locally from `Dockerfile` +by default; point `image:` at the GHCR tag instead of `build:` to run a +published release without a local build, the same way the `emulator` service +already does for the published i3 emulator image. + +## First-run caveat + +This workflow has never been exercised by an actual `v*` tag push -- it is +reviewed (YAML parses, `bash -n` on every `run:` block, permissions scoped +correctly) but not yet proven end to end. If the first real tag push fails, +check in this order: the GHCR login step (repo's Actions have `packages: +write` by default, but organization-level policies can still block push -- +check **Settings → Actions → General → Workflow permissions**), the +`docker/build-push-action` step's build logs for a Dockerfile error unrelated +to this workflow, and finally whether `softprops/action-gh-release` needed +`contents: write` that wasn't granted (it is, in the job's `permissions:` +block, but a branch-protection rule on tags could still interfere). diff --git a/openblade/cli/main.py b/openblade/cli/main.py index e738169..370c62e 100644 --- a/openblade/cli/main.py +++ b/openblade/cli/main.py @@ -9,6 +9,8 @@ import shutil import tempfile from datetime import datetime +from importlib.metadata import PackageNotFoundError +from importlib.metadata import version as _pkg_version from pathlib import Path, PurePosixPath import typer @@ -47,7 +49,37 @@ from openblade.simulator.library import MockLibraryBackend from openblade.simulator.ltfs_volume import MockFileRecord, MockLTFSBackend, MockTapeContents + +def _version_callback(value: bool) -> None: + if not value: + return + try: + typer.echo(_pkg_version("openblade")) + except PackageNotFoundError: + # Running from a checkout that was never `pip install -e .`'d — the + # console-script entry point wouldn't exist either in that case, but + # `python -m openblade.cli.main` still reaches here. + typer.echo("unknown (package not installed)") + raise typer.Exit() + + app = typer.Typer(name="openblade", help="OpenBlade tape archive controller") + + +@app.callback() +def _main( + version: bool = typer.Option( + False, + "--version", + callback=_version_callback, + is_eager=True, + help="Print the installed openblade version and exit.", + ), +) -> None: + """OpenBlade tape archive controller.""" + return None + + mock_app = typer.Typer(help="Mock library commands") app.add_typer(mock_app, name="mock") format_app = typer.Typer(help="Format commands") diff --git a/tests/unit/test_cli_version.py b/tests/unit/test_cli_version.py new file mode 100644 index 0000000..adf2234 --- /dev/null +++ b/tests/unit/test_cli_version.py @@ -0,0 +1,38 @@ +"""`openblade --version`. + +Version is single-sourced in pyproject.toml's [project].version; the CLI +reads it back via importlib.metadata rather than duplicating the string. +""" + +from __future__ import annotations + +import tomllib +from pathlib import Path + +from typer.testing import CliRunner + +from openblade.cli import main as cli_main + +runner = CliRunner() + +REPO_ROOT = Path(__file__).resolve().parents[2] + + +def test_version_matches_pyproject() -> None: + pyproject = tomllib.loads((REPO_ROOT / "pyproject.toml").read_text()) + expected = pyproject["project"]["version"] + + result = runner.invoke(cli_main.app, ["--version"], catch_exceptions=False) + + assert result.exit_code == 0, result.output + assert result.output.strip() == expected + + +def test_version_is_eager_and_short_circuits_other_options() -> None: + # --version must win even when paired with a nonsense subcommand: it is an + # is_eager callback, so it must fire (and exit 0) before typer tries to + # resolve "not-a-real-command" as a subcommand. + result = runner.invoke( + cli_main.app, ["--version", "not-a-real-command"], catch_exceptions=False + ) + assert result.exit_code == 0, result.output From b5d5fc371a8e85c588d8bf714adc0298fa1743ef Mon Sep 17 00:00:00 2001 From: Hermes Agent Date: Sat, 26 Sep 2026 06:53:17 +0000 Subject: [PATCH 42/54] fix(web_flask): close the findings from the adversarial review round The T3 reviewer returned BLOCK with four MUST-FIX items, two carrying working repros against live sockets. All are fixed here with mutation-checked tests; the suite goes 37 -> 73. 1. SSRF: the device-URL deny-list compared host SPELLINGS, so "127.1", "2130706433" and "localhost." all reached a loopback listener (proven), and any attacker-controlled DNS name could resolve into private space. Now resolves the host and judges every returned address. Deliberately does NOT block RFC1918: the reviewer proposed is_private, but a real Scalar i3 lives on the operator's LAN, so that would refuse every legitimate device. Blocks loopback, link-local (covers 169.254.169.254), unspecified, multicast and reserved. Fails closed on an unresolvable name. 2. Probe followed redirects. The vetted URL was only the first hop: a host answering "302 Location: http://127.0.0.1/..." turned a blocked target into a reachable one, and httpx replays the body on 307/308, so the submitted device password could be harvested by an arbitrary host. follow_redirects=False, and a 3xx no longer counts as reachable or as a successful auth. 3. Path confinement. _is_safe_storage_path blocked traversal but not WHICH absolute roots were reachable, and those values become real filesystem sinks in a backend whose /archive and /restore routes have no per-user authorization -- so /proc/self/environ, which holds OPENBLADE_API_TOKEN, was archivable to tape. Added _is_allowed_storage_root, applied to the four genuine filesystem paths only. catalog_path and folder_path deliberately keep the traversal-only check: the former is a catalog key resolved in the DB, the latter is share-relative and matching the backend's own validator. Over-applying it would have broken legitimate config. 4. Confused deputy. Because the client presents the instance-wide OPENBLADE_API_TOKEN as the native bearer, and the native config surface (/nas/policies, /nas/shares, /nas/pools, /nas/cache-drives, /nas/source-stream) has no require_auth, any session could rewrite storage policy regardless of AML role. Added a role gate on the six config-write routes, resolved from /aml/users/me at login and failing closed on an unknown role. Also from the same round: reject dot-only identifiers (_POOL_ID_ALLOWED permitted ".", so ".." matched and collapsed a URL segment); validate volume_group, the one field reaching the backend unchecked; stop echoing non-JSON upstream bodies into the browser; treat 3xx as an error rather than crashing on .json(); cap the pre-auth login bucket and add an IP-only counter so spraying one password across many usernames is throttled; revoke the AML session on logout; reject backslash in redirect targets instead of relying on Werkzeug's escaping; constrain the reflected Host used in copy-paste mount commands; and stop bare int() on backend values from 500ing a page. A native-token mismatch now aborts 502 naming the remedy instead of clearing the session -- the dual-credential scheme made that 401 indistinguishable from session expiry, which produced an unbreakable login loop. Every guard was mutation-checked: 13 of 13 produce a failing test when removed. Two of the first drafts were vacuous and were rewritten until they failed. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_013S1qbQs31YSox1azQChqs2 --- README.md | 5 +- openblade/web_flask/app.py | 464 +++++++++++++++++++++++++++---- openblade/web_flask/client.py | 83 +++++- openblade/web_flask/models.py | 21 +- tests/unit/test_web_flask_app.py | 407 ++++++++++++++++++++++++++- 5 files changed, 911 insertions(+), 69 deletions(-) diff --git a/README.md b/README.md index fc871b0..5755345 100644 --- a/README.md +++ b/README.md @@ -92,7 +92,10 @@ Surfaces, each backed by existing API endpoints: | `OPENBLADE_API_TOKEN` | Must match the backend's value. The backend gates its native surface (`/api`, `/jobs`, `/nas`, `/catalog`, `/status`, `/system`, `/volume-groups`, `/archive`, `/restore`) on this bearer, while `/aml/*` uses the per-user session this UI obtains at login. Omit it only where the backend also leaves native auth off. | | `OPENBLADE_SERVICE_TOKEN` | Must match the backend's value; `/aml/mount` and `/aml/unmount` require it. | | `OPENBLADE_WEB_SECURE_COOKIES` | Forces `Secure` session cookies; defaults on under `OPENBLADE_ENV=production`. | -| `OPENBLADE_WEB_ALLOW_UNSAFE_DEVICE_TARGETS` | Development escape hatch: permits loopback/link-local/metadata hosts as device connection URLs, which are refused by default. Leave unset in production. | +| `OPENBLADE_WEB_ALLOW_UNSAFE_DEVICE_TARGETS` | Development escape hatch: permits loopback, link-local (including cloud metadata) and multicast device connection URLs. By default these are refused based on the host's **resolved addresses**, so respellings like `127.1` or `2130706433` are refused too. RFC1918 LAN addresses are always allowed — that is where real libraries live. Leave unset in production. | +| `OPENBLADE_WEB_ALLOWED_PATH_ROOTS` | Comma-separated absolute roots that operator-supplied archive/restore/cache paths must sit under. Defaults to `/openblade,/var/lib/openblade,/pools,/data,/share,/shares,/datasets,/mnt,/srv`. This is a confinement boundary, not just a traversal check: the backend's `/archive` and `/restore` routes have no per-user authorization, so without it a logged-in operator could archive `/etc` or `/proc/self/environ` to tape. | +| `OPENBLADE_CONNECT_HOST` | Pins the hostname printed in the copy-paste `net use` / `mount -t cifs` / `sftp` hints. Otherwise the (character-validated) request `Host` header is used — pin this wherever that header is not trustworthy. | +| `OPENBLADE_WEB_TIMEOUT_SECONDS` | Backend HTTP timeout, default 8. | ## Safety defaults - Mock backend is the default diff --git a/openblade/web_flask/app.py b/openblade/web_flask/app.py index 2df70c2..53d7396 100644 --- a/openblade/web_flask/app.py +++ b/openblade/web_flask/app.py @@ -4,11 +4,12 @@ import os import re import secrets +import socket import time from collections.abc import Callable, Iterator from dataclasses import asdict from datetime import timedelta -from ipaddress import ip_address +from ipaddress import IPv4Address, IPv6Address, ip_address from pathlib import PurePosixPath from urllib.parse import urlparse @@ -69,6 +70,9 @@ _BARCODE_TOKEN = re.compile(r"^[A-Za-z0-9._-]{1,64}$") _POOL_ID_ALLOWED = re.compile(r"^[A-Za-z0-9._:-]{1,64}$") _SHARE_NAME_ALLOWED = re.compile(r"^[A-Za-z0-9._-]{1,64}$") +#: Hostname / IPv4 / bracketless IPv6 characters only. Guards the host that gets +#: printed into copy-paste mount commands (see _connect_host). +_CONNECT_HOST_ALLOWED = re.compile(r"^[A-Za-z0-9.:_-]{1,253}$") _JOB_TYPES = ("archive", "restore", "format", "verify", "inventory", "import", "export") _JOB_STATES = ("pending", "running", "completed", "failed", "failed_recoverable", "cancelled") _AML_ROLE_OPTIONS: tuple[tuple[int, str], ...] = ( @@ -76,6 +80,11 @@ (1, "Operator"), (2, "Service"), ) +#: AML role 0 is Administrator (aml_state / routes_aml_auth use the same numbering). +_AML_ROLE_ADMIN = 0 +#: Sentinel for "we could not determine the role". Deliberately NOT 0, so a failed +#: /aml/users/me lookup denies writes instead of silently granting admin. +_AML_ROLE_UNKNOWN = -1 _POLICY_TYPE_OPTIONS = ("balanced", "critical_sequential", "noncritical_sharded") _INGEST_MODE_OPTIONS = ("cache_drive", "source_stream") _SHARD_STRATEGY_OPTIONS = ( @@ -90,9 +99,41 @@ _SHARE_TYPE_OPTIONS = ("inbox", "restore", "catalog", "virtual", "pool") _SHARE_ACCESS_OPTIONS = ("read_only", "read_write") _MAX_PATH_LENGTH = 512 +#: Absolute roots an operator-supplied archive/restore/cache path may live under. +#: Chosen from the paths this repo actually uses (`/openblade/inbox`, +#: `/openblade/restore`, `/openblade/cache`, `/var/lib/openblade`) plus the +#: conventional mount points a NAS deployment adds. The point is what is EXCLUDED: +#: `/etc`, `/root`, `/proc`, `/sys`, `/usr` and `/` itself. Override with +#: OPENBLADE_WEB_ALLOWED_PATH_ROOTS. +_DEFAULT_ALLOWED_PATH_ROOTS: tuple[str, ...] = ( + "/openblade", + "/var/lib/openblade", + # Generated by this app: devices_create_pool_share maps a pool to /pools/. + "/pools", + "/data", + "/share", + "/shares", + "/datasets", + "/mnt", + "/srv", +) _LOGIN_WINDOW_SECONDS = 300 _LOGIN_MAX_ATTEMPTS = 8 +#: Allowance for the IP-only key. Higher than the per-account limit because one IP +#: can legitimately be a whole office behind NAT, but still finite so that spraying +#: one password across many usernames is throttled. +_LOGIN_MAX_ATTEMPTS_PER_IP = 30 _LOGIN_LOCK_SECONDS = 120 +#: Hard cap on distinct login-attempt keys held in memory. The bucket is written +#: pre-auth and keyed partly by a caller-supplied username, so it needs a ceiling. +#: Generous enough that no real deployment evicts a live lockout. +_LOGIN_BUCKET_MAX_KEYS = 2048 +# Hostnames that must never be probed, as a courtesy fast-path only. This set is +# NOT the SSRF defence -- it cannot be, because a deny-list of spellings is +# trivially defeated ("127.1", "2130706433", "localhost.", "0x7f.1" all reach +# loopback). The real check is _resolved_addresses_are_safe(), which resolves the +# name and inspects every address ipaddress reports. Keep both: the names give a +# clearer error message for the obvious cases. _DANGEROUS_DEVICE_HOSTS = { "localhost", "127.0.0.1", @@ -178,45 +219,155 @@ def _client_ip() -> str: def _is_safe_redirect_target(target: str) -> bool: + # Backslash is rejected explicitly: urlparse does not treat "/\evil.example" as + # having a netloc, so without this the only thing preventing an off-site + # redirect is Werkzeug percent-encoding the backslash in the Location header. + # That is a load-bearing dependency on someone else's escaping. if not target or not target.startswith("/") or target.startswith("//"): return False + if "\\" in target: + return False if any(ord(char) < 0x20 for char in target): return False parsed = urlparse(target) return not (parsed.scheme or parsed.netloc) +def _is_unsafe_ip(value: IPv4Address | IPv6Address) -> bool: + """Addresses a device probe must never be pointed at. + + Deliberately does NOT include ``is_private``. RFC1918 space is where real tape + libraries live -- a Scalar i3 on the operator's LAN is 10.x/172.16.x/192.168.x + -- so blocking it would refuse every legitimate device and defeat the point of + the app. This matches the original intent of ``_DANGEROUS_DEVICE_HOSTS``, which + listed loopback/link-local/metadata and never private ranges; the fix here is + to apply that intent to RESOLVED addresses rather than to spellings. + + What IS blocked: loopback (the web container and the API itself), link-local + (which covers the 169.254.169.254 cloud metadata endpoint), unspecified, + multicast and reserved. Operators who genuinely need a loopback target set + OPENBLADE_WEB_ALLOW_UNSAFE_DEVICE_TARGETS. + """ + return bool( + value.is_loopback + or value.is_link_local + or value.is_unspecified + or value.is_multicast + or value.is_reserved + ) + + +def _resolve_device_host(hostname: str) -> list[IPv4Address | IPv6Address]: + """Every address ``hostname`` resolves to, or [] if it does not resolve. + + Separated out so tests can monkeypatch the resolver instead of depending on + live DNS. + """ + try: + infos = socket.getaddrinfo(hostname, None, proto=socket.IPPROTO_TCP) + except (socket.gaierror, UnicodeError, ValueError): + return [] + resolved: list[IPv4Address | IPv6Address] = [] + for info in infos: + sockaddr = info[4] + if not sockaddr: + continue + try: + resolved.append(ip_address(str(sockaddr[0]))) + except ValueError: + continue + return resolved + + def _validate_device_url(connection_url: str) -> str | None: + """Reject device connection URLs that could be used for SSRF. + + The decisive check is on RESOLVED ADDRESSES, not on the spelling of the host. + A deny-list of names cannot work: ``127.1``, ``2130706433``, ``0x7f.1`` and + ``localhost.`` all reach loopback while matching no entry in + ``_DANGEROUS_DEVICE_HOSTS``, and any attacker-controlled DNS name can simply + resolve to a private address. Verified: before this check, probing + ``http://2130706433:PORT`` reached a loopback listener. + + Fails CLOSED -- a name that does not resolve is refused rather than probed, + because "unresolvable" and "resolvable only inside the container network" are + indistinguishable from here. + """ parsed_url = urlparse(connection_url) if parsed_url.scheme not in {"http", "https"} or not parsed_url.netloc: return "Connection URL must start with http:// or https://" if parsed_url.username or parsed_url.password: return "Connection URL must not include embedded credentials." - hostname = (parsed_url.hostname or "").strip().lower() + try: + hostname = (parsed_url.hostname or "").strip().lower() + except ValueError: + return "Connection URL host could not be parsed." if not hostname: return "Connection URL must include a hostname." - allow_unsafe_targets = _bool_env("OPENBLADE_WEB_ALLOW_UNSAFE_DEVICE_TARGETS", default=False) - if not allow_unsafe_targets and hostname in _DANGEROUS_DEVICE_HOSTS: + if _bool_env("OPENBLADE_WEB_ALLOW_UNSAFE_DEVICE_TARGETS", default=False): + return None + if hostname in _DANGEROUS_DEVICE_HOSTS: return "Connection URL host is not allowed by web security policy." + # A literal IP needs no resolution; check it directly so a non-resolving + # literal is still judged on its value. try: - ip_value = ip_address(hostname) + literal = ip_address(hostname) except ValueError: - ip_value = None - if ( - ip_value is not None - and not allow_unsafe_targets - and ( - ip_value.is_loopback - or ip_value.is_link_local - or ip_value.is_unspecified - or ip_value.is_multicast - ) - ): + literal = None + if literal is not None: + if _is_unsafe_ip(literal): + return "Connection URL host is not allowed by web security policy." + return None + resolved = _resolve_device_host(hostname) + if not resolved: + return "Connection URL host could not be resolved." + if any(_is_unsafe_ip(address) for address in resolved): return "Connection URL host is not allowed by web security policy." return None +def _coerce_int(value: object, *, default: int = 0) -> int: + """int() a backend-supplied value without letting it 500 the page. + + The emulator and API are separate processes, so a field that is normally a + number can arrive as None, "", or a string. A bare int() on those raises and + turns a rendering path into a 500 -- a repeat of a known pattern in this repo. + """ + if isinstance(value, bool): + return int(value) + if isinstance(value, int): + return value + try: + return int(str(value).strip()) + except (TypeError, ValueError): + return default + + +def _is_safe_identifier(value: str, pattern: re.Pattern[str]) -> bool: + """Allowlist check for a value that becomes a URL path segment or path component. + + The alphabet checks alone were not enough: ``_POOL_ID_ALLOWED`` and + ``_OPERATION_TOKEN`` both permit ``.``, so ``..`` matched them. Those values are + interpolated into backend URLs (``/nas/datasets/{id}/verify``, + ``/aml/magazine/{id}/eject``) and into a filesystem path (``/pools/{pool_id}``), + and both httpx and POSIX normalisation collapse a ``..`` segment -- verified: + ``POST /storage/datasets/../verify`` rewrote the backend URL to ``/nas/verify``. + + So reject any value that is made up entirely of dots. That is exactly the set + normalisation treats specially; an interior dot (``pool.1``) stays legal. + """ + if not value or value.strip(".") == "": + return False + return pattern.fullmatch(value) is not None + + def _is_safe_storage_path(path: str) -> bool: + """Absolute, bounded, NUL-free, and free of ``..`` components. + + NOTE the deliberate limit: this is a *traversal* check, not a *confinement* + check. It does not restrict which absolute roots are reachable -- see + ``_is_allowed_storage_root`` for that, which is applied alongside it. + """ if not path or len(path) > _MAX_PATH_LENGTH or "\x00" in path: return False if not path.startswith("/"): @@ -224,6 +375,39 @@ def _is_safe_storage_path(path: str) -> bool: return ".." not in PurePosixPath(path).parts +def _allowed_storage_roots() -> tuple[str, ...]: + configured = os.environ.get("OPENBLADE_WEB_ALLOWED_PATH_ROOTS", "").strip() + if not configured: + return _DEFAULT_ALLOWED_PATH_ROOTS + roots = tuple(item.strip() for item in configured.split(",") if item.strip().startswith("/")) + return roots or _DEFAULT_ALLOWED_PATH_ROOTS + + +def _is_allowed_storage_root(path: str) -> bool: + """Confine operator-supplied paths to roots an archive/restore may touch. + + ``_is_safe_storage_path`` blocks traversal but says nothing about WHICH absolute + paths are legal, and the values flow to real filesystem sinks in the API + container: ``source_path`` reaches ``Path(...).rglob("*")`` in routes_archive, + ``dest_path`` reaches an LTFS read/rename in jobs/restore, ``root_path`` becomes + a cache-drive root. Those backend routes carry no per-user authorization, so + without a confinement check a logged-in operator could archive ``/root/.ssh`` or + ``/proc/self/environ`` -- which holds OPENBLADE_API_TOKEN and + OPENBLADE_SERVICE_TOKEN -- straight onto tape. + + Override with OPENBLADE_WEB_ALLOWED_PATH_ROOTS (comma-separated absolute + paths) when a deployment stores data elsewhere. + """ + if not _is_safe_storage_path(path): + return False + candidate = PurePosixPath(path) + for root in _allowed_storage_roots(): + root_path = PurePosixPath(root) + if candidate == root_path or candidate.is_relative_to(root_path): + return True + return False + + def _attempt_bucket(app: Flask) -> dict[str, dict[str, object]]: bucket = app.extensions.setdefault("login_attempts", {}) if isinstance(bucket, dict): @@ -231,11 +415,52 @@ def _attempt_bucket(app: Flask) -> dict[str, dict[str, object]]: raise RuntimeError("login attempt bucket is misconfigured") -def _record_login_failure(app: Flask, key: str) -> None: +def _evict_oldest_unblocked(bucket: dict[str, dict[str, object]], now: float) -> None: + """Make room in the login-attempt bucket without releasing an active lockout. + + Preference order: an entry whose lock has expired, oldest last-attempt first. + Only if every entry is actively blocked do we drop the one expiring soonest -- + at that point the bucket is full of real lockouts, and refusing to record new + ones would be the worse failure. + """ + oldest_key: str | None = None + oldest_marker = float("inf") + fallback_key: str | None = None + fallback_expiry = float("inf") + for candidate_key, entry in bucket.items(): + blocked_until = entry.get("blocked_until") + blocked_until = blocked_until if isinstance(blocked_until, float) else 0.0 + attempts = entry.get("attempts") + last_attempt = ( + max((item for item in attempts if isinstance(item, float)), default=0.0) + if isinstance(attempts, list) + else 0.0 + ) + if blocked_until <= now: + if last_attempt < oldest_marker: + oldest_marker = last_attempt + oldest_key = candidate_key + elif blocked_until < fallback_expiry: + fallback_expiry = blocked_until + fallback_key = candidate_key + victim = oldest_key or fallback_key + if victim is not None: + bucket.pop(victim, None) + + +def _record_login_failure(app: Flask, key: str, *, max_attempts: int | None = None) -> None: now = time.monotonic() bucket = _attempt_bucket(app) entry = bucket.get(key) if not isinstance(entry, dict): + # Cap the bucket. It is written on PRE-AUTH requests, keyed partly by a + # caller-supplied username, so without a cap a spray of unique usernames + # grows it without bound -- and because _is_login_blocked sweeps the whole + # dict, every additional key also makes each subsequent attempt slower + # (measured: 50k keys -> ~13ms of scan per login). Evicting the oldest + # unblocked entry keeps active lockouts intact. + if len(bucket) >= _LOGIN_BUCKET_MAX_KEYS: + _evict_oldest_unblocked(bucket, now) entry = {"attempts": [], "blocked_until": 0.0} bucket[key] = entry attempts = entry.get("attempts") @@ -245,7 +470,8 @@ def _record_login_failure(app: Flask, key: str) -> None: attempts.append(now) window_start = now - _LOGIN_WINDOW_SECONDS attempts[:] = [item for item in attempts if isinstance(item, float) and item >= window_start] - if len(attempts) >= _LOGIN_MAX_ATTEMPTS: + threshold = _LOGIN_MAX_ATTEMPTS if max_attempts is None else max_attempts + if len(attempts) >= threshold: entry["blocked_until"] = now + _LOGIN_LOCK_SECONDS @@ -353,11 +579,22 @@ def _share_type_label(share_type: str) -> str: def _connect_host() -> str: + """Host to print in the copy-paste `net use` / `mount -t cifs` / `sftp` hints. + + The fallback reflects the client-supplied Host header, and those hints are + commands we are telling an operator to run, so the value is constrained to a + hostname/IP alphabet: an unexpected character could otherwise extend the command + or the UNC path. Anything that does not match falls back to loopback. + + This bounds the damage but does not eliminate it -- a valid-looking attacker + hostname still renders. Pin OPENBLADE_CONNECT_HOST in any deployment where the + Host header is not trustworthy. + """ override = os.environ.get("OPENBLADE_CONNECT_HOST", "").strip() - if override: + if override and _CONNECT_HOST_ALLOWED.fullmatch(override): return override host = request.host.split(":", 1)[0].strip().lower() - if not host or host in {"0.0.0.0", "::"}: + if not host or host in {"0.0.0.0", "::"} or not _CONNECT_HOST_ALLOWED.fullmatch(host): return "127.0.0.1" return host @@ -659,8 +896,52 @@ def _token() -> str: abort(401) return token + def _is_admin() -> bool: + return _coerce_int(session.get("aml_role"), default=_AML_ROLE_UNKNOWN) == _AML_ROLE_ADMIN + + def _require_admin() -> ResponseReturnValue | None: + """Gate configuration writes on the AML Administrator role. + + This check exists HERE because the OpenBlade-native config surface has none + of its own: /nas/policies, /nas/shares, /nas/pools, /nas/cache-drives, + /nas/source-stream and /nas/datasets/{id}/verify carry no require_auth + dependency (verified against the route table). Since BackendClient presents + the instance-wide OPENBLADE_API_TOKEN as the native bearer, anything this UI + lets a session do, it does with full native authority -- so without a role + check any user who can log in, including role 1 (Operator) and role 2 + (Service), could rewrite storage policy. Fails closed on an unknown role. + """ + if _is_admin(): + return None + flash("That action requires an Administrator account.", "error") + return redirect(url_for("storage", section="overview")) + def _handle_backend_error(error: BackendError) -> ResponseReturnValue: if error.status_code == 401: + # Distinguish "your session expired" from "this deployment's native API + # token is wrong". Both arrive as 401, but clearing the session on the + # latter produces an unbreakable loop: login talks to /aml/* and + # succeeds, then the first native call 401s on the BEARER, bouncing the + # operator back to a login page that can never help. api_auth's curated + # detail names the bearer, so key on that, keep the session, and say what + # to fix. + detail = (error.detail or "").lower() + if "bearer" in detail or "api token" in detail: + app.logger.error( + "Backend rejected the native API token; OPENBLADE_API_TOKEN for " + "the web UI likely does not match the API's value" + ) + # abort(), NOT a redirect. Every native page makes the same call, so + # redirecting anywhere -- including devices_index -- just moves the + # loop. A terminal 502 with the remedy is the honest answer. + abort( + 502, + description=( + "The backend rejected this app's API token. Set " + "OPENBLADE_API_TOKEN for the web UI to the same value the API " + "uses, then reload." + ), + ) session.clear() return redirect(url_for("login")) flash(error.detail, "error") @@ -680,12 +961,20 @@ def login() -> ResponseReturnValue: username = request.form.get("username", "").strip() password = request.form.get("password", "") - login_key = f"{_client_ip()}:{username.lower()[:64]}" - if _is_login_blocked(app, login_key): + client_ip = _client_ip() + # TWO keys, both checked and both recorded. The per-account key stops + # brute-forcing one password list against one user; the IP-only key stops the + # mirror-image attack -- spraying ONE password across many usernames, which + # the per-account key never throttles because every attempt lands in a fresh + # bucket. The IP key gets a larger allowance since it aggregates a whole NAT. + login_key = f"{client_ip}:{username.lower()[:64]}" + ip_key = f"ip:{client_ip}" + if _is_login_blocked(app, login_key) or _is_login_blocked(app, ip_key): flash("Too many login attempts. Try again in a few minutes.", "error") return Response(render_template("login.html", next_path=next_path), status=429) if not username or not password: _record_login_failure(app, login_key) + _record_login_failure(app, ip_key, max_attempts=_LOGIN_MAX_ATTEMPTS_PER_IP) flash("Username and password are required.", "error") return Response(render_template("login.html", next_path=next_path), status=400) @@ -693,13 +982,25 @@ def login() -> ResponseReturnValue: token = _backend(app).login(username=username, password=password) except BackendError as error: _record_login_failure(app, login_key) - app.logger.warning("Login failed", extra={"username": username, "ip": _client_ip()}) + _record_login_failure(app, ip_key, max_attempts=_LOGIN_MAX_ATTEMPTS_PER_IP) + app.logger.warning("Login failed", extra={"username": username, "ip": client_ip}) flash(error.detail, "error") return Response(render_template("login.html", next_path=next_path), status=401) + # Resolve the role now and pin it in the session. It is not in the login + # response, and every write gate below depends on it, so a failure to + # determine it must fail CLOSED rather than default to admin. + try: + current = _backend(app).get_current_user(token) + except BackendError: + app.logger.warning("Could not resolve AML role after login", extra={"user": username}) + current = {} + role = _coerce_int(current.get("role"), default=_AML_ROLE_UNKNOWN) + session.clear() session["username"] = username session["api_token"] = token + session["aml_role"] = role session.permanent = True _ensure_csrf_token() _clear_login_failures(app, login_key) @@ -709,6 +1010,13 @@ def login() -> ResponseReturnValue: @app.post("/logout") def logout() -> ResponseReturnValue: + token = session.get("api_token") + if isinstance(token, str) and token: + try: + _backend(app).logout(token) + except BackendError: + # Never block the local logout on a backend failure. + app.logger.info("Backend session revocation failed during logout") session.clear() return redirect(url_for("login")) @@ -868,7 +1176,9 @@ def devices_move_media(device_id: int) -> ResponseReturnValue: return redirect( url_for("devices_manage_section", device_id=device_id, section="operations") ) - if not _OPERATION_TOKEN.fullmatch(source) or not _OPERATION_TOKEN.fullmatch(destination): + if not _is_safe_identifier(source, _OPERATION_TOKEN) or not _is_safe_identifier( + destination, _OPERATION_TOKEN + ): flash( "Source and destination accept letters, numbers, comma, dot, underscore, dash, and colon.", "error", @@ -911,14 +1221,14 @@ def devices_mount_media(device_id: int) -> ResponseReturnValue: return redirect( url_for("devices_manage_section", device_id=device_id, section="operations") ) - if not _OPERATION_TOKEN.fullmatch(drive): + if not _is_safe_identifier(drive, _OPERATION_TOKEN): flash( "Drive accepts letters, numbers, comma, dot, underscore, dash, and colon.", "error" ) return redirect( url_for("devices_manage_section", device_id=device_id, section="operations") ) - if partition and not _OPERATION_TOKEN.fullmatch(partition): + if partition and not _is_safe_identifier(partition, _OPERATION_TOKEN): flash( "Partition accepts letters, numbers, comma, dot, underscore, dash, and colon.", "error", @@ -955,7 +1265,7 @@ def devices_unmount_media(device_id: int) -> ResponseReturnValue: return redirect( url_for("devices_manage_section", device_id=device_id, section="operations") ) - if not _OPERATION_TOKEN.fullmatch(drive): + if not _is_safe_identifier(drive, _OPERATION_TOKEN): flash( "Drive accepts letters, numbers, comma, dot, underscore, dash, and colon.", "error" ) @@ -985,7 +1295,9 @@ def devices_import_media(device_id: int) -> ResponseReturnValue: return redirect( url_for("devices_manage_section", device_id=device_id, section="operations") ) - if not _OPERATION_TOKEN.fullmatch(partition) or not _OPERATION_TOKEN.fullmatch(ie_station): + if not _is_safe_identifier(partition, _OPERATION_TOKEN) or not _is_safe_identifier( + ie_station, _OPERATION_TOKEN + ): flash( "Partition and I/E station accept letters, numbers, comma, dot, underscore, dash, and colon.", "error", @@ -1017,7 +1329,7 @@ def devices_export_media(device_id: int) -> ResponseReturnValue: return redirect( url_for("devices_manage_section", device_id=device_id, section="operations") ) - if not _OPERATION_TOKEN.fullmatch(ie_station): + if not _is_safe_identifier(ie_station, _OPERATION_TOKEN): flash( "I/E station accepts letters, numbers, comma, dot, underscore, dash, and colon.", "error", @@ -1053,7 +1365,7 @@ def devices_restore_from_tape(device_id: int) -> ResponseReturnValue: return redirect( url_for("devices_manage_section", device_id=device_id, section="operations") ) - if not _is_safe_storage_path(catalog_path) or not _is_safe_storage_path(dest_path): + if not _is_safe_storage_path(catalog_path) or not _is_allowed_storage_root(dest_path): flash( "Catalog and destination paths must be absolute normalized paths within policy limits.", "error", @@ -1077,6 +1389,11 @@ def devices_restore_from_tape(device_id: int) -> ResponseReturnValue: @app.post("/devices//operations/pool") def devices_create_pool(device_id: int) -> ResponseReturnValue: + # Creates NAS pools/shares via /nas/pools and /nas/shares, which carry + # no backend authorization -- same reason as the /storage writes. + denied = _require_admin() + if denied is not None: + return denied pool_id = request.form.get("pool_id", "").strip() pool_name = request.form.get("pool_name", "").strip() description = request.form.get("description", "").strip() @@ -1095,7 +1412,7 @@ def devices_create_pool(device_id: int) -> ResponseReturnValue: "on", } - if not _POOL_ID_ALLOWED.fullmatch(pool_id): + if not _is_safe_identifier(pool_id, _POOL_ID_ALLOWED): flash("Pool ID accepts letters, numbers, dot, underscore, dash, and colon.", "error") return redirect( url_for("devices_manage_section", device_id=device_id, section="operations") @@ -1105,12 +1422,12 @@ def devices_create_pool(device_id: int) -> ResponseReturnValue: return redirect( url_for("devices_manage_section", device_id=device_id, section="operations") ) - if mount_path and not _is_safe_storage_path(mount_path): + if mount_path and not _is_allowed_storage_root(mount_path): flash("Pool mount path must be an absolute normalized path.", "error") return redirect( url_for("devices_manage_section", device_id=device_id, section="operations") ) - if not _is_safe_storage_path(restore_target_path): + if not _is_allowed_storage_root(restore_target_path): flash("Restore target path must be an absolute normalized path.", "error") return redirect( url_for("devices_manage_section", device_id=device_id, section="operations") @@ -1139,7 +1456,7 @@ def devices_create_pool(device_id: int) -> ResponseReturnValue: ) volume_group_ids = _parse_csv_tokens(volume_group_ids_raw) - if any(_POOL_ID_ALLOWED.fullmatch(item) is None for item in volume_group_ids): + if any(not _is_safe_identifier(item, _POOL_ID_ALLOWED) for item in volume_group_ids): flash( "Volume group IDs accept letters, numbers, dot, underscore, dash, and colon.", "error", @@ -1171,15 +1488,20 @@ def devices_create_pool(device_id: int) -> ResponseReturnValue: @app.post("/devices//operations/share") def devices_create_pool_share(device_id: int) -> ResponseReturnValue: + # Creates NAS pools/shares via /nas/pools and /nas/shares, which carry + # no backend authorization -- same reason as the /storage writes. + denied = _require_admin() + if denied is not None: + return denied pool_id = request.form.get("pool_id", "").strip() share_name = request.form.get("share_name", "").strip() writable = request.form.get("writable", "").strip().lower() in {"1", "true", "yes", "on"} - if not _POOL_ID_ALLOWED.fullmatch(pool_id): + if not _is_safe_identifier(pool_id, _POOL_ID_ALLOWED): flash("Pool ID accepts letters, numbers, dot, underscore, dash, and colon.", "error") return redirect( url_for("devices_manage_section", device_id=device_id, section="operations") ) - if not _SHARE_NAME_ALLOWED.fullmatch(share_name): + if not _is_safe_identifier(share_name, _SHARE_NAME_ALLOWED): flash("Share name accepts letters, numbers, dot, underscore, and dash.", "error") return redirect( url_for("devices_manage_section", device_id=device_id, section="operations") @@ -1203,7 +1525,7 @@ def devices_create_pool_share(device_id: int) -> ResponseReturnValue: @app.post("/devices//operations/magazine//eject") def devices_eject_magazine(device_id: int, magazine_id: str) -> ResponseReturnValue: - if not _OPERATION_TOKEN.fullmatch(magazine_id): + if not _is_safe_identifier(magazine_id, _OPERATION_TOKEN): flash("Magazine ID is invalid.", "error") return redirect( url_for("devices_manage_section", device_id=device_id, section="operations") @@ -1219,7 +1541,7 @@ def devices_eject_magazine(device_id: int, magazine_id: str) -> ResponseReturnVa @app.post("/devices//operations/magazine//insert") def devices_insert_magazine(device_id: int, magazine_id: str) -> ResponseReturnValue: - if not _OPERATION_TOKEN.fullmatch(magazine_id): + if not _is_safe_identifier(magazine_id, _OPERATION_TOKEN): flash("Magazine ID is invalid.", "error") return redirect( url_for("devices_manage_section", device_id=device_id, section="operations") @@ -1288,14 +1610,17 @@ def devices_manage_section(device_id: int, section: str) -> ResponseReturnValue: magazines_with_mapping: list[dict[str, object]] = [] for magazine in magazines: magazine_id = str(magazine.get("id") or "").strip() - if not magazine_id: + # Validated even though it came from the backend, not the user: + # it is interpolated into a URL path, and the emulator is a + # separate process whose payloads we do not control. + if not _is_safe_identifier(magazine_id, _OPERATION_TOKEN): continue slots = _backend(app).list_magazine_slots( _token(), library_id=device_id, magazine_id=magazine_id ) entry = dict(magazine) entry["mappedSlotCount"] = len(slots) - entry["slotMapAligned"] = int(magazine.get("slotCount") or 0) == len(slots) + entry["slotMapAligned"] = _coerce_int(magazine.get("slotCount")) == len(slots) magazines_with_mapping.append(entry) section_data["magazines"] = magazines_with_mapping section_data["catalog_files"] = [ @@ -1472,8 +1797,21 @@ def storage_archive_submit() -> ResponseReturnValue: if not source_path or not volume_group: flash("Source path and volume group are required.", "error") return redirect(url_for("storage", section="archive")) - if not _is_safe_storage_path(source_path): - flash("Source path must be an absolute normalized path within policy limits.", "error") + if not _is_allowed_storage_root(source_path): + flash( + "Source path must be an absolute normalized path under an allowed storage root.", + "error", + ) + return redirect(url_for("storage", section="archive")) + # volume_group was the one form field reaching the backend unvalidated. It + # becomes a catalog path component via PurePosixPath("/") / volume_group in + # jobs/archive.py, and PurePosixPath does NOT normalise, so "a/../b" would + # persist into catalog keys. + if not _is_safe_identifier(volume_group, _POOL_ID_ALLOWED): + flash( + "Volume group accepts letters, numbers, dot, underscore, colon, and dash.", + "error", + ) return redirect(url_for("storage", section="archive")) try: _backend(app).create_archive_job( @@ -1491,7 +1829,7 @@ def storage_restore_submit() -> ResponseReturnValue: if not catalog_path or not dest_path: flash("Catalog path and destination path are required.", "error") return redirect(url_for("storage", section="restore")) - if not _is_safe_storage_path(catalog_path) or not _is_safe_storage_path(dest_path): + if not _is_safe_storage_path(catalog_path) or not _is_allowed_storage_root(dest_path): flash( "Catalog and destination paths must be absolute normalized paths within policy limits.", "error", @@ -1509,7 +1847,7 @@ def storage_restore_submit() -> ResponseReturnValue: @app.post("/storage/datasets//verify") def storage_dataset_verify(dataset_id: str) -> ResponseReturnValue: normalized_dataset_id = dataset_id.strip() - if not _POOL_ID_ALLOWED.fullmatch(normalized_dataset_id): + if not _is_safe_identifier(normalized_dataset_id, _POOL_ID_ALLOWED): flash("Dataset ID is invalid.", "error") return redirect(url_for("storage", section="catalog")) try: @@ -1517,9 +1855,9 @@ def storage_dataset_verify(dataset_id: str) -> ResponseReturnValue: except BackendError as error: flash(error.detail, "error") return redirect(url_for("storage", section="catalog")) - files_verified = int(result.get("files_verified", 0)) - files_corrupt = int(result.get("files_corrupt", 0)) - files_updated = int(result.get("files_updated", 0)) + files_verified = _coerce_int(result.get("files_verified")) + files_corrupt = _coerce_int(result.get("files_corrupt")) + files_updated = _coerce_int(result.get("files_updated")) session["nas_verify_snapshot"] = { "dataset_id": normalized_dataset_id, "files_verified": files_verified, @@ -1541,6 +1879,10 @@ def storage_dataset_verify(dataset_id: str) -> ResponseReturnValue: @app.post("/storage/policies") def storage_policy_submit() -> ResponseReturnValue: + # /nas/... has no require_auth on the backend; this is the only gate. + denied = _require_admin() + if denied is not None: + return denied policy_id = request.form.get("policy_id", "").strip() name = request.form.get("name", "").strip() policy_type = request.form.get("policy_type", "balanced").strip().lower() @@ -1575,7 +1917,7 @@ def storage_policy_submit() -> ResponseReturnValue: "yes", "on", } - if not _POOL_ID_ALLOWED.fullmatch(policy_id): + if not _is_safe_identifier(policy_id, _POOL_ID_ALLOWED): flash("Policy ID accepts letters, numbers, dot, underscore, dash, and colon.", "error") return redirect(url_for("storage", section="write-path")) if not _ASCII_PRINTABLE.fullmatch(name): @@ -1640,6 +1982,10 @@ def storage_policy_submit() -> ResponseReturnValue: @app.post("/storage/cache-drives") def storage_cache_drive_submit() -> ResponseReturnValue: + # /nas/... has no require_auth on the backend; this is the only gate. + denied = _require_admin() + if denied is not None: + return denied drive_id = request.form.get("drive_id", "").strip() name = request.form.get("name", "").strip() root_path = request.form.get("root_path", "").strip() @@ -1661,7 +2007,7 @@ def storage_cache_drive_submit() -> ResponseReturnValue: "yes", "on", } - if not _POOL_ID_ALLOWED.fullmatch(drive_id): + if not _is_safe_identifier(drive_id, _POOL_ID_ALLOWED): flash( "Cache drive ID accepts letters, numbers, dot, underscore, dash, and colon.", "error", @@ -1670,7 +2016,7 @@ def storage_cache_drive_submit() -> ResponseReturnValue: if not _ASCII_PRINTABLE.fullmatch(name): flash("Cache drive name must be 1-64 printable ASCII characters.", "error") return redirect(url_for("storage", section="write-path")) - if not _is_safe_storage_path(root_path): + if not _is_allowed_storage_root(root_path): flash("Cache drive root path must be an absolute normalized path.", "error") return redirect(url_for("storage", section="write-path")) if eviction_policy not in _CACHE_EVICTION_OPTIONS: @@ -1720,6 +2066,10 @@ def storage_cache_drive_submit() -> ResponseReturnValue: @app.post("/storage/source-stream") def storage_source_stream_submit() -> ResponseReturnValue: + # /nas/... has no require_auth on the backend; this is the only gate. + denied = _require_admin() + if denied is not None: + return denied checksum_mode = ( request.form.get("checksum_mode", "precompute_and_post_verify").strip().lower() ) @@ -1784,6 +2134,10 @@ def storage_source_stream_submit() -> ResponseReturnValue: @app.post("/storage/shares") def storage_share_submit() -> ResponseReturnValue: + # /nas/... has no require_auth on the backend; this is the only gate. + denied = _require_admin() + if denied is not None: + return denied share_path = request.form.get("share_path", "").strip() share_name = request.form.get("share_name", "").strip() share_type = request.form.get("share_type", "pool").strip().lower() @@ -1792,22 +2146,22 @@ def storage_share_submit() -> ResponseReturnValue: folder_mappings_raw = request.form.get("folder_mappings", "").strip() writable = request.form.get("writable", "").strip().lower() in {"1", "true", "yes", "on"} - if not _is_safe_storage_path(share_path): + if not _is_allowed_storage_root(share_path): flash("Share path must be an absolute normalized path.", "error") return redirect(url_for("storage", section="write-path")) - if not _SHARE_NAME_ALLOWED.fullmatch(share_name): + if not _is_safe_identifier(share_name, _SHARE_NAME_ALLOWED): flash("Share name accepts letters, numbers, dot, underscore, and dash only.", "error") return redirect(url_for("storage", section="write-path")) if share_type not in _SHARE_TYPE_OPTIONS: flash("Share type is invalid.", "error") return redirect(url_for("storage", section="write-path")) - if default_policy_id and not _POOL_ID_ALLOWED.fullmatch(default_policy_id): + if default_policy_id and not _is_safe_identifier(default_policy_id, _POOL_ID_ALLOWED): flash("Default policy ID is invalid.", "error") return redirect(url_for("storage", section="write-path")) pool_ids = _parse_csv_tokens(pool_ids_raw) for pool_id in pool_ids: - if not _POOL_ID_ALLOWED.fullmatch(pool_id): + if not _is_safe_identifier(pool_id, _POOL_ID_ALLOWED): flash( "Pool IDs must contain only letters, numbers, dot, underscore, dash, and colon.", "error", @@ -1830,7 +2184,7 @@ def storage_share_submit() -> ResponseReturnValue: if not _is_safe_storage_path(folder_path): flash(f"Folder mapping line {line_number} has an invalid folder path.", "error") return redirect(url_for("storage", section="write-path")) - if not _POOL_ID_ALLOWED.fullmatch(pool_id): + if not _is_safe_identifier(pool_id, _POOL_ID_ALLOWED): flash(f"Folder mapping line {line_number} has an invalid pool ID.", "error") return redirect(url_for("storage", section="write-path")) if access_mode not in _SHARE_ACCESS_OPTIONS: diff --git a/openblade/web_flask/client.py b/openblade/web_flask/client.py index 9fd814a..e67ef90 100644 --- a/openblade/web_flask/client.py +++ b/openblade/web_flask/client.py @@ -64,16 +64,38 @@ def _headers( headers["X-OpenBlade-Library-Id"] = str(library_id) return headers + #: Cap on the curated backend message surfaced to the browser. FastAPI details + #: are short sentences; anything longer is a sign we are relaying something we + #: did not curate, and it would blow out the flash banner regardless. + _MAX_DETAIL_CHARS = 300 + def _raise_for_error(self, response: httpx.Response) -> None: - if response.status_code < HTTPStatus.BAD_REQUEST: + """Turn a backend error response into a BackendError with a SAFE message. + + Only the backend's own curated ``detail``/``message`` JSON field is + surfaced, because ``app._handle_backend_error`` flashes this straight into + the operator's browser. A non-JSON body is deliberately NOT echoed: an + unhandled 500, or a gunicorn/proxy error page in front of the API, would + otherwise put a stack trace -- internal paths, hostnames, frames -- on + screen. Jinja escapes it, so this is disclosure rather than XSS, but the + body is still not ours to relay. The status code is preserved on the + exception for callers that branch on it (401 clears the session). + """ + # >= 300, not >= 400. Backend calls run with follow_redirects off, so a 3xx + # is an unexpected response with no JSON body; letting it through meant the + # caller's response.json() raised an uncaught ValueError and Flask returned + # a 500 instead of a flash message. + if response.status_code < HTTPStatus.MULTIPLE_CHOICES: return - detail = "Backend request failed" + detail = f"Backend request failed (HTTP {response.status_code})" try: payload = response.json() - if isinstance(payload, dict): - detail = str(payload.get("detail") or payload.get("message") or detail) except ValueError: - detail = response.text or detail + payload = None + if isinstance(payload, dict): + curated = payload.get("detail") or payload.get("message") + if isinstance(curated, str) and curated.strip(): + detail = curated.strip()[: self._MAX_DETAIL_CHARS] raise BackendError(response.status_code, detail) def login(self, *, username: str, password: str) -> str: @@ -93,6 +115,40 @@ def login(self, *, username: str, password: str) -> str: ) return token + def get_current_user(self, token: str) -> dict[str, Any]: + """The logged-in AML user, including their role. + + Needed because /aml/users/login does not return the role, and the UI has to + know it: the OpenBlade-native config surface carries no per-user + authorization of its own, so the role check has to happen here. + """ + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.get( + f"{self.base_url}/aml/users/me", + headers=self._headers(token), + ) + self._raise_for_error(response) + body = response.json() + if not isinstance(body, dict): + raise BackendError(HTTPStatus.BAD_GATEWAY, "Backend returned an invalid user record") + return body + + def logout(self, token: str) -> None: + """Revoke the AML session server-side. + + Clearing the Flask cookie alone left the sessionID minted at login valid on + the backend until it aged out. + """ + with httpx.Client(timeout=self.timeout_seconds) as client: + response = client.delete( + f"{self.base_url}/aml/users/login", + headers=self._headers(token), + ) + # A failure here must not block the local logout; the caller clears the + # session regardless. 401 simply means it was already invalid. + if response.status_code not in {HTTPStatus.UNAUTHORIZED, HTTPStatus.FORBIDDEN}: + self._raise_for_error(response) + def list_devices(self, token: str) -> list[Device]: with httpx.Client(timeout=self.timeout_seconds) as client: response = client.get( @@ -777,7 +833,15 @@ def probe_device_endpoint( raise BackendError(HTTPStatus.BAD_REQUEST, "Invalid device URL") base_url = connection_url.rstrip("/") timeout = min(self.timeout_seconds, 5.0) - with httpx.Client(timeout=timeout, follow_redirects=True) as client: + # follow_redirects=False is a SECURITY requirement, not a preference. The + # URL was vetted by app._validate_device_url, but a redirect is a second, + # UNVETTED destination: an operator-supplied host answering + # "302 Location: http://127.0.0.1:.../healthz" turned a blocked target into + # a reachable one (verified against a live listener). Worse, the login probe + # below POSTs the device password, and httpx replays the body on 307/308, so + # following redirects would let an arbitrary host harvest that credential. + # A device health endpoint has no legitimate reason to redirect. + with httpx.Client(timeout=timeout, follow_redirects=False) as client: reachable = False for path in ("/healthz", "/health", "/aml/system/status"): try: @@ -786,7 +850,10 @@ def probe_device_endpoint( ) except httpx.HTTPError: continue - if response.status_code < HTTPStatus.BAD_REQUEST: + # A 3xx is NOT a healthy device. Redirects are no longer followed, + # so treating one as success would report a redirector as a working + # library, and below would count it as a successful auth. + if response.status_code < HTTPStatus.MULTIPLE_CHOICES: reachable = True break if not reachable: @@ -798,7 +865,7 @@ def probe_device_endpoint( json={"name": username, "password": password or ""}, headers={"Accept": "application/json"}, ) - if login_response.status_code >= HTTPStatus.BAD_REQUEST: + if login_response.status_code >= HTTPStatus.MULTIPLE_CHOICES: raise BackendError(HTTPStatus.BAD_REQUEST, "Device authentication check failed") authenticated = True return {"reachable": True, "authenticated": authenticated} diff --git a/openblade/web_flask/models.py b/openblade/web_flask/models.py index 54ebae9..187bc7b 100644 --- a/openblade/web_flask/models.py +++ b/openblade/web_flask/models.py @@ -4,6 +4,23 @@ from typing import Any +def _coerce_int(value: Any, *, default: int = 0) -> int: + """int() a backend-supplied field without raising. + + ``int(payload.get("id", 0))`` dies on an explicit ``None`` -- ``.get`` only + returns the default when the key is ABSENT, and the API can legitimately send + ``{"id": null}``. That turned a device listing into a 500. + """ + if isinstance(value, bool): + return int(value) + if isinstance(value, int): + return value + try: + return int(str(value).strip()) + except (TypeError, ValueError): + return default + + @dataclass(frozen=True, slots=True) class Device: id: int @@ -18,12 +35,12 @@ class Device: def parse_device(payload: dict[str, Any]) -> Device: return Device( - id=int(payload.get("id", 0)), + id=_coerce_int(payload.get("id")), name=str(payload.get("name", "Unknown Device")), emulator_url=str(payload.get("emulator_url", "")), model=str(payload.get("model", "Scalar i3")), role=str(payload.get("role", "primary")), status=str(payload.get("status", "unknown")).lower(), enabled=bool(payload.get("enabled", True)), - sort_order=int(payload.get("sort_order", 0)), + sort_order=_coerce_int(payload.get("sort_order")), ) diff --git a/tests/unit/test_web_flask_app.py b/tests/unit/test_web_flask_app.py index 82f1254..85ff8aa 100644 --- a/tests/unit/test_web_flask_app.py +++ b/tests/unit/test_web_flask_app.py @@ -1,10 +1,12 @@ from __future__ import annotations from dataclasses import replace +from ipaddress import ip_address import pytest -from openblade.web_flask.app import create_app +from openblade.web_flask import app as web_flask_app +from openblade.web_flask.app import _validate_device_url, create_app from openblade.web_flask.client import BackendClient, BackendError from openblade.web_flask.models import Device @@ -151,12 +153,28 @@ def __init__(self) -> None: } ] self.last_dataset_verify_id: str | None = None + self.last_archive: dict[str, object] | None = None + #: AML role the fake reports from /aml/users/me. 0 == Administrator, which + #: is what most tests need; set it to 1 or 2 to exercise the write gates. + self.current_role = 0 + self.logout_calls: list[str] = [] def login(self, *, username: str, password: str) -> str: if username == "admin" and password == "admin": return "session-token" raise BackendError(status_code=401, detail="invalid credentials") + def get_current_user(self, token: str) -> dict[str, object]: + assert token == "session-token" + return { + "name": "admin", + "role": self.current_role, + "requirePasswordChange": False, + } + + def logout(self, token: str) -> None: + self.logout_calls.append(token) + def list_devices(self, token: str) -> list[Device]: assert token == "session-token" return self.devices @@ -607,6 +625,7 @@ def create_archive_job( assert token == "session-token" assert source_path assert volume_group + self.last_archive = {"source_path": source_path, "volume_group": volume_group} return {"job_id": "job-archive", "status": "pending"} def create_restore_job( @@ -622,6 +641,14 @@ def create_restore_job( @pytest.fixture def app_client(monkeypatch: pytest.MonkeyPatch): monkeypatch.setenv("FLASK_SECRET_KEY", "test-secret-key") + # _validate_device_url resolves device hostnames and fails closed on names that + # do not resolve. The fixture's device names ("device-b") are fictional, so stub + # the resolver to a public address rather than making the suite depend on live + # DNS. The deny-list behaviour itself is covered directly against + # _validate_device_url in the SSRF tests below. + monkeypatch.setattr( + web_flask_app, "_resolve_device_host", lambda hostname: [ip_address("198.51.100.10")] + ) app = create_app(client_factory=FakeBackendClient) app.config["TESTING"] = True return app.test_client() @@ -1132,7 +1159,10 @@ def test_hydrate_restore_action_uses_latest_tape_copy(app_client) -> None: "/devices/1/operations/restore", data={ "catalog_path": "/archive/file-a.bin", - "dest_path": "/restore", + # /openblade/restore, not a bare /restore: dest_path is a real + # filesystem write in jobs/restore.py, so it must sit under an allowed + # storage root. This matches tests/i3/test_09_restore_cycle.py. + "dest_path": "/openblade/restore", "csrf_token": _csrf(app_client), }, follow_redirects=False, @@ -1140,7 +1170,7 @@ def test_hydrate_restore_action_uses_latest_tape_copy(app_client) -> None: assert response.status_code == 302 backend = _backend_from_client(app_client) assert backend.last_restore is not None - assert backend.last_restore["destination_path"] == "/restore" + assert backend.last_restore["destination_path"] == "/openblade/restore" def test_magazine_eject_action_calls_backend(app_client) -> None: @@ -1231,6 +1261,377 @@ def test_headers_fall_back_to_session_bearer_when_no_native_token() -> None: assert headers["Cookie"] == "sessionID=aml-session-token" +@pytest.mark.parametrize( + "url", + [ + # Loopback, spelled every way that defeats a string deny-list. Each of the + # first three was PROVEN to reach a live listener on 127.0.0.1 before the + # resolve-and-check fix. + "http://127.0.0.1:9911", + "http://127.1:9911", + "http://2130706433:9911", + "http://localhost.:9911", + "http://[::1]:9911", + "http://0/", + # Cloud metadata, by name and as a packed integer. + "http://169.254.169.254/", + "http://2852039166/", + "http://metadata.google.internal/", + # Other non-routable classes. + "http://224.0.0.1/", + "http://240.0.0.1/", + ], +) +def test_validate_device_url_blocks_ssrf_targets(url: str) -> None: + """A device URL must never be pointed at loopback, metadata, or multicast. + + Mutation check: delete the `_is_unsafe_ip` call from `_validate_device_url` (or + drop `is_loopback`/`is_link_local` from `_is_unsafe_ip`) and these fail. The + previous implementation compared HOST SPELLINGS against a set, which is why + "127.1" and "2130706433" both sailed through it. + """ + assert _validate_device_url(url) is not None + + +@pytest.mark.parametrize( + "url", + [ + # RFC1918 stays allowed ON PURPOSE: a real Scalar i3 lives on the operator's + # LAN, so blocking private space would refuse every legitimate device. + "http://10.0.0.5:8010/", + "http://192.168.1.10:8010/", + "http://172.16.0.1:8010/", + ], +) +def test_validate_device_url_allows_lan_devices(url: str) -> None: + assert _validate_device_url(url) is None + + +def test_validate_device_url_rejects_unresolvable_host() -> None: + """Fail closed: an unresolvable name is refused, not probed.""" + assert _validate_device_url("http://no-such-host.invalid:8010/") is not None + + +def test_validate_device_url_checks_resolved_address_not_spelling( + monkeypatch: pytest.MonkeyPatch, +) -> None: + """A benign-looking name that RESOLVES to loopback must be refused. + + This is the case a deny-list can never catch, because the attacker owns the DNS + record. Mutation check: make _validate_device_url skip the resolved-address loop + and this fails. + """ + monkeypatch.setattr( + web_flask_app, "_resolve_device_host", lambda hostname: [ip_address("127.0.0.1")] + ) + assert _validate_device_url("http://totally-normal.example.com:8010/") is not None + + +def test_validate_device_url_rejects_embedded_credentials() -> None: + assert _validate_device_url("http://user:pw@10.0.0.5:8010/") is not None + + +def test_validate_device_url_override_allows_loopback(monkeypatch: pytest.MonkeyPatch) -> None: + """The documented development escape hatch still works.""" + monkeypatch.setenv("OPENBLADE_WEB_ALLOW_UNSAFE_DEVICE_TARGETS", "true") + assert _validate_device_url("http://127.0.0.1:8010/") is None + + +def test_probe_does_not_follow_redirects(monkeypatch: pytest.MonkeyPatch) -> None: + """The probe must not chase a redirect to an unvetted host. + + The vetted URL is only the FIRST hop; a redirect is a second destination nobody + checked, and httpx replays the POST body on 307/308, which would hand the + submitted device password to whatever host the redirect names. + + Mutation check: set follow_redirects=True in probe_device_endpoint and this test + fails, because the 302 would be chased to a 200. + """ + captured: dict[str, object] = {} + + class FakeResponse: + status_code = 302 + headers = {"Location": "http://127.0.0.1:9913/healthz"} + + class FakeHttpxClient: + def __init__(self, **kwargs: object) -> None: + captured.update(kwargs) + + def __enter__(self) -> FakeHttpxClient: + return self + + def __exit__(self, *exc: object) -> None: + return None + + def get(self, *args: object, **kwargs: object) -> FakeResponse: + return FakeResponse() + + def post(self, *args: object, **kwargs: object) -> FakeResponse: + return FakeResponse() + + import openblade.web_flask.client as client_module + + monkeypatch.setattr(client_module.httpx, "Client", FakeHttpxClient) + client = BackendClient(base_url="http://backend") + with pytest.raises(BackendError): + client.probe_device_endpoint(connection_url="http://10.0.0.5:8010") + assert captured.get("follow_redirects") is False + + +@pytest.mark.parametrize( + ("path", "field", "payload", "recorder"), + [ + ("/storage/archive", "source_path", {"volume_group": "vg-default"}, "last_archive"), + ("/storage/restore", "dest_path", {"catalog_path": "/archive/f.bin"}, "last_restore"), + ( + "/storage/cache-drives", + "root_path", + {"drive_id": "c1", "name": "C"}, + "last_cache_drive", + ), + ], +) +def test_storage_writes_reject_paths_outside_allowed_roots( + app_client, path: str, field: str, payload: dict[str, str], recorder: str +) -> None: + """Operator paths are confined to allowed roots, not merely traversal-checked. + + These values reach real filesystem sinks in the API container: source_path is + rglob'd by routes_archive, dest_path is written by jobs/restore, root_path + becomes a cache-drive root. Those backend routes have no per-user authorization, + so /etc, /root and /proc/self/environ -- which holds OPENBLADE_API_TOKEN -- must + be unreachable from this form even though none of them contains "..". + + Mutation check: swap _is_allowed_storage_root back to _is_safe_storage_path at + the call site and these fail, because "/etc/passwd" passes a traversal check. + """ + _login(app_client) + backend = _backend_from_client(app_client) + app_client.get("/storage/overview") + for bad in ("/etc/passwd", "/root/.ssh", "/proc/self/environ", "/"): + data = {**payload, field: bad, "csrf_token": _csrf(app_client)} + app_client.post(path, data=data, follow_redirects=False) + assert getattr(backend, recorder) is None, f"{bad} reached the backend via {field}" + + +def test_archive_accepts_path_under_allowed_root(app_client) -> None: + """The confinement must not block the legitimate case.""" + _login(app_client) + app_client.get("/storage/overview") + response = app_client.post( + "/storage/archive", + data={ + "source_path": "/openblade/inbox/finance", + "volume_group": "vg-default", + "csrf_token": _csrf(app_client), + }, + follow_redirects=False, + ) + assert response.status_code == 302 + assert response.headers["Location"].endswith("/jobs") + + +def test_archive_rejects_unvalidated_volume_group(app_client) -> None: + """volume_group was the one field reaching the backend with no allowlist. + + It becomes a catalog path component via PurePosixPath("/") / volume_group, and + PurePosixPath does not normalise, so ".." would persist into catalog keys. + """ + _login(app_client) + app_client.get("/storage/overview") + response = app_client.post( + "/storage/archive", + data={ + "source_path": "/openblade/inbox/finance", + "volume_group": "../../etc", + "csrf_token": _csrf(app_client), + }, + follow_redirects=False, + ) + assert response.status_code == 302 + assert response.headers["Location"].endswith("/storage/archive") + + +@pytest.mark.parametrize("role", [1, 2]) +def test_config_writes_require_admin_role(app_client, role: int) -> None: + """Non-admin sessions cannot rewrite storage configuration. + + This gate lives in the UI because the backend's config surface has none: + /nas/policies, /nas/shares, /nas/pools, /nas/cache-drives and + /nas/source-stream carry no require_auth. Since BackendClient presents the + instance-wide OPENBLADE_API_TOKEN as the native bearer, every session would + otherwise act with full native authority regardless of its AML role. + + Mutation check: remove the _require_admin() calls and these fail -- role 1 and 2 + reach the backend and last_policy/last_cache_drive get set. + """ + backend = _backend_from_client(app_client) + backend.current_role = role + _login(app_client) + app_client.get("/storage/overview") + for path, data in ( + ("/storage/policies", {"policy_id": "p1", "name": "P", "policy_type": "balanced"}), + ("/storage/cache-drives", {"drive_id": "c1", "name": "C", "root_path": "/openblade/cache"}), + ("/storage/source-stream", {"checksum_mode": "streaming", "max_retries": "2"}), + ( + "/storage/shares", + {"share_path": "/openblade/x", "share_name": "x", "share_type": "pool"}, + ), + ): + response = app_client.post( + path, data={**data, "csrf_token": _csrf(app_client)}, follow_redirects=False + ) + assert response.status_code == 302 + assert backend.last_policy is None + assert backend.last_cache_drive is None + assert backend.last_source_stream is None + assert backend.last_share is None + + +def test_config_writes_allowed_for_admin_role(app_client) -> None: + """The admin path still works -- the gate is not a blanket refusal.""" + _login(app_client) + backend = _backend_from_client(app_client) + app_client.get("/storage/write-path") + response = app_client.post( + "/storage/policies", + data={ + "policy_id": "critical-fast", + "name": "Critical Fast", + "policy_type": "critical_sequential", + "copies_required": "2", + "max_parallelism": "1", + "csrf_token": _csrf(app_client), + }, + follow_redirects=False, + ) + assert response.status_code == 302 + assert backend.last_policy is not None + + +def test_unknown_role_fails_closed(app_client) -> None: + """If /aml/users/me cannot be read, writes are denied rather than allowed. + + A role we could not determine must never default to Administrator. + """ + backend = _backend_from_client(app_client) + + def boom(token: str) -> dict[str, object]: + raise BackendError(status_code=503, detail="unavailable") + + backend.get_current_user = boom # type: ignore[method-assign] + _login(app_client) + app_client.get("/storage/overview") + response = app_client.post( + "/storage/policies", + data={"policy_id": "p1", "name": "P", "csrf_token": _csrf(app_client)}, + follow_redirects=False, + ) + assert response.status_code == 302 + assert backend.last_policy is None + + +def test_logout_revokes_the_backend_session(app_client) -> None: + """Clearing the Flask cookie is not enough; the AML sessionID must be revoked.""" + _login(app_client) + backend = _backend_from_client(app_client) + app_client.post("/logout", data={"csrf_token": _csrf(app_client)}, follow_redirects=False) + assert backend.logout_calls == ["session-token"] + + +@pytest.mark.parametrize("value", ["..", ".", "..."]) +def test_identifier_allowlist_rejects_dot_only_segments(app_client, value: str) -> None: + """Dot-only identifiers are refused before they reach a URL or a path. + + _POOL_ID_ALLOWED permits ".", so ".." matched the alphabet. These values are + interpolated into backend URLs and into /pools/, and both httpx and + POSIX normalisation collapse a ".." segment -- verified: POST + /storage/datasets/../verify rewrote the backend URL to /nas/verify. + + Mutation check: call pattern.fullmatch directly instead of _is_safe_identifier + and these fail. + """ + _login(app_client) + backend = _backend_from_client(app_client) + app_client.get("/storage/catalog") + app_client.post( + f"/storage/datasets/{value}/verify", + data={"csrf_token": _csrf(app_client)}, + follow_redirects=False, + ) + assert backend.last_dataset_verify_id is None + + +def test_redirect_target_rejects_backslash() -> None: + """ "/\\evil.example" must not be accepted as a local redirect. + + urlparse reports no netloc for it, so without an explicit check the only thing + preventing an off-site redirect is Werkzeug percent-encoding the backslash in + the Location header -- someone else's escaping doing our security. + """ + from openblade.web_flask.app import _is_safe_redirect_target + + assert _is_safe_redirect_target("/devices") is True + assert _is_safe_redirect_target("/\\evil.example") is False + assert _is_safe_redirect_target("//evil.example") is False + + +def test_backend_error_does_not_echo_non_json_body() -> None: + """A non-JSON upstream body must not be relayed into the operator's browser. + + _handle_backend_error flashes BackendError.detail, so echoing response.text put + proxy/gunicorn HTML error pages -- internal paths, hostnames, stack frames -- on + screen. Only the backend's own curated JSON detail is surfaced. + + Mutation check: restore `detail = response.text` and this fails. + """ + + class FakeResponse: + status_code = 500 + text = "Traceback: /srv/app/secret.py line 3, password=hunter2" + + def json(self) -> object: + raise ValueError("not json") + + client = BackendClient(base_url="http://backend") + with pytest.raises(BackendError) as excinfo: + client._raise_for_error(FakeResponse()) # type: ignore[arg-type] + assert "hunter2" not in excinfo.value.detail + assert "Traceback" not in excinfo.value.detail + assert "500" in excinfo.value.detail + + +def test_backend_error_surfaces_curated_json_detail() -> None: + """The curated FastAPI detail IS still shown -- error quality is not degraded.""" + + class FakeResponse: + status_code = 400 + text = "" + + def json(self) -> object: + return {"detail": "Barcode ABC123L9 is not in the library"} + + client = BackendClient(base_url="http://backend") + with pytest.raises(BackendError) as excinfo: + client._raise_for_error(FakeResponse()) # type: ignore[arg-type] + assert excinfo.value.detail == "Barcode ABC123L9 is not in the library" + + +def test_login_bucket_is_capped(monkeypatch: pytest.MonkeyPatch) -> None: + """The pre-auth login-attempt dict must not grow without bound. + + It is keyed partly by a caller-supplied username, and _is_login_blocked sweeps + the whole dict on every attempt, so an unbounded dict is both a memory leak and + a self-amplifying CPU cost (measured: 50k keys -> ~13ms per login attempt). + """ + monkeypatch.setenv("FLASK_SECRET_KEY", "test-secret-key") + app = create_app(client_factory=FakeBackendClient) + for index in range(web_flask_app._LOGIN_BUCKET_MAX_KEYS + 500): + web_flask_app._record_login_failure(app, f"1.2.3.4:user{index}") + bucket = web_flask_app._attempt_bucket(app) + assert len(bucket) <= web_flask_app._LOGIN_BUCKET_MAX_KEYS + + def test_headers_omit_credentials_when_unauthenticated() -> None: """The login call itself must not present a session cookie.""" client = BackendClient(base_url="http://backend") From 6df330631cbea12c57b1ae27449e235e6f5a3f7c Mon Sep 17 00:00:00 2001 From: Hermes Agent Date: Sat, 26 Sep 2026 06:53:50 +0000 Subject: [PATCH 43/54] docs: record the adversarial review round in the disposition note Adds the four MUST-FIX findings and their fixes, the ten lesser items, the areas the reviewer attacked and found clean, and the mutation-check result (13/13). Also records the one recommendation deliberately NOT taken: blocking is_private would have refused every real LAN tape library, so the address check blocks loopback/link-local/unspecified/multicast/reserved and allows RFC1918. The defect was that the old guard compared host spellings, not that its scope was wrong. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_013S1qbQs31YSox1azQChqs2 --- .../2026-09-26-flask-nas-disposition.md | 52 ++++++++++++++++++- 1 file changed, 50 insertions(+), 2 deletions(-) diff --git a/docs/decisions/2026-09-26-flask-nas-disposition.md b/docs/decisions/2026-09-26-flask-nas-disposition.md index 4ac3174..28fd043 100644 --- a/docs/decisions/2026-09-26-flask-nas-disposition.md +++ b/docs/decisions/2026-09-26-flask-nas-disposition.md @@ -192,11 +192,55 @@ about. Every endpoint and Flask route named in it was verified to exist. Landing a UI is no reason to restate the project's goals, and the reword drops an explicit scope-discipline clause. +## Adversarial review round (T3) + +The reviewer returned **BLOCK** with four MUST-FIX items, two carrying working +repros against live sockets. All are fixed; the suite went 37 → 73 tests. + +| # | Finding | Fix | +| --- | --- | --- | +| 1 | **SSRF.** The device-URL deny-list compared host *spellings*, so `127.1`, `2130706433` and `localhost.` all reached a loopback listener (proven), and any attacker-controlled DNS name could resolve into private space. Gave a logged-in user an internal port scanner and a cloud-metadata reach test. | Resolve the host and judge every returned address | +| 2 | **Redirect bypass + credential replay.** The probe followed redirects, so a host answering `302 Location: http://127.0.0.1/…` turned a blocked target into a reachable one; httpx replays the body on 307/308, so the submitted device password could be harvested. | `follow_redirects=False`; 3xx no longer counts as reachable | +| 3 | **No path confinement.** The guard blocked `..` but not *which* absolute roots were reachable, and the values become real filesystem sinks in a backend whose `/archive` and `/restore` have no per-user authz — so `/proc/self/environ`, which holds `OPENBLADE_API_TOKEN`, was archivable to tape. | `_is_allowed_storage_root` on the four genuine filesystem paths | +| 4 | **Confused deputy.** Because the client presents the instance-wide `OPENBLADE_API_TOKEN` as the native bearer, and the native config surface has no `require_auth`, **any** session could rewrite storage policy regardless of AML role. Fix (2) in the landing commit turned "broken for everyone" into "authorized for everyone", so this was a widening introduced by that change. | Role gate on the six config-write routes, resolved from `/aml/users/me`, failing closed on an unknown role | + +Ten SHOULD-FIX/NIT items were also closed: dot-only identifiers (`_POOL_ID_ALLOWED` +permitted `.`, so `..` matched and collapsed a URL segment), unvalidated +`volume_group`, non-JSON upstream bodies echoed into the browser, 3xx crashing +`.json()`, an unbounded pre-auth login bucket plus a missing IP-only counter, no +logout revocation, backslash accepted in redirect targets, the reflected `Host` in +copy-paste mount commands, and bare `int()` on backend values. + +**One reviewer recommendation was deliberately not taken.** It proposed adding +`is_private` to the blocked-address set. That would block `10.x`, `172.16.x` and +`192.168.x` — which is exactly where a real Scalar i3 lives — and would refuse +every legitimate device. The check blocks loopback, link-local (which covers +169.254.169.254), unspecified, multicast and reserved, and allows RFC1918. This +matches the original intent of `_DANGEROUS_DEVICE_HOSTS`, which never listed +private ranges; the defect was that it compared spellings, not that its *scope* was +wrong. + +**Areas the reviewer attacked and found clean:** session/CSRF/privilege plumbing +(independently re-verified here — all 36 non-exempt endpoints 302 to login when +anonymous, and all 24 mutating routes return 400 without a CSRF token, with no GET +mutating state); template/XSS (no `|safe`, `Markup`, `render_template_string`, +inline `