Basics
Objective
Given a DAG, get a browser page that allows you to interact with its nodes, to try out different scenarios.
Use Cases
Model Evaluation
A model has false positives and negatives. Accuracy, the go to metric, is often not appropriate.
Appropriate would be to associate each (or some) of elements of a confusion matrix to a value,
and compute a model evaluation metric accordingly.
The problem is; what values should we associate? It's not clear. There's a lot of wiggle room and different perspectives. This is where an interactive visual app can help out since we can enter different scenarios and see their effects.
Keywords
Cost structure, inference, what-if scenarios
See this for ugly examples (that I made in ugly JS) or
this rent or buy NYT article for
a beautifully well done example.
Tasks
Explore possibilities
What open source tools do we have at our disposition -- accessible through python.
Find some cool examples
Use meshed.dag.DAG to implement these examples
Make a dag2app using streamlit?
Basics
Objective
Given a DAG, get a browser page that allows you to interact with its nodes, to try out different scenarios.
Use Cases
Model Evaluation
A model has false positives and negatives. Accuracy, the go to metric, is often not appropriate.
Appropriate would be to associate each (or some) of elements of a confusion matrix to a value,
and compute a model evaluation metric accordingly.
The problem is; what values should we associate? It's not clear. There's a lot of wiggle room and different perspectives. This is where an interactive visual app can help out since we can enter different scenarios and see their effects.
Keywords
Cost structure, inference, what-if scenarios
See this for ugly examples (that I made in ugly JS) or
this rent or buy NYT article for
a beautifully well done example.
Tasks
Explore possibilities
What open source tools do we have at our disposition -- accessible through python.
Find some cool examples
Use
meshed.dag.DAGto implement these examplesMake a dag2app using streamlit?