Questions regarding deep active subspaces

[lorenzozanisi]

Hi, I have read with a lot if interest you report on DAS. I can see that it ends with some important questions. Did you investigate those further since the report? For instance, how does the inductive bias in DAS compared to just a standard feedforward NN, where presumably the first layers would, like in DAS, learn a convenient representation of the data space (albeit, unlike DAS, this is unconstrained for a standard NN)?

Thanks
Lorenzo

[wedeling]

@lorenzozanisi Sorry for the very slow reply, I only just saw this now. Most of these questions have been investigated in an article by now: https://www.researchgate.net/publication/363173887_On_the_deep_active-subspace_method

If an active subspace is present in your problem, and you've correctly identified its dimension (by looking at eigenvalue decay), the DAS representation is more robust than an ANN in the sense that it only fits the 'signal' and not the noise. The article should make this clear hopefully. If you have questions let me know, you can also email me (edeling at cwi.nl).