Evaluation-mode could obscure cause of regression differences #3725
Description
There was often an issue when interpreting differences in the regression tests: a slight change to one variable could change the path of the solver and cause many large diffs that were unrelated to the initial change (or mistake). The "once-through" run mitigated this issue by only display the different variable and any direct downstream changes.
I worry that with #3646--where fsolve is used to ensure consistency--we might end up in a similar situation where a difference to a variable upstream of the consistency equations will change the path and hence produce many unrelated differences. Is it possible to introduce some sort of "developer evaluation mode" which is similar to the old once through, where only the idempotence loop is run and no optimisation/consistency checks occurs? I believe this could help when debugging regression differences