A flat L2 penalty can match EWC's exact Task A retention number on Permuted MNIST: 97.63% against 97.63%. It does this by collapsing Task B accuracy to 18%, against EWC's 97%. The same headline number hides two very different costs.

Key takeaways

A registered gate asked whether flat, uniform L2 regularisation matches EWC's Fisher-weighted penalty at comparable cost. The cheapest tested setting landed within 0.86 points of EWC's Task A accuracy, inside the registered 3-point bar - the gate fired.

No tested setting met the registered "fair budget" of costing at most 1 point of Task B accuracy. The cheapest one still cost 2.70 points, over six times what EWC paid.

At a stronger setting, flat L2 hits EWC's exact Task A number, 97.63%, while Task B accuracy collapses to 18.42% against EWC's 97.43%.

Fisher weighting is not required to match EWC's retention number in this setup.…