Interesting Problems

Here are some research directions I sometimes play with. I have spent between one hour and one month thinking on each of these questions. How wide should one-hidden-layer networks be to be robust (when interpolating random data)? The general idea of the tradeoff between neural network robustness; and neural network layer widths is well known1. Of course, one should compare networks with the same accuracy; for simplicity, consider networks that interpolate a random dataset....

April 8, 2021

Links

Shout out to… First, the people who I currently feel deserve way more attention for their writing: 1 Neel Nanda Vitaly Kurin Daniel Seita Writing by established machine learning researchers I think famous researchers who still take time to write blogs are a gift. Please keep it up! Machine learning theory Lilian Weng Francis Bach Sébastien Bubeck Ben Recht AI security and safety Nicholas Carlini Andy Jones Applied machine learning Eric Jang Andrej Karpathy Davis Blalock If you have (or have read) a cool site along these lines, feel free to send me a link....