research

#theme-toggle, .top-link { display: none; } @media (prefers-color-scheme: dark) { :root { --theme: #1d1e20; --entry: #2e2e33; --primary: rgba(255, 255, 255, 0.84); --secondary: rgba(255, 255, 255, 0.56); --tertiary: rgba(255, 255, 255, 0.16); --content: rgba(255, 255, 255, 0.74); --hljs-bg: #2e2e33; --code-bg: #37383e; --border: #333; } .list { background: var(--theme); } .list:not(.dark)::-webkit-scrollbar-track { background: 0 0; } .list:not(.dark)::-webkit-scrollbar-thumb { border-color: var(--theme); } }

Interesting problems in DL theory

Here are some research directions I worked on with before my PhD. Mathematics of machine learning is a very cool field, although these days I think empirical research is a bit more impactful. When are ReLU neural networks injective? Update Nov 2021: I managed to solve some of the problems below in my Master thesis. Update Oct 2022: Antoine Maillard and others have apparently resolved the injectivity phase transition in an upcoming publication!...

Links

Twitter @dpaleka I am a researcher in a field that moves fast. It is hard to overstate how new everything in machine learning is; to paraphrase a famous quote, a DL researcher in 2022 time traveling back to 2012 is Prometheus bringing fire to mankind. Twitter turns out to be: the best way to know what happens; the best way to show other people what I’m doing. Furthermore, it’s a good way to connect with people who could make great collaborators or friends....