Mechanistic Interpretability

• Clearer better understanding of Neural Networks working (white box).
• Strong grounds for Superposition: n-dimensions (neurons) represent more than n-features

References

• https://dynalist.io/d/n2ZWtnoYHrU1s4vnFSAQ519J#z=EuO4CLwSIzX7AEZA1ZOsnwwF
• https://www.neelnanda.io/mechanistic-interpretability/glossary
• https://transformer-circuits.pub/2022/toy_model/index.html
• https://www.anthropic.com/research/superposition-memorization-and-double-descent
• https://transformer-circuits.pub/2023/toy-double-descent/index.html 

State Space Models

• Vector Space of States (of the System)
  
• Alt. to Transformers, reducible to one another 

 

        (Image source: https://en.wikipedia.org/wiki/State-space_representation)

References

• https://newsletter.maartengrootendorst.com/p/a-visual-guide-to-mamba-and-state
• https://huggingface.co/blog/lbourdois/ssm-2022
• https://huggingface.co/blog/lbourdois/get-on-the-ssm-train
• https://en.wikipedia.org/wiki/State-space_representation

Learning Deep

Head out straight to KdNugget's Top 20 Deep Learning Papers of 2018. Has a good listing of research publications spanning over the last 4-5 years. You could further go on to read the papers referred to within these papers & then those referred to in the referred papers & so on for some really deep learning!