UPDATE: Please go here for an updated version:
https://vicco-group.github.io/DNN_vs_brain-and-behavior/
This list is fully up to date (but surely not complete!) until 2018/06, afterwards likely more incomplete
For updates, follow me on Twitter: @martin_hebart
Kriegeskorte, N. (2015) – Deep neural networks: a new framework for modeling biological vision and brain information processing. Annual review of vision science, 1, 417-446.
https://doi.org/10.1146/annurev-vision-082114-035447
Gauthier, I., & Tarr, M. J. (2016). Visual object recognition: Do we (finally) know more now than we did? Annual review of vision science, 2, 377-396.
https://doi.org/10.1146/annurev-vision-111815-114621
Marblestone, A. H., Wayne, G., & Kording, K. P. (2016). Toward an integration of deep learning and neuroscience. Frontiers in Computational Neuroscience, 10, 94.
https://doi.org/10.3389/fncom.2016.00094
Excellent, very complete review of the ML/cortex frontier. It argues in favour of a computational neuroscience that researches the brain’s cost functions and their optimisation processes, rather than specific circuits. It debates what ML can learn from neuroscience and vice-versa.
Yamins, D. L., & DiCarlo, J. J. (2016). Using goal-driven deep learning models to understand sensory cortex. Nature neuroscience, 19(3), 356
https://doi.org/10.1038/nn.4244
Gentle introduction to Convolutional Neural Networks and their use for predicting the encoding strategy of sensory areas. Mostly conversational, does not present hard analytical or data-driven results other than as a review.
Hassabis, D., Kumaran, D., Summerfield, C., & Botvinick, M. (2017). Neuroscience-inspired artificial intelligence. Neuron, 95(2), 245-258.
https://doi.org/10.1016/j.neuron.2017.06.011
General view, including historical perspective, of AI/Neuroscience relationships. Particularly focused on the cognitive/behavioural level (transfer learning, reinforcement learning, attention, memory). By DeepMind founder Demis Hassabis and colleagues.
Kay, K. N. (2017). Principles for models of neural information processing. NeuroImage. http://dx.doi.org/10.1016/j.neuroimage.2017.08.016
Lake, B. M., Ullman, T. D., Tenenbaum, J. B., & Gershman, S. J. (2017). Building machines that learn and think like people. Behavioral and Brain Sciences, 40.
https://doi.org/10.1017/S0140525X16001837
VanRullen, R. (2017). Perception science in the age of deep neural networks. Frontiers in psychology, 8, 142.
https://doi.org/10.3389/fpsyg.2017.00142
Aru, J., & Vicente, R. (2018). What deep learning can tell us about higher cognitive functions like mindreading?. arXiv preprint arXiv:1803.10470.
https://arxiv.org/abs/1803.10470
Barrett, David GT, Ari S. Morcos, and Jakob H. Macke. "Analyzing biological and artificial neural networks: challenges with opportunities for synergy?." arXiv preprint arXiv:1810.13373 (2018). https://arxiv.org/abs/1810.13373
Good overview on challenges and opportunities in comparing DNNs with BioNets. What analysis methods can be used on both? What methods can be used to compare them to each other?
Glaser, J. I., Benjamin, A. S., Farhoodi, R., & Kording, K. P. (2018). The Roles of Supervised Machine Learning in Systems Neuroscience. arXiv preprint arXiv:1805.08239.
https://arxiv.org/abs/1805.08239
How can machine learning be used in neuroscience? 1. Solving engineering problems; 2. Identifying predictive variables; 3. Benchmarking simple models; 4. Serving as a model for the brain.
Majaj, N. J., & Pelli, D. G. (2018). Deep learning—Using machine learning to study biological vision. Journal of vision, 18(13), 2-2
https://doi.org/10.1167/18.13.2
Cichy, R. M., Kaiser, D. (2019). Deep neural networks as scientific models. Trends in Cognitive Sciences.
https://doi.org/10.1016/j.tics.2019.01.009
Kietzmann, T. C., McClure, P., & Kriegeskorte, N. (2019). Deep neural networks in computational neuroscience. In Oxford Research Encyclopedia of Neuroscience. Oxford University Press. doi: http://dx.doi.org/10.1093/acrefore/9780190264086.013.46
Storrs, K. & Kriegeskorte, N. (2019). Deep Learning for Cognitive Neuroscience. In The Cognitive Neurosciences, 6th Edition. https://arxiv.org/abs/1903.01458
Chén, O. Y. (2019), The Roles of Statistics in Human Neuroscience. Brain Science, 9(8), 194.
Sinz FH, Pitkow X, Reimer J, Bethge M, Tolias AS, Engineering a less artificial intelligence Neuron 103(6). doi: https://doi.org/10.1016/j.neuron.2019.08.034
Marblestone et al. (2016). Toward an integration of deep learning and neuroscience. See above.
Whittington and Bogacz, Theories of Error Back-propagation in the Brain, Trends in Cog Sci 2018 https://www.cell.com/trends/cognitive-sciences/fulltext/S1364-6613(19)30012-9
A review with a particular focus on bio-plausible implementations of backpropagation in artificial neural networks.
Pieter R. Roelfsema & Anthony Holtmaat, Control of synaptic plasticity in deep cortical networks, Nat Rev Neurosci 19, 166–180 (2018) https://www.nature.com/articles/nrn.2018.6
Bio-oriented. Neuroscience review on how the brain may solve the “credit assignment problem” by means of neuromodulation and feedback that “gate” and “steer” plasticity in earlier layers, based on the outcome of the action and on what synapses contributed to the decision.
Lillicrap, T., Santoro, A., Marris, L., Akerman, C., Hinton, G. (2020). Backpropagation and the brain Nature Reviews Neuroscience https://dx.doi.org/10.1038/s41583-020-0277-3
Please no medical imaging papers! Also, any paper before 2014 please at the bottom, there is a separate section for that.
2014:
Cadieu et al (2014) – Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognition – PloS Comput Biol
Khaligh-Razavi & Kriegeskorte (2014) – Deep Supervised, but Not Unsupervised, Models May Explain IT Cortical Representation – PloS Comput Biol
https://doi.org/10.1371/journal.pcbi.1003915
Yamins et al (2014) – Performance-optimized hierarchical models predict neural responses in higher visual cortex – PNAS
Performance-optimized hierarchical models predict neural responses in higher visual cortex
2015:
Güçlü & van Gerven (2015) – Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream – J Neurosci
https://doi.org/10.1523/JNEUROSCI.5023-14.2015
Güçlü & van Gerven (2015) – Increasingly complex representations of natural movies across the dorsal stream are shared between subjects – Neuroimage
https://doi.org/10.1016/j.neuroimage.2015.12.036
B. Vintch, J. A. Movshon, and E. P. Simoncelli. A Convolutional Subunit Model for Neuronal Re- sponses in Macaque V1. The Journal of neuroscience : the official journal of the Society for Neuroscience, 35(44):14829–41, 2015.
2016:
Cichy et al (2016) - Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence – Sci Reports
https://www.nature.com/articles/srep27755
Güçlü et al (2016) – Brains on beats – NeurIPS
https://papers.nips.cc/paper/6222-brains-on-beats.pdf
Hong et al (2016) – Explicit information for category-orthogonal object properties increases along the ventral stream – Nature Neuroscience
https://www.nature.com/articles/nn.4247
Yamins & DiCarlo (2016) – Using goal-driven deep learning models to understand sensory cortex – Nature Neuroscience
https://www.nature.com/articles/nn.4244
J. Antolík, S. B. Hofer, J. A. Bednar, and T. D. Mrsic-flogel. Model Constrained by Visual Hierarchy Improves Prediction of Neural Responses to Natural Scenes. PLoS Comput Biol, pages 1–22, 2016. ISSN 1553-7358. doi: 10.1371/journal.pcbi.1004927.
E. Batty, J. Merel, N. Brackbill, A. Heitman, A. Sher, A. Litke, E. J. Chichilnisky, and L. Panin- ski. Multilayer network models of primate retinal ganglion cells. Nips, 2016.
2017:
Cichy et al (2017) – Dynamics of scene representations in the human brain revealed by magnetoencephalography and deep neural networks – Neuroimage
https://doi.org/10.1016/j.neuroimage.2016.03.063
Eickenberg et al (2017) – Seeing it all: Convolutional network layers map the function of the human visual system – Neuroimage
https://doi.org/10.1016/j.neuroimage.2016.10.001
Güçlütürk, Güçlü, Seeliger, Bosch, van Lier, van Gerven (2017) – Reconstructing perceived faces from brain activations with deep adversarial neural decoding – NeurIPS
Güçlü & van Gerven (2017) – Modeling the Dynamics of Human Brain Activity with Recurrent Neural Networks – Front Comput Neurosci
https://dx.doi.org/10.3389%2Ffncom.2017.00007
Horikawa & Kamitani (2017) – Hierarchical neural representation of dreamed objects revealed by brain decoding with deep neural network features – Front Comput Neurosci https://doi.org/10.3389/fncom.2017.00004
Horikawa & Kamitani (2017) – Generic decoding of seen and imagined objects using hierarchical visual features – Nat Commun
https://dx.doi.org/10.1038%2Fncomms15037
Kalfas, Kumar, Vogels (2017) – Shape Selectivity of Middle Superior Temporal Sulcus Body Patch Neurons – eNeuro
https://doi.org/10.1523/ENEURO.0113-17.2017
Karimi-Rouzbahani et al (2017) - Hard-wired feed-forward visual mechanisms of the brain compensate for affine variations in object recognition - Journal of Neuroscience
https://doi.org/10.1016/j.neuroscience.2017.02.050
Khaligh-Razavi et al (2017) – Fixed versus mixed RSA: Explaining visual representations by fixed and mixed feature sets from shallow and deep computational models – J Math Psych
https://doi.org/10.1016/j.jmp.2016.10.007
Klindt, Ecker, Euler, Bethge (2017) – Neural system identification for large populations separating “what” and “where” – NeurIPS
Ratan Murty & Arun (2017) – A balanced comparison of object invariances in monkey IT neurons – eNeuro
https://doi.org/10.1523/ENEURO.0333-16.2017
Scholte et al (2017) - Visual pathways from the perspective of cost functions and multi-task deep neural networks - Cortex
https://doi.org/10.1016/j.cortex.2017.09.019
Seeliger et al (2017) – Convolutional neural network-based encoding and decoding of visual object recognition in space and time – Neuroimage
https://doi.org/10.1016/j.neuroimage.2017.07.018
Seeliger et al (2017) - Generative adversarial networks for reconstructing natural images from brain activity. NeuroImage 181 http://www.sciencedirect.com/science/article/pii/S105381191830658X
(preprint: https://doi.org/10.1101/226688 )
Tacchetti, Isik, Poggio (2017) – Invariant recognition drives neural representations of action sequences – PloS Comput Biology
https://doi.org/10.1371/journal.pcbi.1005859
Tripp et al (2017) – Similarities and differences between stimulus tuning in the inferotemporal visual cortex and convolutional networks - IJCNN
https://arxiv.org/abs/1612.06975
Wen et al (2017) – Neural Encoding and Decoding with Deep Learning for Dynamic Natural Vision – Cerebral Cortex
https://doi.org/10.1093/cercor/bhx268
Zhuang, Wang, Yamins, Hu (2017) – Deep learning predicts correlation between a functional signature of higher visual areas and sparse firing of neurons – Front Comput Neurosci
https://doi.org/10.3389/fncom.2017.00100
2018:
Abdelhack, Kamitani (2018) – Sharpening of hierarchical visual feature representations of blurred images – eNeuro
https://doi.org/10.1523/ENEURO.0443-17.2018
Bankson, Hebart, Groen, Baker (2018) – The temporal evolution of conceptual object representations revealed through models of behavior, semantics and deep neural networks – Neuroimage
https://doi.org/10.1016/j.neuroimage.2018.05.037
Banino et al (2018) - Vector-based navigation using grid-like representations in artificial agents - Nature
https://doi.org/10.1038/s41586-018-0102-6
Bonner, Epstein (2018) – Computational mechanisms underlying cortical responses to the affordance properties of visual scenes – PloS Computational Biology
https://doi.org/10.1371/journal.pcbi.1006111
Devereux, Clarke, Tyler (2018) - Integrated deep visual and semantic attractor neural networks predict fMRI pattern-information along the ventral object processing pathway
- Scientific Reports https://doi.org/10.1038/s41598-018-28865-1
- biorXiv: https://doi.org/10.1101/302406
Dezfouli, A., Morris, R., Ramos, F. T., Dayan, P., & Balleine, B. (2018). Integrated accounts of behavioral and neuroimaging data using flexible recurrent neural network models. In Advances in Neural Information Processing Systems (NeurIPS; pp. 4228-4237).
https://www.biorxiv.org/content/10.1101/328849v2
Fong, Scheirer, Cox (2018) - Using human brain activity to guide machine learning - Scientific Reports
https://doi.org/10.1038/s41598-018-23618-6
Greene & Hansen (2018) - Shared spatiotemporal category representations in biological and artificial deep neural networks - PloS Comput Biol
https://doi.org/10.1371/journal.pcbi.1006327
Groen et al (2018) – Distinct contributions of functional and deep neural network features to scene representation in brain and behavior – eLife
https://doi.org/10.7554/eLife.32962
Kell, Yamins, Shook, Norman-Haignere, McDermott (2018) – A task-optimized neural network replicates human auditory behavior, predicts brain responses, and reveals a cortical processing hierarchy – Neuron
https://doi.org/10.1016/j.neuron.2018.03.044
Kuzovkin, Vicente, Petton, Lachaux, Baciu, Kahane, Rheims, Vidal, and Aru. - Activations of deep convolutional neural networks are aligned with gamma band activity of human visual cortex - Communications biology
https://doi.org/10.1038/s42003-018-0110-y
O’Connell and Chun (2018) – Predicting eye movements from fMRI responses to natural scenes – Nature Communications
https://doi.org/10.1038/s41467-018-07471-9
Rajalingham et al (2018): Large-Scale, Comparison of the Core Visual Object Recognition Behavior of Humans, Monkeys, and State-of-the-Art Deep Artificial Neural Networks. Journal of Neuroscience 38. (preprint: http://dx.doi.org/10.1101/240614 )
http://www.jneurosci.org/content/38/33/7255
Ratan Murty, Arun (2018) – Multiplicative mixing of object identity and image attributes in single inferior temporal neurons – PNAS
https://doi.org/10.1073/pnas.1714287115
Shi et al (2018) – Deep recurrent neural network reveals a hierarchy of process memory during dynamic natural vision – Human Brain Mapping
http://dx.doi.org/10.1002/hbm.24006
Sinz, Ecker, Fahey, Walker, Cobos, Froudarakis, Yatsenko, Pitkow, Reimer, Tolias Stimulus domain transfer in recurrent models for large scale cortical population prediction on video -- NeurIPS https://papers.nips.cc/paper/7950-stimulus-domain-transfer-in-recurrent-models-for-large-scale-cortical-population-prediction-on-video.pdf
Tang, Schrimpf, Lotter, Moerman, Paredes, Caro, Hardesty, Cox, and Kreiman (2018) – Recurrent computations for visual pattern completion –
Wen et al (2018) – Deep Residual Network Reveals a Nested Hierarchy of Distributed Cortical Representation for Visual Categorization – Scientific Reports
https://www.nature.com/articles/s41598-018-22160-9
Wen et al (2018) – Transferring and Generalizing Deep-Learning-based Neural Encoding Models across Subjects –
Wenliang, Seitz (2018) - Deep neural networks for modeling visual perceptual learning - J Neurosci
https://doi.org/10.1523/JNEUROSCI.1620-17.2018
Zhang, Qiao, Wang, Tong, Zeng, Yan (2018) – Constraint-free natural image reconstruction from fMRI signals based on convolutional neural network –
2019:
Bashivan, Kar, DiCarlo (2019) - Neural population control via deep image synthesis -
Cadena, Sinz, Muhammad, Froudarakis, Cobos, Walker, Reimer, Bethge, Tolias, Ecker. "How well do deep neural networks trained on object recognition characterize the mouse visual system?." (2019). https://openreview.net/forum?id=rkxcXmtUUS
Cadena, Denfield, Walker, Gatys, Tolias, Bethge, Ecker (2019) - Deep convolutional models improve predictions of macaque V1 responses to natural images - PLoS Computational Biology
https://doi.org/10.1371/journal.pcbi.1006897
Preprint: https://doi.org/10.1101/201764
Kar, Kubilius, Schmidt, Issa, DiCarlo (2019). Evidence that recurrent circuits are critical to the ventral stream’s execution of core object recognition behavior. Nature Neuroscience
https://doi.org/10.1038/s41593-019-0392-5
Preprint: https://doi.org/10.1101/354753
Kietzmann, T.C., Spoerer, C.J., Sörensen, L., Cichy, R.M., Hauk, O., & Kriegeskorte, N. (2019). Recurrence is required to capture the representational dynamics of the human visual system. Proceedings of the National Academy of Sciences, p. 1-10
https://doi.org/10.1073/pnas.1905544116
Kubilius*, Schrimpf*, Kar, Hong, Majaj, Rajalingham, Issa, Bashivan, Prescott-Roy, Schmidt, Nayebi, Bear, Yamins, DiCarlo - Brain-Like Object Recognition with High-Performing Shallow Recurrent ANNs - NeurIPS 2019 (Oral)
https://papers.nips.cc/paper/9441-brain-like-object-recognition-with-high-performing-shallow-recurrent-anns
Ponce, C., Xiao, W., Schade, P., Hartmann, T., Kreiman, G., Livingstone, M. (2019). Evolving Images for Visual Neurons Using a Deep Generative Network Reveals Coding Principles and Neuronal Preferences Cell 177(4), 999-1009.e10.
Sharmistha Jat, Hao Tang, Partha Talukdar, Tom M Mitchell -Relating Simple Sentence Representations in Deep Neural Networks and Brain -ACL 2019
Link: https://www.aclweb.org/anthology/P19-1507/
Shen, G., Horikawa, T., Majima, K., & Kamitani, Y. (2019). Deep image reconstruction from human brain activity -
Ukita, Yoshida, Ohki, K (2019) - Characterisation of nonlinear receptive fields of visual neurons by convolutional neural network - Scientific Reports https://www.nature.com/articles/s41598-019-40535-4
VanRullen & Reddy (2019) – Reconstructing faces from fMRI patterns using deep generative neural networks –
Walker, Sinz, Cobos, Muhammad,Froudarakis, Fahey, Ecker, Reimer, Pitkow, & Tolias (2019) - Inception loops discover what excites neurons most using deep predictive models -
2020
Zeman, A., Ritchie, J. B., Bracci, S., & de Beeck, H. O. (2020). Orthogonal Representations of Object Shape and Category in Deep Convolutional Neural Networks and Human Visual Cortex.
Dwivedi, Cichy, and Roig (2020). Unraveling Representations in Scene-selective Brain Regions Using Scene-Parsing Deep Neural Networks
Preprints:
Agrawal et al (2014) - Pixels to Voxels: Modeling Visual Representation in the Human Brain - arXiv https://arxiv.org/pdf/1407.5104.pdf
Bracci, Kalfas, Op de Beeck (2017) – The ventral visual pathway represents animal appearance over animacy, unlike human behavior and deep neural networks – bioRxiv
https://doi.org/10.1101/228932
Burg, Cadena, Denfield, Walker, Tolias, Bethge, Ecker. "Learning Divisive Normalization in Primary Visual Cortex." bioRxiv (2019): 767285. https://www.biorxiv.org/content/10.1101/767285v3
Cichy et al (2017) - Neural dynamics of real-world object vision that guide behaviour - bioRxiv
https://www.biorxiv.org/content/early/2017/06/08/147298
Dwivedi, Roig (2018) - Task-specific vision models explain task-specific areas of visual cortex - bioRxiv
https://www.biorxiv.org/content/early/2018/08/28/402735
Gwilliams & King (2017) - Performance-optimized hierarchical models only partially predict neural responses during perceptual decision making - bioRxiv
https://www.biorxiv.org/content/early/2017/11/20/221630.full
King, Groen, Steel, Kravitz, Baker (2018) - Similarity judgments and cortical visual responses reflect different properties of object and scene categories in naturalistic images – bioRxiv
https://doi.org/10.1101/316554
Kindel, Christensen, Zylberberg (2017) – Using deep learning to reveal the neural code for images in primary visual cortex – arxiV
https://arxiv.org/abs/1706.06208
Kuzovkin et al (2017) - Activations of Deep Convolutional Neural Network are Aligned with Gamma Band Activity of Human Visual Cortex - BioRxiv
https://doi.org/10.1101/133694
Long et al (2017) - A mid-level organization of the ventral stream - BioRXiv
https://doi.org/10.1101/213934
Lotter, Kreiman, Cox (2018) – A neural network trained to predict future video frames mimics critical properties of biological neuronal responses and perception – arXiv
https://arxiv.org/abs/1805.10734
Mehrer, J., Spoerer, C.J., Kriegeskorte, N., & Kietzmann, T.C. (2019). Individual differences among deep neural network models. bioRxiv, 898288
https://doi.org/10.1101/2020.01.08.898288
Nayebi, Bear, Kubilius, Kar, Ganguli, Sussillo, DiCarlo, Yamins (2018) - Task-Driven Convolutional Recurrent Models of the Visual System - arXiv
https://arxiv.org/abs/1807.00053
O’Connell & Chun (2017) - Predicting eye movements with deep neural network activity decoded from fMRI responses to natural scenes - bioRxiv
https://doi.org/10.1101/166421
Pinotsis, Siegel, Miller (2019) - Sensory processing and categorization in cortical and deep neural networks - bioRxiv
https://doi.org/10.1101/647222
Qiao, Zhang, Wang, Yan, Chen, Zeng, Tong (2018) – Accurate reconstruction of image stimuli from human fMRI based on the decoding model with capsule network architecture – arXiv
https://arxiv.org/abs/1801.00602
Rajaei, Mohsenzadeh, Ebrahimpour, Khaligh-Razavi (2018) – Beyond core object recognition: Recurrent processes account for object recognition under occlusion – bioRxiv
https://doi.org/10.1101/302034
Ramakrishnan et al (2017) - Characterizing the temporal dynamics of object recognition by deep neural networks: role of depth - bioRxiv
https://doi.org/10.1101/178541
Schrimpf*, Kubilius*, Hong, Majaj, Rajalingham, Issa, Kar, Bashivan, Prescott-Roy, Schmidt, Yamins, DiCarlo - Brain-Score: Which Artificial Neural Network for Object Recognition is most Brain-Like? - bioRxiv 2018
https://www.biorxiv.org/content/10.1101/407007v1
Kshitij Dwivedi, Michael F. Bonner, Radoslaw Martin Cichy, Gemma Roig (2020) –
Unveiling functions of the visual cortex using task-specific deep neural networks – bioRxiv
https://www.biorxiv.org/content/10.1101/2020.11.27.401380v1
2014:
Ghodrati et al (2014) – Feedforward object-vision models only tolerate small image variations compared to human – Front Comput Neurosci
https://dx.doi.org/10.3389%2Ffncom.2014.00074
2015:
Lake et al (2015) - Deep neural networks predict category typicality ratings for images - Cogn Psych
http://gureckislab.org/papers/LakeZarembaFergusGureckis.CogSci2015.pdf
Nguyen et al (2015) – Deep Neural Networks Are Easily Fooled: High Confidence Predictions for Unrecognizable Images – IEEE Comput Vis Patt Recog
Rajalingham et al (2015) – Comparison of object recognition behavior in human and monkey. Journal of Neuroscience
http://www.jneurosci.org/content/35/35/12127
2016:
Eberhardt et al (2016) – How deep is the feature analysis underlying rapid visual categorization – NIPS
Farzmahdi et al (2016) – A specialized face-processing model inspired by the organization of monkey face patches explains several face-specific phenomena observed in humans – Sci Rep
https://dx.doi.org/10.1038%2Fsrep25025
Greene et al (2016) – Visual Scenes Are Categorized by Function – JEP:General
http://psycnet.apa.org/fulltext/2015-58122-004.html
Kheradpisheh et al (2016) – Deep Networks Can Resemble Human Feed-forward Vision in Invariant Object Recognition – Sci Rep
https://www.nature.com/articles/srep32672
Kheradpisheh et al (2016) – Humans and Deep Networks Largely Agree on Which Kinds of Variation Make Object Recognition Harder – Front Comput Neurosci
https://dx.doi.org/10.3389%2Ffncom.2016.00092
Kubilius et al (2016) – Deep Neural Networks as a Computational Model for Human Shape Sensitivity – PloS Comput Biol
https://doi.org/10.1371/journal.pcbi.1004896
2017:
Jozwik et al (2017) – Deep Convolutional Neural Networks Outperform Feature-Based But Not Categorical Models in Explaining Object Similarity Judgments – Front Psychol
https://doi.org/10.3389/fpsyg.2017.01726
Karimi-Rouzbahani, Bagheri & Ebrahimpour (2017) – Invariant object recognition is a personalized selection of invariant features in humans, not simply explained by hierarchical feed-forward vision models – Scientific Reports
https://doi.org/10.1038/s41598-017-13756-8
Love et al (2017) – Deep Networks as Models of Human and Animal Categorization - Cogn Psych
https://mindmodeling.org/cogsci2017/papers/0283/paper0283.pdf
Lukavský, J., & Děchtěrenko, F. (2017). Visual properties and memorising scenes: Effects of image-space sparseness and uniformity. Attention, Perception, & Psychophysics, 79(7), 2044–2054.
https://doi.org/10.3758/s13414-017-1375-9
Song, Yang, Wang (2017) – Reward-based training of recurrent neural networks for cognitive and value-based tasks – eLife
https://dx.doi.org/10.7554%2FeLife.21492
Suzuki et al (2017) - A Deep-Dream Virtual Reality Platform for Studying Altered Perceptual Phenomenology - Scientific Reports
https://doi.org/10.1038/s41598-017-16316-2
Wallis et al (2017) – A parametric texture model based on deep convolutional features closely matches texture appearance for humans – JoV
http://jov.arvojournals.org/article.aspx?articleid=2657215
Wichmann, F. A., Janssen, D. H., Geirhos, R., Aguilar, G., Schütt, H. H., Maertens, M., & Bethge, M. (2017). Methods and measurements to compare men against machines. Electronic Imaging, 2017(14), 36-45.
https://doi.org/10.2352/ISSN.2470-1173.2017.14.HVEI-113
Tang, Schrimpf, Lotter, et al (2017) - Recurrent computations for visual pattern completion - arXiv https://arxiv.org/abs/1706.02240
2018:
Baker, Lu, Erlikhman, Kellman (2018) – Deep convolutional networks do not classify based on global object shape – PLoS Computational Biology
https://doi.org/10.1371/journal.pcbi.1006613
Flesch, Balaguer, Dekker, Nili, & Summerfield (2018) – Comparing continual task learning in minds and machines – PNAS
https://doi.org/10.1073/pnas.1803839115
Geirhos, Medina Temme, Rauber, Schütt, Bethge, Wichmann (2018) - Generalisation in humans and deep neural networks - NeurIPS
https://arxiv.org/abs/1808.08750
Masse, Grant, & Freedman (2018) – Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization – PNAS
https://doi.org/10.1073/pnas.1803839115
Ponce, Lomber, Livingstone (2018) - Posterior Inferotemporal Cortex Cells Use Multiple Input Pathways for Shape Encoding - Journal of Neuroscience
http://www.jneurosci.org/content/37/19/5019
Rajalingham et al (2018): Large-Scale, High-Resolution Comparison of the Core Visual Object Recognition Behavior of Humans, Monkeys, and State-of-the-Art Deep Artificial Neural Networks. Journal of Neuroscience 38. (preprint: http://dx.doi.org/10.1101/240614 )
http://www.jneurosci.org/content/38/33/7255
Watanabe et al (2018) - Illusory Motion Reproduced by Deep Neural Networks Trained for Prediction - Frontiers in Psychology
https://doi.org/10.3389/fpsyg.2018.00345
Wenliang, Seitz (2018) - Deep neural networks for modeling visual perceptual learning - J Neurosci
https://doi.org/10.1523/JNEUROSCI.1620-17.2018
Xu, Garrod, Scholte, Ince, Schyns (2018) - Using Psychophysical Methods to Understand Mechanisms of Face Identification in a Deep Neural Network - CVPRW
Dezfouli, A., Griffiths, K., Ramos, F., Dayan, P., & Balleine, B. W. (2019). Models that learn how humans learn: the case of decision-making and its disorders. PLoS computational biology, 15(6), e1006903.
https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1006903
Dezfouli, A., Morris, R., Ramos, F. T., Dayan, P., & Balleine, B. (2018). Integrated accounts of behavioral and neuroimaging data using flexible recurrent neural network models. In Advances in Neural Information Processing Systems (NeurIPS) (pp. 4228-4237).
https://www.biorxiv.org/content/10.1101/328849v2
2019:
Dezfouli, A., Ashtiani, H., Ghattas, O., Nock, R., Dayan, P., & Ong, C. S. (2019). Disentangled behavioral representations. In Advances in Neural Information Processing Systems (NeurIPS).
Geirhos; Rubisch, Michaelis, Bethge, Wichmann, & Brendel (2018) - ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness
ICLR 2019 (Oral) https://openreview.net/forum?id=Bygh9j09KX
Kubilius*, Schrimpf*, Kar, Hong, Majaj, Rajalingham, Issa, Bashivan, Prescott-Roy, Schmidt, Nayebi, Bear, Yamins, DiCarlo - Brain-Like Object Recognition with High-Performing Shallow Recurrent ANNs - NeurIPS 2019 (Oral)
https://papers.nips.cc/paper/9441-brain-like-object-recognition-with-high-performing-shallow-recurrent-anns
Lindh, D., Sligte, I.G., Assecondi, S. et al. Conscious perception of natural images is constrained by category-related visual features. Nat Commun 10, 4106 (2019) doi:10.1038/s41467-019-12135-3
Zhou, Z., & Firestone, C. (2019). Humans can decipher adversarial images. Nature Communications, 10, 1334.
https://www.nature.com/articles/s41467-019-08931-6
2020:
Doerig, Bornet, Choung, & M. H. Herzog (2020) - Crowding Reveals Fundamental Differences in Local vs. Global Processing in Humans and Machines
Vision Research; doi: https://doi.org/10.1016/j.visres.2019.12.006
Geirhos, R., Meding, K. and Wichmann, F.A. (2020). Beyond accuracy: quantifying trial-by-trial behaviour of CNNs and humans by measuring error consistency - NeurIPS 2020 https://arxiv.org/pdf/2006.16736.pdf
Preprints:
Peterson et al (2016) – Adapting deep network features to capture psychological representations – ArXiv
https://arxiv.org/abs/1608.02164
Battleday, Peterson, Griffiths (2017) – Modeling Human Categorization of Natural Images Using Deep Feature Representations - ArXiv
https://arxiv.org/abs/1711.04855
Dekel (2017) - Human perception in computer vision - arXiv
https://arxiv.org/abs/1701.04674
Doerig, Schmittwilken, Sayim, Manassi, & Herzog (2019) - Capsule Networks as Recurrent Models of Grouping and Segmentation
bioRxiv 747394; doi: https://doi.org/10.1101/747394
Fan, Yamins, & Turk-Browne (2017) – Common object representations for visual production and recognition – bioRxiv
https://www.biorxiv.org/content/early/2017/01/03/097840
Geirhos, Janssen, Schütt, Rauber, Bethge, & Wichmann (2017) – Comparing deep neural networks against humans: object recognition when the signal gets weaker – ArXiv
https://arxiv.org/abs/1706.06969
Lin et al (2017) – Transfer of view-manifold learning to similarity perception of novel objects – ArXiv
https://arxiv.org/abs/1704.00033
Lindsay & Miller (2017) - Understanding Biological Visual Attention Using Convolutional Neural Networks – biorXiv
https://www.biorxiv.org/content/early/2017/12/20/233338
Fruend, Stalker (2018) – Measuring human sensitivity to perturbations within the manifold of natural images – bioRxiv
https://doi.org/10.1101/320531
Leibo et al (2018) - Psychlab: A Psychology Laboratory for Deep Reinforcement Learning Agents - arXiv
https://arxiv.org/abs/1801.08116
Linsley, Scheibler, Eberhardt, Serre (2018) – Global-and-local attention networks for visual recognition – arXiv
https://arxiv.org/abs/1805.08819
Hugo Richard, Ana Pinho, Bertrand Thirion, Guillaume Charpiat (2018) – Optimizing deep video representations to match brain activity – arXiv
https://arxiv.org/abs/1809.02440
Rosenfeld, Solbach, Tsotsos (2018) – Totally Looks Like - How Humans Compare, Compared to Machines – arXiv
https://arxiv.org/abs/1803.01485
Wallis, T. S. A., Funke, C. M., Ecker, A. S., Gatys, L. A., Wichmann, F. A., & Bethge, M. (2018). Image content is more important than Boumas Law for scene metamers. BioRxiv. https://doi.org/10.1101/378521
Fruend, I. Simple, biologically informed models, but not convolutional neural networks describe target detection in naturalistic images. bioRxiv. https://doi.org/10.1101/578633
Kim, B., Reif, E., Wattenberg, M., Bengio, S. (2019). Do neural networks show gestalt phenomena? An exploration of the law of closure. https://arxiv.org/abs/1903.01069
Spoerer, C.J., Kietzmann, T.C., & Kriegeskorte, N. (2019). Recurrent networks can recycle neural resources to flexibly trade speed for accuracy in visual recognition. bioRxiv, 677237
https://www.biorxiv.org/content/10.1101/677237v3
Koch, G.E., Akpan, E., & Coutanche, M.N. (2020). Image memorability is predicted at different stages of a convolutional neural network. bioRxiv. https://www.biorxiv.org/content/10.1101/834796v2
(everything else, obviously incomplete)
Fukushima, K. (1988). Neocognitron: A hierarchical neural network capable of visual pattern recognition. Neural networks, 1(2), 119-130.
Zipser, D., & Andersen, R. A. (1988). A back-propagation programmed network that simulates response properties of a subset of posterior parietal neurons. Nature, 331(6158), 679. https://www.nature.com/articles/331679a0
van Gerven, Marcel AJ, Floris P. de Lange, and Tom Heskes (2010). "Neural decoding with hierarchical generative models." Neural computation 22.12 (2010): 3127-3142.
Zeman et al (2013) The Müller-Lyer Illusion in a Computational Model of Biological Object Recognition. PLoS ONE8(2): e56126. https://doi.org/10.1371/journal.pone.0056126
Zeman et al (2014) - Complex cells decrease errors for the Müller-Lyer illusion in a model of the visual ventral stream - Front. Comput. Neurosci. 8:112. https://doi.org/10.3389/fncom.2014.00112
Ramakrishnan K, Scholte HS, Groen IIA, Smeulders AWM, Ghebreab S (2015) -Visual dictionaries as intermediate features in the human brain - Front Comput Neurosci
https://www.frontiersin.org/articles/10.3389/fncom.2014.00168/full
(HMAX vs bag-of-words tested against fMRI)
Tang, Schrimpf, Lotter, et al (2017) - Recurrent computations for visual pattern completion - arXiv https://arxiv.org/abs/1706.02240
S. Nishimoto and J. L. Gallant. A Three-Dimensional Spatiotemporal Receptive Field Model Explains Responses of Area MT Neurons to Naturalistic Movies. Journal of Neuroscience, 31 (41):14551–14564, 2011. ISSN 0270-6474. doi: 10.1523/JNEUROSCI.6801-10.2011. URL http://www.jneurosci.org/cgi/doi/10.1523/JNEUROSCI.6801-10.2011
R. Prenger, M. C. K. Wu, S. V. David, and J. L. Gallant. Nonlinear V1 responses to natural scenes revealed by neural network analysis. Neural Networks, 17(5-6):663–679, 2004. ISSN 08936080. doi: 10.1016/j.neunet.2004.03.008.
B. Lau, G. B. Stanley, and Y. Dan. Computational subunits of visual cortical neurons revealed by artificial neural networks. Proceedings of the National Academy of Sciences, 99(13):8974–8979, 2002. ISSN 0027-8424. doi: 10.1073/pnas.122173799. URL http://www.pnas.org/cgi/ doi/10.1073/pnas.122173799.
S. R. Lehky, T. J. Sejnowski, and R. Desimone. Predicting responses of nonlinear neurons in monkey striate cortex to complex patterns. The Journal of Neuroscience, 12(9):3568– 3581, 1992. ISSN 0270-6474. URL http://www.jneurosci.org/content/12/9/3568. short{%}5Cnhttp://www.ncbi.nlm.nih.gov/pubmed/1527596.