Neuroscience-Inspired Encoding and Learning: A Path to Robust Representation Learning

Abstract

We present a neuroscience-inspired strategy to improve the robustness of encoder models by combining Artificial Kuramoto Oscillatory Neurons (AKOrN) with PhiNet, a non-contrastive self-supervised learning approach.

AKOrN models neurons as interacting oscillators using the Kuramoto framework, while PhiNet draws inspiration from the hippocampus and temporal prediction theory to build stable, generalizable representations without relying on negative pairs.

Our motivation stems from the challenge of learning robust representations in domains such as brain imaging, where datasets are small, noisy, and highly variable. Conventional vision models tend to overfit and are vulnerable to adversarial perturbations. By integrating the oscillatory dynamics of AKOrN with the predictive structure of PhiNet, we aim to obtain models that are inherently more robust without the need for adversarial training, synthetic augmentation, or heavy post-training procedures.

Key results

Our experiments evaluate AKOrN encoders trained with several self-supervised learning methods on CIFAR-10:

This work was carried out during my research internship at OIST in the Machine Learning and Data Science Unit of Prof. Makoto Yamada. An extended version is available as an arXiv preprint.

Citation

Habibi, M.-Y., Ziu, K., & Yamada, M. (2025). "Neuroscience-Inspired Encoding and Learning: A Path to Robust Representation Learning." Women in Machine Learning (WiML) Workshop @ NeurIPS 2025.

@inproceedings{habibi2025neuroscience,
  title     = {Neuroscience-Inspired Encoding and Learning: A Path to Robust Representation Learning},
  author    = {Habibi, Mohammed-Yassine and Ziu, Klea and Yamada, Makoto},
  booktitle = {Women in Machine Learning (WiML) Workshop @ NeurIPS},
  year      = {2025},
  url       = {https://openreview.net/forum?id=3LERuKSX1B}
}