fine tuning worklog
trying to learn about
- freezing hierarchy
- lr disrimination
- layernorm freezing
- optimizer selection
the setup
take a pre-trained MiniMAE (masked auto encoder) trained on FashionMNIST dataset and then use it for downstream task of classification and experiment with the above mentioned learning goals
resources: