39 lines
2.1 KiB
Markdown
39 lines
2.1 KiB
Markdown
# MSc Thesis — QK-Norm at Small Scale
|
||
|
||
**Working title:** When Does a Small Language Model Need QK-Norm? An Isolated
|
||
Learning-Rate Characterization of Query–Key Normalization at <100M Parameters
|
||
|
||
Organised copy of the project, made 28/10/2026, and tracked in its own git repo (no remote yet).
|
||
The originals are still in the parent folder, and the CS6513 git repo
|
||
(`../Research Project/`, github.com/NikScorch/MSc_Research_Project) is untouched.
|
||
|
||
Not in git (see `.gitignore`): the pitch video (52 MB), `05-course-materials/`, datasets,
|
||
checkpoints, the API cache and the AI-assisted `ref_draft`.
|
||
|
||
## Layout
|
||
|
||
| Folder | What's in it |
|
||
|---|---|
|
||
| `00-supervisor/` | Supervisor meeting notes and to-dos, `timeline.md` |
|
||
| `01-proposal/` | CS6513 proposal pitch: Marp slides (`pitch-slides.md`), script, exported PDF, recorded video |
|
||
| `02-literature/` | Lit review: `protocol.md`, search log, screening sheet, extraction table, `references.bib` — see its README |
|
||
| `03-cs6513-paper/` | CS6513 Introduction + Lit Review paper (IEEE format), see below |
|
||
| `04-cs6513-notebook/` | CS6513 research notebook (`.rmd` source + exported PDF) |
|
||
| `05-course-materials/` | CS6513 lecture slides and set readings (wk1 methods, wk2 lit review incl. Xiao 2017 SLR guidance) |
|
||
| `06-experiments/` | nanoGPT + TinyStories, QK-norm toggle, automatic run logging, `preregistration.md` — see its README |
|
||
| `AI-USE-LOG.md` | GenAI transparency log |
|
||
|
||
### `03-cs6513-paper/` — which file is which
|
||
|
||
| File | Status |
|
||
|---|---|
|
||
| `draft.md` → `draft.tex` / `draft.pdf` | **The submitted paper.** Cormac's own prose; `.tex` is a format conversion of the `.md` |
|
||
| `main.tex` / `main.pdf` | Earlier bullet-point scaffold (AI-assisted outline, not submitted) |
|
||
| `ref_draft.tex` / `ref_draft.pdf` | Personal AI-assisted reference draft — never for submission; kept out of the git repo |
|
||
| `build.sh`, `IEEEtran.cls`, `README.md` | Build tooling: `./build.sh draft` |
|
||
|
||
## Not copied
|
||
|
||
- `../Research Project/msc-research-notebook-outline.md` — byte-identical duplicate of the `.rmd`
|
||
- `.obsidian/`, `.DS_Store`, `.git/` — editor/system/VCS metadata
|