Research proposal · Author-reviewed
A data queue with quarantine: proposal for online data selection in GoLLeM-v6
No results
Fabryka AI / Research agenda 2026
Questions worth measuring.
Three connected programmes — from learning with limited compute to measuring the systems we deploy. Current artifacts and planned studies are identified separately.
01 — FAB-SLM-001 / SMALL LANGUAGE MODELS
Scaling data mixtures and distillation for compact language models
RESEARCH QUESTION
How do data composition, tokenization and teacher supervision affect model quality under a fixed parameter and training-compute budget?
STUDY DESIGN
Matched training runs on fixed evaluation splits; vary one factor at a time; record tokenizer, data revisions, tokens, seeds and compute.SCOPE & COMPUTE
Sub-150M models first. 1B and 10B studies are roadmap directions, subject to compute and data readiness.Fig. 1 — Loss vs. training compute, by data recipe
SCHEMATIC · STUDY DESIGN, NOT RESULTS
A · baseline web mix
B · + curated Polish data
C · + teacher distillation
planned run
Fig. 2 — One-factor-at-a-time ablation grid
Each run changes exactly one factor against the R0 control. Everything else is logged and held fixed.
R0 ctrl
R1
R2
R3
R4
Tokenizer
PL-32k
Data mixture
+PL
Teacher
KD
Seed
×3
02 — FAB-SYS-001 / EFFICIENT INFERENCE
Characterizing LLM inference on commodity GPUs
RESEARCH QUESTION
How do memory bandwidth, attention implementations, quantization and decoding strategies affect latency, throughput, memory use and output quality?
STUDY DESIGN
Same model and workload across documented hardware and software. Measure prefill, TTFT, decode speed and latency distributions under specified concurrency. Quality is evaluated separately from speed.SCOPE
Quantization, speculative decoding incl. DFlash, routing, commodity GPUs. Every result states context length, batch size and load.Fig. 3 — Anatomy of a request: what we time
MEASUREMENT PROTOCOL
Fig. 4 — Report distributions, not means
End-to-end latency under fixed concurrency (illustrative shape)
Fig. 5 — Configuration matrix
Same model, same workload, every cell
FP16INT8INT4+spec
RTX 3090
RTX 4090
RTX 5090
L4
planned cell
DFlash priority
03 — FAB-DATA-001 / POLISH AI EVALUATION & DATA
Polish data and reproducible AI evaluation
RESEARCH QUESTION
How do dataset composition and evaluation design affect our estimates of model capability in Polish, including document and OCR tasks?
STUDY DESIGN
Document provenance, licensing, splits and scoring. Check contamination and reference quality. Use task-appropriate metrics and matched protocols; separate our measurements from externally reported results.Fig. 6 — From raw source to reported score
Every stage leaves an inspectable record.
01SourcesWeb, documents, scans
02ProvenanceLicence & origin per record
03Decontam.Dedup vs. eval sets
04SplitsFrozen train / dev / test
05ScoringTask-matched metric
Bar widths indicate stage order, not measured volumes.
Fig. 7 — Evaluation coverage map
public
in agenda
planned
CORPUSPolish text · DynaWord
BENCHMARKSCodeSOTA
LLM EVALPolish language understanding
OCRPrinted documents
OCRHandwriting & archival scans
DOCUMENTSLayout & form extraction
FABRYKA AI NOTES / REPORTS & PROPOSALS
Research in writing.
Technical reports and proposals shared by SlayerLab, Fabryka AI and collaborators. Each entry retains its authors, version and result status from the source repository.
Technical report · Draft
GoLLeM-v5: Small Language Models for the Glint Tiny-ML Leaderboard. Measurement, Method and Negative Results
Preliminary results
“Author-reviewed” is the repository’s review status; it does not indicate journal peer review. Drafts and proposals are works in progress. See each entry’s metadata for licensing; an undecided license does not grant reuse rights.
Browse Fabryka AI Notes on GitHub ↗OPEN RESEARCH / ACROSS THE REPOSITORIES
More questions.
More evidence.
Nine selected notes across six public repositories: small models, Polish tokenizers, evaluation and data preparation. Browse measured results, provisional pilots and technical methods with their source records.
RESEARCH ATLAS / FOLLOW THE CONNECTIONS
What builds on what?
From data and representation to measurement and decisions. Select a note to see the connection, its purpose and the source evidence.
Documented reuseConceptual connectionArrows show relationships, not a timeline.
Scroll the diagram sideways, or select a note below.
01 / REPRESENTATION
Polish BPE
A handwritten byte-level BPE is compared with Hugging Face variants at the same 6,756-token vocabulary. The question is how implementation and pretokenisation change compression.
The reported range is 833–844 tokens on a small held-out. It does not establish a downstream model-quality gain.
Read this research note ↗What connects it — and why
- Conceptual connection
Polish BPE → Probierz
Different token counts → a common unit
The BPE experiment shows that token counts depend on tokenisation. This motivates looking beyond per-token quantities when comparing models. The recorded panel uses bits per byte; this is a methodological connection, not evidence that Probierz used this tokenizer.
02 / DATA QUALITY
Dataset cleaning
The audit compares the existing data pipeline with published corpus-cleaning approaches and identifies candidate quality, deduplication and filtering checks.
This is a gap analysis for future experiments, not a measured cleaning ablation.
Read this research note ↗What connects it — and why
- Conceptual connection
Dataset cleaning → Polish agreement probe
Audit overlap before interpreting scores
Data-cleaning research includes evaluation-overlap checks. The Ladder pilot explicitly leaves training-overlap review pending. Both address the same validity problem; no shared implementation is established by these notes.
03 / MEASUREMENT
Probierz
Probierz defines measurement dimensions, provenance and overlap checks, and a programme of falsifiable evaluation. It also provides scoring tools used by the recorded model panel.
The panel is evidence for particular metrics, not validation of the entire multidimensional measurement programme.
Read this research note ↗What connects it — and why
- Conceptual connection
Polish BPE → Probierz
Different token counts → a common unit
The BPE experiment shows that token counts depend on tokenisation. This motivates looking beyond per-token quantities when comparing models. The recorded panel uses bits per byte; this is a methodological connection, not evidence that Probierz used this tokenizer.
- Documented reuse
Probierz → Fabryka Track panel
Scoring tools + syntax items
The Track result manifest identifies Probierz’s scorer_hf.py, bpb_scorer.py, run_forced_choice.py and d3 syntax eval items, including hashes. These tools and items turn part of the measurement programme into a recorded model panel.
04 / PILOT
Polish agreement probe
A matched-item pilot records continuous pair probabilities and runtime for Koliber and Pollock. Its measurements supply the cost-planning note.
Native review and overlap checks are pending. Corpus BPB and training-seed variance were not measured; language-training domains differ.
Read this research note ↗What connects it — and why
- Conceptual connection
Dataset cleaning → Polish agreement probe
Audit overlap before interpreting scores
Data-cleaning research includes evaluation-overlap checks. The Ladder pilot explicitly leaves training-overlap review pending. Both address the same validity problem; no shared implementation is established by these notes.
- Documented reuse
Polish agreement probe → Benchmark cost
Measured runtime + observed contrast
The cost note explicitly reuses the 2,000-pair pilot: 111.3 seconds for Koliber, 100.4 seconds for Pollock and an observed probability difference of 0.3169. It uses these measurements to anchor workload estimates; it cannot compute seed-calibrated DF.
05 / RECORDED EVIDENCE
Fabryka Track panel
The result archive stores per-document compression scores, paired-bootstrap artifacts and a syntax panel. Its manifest names and hashes Probierz scoring tools and syntax items.
BPB uses 300 documents; the reported syntax core contains 148 pairs. This panel and the Ladder pilot use different items and metrics.
Read this research note ↗What connects it — and why
- Documented reuse
Probierz → Fabryka Track panel
Scoring tools + syntax items
The Track result manifest identifies Probierz’s scorer_hf.py, bpb_scorer.py, run_forced_choice.py and d3 syntax eval items, including hashes. These tools and items turn part of the measurement programme into a recorded model panel.
- Conceptual connection
Fabryka Track panel → Benchmark cost
From a score to a useful decision
The historical panel reports scores and uncertainty. Ladder asks the additional question: can a metric distinguish the intended models reliably enough to justify its cost? This connects research questions; the cost note does not claim to reuse the Track panel.
06 / NEXT DECISION
Benchmark cost
The planning note reuses the Polish pilot’s observed runtimes and contrast, then estimates the workload of other benchmarks to plan the next paired evaluation.
English-battery times are proxies. Seed-calibrated discrimination is unavailable, so the note does not establish which benchmark should be adopted.
Read this research note ↗What connects it — and why
- Documented reuse
Polish agreement probe → Benchmark cost
Measured runtime + observed contrast
The cost note explicitly reuses the 2,000-pair pilot: 111.3 seconds for Koliber, 100.4 seconds for Pollock and an observed probability difference of 0.3169. It uses these measurements to anchor workload estimates; it cannot compute seed-calibrated DF.
- Conceptual connection
Fabryka Track panel → Benchmark cost
From a score to a useful decision
The historical panel reports scores and uncertainty. Ladder asks the additional question: can a metric distinguish the intended models reliably enough to justify its cost? This connects research questions; the cost note does not claim to reuse the Track panel.
Two reuse links are documented in the inspected notes. Three dashed links are editorial interpretations of shared research questions. They do not imply a shared codebase, dataset or chronological dependency. Sources are pinned to the revisions used in this catalogue.
Illustrated research note · micro-models
Small models, complementary errors
Completed pilot · limited sample
Illustrated research note · tokenizer
A smaller vocabulary, fitted to Polish
Workshop research · measured tokenizer metrics
Illustrated research note · ladder
A Polish probe, with its limits visible
Provisional diagnostic · incomplete tier
Experiment report · micro-models
Music experts: ensemble, stitching and representation similarity
Reported results · corpus-specific
Workshop research · tokenizer
Polish BPE: implementation and pretokenisation
Small held-out · 371 words
Benchmark methodology · Probierz
Probierz: evaluation dimensions and falsifiable measurement
Experimental programme
Technical research note · slayer-dsbench
Dataset cleaning: pipeline gaps and proposed experiments
Gap analysis · not an ablation result
Benchmark artifact · fabryka-track
Polish compression and syntax: recorded model panel
Historical matched panel
Evaluation planning note · ladder
Benchmark cost versus discrimination
Measured probe + explicit cost proxies
These are selected research notes and artifacts, not nine peer-reviewed papers. Measurements retain their own datasets and protocols; scores from different studies are not combined.
PUBLICATION STANDARD
From hypothesis to evidence.
Each report connects question to limitations, names authors, dates, funding and compute, and links every artifact needed to inspect or reproduce it. Results appear here only once a corresponding artifact exists.
01Question
02Hypothesis
03Baseline
04Method
05Experiments
06Results
07Limitations
Artifact register
Browse current outputs →ARTIFACTPROGRAMMETYPESTATUS
Pollock Mini LM 125M
SLM-001
Model
Public
Polish DynaWord
DATA-001
Dataset
Public
CodeSOTA
DATA-001
Benchmark
Public
Token-probability tools
SYS-001
Tool
Live
Inference platform
SYS-001
Service
Live
Data-mixture scaling study
SLM-001
Technical report
Planned
Commodity-GPU inference comparison
SYS-001
Technical report
Planned