FABRYKA AI / RESEARCH METHODOLOGY
Make the result
inspectable.
A research question, a controlled comparison and enough evidence for someone else to understand what happened.
Start with a question
State the hypothesis, baseline and success criterion before the experiment. Separate the research agenda from completed work.
Keep comparisons matched
Document model and data revisions, tokenizer, seeds, training tokens, compute and evaluation protocol. Compare results within the same protocol; do not combine incompatible task scores into a single claim.
Separate training from evaluation
Record data provenance, licensing and splits. Check contamination and reference quality. Closed or private external benchmarks are for final evaluation, not training or development decisions.
Measure systems and quality separately
Serving experiments should specify hardware, software, context length, batch size and concurrency. Report latency distributions, throughput and memory use alongside a separate quality evaluation.
Publish context and limitations
Reports should identify authors, dates, funding and compute sources, and link relevant code, data, configurations and artifacts. Label schematic figures, planned studies and historical results explicitly.