This technical guide outlines an end-to-end pipeline for developing a reasoning-focused language model by curating and fine-tuning the SupraLabs/reasoning-corpus-4K-5M-v1 dataset. The workflow begins with streaming a representative subset directly from the Hugging Face Hub, enabling efficient data access without full download. Initial data analysis involves inspecting source distribution, token length patterns, task composition (e.g., code, math, medical), and the ratio of reasoning content to final answers. This exploratory phase informs subsequent quality filtering steps.
A multi-stage filtering process is applied to refine the dataset, removing examples unsuitable for training. Criteria include:
- Token Length: Samples between 200 and 3000 tokens.
- Degeneracy: Ensuring thought traces exceed 100 characters and assistant responses exceed 20 characters.
- Repetition: Filtering out examples where a single line repeats excessively (over 30% of lines).
- Reasoning Ratio: Maintaining a balance where reasoning content comprises 15% to 97% of the combined thought and answer length.
The filtered data is then transformed into a chat-based supervised fine-tuning format, explicitly embedding reasoning steps within
<think>...</think>tags in the assistant's response, guided by a system prompt emphasizing step-by-step thought.
The SmolLM2-135M-Instruct model serves as the base for fine-tuning, utilizing LoRA for parameter-efficient adaptation. The LoRA configuration specifies r=16, lora_alpha=32, and lora_dropout=0.05. Training is conducted via TRL’s SFTTrainer with a max_length of 2048 tokens, a batch size of 2, and 8 gradient accumulation steps over one epoch, employing bf16 precision on CUDA-enabled devices. The resulting model demonstrates structured inference, separating generated thought processes from final answers. The curated training and evaluation datasets are exported as Parquet files, providing a reusable foundation for further research into reasoning model development.