Beyond Accuracy
One of the recurring lessons:
- Accuracy is not enough
- In imbalanced settings, metrics like macro-F1, precision/recall trade-offs, or PR-AUC are far more informative
- Threshold tuning often matters as much as the underlying architecture
Iterative Process
A robust NLP project is never a single run — it’s a series of experiments. Typical steps involve:
- adjusting maximum sequence length, dropout, and learning rates
- balancing loss functions with class weights or sampling
- exploring optimizers and schedulers
- applying techniques like EMA, label smoothing, or gradient accumulation
What looks like trial-and-error becomes a scientific process when documented.
Tracking and Reproducibility
Modern ML practice is inseparable from experiment tracking:
- MLflow for storing artifacts, metrics, and parameters
- Weights & Biases (W&B) for visualizing learning curves, validation performance, and threshold sweeps
- Notion or similar tools as a lab notebook to capture the “story” of the experiments
This turns raw training logs into a transparent and reproducible workflow — where each run, successful or not, contributes to understanding.
Key Takeaway
The most important insight is not about squeezing the last percentage point of accuracy, but about:
- choosing the right metric for the real-world task
- designing a process that can be explained and reproduced
- building trust in the model through transparency and iteration
In applied NLP, it’s less about chasing a perfect number, and more about cultivating a scientific practice around model development.