A NLP course project to train a small chatGPT(1.5B).

The master branch is for welcome. Code is in the pretrain and sft branch.

Please leave a ⭐star⭐ if you think helpful, thanks.

We wrote two blogs to record this project(pretrain and SFT separately):

1. Pretrain

2. Supervised Finetune

Parameters: