Mistral 7B v0.2 Base (released at SHACK15sf hackathon)

GitHub: https://github.com/mistralai-sf24/hackathon
X: https://twitter.com/MistralAILabs/status/1771670765521281370

New release: Mistral 7B v0.2 Base (Raw pretrained model used to train Mistral-7B-Instruct-v0.2)
🔸 https://models.mistralcdn.com/mistral-7b-v0-2/mistral-7B-v0.2.tar
🔸 32k context window
🔸 Rope Theta = 1e6
🔸 No sliding window
🔸 How to fine-tune: