Llama2 70b support

[bretello]

Add support for llama2 70b (uses n_gqa parameter)

See #488 (comment) and ggml-org/llama.cpp#2276

[bretello]

From my limited testing, it seems that the GPU is not being used when providing n_gpu_layers=1, even when compiled with metal. I tried running other models (yet) but I'm probably going to work on these in the coming days.

[oobabooga]

I have made a test and was able to get 70b working with GPU acceleration by compiling it like this (probably half of it is redundant):

pip uninstall llama-cpp-python
git clone 'https://github.com/bretello/llama-cpp-python' -b llama2-70b-support
cd llama-cpp-python/
git submodule update --init --recursive
cd vendor/llama.cpp/
make LLAMA_CUBLAS=1 -j8
cd ..
CMAKE_ARGS="-DLLAMA_CUBLAS=on" FORCE_CMAKE=1 pip install . --no-cache-dir

[oobabooga]

Maybe rms_norm_eps is also necessary?

ggml-org/llama.cpp#2374

It seems like it currently needs to be manually set to 1e-5 for llama-2 (the default is 1e-6).

[bretello]

@oobabooga Rebased on main (v0.1.76), now that's also included.

[abetlen]

@bretello thanks for rebasing, I'll merge this in and publish a new release since I don't think it can work without setting that param during initialisation.