Table of Contents
https://github.com/simonw/llm + https://github.com/simonw/llm-mlc
pip install llm
llm install llm-mlc
llm mlc pip install --pre --force-reinstall mlc-ai-nightly mlc-chat-nightly -f <https://mlc.ai/wheels>
llm mlc setup
llm mlc download-model Llama-2-70b-chat --alias llama2git
Do recall that we’ve aliases llama2 and can use that above…
https://github.com/Mozilla-Ocho/llamafile
Very similar to simonw llm tooling, but a touch easier for folks. There is a web interface option but this app should basically be above under CLI.
GUI with ability to retrieve and interact with multiple models locally and via API. Has the ability to create collection of local docs to query:
