[$] Portable LLMs with llamafile
Large language models (LLMs) have been the subject of much discussion and scrutiny recently. Of particular interest to open-source enthusiasts are the problems with running LLMs on one's own hardware — especially when doing so requires NVIDIA's proprietary CUDA toolkit, which remains unavailable in many environments. Mozilla has developed llamafile as a potential solution to these problems. Llamafile can compile LLM weights into portable, native executables for easy integration, archival, or distribution. These executables can take advantage of supported GPUs when present, but do not require them.