Models & resources
LLaMA
LLaMA is a series of large language models released by Meta, designed to provide accessible foundation model resources for researchers, developers, and businesses.
What is LLaMA?
LLaMA is a series of large language models released by Meta, designed to provide accessible foundation model resources for researchers, developers, and businesses.The series covers various scales from 7 billion to 70 billion parameters, offering both pre-trained and instruction-tuned versions to help users experiment, innovate, and scale applications responsibly in local environments.
The official repository provides minimal example code for loading LLaMA models and running inference, including text completion and chat completion functionalities.With the release of Llama 3.1, Meta has consolidated resources into the Llama Stack ecosystem, including llama-models for foundation models, PurpleLlama for safety components, and llama-agentic-system for end-to-end agentic applications.
Users can request model weights and tokenizers via the Meta website or access them through Hugging Face.The toolkit supports quick deployment in environments equipped with PyTorch and CUDA, allowing developers to adjust parallel processing parameters and sequence lengths based on hardware capabilities for efficient local inference.
Key features
- Multi-Scale ModeOffers pre-trained and fine-tuned models ranging from 7B to 70B parameters to accommodate diverse computational resources.
- Local InferenceIncludes minimal Python script examples supporting local execution for both text completion and chat completion modes.
- Llama Stack IntePart of the broader Llama ecosystem, compatible with components like llama-toolchain and llama-agentic-system.
- Flexible ShardinSupports model parallelism with recommended GPU process counts based on model size (e.g., 8 processes for 70B).
- Long Context SupAll models support sequence lengths up to 4096 tokens, with cache pre-allocation adjustable based on hardware memory.
- Safety & ResponsProvides detailed use policies, responsible use guides, and PurpleLlama safety mitigation components for compliant usage.
How to use it
- Visit the Meta website or Hugging Face page, accept the license agreement, and request access to download model weights and tokenizers.
- After receiving the email with the signed URL, clone the official GitHub repository and run pip install -e .in the top-level directory to install dependencies.
- Grant execution permissions to the download.sh script, run it, and manually paste the download link provided in the email when prompted.
- Configure model parallel parameters based on model size (e.g., 1 for 7B, 8 for 70B) and adjust max_seq_len and max_batch_size as needed.
- Use the torchrun command to execute example_chat_completion.py or example_text_completion.py to start local inference testing.
Where it fits
- Local Research &Researchers can load models of various scales on local hardware for algorithm validation and performance benchmarking.
- Private DeploymeEnterprises can download model weights to internal servers to build secure chatbots or text generation tools without relying on external APIs.
- Fine-Tuning FounDevelopers can use pre-trained weights as a starting point for domain-specific instruction fine-tuning using scripts from llama-cookbook.
Pricing and free quota
Free to access, but requires requesting permission via the Meta website or Hugging Face and accepting the license agreement.Pricing follows the official site.
Who it is for
- AI Researchers
- Machine Learning Engineers
- Privacy-First Enterprise Developers
- Open Source Community Contributors
FAQ
How do I get the download link for LLaMA models?
You need to visit the Meta website or the Hugging Face model card, accept the license agreement, and submit a request.Once approved, you will receive an email with a time-limited signed download link.
How many GPU resources are needed to run the 70B model?
According to the official examples, running the 70B model recommends setting the model parallelism (MP) to 8, meaning 8 GPU processes are required.Specific VRAM usage depends on batch size and sequence length settings.
Is the LLaMA 2 repository still maintained?
This repository is marked as deprecated.Meta recommends users move to the new Llama Stack related repositories, such as llama-models and llama-toolchain, for the latest features and support.