The Llama connector integrates Meta’s powerful Llama large language models into your workflow. This documentation helps you understand how to use and configure the Llama node effectively. This is typically used for tasks such as reasoning, summarization, content generation, and conversational response.
Configuration:
When setting up the Llama node, you’ll need to configure several parameters:
- Model Selection: Choose the appropriate Llama model variant based on your needs (e.g., llama-3-8b, llama-2-13b)
- API Key: Enter your Meta Llama API key if using a hosted API, Llama APIÂ Remember to store your API key securely and never share it publicly.
- Available Models: The Llama connector provides access to Meta’s latest Llama models, ensuring you can leverage advanced AI capabilities for your workflows.
Supported Model Variants:
| Model | Parameters | Input Tokens | Optimized For |
|---|---|---|---|
| Llama 3.1 70B | 70 billion | 128K | State-of-the-art reasoning, complex tasks, long context understanding |
| Llama 3 8B | 8 billion | 8K | Balanced performance and efficiency, general-purpose applications |
| Llama 3 70B | 70 billion | 8K | Advanced reasoning, complex tasks, highly accurate responses |
| Llama 2 7B | 7 billion | 4K | Lightweight deployment, resource-constrained environments |
| Llama 2 13B | 13 billion | 4K | Improved quality over 7B, mid-range performance |
| Llama 2 70B | 70 billion | 4K | High-quality responses for complex tasks |
Inputs and Outputs
-
Input Channels
Prompt – Primary text input for the model.
Questions – Accepts plain text prompts or user-generated questions, forwarded to the Llama model for processing.
Documents – Document objects for context (if supported by your Llama integration).
System – System instructions for the model (if supported).
-
Output Channels
Text – Generated text output.
Answers – The model-generated response based on the input received. The output is a string of generated text that can be passed to other nodes.
Common Use Cases
-
Content Creation
– Generate drafts, summaries, and creative materials.
– Develop reports from structured information.
– Create variations of existing content.
-
Information Analysis
– Extract key insights from unstructured text.
– Organize and classify information.
– Recognize patterns and emerging trends.
-
Interactive AI
– Design responsive chatbots and virtual assistants.
– Build customer support automation solutions.
– Create intuitive interfaces for complex systems.
Frequently Asked Questions:
-
Authentication Errors
Invalid API key: Verify that LLAMA_API_KEY is set and has the correct scope.
Endpoint unreachable: Check that LLAMA_API_URL is correct and that your server/network is accessible
-
Rate Limiting & Timeouts
429 Too Many Requests: Implement back-off or upgrade your plan.
Timeouts: Increase the timeout setting or reduce prompt/context size.
-
Quality Issues
Repetitive output: Increase frequency, Penalty, or lower temperature.
Off-topic responses: Refine system prompts or include more relevant context.
