This weblog put up focuses on new options and enhancements. For a complete record, together with bug fixes, please see the launch notes.
Introducing the template to fine-tune Llama 3.1
Llama 3.1 is a set of pre-trained and instruction-tuned giant language fashions (LLMs) developed by Meta AI. It’s identified for its open-source nature and spectacular capabilities, similar to being optimized for multilingual dialogue use circumstances, prolonged context size of 128K, superior software utilization, and improved reasoning capabilities.
It’s out there in three mannequin sizes:
- 405 billion parameters: The flagship basis mannequin designed to push the boundaries of AI capabilities.
- 70 billion parameters: A extremely performant mannequin that helps a variety of use circumstances.
- 8 billion parameters: A light-weight, ultra-fast mannequin that retains most of the superior options of its bigger counterpart, which makes it extremely succesful.
At Clarifai, we provide the 8 billion parameter model of Llama 3.1, which you’ll fine-tune utilizing the Llama 3.1 coaching template inside the Platform UI for prolonged context, instruction-following, or functions similar to textual content era and textual content classification duties. We transformed it into the Hugging Face Transformers format to reinforce its compatibility with our platform and pipelines, ease its consumption, and optimize its deployment in varied environments.
To get probably the most out of the Llama 3.1 8B mannequin, we additionally quantized it utilizing the GPTQ quantization methodology. Moreover, we employed the LoRA (Low-Rank Adaptation) methodology to attain environment friendly and quick fine-tuning of the pre-trained Llama 3.1 8B mannequin.
Positive-tuning Llama 3.1 is simple: Begin by creating your Clarifai app and importing the information you need to fine-tune. Subsequent, add a brand new mannequin inside your app, and choose the “Textual content-Generator” mannequin sort. Select your uploaded information, customise the fine-tuning parameters, and prepare the mannequin. You possibly can even consider the mannequin instantly inside the UI as soon as the coaching is completed.
Comply with this information to fine-tune the Llama 3.1 8b instruct mannequin with your individual information.
Revealed new fashions
(Clarifai-hosted fashions are those we host inside our Clarifai Cloud. Wrapped fashions are these hosted externally, however we deploy them on our platform utilizing their third-party API keys)
- Revealed Llama 3.1-8b-Instruct, a multilingual, extremely succesful LLM optimized for prolonged context, instruction-following, and superior functions.
- Revealed GPT-4o-mini, an inexpensive, high-performing small mannequin excelling in textual content and imaginative and prescient duties with in depth context help.
- Revealed Qwen1.5-7B-Chat, an open-source, multilingual LLM with 32K token help, excelling in language understanding, alignment with human preferences, and aggressive tool-use capabilities.
- Revealed Qwen2-7B-Instruct, a state-of-the-art multilingual language mannequin with 7.07 billion parameters, excelling in language understanding, era, coding, and arithmetic, and supporting as much as 128,000 tokens.
- Revealed Whisper-Giant-v3, a Transformer-based speech-to-text mannequin displaying 10-20% error discount in comparison with Whisper-Giant-v2, skilled on 1 million hours of weakly labeled audio, and can be utilized for translation and transcription duties.
- Revealed Llama-3-8b-Instruct-4bit, an instruction-tuned LLM optimized for dialogue use circumstances. It could actually outperform most of the out there open-source chat LLMs on widespread trade benchmarks.
- Revealed Mistral-Nemo-Instruct, a state-of-the-art 12B multilingual LLM with a 128k token context size, optimized for reasoning, code era, and world functions.
- Revealed Phi-3-Mini-4K-Instruct, a 3.8B parameter small language mannequin providing state-of-the-art efficiency in reasoning and instruction-following duties. It outperforms bigger fashions with its high-quality information coaching.
Python SDK
Added patch operations
- Launched patch operations for enter annotations and ideas.
- Launched patch operations for apps and datasets.
Improved the RAG SDK
- We enabled the RAG SDK to make use of atmosphere variables for enhanced safety, flexibility, and simplified configuration administration.
Improved the logging expertise
- Enhanced the logging expertise by including a relentless width worth to wealthy logging.
Group Settings and Administration
Launched a brand new Group Consumer function
- This function has entry privileges just like these of an Group Contributor for all apps and scopes. Nevertheless, it comes with view-only permissions with out create, replace, or delete privileges.
Applied restrictions on the power so as to add new organizations primarily based on the person’s present group rely and have entry
- If a person has created one group and doesn’t have entry to the a number of organizations function, the “Add a company” button is now disabled. We additionally show an applicable tooltip to them.
- If a person has entry to the a number of organizations function however has reached the utmost creation restrict of 20 organizations, the “Add a company” button is disabled. We additionally show an applicable tooltip to them.
Improved the performance of the Hyperparamater Sweeps module
- Now you can use the module to successfully prepare your mannequin on a spread and mixtures of hyperparameter values.
Docs Refresh
Made vital enhancements to our documentation web site
- Upgraded the location to make use of Docusaurus model 3.4.
- Different enhancements embody aesthetic updates, a extra intuitive menu-based navigation, and a brand new complete API reference information.
Fashions
Enabled deletion of related mannequin property when eradicating a mannequin annotation
- Now, when deleting a mannequin annotation, the related mannequin property are additionally marked as deleted.
Workflows
Improved the performance of the Face workflow
- Now you can use the Face workflow to successfully generate face landmarks and carry out face visible searches inside your functions.
Mounted points with Python and Node.js SDK code snippets
In the event you click on the “Use Mannequin” button on a person mannequin’s web page, the “Name by API / Use in a Workflow” modal seems. You possibly can then combine the displayed code snippets in varied programming languages into your individual use case.
- Beforehand, the code snippets for Python and Node.js SDKs for image-to-text fashions incorrectly outputted ideas as a substitute of the anticipated textual content. We mounted the problem to make sure the output is now accurately supplied as textual content.
Added help for non-ASCII characters
- Beforehand, non-ASCII characters have been fully filtered out from the UI when creating ideas. We mounted this concern, and now you can use non-ASCII characters throughout all elements.
Improved the show of idea relations
- Idea relations at the moment are proven subsequent to their respective idea names, offering clearer and extra quick context.