# BLOOM

- Category: Models & resources
- Official site: https://huggingface.co/docs/transformers/model_doc/bloom
- HTML: https://www.88z.ai/en/tools/bloom
- Markdown: https://www.88z.ai/en/tools/bloom.md

BigScience’s open-source multilingual LLM supporting 46 natural languages and 13

## What is BLOOM?

BLOOM is a large language model proposed by the BigScience Workshop, designed to advance and democratize artificial intelligence through open source and open science.Its architecture is similar to GPT-3, using an auto-regressive approach for next-token prediction, but it distinguishes itself with extensive multilingual training data.

The model family includes various sizes ranging from 560 million to 176 billion parameters (such as bloom-560m, bloom-1b7, bloom-7b1, and bloom-176B), all trained on the same diverse dataset.BLOOM supports not only English but also 46 natural languages including Chinese, French, and Spanish, as well as 13 programming languages.

Fully supported in the Hugging Face Transformers library, developers can utilize classes like BloomForCausalLM for tasks such as text generation, classification, and question answering.The community also provides optimization guides for DeepSpeed and Accelerate to enable efficient inference and training.

## Key features

- **Multilingual Sup**：Trained on 46 natural languages and 13 programming languages, offering robust cross-lingual capabilities.
- **Multiple Model S**：Available in various versions from bloom-560m to bloom-176B, catering to different computational resources.
- **GPT-3-like Archi**：Uses an auto-regressive decoder structure focused on next-token prediction to ensure coherent generation quality.
- **Open Source**：Contributed by the BigScience Workshop following open science principles to promote transparency and sharing in AI.
- **Transformers Int**：Natively supported in the Hugging Face Transformers library for easy loading and fine-tuning.
- **Inference Optimi**：Official and community resources provide blogs and code examples for fast inference using DeepSpeed and Accelerate.

## How to use it

1. 1.Install the Hugging Face Transformers library: pip install transformers.
2. 2.Choose a suitable BLOOM version, such as bloom-560m or bloom-7b1.
3. 3.Load the pre-trained model using AutoModelForCausalLM.from_pretrained.
4. 4.Prepare input text and perform tokenization.
5. 5.Call the generate method to produce text output.
6. 6.(Optional) Refer to official documentation to accelerate inference using DeepSpeed or Accelerate.

## Where it fits

- **Multilingual Con**：Leverage support for 46 languages to generate high-quality articles, stories, or marketing copy in multiple languages.
- **Code Assistance**：Utilize understanding of 13 programming languages to help developers generate code snippets or explain programming logic.
- **Academic Researc**：As part of an open science initiative, it is suitable for benchmarking and algorithmic research in the NLP field.

## Pricing and free quota

BLOOM is an open-source model available for free download and use.Specific deployment costs depend on selected hardware resources; refer to the official site for details.

## Who it is for

- NLP Researchers
- Multilingual Application Developers
- Open Source AI Advocates
- Enterprises needing local LLM deployment

## FAQ

### Which languages does BLOOM support?

BLOOM supports 46 natural languages and 13 programming languages.The specific list can be found on the Hugging Face model card.

### How do I get the smallest BLOOM model?

You can load the bloom-560m version, which has approximately 560 million parameters, suitable for resource-constrained environments.

### How is BLOOM's architecture different from GPT-3?

The architecture is essentially similar as both are auto-regressive models, but BLOOM is trained on a more diverse dataset covering more languages and code.

## Tags

Open Source Model · Multilingual LLM

Features, pricing and availability follow the official site. Except for 88z.ai’s own API pages, this is a third-party product; 88z.ai only provides a directory introduction.