‘Most important AI Model in the Last Decade’ — Meet Large Language Model BLOOM

'上个十年最重要的人工智能模型' -- 满足大型语言模型布鲁姆

Major tech companies have been playing with large language models (LLMs) for a while, with OpenAI’s GPT-3 making its way onto the global stage in 2020. Google also unveiled its Pathways Language Model (PaLM) in April 2022.

大型科技公司使用大型语言模型 (llm) 已有一段时间了，OpenAI的GPT-3 2020年进入了全球舞台。Google还在2022年4月发布了其Pathways语言模型 (PaLM)。

Meta’s May 2022 decision to open-source OPT-175B, meanwhile, was considered a step forward in leveling the playing field for researchers without major capital. As mentioned, Hugging Face Researcher and Chief Ethics Scientist, Margaret Mitchell, warned that the model could be used — both on its own and through downstream applications — to generate harmful content.

同时，Meta 2022年5月决定开放源代码OPT-175B，被认为是在为没有重大资本的研究人员提供公平竞争环境方面迈出的一步。如前所述，拥抱面部研究员和首席伦理学科学家玛格丽特·米切尔 (Margaret Mitchell) 警告说，该模型可以单独使用，也可以通过下游应用来生成有害内容。

Concerns about misuse of LLMs inspired Cohere, OpenAI, and AI21 Labs to publish, in June 2022, a preliminary set of best practices for their responsible development and deployment.

对LLMs滥用的担忧促使Cohere，OpenAI和AI21实验室2022年6月发布了一套负责任的开发和部署的最佳实践。

Now, a new LLM is generating buzz as potentially bypassing some of those concerns.

现在，一个新的LLM正在引起嗡嗡声，因为它有可能绕过其中一些担忧。

BLOOM — an acronym for BigScience Large Open-science Open-access Multilingual Language Model — is the brainchild of BigScience, a collective of more than 1,000 volunteer researchers worldwide.

BLOOM-BigScience大型开放科学开放多语言模型的首字母缩写-是BigScience的创意，BigScience是全球1,000多名志愿研究人员的集体。

While French machine learning platform Hugging Face led the project, starting in 2021, contributors included Nvidia, Microsoft, and support from the French National Research Agency, CNRS. BLOOM was built and trained using the Jean Zay supercomputer.

虽然法国机器学习平台拥抱面孔领导了这个项目，从2021年开始，贡献者包括英伟达、微软和法国国家研究机构CNRS的支持。BLOOM是使用Jean Zay超级计算机构建和培训的。

In a June 28, 2022 tweet, Cambrian AI analyst Alberto Romero declared BLOOM “the most important AI model in the last decade.”

在一条2022年6月28日的推文中，寒武纪人工智能分析师阿尔贝托·罗梅罗 (Alberto Romero) 宣布布鲁姆为 “过去十年中最重要的人工智能模型”。

“BLOOM isn’t architecturally different from GPT-3,” Romero explained. “What makes it unique is that it represents the starting point of a socio-political paradigm shift that will define the future of the AI field.”

“布鲁姆在建筑上与GPT-3没有什么不同，” 罗梅罗解释道。“它的独特之处在于，它代表了社会政治范式转变的起点，这将定义人工智能领域的未来。”

Romero pointed out that current SOTA language models follow a certain trend; that is, “large transformer-based and trained with lots of data, using big computers.” Most significantly, however, “they all stem from the immense resources of private tech companies.”

罗梅罗指出，目前的SOTA语言模型遵循一定的趋势; 也就是说，“基于大型变压器，使用大型计算机进行大量数据培训。” 然而，最重要的是，“它们都源于私营科技公司的巨大资源。”

Designed to be multilingual from conception — as opposed to many AI language models that rely on either Chinese or English — BLOOM now supports 46 human languages and 13 programming languages.

与许多依赖中文或英文的人工智能语言模型相反，布鲁姆现在支持46种人类语言和13种编程语言。

The Hugging Face model card breaks down the distribution of languages used in BLOOM’s training data, with English (30.04%), Simplified Chinese (16.2%), and French (12.9%) accounting for the greatest swaths.

拥抱人脸模型卡分解了BLOOM训练数据中使用的语言分布，其中英语 (30.04%)，简体中文 (16.2%) 和法语 (12.9%) 占了最大的范围。

Spanish and “Code,” each at 10.8%, were tied for fourth place. Training data also came from languages belonging to the Indic family (4.4%) and the Niger-Congo family (0.03%).

西班牙语和 “代码” 各在10.8%，并列第四。培训数据也来自印度语族 (4.4%) 和尼日尔-刚果语族 (0.03%) 的语言。

As an example of the project’s reach, Masakhane researcher and Hugging Face intern Chris Emezue told MIT Technology Review that Hugging Face coordinated with African AI researchers to find data sets, “such as records from local authorities or universities,” to train the model on African languages historically underrepresented online.

作为该项目覆盖范围的一个例子，Masakhane研究人员和拥抱面孔实习生Chris Emezue告诉《麻省理工学院技术评论》，拥抱面孔与非洲人工智能研究人员协调寻找数据集，“例如来自地方当局或大学的记录”，以在历史上在线代表性不足的非洲语言上训练该模型。

Training started on March 11, 2022 and wrapped up July 5, 2022, with an estimated cost of USD 2–5m in cloud computing, according to the same model card. With regard to environmental impact, “the training supercomputer, Jean Zay, uses mostly nuclear energy. The heat generated by it is reused for heating campus housing.”

根据同一型号卡，培训于2022年3月11日开始，2022年7月5日结束，云计算的估计成本为2-50万美元。关于环境影响，“培训超级计算机让·扎伊 (Jean Zay) 主要使用核能。它产生的热量被重新用于校园住房供暖。”

As reported by VentureBeat, researchers trained BLOOM using existing open-source ML models and, building on the open-source PyTorch ML framework, enabling the model to look at dozens of different languages.

据VentureBeat报道，研究人员使用现有的开源ML模型对BLOOM进行了培训，并基于开源PyTorch ML框架，使该模型能够查看数十种不同的语言。

“This model is being created in order to enable public research on [LLMs, which] are intended to be used for language generation or as a pretrained base model that can be further fine-tuned for specific tasks,” according to the Hugging Face model card.

“这个模型的创建是为了使公众能够对 [LLMs进行研究，这些研究旨在用于语言生成或作为预先训练的基础模型，可以针对特定任务进行进一步微调，” 根据拥抱脸模型卡。

BLOOM uses its own open license, modeled on the Responsible AI, with the goal of keeping the model as open as possible while limiting the possibilities for misuse.

BLOOM使用自己的开放许可证，以负责任的AI为模型，目的是保持模型尽可能开放，同时限制滥用的可能性。

TechCrunch has reported that BigScience plans to charge researchers less than USD 40 per hour to access BLOOM on a cloud provider. To further equalize access, the organization is also developing “smaller, less hardware-intensive versions” of the model, a system to allow labs to share the model across servers, and an API.

TechCrunch报告说，BigScience计划向研究人员收取每小时不到40美元的费用，以访问云提供商上的BLOOM。为了进一步均衡访问，该组织还在开发该模型的 “较小，较少硬件密集型版本”，允许实验室跨服务器共享该模型的系统以及API。