SOLIDOT Telegram 25738
压缩算法为大模型“瘦身”

2024-11-22 22:26 by 人猿泰山之英雄归来

普林斯顿大学和斯坦福大学团队开发出一种新压缩算法 CALDERA,能精简大模型(LLM)的海量数据,为 LLM“瘦身”。这项算法不仅有助保护数据隐私、节约能源、降低成本,还能推动 LLM 在手机和笔记本电脑上高效使用。
团队举例称,当人们使用 ChatGP T时,请求会被发送到 OpenAI 公司的后端服务器进行处理。这一过程不仅成本高昂、能耗巨大,通常还很慢。如果用户想要使用消费级图形处理单元运行 LLM,就需要对这些 LLM 进行压缩。
CALDERA 算法通过减少 LLM 冗余并降低信息层的精度来发挥作用。“瘦身”后的 LLM 更加精简,可在手机或笔记本电脑等设备上存储和访问,同时提供了与未压缩版本几乎一样准确而微妙的性能。虽然 CALDERA 并非首个压缩LLM的算法,但其独特之处在于兼具“低精度”和“低排序”两种特性。其中,“低精度”减少了比特数,加快了数据存储和处理速度。而“低排序”则降低了 LLM 数据中的冗余。

https://github.com/pilancilab/caldera
https://news.sciencenet.cn/htmlnews/2024/11/534486.shtm

#人工智能



tgoop.com/solidot/25738
Create:
Last Update:

压缩算法为大模型“瘦身”

2024-11-22 22:26 by 人猿泰山之英雄归来

普林斯顿大学和斯坦福大学团队开发出一种新压缩算法 CALDERA,能精简大模型(LLM)的海量数据,为 LLM“瘦身”。这项算法不仅有助保护数据隐私、节约能源、降低成本,还能推动 LLM 在手机和笔记本电脑上高效使用。
团队举例称,当人们使用 ChatGP T时,请求会被发送到 OpenAI 公司的后端服务器进行处理。这一过程不仅成本高昂、能耗巨大,通常还很慢。如果用户想要使用消费级图形处理单元运行 LLM,就需要对这些 LLM 进行压缩。
CALDERA 算法通过减少 LLM 冗余并降低信息层的精度来发挥作用。“瘦身”后的 LLM 更加精简,可在手机或笔记本电脑等设备上存储和访问,同时提供了与未压缩版本几乎一样准确而微妙的性能。虽然 CALDERA 并非首个压缩LLM的算法,但其独特之处在于兼具“低精度”和“低排序”两种特性。其中,“低精度”减少了比特数,加快了数据存储和处理速度。而“低排序”则降低了 LLM 数据中的冗余。

https://github.com/pilancilab/caldera
https://news.sciencenet.cn/htmlnews/2024/11/534486.shtm

#人工智能

BY Solidot


Share with your friend now:
tgoop.com/solidot/25738

View MORE
Open in Telegram


Telegram News

Date: |

With the sharp downturn in the crypto market, yelling has become a coping mechanism for many crypto traders. This screaming therapy became popular after the surge of Goblintown Ethereum NFTs at the end of May or early June. Here, holders made incoherent groaning sounds in late-night Twitter spaces. They also role-played as urine-loving Goblin creatures. The group also hosted discussions on committing arson, Judge Hui said, including setting roadblocks on fire, hurling petrol bombs at police stations and teaching people to make such weapons. The conversation linked to arson went on for two to three months, Hui said. During a meeting with the president of the Supreme Electoral Court (TSE) on June 6, Telegram's Vice President Ilya Perekopsky announced the initiatives. According to the executive, Brazil is the first country in the world where Telegram is introducing the features, which could be expanded to other countries facing threats to democracy through the dissemination of false content. How to create a business channel on Telegram? (Tutorial) It’s easy to create a Telegram channel via desktop app or mobile app (for Android and iOS):
from us


Telegram Solidot
FROM American