LoRA Homepage, Documentation and Downloads – Low Rank Adaptation of Large Language Models – News Fast Delivery
LoRA is an abbreviation for Low-Rank Adaptation of Large Language Models, that is, low-rank adaptation of large language models. It freezes the weights of the pre-trained model and injects a trainable rank factorization matrix into each layer of the Transformer architecture, greatly reducing the number of trainable parameters for downstream tasks. Compared to GPT-3 175B […]









