LoraMap: Harnessing the Power of LoRA Connections


Using the power of LoRAs to... fact check?!

Researchers have found a way to combine multiple reasoning LoRAs (Low-Rank Adaptations) to significantly improve the fact-checking capabilities of Large Language Models (LLMs).

LoraMap is a new approach that learns the connections between different reasoning LoRAs, each trained on a specific dataset.

Unlike existing methods like LoraHub (which linearly adds LoRA weights) or LoraConcat (which concatenates LoRAs and fine-tunes them), LoraMap preserves the original LoRA matrices and only learns the mapping between them.

This allows the model to reason from diverse perspectives while avoiding the pitfalls of weight averaging or catastrophic forgetting.

The results? LoraMap outperforms both LoraHub and LoraConcat on fact-checking tasks with significantly fewer parameters.

Enabling LLMs to better distinguish between claims and contexts and leverage domain-specific knowledge is important if we want to build systems that people can rely on.

 

https://arxiv.org/abs/2408.16264v1

 

Comments

Post a Comment

Popular posts from this blog

AlphaProteo generates novel proteins for biology and health research

AlphaProteo: A New AI System for Protein Design  A powerful tool for biological research In a recent paper published in Nature, researchers from DeepMind introduced AlphaProteo, a new AI system for protein design. AlphaProteo can design proteins that bind to specific target molecules, which could be helpful for drug discovery, disease understanding, and more. AlphaProteo is better than other methods at designing these proteins. In one experiment, AlphaProteo designed binders that were 10 times stronger than those designed by other methods. AlphaProteo is still under development, but it has the potential to revolutionize many areas of science. What is AlphaProteo? AlphaProteo is a deep learning system that was trained on a large dataset of protein structures. The system can learn the rules that govern how proteins fold and how they interact with other molecules. This allows AlphaProteo to design new proteins that have specific properties, such as the ability to bind to a particular ...
Read more
  A New Kind of Hidden Networking Science Building off the success of the program  Resilient Anonymous Communication for Everyone (RACE) , which recently released its code on  GitHub , PWND 2  will develop formal models of emergent communication pathways (AKA weird networks) to fundamentally improve the deployment and detection of robust and resilient hidden networks. What is RACE? RACE is an open source project aimed at developing technologies to provide metadata-anonymous, secure, and resilient messaging for users around the world. RACE provides anonymity by routing messages through an overlay network of volunteer servers using cryptographic algorithms that prevent a malicious subset of these servers from determining who is messaging whom. RACE uses specialized networking protocols to prevent connections between individual members of the network from being detected or blocked. RACE is built to run in a dockerized linux environment and on Android devices. For more d...
Read more

A New Tool for ADHD Management

AI Apps: A New Tool for ADHD Management   In the ever-evolving landscape of technology, artificial intelligence (AI) is making waves in an unexpected area: helping individuals with Attention Deficit Hyperactivity Disorder (ADHD). Recent developments have shown that AI apps can be powerful allies for those struggling with the challenges of ADHD, offering support in organization, task management, and focus.   ## The AI Advantage   Take Becky Litvintchouk, for example. As an entrepreneur with ADHD, she found starting her own business, GetDirty, a daunting task. However, by leveraging an AI app called Claude, she was able to navigate complex contracts and create business plans with ease. The app's ability to summarize information and generate step-by-step plans based on her goals proved "massively instrumental" in her success.   Litvintchouk's experience is not unique. Many individuals with ADHD are turning to AI tools to help manage their symptoms and improve their prod...
Read more