Large Language Models Paper 分享

论文1： ChatGPT's One-year Anniversary: Are Open-Source Large Language Models Catching up?

简介

2022年11月，OpenAI发布了ChatGPT，这一事件在AI社区甚至全世界引起了轰动。首次，一个基于应用的AI聊天机器人能够提供有帮助、安全和有用的答案，遵循人类指令，甚至承认并纠正之前的错误。作为第一个这样的应用，ChatGPT在其推出仅两个月内，用户数量就达到了1亿，远远快于其他流行应用如TikTok或YouTube。因此，它也吸引了巨额的商业投资，因为它有望降低劳动成本，自动化工作流程，甚至为客户带来新的体验。

但ChatGPT的闭源特性可能引发诸多问题。首先，由于不了解内部细节，比如预训练和微调过程，很难正确评估其潜在风险，尤其是考虑到大模型可能生成有害、不道德和虚假的内容。其次，有报道称ChatGPT的性能随时间变化，妨碍了可重复的结果。第三，ChatGPT经历了多次故障，仅在2023年11月就发生了两次重大故障，期间无法访问ChatGPT网站及其API。最后，采用ChatGPT的企业可能会关注API调用的高成本、服务中断、数据所有权和隐私问题，以及其他不可预测的事件，比如最近有关CEO Sam Altman被解雇并最终回归的董事会闹剧。

此时，开源大模型应运而生，社区一直在积极推动将高性能的大模型保持开源。然而，截至2023年末，大家还普遍认为类似Llama-2或Falcon这样的开源大模型在性能上落后于它们的闭源模型，如OpenAI的GPT3.5（ChatGPT）和GPT-4，Anthropic的Claude2或Google的Bard3，其中GPT-4通常被认为是最出色的。然而，令人鼓舞的是差距正在变得越来越小，开源大模型正在迅速赶上。

地址：[2311.16989] ChatGPT's One-year Anniversary: Are Open-Source Large Language Models Catching up? (arxiv.org)

更有趣的 AI Agent

  • Generative Agents: Interactive Simulacra of Human Behavior https://arxiv.org/abs/2304.03442

  • RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Models https://arxiv.org/abs/2310.00746

  • Role play with large language models https://www.nature.com/articles/s41586-023-06647-8

  • Exploring Large Language Models for Communication Games: An Empirical Study on Werewolf https://arxiv.org/abs/2309.04658

  • MemGPT: Towards LLMs as Operating Systems https://arxiv.org/abs/2310.08560

  • Augmenting Language Models with Long-Term Memory https://arxiv.org/abs/2306.07174

  • Do LLMs Possess a Personality? Making the MBTI Test an Amazing Evaluation for Large Language Models https://arxiv.org/pdf/2307.16180.pdf

更有用的 AI Agent

  • The Rise and Potential of Large Language Model Based Agents: A Survey https://arxiv.org/abs/2309.07864

  • MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework https://arxiv.org/abs/2308.00352

  • Communicative Agents for Software Development https://arxiv.org/pdf/2307.07924.pdf

  • Large Language Models Can Self-Improve https://arxiv.org/abs/2210.11610

  • Evaluating Human-Language Model Interaction https://arxiv.org/abs/2212.09746

  • Large Language Models can Learn Rules https://arxiv.org/abs/2310.07064

  • AgentBench: Evaluating LLMs as Agents https://arxiv.org/abs/2308.03688

  • WebArena: A Realistic Web Environment for Building Autonomous Agents https://arxiv.org/abs/2307.13854

  • TableGPT: Towards Unifying Tables, Nature Language and Commands into One GPT https://arxiv.org/abs/2307.08674

任务规划与分解

  • Chain-of-Thought Prompting Elicits Reasoning in Large Language Models https://arxiv.org/abs/2201.11903

  • Tree of Thoughts: Deliberate Problem Solving with Large Language Models https://arxiv.org/abs/2305.10601

  • Implicit Chain of Thought Reasoning via Knowledge Distillation https://arxiv.org/abs/2311.01460

  • ReAct: Synergizing Reasoning and Acting in Language Models https://arxiv.org/abs/2210.03629

  • ART: Automatic multi-step reasoning and tool-use for large language models https://arxiv.org/abs/2303.09014

  • Branch-Solve-Merge Improves Large Language Model Evaluation and Generation https://arxiv.org/abs/2310.15123

  • WizardLM: Empowering Large Language Models to Follow Complex Instructionshttps://arxiv.org/pdf/2304.12244.pdf

幻觉

  • Siren’s Song in the AI Ocean: A Survey on Hallucination in Large Language Modelshttps://arxiv.org/pdf/2309.01219.pdf

  • Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback https://arxiv.org/abs/2302.12813

  • SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models https://arxiv.org/abs/2303.08896

  • WebBrain: Learning to Generate Factually Correct Articles for Queries by Grounding on Large Web Corpus https://arxiv.org/abs/2304.04358

多模态

  • Learning Transferable Visual Models From Natural Language Supervision (CLIP) https://arxiv.org/abs/2103.00020

  • An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (ViT): https://arxiv.org/abs/2010.11929

  • MiniGPT-v2: large language model as a unified interface for vision-language multi-task learninghttps://arxiv.org/abs/2310.09478

  • MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models https://arxiv.org/abs/2304.10592

  • NExT-GPT: Any-to-Any Multimodal LLM https://arxiv.org/pdf/2309.05519.pdf

  • Visual Instruction Tuning (LLaVA) https://arxiv.org/pdf/2304.08485.pdf

  • Improved Baselines with Visual Instruction Tuning (LLaVA-1.5) https://arxiv.org/abs/2310.03744

  • Sequential Modeling Enables Scalable Learning for Large Vision Models (LVM) https://arxiv.org/pdf/2312.00785.pdf

  • CoDi-2: In-Context, Interleaved, and Interactive Any-to-Any Generation https://arxiv.org/pdf/2311.18775.pdf

  • Neural Discrete Representation Learning (VQ-VAE) https://browse.arxiv.org/pdf/1711.00937.pdf

  • Taming Transformers for High-Resolution Image Synthesis (VQ-GAN) https://arxiv.org/abs/2012.09841

  • Swin Transformer: Hierarchical Vision Transformer using Shifted Windows https://arxiv.org/abs/2103.14030

  • BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models https://browse.arxiv.org/pdf/2301.12597.pdf

  • InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning https://browse.arxiv.org/pdf/2305.06500.pdf

  • ImageBind: One Embedding Space To Bind Them All https://arxiv.org/abs/2305.05665

  • Meta-Transformer: A Unified Framework for Multimodal Learning https://arxiv.org/abs/2307.10802

图片/视频生成

  • High-Resolution Image Synthesis with Latent Diffusion Models https://arxiv.org/pdf/2112.10752.pdf

  • Structure and Content-Guided Video Synthesis with Diffusion Models (RunwayML Gen1) https://browse.arxiv.org/pdf/2302.03011.pdf

  • Hierarchical Text-Conditional Image Generation with CLIP Latents (DaLLE-2) https://arxiv.org/pdf/2204.06125.pdf

  • AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning https://arxiv.org/abs/2307.04725

  • Adding Conditional Control to Text-to-Image Diffusion Models (ControlNet) https://arxiv.org/abs/2302.05543

  • SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesishttps://arxiv.org/abs/2307.01952

  • Zero-1-to-3: Zero-shot One Image to 3D Object https://arxiv.org/abs/2303.11328

  • Scaling Vision Transformers to 22 Billion Parameters https://arxiv.org/abs/2302.05442

  • Glow: Generative Flow with Invertible 1×1 Convolutions https://browse.arxiv.org/pdf/1807.03039.pdf

  • Language Model Beats Diffusion – Tokenizer is Key to Visual Generation https://arxiv.org/pdf/2310.05737.pdf

  • InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generationhttps://arxiv.org/pdf/2309.06380.pdf

  • Perceptual Losses for Real-Time Style Transfer and Super-Resolution https://arxiv.org/pdf/1603.08155.pdf

  • CogView: Mastering Text-to-Image Generation via Transformers https://arxiv.org/abs/2105.13290

  • Diffusion Models for Video Prediction and Infilling https://arxiv.org/abs/2206.07696

语音合成

  • Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech (VITS)https://browse.arxiv.org/pdf/2106.06103.pdf

  • Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers (VALL-E)https://arxiv.org/abs/2301.02111

  • Speak Foreign Languages with Your Own Voice: Cross-Lingual Neural Codec Language Modeling (VALL-E X) https://arxiv.org/pdf/2303.03926.pdf

  • MusicLM: Generating Music From Text https://arxiv.org/abs/2301.11325

大模型基础

  • Attention Is All You Need https://arxiv.org/abs/1706.03762

  • Sequence to Sequence Learning with Neural Networks https://arxiv.org/abs/1409.3215

  • Neural Machine Translation by Jointly Learning to Align and Translate https://arxiv.org/abs/1409.0473

  • BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding https://arxiv.org/abs/1810.04805

  • Scaling Laws for Neural Language Models https://arxiv.org/pdf/2001.08361.pdf

  • Emergent Abilities of Large Language Models https://openreview.net/pdf?id=yzkSU5zdwD

  • Training Compute-Optimal Large Language Models (ChinChilla scaling law) https://arxiv.org/abs/2203.15556

  • Scaling Instruction-Finetuned Language Models https://arxiv.org/pdf/2210.11416.pdf

  • Direct Preference Optimization:

  • Your Language Model is Secretly a Reward Model https://arxiv.org/pdf/2305.18290.pdf

  • Progress measures for grokking via mechanistic interpretability https://arxiv.org/abs/2301.05217

  • Language Models Represent Space and Time https://arxiv.org/abs/2310.02207

  • GLaM: Efficient Scaling of Language Models with Mixture-of-Experts https://arxiv.org/abs/2112.06905

  • Adam: A Method for Stochastic Optimization https://arxiv.org/abs/1412.6980

  • Efficient Estimation of Word Representations in Vector Space (Word2Vec) https://arxiv.org/abs/1301.3781

  • Distributed Representations of Words and Phrases and their Compositionality https://arxiv.org/abs/1310.4546

GPT

  • Language Models are Few-Shot Learners (GPT-3) https://arxiv.org/abs/2005.14165

  • Language Models are Unsupervised Multitask Learners (GPT-2) https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf

  • Improving Language Understanding by Generative Pre-Training (GPT-1) https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf

  • Training language models to follow instructions with human feedback (InstructGPT)https://arxiv.org/pdf/2203.02155.pdf

  • Evaluating Large Language Models Trained on Code https://arxiv.org/pdf/2107.03374.pdf

  • Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond https://arxiv.org/abs/2304.13712

  • Instruction Tuning with GPT-4 https://arxiv.org/pdf/2304.03277.pdf

  • The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision) https://arxiv.org/abs/2309.17421

  • Sparks of Artificial General Intelligence: Early experiments with GPT-4 https://arxiv.org/abs/2303.12712

  • Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision https://arxiv.org/abs/2312.09390

开源大模型

  • LLaMA: Open and Efficient Foundation Language Models https://arxiv.org/abs/2302.13971

  • Llama 2: Open Foundation and Fine-Tuned Chat Models https://arxiv.org/pdf/2307.09288.pdf

  • Vicuna: An Open-Source Chatbot Impressing GPT-4 with 90%* ChatGPT Quality https://lmsys.org/blog/2023-03-30-vicuna/

  • LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Dataset https://arxiv.org/abs/2309.11998

  • Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena https://arxiv.org/abs/2306.05685

  • How Long Can Open-Source LLMs Truly Promise on Context Length? https://lmsys.org/blog/2023-06-29-longchat/

  • Mixtral of experts https://mistral.ai/news/mixtral-of-experts/

  • OpenChat: Advancing Open-source Language Models with Mixed-Quality Data https://arxiv.org/abs/2309.11235

  • RWKV: Reinventing RNNs for the Transformer Era https://arxiv.org/abs/2305.13048

  • Mamba: Linear-Time Sequence Modeling with Selective State Spaces https://arxiv.org/ftp/arxiv/papers/2312/2312.00752.pdf

  • Retentive Network: A Successor to Transformer for Large Language Models https://arxiv.org/abs/2307.08621

  • Baichuan 2: Open Large-scale Language Models https://arxiv.org/abs/2309.10305

  • GLM-130B: An Open Bilingual Pre-trained Model https://arxiv.org/abs/2210.02414

  • Qwen Technical Report https://arxiv.org/abs/2309.16609

  • Skywork: A More Open Bilingual Foundation Model https://arxiv.org/abs/2310.19341

微调

  • Learning to summarize from human feedback https://arxiv.org/abs/2009.01325

  • Self-Instruct: Aligning Language Model with Self Generated Instruction https://arxiv.org/abs/2212.10560

  • Scaling Down to Scale Up: A Guide to Parameter-Efficient Fine-Tuning https://arxiv.org/abs/2303.15647

  • LoRA: Low-Rank Adaptation of Large Language Models https://arxiv.org/abs/2106.09685

  • Vera: Vector-Based Random Matrix Adapation https://arxiv.org/pdf/2310.11454.pdf

  • QLoRA: Efficient Finetuning of Quantized LLMs https://arxiv.org/abs/2305.14314

  • Chain of Hindsight Aligns Language Models with Feedback https://arxiv.org/abs/2302.02676

  • Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Models https://arxiv.org/pdf/2312.06585.pdf

性能优化

  • Efficient Memory Management for Large Language Model Serving with PagedAttention (vLLM) https://arxiv.org/abs/2309.06180

  • FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness https://arxiv.org/abs/2205.14135

  • S-LoRA: Serving Thousands of Concurrent LoRA Adapters https://arxiv.org/abs/2311.03285

  • GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism https://proceedings.neurips.cc/paper/2019/file/093f65e080a295f8076b1c5722a46aa2-Paper.pdf

  • Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism https://arxiv.org/pdf/1909.08053.pdf

  • ZeRO: Memory Optimizations Toward Training Trillion Parameter Models https://arxiv.org/pdf/1910.02054.pdf

本文来自互联网用户投稿，该文观点仅代表作者本人，不代表本站立场。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如若转载，请注明出处：http://www.mzph.cn/news/607057.shtml

如若内容造成侵权/违法违规/事实不符，请联系多彩编程网进行投诉反馈email:809451989@qq.com，一经查实，立即删除！

相关文章

属性动画的使用

属性动画的使用

文章目录 1 概述2 创建属性动画页面3 属性动画参数调整延时播放时间delay的设置onFinish回调函数的使用 4 关闭属性动画页面5 参考 1 概述 属性动画，是最为基础的动画，其功能强大、使用场景多，应用范围较广。常用于如下场景中： …
阅读更多...
第九节HarmonyOS 常用基础组件12-TextTimer

第九节HarmonyOS 常用基础组件12-TextTimer

1、描述 通过文本显示计时信息并控制其计时器状态的组件。 2、接口 TextTimer(options?: {isCountDown?: boolean, count?: number, controller?: TextTimerController}) 3、参数 参数名称 参数类型 必填 描述 isCountDown boolean 否 是否倒计时。默认值&#…
阅读更多...
Kafka与RabbitMQ的区别

Kafka与RabbitMQ的区别

消息队列介绍 消息队列（Message Queue）是一种在分布式系统中进行异步通信的机制。它允许一个或多个生产者在发送消息时暂时将消息存储在队列中，然后由一个或多个消费者按顺序读取并处理这些消息。 消息队列具有以下特点： 异步通…
阅读更多...
64.网游逆向分析与插件开发-游戏增加自动化助手接口-优化自动助手与游戏焦点的切换

64.网游逆向分析与插件开发-游戏增加自动化助手接口-优化自动助手与游戏焦点的切换

内容来源于：易道云信息技术研究院VIP课 上一个内容：自动化助手UI与游戏菜单的对接-CSDN博客 码云地址（master分支）：https://gitee.com/dye_your_fingers/sro_-ex.git 码云版本号：617ac3477ef18273fb9cc2…
阅读更多...
Linux文件系统与日志服务管理

Linux文件系统与日志服务管理

目录 一.Linux文件系统 1.inode表和block （1）inode （2）block 2.查看inode号命令 3.Linux系统文件三种主要时间属性 4.磁盘空间还剩余很多但无法继续创建文件 5.inode大小 二.日志 1.日志保存位置 2.日志文件的分类 &a…
阅读更多...
Selenium 学习（0.17）——软件测试之流程图绘制方法

Selenium 学习（0.17）——软件测试之流程图绘制方法

病假5天，出去野20天，成功错过了慕课网上的期末考试。 害，都怪玩乐太开心了…… 反正咱又不指着全靠这个行当来吃饭，错过也就错过了，立的Flag能抢救一下还是要抢救一下吧。【这个其实早都会画了，而且基本也正…
阅读更多...
Mac robotframework+pycharm运行suite报错情况：ImportError: No module named request

Mac robotframework+pycharm运行suite报错情况：ImportError: No module named request

报错实例： 当前Preferences–>Tool–>External Tools Suite配置，显示使用的python为2.7版本，robotframework安装在当前版本的python中： 但是我pycharm现在的环境配置的python为3.11，当前使用的RF与当前使用的py…
阅读更多...
NoSQL概述与Redis入门-redis安装与测试

NoSQL概述与Redis入门-redis安装与测试

一、Nosql概述 1、为什么使用Nosql 1、单机Mysql时代 90年代,一个网站的访问量一般不会太大，单个数据库完全够用。随着用户增多，网站出现以下问题 数据量增加到一定程度，单机数据库就放不下了数据的索引（B Tree）,一个…
阅读更多...
关于外连接、内连接和子查询的使用（2）

关于外连接、内连接和子查询的使用（2）

目录 一. 前言 二. 使用外连接、内连接和子查询进行解答 三. 思维导图 一. 前言 在前面我们对外连接、内连接和子查询的使用有了一些了解，今天我们将继续更深入的进行学习。（这里缺少的八个题目在博主的前面博客有解答，大家可以移步前面一…
阅读更多...
zabbix-proxy代理安装

zabbix-proxy代理安装

zabbix-proxy代理安装 安装zabbix-proxyserver端配置zabbix-proxy配置被监控的agent安装中问题解决监控网络设备JMX和IPMI监控方式 zabbix-proxy的安装，至少需要准备三台机器，一台安装服务端，一台安装agent端，这里就不叙述了&…
阅读更多...
OBD汽车

OBD汽车

相当于客户端与服务器 诊断设备流程 》》》》诊断服务 OBD很多的定死了 Vme就很灵活 WWH 就是两个的结合 OBD15031 SID PID 可以自己定义一些 一个字节255个 两个有效字节 02 01 0D 5555（随机值）这是请求 两个有效字节 01 OD（请求速…
阅读更多...
51单片机之按键和数码管

51单片机之按键和数码管

51单片机之按键和数码管 ✍前言：♐独立按键😀独立按键的原理😀软件实现按键控制LED灯的亮灭 ♐数码管😊数码管显示数字或者字母的原理🐉共阳极数码管🐉共阴极极数码管🐉4位1体数码管 &#x1f6…
阅读更多...
大数据StarRocks(六) ：Catalog

大数据StarRocks(六) ：Catalog

StarRocks 自 2.3 版本起支持 Catalog（数据目录）功能，实现在一套系统内同时维护内、外部数据，方便您轻松访问并查询存储在各类外部源的数据。 1. 基本概念 内部数据：指保存在 StarRocks 中的数据。 外部数据&#xf…
阅读更多...
分布式系统架构设计之分布式消息队列架构解析

分布式系统架构设计之分布式消息队列架构解析

分布式消息队列架构是构建在分布式系统之上的消息队列架构，旨在提高高性能、高可用性和可伸缩性。它包括以下架构相关部分： 1、架构优势 分布式消息队列架构的优势主要体现在以下几个方面： 01 高可用性 在分布式消息队列架构中&#xff0…
阅读更多...
ARM架构—— Cortex-M3与Cortex-M4特点概述

ARM架构—— Cortex-M3与Cortex-M4特点概述

一、Cortex-M3与Cortex-M4异同点 相同点： 基于ARM-v7-M架构。三级流水线设计。哈佛总线架构，统一的存储器空间：指令和地址总线使用相同的地址空间。32位寻址，支持4GB 存储空间。基于ARM AMBA（高级微控制器总线架构&a…
阅读更多...
在docker上运行LCM

在docker上运行LCM

目录 1.加载镜像并进入容器 2.安装依赖 3.在docker外部git-clone lcm 4.将get-clone的lcm复制到容器中 5.编译库 6.将可执行文件复制到容器中 7.进入可执行文件 8.编译可执行文件 9.再开一个终端运行程序 10.将以上容器打成镜像并导出 1.加载镜像并进入容器 sudo do…
阅读更多...
基于多反应堆的高并发服务器【C/C++/Reactor】（中）在TcpConnection 中接收并解析Http请求消息

基于多反应堆的高并发服务器【C/C++/Reactor】（中）在TcpConnection 中接收并解析Http请求消息

一、在TcpConnection 中多添加和http协议相关的request和response struct TcpConnection {struct EventLoop* evLoop;struct Channel* channel;struct Buffer* readBuf;struct Buffer* writeBuf;char name[32];// http协议struct HttpRequest* request;struct HttpResponse* r…
阅读更多...
LabVIEW在旋转机械故障诊断中的随机共振增强应用

LabVIEW在旋转机械故障诊断中的随机共振增强应用

在现代工业自动化领域，准确的故障诊断对于保障机械设备的稳定运行至关重要。传统的故障检测方法往往因噪声干扰而难以捕捉到微弱的故障信号。随着LabVIEW在数据处理和系统集成方面的优势日益凸显，其在旋转机械故障诊断中的应用开始发挥重要作用&#xff…
阅读更多...
Spring学习 Spring整合MyBatis

Spring学习 Spring整合MyBatis

6.1.创建工程 6.1.1.pom.xml <?xml version"1.0" encoding"UTF-8"?> <project xmlns"http://maven.apache.org/POM/4.0.0"xmlns:xsi"http://www.w3.org/2001/XMLSchema-instance"xsi:schemaLocation"http://maven.ap…
阅读更多...
20240108移远的4G模块EC20在Firefly的AIO-3399J开发板的Android11下调通的步骤

20240108移远的4G模块EC20在Firefly的AIO-3399J开发板的Android11下调通的步骤

20240108移远的4G模块EC20在Firefly的AIO-3399J开发板的Android11下调通的步骤 2024/1/8 17:50 缘起&#xff1a;使用友善之臂的Android11可以让EC20上网&#xff0c;但是同样的修改步骤&#xff0c;Toybrick的Android11不能让EC20上网。最后确认是selinux的问题&#xff01; …
阅读更多...
推荐文章
最新文章

Copyright @ 2022~2023