langchain4j+local-ai小试牛刀

序

本文主要研究一下如何本地运行local-ai并通过langchain4j集成调用。

步骤

curl安装

curl https://localai.io/install.sh | sh% Total    % Received % Xferd  Average Speed   Time    Time     Time  CurrentDload  Upload   Total   Spent    Left  Speed
100 21509    0 21509    0     0  45959      0 --:--:-- --:--:-- --:--:-- 45861Downloading local-ai...
######################################################################## 100.0%Installing local-ai to /usr/local/bin...Creating local-ai user...Adding current user to local-ai group...Creating local-ai systemd service...The LocalAI API is now available at 127.0.0.1:8080.Install complete. Run "local-ai" from the command line.
[WARN]  No NVIDIA/AMD GPU detected. LocalAI will run in CPU-only mode.

运行

local-ai
9:49AM INF env file found, loading environment variables from file envFile=/etc/localai.env
9:49AM INF Setting logging to info
9:49AM INF Starting LocalAI using 16 threads, with models path: /usr/share/local-ai/models
9:49AM INF LocalAI version: v2.26.0 (09941c0bfb9119bb01a04b2a0a16897ecf2cd087)
9:49AM INF Preloading models from /usr/share/local-ai/models
9:49AM INF core/startup process completed!
9:49AM INF LocalAI API is listening! Please connect to the endpoint for API documentation. endpoint=http://0.0.0.0:8080

访问http://localhost:8080

安装model

访问http://localhost:8080/browse 浏览模型并选择安装
或者使用命令行，这里选择一个小模型

local-ai models install llama-3.2-1b-instruct:q4_k_m

https://hf-mirror.com/hugging-quants/Llama-3.2-1B-Instruct-Q4_K_M-GGUF

pom.xml

        <dependency><groupId>dev.langchain4j</groupId><artifactId>langchain4j-local-ai</artifactId><version>1.0.0-beta1</version></dependency>

controller

@RequestMapping("/local-ai")
@RestController
@Slf4j
public class LocalAiController {ChatLanguageModel model = LocalAiChatModel.builder().baseUrl("http://localhost:8080/v1").modelName("llama-3.2-1b-instruct:q4_k_m")
//            .maxTokens(3).logRequests(true).logResponses(true).build();/*** curl -i http://localhost:8080/local-ai/chat?prompt=who are you* @param prompt* @return*/@GetMapping("/chat")public String chat(@RequestParam("prompt") String prompt) {return model.chat(prompt);}}

请求

http://localhost:8080/local-ai/chat?prompt=who%20are%20you
返回

I'm an AI assistant, which means I'm a computer program designed to understand and respond to human language. I'm here to help with a wide range of topics, answer questions, provide information, and even chat with you in a friendly and helpful way. I'm a large language model, which means I've been trained on a massive database of text from the internet, books, and other sources, which allows me to generate responses that are often surprisingly accurate and insightful. Some common ways I can assist you include: * Answering questions on a particular topic * Providing definitions and explanations * Offering suggestions and ideas * Summarizing long pieces of text * Translating languages (I can translate text from one language to another) * Offering writing and proofreading services * Generating text on a given topic or topic I'm always learning and improving, so please bear with me if I make any mistakes or don't quite understand the question. I'm here to help, so feel free to ask me anything that's on your mind!