1.ollama导入自己微调后的模型胡言乱语
原因：Modelfile的模板格式不对，对的如下所示
##格式0FROM tinyllama-my-model.gguf### Set the system message
SYSTEM """
You are a super helpful helper.
"""PARAMETER stop <s>
PARAMETER stop </s>#格式0的运行方式：ollama run my-model "<s>\nQ: \nWhat is the capital of France?\nA:\n"
##格式1
FROM ./llama3-unsloth.Q8_0.ggufTEMPLATE """{{- if .System }}
<|system|>
{{ .System }}
{{- end }}
<|user|>
{{ .Prompt }}
<|assistant|>
"""SYSTEM """You are a helpful, smart, kind, and efficient AI assistant.Your name is Aila. You always fulfill the user's requests to the best of your ability."""PARAMETER temperature 0.8
PARAMETER num_ctx 8192
PARAMETER stop "<|system|>"
PARAMETER stop "<|user|>"
PARAMETER stop "<|assistant|>"##格式2
FROM GEITje-7B-chat-v2.gguf
TEMPLATE """{{- if .System }}
<|system|>
{{ .System }}
</s>
{{- end }}
<|user|>
{{ .Prompt }}
</s>
<|assistant|>
"""
PARAMETER temperature 0.2
PARAMETER num_ctx 8192
PARAMETER stop "<|system|>"
PARAMETER stop "<|user|>"
PARAMETER stop "<|assistant|>"
PARAMETER stop "</s>"
##格式3
FROM models/tinyllama-1.1b-chat-v0.3.Q6_K.gguf
PARAMETER temperature 0.7
PARAMETER stop "<|im_start|>"
PARAMETER stop "<|im_end|>"
TEMPLATE """
<|im_start|>system
{{ .System }}<|im_end|>
<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
"""
SYSTEM """You are a helpful assistant."""
ollama create example -f Modelfileollama run example2.将safesensor转为gguf：
下载llama.cpp
https://github.com/ggerganov/llama.cppcd llama.cpp && make clean && make all -j
python convert.py /home/Ubuntu/phit/model-00001-of-00002.safetensors --outfile /home/Ubuntu/phit/your_output.gguf --pad-vocab
Import GGUF into Ollama
微调Phi-3和Llama3！AutoGen实现Phi-3+Llama3多LLMs对话！打造自己的AI Agent！windows系统也支持_哔哩哔哩_bilibili
)
