IBM watsonx.ai
WatsonxLLM 是 IBM watsonx.ai 基础模型的包装器。
此示例演示了如何使用LangChain与watsonx.ai模型进行通信。
设置
安装包langchain-ibm。
!pip install -qU langchain-ibm
此单元格定义了与 watsonx 基础模型推理配合使用所需的 WML 凭据。
操作：提供 IBM Cloud 用户 API 密钥。有关详细信息，请参阅文档。
import os
from getpass import getpass
watsonx_api_key = getpass()
os.environ["WATSONX_APIKEY"] = watsonx_api_key
此外，您可以将其他机密作为环境变量传递。
import os
os.environ["WATSONX_URL"] = "your service instance url"
os.environ["WATSONX_TOKEN"] = "your token for accessing the CPD cluster"
os.environ["WATSONX_PASSWORD"] = "your password for accessing the CPD cluster"
os.environ["WATSONX_USERNAME"] = "your username for accessing the CPD cluster"
os.environ["WATSONX_INSTANCE_ID"] = "your instance_id for accessing the CPD cluster"
加载模型
您可能需要为不同的模型或任务调整模型parameters。有关详细信息，请参阅文档。
parameters = {
"decoding_method": "sample",
"max_new_tokens": 100,
"min_new_tokens": 1,
"temperature": 0.5,
"top_k": 50,
"top_p": 1,
}
使用先前设置的参数初始化WatsonxLLM类。
注意:
在此示例中，我们将使用project_id和达拉斯 URL。
您需要指定将用于推理的model_id。所有可用模型都可以在文档中找到。
from langchain_ibm import WatsonxLLM
watsonx_llm = WatsonxLLM(
model_id="ibm/granite-13b-instruct-v2",
url="https://us-south.ml.cloud.ibm.com",
project_id="PASTE YOUR PROJECT_ID HERE",
params=parameters,
)
或者，您可以使用 Cloud Pak for Data 凭据。有关详细信息，请参阅文档。
watsonx_llm = WatsonxLLM(
model_id="ibm/granite-13b-instruct-v2",
url="PASTE YOUR URL HERE",
username="PASTE YOUR USERNAME HERE",
password="PASTE YOUR PASSWORD HERE",
instance_id="openshift",
version="4.8",
project_id="PASTE YOUR PROJECT_ID HERE",
params=parameters,
)
除了model_id，您还可以传递先前调整的模型的deployment_id。整个模型调整工作流程此处进行了描述。
watsonx_llm = WatsonxLLM(
deployment_id="PASTE YOUR DEPLOYMENT_ID HERE",
url="https://us-south.ml.cloud.ibm.com",
project_id="PASTE YOUR PROJECT_ID HERE",
params=parameters,
)
对于某些要求，可以选择将 IBM 的APIClient对象传递到WatsonxLLM类中。
from ibm_watsonx_ai import APIClient
api_client = APIClient(...)
watsonx_llm = WatsonxLLM(
model_id="ibm/granite-13b-instruct-v2",
watsonx_client=api_client,
)
您还可以将 IBM 的ModelInference对象传递到WatsonxLLM类中。
from ibm_watsonx_ai.foundation_models import ModelInference
model = ModelInference(...)
watsonx_llm = WatsonxLLM(watsonx_model=model)
创建链
创建PromptTemplate对象，这些对象将负责创建随机问题。
from langchain_core.prompts import PromptTemplate
template = "Generate a random question about {topic}: Question: "
prompt = PromptTemplate.from_template(template)
API 参考：PromptTemplate
提供主题并运行链。
llm_chain = prompt | watsonx_llm
topic = "dog"
llm_chain.invoke(topic)
'What is the difference between a dog and a wolf?'
直接调用模型
要获取完成，您可以使用字符串提示直接调用模型。
# Calling a single prompt
watsonx_llm.invoke("Who is man's best friend?")
"Man's best friend is his dog. "
# Calling multiple prompts
watsonx_llm.generate(
[
"The fastest dog in the world?",
"Describe your chosen dog breed",
]
)
LLMResult(generations=[[Generation(text='The fastest dog in the world is the greyhound, which can run up to 45 miles per hour. This is about the same speed as a human running down a track. Greyhounds are very fast because they have long legs, a streamlined body, and a strong tail. They can run this fast for short distances, but they can also run for long distances, like a marathon. ', generation_info={'finish_reason': 'eos_token'})], [Generation(text='The Beagle is a scent hound, meaning it is bred to hunt by following a trail of scents.', generation_info={'finish_reason': 'eos_token'})]], llm_output={'token_usage': {'generated_token_count': 106, 'input_token_count': 13}, 'model_id': 'ibm/granite-13b-instruct-v2', 'deployment_id': ''}, run=[RunInfo(run_id=UUID('52cb421d-b63f-4c5f-9b04-d4770c664725')), RunInfo(run_id=UUID('df2ea606-1622-4ed7-8d5d-8f6e068b71c4'))])
流式传输模型输出
您可以流式传输模型输出。
for chunk in watsonx_llm.stream(
"Describe your favorite breed of dog and why it is your favorite."
):
print(chunk, end="")
My favorite breed of dog is a Labrador Retriever. Labradors are my favorite because they are extremely smart, very friendly, and love to be with people. They are also very playful and love to run around and have a lot of energy.