概述#
在这里，我们将看到如何定义一个可以调用外部 python 函数的 Kork 代码链。
import langchain
from langchain.chat_models import ChatOpenAI
from langchain.llms import OpenAI
from kork import CodeChain
外部函数#
让我们创建一个我们希望链能够调用的 python 函数列表。
关于这些函数的信息将被添加到提示中，以告知 LLM 它可以使用它们。
这些函数也将作为外部函数添加到 kork 解释器环境中，以便解释器可以实际使用它们！
def output_with_matplotlib(output: str) -> None:
"""Function that will output a plot using matplotlib."""
if not isinstance(output, str):
raise ValueError(f"LLM failed to produce a string!")
import matplotlib.pyplot as plt
fig, ax = plt.subplots()
ax.axis("off")
ax.text(0.5, 0.5, output, ha="center", va="center")
plt.show()
def foo(what_to_print: str) -> str:
"""A function that something to the screen"""
print("Invoking a custom user defined function")
try:
import matplotlib
output_with_matplotlib(what_to_print)
except ImportError:
print("Matplotlib missing so using print")
print(what_to_print)
print("Finished invoking a custom user defined function")
import math
foreign_functions = [foo, math.pow, math.log2, math.log10, math.sqrt]
示例#
使用示例来教导 llm 如何在目标问题域中工作。在文档的后面，我们将看到如何自定义整个提示。
如果程序调用函数，请确保函数名称与提供的外部函数名称完全匹配（目前没有验证）。
我们可以以文本格式指定程序。
examples = [
("calculate the sqrt of 2", "let result = pow(2, 0.5)"),
("2*5 + 1", "let result = 2 * 5 + 1"),
("1.3e-3", "let result = 1.3 * pow(10, -3)"),
("2**5", "let result = pow(2, 5)"),
("calculate log of 2", "let result = log2(2)"),
("set the value of result to the string hello", 'let result = "hello";'),
(
"every day i eat 3 donuts. how many donuts do i eat during the week",
"let days_in_week = 7; let result = days_in_week * 3;",
),
]
但在将它们输入到链之前，我们需要将它们转换为抽象语法树 (ast)
from kork.parser import parse
examples_in_ast = [(query, parse(code)) for query, code in examples]
现在，第一个示例由查询的 2 元组和以 Kork AST 表示的相应程序组成。
examples_in_ast[0]
('calculate the sqrt of 2',
Program(stmts=(VarDecl(name='result', value=FunctionCall(name='pow', args=(Literal(value=2), Literal(value=0.5)))),)))
代码链#
链的输入将在文档的后面详细解释。
现在，让我们运行代码链，看看生成的提示和链的输出！
llm = OpenAI(
model_name="text-davinci-003",
temperature=0,
max_tokens=2000,
frequency_penalty=0,
presence_penalty=0,
top_p=1.0,
verbose=True,
)
chain = CodeChain.from_defaults(
llm=llm,
examples=examples_in_ast,
context=foreign_functions,
)
让我们看看我们是否可以让 LLM 调用我们的 print 函数？
code_result = chain(
inputs={
"query": "write a program to output a funny sentence about a cat to the screen"
}
)
Invoking a custom user defined function
Finished invoking a custom user defined function
这是 LLM 编写的代码。
from kork.display import display_code_result
display_code_result(code_result, columns=["query", "code"])
| 查询 | 代码 | |
|---|---|---|
| 0 | 编写一个程序来输出一个 关于猫的有趣的句子到 屏幕上 |
var result = foo("Cats are so funny, they make me LOL!") |
提示
LLM 成功调用了一个名为 foo 的自定义用户函数，该函数打印到屏幕上。🤔
随意尝试并创建可以创建图像、播放声音等功能的函数。
该链将能够调用您提供的任何函数。
注意
LLM 可能会在程序末尾返回 var result = 0。
示例和提示偏向于使其以变量声明语句结束。
提示和示例可以自定义以更改此行为。
警告
即使在像这样的简单任务中，您也可能会看到失败。请参阅指南部分。
提示#
让我们打开 verbose 模式，看看生成的提示。
langchain.verbose = True
让我们尝试一个更复杂的句子。
query = "An ecoli cell doubles every 30 mins in a test tube. How long will it take 1 cell to become 16 cells? Use variable declarations and function invocations."
code_result = chain(inputs={"query": query})
> Entering new LLMChain chain...
Prompt after formatting:
You are programming in a language called "😼".
You are an expert programmer and must follow the instructions below exactly.
Your goal is to translate a user query into a corresponding and valid 😼
program.
You have access to the following external functions:
```😼
extern fn foo(what_to_print: str) -> str // A function that something to the screen
extern fn pow(x: Any, y: Any) -> Any // Return x**y (x to the power of y).
extern fn log2(x: Any) -> Any // Return the base 2 logarithm of x.
extern fn log10(x: Any) -> Any // Return the base 10 logarithm of x.
extern fn sqrt(x: Any) -> Any // Return the square root of x.
```
Do not assume that any other functions except for the ones listed above exist.
Wrap the program in <code> and </code> tags.
Store the solution to the query in a variable called "result".
Here is a sample valid program:
<code>
var x = 1 # Assign 1 to the variable x
var result = 1 + 2 # Calculate the sum of 1 + 2 and assign to result
var result = x # Assign the value of x to result
</code>
Guidelines:
- Do not use operators, instead invoke appropriate external functions.
- Do not declare functions, do not use loops, do not use conditionals.
- Solve the problem only using variable declarations and function invocations.
Begin!
Input: ```text
calculate the sqrt of 2
```
Output: <code>var result = pow(2, 0.5)</code>
Input: ```text
2*5 + 1
```
Output: <code>var result = 2 * 5 + 1</code>
Input: ```text
1.3e-3
```
Output: <code>var result = 1.3 * pow(10, -3)</code>
Input: ```text
2**5
```
Output: <code>var result = pow(2, 5)</code>
Input: ```text
calculate log of 2
```
Output: <code>var result = log2(2)</code>
Input: ```text
set the value of result to the string hello
```
Output: <code>var result = "hello"</code>
Input: ```text
every day i eat 3 donuts. how many donuts do i eat during the week
```
Output: <code>var days_in_week = 7
var result = days_in_week * 3</code>
Input: ```text
An ecoli cell doubles every 30 mins in a test tube. How long will it take 1 cell to become 16 cells? Use variable declarations and function invocations.
```
Output:
> Finished chain.
llm 输出为
display_code_result(code_result, expected_answer=120.0)
| 查询 | 代码 | 结果 | 预期 | 正确 | 错误 | 原始 | |
|---|---|---|---|---|---|---|---|
| 0 | 一个大肠杆菌细胞每 30 分钟在试管中翻倍。需要多久 一个细胞变成 16 个细胞？使用变量 声明和函数 调用。 |
var cell_count = 1 |
120.0 | 120.0 | ✅ | [] | <code>var cell_count = 1 var target_cell_count = 16 var doubling_time = 30 var result = log2(target_cell_count / cell_count) * doubling_time</code> |
环境#
运行链后，在解释器完成代码运行后，我们可以访问环境的状态。
我们可以从环境中获取 result 变量。
type(code_result["environment"])
kork.environment.Environment
大肠杆菌细胞计算的结果存储在 result 符号中
print(code_result["environment"].get_symbol("result"))
120.0
相对论动能#
我们可以尝试进行更复杂的物理计算。
重要
LLM 在处理更长更复杂的任务时可能会遇到困难。
指南
使用更好的 LLM
为您的任务自定义提示
提供示例
提供简单且包含类型注释和简短文档字符串的外部函数。
注意
Kork 仍处于开发阶段。
它尚未与 LLM 进行第二次传递，以帮助纠正生成的程序中的错误，这可能会有助于提高结果。
query = """\
Let's calculate the kinetic energy of a relativistic electron moving at 0.9999c in MeV.
First list all relevant physical constants and conver them into an appropriate units.
Now, calculate the lorentz factor.
Then subtract one from it, and multiply the result by the rest mass of an electron in MeV.
Make sure to use the listed foreign functions and no binary arithemtic operators."""
code_result = chain(inputs={"query": query})
> Entering new LLMChain chain...
Prompt after formatting:
You are programming in a language called "😼".
You are an expert programmer and must follow the instructions below exactly.
Your goal is to translate a user query into a corresponding and valid 😼
program.
You have access to the following external functions:
```😼
extern fn foo(what_to_print: str) -> str // A function that something to the screen
extern fn pow(x: Any, y: Any) -> Any // Return x**y (x to the power of y).
extern fn log2(x: Any) -> Any // Return the base 2 logarithm of x.
extern fn log10(x: Any) -> Any // Return the base 10 logarithm of x.
extern fn sqrt(x: Any) -> Any // Return the square root of x.
```
Do not assume that any other functions except for the ones listed above exist.
Wrap the program in <code> and </code> tags.
Store the solution to the query in a variable called "result".
Here is a sample valid program:
<code>
var x = 1 # Assign 1 to the variable x
var result = 1 + 2 # Calculate the sum of 1 + 2 and assign to result
var result = x # Assign the value of x to result
</code>
Guidelines:
- Do not use operators, instead invoke appropriate external functions.
- Do not declare functions, do not use loops, do not use conditionals.
- Solve the problem only using variable declarations and function invocations.
Begin!
Input: ```text
calculate the sqrt of 2
```
Output: <code>var result = pow(2, 0.5)</code>
Input: ```text
2*5 + 1
```
Output: <code>var result = 2 * 5 + 1</code>
Input: ```text
1.3e-3
```
Output: <code>var result = 1.3 * pow(10, -3)</code>
Input: ```text
2**5
```
Output: <code>var result = pow(2, 5)</code>
Input: ```text
calculate log of 2
```
Output: <code>var result = log2(2)</code>
Input: ```text
set the value of result to the string hello
```
Output: <code>var result = "hello"</code>
Input: ```text
every day i eat 3 donuts. how many donuts do i eat during the week
```
Output: <code>var days_in_week = 7
var result = days_in_week * 3</code>
Input: ```text
Let's calculate the kinetic energy of a relativistic electron moving at 0.9999c in MeV.
First list all relevant physical constants and conver them into an appropriate units.
Now, calculate the lorentz factor.
Then subtract one from it, and multiply the result by the rest mass of an electron in MeV.
Make sure to use the listed foreign functions and no binary arithemtic operators.
```
Output:
> Finished chain.
expected_answer = 0.511 * ((1 - 0.9999**2) ** (-0.5) - 1)
警告
此计算的成功率高于 5%，但肯定低于 80%。由于底层 LLM，存在巨大差异！
这是一个在事情按预期工作时精心挑选的运行。可能需要与 LLM 进行第二次传递以获得更强的鲁棒性。
display_code_result(code_result, expected_answer=expected_answer)
| 查询 | 代码 | 结果 | 预期 | 正确 | 错误 | 原始 | |
|---|---|---|---|---|---|---|---|
| 0 | 让我们计算 一个 相对论电子的动能，该电子以 0.9999c 的速度（单位为 MeV）移动。首先列出 所有相关的物理 常数，并将它们转换为 适当的单位。现在， 计算洛伦兹因子。 然后从中减去一，并将结果 乘以 电子的静止质量（单位为 MeV）。确保使用 列出的外部函数，并且 不使用二进制算术 运算符。 |
var c = 299792458 // Speed of light in m/s |
35.62305988140826 | 35.62305988142832 | ✅ | [] | <code>var c = 299792458 // Speed of light in m/s var rest_mass_electron = 0.511 // Rest mass of an electron in MeV var speed = 0.9999 * c // Speed of the electron in m/s var lorentz_factor = 1 / sqrt(1 - pow(speed, 2) / pow(c, 2)) // Calculate the lorentz factor var result = (lorentz_factor - 1) * rest_mass_electron // Calculate the kinetic energy in MeV</code> |
使用 LLM 进行计算#
随着 LLM 变得越来越好，它们将更好地逼近正确答案（甚至无需指定算法）。
您可以尝试使用 GPT-4 或 chat gpt 进行此练习。使用适当的提示和模型，结果看起来会相当合理！
或者，您可以将结果与 langchain 数学链进行比较！
作为一项实验，您可以使用 LLM 为给定的科学问题生成计算计划，然后使用代码链将指令转换为程序。
警告
与人类类似，LLM 似乎在单位转换方面有点困难。😹
但是知识就在那里，所以也许我们只需要足够聪明！
print(
llm.generate(
prompts=[
"What is the relativistic kinetic energy of an electron moving at 0.999999 c expressed in MeV?"
]
)
.generations[0][0]
.text
)
The relativistic kinetic energy of an electron moving at 0.999999 c is 7.979 MeV.
print(
llm.generate(
prompts=[
"What is the relativistic kinetic energy of an electron moving at 0.9999 c expressed in MeV explain step by step calculation?"
]
)
.generations[0][0]
.text
)
The relativistic kinetic energy of an electron moving at 0.9999 c can be calculated using the equation:
KE = (γ - 1)mc2
Where γ is the Lorentz factor, m is the mass of the electron, and c is the speed of light.
Step 1: Calculate the Lorentz factor
γ = 1/√(1 - v2/c2)
γ = 1/√(1 - (0.9999)2)
γ = 1/√(1 - 0.99980001)
γ = 1/√(0.00019999)
γ = 1/0.0141
γ = 70.7107
Step 2: Calculate the relativistic kinetic energy
KE = (γ - 1)mc2
KE = (70.7107 - 1)mc2
KE = 69.7107mc2
KE = 69.7107 x (9.109 x 10-31 kg) x (3 x 108 m/s)2
KE = 1.945 x 10-13 J
Step 3: Convert Joules to MeV
1 MeV = 1.602 x 10-13 J
KE = 1.945 x 10-13 J / 1.602 x 10-13 J/MeV
KE = 1.21 MeV
print(
llm.generate(
prompts=[
"Generate step by step instructions to calculate the relativistic kinetic energy of an electron moving at 0.9999c?"
]
)
.generations[0][0]
.text
)
1. Determine the speed of the electron in terms of the speed of light (c). In this case, the electron is moving at 0.9999c.
2. Calculate the relativistic factor (γ) using the equation γ = 1/√(1-v2/c2).
3. Calculate the relativistic kinetic energy (Ek) using the equation Ek = γmc2 - mc2.
4. Substitute the values for γ and c into the equation and solve for Ek.