190 lines
3.7 KiB
Plaintext
190 lines
3.7 KiB
Plaintext
Metadata-Version: 2.1
|
|
Name: ollama
|
|
Version: 0.1.5
|
|
Summary: The official Python client for Ollama.
|
|
Home-page: https://ollama.ai
|
|
License: MIT
|
|
Author: Ollama
|
|
Author-email: hello@ollama.com
|
|
Requires-Python: >=3.8,<4.0
|
|
Classifier: License :: OSI Approved :: MIT License
|
|
Classifier: Programming Language :: Python :: 3
|
|
Classifier: Programming Language :: Python :: 3.8
|
|
Classifier: Programming Language :: Python :: 3.9
|
|
Classifier: Programming Language :: Python :: 3.10
|
|
Classifier: Programming Language :: Python :: 3.11
|
|
Classifier: Programming Language :: Python :: 3.12
|
|
Requires-Dist: httpx (>=0.25.2,<0.26.0)
|
|
Project-URL: Repository, https://github.com/jmorganca/ollama-python
|
|
Description-Content-Type: text/markdown
|
|
|
|
# Ollama Python Library
|
|
|
|
The Ollama Python library provides the easiest way to integrate Python 3.8+ projects with [Ollama](https://github.com/jmorganca/ollama).
|
|
|
|
## Install
|
|
|
|
```sh
|
|
pip install ollama
|
|
```
|
|
|
|
## Usage
|
|
|
|
```python
|
|
import ollama
|
|
response = ollama.chat(model='llama2', messages=[
|
|
{
|
|
'role': 'user',
|
|
'content': 'Why is the sky blue?',
|
|
},
|
|
])
|
|
print(response['message']['content'])
|
|
```
|
|
|
|
## Streaming responses
|
|
|
|
Response streaming can be enabled by setting `stream=True`, modifying function calls to return a Python generator where each part is an object in the stream.
|
|
|
|
```python
|
|
import ollama
|
|
|
|
stream = ollama.chat(
|
|
model='llama2',
|
|
messages=[{'role': 'user', 'content': 'Why is the sky blue?'}],
|
|
stream=True,
|
|
)
|
|
|
|
for chunk in stream:
|
|
print(chunk['message']['content'], end='', flush=True)
|
|
```
|
|
|
|
## API
|
|
|
|
The Ollama Python library's API is designed around the [Ollama REST API](https://github.com/jmorganca/ollama/blob/main/docs/api.md)
|
|
|
|
### Chat
|
|
|
|
```python
|
|
ollama.chat(model='llama2', messages=[{'role': 'user', 'content': 'Why is the sky blue?'}])
|
|
```
|
|
|
|
### Generate
|
|
|
|
```python
|
|
ollama.generate(model='llama2', prompt='Why is the sky blue?')
|
|
```
|
|
|
|
### List
|
|
|
|
```python
|
|
ollama.list()
|
|
```
|
|
|
|
### Show
|
|
|
|
```python
|
|
ollama.show('llama2')
|
|
```
|
|
|
|
### Create
|
|
|
|
```python
|
|
modelfile='''
|
|
FROM llama2
|
|
SYSTEM You are mario from super mario bros.
|
|
'''
|
|
|
|
ollama.create(model='example', modelfile=modelfile)
|
|
```
|
|
|
|
### Copy
|
|
|
|
```python
|
|
ollama.copy('llama2', 'user/llama2')
|
|
```
|
|
|
|
### Delete
|
|
|
|
```python
|
|
ollama.delete('llama2')
|
|
```
|
|
|
|
### Pull
|
|
|
|
```python
|
|
ollama.pull('llama2')
|
|
```
|
|
|
|
### Push
|
|
|
|
```python
|
|
ollama.push('user/llama2')
|
|
```
|
|
|
|
### Embeddings
|
|
|
|
```python
|
|
ollama.embeddings(model='llama2', prompt='They sky is blue because of rayleigh scattering')
|
|
```
|
|
|
|
## Custom client
|
|
|
|
A custom client can be created with the following fields:
|
|
|
|
- `host`: The Ollama host to connect to
|
|
- `timeout`: The timeout for requests
|
|
|
|
```python
|
|
from ollama import Client
|
|
client = Client(host='http://localhost:11434')
|
|
response = client.chat(model='llama2', messages=[
|
|
{
|
|
'role': 'user',
|
|
'content': 'Why is the sky blue?',
|
|
},
|
|
])
|
|
```
|
|
|
|
## Async client
|
|
|
|
```python
|
|
import asyncio
|
|
from ollama import AsyncClient
|
|
|
|
async def chat():
|
|
message = {'role': 'user', 'content': 'Why is the sky blue?'}
|
|
response = await AsyncClient().chat(model='llama2', messages=[message])
|
|
|
|
asyncio.run(chat())
|
|
```
|
|
|
|
Setting `stream=True` modifies functions to return a Python asynchronous generator:
|
|
|
|
```python
|
|
import asyncio
|
|
from ollama import AsyncClient
|
|
|
|
async def chat():
|
|
message = {'role': 'user', 'content': 'Why is the sky blue?'}
|
|
async for part in await AsyncClient().chat(model='llama2', messages=[message], stream=True):
|
|
print(part['message']['content'], end='', flush=True)
|
|
|
|
asyncio.run(chat())
|
|
```
|
|
|
|
## Errors
|
|
|
|
Errors are raised if requests return an error status or if an error is detected while streaming.
|
|
|
|
```python
|
|
model = 'does-not-yet-exist'
|
|
|
|
try:
|
|
ollama.chat(model)
|
|
except ollama.ResponseError as e:
|
|
print('Error:', e.error)
|
|
if e.status_code == 404:
|
|
ollama.pull(model)
|
|
```
|
|
|