---
url: /providers/anthropic.md
---
# Anthropic
## Configuration
```php
'anthropic' => [
'api_key' => env('ANTHROPIC_API_KEY', ''),
'version' => env('ANTHROPIC_API_VERSION', '2023-06-01'),
'default_thinking_budget' => env('ANTHROPIC_DEFAULT_THINKING_BUDGET', 1024),
// Include beta strings as a comma separated list.
'anthropic_beta' => env('ANTHROPIC_BETA', null),
]
```
## Prompt caching
Anthropic's prompt caching feature allows you to drastically reduce latency and your API bill when repeatedly re-using blocks of content within five minutes of each other.
We support Anthropic prompt caching on:
* System Messages (text only)
* User Messages (Text, Image and PDF (pdf only))
* Assistant Messages (text only)
* Tools
The API for enabling prompt caching is the same for all, enabled via the `withProviderOptions()` method. Where a UserMessage contains both text and an image or document, both will be cached.
```php
use Prism\Prism\Enums\Provider;
use Prism\Prism\Prism;
use Prism\Prism\Tool;
use Prism\Prism\ValueObjects\Messages\UserMessage;
use Prism\Prism\ValueObjects\Messages\SystemMessage;
Prism::text()
->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
->withMessages([
(new SystemMessage('I am a long re-usable system message.'))
->withProviderOptions(['cacheType' => 'ephemeral']),
(new UserMessage('I am a long re-usable user message.'))
->withProviderOptions(['cacheType' => 'ephemeral'])
])
->withTools([
Tool::as('cache me')
->withProviderOptions(['cacheType' => 'ephemeral'])
])
->asText();
```
If you prefer, you can use the `AnthropicCacheType` Enum like so:
```php
use Prism\Prism\Enums\Provider;
use Prism\Prism\Providers\Anthropic\Enums\AnthropicCacheType;
use Prism\Prism\ValueObjects\Messages\UserMessage;
use Prism\Prism\ValueObjects\Media\Document;
(new UserMessage('I am a long re-usable user message.'))->withProviderOptions(['cacheType' => AnthropicCacheType::ephemeral])
```
Note that you must use the `withMessages()` method in order to enable prompt caching, rather than `withPrompt()` or `withSystemPrompt()`.
### Tool result caching
In addition to caching prompts and tool definitions, Prism supports caching tool results. This is particularly useful when making multiple tool calls where results might be referenced repeatedly.
To enable tool result caching, use the `tool_result_cache_type` provider option on your request:
```php
use Prism\Prism\Prism;
$response = Prism::text()
->using('anthropic', 'claude-3-5-sonnet-20241022')
->withMaxSteps(30)
->withTools([new WeatherTool()])
->withProviderOptions([
'tool_result_cache_type' => 'ephemeral'
])
->withPrompt('Check the weather in New York, London, Tokyo, Paris, and Sydney')
->asText();
```
When multiple tool results are returned, Prism automatically applies caching to only the last result, which caches all preceding results as well. This avoids Anthropic's 4-cache-breakpoint limitation.
Please ensure you read Anthropic's [prompt caching documentation](https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching), which covers some important information on e.g. minimum cacheable tokens and message order consistency.
## Extended thinking
Claude Sonnet 3.7 supports an optional extended thinking mode, where it will reason before returning its answer. Please ensure your consider [Anthropic's own extended thinking documentation](https://docs.anthropic.com/en/docs/build-with-claude/extended-thinking) before using extended thinking with caching and/or tools, as there are some important limitations and behaviours to be aware of.
### Enabling extended thinking and setting budget
Prism supports thinking mode for text and structured with the same API:
```php
use Prism\Prism\Enums\Provider;
use Prism\Prism\Prism;
Prism::text()
->using('anthropic', 'claude-3-7-sonnet-latest')
->withPrompt('What is the meaning of life, the universe and everything in popular fiction?')
// enable thinking
->withProviderOptions(['thinking' => ['enabled' => true]])
->asText();
```
By default Prism will set the thinking budget to the value set in config, or where that isn't set, the minimum allowed (1024).
You can overide the config (or its default) using `withProviderOptions`:
```php
use Prism\Prism\Enums\Provider;
use Prism\Prism\Prism;
Prism::text()
->using('anthropic', 'claude-3-7-sonnet-latest')
->withPrompt('What is the meaning of life, the universe and everything in popular fiction?')
// Enable thinking and set a budget
->withProviderOptions([
'thinking' => [
'enabled' => true,
'budgetTokens' => 2048
]
]);
```
Note that thinking tokens count towards output tokens, so you will be billed for them and your token budget must be less than the max tokens you have set for the request.
If you expect a long response, you should ensure there's enough tokens left for the response - i.e. does (maxTokens - thinkingBudget) leave a sufficient remainder.
### Inspecting the thinking block
Anthropic returns the thinking block with its response.
You can access it via the additionalContent property on either the Response or the relevant step.
On the Response (easiest if not using tools):
```php
use Prism\Prism\Enums\Provider;
use Prism\Prism\Prism;
Prism::text()
->using('anthropic', 'claude-3-7-sonnet-latest')
->withPrompt('What is the meaning of life, the universe and everything in popular fiction?')
->withProviderOptions(['thinking' => ['enabled' => true']])
->asText();
$response->additionalContent['thinking'];
```
On the Step (necessary if using tools, as Anthropic returns the thinking block on the ToolCall step):
```php
$tools = [...];
$response = Prism::text()
->using('anthropic', 'claude-3-7-sonnet-latest')
->withTools($tools)
->withMaxSteps(3)
->withPrompt('What time is the tigers game today and should I wear a coat?')
->withProviderOptions(['thinking' => ['enabled' => true]])
->asText();
$response->steps->first()->additionalContent->thinking;
```
### Extended output mode
Claude Sonnet 3.7 also brings extended output mode which increase the output limit to 128k tokens.
This feature is currently in beta, so you will need to enable to by adding `output-128k-2025-02-19` to your Anthropic anthropic\_beta config (see [Configuration](#configuration) above).
## Documents
Anthropic supports PDF, text and markdown documents. Note that Anthropic uses vision to process PDFs under the hood, and consequently there are some limitations detailed in their [feature documentation](https://docs.anthropic.com/en/docs/build-with-claude/pdf-support).
See the [Documents](/input-modalities/documents.html) on how to get started using them.
Anthropic also supports "custom content documents", separately documented below, which are primarily for use with citations.
### Custom content documents
Custom content documents are primarily for use with citations (see below), if you need citations to reference your own chunking strategy.
```php
use Prism\Prism\Enums\Provider;
use Prism\Prism\Prism;
use Prism\Prism\ValueObjects\Messages\UserMessage;
use Prism\Prism\ValueObjects\Media\Document;
Prism::text()
->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
->withMessages([
new UserMessage(
content: "Is the grass green and the sky blue?",
additionalContent: [
Document::fromChunks(["The grass is green.", "Flamingos are pink.", "The sky is blue."])
]
)
])
->asText();
```
## Citations
Prism supports [Anthropic's citations feature](https://docs.anthropic.com/en/docs/build-with-claude/citations) for both text and structured.
Please note however that due to Anthropic not supporting "native" structured output, and Prism's workaround for this, the output can be unreliable. You should therefore ensure you implement proper error handling for the scenario where Anthropic does not return a valid decodable schema.
## Code execution
Anthropic offers built-in code execution capabilities that allow your AI to run code in a secure environment. This is a provider tool that executes code using Anthropic's infrastructure. For more information about the difference between custom tools and provider tools, see [Tools & Function Calling](/core-concepts/tools-function-calling#provider-tools).
To enable code execution, you will first need to enable the beta feature.
Either in prism/config.php:
```php
'anthropic' => [
...
'anthropic_beta' => 'code-execution-2025-05-22',
],
```
Or in your env file (assuming config/prism.php reflects the default prism setup):
```
ANTHROPIC_BETA="code-execution-2025-05-22"
```
You may then use code execution as follows:
```php
use Prism\Prism\Prism;
use Prism\Prism\ValueObjects\ProviderTool;
Prism::text()
->using('anthropic', 'claude-3-5-haiku-latest')
->withPrompt('Solve the equation 3x + 10 = 14.')
->withProviderTools([new ProviderTool(type: 'code_execution_20250522', name: 'code_execution')])
->asText();
```
### Enabling citations
Anthropic require citations to be enabled on all documents in a request. To enable them, using the `withProviderOptions()` method when building your request:
```php
use Prism\Prism\Enums\Provider;
use Prism\Prism\Prism;
use Prism\Prism\ValueObjects\Messages\UserMessage;
use Prism\Prism\ValueObjects\Media\Document;
$response = Prism::text()
->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
->withMessages([
new UserMessage(
content: "Is the grass green and the sky blue?",
additionalContent: [
Document::fromChunks(
chunks: ["The grass is green.", "Flamingos are pink.", "The sky is blue."],
title: 'The colours of nature',
context: 'The go-to textbook on the colours found in nature!'
)
]
)
])
->withProviderOptions(['citations' => true])
->asText();
```
### Accessing citations
You can access the chunked output with its citations via the additionalContent property on a response, which returns an array of `Providers\Anthropic\ValueObjects\MessagePartWithCitations`s.
As a rough worked example, let's assume you want to implement footnotes. You'll need to loop through those chunks and (1) re-construct the message with links to the footnotes; and (2) build an array of footnotes to loop through in your frontend.
```php
use Prism\Prism\Providers\Anthropic\ValueObjects\MessagePartWithCitations;
use Prism\Prism\Providers\Anthropic\ValueObjects\Citation;
$messageChunks = $response->additionalContent['messagePartsWithCitations'];
$text = '';
$footnotes = [];
$footnoteId = 1;
/** @var MessagePartWithCitations $messageChunk */
foreach ($messageChunks as $messageChunk) {
$text .= $messageChunk->text;
/** @var Citation $citation */
foreach ($messageChunk->citations as $citation) {
$footnotes[] = [
'id' => $footnoteId,
'document_title' => $citation->documentTitle,
'reference_start' => $citation->startIndex,
'reference_end' => $citation->endIndex
];
$text .= ''.$footnoteId.'';
$footnoteId++;
}
}
```
Note that when using streaming, Anthropic does not stream citations in the same way. Instead, of building the context as above, yield text to the browser in the usual way and pair text up with the relevant footnote using the `citationIndex` on the text chunk's additionalContent parameter.
## Considerations
### Message Order
* Message order matters. Anthropic is strict about the message order being:
1. `UserMessage`
2. `AssistantMessage`
3. `UserMessage`
### Structured Output
While Anthropic models don't have native JSON mode or structured output like some providers, Prism implements two approaches for structured output:
#### Default JSON Mode (Prompt-based)
* We automatically append instructions to your prompt that guide the model to output valid JSON matching your schema
* If the response isn't valid JSON, Prism will raise a PrismException
* This method can sometimes struggle with complex JSON containing quotes, especially in non-English languages
#### Tool Calling Mode (Recommended)
For more reliable structured output, especially when dealing with complex content or non-English text that may contain quotes, you can enable tool calling mode:
```php
use Prism\Prism\Enums\Provider;
use Prism\Prism\Prism;
use Prism\Prism\Schema\ObjectSchema;
use Prism\Prism\Schema\StringSchema;
$response = Prism::structured()
->withSchema(new ObjectSchema(
'weather_report',
'Weather forecast with recommendations',
[
new StringSchema('forecast', 'The weather forecast'),
new StringSchema('recommendation', 'Clothing recommendation')
],
['forecast', 'recommendation']
))
->using(Provider::Anthropic, 'claude-3-5-sonnet-latest')
->withPrompt('What\'s the weather like and what should I wear?')
->withProviderOptions(['use_tool_calling' => true])
->asStructured();
```
**Benefits of tool calling mode:**
* More reliable JSON parsing, especially with quotes and special characters
* Better handling of non-English content (Chinese, Japanese, etc.)
* Reduced risk of malformed JSON responses
* Compatible with thinking mode
**Limitations:**
* Cannot be used with citations (citations are not supported in tool calling mode)
* Slightly more complex under the hood but identical API usage
## Limitations
### Messages
Most providers' API include system messages in the messages array with a "system" role. Anthropic does not support the system role, and instead has a "system" property, separate from messages.
Therefore, for Anthropic we:
* Filter all `SystemMessage`s out, omitting them from messages.
* Always submit the prompt defined with `->withSystemPrompt()` at the top of the system prompts array.
* Move all `SystemMessage`s to the system prompts array in the order they were declared.
### Images
Does not support `Image::fromURL`
---
---
url: /input-modalities/audio.md
---
# Audio
Prism supports including audio files in your messages for advanced analysis with supported providers like Gemini.
See the [provider support table](/getting-started/introduction.html#provider-support) to check whether Prism supports your chosen provider.
Note however that provider support may differ by model. If you receive error messages with a provider that Prism indicates is supported, check the provider's documentation as to whether the model you are using supports audio files.
::: tip
For other input modalities like videos and images, see their respective documentation pages:
* [Video documentation](/input-modalities/video.html)
* [Images documentation](/input-modalities/images.html)
:::
## Getting started
To add an audio file to your prompt, use the `withPrompt` method with an `Audio` value object:
```php
use Prism\Prism\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\ValueObjects\Media\Audio;
// From a local path
$response = Prism::text()
->using(Provider::Gemini, 'gemini-1.5-flash')
->withPrompt(
"What's in this audio?",
[Audio::fromLocalPath(path: '/path/to/audio.mp3')]
)
->asText();
// From a path on a storage disk
$response = Prism::text()
->using(Provider::Gemini, 'gemini-1.5-flash')
->withPrompt(
"What's in this audio?",
[Audio::fromStoragePath(
path: '/path/to/audio.mp3',
diskName: 'my-disk' // optional - omit/null for default disk
)]
)
->asText();
// From a URL
$response = Prism::text()
->using(Provider::Gemini, 'gemini-1.5-flash')
->withPrompt(
'Analyze this audio:',
[Audio::fromUrl(url: 'https://example.com/audio.mp3')]
)
->asText();
// From base64
$response = Prism::text()
->using(Provider::Gemini, 'gemini-1.5-flash')
->withPrompt(
'Analyze this audio:',
[Audio::fromBase64(
base64: base64_encode(file_get_contents('/path/to/audio.mp3')),
mimeType: 'audio/mpeg'
)]
)
->asText();
// From raw content
$response = Prism::text()
->using(Provider::Gemini, 'gemini-1.5-flash')
->withPrompt(
'Analyze this audio:',
[Audio::fromRawContent(
rawContent: file_get_contents('/path/to/audio.mp3'),
mimeType: 'audio/mpeg'
)]
)
->asText();
```
## Alternative: Using withMessages
You can also include audio files using the message-based approach:
```php
use Prism\Prism\ValueObjects\Messages\UserMessage;
use Prism\Prism\ValueObjects\Media\Audio;
$message = new UserMessage(
"What's in this audio?",
[Audio::fromLocalPath(path: '/path/to/audio.mp3')]
);
$response = Prism::text()
->using(Provider::Gemini, 'gemini-1.5-flash')
->withMessages([$message])
->asText();
```
## Supported Audio Types
Prism supports a variety of audio formats, including:
* MP3 (audio/mpeg)
* WAV (audio/x-wav, audio/wav)
* AAC (audio/aac)
* FLAC (audio/flac)
The specific supported formats depend on the provider. Gemini is currently the main provider with comprehensive audio analysis capabilities. Check the provider's documentation for a complete list of supported formats.
## Transfer mediums
Providers are not consistent in their support of sending raw contents, base64 and/or URLs.
Prism tries to smooth over these rough edges, but its not always possible.
### Supported conversions
* Where a provider does not support URLs: Prism will fetch the URL and use base64 or rawContent.
* Where you provide a file, base64 or rawContent: Prism will switch between base64 and rawContent depending on what the provider accepts.
### Limitations
* Where a provider only supports URLs: if you provide a file path, raw contents or base64, for security reasons Prism does not create a URL for you and your request will fail.
---
---
url: /core-concepts/audio.md
---
# Audio Processing
Transform text into speech and speech into text using AI-powered audio models. Prism provides a unified API for audio processing across different providers, enabling both text-to-speech (TTS) and speech-to-text (STT) functionality.
## Getting Started
### Text-to-Speech
Convert text into natural-sounding speech:
```php
use Prism\Prism\Prism;
use Prism\Prism\Enums\Provider;
$response = Prism::audio()
->using(Provider::OpenAI, 'tts-1')
->withInput('Hello, this is a test of text-to-speech functionality.')
->withVoice('alloy')
->asAudio();
$audio = $response->audio;
if ($audio->hasBase64()) {
file_put_contents('output.mp3', base64_decode($audio->base64));
echo "Audio saved as: output.mp3";
}
```
### Speech-to-Text
Convert audio files into text transcriptions:
```php
use Prism\Prism\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\ValueObjects\Media\Audio;
$audioFile = Audio::fromPath('/path/to/audio.mp3');
$response = Prism::audio()
->using(Provider::OpenAI, 'whisper-1')
->withInput($audioFile)
->asText();
echo "Transcription: " . $response->text;
```
## Provider Support
Currently, Prism supports audio processing through:
* **OpenAI**: TTS-1, TTS-1-HD (text-to-speech) and Whisper-1 (speech-to-text)
* **Groq**: PlayAI TTS models (text-to-speech) and Whisper Large V3 models (speech-to-text)
Additional providers will be added in future releases as the ecosystem evolves.
## Basic Usage
### Simple Text-to-Speech
Generate speech from text input:
```php
$response = Prism::audio()
->using('openai', 'tts-1')
->withInput('Welcome to our application!')
->withVoice('alloy')
->asAudio();
$audio = $response->audio;
echo "Audio type: " . $audio->getMimeType(); // audio/mpeg
echo "Has audio data: " . ($audio->hasBase64() ? 'Yes' : 'No');
```
### Simple Speech-to-Text
Transcribe audio files to text:
```php
use Prism\Prism\ValueObjects\Media\Audio;
// From file path
$audioFile = Audio::fromPath('/path/to/recording.wav');
// From URL
$audioFile = Audio::fromUrl('https://example.com/audio.mp3');
// From base64 data
$audioFile = Audio::fromBase64($base64AudioData, 'audio/wav');
$response = Prism::audio()
->using('openai', 'whisper-1')
->withInput($audioFile)
->asText();
// Access the transcription
echo $response->text;
```
## Working with Audio Files
### Creating Audio Objects
The `Audio` class provides several ways to work with audio files:
```php
use Prism\Prism\ValueObjects\Media\Audio;
// From local file
$audio = Audio::fromPath('/path/to/audio.mp3');
// From remote URL
$audio = Audio::fromUrl('https://example.com/speech.wav');
// From base64 encoded data
$audio = Audio::fromBase64($base64Data, 'audio/mpeg');
// From raw binary content
$audio = Audio::fromContent($binaryData, 'audio/wav');
```
### Audio Properties
Access audio file information:
```php
$audio = Audio::fromPath('/path/to/audio.mp3');
echo "MIME type: " . $audio->mimeType();
echo "Has local path: " . ($audio->hasLocalPath() ? 'Yes' : 'No');
echo "File size: " . $audio->size() . " bytes";
```
## Working with Responses
### Text-to-Speech Responses
```php
$response = Prism::audio()
->using('openai', 'tts-1-hd')
->withInput('This is high-quality text-to-speech.')
->withVoice('nova')
->asAudio();
// Access the generated audio
$audio = $response->audio;
if ($audio->hasBase64()) {
// Save to file
$audioData = base64_decode($audio->base64);
file_put_contents('speech.mp3', $audioData);
// Get MIME type
echo "Content type: " . $audio->getMimeType();
}
// Access additional response data
foreach ($response->additionalContent as $key => $value) {
echo "{$key}: {$value}\n";
}
```
### Speech-to-Text Responses
```php
$audioFile = Audio::fromPath('/path/to/speech.mp3');
$response = Prism::audio()
->using('openai', 'whisper-1')
->withInput($audioFile)
->asText();
// Access the transcription
$text = $response->text;
echo "Transcription: " . $text;
// Check token usage
if ($response->usage) {
echo "Prompt tokens: " . $response->usage->promptTokens;
echo "Completion tokens: " . $response->usage->completionTokens;
}
// Access raw response data
print_r($response->additionalContent);
```
## Voice Selection
Prism provides a dedicated `withVoice()` method for selecting voices in text-to-speech, making voice selection a first-class citizen in the API:
```php
$response = Prism::audio()
->using('openai', 'tts-1')
->withInput('Hello, how are you today?')
->withVoice('alloy') // Voice options vary by provider
->asAudio();
```
## Provider-Specific Options
While Prism provides a consistent API, you can access provider-specific features using the `withProviderOptions()` method.
### OpenAI Text-to-Speech Options
Customize format, speed, and other options:
```php
$response = Prism::audio()
->using('openai', 'tts-1')
->withInput('Hello, how are you today?')
->withVoice('nova')
->withProviderOptions([
'response_format' => 'mp3', // mp3, opus, aac, flac, wav, pcm
'speed' => 1.0, // 0.25 to 4.0
])
->asAudio();
```
### OpenAI Speech-to-Text Options
Configure transcription settings:
```php
$audioFile = Audio::fromPath('/path/to/multilingual.mp3');
$response = Prism::audio()
->using('openai', 'whisper-1')
->withInput($audioFile)
->withProviderOptions([
'language' => 'en',
'prompt' => 'Previous context...'
])
->asText();
```
### Response Formats
Different response formats provide varying levels of detail:
```php
// Verbose JSON format includes timestamps and confidence scores
$response = Prism::audio()
->using('openai', 'whisper-1')
->withInput($audioFile)
->withProviderOptions([
'response_format' => 'verbose_json',
])
->asText();
// Access additional metadata
$metadata = $response->additionalContent;
if (isset($metadata['segments'])) {
foreach ($metadata['segments'] as $segment) {
echo "Segment: " . $segment['text'] . "\n";
echo "Start: " . $segment['start'] . "s\n";
echo "End: " . $segment['end'] . "s\n";
}
}
```
## Advanced Usage
Audio can be integrated into multi-modal conversations:
```php
use Prism\Prism\ValueObjects\Messages\UserMessage;
use Prism\Prism\ValueObjects\Media\Audio;
use Prism\Prism\ValueObjects\Media\Text;
$audioFile = Audio::fromPath('/path/to/question.mp3');
// First transcribe the audio
$transcription = Prism::audio()
->using('openai', 'whisper-1')
->withInput($audioFile)
->asText();
// Then use in a text conversation
$response = Prism::text()
->using('openai', 'gpt-4')
->withMessages([
new UserMessage('', [
new Text('User asked: '),
new Text($transcription->text),
new Text(' - Please provide a detailed response.')
])
])
->asText();
// Convert response back to speech
$speechResponse = Prism::audio()
->using('openai', 'tts-1')
->withInput($response->text)
->withVoice('alloy')
->asAudio();
```
## Configuration Options
### Client Configuration
Configure HTTP client options for audio processing:
```php
$response = Prism::audio()
->using('openai', 'tts-1')
->withInput('This might take a while to process.')
->withClientOptions([
'timeout' => 60, // Increase timeout for large files
'connect_timeout' => 10, // Connection timeout
])
->withClientRetry(3, 1000) // Retry 3 times with 1s delay
->asAudio();
```
### Provider Configuration
Override provider configuration for multi-tenant applications:
```php
$customConfig = [
'api_key' => 'user-specific-api-key',
'organization' => 'user-org-id',
];
$response = Prism::audio()
->using('openai', 'whisper-1')
->usingProviderConfig($customConfig)
->withInput($audioFile)
->asText();
```
## Error Handling
Handle potential errors in audio processing:
```php
use Prism\Prism\Exceptions\PrismException;
use Throwable;
try {
$response = Prism::audio()
->using('openai', 'tts-1')
->withInput('Text to convert to speech')
->withVoice('alloy')
->asAudio();
// Process successful response
file_put_contents('output.mp3', base64_decode($response->audio->base64));
} catch (PrismException $e) {
Log::error('Audio processing failed:', ['error' => $e->getMessage()]);
// Handle Prism-specific errors
} catch (Throwable $e) {
Log::error('General error:', ['error' => $e->getMessage()]);
// Handle any other errors
}
```
## Testing
Prism provides convenient fakes for testing audio functionality:
```php
use Prism\Prism\Prism;
use Prism\Prism\Testing\PrismFake;
use Prism\Prism\Audio\AudioResponse;
use Prism\Prism\Audio\TextResponse;
use Prism\Prism\ValueObjects\GeneratedAudio;
test('can generate text-to-speech', function () {
$fakeAudio = new AudioResponse(
audio: new GeneratedAudio(
base64: base64_encode('fake-audio-data'),
type: 'audio/mpeg'
)
);
Prism::fake([$fakeAudio]);
$response = Prism::audio()
->using('openai', 'tts-1')
->withInput('Test audio generation')
->withVoice('alloy')
->asAudio();
expect($response->audio->hasBase64())->toBeTrue();
expect($response->audio->getMimeType())->toBe('audio/mpeg');
});
test('can transcribe speech-to-text', function () {
$fakeTranscription = new TextResponse(
text: 'This is a fake transcription'
);
Prism::fake([$fakeTranscription]);
$audioFile = Audio::fromPath('/fake/path/test.mp3');
$response = Prism::audio()
->using('openai', 'whisper-1')
->withInput($audioFile)
->asText();
expect($response->text)->toBe('This is a fake transcription');
});
```
---
---
url: /getting-started/configuration.md
---
# Configuration
Prism's flexible configuration allows you to easily set up and switch between different AI providers. Let's dive into how you can configure Prism to work with your preferred providers.
## Configuration File
After installation, you'll find the Prism configuration file at `config/prism.php`. If you haven't published it yet, you can do so with:
```bash
php artisan vendor:publish --tag=prism-config
```
Let's break down the key sections of this configuration file:
```php
return [
'prism_server' => [
'enabled' => env('PRISM_SERVER_ENABLED', false),
],
'providers' => [
// Provider configurations here
],
];
```
## Provider Configuration
Prism uses a straightforward provider configuration system that lets you set up multiple AI providers in one place. Each provider has its own section in the configuration file where you can specify:
* API credentials
* Base URLs (useful for self-hosted instances or custom endpoints)
* Other Provider-specific settings
Here's a general template for how providers are configured:
```php
'providers' => [
'provider-name' => [
'api_key' => env('PROVIDER_API_KEY', ''),
'url' => env('PROVIDER_URL', 'https://api.provider.com'),
// Other provider-specific settings
],
],
```
## Environment Variables
Prism follows Laravel's environment configuration best practices. All sensitive or environment-specific values should be stored in your `.env` file. Here's how it works:
1. Each provider's configuration pulls values from environment variables
2. Default values are provided as fallbacks
3. Environment variables follow a predictable naming pattern:
* API keys: `PROVIDER_API_KEY`
* URLs: `PROVIDER_URL`
* Other settings: `PROVIDER_SETTING_NAME`
For example:
```shell
# Prism Server Configuration
PRISM_SERVER_ENABLED=true
# Provider Configuration
PROVIDER_API_KEY=your-api-key-here
PROVIDER_URL=https://custom-endpoint.com
```
> \[!NOTE]
> Remember to always refer to your chosen provider's documentation pages for the most up-to-date configuration options and requirements specific to that provider.
## Overriding config in your code
You can override config in your code in two ways:
```php
use Prism\Prism\Prism;
use Prism\Prism\Enums\Provider;
// Via the third parameter of `using()`
$response = Prism::text()
->using(Provider::OpenAI, 'claude-3-5-sonnet-20241022', [
'url' => 'new-base-url'
])
->withPrompt('Explain quantum computing.')
->asText();
// Or via `usingProviderConfig()` (note that this will re-resolve the provider).
$response = Prism::text()
->using(Provider::OpenAI, 'claude-3-5-sonnet-20241022')
->usingProviderConfig([
'url' => 'new-base-url'
])
->withPrompt('Explain quantum computing.')
->asText();
```
---
---
url: /advanced/custom-providers.md
---
# Custom Providers
Want to add support for a new AI provider in Prism? This guide will walk you through creating and registering your own custom provider implementation.
## Building Your Provider
All providers must extend the `Prism\Prism\Providers\Provider` abstract class. This base class provides default implementations for all required methods, throwing exceptions for unsupported actions.
When creating your provider, you'll only need to override the methods for the features you want to support:
* `text()` - For text generation
* `structured()` - For structured output generation
* `embeddings()` - For creating embeddings
* `images()` - For image generation
* `stream()` - For streaming text responses
Here's what that looks like in practice:
```php
namespace App\Prism\Providers;
use Prism\Prism\Providers\Provider;
use Prism\Prism\Text\Request as TextRequest;
use Prism\Prism\Text\Response as TextResponse;
class MyCustomProvider extends Provider
{
public function __construct(
protected string $apiKey,
) {}
public function text(TextRequest $request): TextResponse
{
// Your text generation logic here
// Make API calls, process the response, and return a TextResponse
}
// Only override the methods you need!
}
```
## Registration Process
Once you've created your provider, you'll need to register it with Prism. Let's add it to a service provider:
```php
namespace App\Providers;
use App\Prism\Providers\MyCustomProvider;
use Illuminate\Support\ServiceProvider;
class AppServiceProvider extends ServiceProvider
{
public function boot(): void
{
$this->app['prism-manager']->extend('my-custom-provider', function ($app, $config) {
return new MyCustomProvider(
apiKey: $config['api_key'] ?? '',
);
});
}
}
```
Next, add your provider configuration to `config/prism.php`:
```php
return [
'providers' => [
// ... other providers ...
'my-custom-provider' => [
'api_key' => env('MY_CUSTOM_PROVIDER_API_KEY'),
],
],
];
```
That's it! You're ready to use your custom provider:
```php
use Prism\Prism\Facades\Prism;
$response = Prism::text()
->using('my-custom-provider', 'model-name')
->withPrompt('Hello, custom AI!')
->asText();
```
## Custom Error Handling
Your provider inherits a default `handleRequestException` method that handles common HTTP status codes. You can override this method to handle provider-specific errors or add custom logic:
```php
use Illuminate\Http\Client\RequestException;
use Prism\Prism\Exceptions\PrismException;
class MyCustomProvider extends Provider
{
// ... other methods ...
public function handleRequestException(string $model, RequestException $e): never
{
// Handle provider-specific error codes
match ($e->response->getStatusCode()) {
429 => throw PrismRateLimitedException::make(
rateLimits: $this->processRateLimits($e->response),
retryAfter: $e->response->header('retry-after') === ''
? null
: (int) $e->response->header('retry-after'),
),
default => parent::handleRequestException($model, $e),
};
}
}
```
The method must throw an exception (return type `never`). If you don't handle a specific status code, make sure to call the parent method to maintain the default error handling.
## Best Practices
* **Start small**: Begin by implementing just the methods you need. You don't have to support every feature right away.
* **Handle errors gracefully**: Leverage the inherited error handling or override `handleRequestException()` for provider-specific errors (see Custom Error Handling section above).
* **Test thoroughly**: Make sure to test your provider with various inputs and edge cases.
* **Document your models**: Let users know which models your provider supports and any special parameters they can use.
> \[!TIP]
> Looking at existing provider implementations in Prism's source code can give you great insights into best practices and patterns to follow.
---
---
url: /providers/deepseek.md
---
# DeepSeek
## Configuration
```php
'deepseek' => [
'api_key' => env('DEEPSEEK_API_KEY', ''),
'url' => env('DEEPSEEK_URL', 'https://api.deepseek.com/v1')
]
```
## Provider-specific options
## Limitations
### Embeddings
Does not support embeddings.
### Tool Choice
Does not support tool choice.
### Images
Does not support images.
---
---
url: /input-modalities/documents.md
---
# Documents
Prism supports including documents in your messages with some providers.
See the [provider support table](/getting-started/introduction.html#provider-support) to check whether Prism supports your chosen provider.
Note however that provider support may differ by model. If you receive error messages with a provider that Prism indicates is supported, check the provider's documentation as to whether the model you are using supports documents.
## Supported file types
> \[!TIP]
> If provider interoperability is important to your app, we recommend converting documents to markdown.
Please check provider documentation for supported file/mime types, as support differs widely.
The most supported file types are pdf and text/plain (which may include markdown).
## Transfer mediums
> \[!TIP]
> If provider interoperability is important to your app, we recommend using rawContent or base64.
Providers are not consistent in their support of sending file raw contents, base64 and/or URLs.
Prism tries to smooth over these rough edges, but its not always possible.
### Supported conversions
* Where a provider does not support URLs: Prism will fetch the URL and use base64 or rawContent.
* Where you provide a file, base64 or rawContent: Prism will switch between base64 and rawContent depending on what the provider accepts.
### Limitations
* Where a provider only supports URLs: if you provide a file path, raw contents, base64 or chunks, for security reasons Prism does not create a URL for you and your request will fail.
* Chunks cannot be passed between providers, as they could be in different formats (however, currently only Anthropic supports them).
## Getting started
To add a document to your prompt, use the `withPrompt` method with a `Document` value object:
```php
use Prism\Prism\Enums\Provider;
use Prism\Prism\Prism;
use Prism\Prism\ValueObjects\Media\Document;
// From a local path
$response = Prism::text()
->using('my-provider', 'my-model')
->withPrompt(
'Analyze this document',
[Document::fromLocalPath(
path: 'tests/Fixtures/test-pdf.pdf',
title: 'My document title' // optional
)]
)
->asText();
// From a storage path
$response = Prism::text()
->using('my-provider', 'my-model')
->withPrompt(
'Summarize this document',
[Document::fromStoragePath(
path: 'mystoragepath/file.pdf',
diskName: 'my-disk', // optional - omit/null for default disk
title: 'My document title' // optional
)]
)
->asText();
// From base64
$response = Prism::text()
->using('my-provider', 'my-model')
->withPrompt(
'Extract key points from this document',
[Document::fromBase64(
base64: $baseFromDB,
mimeType: 'optional/mimetype', // optional
title: 'My document title' // optional
)]
)
->asText();
// From raw content
$response = Prism::text()
->using('my-provider', 'my-model')
->withPrompt(
'Review this document',
[Document::fromRawContent(
rawContent: $rawContent,
mimeType: 'optional/mimetype', // optional
title: 'My document title' // optional
)]
)
->asText();
// From a text string
$response = Prism::text()
->using('my-provider', 'my-model')
->withPrompt(
'Process this text document',
[Document::fromText(
text: 'Hello world!',
title: 'My document title' // optional
)]
)
->asText();
// From an URL
$response = Prism::text()
->using('my-provider', 'my-model')
->withPrompt(
'Analyze this document from URL',
[Document::fromUrl(
url: 'https://example.com/test-pdf.pdf',
title: 'My document title' // optional
)]
)
->asText();
// From chunks
$response = Prism::text()
->using('my-provider', 'my-model')
->withPrompt(
'Process this chunked document',
[Document::fromChunks(
chunks: [
'chunk one',
'chunk two'
],
title: 'My document title' // optional
)]
)
->asText();
// From a provider file ID
$response = Prism::text()
->using('my-provider', 'my-model')
->withPrompt(
'Analyze this document from provider file',
[Document::fromFileId(
fileId: 'my-provider-file-id'
)]
)
->asText();
```
## Alternative: Using withMessages
You can also include documents using the message-based approach:
```php
use Prism\Prism\ValueObjects\Messages\UserMessage;
use Prism\Prism\ValueObjects\Media\Document;
$message = new UserMessage(
'Analyze this document',
[Document::fromLocalPath(
path: 'tests/Fixtures/test-pdf.pdf',
title: 'My document title' // optional
)]
);
$response = Prism::text()
->using('my-provider', 'my-model')
->withMessages([$message])
->asText();
```
Or, if using a provider file\_id - use fromFileId:
```php
use Prism\Prism\Enums\Provider;
use Prism\Prism\Prism;
use Prism\Prism\ValueObjects\Media\Document;
$response = Prism::text()
->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
->withPrompt(
'Analyze this OpenAI file',
[Document::fromFileId(
fileId: 'file-lsfgSXyV2xEb8gw8fYjXU6'
)]
)
->asText();
```
---
---
url: /providers/elevenlabs.md
---
# ElevenLabs
## Configuration
```php
'elevenlabs' => [
'api_key' => env('ELEVENLABS_API_KEY', ''),
'url' => env('ELEVENLABS_URL', 'https://api.elevenlabs.io/v1/'),
]
```
## Speech-to-Text
ElevenLabs provides speech-to-text through their Scribe model with support for diarization and audio event tagging.
### Basic Usage
```php
use Prism\Prism\Prism;
use Prism\Prism\ValueObjects\Media\Audio;
$audioFile = Audio::fromPath('/path/to/recording.mp3');
$response = Prism::audio()
->using('elevenlabs', 'scribe_v1')
->withInput($audioFile)
->asText();
```
## Provider-specific Options
### Language Detection
```php
$response = Prism::audio()
->using('elevenlabs', 'scribe_v1')
->withInput($audioFile)
->withProviderOptions([
'language_code' => 'en',
])
->asText();
```
### Speaker Diarization
```php
$response = Prism::audio()
->using('elevenlabs', 'scribe_v1')
->withInput($audioFile)
->withProviderOptions([
'diarize' => true,
'num_speakers' => 2,
])
->asText();
```
### Audio Event Tagging
```php
$response = Prism::audio()
->using('elevenlabs', 'scribe_v1')
->withInput($audioFile)
->withProviderOptions([
'tag_audio_events' => true,
])
->asText();
```
## Limitations
* Text-to-speech is not yet implemented
---
---
url: /core-concepts/embeddings.md
---
# Embeddings
Transform your text into powerful vector representations! Embeddings let you add semantic search, recommendation systems, and other advanced natural language features to your applications.
## Quick Start
Here's how to generate an embedding with just a few lines of code:
```php
use Prism\Prism\Prism;
use Prism\Prism\Enums\Provider;
$response = Prism::embeddings()
->using(Provider::OpenAI, 'text-embedding-3-large')
->fromInput('Your text goes here')
->asEmbeddings();
// Get your embeddings vector
$embeddings = $response->embeddings[0]->embedding;
// Check token usage
echo $response->usage->tokens;
```
## Generating multiple embeddings
You can generate multiple embeddings at once with all providers that support embeddings, other than Gemini:
```php
use Prism\Prism\Prism;
use Prism\Prism\Enums\Provider;
$response = Prism::embeddings()
->using(Provider::OpenAI, 'text-embedding-3-large')
// First embedding
->fromInput('Your text goes here')
// Second embedding
->fromInput('Your second text goes here')
// Third and fourth embeddings
->fromArray([
'Third',
'Fourth'
])
->asEmbeddings();
/** @var Embedding $embedding */
foreach ($embeddings as $embedding) {
// Do something with your embeddings
$embedding->embedding;
}
// Check token usage
echo $response->usage->tokens;
```
## Input Methods
You've got two convenient ways to feed text into the embeddings generator:
### Direct Text Input
```php
use Prism\Prism\Prism;
use Prism\Prism\Enums\Provider;
$response = Prism::embeddings()
->using(Provider::OpenAI, 'text-embedding-3-large')
->fromInput('Analyze this text')
->asEmbeddings();
```
### From File
Need to analyze a larger document? No problem:
```php
use Prism\Prism\Prism;
use Prism\Prism\Enums\Provider;
$response = Prism::embeddings()
->using(Provider::OpenAI, 'text-embedding-3-large')
->fromFile('/path/to/your/document.txt')
->asEmbeddings();
```
> \[!NOTE]
> Make sure your file exists and is readable. The generator will throw a helpful `PrismException` if there's any issue accessing the file.
## Common Settings
Just like with text generation, you can fine-tune your embeddings requests:
```php
use Prism\Prism\Prism;
use Prism\Prism\Enums\Provider;
$response = Prism::embeddings()
->using(Provider::OpenAI, 'text-embedding-3-large')
->fromInput('Your text here')
->withClientOptions(['timeout' => 30]) // Adjust request timeout
->withClientRetry(3, 100) // Add automatic retries
->asEmbeddings();
```
## Response Handling
The embeddings response gives you everything you need:
```php
namespace Prism\Prism\ValueObjects\Embedding;
// Get an array of Embedding value objects
$embeddings = $response->embeddings;
// Just get first embedding
$firstVectorSet = $embeddings[0]->embedding;
// Loop over all embeddings
/** @var Embedding $embedding */
foreach ($embeddings as $embedding) {
$vectorSet = $embedding->embedding;
}
// Check token usage
$tokenCount = $response->usage->tokens;
```
## Error Handling
Always handle potential errors gracefully:
```php
use Prism\Prism\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\Exceptions\PrismException;
try {
$response = Prism::embeddings()
->using(Provider::OpenAI, 'text-embedding-3-large')
->fromInput('Your text here')
->asEmbeddings();
} catch (PrismException $e) {
Log::error('Embeddings generation failed:', [
'error' => $e->getMessage()
]);
}
```
## Pro Tips 🌟
**Vector Storage**: Consider using a vector database like Milvus, Qdrant, or pgvector to store and query your embeddings efficiently.
**Text Preprocessing**: For best results, clean and normalize your text before generating embeddings. This might include:
* Removing unnecessary whitespace
* Converting to lowercase
* Removing special characters
* Handling Unicode normalization
> \[!IMPORTANT]
> Different providers and models produce vectors of different dimensions. Always check your provider's documentation for specific details about the embedding model you're using.
---
---
url: /advanced/error-handling.md
---
# Error handling
By default, Prism throws a `PrismException` for Prism errors, or a `PrismServerException` for Prism Server errors.
For production use cases, you may find yourself needing to catch Exceptions more granularly, for instance to provide more useful error messages to users or to implement failover or retry logic.
Prism has begun rolling out more specific exceptions as use cases arise.
## Provider agnostic exceptions
* `PrismStructuredDecodingException` where a provider has returned invalid JSON for a structured request.
## Exceptions based on provider feedback
Prism currently supports three exceptions based on provider feedback:
* `PrismRateLimitedException` where you have hit a rate limit or quota (see [Handling rate limits](/advanced/rate-limits.html) for more info).
* `PrismProviderOverloadedException` where the provider is unable to fulfil your request due to capacity issues.
* `PrismRequestTooLargeException` where your request is too large.
However, as providers all handle errors differently, support is being rolled out incrementally. If you'd like to make your first contribution, adding one or more of these exceptions for a provider would make a great first contribution. If you'd like to discuss, start an issue on Github, or just jump straight into a pull request.
---
---
url: /providers/gemini.md
---
# Gemini
## Configuration
```php
'gemini' => [
'api_key' => env('GEMINI_API_KEY', ''),
'url' => env('GEMINI_URL', 'https://generativelanguage.googleapis.com/v1beta/models'),
],
```
## Search grounding
Google Gemini offers built-in search grounding capabilities that allow your AI to search the web for real-time information. This is a provider tool that uses Google's search infrastructure. For more information about the difference between custom tools and provider tools, see [Tools & Function Calling](/core-concepts/tools-function-calling#provider-tools).
You may enable Google search grounding on text requests using withProviderTools:
```php
use Prism\Prism\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\ValueObjects\ProviderTool;
$response = Prism::text()
->using(Provider::Gemini, 'gemini-2.0-flash')
->withPrompt('What is the stock price of Google right now?')
// Enable search grounding
->withProviderTools([
new ProviderTool('google_search')
])
->asText();
```
If you use search groundings, Google require you meet certain [display requirements](https://ai.google.dev/gemini-api/docs/grounding/search-suggestions).
The data you need to meet these display requirements, and to build e.g. footnote functionality will be saved to the response's `additionalContent` property.
```php
// The Google supplied and styled widget to click through to results.
$response->additionalContent['searchEntryPoint'];
// The search queries made by the model
$response->additionalContent['searchQueries'];
// The detail needed to build your citations
$response->additionalContent['groundingSupports'];
```
`groundingSupports` is an array of `MessagePartWithSearchGroundings`, which you can use to build up footnotes as follows:
```php
use Prism\Prism\Providers\Gemini\ValueObjects\MessagePartWithSearchGroundings;
use Prism\Prism\Providers\Gemini\ValueObjects\SearchGrounding;
$text = '';
$footnotes = [];
$footnoteId = 1;
/** @var MessagePartWithSearchGrounding $part */
foreach ($response->additionalContent['groundingSupports'] as $part) {
$text .= $part->text;
/** @var SearchGrounding $grounding */
foreach ($part->groundings as $grounding) {
$footnotes[] = [
'id' => $footnoteId,
'firstCharacter' => $part->startIndex,
'lastCharacter' => $part->endIndex,
'title' => $grounding->title,
'uri' => $grounding->uri,
'confidence' => $grounding->confidence // Float 0-1
];
$text .= ''.$footnoteId.'';
$footnoteId++;
}
}
// Pass $text and $footnotes to your frontend.
```
## Caching
Prism supports Gemini prompt caching, though due to Gemini requiring you first upload the cached content, it works a little differently to other providers.
To store content in the cache, use the Gemini provider cache method as follows:
```php
use Prism\Prism\Enums\Provider;
use Prism\Prism\Prism;
use Prism\Prism\Providers\Gemini\Gemini;
use Prism\Prism\ValueObjects\Media\Document;
use Prism\Prism\ValueObjects\Messages\SystemMessage;
use Prism\Prism\ValueObjects\Messages\UserMessage;
/** @var Gemini */
$provider = Prism::provider(Provider::Gemini);
$object = $provider->cache(
model: 'gemini-1.5-flash-002',
messages: [
new UserMessage('', [
Document::fromLocalPath('tests/Fixtures/long-document.pdf'),
]),
],
systemPrompts: [
new SystemMessage('You are a legal analyst.'),
],
ttl: 60
);
```
Then reference that object's name in your request using withProviderOptions:
```php
$response = Prism::text()
->using(Provider::Gemini, 'gemini-1.5-flash-002')
->withProviderOptions(['cachedContentName' => $object->name])
->withPrompt('In no more than 100 words, what is the document about?')
->asText();
```
## Embeddings
You can customize your Gemini embeddings request with additional parameters using `->withProviderOptions()`.
### Title
You can add a title to your embedding request. Only applicable when TaskType is `RETRIEVAL_DOCUMENT`
```php
use Prism\Prism\Enums\Provider;
use Prism\Prism\Prism;
Prism::embeddings()
->using(Provider::Gemini, 'text-embedding-004')
->fromInput('The food was delicious and the waiter...')
->withProviderOptions(['title' => 'Restaurant Review'])
->asEmbeddings();
```
### Task Type
Gemini allows you to specify the task type for your embeddings to optimize them for specific use cases:
```php
use Prism\Prism\Enums\Provider;
use Prism\Prism\Prism;
Prism::embeddings()
->using(Provider::Gemini, 'text-embedding-004')
->fromInput('The food was delicious and the waiter...')
->withProviderOptions(['taskType' => 'RETRIEVAL_QUERY'])
->asEmbeddings();
```
[Available task types](https://ai.google.dev/api/embeddings#tasktype)
### Output Dimensionality
You can control the dimensionality of your embeddings:
```php
use Prism\Prism\Enums\Provider;
use Prism\Prism\Prism;
Prism::embeddings()
->using(Provider::Gemini, 'text-embedding-004')
->fromInput('The food was delicious and the waiter...')
->withProviderOptions(['outputDimensionality' => 768])
->asEmbeddings();
```
### Thinking Mode
Gemini 2.5 series models use an internal "thinking process" during response generation. Thinking is on by default as these models have the ability to automatically decide when and how much to think based on the prompt. If you would like to customize how many tokens the model may use for thinking, or disable thinking altogether, utilize the `withProviderOptions()` method, and pass through an array with a key value pair with `thinkingBudget` and an integer representing the budget of tokens. Set this value to `0` to disable thinking.
```php
use Prism\Prism\Prism;
use Prism\Prism\Enums\Provider;
$response = Prism::text()
->using(Provider::Gemini, 'gemini-2.5-flash-preview')
->withPrompt('Explain the concept of Occam\'s Razor and provide a simple, everyday example.')
// Set thinking budget
->withProviderOptions(['thinkingBudget' => 300])
->asText();
```
> \[!NOTE]
> Do not specify a `thinkingBudget` on 2.0 or prior series Gemini models as your request will fail.
## Media Support
Gemini has robust support for processing multimedia content:
### Video Analysis
Gemini can process and analyze video content including standard video files and YouTube videos. Prism implements this through the `Video` value object which maps to Gemini's video processing capabilities.
```php
use Prism\Prism\ValueObjects\Messages\UserMessage;
use Prism\Prism\ValueObjects\Media\Video;
use Prism\Prism\Enums\Provider;
$response = Prism::text()
->using(Provider::Gemini, 'gemini-1.5-flash')
->withMessages([
new UserMessage(
'What is happening in this video?',
additionalContent: [
Video::fromUrl('https://example.com/sample-video.mp4'),
],
),
])
->asText();
```
### YouTube Integration
Gemini has special support for YouTube videos. You can easily `analyze/summarize` YouTube content by providing the URL:
```php
use Prism\Prism\ValueObjects\Messages\UserMessage;
use Prism\Prism\ValueObjects\Media\Video;
use Prism\Prism\Enums\Provider;
$response = Prism::text()
->using(Provider::Gemini, 'gemini-1.5-flash')
->withMessages([
new UserMessage(
'Summarize this YouTube video:',
additionalContent: [
Video::fromUrl('https://www.youtube.com/watch?v=dQw4w9WgXcQ'),
],
),
])
->asText();
```
### Audio Processing
Gemini can analyze audio files for various tasks like transcription, content analysis, and audio scene understanding. The implementation in Prism uses the `Audio` value object which is specifically designed for Gemini's audio processing capabilities.
```php
use Prism\Prism\ValueObjects\Messages\UserMessage;
use Prism\Prism\ValueObjects\Media\Audio;
use Prism\Prism\Enums\Provider;
$response = Prism::text()
->using(Provider::Gemini, 'gemini-1.5-flash')
->withMessages([
new UserMessage(
'Transcribe this audio file:',
additionalContent: [
Audio::fromLocalPath('/path/to/audio.mp3'),
],
),
])
->asText();
```
---
---
url: /providers/groq.md
---
# Groq
## Configuration
```php
'groq' => [
'api_key' => env('GROQ_API_KEY', ''),
'url' => env('GROQ_URL', 'https://api.groq.com/openai/v1'),
],
```
## Audio Processing
Groq provides high-performance audio processing capabilities through their ultra-fast Language Processing Unit (LPU) architecture, enabling both text-to-speech (TTS) and speech-to-text (STT) functionality with exceptional speed and quality.
### Text-to-Speech
Groq offers PlayAI TTS models that can convert text into natural-sounding speech with support for multiple languages and voices.
#### Basic TTS Usage
```php
use Prism\Prism\Prism;
$response = Prism::audio()
->using('groq', 'playai-tts')
->withInput('Hello, welcome to our application!')
->withVoice('Fritz-PlayAI')
->asAudio();
// Save the audio file
$audioData = base64_decode($response->audio->base64);
file_put_contents('welcome.wav', $audioData);
```
#### TTS Configuration Options
Control audio format and quality:
```php
$response = Prism::audio()
->using('groq', 'playai-tts')
->withInput('Testing different audio settings.')
->withVoice('Celeste-PlayAI')
->withProviderOptions([
'response_format' => 'wav', // wav (default)
'speed' => 1.2, // Speed: 0.5 to 5.0
'sample_rate' => 48000, // Sample rate options: 8000, 16000, 22050, 24000, 32000, 44100, 48000
])
->asAudio();
echo "Audio type: " . $response->audio->getMimeType();
```
#### Arabic Text-to-Speech
```php
$response = Prism::audio()
->using('groq', 'playai-tts-arabic')
->withInput('مرحبا بكم في تطبيقنا')
->withVoice('Amira-PlayAI')
->asAudio();
file_put_contents('arabic_speech.wav', base64_decode($response->audio->base64));
```
### Speech-to-Text
Groq provides ultra-fast speech recognition using Whisper models, offering exceptional speed with real-time factors of up to 299x.
#### Basic STT Usage
```php
use Prism\Prism\ValueObjects\Media\Audio;
$audioFile = Audio::fromPath('/path/to/recording.mp3');
$response = Prism::audio()
->using('groq', 'whisper-large-v3')
->withInput($audioFile)
->asText();
echo "Transcription: " . $response->text;
```
#### Model Selection Guide
Choose the right model for your use case:
```php
$response = Prism::audio()
->using('groq', 'whisper-large-v3')
->withInput($audioFile)
->asText();
// For fastest English-only transcription
$response = Prism::audio()
->using('groq', 'distil-whisper-large-v3-en')
->withInput($audioFile)
->asText();
// For balanced speed and multilingual capability
$response = Prism::audio()
->using('groq', 'whisper-large-v3-turbo')
->withInput($audioFile)
->asText();
```
#### Language Detection and Specification
Whisper can automatically detect languages or you can specify them:
```php
$response = Prism::audio()
->using('groq', 'whisper-large-v3')
->withInput($audioFile)
->withProviderOptions([
'language' => 'es', // ISO-639-1 code (optional)
'temperature' => 0.2, // Lower for more focused results
])
->asText();
```
#### Response Formats
Get transcriptions in different formats:
```php
// Standard JSON response
$response = Prism::audio()
->using('groq', 'whisper-large-v3')
->withInput($audioFile)
->withProviderOptions([
'response_format' => 'json', // json, text, verbose_json
])
->asText();
// Verbose JSON includes timestamps and segments
$response = Prism::audio()
->using('groq', 'whisper-large-v3')
->withInput($audioFile)
->withProviderOptions([
'response_format' => 'verbose_json',
'timestamp_granularities' => ['segment'], // word, segment
])
->asText();
// Access detailed segment information
$segments = $response->additionalContent['segments'] ?? [];
foreach ($segments as $segment) {
echo "Text: " . $segment['text'] . "\n";
echo "Start: " . $segment['start'] . "s\n";
echo "End: " . $segment['end'] . "s\n";
}
```
#### Context and Prompts
Improve transcription accuracy with context:
```php
$response = Prism::audio()
->using('groq', 'whisper-large-v3')
->withInput($audioFile)
->withProviderOptions([
'prompt' => 'This is a technical discussion about machine learning and artificial intelligence.',
'language' => 'en',
'temperature' => 0.1, // Lower temperature for technical content
])
->asText();
```
#### Creating Audio Objects
```php
use Prism\Prism\ValueObjects\Media\Audio;
// From local file path
$audio = Audio::fromPath('/path/to/audio.mp3');
// From remote URL (recommended for large files)
$audio = Audio::fromUrl('https://example.com/recording.wav');
// From base64 encoded data
$audio = Audio::fromBase64($base64AudioData, 'audio/mpeg');
// From binary content
$audioContent = file_get_contents('/path/to/audio.wav');
$audio = Audio::fromContent($audioContent, 'audio/wav');
```
---
---
url: /advanced/rate-limits.md
---
# Handling Rate Limits
Hitting issues with rate limits? We've got you covered!
In this guide we will look at handling:
* situations where you actually hit a rate limit (i.e. HTTP 429); and
* dynamic rate limiting (figuring out when you can make your next request, from a successful request).
## Provider support
Prism throws a `PrismRateLimitedException` for all providers other than DeepSeek (which does not have rate limits).
Prism provides an array of `ProviderRateLimit` value objects on the exception and on meta for all providers other than OpenAI, Gemini, xAI and VoyageAI - as they do not provide the necessary headers to do so.
## The ProviderRateLimit value object
Throughout this guide, we'll talk about the `ProviderRateLimit` value object.
Each `ProviderRateLimit` has four properties:
* name - the name given to that rate limit by the provider - e.g. "input-tokens"
* limit - the current limit set on your API key by the provider - e.g. for input-tokens, perhaps 80000
* remaining - how many you have left - e.g. for input-tokens if you have used 30000 out of your 80000 limit - this will be 50000
* resetsAt - a Carbon instance with the date and time at which remaining will reset to limit
## Handling a rate limit hit
Prism throws a `PrismRateLimitedException` when you hit a rate limit.
You can catch that exception, gracefully fail and inspect the `rateLimits` property which contains an array of `ProviderRateLimit`s.
```php
use Prism\Prism\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\ValueObjects\ProviderRateLimit;
use Prism\Prism\Exceptions\PrismRateLimitedException;
try {
Prism::text()
->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
->withPrompt('Hello world!')
->asText();
}
catch (PrismRateLimitedException $e) {
/** @var ProviderRateLimit $rate_limit */
foreach ($e->rateLimits as $rate_limit) {
// Loop through rate limits...
}
// Log, fail gracefully, etc.
}
```
### Figuring out which rate limit you have hit
In a simple world, they'd only be one rate limit.
However most providers implement various rate limits (e.g. request, input tokens, output tokens, etc.) and provide you with information on all of them on all requests, regardless of which you have hit.
For simple rate limits like "requests", the `remaining` property on `ProviderRateLimit` will be 0 if you have hit it. These are easy to find:
```php
use Prism\Prism\ValueObjects\ProviderRateLimit;
use Illuminate\Support\Arr;
try {
// Your request
}
catch (PrismRateLimitedException $e) {
$hit_limit = Arr::first($e->rateLimits, fn(ProviderRateLimit $rate_limit) => $rate_limit->remaining === 0);
}
```
For less simple rate limits like input tokens, the `remaining` property may not be zero. For instance, if you have 5,000 input tokens remaining and submit a request requiring 6,000 tokens, you'll be rate limited but remaining will still show 5,000.
Here, you may need to implement some logic to approximate how many tokens your request will use before sending it, and then test against that:
```php
use Prism\Prism\ValueObjects\ProviderRateLimit;
use Illuminate\Support\Arr;
try {
// Your request
}
catch (PrismRateLimitedException $e) {
$input_token_limit = Arr::first($e->rateLimits, fn(ProviderRateLimit $rate_limit) => $rate_limit->name === 'input-tokens');
if ($input_token_limit < $your_token_estimate) {
// Handle
}
}
```
To help with approximating input token usage, we plan to implement Anthopic's token counting endpoint in a future release.
For providers that don't have a token counting endpoint, you could either roll your own token counter or use something like [tiktoken](https://github.com/openai/tiktoken) if you are comfortable calling out to Python.
Once you know which rate limit you have hit, you'll want to ensure your app does not continue making requests until after the `ProviderRateLimit` `resetsAt` property.
If you aren't sure where to start with that, check out the [What should you do with rate limit information](#what-should-you-do-with-rate-limit-information) section below.
## Dynamic rate limiting
Prism adds the same rate limit information to every successful request:
```php
use Prism\Prism\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\ValueObjects\ProviderRateLimit;
$response = Prism::text()
->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
->withPrompt('Hello world!')
->asText();
/** @var ProviderRateLimit $rate_limit */
foreach ($response->meta->rateLimits as $rate_limit) {
// Handle
}
```
Armed with that information, you'll probably want to [update your app's rate limiter(s)](#what-should-you-do-with-rate-limit-information).
## What should you do with rate limit information?
You'll likely want to implement a rate limiter within your app. Thankfully Laravel, as always, makes this very easy!
You should take a look at the [rate limiting](https://laravel.com/docs/11.x/rate-limiting) docs, and if you are firing requests from your queue, check out the [job middleware](https://laravel.com/docs/11.x/queues#job-middleware) docs.
You should implement a rate limiter / job middleware for each of the provider rate limits your application typically hits.
---
---
url: /core-concepts/image-generation.md
---
# Image Generation
Generate stunning images from text prompts using AI-powered models. Prism provides a clean, consistent API for image generation across different providers, starting with comprehensive OpenAI support.
## Getting Started
Creating images with Prism is as simple as describing what you want:
```php
use Prism\Prism\Prism;
$response = Prism::image()
->using('openai', 'dall-e-3')
->withPrompt('A cute baby sea otter floating on its back in calm blue water')
->generate();
$image = $response->firstImage();
echo $image->url; // https://oaidalleapiprodscus.blob.core.windows.net/...
```
## Provider Support
Currently, Prism supports image generation through:
* **OpenAI**: DALL-E 2, DALL-E 3, and GPT-Image-1 models
Additional providers will be added in future releases as the ecosystem evolves.
## Basic Usage
### Simple Generation
The most straightforward way to generate an image:
```php
$response = Prism::image()
->using('openai', 'dall-e-3')
->withPrompt('A serene mountain landscape at sunset')
->generate();
// Access the generated image
$image = $response->firstImage();
if ($image->hasUrl()) {
echo "Image URL: " . $image->url;
}
if ($image->hasBase64()) {
echo "Base64 Image Data: " . $image->base64;
}
```
### Working with Responses
The response object provides helpful methods for accessing generated content:
```php
$response = Prism::image()
->using('openai', 'dall-e-2')
->withPrompt('Abstract geometric patterns in vibrant colors')
->generate();
// Check if images were generated
if ($response->hasImages()) {
echo "Generated {$response->imageCount()} image(s)";
// Access all images
foreach ($response->images as $image) {
if ($image->hasUrl()) {
echo "Image: {$image->url}\n";
}
if ($image->hasBase64()) {
echo "Base64 Image: " . substr($image->base64, 0, 50) . "...\n";
}
if ($image->hasRevisedPrompt()) {
echo "Revised prompt: {$image->revisedPrompt}\n";
}
}
// Or just get the first one
$firstImage = $response->firstImage();
}
// Check usage information
echo "Prompt tokens: {$response->usage->promptTokens}";
echo "Model used: {$response->meta->model}";
```
## Provider-Specific Options
While Prism provides a consistent API, you can access provider-specific features using the `withProviderOptions()` method.
### OpenAI Options
OpenAI offers various customization options depending on the model:
#### DALL-E 3 Options
```php
$response = Prism::image()
->using('openai', 'dall-e-3')
->withPrompt('A beautiful sunset over mountains')
->withProviderOptions([
'size' => '1792x1024', // 1024x1024, 1024x1792, 1792x1024
'quality' => 'hd', // standard, hd
'style' => 'vivid', // vivid, natural
'response_format' => 'url', // url, b64_json
])
->generate();
```
#### GPT-Image-1 (Base64 Only)
The GPT-Image-1 model always returns base64-encoded images, regardless of the `response_format` setting:
```php
$response = Prism::image()
->using('openai', 'gpt-image-1')
->withPrompt('A cute baby sea otter floating on its back')
->withProviderOptions([
'size' => '1024x1024', // 1024x1024, 1536x1024, 1024x1536, auto
'quality' => 'high', // auto, high, medium, low
'background' => 'transparent', // transparent, opaque, auto
'output_format' => 'png', // png, jpeg, webp
'output_compression' => 90, // 0-100 (for jpeg/webp)
])
->generate();
$image = $response->firstImage();
if ($image->hasBase64()) {
// Save the base64 image to a file
file_put_contents('generated-image.png', base64_decode($image->base64));
echo "Base64 image saved to generated-image.png";
}
```
#### Base64 vs URL Responses
Different models return images in different formats:
* **GPT-Image-1**: Always returns base64-encoded images in the `base64` property
* **DALL-E 2 & 3**: Return URLs by default, but can return base64 when `response_format` is set to `'b64_json'`
```php
// Request base64 format from DALL-E 3
$response = Prism::image()
->using('openai', 'dall-e-3')
->withPrompt('Abstract art')
->withProviderOptions([
'response_format' => 'b64_json',
])
->generate();
$image = $response->firstImage();
if ($image->hasBase64()) {
echo "Received base64 image data";
}
```
## Testing
Prism provides convenient fakes for testing image generation:
```php
use Prism\Prism\Prism;
use Prism\Prism\Testing\PrismFake;
test('can generate images', function () {
$fake = PrismFake::create()->image();
Prism::fake($fake);
$response = Prism::image()
->using('openai', 'dall-e-3')
->withPrompt('Test image')
->generate();
expect($response->hasImages())->toBeTrue();
expect($response->firstImage()->url)->toContain('fake-image-url');
});
```
Need help with a specific provider or use case? Check the [provider documentation](/providers/openai) for detailed configuration options and examples.
---
---
url: /input-modalities/images.md
---
# Images
Prism supports including images in your messages for vision analysis for most providers.
See the [provider support table](/getting-started/introduction.html#provider-support) to check whether Prism supports your chosen provider.
Note however that provider support may differ by model. If you receive error messages with a provider that Prism indicates is supported, check the provider's documentation as to whether the model you are using supports images.
## Getting started
To add an image to your prompt, use the `withPrompt` method with an `Image` value object:
```php
use Prism\Prism\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\ValueObjects\Media\Image;
// From a local path
$response = Prism::text()
->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
->withPrompt(
"What's in this image?",
[Image::fromLocalPath(path: '/path/to/image.jpg')]
)
->asText();
// From a path on a storage disk
$response = Prism::text()
->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
->withPrompt(
"What's in this image?",
[Image::fromStoragePath(
path: '/path/to/image.jpg',
diskName: 'my-disk' // optional - omit/null for default disk
)]
)
->asText();
// From a URL
$response = Prism::text()
->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
->withPrompt(
'Analyze this diagram:',
[Image::fromUrl(url: 'https://example.com/diagram.png')]
)
->asText();
// From base64
$response = Prism::text()
->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
->withPrompt(
'Analyze this diagram:',
[Image::fromBase64(base64: base64_encode(file_get_contents('/path/to/image.jpg')))]
)
->asText();
// From raw content
$response = Prism::text()
->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
->withPrompt(
'Analyze this diagram:',
[Image::fromRawContent(rawContent: file_get_contents('/path/to/image.jpg'))]
)
->asText();
```
## Alternative: Using withMessages
You can also include images using the message-based approach:
```php
use Prism\Prism\ValueObjects\Messages\UserMessage;
use Prism\Prism\ValueObjects\Media\Image;
$message = new UserMessage(
"What's in this image?",
[Image::fromLocalPath(path: '/path/to/image.jpg')]
);
$response = Prism::text()
->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
->withMessages([$message])
->asText();
```
## Transfer mediums
Providers are not consistent in their support of sending raw contents, base64 and/or URLs (as noted above).
Prism tries to smooth over these rough edges, but its not always possible.
### Supported conversions
* Where a provider does not support URLs: Prism will fetch the URL and use base64 or rawContent.
* Where you provide a file, base64 or rawContent: Prism will switch between base64 and rawContent depending on what the provider accepts.
### Limitations
* Where a provider only supports URLs: if you provide a file path, raw contents or base64, for security reasons Prism does not create a URL for you and your request will fail.
---
---
url: /getting-started/installation.md
---
# Installation
Getting started with Prism is a breeze.
## Requirements
Before we dive in, make sure your project meets these requirements:
* PHP 8.2 or higher
* Laravel 11.0 or higher
## Step 1: Composer Installation
::: tip
Prism is actively evolving. To prevent unexpected issues from breaking changes, we strongly recommend pinning your installation to a specific version. Example: "prism-php/prism": "^0.3.0".
:::
First, let's add Prism to your project using Composer. Open your terminal, navigate to your project directory, and run:
```bash
composer require prism-php/prism
```
This command will download Prism and its dependencies into your project.
## Step 2: Publish the Configuration
Prism comes with a configuration file that you'll want to customize. Publish it to your config directory by running:
```bash
php artisan vendor:publish --tag=prism-config
```
This will create a new file at `config/prism.php`. We'll explore how to configure Prism in the next section.
---
---
url: /getting-started/introduction.md
---
# Introduction
Large Language Models (LLMs) have revolutionized how we interact with artificial intelligence, enabling applications to understand, generate, and manipulate human language with unprecedented sophistication. These powerful models open up exciting possibilities for developers, from creating chatbots and content generators to building complex AI-driven applications.
Prism **simplifies the process of integrating LLMs into your Laravel projects**, providing a unified interface to work with various AI providers. This allows you to focus on crafting innovative AI features for your users, rather than getting bogged down in the intricacies of different APIs and implementation details.
Here's a quick example of how you can generate text using Prism:
::: code-group
```php [Anthropic]
use Prism\Prism\Prism;
use Prism\Prism\Enums\Provider;
$response = Prism::text()
->using(Provider::Anthropic, 'claude-3-7-sonnet-latest')
->withSystemPrompt(view('prompts.system'))
->withPrompt('Explain quantum computing to a 5-year-old.')
->asText();
echo $response->text;
```
```php [Mistral]
use Prism\Prism\Prism;
use Prism\Prism\Enums\Provider;
$response = Prism::text()
->using(Provider::Mistral, 'mistral-medium')
->withSystemPrompt(view('prompts.system'))
->withPrompt('Explain quantum computing to a 5-year-old.')
->asText();
echo $response->text;
```
```php [Ollama]
use Prism\Prism\Prism;
use Prism\Prism\Enums\Provider;
$response = Prism::text()
->using(Provider::Ollama, 'llama2')
->withSystemPrompt(view('prompts.system'))
->withPrompt('Explain quantum computing to a 5-year-old.')
->asText();
echo $response->text;
```
```php [OpenAI]
use Prism\Prism\Prism;
use Prism\Prism\Enums\Provider;
$response = Prism::text()
->using(Provider::OpenAI, 'gpt-4')
->withSystemPrompt(view('prompts.system'))
->withPrompt('Explain quantum computing to a 5-year-old.')
->asText();
echo $response->text;
```
:::
Prism draws significant inspiration from the [Vercel AI SDK](https://sdk.vercel.ai/docs/ai-sdk-core), adapting its powerful concepts and developer-friendly approach to the Laravel ecosystem.
## Key Features
* **Unified Provider Interface**: Switch seamlessly between AI providers like OpenAI, Anthropic, and Ollama without changing your application code.
* **Tool System**: Extend AI capabilities by defining custom tools that can interact with your application's business logic.
* **Image Support**: Work with multi-modal models that can process both text and images.
Prism also provides a fluent `prism()` helper function to resolve the `Prism` instance from the application container.
```php
prism()
->text()
->using(Provider::OpenAI, 'gpt-4')
->withPrompt('Explain quantum computing to a 5-year-old.')
->asText();
```
## Providers
We currently offer first-party support for these leading AI providers:
* [Anthropic](/providers/anthropic.md)
* [DeepSeek](/providers/deepseek.md)
* [Groq](/providers/groq.md)
* [Mistral](/providers/mistral.md)
* [Ollama](/providers/ollama.md)
* [OpenAI](/providers/openai.md)
* [xAI](/providers/xai.md)
Each provider brings its own strengths to the table, and Prism makes it easy to use them all through a consistent, elegant interface.
## Provider Support
Make sure you check the dedicated provider pages for considerations, limitations, and options. Support may be model dependant, check with your provider for model specific features and support.
---
---
url: /providers/mistral.md
---
# Mistral
## Configuration
```php
'mistral' => [
'api_key' => env('MISTRAL_API_KEY', ''),
'url' => env('MISTRAL_URL', 'https://api.mistral.ai/v1'),
],
```
## Provider-specific options
## Audio Processing
Mistral provides advanced speech-to-text capabilities through their Voxtral models, offering state-of-the-art transcription accuracy with native multilingual support and audio understanding features.
### Speech-to-Text
Mistral's Voxtral models deliver exceptional speech recognition performance, outperforming industry standards like Whisper large-v3 across multiple languages and acoustic environments.
#### Basic STT Usage
```php
use Prism\Prism\ValueObjects\Media\Audio;
$audioFile = Audio::fromPath('/path/to/recording.mp3');
$response = Prism::audio()
->using('mistral', 'voxtral-mini-2507')
->withInput($audioFile)
->asText();
echo "Transcription: " . $response->text;
```
#### Model Selection Guide
Choose the right Voxtral model for your use case:
```php
// For production transcription with highest accuracy
$response = Prism::audio()
->using('mistral', 'voxtral-small-latest')
->withInput($audioFile)
->asText();
// For efficient transcription and edge deployment
$response = Prism::audio()
->using('mistral', 'voxtral-mini-latest')
->withInput($audioFile)
->asText();
// For optimized transcription-only service
$response = Prism::audio()
->using('mistral', 'voxtral-mini-2507')
->withInput($audioFile)
->asText();
```
#### Language Detection and Specification
Voxtral automatically detects languages or you can specify them for better accuracy:
```php
$response = Prism::audio()
->using('mistral', 'voxtral-mini-2507')
->withInput($audioFile)
->withProviderOptions([
'language' => 'en', // ISO-639-1 code (optional)
'temperature' => 0.0, // Lower for more deterministic results
])
->asText();
// Multilingual support - single model handles multiple languages
$response = Prism::audio()
->using('mistral', 'voxtral-mini-2507')
->withInput($multilingualAudioFile)
->withProviderOptions([
// Auto-detection works well for mixed-language content
'temperature' => 0.1,
])
->asText();
```
#### Timestamps and Segmentation
Get detailed timing information with your transcriptions:
```php
$response = Prism::audio()
->using('mistral', 'voxtral-mini-2507')
->withInput($audioFile)
->withProviderOptions([
'timestamp_granularities' => ['segment'], // Available: word, segment
'response_format' => 'json',
])
->asText();
// Access segment information with timestamps
$segments = $response->additionalContent['segments'] ?? [];
foreach ($segments as $segment) {
echo "Text: " . $segment['text'] . "\n";
echo "Start: " . $segment['start'] . "s\n";
echo "End: " . $segment['end'] . "s\n";
}
```
#### Context and Prompts
Improve transcription accuracy with contextual information:
```php
$response = Prism::audio()
->using('mistral', 'voxtral-mini-2507')
->withInput($audioFile)
->withProviderOptions([
'prompt' => 'This is a medical consultation discussing patient symptoms and treatment options.',
'language' => 'en',
'temperature' => 0.0, // Deterministic for medical content
])
->asText();
```
#### Long-form Audio Processing
Voxtral handles extended audio without chunking:
```php
// Process up to 30 minutes of audio in a single request
$longAudioFile = Audio::fromPath('/path/to/long_meeting.wav');
$response = Prism::audio()
->using('mistral', 'voxtral-small-latest')
->withInput($longAudioFile)
->withProviderOptions([
'timestamp_granularities' => ['segment'],
'language' => 'en',
])
->asText();
echo "Full transcription: " . $response->text;
// Access usage information
if ($response->usage) {
echo "Audio duration: " . $response->usage->promptTokens . " tokens\n";
echo "Total tokens: " . $response->usage->totalTokens . "\n";
}
```
#### Creating Audio Objects
```php
use Prism\Prism\ValueObjects\Media\Audio;
// From local file path
$audio = Audio::fromPath('/path/to/audio.mp3');
// From remote URL
$audio = Audio::fromUrl('https://example.com/recording.wav');
// From base64 encoded data
$audio = Audio::fromBase64($base64AudioData, 'audio/mpeg');
// From binary content
$audioContent = file_get_contents('/path/to/audio.wav');
$audio = Audio::fromContent($audioContent, 'audio/wav');
```
## Documents
The text generation part of the exposed Facade only allows documents to be passed in through via URL.
See the [documents](./../input-modalities/documents.md) on how to do that.
## OCR
Mistral provides an OCR endpoint which can be used to extract text from documents.
This OCR endpoint can be used like this:
```php
use Prism\Prism\Enums\Provider;
use Prism\Prism\Prism;
use Prism\Prism\Tool;
use Prism\Prism\ValueObjects\Messages\UserMessage;
use Prism\Prism\ValueObjects\Messages\SystemMessage;
use Prism\Prism\Providers\Mistral\Mistral;
use Prism\Prism\Providers\Mistral\ValueObjects\OCRResponse;
/** @var Mistral $provider */
$provider = Prism::provider(\Prism\Prism\Enums\Provider::Mistral);
/** @var OCRResponse $ocrResponse */
$ocrResponse = $provider->ocr(
'mistral-ocr-latest',
Document::fromUrl('https://prismphp.com/storage/prism-text-generation.pdf')
);
/**
* Just need the full text of all the pages combined? Use the toText() method.
*/
$text = $ocrResponse->toText();
```
::: tip
The OCR endpoint response time can vary depending on the size of the document. We recommend doing this in the background like a queue with a longer timeout.
:::
---
---
url: /providers/ollama.md
---
# Ollama
## Configuration
```php
'ollama' => [
'url' => env('OLLAMA_URL', 'http://localhost:11434/v1'),
],
```
## Ollama Options
Ollama allows you to customize how the model is run via [options](https://github.com/ollama/ollama/blob/main/docs/modelfile.md#parameter). These options can be passed via the `->withProviderOptions()` method.
```php
Prism::text() // [!code focus]
->using(Provider::Ollama, 'gemma3:1b')
->withPrompt('Who are you?')
->withClientOptions(['timeout' => 60])
->withProviderOptions([ // [!code focus]
'top_p' => 0.9, // [!code focus]
'num_ctx' => 4096, // [!code focus]
]) // [!code focus]
```
> \[!NOTE]
> Using `withProviderOptions` will override settings like `topP` and `temperature`
## Considerations
### Timeouts
Depending on your configuration, responses tend to time out. You may need to extend the client's timeout using `->withClientOptions(['timeout' => $seconds])`.
```php
Prism::text() // [!code focus]
->using(Provider::Ollama, 'gemma3:1b')
->withPrompt('Who are you?')
->withClientOptions(['timeout' => 60]) // [!code focus]
```
### Structured Output
Ollama doesn't have native JSON mode or structured output like some providers, Prism implements a robust workaround for structured output:
* We automatically append instructions to your prompt that guide the model to output valid JSON matching your schema
* If the response isn't valid JSON, Prism will raise a PrismException
## Limitations
### Image URL
Ollama does not support images using `Image::fromUrl()`.
### Tool Choice
Ollama does not currently support tool choice / required tools.
---
---
url: /providers/openai.md
---
# OpenAI
## Configuration
```php
'openai' => [
'url' => env('OPENAI_URL', 'https://api.openai.com/v1'),
'api_key' => env('OPENAI_API_KEY', ''),
'organization' => env('OPENAI_ORGANIZATION', null),
]
```
## Provider-specific options
### Strict Tool Schemas
Prism supports OpenAI's [function calling with Structured Outputs](https://platform.openai.com/docs/guides/function-calling#function-calling-with-structured-outputs) via provider-specific meta.
```php
Tool::as('search') // [!code focus]
->for('Searching the web')
->withStringParameter('query', 'the detailed search query')
->using(fn (): string => '[Search results]')
->withProviderOptions([ // [!code focus]
'strict' => true, // [!code focus]
]); // [!code focus]
```
### Strict Structured Output Schemas
```php
$response = Prism::structured()
->withProviderOptions([ // [!code focus]
'schema' => [ // [!code focus]
'strict' => true // [!code focus]
] // [!code focus]
]) // [!code focus]
```
### Metadata
```php
$response = Prism::structured()
->withProviderOptions([ // [!code focus]
'metadata' => [ // [!code focus]
'project_id' => 23 // [!code focus]
] // [!code focus]
]) // [!code focus]
```
### Previous Responses
Prism supports OpenAI's [conversation state](https://platform.openai.com/docs/guides/conversation-state#openai-apis-for-conversation-state) with the `previous_response_id` parameter.
```php
$response = Prism::structured()
->withProviderOptions([ // [!code focus]
'previous_response_id' => 'response_id' // [!code focus]
]) // [!code focus]
```
### Truncation
```php
$response = Prism::structured()
->withProviderOptions([ // [!code focus]
'truncation' => 'auto' // [!code focus]
]) // [!code focus]
```
### Reasoning Models
OpenAI's reasoning models like `gpt-5`, `gpt-5-mini`, and `gpt-5-nano` use advanced reasoning capabilities to think through complex problems before responding. These models excel at multi-step problem solving, coding, scientific reasoning, and complex analysis tasks.
#### Reasoning Effort
Control how much reasoning the model performs before generating a response using the `reasoning` parameter:
```php
$response = Prism::text()
->using('openai', 'gpt-5')
->withPrompt('Write a PHP function to implement a binary search algorithm with proper error handling')
->withProviderOptions([ // [!code focus]
'reasoning' => ['effort' => 'high'] // [!code focus]
]) // [!code focus]
->asText();
```
Available reasoning effort levels:
* **`low`**: Faster responses with economical token usage, suitable for simpler tasks
* **`medium`**: Balanced approach between speed and reasoning depth (default)
* **`high`**: More thorough reasoning for complex problems requiring deep analysis
> \[!NOTE]
> Reasoning models generate internal "reasoning tokens" that help them think through problems. These tokens are included in your usage costs but aren't visible in the response.
#### Reasoning Token Usage
You can track reasoning token usage through the response's usage information:
```php
$response = Prism::text()
->using('openai', 'gpt-5-mini')
->withPrompt('Refactor this PHP code to use dependency injection')
->withProviderOptions([
'reasoning' => ['effort' => 'medium']
])
->asText();
// Access reasoning token usage
$usage = $response->firstStep()->usage;
echo "Reasoning tokens: " . $usage->thoughtTokens;
echo "Total completion tokens: " . $usage->completionTokens;
```
### Caching
Automatic caching does not currently work with JsonMode. Please ensure you use StructuredMode if you wish to utilise automatic caching.
## Provider Tools
OpenAI offers built-in provider tools that can be used alongside your custom tools. These tools are executed by OpenAI's infrastructure and provide specialized capabilities. For more information about the difference between custom tools and provider tools, see [Tools & Function Calling](/core-concepts/tools-function-calling#provider-tools).
### Code Interpreter
The OpenAI code interpreter allows your AI to execute Python code in a secure, sandboxed environment. This is particularly useful for mathematical calculations, data analysis, and code execution tasks.
```php
use Prism\Prism\Prism;
use Prism\Prism\ValueObjects\ProviderTool;
Prism::text()
->using('openai', 'gpt-4.1')
->withPrompt('Solve the equation 3x + 10 = 14.')
->withProviderTools([
new ProviderTool(type: 'code_interpreter', options: ['container' => ['type' => 'auto']])
])
->asText();
```
#### Configuration Options
* **container**: Configure the execution environment
* `type`: Set to `'auto'` for automatic environment selection
## Additional Message Attributes
Adding optional parameters to a `UserMessage` like the `name` field can be done through the `additionalAttributes` parameter.
```php
Prism::text()
->using('openai', 'gpt-4.1')
->withMessages([
new UserMessage('Who are you?', additionalAttributes: ['name' => 'TJ']),
])
->asText()
```
## Image Generation
OpenAI provides powerful image generation capabilities through multiple models. Prism supports all of OpenAI's image generation models with their full feature sets.
### Supported Models
| Model | Description |
|-------|-------------|
| `dall-e-3` | Latest DALL-E model |
| `dall-e-2` | Previous generation |
| `gpt-image-1` | GPT-based image model |
### Basic Usage
```php
$response = Prism::image()
->using('openai', 'dall-e-3')
->withPrompt('A serene mountain landscape at sunset')
->generate();
$image = $response->firstImage();
echo $image->url; // Generated image URL
```
### DALL-E 3 Options
DALL-E 3 is the most advanced model with the highest quality output:
```php
$response = Prism::image()
->using('openai', 'dall-e-3')
->withPrompt('A futuristic cityscape with flying cars')
->withProviderOptions([
'size' => '1792x1024', // 1024x1024, 1024x1792, 1792x1024
'quality' => 'hd', // standard, hd
'style' => 'vivid', // vivid, natural
])
->generate();
// DALL-E 3 automatically revises prompts for better results
if ($response->firstImage()->hasRevisedPrompt()) {
echo "Revised prompt: " . $response->firstImage()->revisedPrompt;
}
```
### DALL-E 2 Options
DALL-E 2 supports generating multiple images and is more cost-effective:
```php
$response = Prism::image()
->using('openai', 'dall-e-2')
->withPrompt('Abstract geometric patterns')
->withProviderOptions([
'n' => 4, // Number of images (1-10)
'size' => '1024x1024', // 256x256, 512x512, 1024x1024
'response_format' => 'url', // url only
'user' => 'user-123', // Optional user identifier
])
->generate();
// Process multiple images
foreach ($response->images as $image) {
echo "Image: {$image->url}\n";
}
```
### GPT-Image-1 Options
GPT-Image-1 offers advanced features including image editing and format control:
```php
$response = Prism::image()
->using('openai', 'gpt-image-1')
->withPrompt('A detailed architectural rendering of a modern house')
->withProviderOptions([
'size' => '1536x1024', // Various sizes supported
'quality' => 'high', // standard, high
'output_format' => 'webp', // png, webp, jpeg
'output_compression' => 85, // Compression level (0-100)
'background' => 'transparent', // transparent, white, black
'moderation' => true, // Enable content moderation
])
->generate();
```
### Image Editing with GPT-Image-1
GPT-Image-1 supports sophisticated image editing operations:
```php
$originalImage = fopen('tests/Fixtures/diamond.png', 'r');
$mask = fopen('tests/Fixtures/diamond-mask.png', 'r');
$response = Prism::image()
->using('openai', 'gpt-image-1')
->withPrompt('Add a vaporwave sunset to the background')
->withProviderOptions([
'image' => $originalImage,
'mask' => $mask,
'size' => '1024x1024',
'output_format' => 'png',
'quality' => 'high',
])
->withClientOptions(['timeout' => 9999])
->generate();
file_put_contents('edited-image.png', base64_decode($response->firstImage()->base64));
```
### Response Format
Generated images are returned as URLs:
```php
$response = Prism::image()
->using('openai', 'dall-e-3')
->withPrompt('Digital artwork')
->generate();
$image = $response->firstImage();
if ($image->hasUrl()) {
echo "
";
}
```
## Audio Processing
OpenAI provides comprehensive audio processing capabilities through their TTS (Text-to-Speech) and Whisper (Speech-to-Text) models. Prism supports all of OpenAI's audio models with their full feature sets.
### Text-to-Speech
Convert text into natural-sounding speech with various voice options:
#### Basic TTS Usage
```php
use Prism\Prism\Prism;
$response = Prism::audio()
->using('openai', 'gpt-4o-mini-tts')
->withInput('Hello, welcome to our application!')
->withVoice('alloy')
->asAudio();
// Save the audio file
$audioData = base64_decode($response->audio->base64);
file_put_contents('welcome.mp3', $audioData);
```
\\
#### High-Definition Audio
For higher quality audio output, use the model:
```php
$response = Prism::audio()
->using('openai', 'gpt-4o-mini-tts')
->withInput('This is high-quality audio generation.')
->withProviderOptions([
'voice' => 'nova',
'response_format' => 'wav', // Higher quality format
])
->asAudio();
```
#### Audio Format Options
Control the output format and quality:
```php
$response = Prism::audio()
->using('openai', 'gpt-4o-mini-tts')
->withInput('Testing different audio formats.')
->withProviderOptions([
'voice' => 'echo',
'response_format' => 'opus', // mp3, opus, aac, flac, wav, pcm
'speed' => 1.25, // Speed: 0.25 to 4.0
])
->asAudio();
echo "Audio type: " . $response->audio->getMimeType();
```
For more information on the available options, please refer to the [OpenAI API documentation](https://platform.openai.com/docs/guides/text-to-speech).
### Speech-to-Text
Convert audio files into accurate text transcriptions using Whisper:
#### Basic STT Usage
```php
use Prism\Prism\ValueObjects\Media\Audio;
$audioFile = Audio::fromPath('/path/to/recording.mp3');
$response = Prism::audio()
->using('openai', 'whisper-1')
->withInput($audioFile)
->asText();
echo "Transcription: " . $response->text;
```
#### Language Detection
Whisper can automatically detect the language or you can specify it:
```php
$response = Prism::audio()
->using('openai', 'whisper-1')
->withInput($audioFile)
->withProviderOptions([
'language' => 'es', // ISO-639-1 code (optional)
'temperature' => 0.2, // Lower temperature for more focused results
])
->asText();
```
#### Response Formats
Get transcriptions in different formats with varying detail levels:
```php
// Standard JSON response
$response = Prism::audio()
->using('openai', 'whisper-1')
->withInput($audioFile)
->withProviderOptions([
'response_format' => 'json', // json, text, srt, verbose_json, vtt
])
->asText();
// Verbose JSON includes timestamps and confidence scores
$response = Prism::audio()
->using('openai', 'whisper-1')
->withInput($audioFile)
->withProviderOptions([
'response_format' => 'verbose_json',
])
->asText();
// Access detailed segment information
$segments = $response->additionalContent['segments'] ?? [];
foreach ($segments as $segment) {
echo "Text: " . $segment['text'] . "\n";
echo "Start: " . $segment['start'] . "s\n";
echo "End: " . $segment['end'] . "s\n";
echo "Confidence: " . ($segment['no_speech_prob'] ?? 'N/A') . "\n\n";
}
```
#### Subtitle Generation
Generate subtitle files directly:
```php
// SRT format subtitles
$response = Prism::audio()
->using('openai', 'whisper-1')
->withInput($audioFile)
->withProviderOptions([
'response_format' => 'srt',
])
->asText();
file_put_contents('subtitles.srt', $response->text);
// VTT format subtitles
$response = Prism::audio()
->using('openai', 'whisper-1')
->withInput($audioFile)
->withProviderOptions([
'response_format' => 'vtt',
])
->asText();
file_put_contents('subtitles.vtt', $response->text);
```
#### Context and Prompts
Improve transcription accuracy with context:
```php
$response = Prism::audio()
->using('openai', 'whisper-1')
->withInput($audioFile)
->withProviderOptions([
'prompt' => 'This is a technical discussion about machine learning and artificial intelligence.',
'language' => 'en',
'temperature' => 0.1, // Lower temperature for technical content
])
->asText();
```
### Audio File Handling
#### Creating Audio Objects
Load audio from various sources:
```php
use Prism\Prism\ValueObjects\Media\Audio;
// From local file path
$audio = Audio::fromPath('/path/to/audio.mp3');
// From remote URL
$audio = Audio::fromUrl('https://example.com/recording.wav');
// From base64 encoded data
$audio = Audio::fromBase64($base64AudioData, 'audio/mpeg');
// From binary content
$audioContent = file_get_contents('/path/to/audio.wav');
$audio = Audio::fromContent($audioContent, 'audio/wav');
```
#### File Size Considerations
Whisper has a file size limit of 25 MB. For larger files, consider:
```php
// Check file size before processing
$audio = Audio::fromPath('/path/to/large-audio.mp3');
if ($audio->size() > 25 * 1024 * 1024) { // 25 MB
echo "File too large for processing";
} else {
$response = Prism::audio()
->using('openai', 'whisper-1')
->withInput($audio)
->asText();
}
```
For more information on the available options, please refer to the [OpenAI API documentation](https://platform.openai.com/docs/guides/speech-to-text).
---
---
url: /providers/openrouter.md
---
# OpenRouter
OpenRouter provides access to multiple AI models through a single API. This provider allows you to use various models from different providers through OpenRouter's routing system.
## Configuration
Add your OpenRouter configuration to `config/prism.php`:
```php
'providers' => [
'openrouter' => [
'api_key' => env('OPENROUTER_API_KEY'),
'url' => env('OPENROUTER_URL', 'https://openrouter.ai/api/v1'),
],
],
```
## Environment Variables
Set your OpenRouter API key and URL in your `.env` file:
```env
OPENROUTER_API_KEY=your_api_key_here
OPENROUTER_URL=https://openrouter.ai/api/v1
```
## Usage
### Text Generation
```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
$response = Prism::text()
->using(Provider::OpenRouter, 'openai/gpt-4-turbo')
->withPrompt('Tell me a story about AI.')
->generate();
echo $response->text;
```
### Structured Output
```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\Schema\ObjectSchema;
use Prism\Prism\Schema\StringSchema;
$schema = new ObjectSchema('person', 'Person information', [
new StringSchema('name', 'The person\'s name'),
new StringSchema('occupation', 'The person\'s occupation'),
]);
$response = Prism::structured()
->using(Provider::OpenRouter, 'openai/gpt-4-turbo')
->withPrompt('Generate a person profile for John Doe.')
->withSchema($schema)
->generate();
echo $response->text;
```
### Tool Calling
```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\Tool;
$weatherTool = Tool::as('get_weather')
->for('Get the current weather for a location')
->withStringParameter('location', 'The location to get weather for')
->using(function (string $location) {
return "The weather in {$location} is sunny and 72°F";
});
$response = Prism::text()
->using(Provider::OpenRouter, 'openai/gpt-4-turbo')
->withPrompt('What is the weather like in New York?')
->withTools([$weatherTool])
->generate();
echo $response->text;
```
### Streaming
```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
$stream = Prism::text()
->using(Provider::OpenRouter, 'openai/gpt-4-turbo')
->withPrompt('Tell me a long story about AI.')
->asStream();
foreach ($stream as $chunk) {
echo $chunk->text;
}
```
### Streaming with Tools
```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\Tool;
$weatherTool = Tool::as('get_weather')
->for('Get the current weather for a location')
->withStringParameter('location', 'The location to get weather for')
->using(function (string $location) {
return "The weather in {$location} is sunny and 72°F";
});
$stream = Prism::text()
->using(Provider::OpenRouter, 'openai/gpt-4-turbo')
->withPrompt('What is the weather like in multiple cities?')
->withTools([$weatherTool])
->asStream();
foreach ($stream as $chunk) {
echo $chunk->text;
// Handle tool calls
if ($chunk->toolCalls) {
foreach ($chunk->toolCalls as $toolCall) {
echo "Tool called: {$toolCall->name}\n";
}
}
// Handle tool results
if ($chunk->toolResults) {
foreach ($chunk->toolResults as $result) {
echo "Tool result: {$result->result}\n";
}
}
}
```
### Reasoning/Thinking Tokens
Some models (like OpenAI's o1 series) support reasoning tokens that show the model's thought process:
```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\Enums\ChunkType;
$stream = Prism::text()
->using(Provider::OpenRouter, 'openai/o1-preview')
->withPrompt('Solve this complex math problem: What is the derivative of x^3 + 2x^2 - 5x + 1?')
->asStream();
foreach ($stream as $chunk) {
if ($chunk->chunkType === ChunkType::Thinking) {
// This is the model's reasoning/thinking process
echo "Thinking: " . $chunk->text . "\n";
} else {
// This is the final answer
echo $chunk->text;
}
}
```
#### Reasoning Effort
Control how much reasoning the model performs before generating a response using the `reasoning` parameter. The way this is structured depends on the underlying model you are calling:
```php
$response = Prism::text()
->using(Provider::OpenRouter, 'openai/gpt-5-mini')
->withPrompt('Write a PHP function to implement a binary search algorithm with proper error handling')
->withProviderOptions([
'reasoning' => [
'effort' => 'high', // Can be "high", "medium", or "low" (OpenAI-style)
'max_tokens' => 2000, // Specific token limit (Gemini / Anthropic-style)
// Optional: Default is false. All models support this.
'exclude': false, // Set to true to exclude reasoning tokens from response
// Or enable reasoning with the default parameters:
'enabled': true // Default: inferred from `effort` or `max_tokens`
]
])
->asText();
```
## Available Models
OpenRouter supports many models from different providers. Some popular options include:
* `openai/gpt-4-turbo`
* `openai/gpt-3.5-turbo`
* `anthropic/claude-3-5-sonnet`
* `meta-llama/llama-3.1-70b`
* `google/gemini-pro`
* `mistralai/mistral-7b-instruct`
Visit [OpenRouter's models page](https://openrouter.ai/models) for a complete list of available models.
## Features
* ✅ Text Generation
* ✅ Structured Output
* ✅ Tool Calling
* ✅ Multiple Model Support
* ✅ Provider Routing
* ✅ Streaming
* ✅ Reasoning/Thinking Tokens (for compatible models)
* ❌ Embeddings (not yet implemented)
* ❌ Image Generation (not yet implemented)
## API Reference
For detailed API documentation, visit [OpenRouter's API documentation](https://openrouter.ai/docs/api-reference/chat-completion).
## Error Handling
The OpenRouter provider includes standard error handling for common issues:
* Rate limiting
* Request too large
* Provider overload
* Invalid API key
Errors are automatically mapped to appropriate Prism exceptions for consistent error handling across all providers.
---
---
url: /core-concepts/prism-server.md
---
# Prism Server
Prism Server is a powerful feature that allows you to expose your Prism-powered AI models through a standardized API. This makes it easy to integrate your custom AI solutions into various applications, including chat interfaces and other tools that support OpenAI-compatible APIs.
## How It Works
Prism Server acts as a middleware, translating requests from OpenAI-compatible clients into Prism-specific operations. This means you can use tools like ChatGPT web UIs or any OpenAI SDK to interact with your custom Prism models.
## Setting Up Prism Server
### 1. Enable Prism Server
First, make sure Prism Server is enabled in your `config/prism.php` file:
```php
'prism_server' => [
// The middleware that will be applied to the Prism Server routes.
'middleware' => [],
'enabled' => env('PRISM_SERVER_ENABLED', false),
]
```
### 2. Register Your Prisms
To make your Prism models available through the server, you need to register them. This is typically done in a service provider, such as `AppServiceProvider`:
```php
use Prism\Prism\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\Facades\PrismServer;
use Illuminate\Support\ServiceProvider;
class AppServiceProvider extends ServiceProvider
{
public function boot(): void
{
PrismServer::register(
'my-custom-model',
fn () => Prism::text()
->using(Provider::Anthropic, 'claude-3-5-sonnet-latest')
->withSystemPrompt('You are a helpful assistant.')
);
}
}
```
In this example, we're registering a model named `my-custom-model` that uses the Anthropic Claude 3 Sonnet model with a custom system message.
## Using Prism Server
Once set up, Prism Server exposes two main endpoints:
### Chat Completions
To generate text using your registered Prism models:
```bash
curl -X POST "http://your-app.com/prism/openai/v1/chat/completions" \
-H "Content-Type: application/json" \
-d '{
"model": "my-custom-model",
"messages": [
{"role": "user", "content": "Hello, who are you?"}
]
}'
```
### List Available Models
To get a list of all registered Prism models:
```bash
curl "http://your-app.com/prism/openai/v1/models"
```
## Integration with Open WebUI
Prism Server works seamlessly with OpenAI-compatible chat interfaces like [Open WebUI](https://openwebui.com). Here's an example Docker Compose configuration:
```yaml
services:
open-webui:
image: ghcr.io/open-webui/open-webui:main
ports:
- "3000:8080"
environment:
OPENAI_API_BASE_URLS: "http://laravel:8080/prism/openai/v1"
WEBUI_SECRET_KEY: "your-secret-key"
laravel:
image: serversideup/php:8.3-fpm-nginx
volumes:
- ".:/var/www/html"
environment:
OPENAI_API_KEY: ${OPENAI_API_KEY}
ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY}
depends_on:
- open-webui
```
With this setup, you can access your Prism models through a user-friendly chat interface at `http://localhost:3000`.
By leveraging Prism Server, you can create powerful, custom AI experiences while maintaining compatibility with a wide ecosystem of tools and libraries. Whether you're building a chatbot, a content generation tool, or something entirely new, Prism Server provides the flexibility and standardization you need to succeed.
## Adding Middleware
You can add middleware to the Prism Server routes by setting the `middleware` option in your `config/prism.php` file:
```php
'prism_server' => [
'middleware' => ['api'],
],
```
---
---
url: /advanced/provider-interoperability.md
---
# Provider Interoperability
When working with Prism, you might need to customize requests based on which provider you're using. Different providers have unique capabilities, configuration options, and requirements that can affect how you structure your requests for optimal results.
## Using the `whenProvider` Method
The `whenProvider` method lets you easily customize your requests for specific providers while maintaining clean, readable code.
```php
$response = Prism::text()
->using(Provider::OpenAI, 'gpt-4o')
->withPrompt('Who are you?')
->whenProvider(
Provider::Anthropic,
fn ($request) => $request
->withProviderOptions([
'cacheType' => 'ephemeral',
])
)
->asText();
```
In this example, the `withProviderOptions` settings will only be applied when using Anthropic's provider. If you're using OpenAI (as specified in the `using` method), these customizations are simply skipped.
## Key Benefits
* **Cleaner Code**: Keep your provider-specific customizations encapsulated and only apply them when needed
* **Easy Provider Switching**: Swap between providers without rewriting your configuration code
* **Maintainable Applications**: Define provider-specific behaviors in one place
## Advanced Usage
You can chain multiple `whenProvider` calls to handle different provider scenarios:
```php
$response = Prism::text()
->using(Provider::OpenAI, 'gpt-4o')
->withPrompt('Generate a creative story about robots.')
->whenProvider(
Provider::Anthropic,
fn ($request) => $request
->withMaxTokens(4000)
->withProviderOptions(['cacheType' => 'ephemeral'])
)
->whenProvider(
Provider::OpenAI,
fn ($request) => $request
->withMaxTokens(2000)
->withProviderOptions(['response_format' => ['type' => 'text']])
)
->asText();
```
## Using Invokable Classes
For more complex provider-specific configurations, you can use invokable classes instead of closures:
```php
class AnthropicConfigurator
{
public function __invoke($request)
{
return $request
->withMaxTokens(4000)
->withProviderOptions([
'cacheType' => 'ephemeral',
'citations' => true,
]);
}
}
$response = Prism::text()
->using(Provider::Anthropic, 'claude-3-sonnet')
->withPrompt('Explain the theory of relativity.')
->whenProvider(Provider::Anthropic, new AnthropicConfigurator())
->asText();
```
This approach can be especially helpful when you have complex or reusable provider configurations.
> \[!TIP]
> The `whenProvider` method works with all request types in Prism including text, structured output, and embeddings requests.
## Best Practices
### Avoiding SystemMessages with Multiple Providers
When working with multiple providers, it's best to avoid using `SystemMessages` directly in your `withMessages` array. Instead, use the `withSystemPrompt` method as it offers better provider interoperability.
```php
// Avoid this when switching between providers
$response = Prism::text()
->using(Provider::OpenAI, 'gpt-4o')
->withMessages([
new SystemMessage('You are a helpful assistant.'),
new UserMessage('Tell me about AI'),
])
->asText();
// Prefer this instead
$response = Prism::text()
->using(Provider::OpenAI, 'gpt-4o')
->withSystemPrompt('You are a helpful assistant.')
->withPrompt('Tell me about AI')
->asText();
```
This approach allows Prism to handle the provider-specific formatting of system messages, making your code more portable across different LLM providers.
---
---
url: /core-concepts/schemas.md
---
# Schemas
Schemas are the blueprints that help you define the shape of your data in Prism. Whether you're building tool parameters or crafting structured outputs, schemas help you clearly communicate what your data should look like.
## Quick Start
Let's dive right in with a practical example:
```php
use Prism\Prism\Schema\ArraySchema;
use Prism\Prism\Schema\ObjectSchema;
use Prism\Prism\Schema\StringSchema;
$userSchema = new ObjectSchema(
name: 'user',
description: 'A user profile with their hobbies',
properties: [
new StringSchema('name', 'The user\'s full name'),
new ArraySchema(
name: 'hobbies',
description: 'The user\'s list of hobbies',
items: new ObjectSchema(
name: 'hobby',
description: 'A detailed hobby entry',
properties: [
new StringSchema('name', 'The name of the hobby'),
new StringSchema('description', 'A brief description of the hobby'),
],
requiredFields: ['name', 'description']
)
),
],
requiredFields: ['name', 'hobbies']
);
```
## Available Schema Types
### StringSchema
For text values of any length. Perfect for names, descriptions, or any textual data.
```php
use Prism\Prism\Schema\StringSchema;
$nameSchema = new StringSchema(
name: 'full_name',
description: 'The user\'s full name including first and last name'
);
```
### NumberSchema
Handles both integers and floating-point numbers. Great for ages, quantities, or measurements.
```php
use Prism\Prism\Schema\NumberSchema;
$ageSchema = new NumberSchema(
name: 'age',
description: 'The user\'s age in years'
);
```
### BooleanSchema
For simple true/false values. Perfect for flags and toggles.
```php
use Prism\Prism\Schema\BooleanSchema;
$activeSchema = new BooleanSchema(
name: 'is_active',
description: 'Whether the user account is active'
);
```
### ArraySchema
For lists of items, where each item follows a specific schema.
```php
use Prism\Prism\Schema\ArraySchema;
use Prism\Prism\Schema\StringSchema;
$tagsSchema = new ArraySchema(
name: 'tags',
description: 'List of tags associated with the post',
items: new StringSchema('tag', 'A single tag')
);
```
### EnumSchema
When you need to restrict values to a specific set of options.
```php
use Prism\Prism\Schema\EnumSchema;
$statusSchema = new EnumSchema(
name: 'status',
description: 'The current status of the post',
options: ['draft', 'published', 'archived']
);
```
### ObjectSchema
For complex, nested data structures. The Swiss Army knife of schemas!
```php
use Prism\Prism\Schema\ObjectSchema;
use Prism\Prism\Schema\StringSchema;
use Prism\Prism\Schema\NumberSchema;
$profileSchema = new ObjectSchema(
name: 'profile',
description: 'A user\'s public profile information',
properties: [
new StringSchema('username', 'The unique username'),
new StringSchema('bio', 'A short biography'),
new NumberSchema('joined_year', 'Year the user joined'),
],
requiredFields: ['username']
);
```
## Nullable Fields
Sometimes, not every field is required. You can make any schema nullable by setting the `nullable` parameter to `true`:
```php
use Prism\Prism\Schema\StringSchema;
$bioSchema = new StringSchema(
name: 'bio',
description: 'Optional user biography',
nullable: true
);
```
> \[!NOTE]
> When using OpenAI in strict mode, all fields must be marked as required, so optional fields must be marked as nullable.
## Required vs Nullable Fields
Understanding the difference between required fields and nullable fields is crucial when working with schemas in Prism:
### Required Fields
Required fields are specified at the object level using the `requiredFields` parameter. They indicate which properties must be present in the data structure:
```php
$userSchema = new ObjectSchema(
name: 'user',
description: 'User profile',
properties: [
new StringSchema('email', 'Primary email address'),
new StringSchema('name', 'User\'s full name'),
new StringSchema('bio', 'User biography'),
],
requiredFields: ['email', 'name'] // email and name must be present
);
```
### Nullable Fields
Nullable fields, on the other hand, are specified at the individual field level using the `nullable` parameter. They indicate that a field can contain a `null` value:
```php
$userSchema = new ObjectSchema(
name: 'user',
description: 'User profile',
properties: [
new StringSchema('email', 'Primary email address'),
new StringSchema('name', 'User\'s full name'),
new StringSchema('bio', 'User biography', nullable: true), // bio can be null
],
requiredFields: ['email', 'name', 'bio'] // bio must be present, but can be null
);
```
### Key Differences
1. **Required vs Present**:
* A required field must be present in the data structure
* A non-nullable field must contain a non-null value when present
* A field can be required but nullable (must be present, can be null)
* A field can be non-required and non-nullable (when present, cannot be null)
2. **Common Patterns**:
```php
// Required and Non-nullable (most strict)
new StringSchema('email', 'Primary email', nullable: false);
// requireFields: ['email']
// Required but Nullable (must be present, can be null)
new StringSchema('bio', 'User bio', nullable: true);
// requireFields: ['bio']
// Optional and Non-nullable (can be omitted, but if present cannot be null)
new StringSchema('phone', 'Phone number', nullable: false);
// requireFields: []
// Optional and Nullable (most permissive)
new StringSchema('website', 'Personal website', nullable: true);
// requireFields: []
```
### Provider Considerations
When working with providers that support strict mode (like OpenAI), you'll want to be especially careful with these settings:
```php
// For OpenAI strict mode:
// - All fields should be required
// - Use nullable: true for optional fields
$userSchema = new ObjectSchema(
name: 'user',
description: 'User profile',
properties: [
new StringSchema('email', 'Required email address'),
new StringSchema('bio', 'Optional biography', nullable: true),
],
requiredFields: ['email', 'bio'] // Note: bio is required but nullable
);
```
> \[!TIP]
> When in doubt, be explicit about both requirements. Specify both the `nullable` status of each field AND which fields are required in your object schemas. This makes your intentions clear to both other developers and AI providers.
## Best Practices
1. **Clear Descriptions**: Write clear, concise descriptions for each field. Future you (and other developers) will thank you!
```php
// ❌ Not helpful
new StringSchema('name', 'the name');
// ✅ Much better
new StringSchema('name', 'The user\'s display name (2-50 characters)');
```
2. **Thoughtful Required Fields**: Only mark fields as required if they're truly necessary:
```php
new ObjectSchema(
name: 'user',
description: 'User profile',
properties: [
new StringSchema('email', 'Primary email address'),
new StringSchema('phone', 'Optional phone number', nullable: true),
],
requiredFields: ['email']
);
```
3. **Nested Organization**: Keep your schemas organized when dealing with complex structures:
```php
// Define child schemas first
$addressSchema = new ObjectSchema(/*...*/);
$contactSchema = new ObjectSchema(/*...*/);
// Then use them in your parent schema
$userSchema = new ObjectSchema(
name: 'user',
description: 'Complete user profile',
properties: [$addressSchema, $contactSchema]
);
```
> \[!NOTE]
> Remember that while schemas help define the structure of your data, Prism doesn't currently validate the data against these schemas. Schema validation is planned for a future release!
---
---
url: /core-concepts/streaming-output.md
---
# Streaming Output
Want to show AI responses to your users in real-time? Streaming lets you display text as it's generated, creating a more responsive and engaging user experience.
> \[!WARNING]
> When using Laravel Telescope or other packages that intercept Laravel's HTTP client events, they may consume the stream before Prism can emit the stream chunks. This can cause streaming to appear broken or incomplete. Consider disabling such interceptors when using streaming functionality, or configure them to ignore Prism's HTTP requests.
## Basic Streaming
At its simplest, streaming works like this:
```php
use Prism\Prism\Prism;
$response = Prism::text()
->using('openai', 'gpt-4')
->withPrompt('Tell me a story about a brave knight.')
->asStream();
// Process each chunk as it arrives
foreach ($response as $chunk) {
echo $chunk->text;
// Flush the output buffer to send text to the browser immediately
ob_flush();
flush();
}
```
## Understanding Chunks
Each chunk from the stream contains a piece of the generated content:
```php
foreach ($response as $chunk) {
// The text fragment in this chunk
echo $chunk->text;
if ($chunk->usage) {
echo "Prompt tokens: " . $chunk->usage->promptTokens;
echo "Completion tokens: " . $chunk->usage->completionTokens;
}
// Check if this is the final chunk
if ($chunk->finishReason === FinishReason::Stop) {
echo "Generation complete: " . $chunk->finishReason->name;
}
}
```
## Streaming with Tools
Streaming works seamlessly with tools, allowing real-time interaction:
```php
use Prism\Prism\Facades\Tool;
use Prism\Prism\Prism;
$weatherTool = Tool::as('weather')
->for('Get current weather information')
->withStringParameter('city', 'City name')
->using(function (string $city) {
return "The weather in {$city} is sunny and 72°F.";
});
$response = Prism::text()
->using('openai', 'gpt-4o')
->withTools([$weatherTool])
->withMaxSteps(3) // Control maximum number of back-and-forth steps
->withPrompt('What\'s the weather like in San Francisco today?')
->asStream();
$fullResponse = '';
foreach ($response as $chunk) {
// Append each chunk to build the complete response
$fullResponse .= $chunk->text;
// Check for tool calls
if ($chunk->chunkType === ChunkType::ToolCall) {
foreach ($chunk->toolCalls as $call) {
echo "Tool called: " . $call->name;
}
}
// Check for tool results
if ($chunk->chunkType === ChunkType::ToolResult) {
foreach ($chunk->toolResults as $result) {
echo "Tool result: " . $result->result;
}
}
}
echo "Final response: " . $fullResponse;
```
## Configuration Options
Streaming supports the same configuration options as regular [text generation](/core-concepts/text-generation#generation-parameters).
## Handling Streaming in Web Applications
Here's how to integrate streaming in a Laravel controller:
Alternatively, you might consider using Laravel's [Broadcasting feature](https://laravel.com/docs/12.x/broadcasting) to send the chunks to your frontend.
```php
use Prism\Prism\Prism;
use Illuminate\Http\Response;
public function streamResponse()
{
return response()->stream(function () {
$stream = Prism::text()
->using('openai', 'gpt-4')
->withPrompt('Explain quantum computing step by step.')
->asStream();
foreach ($stream as $chunk) {
echo $chunk->text;
ob_flush();
flush();
}
}, 200, [
'Cache-Control' => 'no-cache',
'Content-Type' => 'text/event-stream',
'X-Accel-Buffering' => 'no', // Prevents Nginx from buffering
]);
}
```
### Laravel 12 Event Streams
Stream the output via Laravel event streams ([docs](https://laravel.com/docs/12.x/responses#event-streams)).
```php
Route::get('/chat', function () {
return response()->eventStream(function () {
$stream = Prism::text()
->using('openai', 'gpt-4')
->withPrompt('Explain quantum computing step by step.')
->asStream();
foreach ($stream as $response) {
yield $response->text;
}
});
});
```
Streaming gives your users a more responsive experience by showing AI-generated content as it's created, rather than making them wait for the complete response. This approach feels more natural and keeps users engaged, especially for longer responses or complex interactions with tools.
---
---
url: /core-concepts/structured-output.md
---
# Structured Output
Want your AI responses as neat and tidy as a Marie Kondo-approved closet? Structured output lets you define exactly how you want your data formatted, making it perfect for building APIs, processing forms, or any time you need data in a specific shape.
## Quick Start
Here's how to get structured data from your AI:
```php
use Prism\Prism\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\Schema\ObjectSchema;
use Prism\Prism\Schema\StringSchema;
$schema = new ObjectSchema(
name: 'movie_review',
description: 'A structured movie review',
properties: [
new StringSchema('title', 'The movie title'),
new StringSchema('rating', 'Rating out of 5 stars'),
new StringSchema('summary', 'Brief review summary')
],
requiredFields: ['title', 'rating', 'summary']
);
$response = Prism::structured()
->using(Provider::OpenAI, 'gpt-4o')
->withSchema($schema)
->withPrompt('Review the movie Inception')
->asStructured();
// Access your structured data
$review = $response->structured;
echo $review['title']; // "Inception"
echo $review['rating']; // "5 stars"
echo $review['summary']; // "A mind-bending..."
```
> \[!TIP]
> This is just a basic example of schema usage. Check out our [dedicated schemas guide](/core-concepts/schemas) to learn about all available schema types, nullable fields, and best practices for structuring your data.
## Understanding Output Modes
Different AI providers handle structured output in two main ways:
1. **Structured Mode**: Some providers support strict schema validation, ensuring responses perfectly match your defined structure.
2. **JSON Mode**: Other providers simply guarantee valid JSON output that approximately matches your schema.
> \[!NOTE]
> Check your provider's documentation to understand which mode they support. Provider support can vary by model, so always verify capabilities for your specific use case.
## Provider-Specific Options
Providers may offer additional options for structured output:
### OpenAI: Strict Mode
OpenAI supports a "strict mode" for even tighter schema validation:
```php
use Prism\Prism\Prism;
use Prism\Prism\Enums\Provider;
$response = Prism::structured()
->withProviderOptions([
'schema' => [
'strict' => true
]
])
// ... rest of your configuration
```
### Anthropic: Tool Calling Mode
Anthropic doesn't have native structured output, but Prism provides two approaches. For more reliable JSON parsing, especially with complex content or non-English text, use tool calling mode:
```php
use Prism\Prism\Prism;
use Prism\Prism\Enums\Provider;
$response = Prism::structured()
->using(Provider::Anthropic, 'claude-3-5-sonnet-latest')
->withSchema($schema)
->withPrompt('天氣怎麼樣？應該穿什麼？') // Chinese text with potential quotes
->withProviderOptions(['use_tool_calling' => true])
->asStructured();
```
**When to use tool calling mode with Anthropic:**
* Working with non-English content that may contain quotes
* Complex JSON structures that might confuse prompt-based parsing
* When you need the most reliable structured output possible
> \[!NOTE]
> Tool calling mode cannot be used with Anthropic's citations feature.
> \[!TIP]
> Check the provider-specific documentation pages for additional options and features that might be available for structured output.
## Response Handling
When working with structured responses, you have access to both the structured data and metadata about the generation:
```php
use Prism\Prism\Prism;
$response = Prism::structured()
->withSchema($schema)
->asStructured();
// Access the structured data as a PHP array
$data = $response->structured;
// Get the raw response text if needed
echo $response->text;
// Check why the generation stopped
echo $response->finishReason->name;
// Get token usage statistics
echo "Prompt tokens: {$response->usage->promptTokens}";
echo "Completion tokens: {$response->usage->completionTokens}";
// Access provider-specific response data
$rawResponse = $response->response;
```
> \[!TIP]
> Always validate the structured data before using it in your application:
```php
if ($response->structured === null) {
// Handle parsing failure
}
if (!isset($response->structured['required_field'])) {
// Handle missing required data
}
```
## Common Settings
Structured output supports several configuration options to fine-tune your generations:
### Model Configuration
* `maxTokens` - Set the maximum number of tokens to generate
* `temperature` - Control output randomness (provider-dependent)
* `topP` - Alternative to temperature for controlling randomness (provider-dependent)
### Input Methods
* `withPrompt` - Single prompt for generation
* `withMessages` - Message history for more context
* `withSystemPrompt` - System-level instructions
### Request Configuration
* `withClientOptions` - Set HTTP client options (e.g., timeouts)
* `withClientRetry` - Configure automatic retries on failures
* `usingProviderConfig` - Override provider configuration
* `withProviderOptions` - Set provider-specific options
> \[!NOTE]
> Unlike text generation, structured output does not support tools/function calling. For those features, use the text generation API instead.
See the [Text Generation](./text-generation.md) documentation for comparison with standard text generation capabilities.
## Error Handling
When working with structured output, it's especially important to handle potential errors:
```php
use Prism\Prism\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\Exceptions\PrismException;
try {
$response = Prism::structured()
->using('anthropic', 'claude-3-sonnet')
->withSchema($schema)
->withPrompt('Generate product data')
->asStructured();
} catch (PrismException $e) {
// Handle validation or generation errors
Log::error('Structured generation failed:', [
'error' => $e->getMessage()
]);
}
```
> \[!IMPORTANT]
> Always validate the structured response before using it in your application, as different providers may have varying levels of schema adherence.
---
---
url: /core-concepts/testing.md
---
# Testing
Want to make sure your Prism integrations work flawlessly? Let's dive into testing! Prism provides a powerful fake implementation that makes it a breeze to test your AI‑powered features.
## Basic Test Setup
First, let's look at how to set up basic response faking:
```php
use Prism\Prism\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\ValueObjects\Usage;
use Prism\Prism\Testing\TextResponseFake;
it('can generate text', function () {
$fakeResponse = TextResponseFake::make()
->withText('Hello, I am Claude!')
->withUsage(new Usage(10, 20));
// Set up the fake
$fake = Prism::fake([$fakeResponse]);
// Run your code
$response = Prism::text()
->using(Provider::Anthropic, 'claude-3-5-sonnet-latest')
->withPrompt('Who are you?')
->asText();
// Make assertions
expect($response->text)->toBe('Hello, I am Claude!');
});
```
The response fakes create a new response with default values and let you fluently set the values you need for your test.
## Testing Multiple Responses
When testing conversations or tool usage, you might need to simulate multiple responses:
```php
use Prism\Prism\ValueObjects\Usage;
use Prism\Prism\ValueObjects\ToolCall;
use Prism\Prism\Testing\TextResponseFake;
use Prism\Prism\ValueObjects\Meta;
it('can handle tool calls', function () {
$responses = [
TextResponseFake::make()
->withToolCalls([
new ToolCall(
id: 'call_1',
name: 'search',
arguments: ['query' => 'Latest news']
)
])
->withUsage(new Usage(15, 25))
->withMeta(new Meta('fake-1', 'fake-model')),
TextResponseFake::make()
->withText('Here are the latest news...')
->withUsage(new Usage(20, 30))
->withMeta(new Meta('fake-2', 'fake-model')),
];
$fake = Prism::fake($responses);
});
```
## Using the ResponseBuilder
If you need to test a richer response object, e.g. with Steps, you may find it easier to use the `ResponseBuilder` together with the fake Step helpers.
This is especially useful when you want to test complex streamed responses.
```php
use Prism\Prism\Text\ResponseBuilder;
use Prism\Prism\Testing\TextStepFake;
use Prism\Prism\ValueObjects\Usage;
use Prism\Prism\ValueObjects\Meta;
use Prism\Prism\Enums\FinishReason;
use Prism\Prism\ValueObjects\ToolCall;
use Prism\Prism\ValueObjects\ToolResult;
use Prism\Prism\ValueObjects\Messages\{UserMessage,AssistantMessage,SystemMessage};
use Prism\Prism\ValueObjects\Media\Document;
Prism::fake([
(new ResponseBuilder)
->addStep(
TextStepFake::make()
->withText('Step 1 response text')
->withFinishReason(FinishReason::Stop)
->withToolCalls([/* tool calls */])
->withToolResults([/* tool results */])
->withUsage(new Usage(1000, 750))
->withMeta(new Meta('step1', 'test-model'))
->withMessages([
new UserMessage('Test message 1', [
new Document(
document: '',
mimeType: 'text/plain',
dataFormat: 'text',
documentTitle: 'Test document',
documentContext: 'Test context'
),
]),
new AssistantMessage('Test message 2')
])
->withSystemPrompts([
new SystemMessage('Test system')
])
->withAdditionalContent(['test' => 'additional'])
)
->addStep(
TextStepFake::make()
->withText('Step 2 response text')
->withFinishReason(FinishReason::Stop)
->withToolCalls([/* tool calls */])
->withToolResults([/* tool results */])
->withUsage(new Usage(1000, 750))
->withMeta(new Meta(id: 123, model: 'test-model'))
->withMessages([/* Second step messages */])
->withSystemPrompts([/* Second step system prompts */])
->withAdditionalContent([/* Second step additional data */])
)
->toResponse()
]);
```
## Testing Tools
```php
use Prism\Prism\Enums\FinishReason;
use Prism\Prism\Enums\Provider;
use Prism\Prism\Facades\Tool;
use Prism\Prism\Prism;
use Prism\Prism\Testing\TextStepFake;
use Prism\Prism\Text\ResponseBuilder;
use Prism\Prism\ValueObjects\Meta;
use Prism\Prism\ValueObjects\ToolCall;
use Prism\Prism\ValueObjects\ToolResult;
use Prism\Prism\ValueObjects\Usage;
it('can use weather tool', function () {
// Define the expected tool call and response sequence
$responses = [
(new ResponseBuilder)
->addStep(
// First response: AI decides to use the weather tool
TextStepFake::make()
->withToolCalls([
new ToolCall(
id: 'call_123',
name: 'weather',
arguments: ['city' => 'Paris']
),
])
->withFinishReason(FinishReason::ToolCalls)
->withUsage(new Usage(15, 25))
->withMeta(new Meta('fake-1', 'fake-model'))
)
->addStep(
// Second response: AI uses the tool result to form a response
TextStepFake::make()
->withText('Based on current conditions, the weather in Paris is sunny with a temperature of 72°F.')
->withToolResults([
new ToolResult(
toolCallId: 'call_123',
toolName: 'weather',
args: ['city' => 'Paris'],
result: 'Sunny, 72°F'
),
])
->withFinishReason(FinishReason::Stop)
->withUsage(new Usage(20, 30))
->withMeta(new Meta('fake-2', 'fake-model')),
)
->toResponse(),
];
// Set up the fake
Prism::fake($responses);
// Create the weather tool
$weatherTool = Tool::as('weather')
->for('Get weather information')
->withStringParameter('city', 'City name')
->using(fn (string $city) => "The weather in {$city} is sunny with a temperature of 72°F");
// Run the actual test
$response = Prism::text()
->using(Provider::Anthropic, 'claude-3-5-sonnet-latest')
->withPrompt('What\'s the weather in Paris?')
->withTools([$weatherTool])
->withMaxSteps(2)
->asText();
// Assert the response has the correct number of steps
expect($response->steps)->toHaveCount(2);
// Assert tool calls were made correctly
expect($response->steps[0]->toolCalls)->toHaveCount(1);
expect($response->steps[0]->toolCalls[0]->name)->toBe('weather');
expect($response->steps[0]->toolCalls[0]->arguments())->toBe(['city' => 'Paris']);
// Assert tool results were processed
expect($response->toolResults)->toHaveCount(1);
expect($response->toolResults[0]->result)
->toBe('Sunny, 72°F');
// Assert final response
expect($response->text)
->toBe('Based on current conditions, the weather in Paris is sunny with a temperature of 72°F.');
});
```
## Testing Streamed Responses
To test streamed responses, you can use any text response for a fake. The fake Provider will turn the text response into a fake stream of text chunks.
It will always finish with an empty chunk including your given finish reason.
```php
Prism::fake([
TextResponseFake::make()
->withText('fake response text') // text to be streamed
->withFinishReason(FinishReason::Stop), // finish reason for final chunk
]);
$text = Prism::text()
->using('anthropic', 'claude-3-sonnet')
->withPrompt('What is the meaning of life?')
->asStream();
$outputText = '';
foreach ($text as $chunk) {
$outputText .= $chunk->text; // will be ['fake ', 'respo', 'nse t', 'ext', ''];
}
expect($outputText)->toBe('fake response text');
```
You can adjust the chunk size by using `withFakeChunkSize` on the fake.
```php
Prism::fake([
TextResponseFake::make()->withText('fake response text'),
])->withFakeChunkSize(1);
```
Now, the text will be streamed in chunks of one character (`['f', 'a', 'k', ...]`).
### Testing Tool Calling while Streaming
When testing streamed responses with tool calls, you can use the `ResponseBuilder` to create a more complex response.
Given a text response with steps, the fake provider will not only generate text chunks, but also include chunks for tool calls and results.
```php
Prism::fake([
(new ResponseBuilder)
->addStep(
TextStepFake::make()
->withToolCalls(
[
new ToolCall('id-123', 'tool', ['input' => 'value']),
]
)
)
->addStep(
TextStepFake::make()
->withToolResults(
[
new ToolResult('id-123', 'tool', ['input' => 'value'], 'result'),
]
)
)
->addStep(
TextStepFake::make()
->withText('fake response text')
)
->toResponse(),
]);
$text = Prism::text()
->using('anthropic', 'claude-3-sonnet')
->withPrompt('What is the meaning of life?')
->asStream();
$outputText = '';
$toolCalls = [];
$toolResults = [];
foreach ($text as $chunk) {
$outputText .= $chunk->text;
// Accumulate tool calls
if ($chunk->toolCalls) {
foreach ($chunk->toolCalls as $call) {
$toolCalls[] = $call;
}
}
// Accumulate tool results
if ($chunk->toolResults) {
foreach ($chunk->toolResults as $result) {
$toolResults[] = $result;
}
}
}
expect($outputText)->toBe('fake response text')
->and($toolCalls)->toHaveCount(1)
->and($toolCalls[0])->toBeInstanceOf(ToolCall::class)
->and($toolCalls[0]->id)->toBe('id-123')
->and($toolCalls[0]->name)->toBe('tool')
->and($toolCalls[0]->arguments())->toBe(['input' => 'value'])
->and($toolResults)->toHaveCount(1)
->and($toolResults[0])->toBeInstanceOf(ToolResult::class)
->and($toolResults[0]->toolCallId)->toBe('id-123')
->and($toolResults[0]->toolName)->toBe('tool')
->and($toolResults[0]->args)->toBe(['input' => 'value'])
->and($toolResults[0]->result)->toBe('result');
```
## Testing Structured Output
```php
use Prism\Prism\Prism;
use Prism\Prism\Testing\StructuredResponseFake;
use Prism\Prism\ValueObjects\Usage;
use Prism\Prism\ValueObjects\Meta;
use Prism\Prism\Enums\FinishReason;
use Prism\Prism\Schema\ObjectSchema;
use Prism\Prism\Schema\StringSchema;
it('can generate structured response', function () {
$schema = new ObjectSchema(
name: 'user',
description: 'A user object, because we love organizing things!',
properties: [
new StringSchema('name', 'The user\'s name (hopefully not "test test")'),
new StringSchema('bio', 'A brief bio (no novels, please)'),
],
requiredFields: ['name', 'bio']
);
$fakeResponse = StructuredResponseFake::make()
->withText(json_encode([
'name' => 'Alice Tester',
'bio' => 'Professional bug hunter and code wrangler'
], JSON_THROW_ON_ERROR))
->withStructured([
'name' => 'Alice Tester',
'bio' => 'Professional bug hunter and code wrangler'
])
->withFinishReason(FinishReason::Stop)
->withUsage(new Usage(10, 20))
->withMeta(new Meta('fake-1', 'fake-model'));
$fake = Prism::fake([$fakeResponse]);
$response = Prism::structured()
->using('anthropic', 'claude-3-sonnet')
->withPrompt('Generate a user profile')
->withSchema($schema)
->asStructured();
// Assertions
expect($response->structured)->toBeArray();
expect($response->structured['name'])->toBe('Alice Tester');
expect($response->structured['bio'])->toBe('Professional bug hunter and code wrangler');
});
```
## Testing Embeddings
```php
use Prism\Prism\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\ValueObjects\Embedding;
use Prism\Prism\ValueObjects\EmbeddingsUsage;
use Prism\Prism\Testing\EmbeddingsResponseFake;
use Prism\Prism\ValueObjects\Meta;
it('can generate embeddings', function () {
$fakeResponse = EmbeddingsResponseFake::make()
->withEmbeddings([Embedding::fromArray(array_fill(0, 1536, 0.1))])
->withUsage(new EmbeddingsUsage(10))
->withMeta(new Meta('fake-emb-1', 'fake-model'));
Prism::fake([$fakeResponse]);
$response = Prism::embeddings()
->using(Provider::OpenAI, 'text-embedding-3-small')
->fromInput('Test content for embedding generation.')
->asEmbeddings();
expect($response->embeddings)->toHaveCount(1)
->and($response->embeddings[0]->embedding)
->toBeArray()
->toHaveCount(1536);
});
```
## Assertions
`PrismFake` provides several helpful assertion methods:
```php
// Assert specific prompt was sent
$fake->assertPrompt('Who are you?');
// Assert number of calls made
$fake->assertCallCount(2);
// Assert detailed request properties
$fake->assertRequest(function ($requests) {
expect($requests[0]->provider())->toBe('anthropic');
expect($requests[0]->model())->toBe('claude-3-sonnet');
});
// Assert provider configuration
$fake->assertProviderConfig(['api_key' => 'sk-1234']);
```
## Using the real response classes
While the fake helpers make tests concise, you can still build responses with the real
classes if you know you will need to test against all the properties of the response:
```php
use Prism\Prism\Text\Response;
use Illuminate\Support\Collection;
use Prism\Prism\Enums\FinishReason;
use Prism\Prism\ValueObjects\Usage;
use Prism\Prism\ValueObjects\Meta;
$response = new Response(
steps: collect([]),
responseMessages: collect([]),
text: 'The meaning of life is 42',
finishReason: FinishReason::Stop,
toolCalls: [],
toolResults: [],
usage: new Usage(42, 42),
meta: new Meta('resp_1', 'real-model'),
messages: collect([]),
additionalContent: [],
);
```
This approach is perfectly valid—but for most tests the fake builders are shorter and
easier to read.
---
---
url: /core-concepts/text-generation.md
---
# Text Generation
Prism provides a powerful interface for generating text using Large Language Models (LLMs). This guide covers everything from basic usage to advanced features like multi-modal interactions and response handling.
## Basic Text Generation
At its simplest, you can generate text with just a few lines of code:
```php
use Prism\Prism\Prism;
use Prism\Prism\Enums\Provider;
$response = Prism::text()
->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
->withPrompt('Tell me a short story about a brave knight.')
->asText();
echo $response->text;
```
## System Prompts and Context
System prompts help set the behavior and context for the AI. They're particularly useful for maintaining consistent responses or giving the LLM a persona:
```php
use Prism\Prism\Prism;
use Prism\Prism\Enums\Provider;
$response = Prism::text()
->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
->withSystemPrompt('You are an expert mathematician who explains concepts simply.')
->withPrompt('Explain the Pythagorean theorem.')
->asText();
```
You can also use Laravel views for complex system prompts:
```php
use Prism\Prism\Prism;
use Prism\Prism\Enums\Provider;
$response = Prism::text()
->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
->withSystemPrompt(view('prompts.math-tutor'))
->withPrompt('What is calculus?')
->asText();
```
You an also pass a View to the `withPrompt` method.
## Multi-Modal Input
Prism supports including images, documents, audio, and video files in your prompts for rich multi-modal analysis:
```php
use Prism\Prism\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\ValueObjects\Media\Image;
use Prism\Prism\ValueObjects\Media\Document;
use Prism\Prism\ValueObjects\Media\Audio;
use Prism\Prism\ValueObjects\Media\Video;
// Analyze an image
$response = Prism::text()
->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
->withPrompt(
'What objects do you see in this image?',
[Image::fromLocalPath('/path/to/image.jpg')]
)
->asText();
// Process a document
$response = Prism::text()
->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
->withPrompt(
'Summarize the key points from this document',
[Document::fromLocalPath('/path/to/document.pdf')]
)
->asText();
// Analyze audio content
$response = Prism::text()
->using(Provider::Gemini, 'gemini-1.5-flash')
->withPrompt(
'What is being discussed in this audio?',
[Audio::fromLocalPath('/path/to/audio.mp3')]
)
->asText();
// Process video content
$response = Prism::text()
->using(Provider::Gemini, 'gemini-1.5-flash')
->withPrompt(
'Describe what happens in this video',
[Video::fromUrl('https://www.youtube.com/watch?v=dQw4w9WgXcQ')]
)
->asText();
// Multiple media types in one prompt
$response = Prism::text()
->using(Provider::Gemini, 'gemini-1.5-flash')
->withPrompt(
'Compare this image with the information in this document',
[
Image::fromLocalPath('/path/to/chart.png'),
Document::fromLocalPath('/path/to/report.pdf')
]
)
->asText();
```
## Message Chains and Conversations
For interactive conversations, use message chains to maintain context:
```php
use Prism\Prism\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\ValueObjects\Messages\UserMessage;
use Prism\Prism\ValueObjects\Messages\AssistantMessage;
$response = Prism::text()
->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
->withMessages([
new UserMessage('What is JSON?'),
new AssistantMessage('JSON is a lightweight data format...'),
new UserMessage('Can you show me an example?')
])
->asText();
```
### Message Types
* `SystemMessage`
* `UserMessage`
* `AssistantMessage`
* `ToolResultMessage`
> \[!NOTE]
> Some providers, like Anthropic, do not support the `SystemMessage` type. In those cases we convert `SystemMessage` to `UserMessage`.
## Generation Parameters
Fine-tune your generations with various parameters:
`withMaxTokens`
Maximum number of tokens to generate.
`usingTemperature`
Temperature setting.
The value is passed through to the provider. The range depends on the provider and model. For most providers, 0 means almost deterministic results, and higher values mean more randomness.
> \[!TIP]
> It is recommended to set either temperature or topP, but not both.
`usingTopP`
Nucleus sampling.
The value is passed through to the provider. The range depends on the provider and model. For most providers, nucleus sampling is a number between 0 and 1. E.g. 0.1 would mean that only tokens with the top 10% probability mass are considered.
> \[!TIP]
> It is recommended to set either temperature or topP, but not both.
`withClientOptions`
Under the hood we use Laravel's [HTTP client](https://laravel.com/docs/11.x/http-client#main-content). You can use this method to pass any of Guzzles [request options](https://docs.guzzlephp.org/en/stable/request-options.html) e.g. `->withClientOptions(['timeout' => 30])`.
`withClientRetry`
Under the hood we use Laravel's [HTTP client](https://laravel.com/docs/11.x/http-client#main-content). You can use this method to set [retries](https://laravel.com/docs/11.x/http-client#retries) e.g. `->withClientRetry(3, 100)`.
`usingProviderConfig`
This allows for complete or partial override of the providers configuration. This is great for multi-tenant applications where users supply their own API keys. These values are merged with the original configuration allowing for partial or complete config override.
## Response Handling
The response object provides rich access to the generation results:
```php
use Prism\Prism\Prism;
use Prism\Prism\Enums\Provider;
$response = Prism::text()
->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
->withPrompt('Explain quantum computing.')
->asText();
// Access the generated text
echo $response->text;
// Check why the generation stopped
echo $response->finishReason->name;
// Get token usage statistics
echo "Prompt tokens: {$response->usage->promptTokens}";
echo "Completion tokens: {$response->usage->completionTokens}";
// For multi-step generations, examine each step
foreach ($response->steps as $step) {
echo "Step text: {$step->text}";
echo "Step tokens: {$step->usage->completionTokens}";
}
// Access message history
foreach ($response->responseMessages as $message) {
if ($message instanceof AssistantMessage) {
echo $message->content;
}
}
```
### Finish Reasons
```php
FinishReason::Stop;
FinishReason::Length;
FinishReason::ContentFilter;
FinishReason::ToolCalls;
FinishReason::Error;
FinishReason::Other;
FinishReason::Unknown;
```
## Error Handling
Remember to handle potential errors in your generations:
```php
use Prism\Prism\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\Exceptions\PrismException;
use Throwable;
try {
$response = Prism::text()
->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
->withPrompt('Generate text...')
->asText();
} catch (PrismException $e) {
Log::error('Text generation failed:', ['error' => $e->getMessage()]);
} catch (Throwable $e) {
Log::error('Generic error:', ['error' => $e->getMessage()]);
}
```
---
---
url: /core-concepts/tools-function-calling.md
---
# Tools & Function Calling
Need your AI assistant to check the weather, search a database, or call your API? Tools are here to help! They let you extend your AI's capabilities by giving it access to specific functions it can call.
## Tool Concept Overview
Think of tools as special functions that your AI assistant can use when it needs to perform specific tasks. Just like how Laravel's facades provide a clean interface to complex functionality, Prism tools give your AI a clean way to interact with external services and data sources.
```php
use Prism\Prism\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\Facades\Tool;
$weatherTool = Tool::as('weather')
->for('Get current weather conditions')
->withStringParameter('city', 'The city to get weather for')
->using(function (string $city): string {
// Your weather API logic here
return "The weather in {$city} is sunny and 72°F.";
});
$response = Prism::text()
->using(Provider::Anthropic, 'claude-3-5-sonnet-latest')
->withMaxSteps(2)
->withPrompt('What is the weather like in Paris?')
->withTools([$weatherTool])
->asText();
```
## Max Steps
Prism defaults to allowing a single step. To use Tools, you'll need to increase this using `withMaxSteps`:
```php
use Prism\Prism\Prism;
use Prism\Prism\Enums\Provider;
Prism::text()
->using(Provider::Anthropic, 'claude-3-5-sonnet-latest')
// Increase max steps to at least 2
->withMaxSteps(2)
->withPrompt('What is the weather like in Paris?')
->withTools([$weatherTool])
->asText();
```
You should use a higher number of max steps if you expect your initial prompt to make multiple tool calls.
## Creating Basic Tools
Creating tools in Prism is straightforward and fluent. Here's how you can create a simple tool:
```php
use Prism\Prism\Facades\Tool;
$searchTool = Tool::as('search')
->for('Search for current information')
->withStringParameter('query', 'The search query')
->using(function (string $query): string {
// Your search implementation
return "Search results for: {$query}";
});
```
Tools can take a variety of parameters, but must always return a string.
## Error Handling
By default, tools handle invalid parameters gracefully by returning error messages instead of throwing exceptions. This helps AI assistants understand and potentially correct their mistakes.
```php
$tool = Tool::as('calculate')
->for('Add two numbers')
->withNumberParameter('a', 'First number')
->withNumberParameter('b', 'Second number')
->using(fn (int $a, int $b): string => (string) ($a + $b));
// If AI provides invalid parameters, it receives:
// "Parameter validation error: Type mismatch. Expected: [a (NumberSchema, required), b (NumberSchema, required)]. Received: {"a":"five","b":10}"
```
### Opting Out
If you prefer exceptions for invalid parameters:
```php
// Per-tool
$tool->withoutErrorHandling();
// Per-request
Prism::text()->withoutToolErrorHandling();
```
**Best Practice**: Use default error handling for conversational AI. Disable it only when you need strict validation that stops execution.
## Parameter Definition
Prism offers multiple ways to define tool parameters, from simple primitives to complex objects.
### String Parameters
Perfect for text inputs:
```php
use Prism\Prism\Facades\Tool;
$tool = Tool::as('search')
->for('Search for information')
->withStringParameter('query', 'The search query')
->using(function (string $query): string {
return "Search results for: {$query}";
});
```
### Number Parameters
For integer or floating-point values:
```php
use Prism\Prism\Facades\Tool;
$tool = Tool::as('calculate')
->for('Perform calculations')
->withNumberParameter('value', 'The number to process')
->using(function (float $value): string {
return "Calculated result: {$value * 2}";
});
```
### Boolean Parameters
For true/false flags:
```php
use Prism\Prism\Facades\Tool;
$tool = Tool::as('feature_toggle')
->for('Toggle a feature')
->withBooleanParameter('enabled', 'Whether to enable the feature')
->using(function (bool $enabled): string {
return "Feature is now " . ($enabled ? 'enabled' : 'disabled');
});
```
### Array Parameters
For handling lists of items:
```php
use Prism\Prism\Facades\Tool;
$tool = Tool::as('process_tags')
->for('Process a list of tags')
->withArrayParameter(
'tags',
'List of tags to process',
new StringSchema('tag', 'A single tag')
)
->using(function (array $tags): string {
return "Processing tags: " . implode(', ', $tags);
});
```
### Enum Parameters
When you need to restrict values to a specific set:
```php
use Prism\Prism\Facades\Tool;
$tool = Tool::as('set_status')
->for('Set the status')
->withEnumParameter(
'status',
'The new status',
['draft', 'published', 'archived']
)
->using(function (string $status): string {
return "Status set to: {$status}";
});
```
### Object Parameters
For complex objects without needing to create separate schema instances:
```php
use Prism\Prism\Facades\Tool;
use Prism\Prism\Schema\StringSchema;
use Prism\Prism\Schema\NumberSchema;
$tool = Tool::as('update_user')
->for('Update a user profile')
->withObjectParameter(
'user',
'The user profile data',
[
new StringSchema('name', 'User\'s full name'),
new NumberSchema('age', 'User\'s age'),
new StringSchema('email', 'User\'s email address')
],
requiredFields: ['name', 'email']
)
->using(function (array $user): string {
return "Updated user profile for: {$user['name']}";
});
```
### Schema-based Parameters
For complex, nested data structures, you can use Prism's schema system:
```php
use Prism\Prism\Facades\Tool;
use Prism\Prism\Schema\ObjectSchema;
use Prism\Prism\Schema\StringSchema;
use Prism\Prism\Schema\NumberSchema;
$tool = Tool::as('create_user')
->for('Create a new user profile')
->withParameter(new ObjectSchema(
name: 'user',
description: 'The user profile data',
properties: [
new StringSchema('name', 'User\'s full name'),
new NumberSchema('age', 'User\'s age'),
new StringSchema('email', 'User\'s email address')
],
requiredFields: ['name', 'email']
))
->using(function (array $user): string {
return "Created user profile for: {$user['name']}";
});
```
> \[!TIP]
> For more complex parameter definitions, Prism provides a powerful schema system. See our [complete schemas guide](/core-concepts/schemas) to learn how to define complex nested objects, arrays, enums, and more.
## Complex Tool Implementation
For more sophisticated tools, you can create dedicated classes:
```php
namespace App\Tools;
use Prism\Prism\Tool;
use Illuminate\Support\Facades\Http;
class SearchTool extends Tool
{
public function __construct()
{
$this
->as('search')
->for('useful when you need to search for current events')
->withStringParameter('query', 'Detailed search query. Best to search one topic at a time.')
->using($this);
}
public function __invoke(string $query): string
{
$response = Http::get('https://serpapi.com/search', [
'engine' => 'google',
'q' => $query,
'google_domain' => 'google.com',
'gl' => 'us',
'hl' => 'en',
'api_key' => config('services.serpapi.api_key'),
]);
$results = collect($response->json('organic_results'));
$results->map(function ($result) {
return [
'title' => $result['title'],
'link' => $result['link'],
'snippet' => $result['snippet'],
];
})->take(4);
return view('prompts.search-tool-results', [
'results' => $results,
])->render();
}
}
```
## Tool Choice Options
You can control how the AI uses tools with the `withToolChoice` method:
```php
use Prism\Prism\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\Enums\ToolChoice;
$prism = Prism::text()
->using(Provider::Anthropic, 'claude-3-5-sonnet-latest')
->withMaxSteps(2)
->withPrompt('How is the weather in Paris?')
->withTools([$weatherTool])
// Let the AI decide whether to use tools
->withToolChoice(ToolChoice::Auto)
// Force the AI to use a tool
->withToolChoice(ToolChoice::Any)
// Force the AI to use a specific tool
->withToolChoice('weather');
```
> \[!WARNING]
> Tool choice support varies by provider. Check your provider's documentation for specific capabilities.
## Response Handling with Tools
When your AI uses tools, you can inspect the results and see how it arrived at its answer:
```php
use Prism\Prism\Prism;
use Prism\Prism\Enums\Provider;
$response = Prism::text()
->using(Provider::Anthropic, 'claude-3-5-sonnet-latest')
->withMaxSteps(2)
->withPrompt('What is the weather like in Paris?')
->withTools([$weatherTool])
->asText();
// Get the final answer
echo $response->text;
// ->text is empty for tool calls
// Inspect tool usage
if ($response->toolResults) {
foreach ($response->toolResults as $toolResult) {
echo "Tool: " . $toolResult->toolName . "\n";
echo "Result: " . $toolResult->result . "\n";
}
}
foreach ($response->steps as $step) {
if ($step->toolCalls) {
foreach ($step->toolCalls as $toolCall) {
echo "Tool: " . $toolCall->name . "\n";
echo "Arguments: " . json_encode($toolCall->arguments()) . "\n";
}
}
}
```
## Provider Tools
In addition to custom tools that you define, Prism supports **provider tools** - built-in capabilities offered directly by AI providers. These are specialized tools that leverage the provider's own infrastructure and services.
### Understanding Provider Tools vs Custom Tools
**Custom Tools** (covered above) are functions you define and implement yourself:
* You control the logic and implementation
* Called by the AI, executed by your code
* Can access your databases, APIs, and services
**Provider Tools** are built-in capabilities offered by the AI provider:
* Implemented and executed by the provider
* Access the provider's own services and infrastructure
* Enable capabilities like code execution, web search, and more
### Using Provider Tools
Provider tools are added to your requests using the `withProviderTools()` method with `ProviderTool` objects:
```php
use Prism\Prism\Prism;
use Prism\Prism\ValueObjects\ProviderTool;
$response = Prism::text()
->using('anthropic', 'claude-3-5-sonnet-latest')
->withPrompt('Calculate the fibonacci sequence up to 100')
->withProviderTools([
new ProviderTool(type: 'code_execution_20250522', name: 'code_execution')
])
->asText();
```
### Available Provider Tools
Each provider offers different built-in capabilities. Check the provider-specific documentation for detailed information about available tools, configuration options, and usage examples.
### ProviderTool Object
The `ProviderTool` class accepts three parameters:
```php
new ProviderTool(
type: 'code_execution_20250522', // Required: The provider tool identifier
name: 'code_execution', // Optional: Custom name for the tool
options: [] // Optional: Provider-specific options
)
```
* **type**: The provider-specific tool identifier (required)
* **name**: Optional custom name for the tool
* **options**: Additional provider-specific configuration options
### Combining Provider Tools and Custom Tools
You can use both provider tools and custom tools in the same request:
```php
use Prism\Prism\Prism;
use Prism\Prism\ValueObjects\ProviderTool;
use Prism\Prism\Facades\Tool;
$customTool = Tool::as('database_lookup')
->for('Look up user information')
->withStringParameter('user_id', 'The user ID to look up')
->using(function (string $userId): string {
// Your database lookup logic
return "User data for ID: {$userId}";
});
$response = Prism::text()
->using('anthropic', 'claude-3-5-sonnet-latest')
->withMaxSteps(5)
->withPrompt('Look up user 123 and calculate their usage statistics')
->withTools([$customTool])
->withProviderTools([
new ProviderTool(type: 'code_execution_20250522', name: 'code_execution')
])
->asText();
```
---
---
url: /index.md
---
---
---
url: /input-modalities/video.md
---
# Video
Prism supports including video files and YouTube videos in your messages for advanced analysis with supported providers like Gemini.
See the [provider support table](/getting-started/introduction.html#provider-support) to check whether Prism supports your chosen provider.
Note however that provider support may differ by model. If you receive error messages with a provider that Prism indicates is supported, check the provider's documentation as to whether the model you are using supports video files.
::: tip
For other input modalities like audio and images, see their respective documentation pages:
* [Audio documentation](/input-modalities/audio.html)
* [Images documentation](/input-modalities/images.html)
:::
## Getting started
To add a video to your prompt, use the `withPrompt` method with a `Video` value object:
```php
use Prism\Prism\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\ValueObjects\Media\Video;
// From a local path
$response = Prism::text()
->using(Provider::Gemini, 'gemini-1.5-flash')
->withPrompt(
"What's in this video?",
[Video::fromLocalPath(path: '/path/to/video.mp4')]
)
->asText();
// From a path on a storage disk
$response = Prism::text()
->using(Provider::Gemini, 'gemini-1.5-flash')
->withPrompt(
"What's in this video?",
[Video::fromStoragePath(
path: '/path/to/video.mp4',
diskName: 'my-disk' // optional - omit/null for default disk
)]
)
->asText();
// From a URL
$response = Prism::text()
->using(Provider::Gemini, 'gemini-1.5-flash')
->withPrompt(
'Analyze this video:',
[Video::fromUrl(url: 'https://example.com/video.mp4')]
)
->asText();
// From a YouTube URL (automatically extracts the video ID)
$response = Prism::text()
->using(Provider::Gemini, 'gemini-1.5-flash')
->withPrompt(
'What is this YouTube video about?',
[Video::fromUrl(url: 'https://www.youtube.com/watch?v=dQw4w9WgXcQ')]
)
->asText();
// From shortened YouTube URL
$response = Prism::text()
->using(Provider::Gemini, 'gemini-1.5-flash')
->withPrompt(
'What is this YouTube video about?',
[Video::fromUrl(url: 'https://youtu.be/dQw4w9WgXcQ')]
)
->asText();
// From base64
$response = Prism::text()
->using(Provider::Gemini, 'gemini-1.5-flash')
->withPrompt(
'Analyze this video:',
[Video::fromBase64(
base64: base64_encode(file_get_contents('/path/to/video.mp4')),
mimeType: 'video/mp4'
)]
)
->asText();
// From raw content
$response = Prism::text()
->using(Provider::Gemini, 'gemini-1.5-flash')
->withPrompt(
'Analyze this video:',
[Video::fromRawContent(
rawContent: file_get_contents('/path/to/video.mp4'),
mimeType: 'video/mp4'
)]
)
->asText();
```
## Alternative: Using withMessages
You can also include videos using the message-based approach:
```php
use Prism\Prism\ValueObjects\Messages\UserMessage;
use Prism\Prism\ValueObjects\Media\Video;
$message = new UserMessage(
"What's in this video?",
[Video::fromLocalPath(path: '/path/to/video.mp4')]
);
$response = Prism::text()
->using(Provider::Gemini, 'gemini-1.5-flash')
->withMessages([$message])
->asText();
```
## Supported Video Types
Prism supports a variety of video formats, including:
* MP4 (video/mp4)
* MOV (video/quicktime)
* WEBM (video/webm)
* AVI (video/x-msvideo)
* YouTube videos (via URL)
The specific supported formats depend on the provider. Gemini is currently the main provider with comprehensive video analysis capabilities. Check the provider's documentation for a complete list of supported formats.
## YouTube Video Support
Prism provides seamless support for YouTube videos. When you pass a YouTube URL to `Video::fromUrl()`, Prism automatically extracts the video ID and sends it to the provider in the appropriate format.
Supported YouTube URL formats:
* Standard: `https://www.youtube.com/watch?v=VIDEO_ID`
* Shortened: `https://youtu.be/VIDEO_ID`
Example:
```php
$response = Prism::text()
->using(Provider::Gemini, 'gemini-1.5-flash')
->withPrompt(
'What is this YouTube video about?',
[Video::fromUrl('https://www.youtube.com/watch?v=dQw4w9WgXcQ')]
)
->asText();
```
## Transfer mediums
Providers are not consistent in their support of sending raw contents, base64 and/or URLs.
Prism tries to smooth over these rough edges, but its not always possible.
### Supported conversions
* Where a provider does not support URLs: Prism will fetch the URL and use base64 or rawContent.
* Where you provide a file, base64 or rawContent: Prism will switch between base64 and rawContent depending on what the provider accepts.
### Limitations
* Where a provider only supports URLs: if you provide a file path, raw contents or base64, for security reasons Prism does not create a URL for you and your request will fail.
---
---
url: /providers/voyageai.md
---
# Voyage AI
## Configuration
```php
'voyageai' => [
'api_key' => env('VOYAGEAI_API_KEY', ''),
'url' => env('VOYAGEAI_URL', 'https://api.voyageai.com/v1'),
],
```
## Provider specific options
You can change some options on your request specific to Voyage AI by using `->withProviderOptions()`.
### Input type
By default, Voyage AI generates general purpose vectors.
However, they taylor your vectors for the task they are intended for - for search ("query") or for retrieval ("document"):
For search / querying:
```php
use Prism\Prism\Enums\Provider;
use Prism\Prism\Prism;
Prism::embeddings()
->using(Provider::VoyageAI, 'voyage-3-lite')
->fromInput('The food was delicious and the waiter...')
->withProviderOptions(['inputType' => 'query'])
->asEmbeddings();
```
For document retrieval:
```php
use Prism\Prism\Enums\Provider;
use Prism\Prism\Prism;
Prism::embeddings()
->using(Provider::VoyageAI, 'voyage-3-lite')
->fromInput('The food was delicious and the waiter...')
->withProviderOptions(['inputType' => 'document'])
->asEmbeddings();
```
### Truncation
By default, Voyage AI truncates inputs that are over the context length.
You can force it to throw an error instead by setting truncation to false.
```php
use Prism\Prism\Enums\Provider;
use Prism\Prism\Prism;
Prism::embeddings()
->using(Provider::VoyageAI, 'voyage-3-lite')
->fromInput('The food was delicious and the waiter...')
->withProviderOptions(['truncation' => false])
->asEmbeddings();
```
---
---
url: /providers/xai.md
---
# xAI
## Configuration
```php
'xai' => [
'api_key' => env('XAI_API_KEY', ''),
'url' => env('XAI_URL', 'https://api.x.ai/v1'),
],
```
## Provider-specific options
### Extended Thinking/Reasoning
xAI's Grok models support an optional extended thinking mode, where the model will reason through problems before returning its answer. This is particularly useful for complex mathematical problems, logical reasoning, and detailed analysis tasks.
#### Enabling thinking mode
```php
use Prism\Prism\Enums\Provider;
use Prism\Prism\Prism;
$response = Prism::text()
->using(Provider::XAI, 'grok-4')
->withPrompt('Solve this complex equation: 3x² + 5x - 2 = 0')
->withProviderOptions([
'thinking' => ['enabled' => true]
])
->asText();
```
Thinking content is automatically extracted when present in the response and can be accessed through streaming chunks.
If you prefer not to process thinking content, you can disable it. Set the `thinking` option to `false`.
#### Streaming thinking content
When using streaming, thinking content is yielded as separate chunks with `ChunkType::Thinking`:
```php
use Prism\Prism\Enums\ChunkType;
$stream = Prism::text()
->using(Provider::XAI, 'grok-4')
->withPrompt('Explain quantum entanglement in detail')
->asStream();
foreach ($stream as $chunk) {
if ($chunk->chunkType === ChunkType::Thinking) {
echo $chunk->text . PHP_EOL; // Outputs: Thinking...
} elseif ($chunk->chunkType === ChunkType::Text) {
echo $chunk->text;
}
}
```
### Structured Output
xAI supports structured output through JSON schema validation. The following models support structured output:
* `grok-3`
* `grok-4`
```php
use Prism\Prism\Schema\ObjectSchema;
use Prism\Prism\Schema\StringSchema;
use Prism\Prism\Schema\BooleanSchema;
$schema = new ObjectSchema(
'weather_report',
'Weather forecast with recommendations',
[
new StringSchema('forecast', 'The weather forecast'),
new StringSchema('clothing', 'Clothing recommendation'),
new BooleanSchema('coat_required', 'Whether a coat is needed'),
],
['forecast', 'clothing', 'coat_required']
);
$response = Prism::structured()
->withSchema($schema)
->using(Provider::XAI, 'grok-4')
->withPrompt('What\'s the weather like in Detroit and should I wear a coat?')
->asStructured();
// Access structured data
echo $response->structured['forecast']; // "75° and sunny"
echo $response->structured['coat_required']; // false
```
#### Strict schema mode
Enable strict schema validation for more reliable structured output:
```php
$response = Prism::structured()
->withSchema($schema)
->using(Provider::XAI, 'grok-4')
->withProviderOptions([
'schema' => ['strict' => true]
])
->withPrompt('Analyze this data')
->asStructured();
```
### Tool Calling
xAI supports function calling with tools. Tools can be used alongside thinking mode for complex problem-solving scenarios.
```php
use Prism\Prism\Facades\Tool;
$tools = [
Tool::as('calculator')
->for('Perform mathematical calculations')
->withStringParameter('expression', 'Mathematical expression to calculate')
->using(fn (string $expression): string => "Result: " . eval("return $expression;")),
Tool::as('weather')
->for('Get current weather information')
->withStringParameter('city', 'City name')
->using(fn (string $city): string => "Weather in {$city}: 72°F and sunny"),
];
$response = Prism::text()
->using(Provider::XAI, 'grok-4')
->withTools($tools)
->withMaxSteps(3)
->withPrompt('Calculate 15 * 23 and tell me the weather in Detroit')
->asText();
```
### Model Parameters
#### Temperature Control
Control the randomness of responses:
```php
$response = Prism::text()
->using(Provider::XAI, 'grok-4')
->withTemperature(0.7) // 0.0 = deterministic, 1.0 = very creative
->withPrompt('Write a creative story')
->asText();
```
#### Top-P Sampling
Control nucleus sampling:
```php
$response = Prism::text()
->using(Provider::XAI, 'grok-4')
->withTopP(0.9) // Consider top 90% probability mass
->withPrompt('Generate diverse responses')
->asText();
```
#### Token Limits
Set maximum output tokens:
```php
$response = Prism::text()
->using(Provider::XAI, 'grok-4')
->withMaxTokens(1000)
->withPrompt('Write a detailed explanation')
->asText();
```
## Advanced Examples
### Complex Analysis with Thinking
```php
$response = Prism::text()
->using(Provider::XAI, 'grok-4')
->withPrompt('
Analyze the economic implications of implementing a universal basic income program.
Consider both potential benefits and drawbacks, and provide specific examples.
')
->asStream();
$analysis = '';
$reasoning = '';
foreach ($response as $chunk) {
if ($chunk->chunkType === ChunkType::Thinking) {
$reasoning .= $chunk->text;
} else {
$analysis .= $chunk->text;
echo $chunk->text; // Stream to user
}
}
// Save the reasoning process for later review
file_put_contents('analysis_reasoning.txt', $reasoning);
```
### Structured Data Extraction
```php
use Prism\Prism\Schema\ArraySchema;
use Prism\Prism\Schema\IntegerSchema;
use Prism\Prism\Schema\NumberSchema;
$schema = new ObjectSchema(
'financial_analysis',
'Complete financial analysis result',
[
new StringSchema('summary', 'Executive summary'),
new NumberSchema('total_revenue', 'Total revenue amount'),
new NumberSchema('profit_margin', 'Profit margin percentage'),
new ArraySchema('recommendations', 'List of recommendations',
new StringSchema('recommendation', 'Individual recommendation')
),
new ObjectSchema('risk_assessment', 'Risk analysis', [
new StringSchema('level', 'Risk level (low/medium/high)'),
new IntegerSchema('score', 'Risk score from 1-10'),
], ['level', 'score']),
],
['summary', 'total_revenue', 'profit_margin', 'recommendations', 'risk_assessment']
);
$response = Prism::structured()
->withSchema($schema)
->using(Provider::XAI, 'grok-4')
->withPrompt('
Analyze this financial data:
Q1 Revenue: $1.2M, Q1 Costs: $800K
Q2 Revenue: $1.5M, Q2 Costs: $900K
Provide a complete analysis with recommendations.
')
->asStructured();
$analysis = $response->structured;
echo "Revenue: $" . number_format($analysis['total_revenue']);
echo "Risk Level: " . $analysis['risk_assessment']['level'];
```
### Model Validation
Structured output is only supported on specific models. Prism will throw an exception for unsupported models:
```php
use Prism\Prism\Exceptions\PrismException;
try {
$response = Prism::structured()
->withSchema($schema)
->using(Provider::XAI, 'unsupported-model')
->asStructured();
} catch (PrismException $e) {
// Handle unsupported model error
echo "Error: " . $e->getMessage();
}
```
## Considerations
### Thinking Content Processing
* Thinking content is automatically filtered to remove repetitive "Thinking..." patterns
* Only meaningful reasoning content is yielded in thinking chunks
* Thinking content appears before regular response content in streams
* Thinking can be disabled if not needed to reduce processing overhead
### API Compatibility
xAI uses an OpenAI-compatible API structure, which means:
* Request/response formats are similar to OpenAI
* Tool calling follows OpenAI's function calling specification
* Structured output uses JSON schema format
* Streaming follows server-sent events (SSE) format
### Token Management
* Thinking tokens count toward your total token usage
* Set appropriate `maxTokens` limits when expecting long thinking sequences
* Monitor usage through the response objects for cost tracking