---
url: /providers/anthropic.md
---
# Anthropic

## Configuration

```php
'anthropic' => [
    'api_key' => env('ANTHROPIC_API_KEY', ''),
    'version' => env('ANTHROPIC_API_VERSION', '2023-06-01'),
    'default_thinking_budget' => env('ANTHROPIC_DEFAULT_THINKING_BUDGET', 1024),
    // Include beta strings as a comma separated list (e.g. output-128k-2025-02-19, code-execution-2025-05-22).
    'anthropic_beta' => env('ANTHROPIC_BETA', null),
]
```

## Prompt caching

Anthropic's prompt caching feature allows you to drastically reduce latency and your API bill when repeatedly re-using blocks of content within five minutes or one hour of each other, depending on the Anthropic compatible TTL option you provide.

There are two ways to enable prompt caching:

* Automatic caching
* Explicit cache breakpoints

To enable automatic caching, simply add a single cache\_control field at the top level of your request:

```php
use Prism\Prism\Enums\Provider;
use Prism\Prism\Facades\Prism;
use Prism\Prism\Tool;
use Prism\Prism\ValueObjects\Messages\UserMessage;
use Prism\Prism\ValueObjects\Messages\SystemMessage;

Prism::text()
    ->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
    ->withSystemPrompt(
        (new SystemMessage('I am a long re-usable system message.'))
    )
    ->withMessages([
        (new UserMessage('I am a long re-usable user message.'))
    ])
    ->withTools([
        Tool::as('cache me')
    ])
    ->withProviderOptions(['cache_control' => ['type' => 'ephemeral']])
    ->asText();
```

We support Anthropic explicit cache breakpoints on:

* System Messages (text only)
* User Messages (Text, Image and PDF (pdf only))
* Assistant Messages (text only)
* Tools

The API for enabling prompt caching is the same for all, enabled via the `withProviderOptions()` method. Where a UserMessage contains both text and an image or document, both will be cached.

```php
use Prism\Prism\Enums\Provider;
use Prism\Prism\Facades\Prism;
use Prism\Prism\Tool;
use Prism\Prism\ValueObjects\Messages\UserMessage;
use Prism\Prism\ValueObjects\Messages\SystemMessage;

Prism::text()
    ->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
    ->withSystemPrompt(
        (new SystemMessage('I am a long re-usable system message.'))
            ->withProviderOptions(['cacheType' => 'ephemeral', 'cacheTtl' => '1h'])
    )
    ->withMessages([
        (new UserMessage('I am a long re-usable user message.'))
            ->withProviderOptions(['cacheType' => 'ephemeral'])
    ])
    ->withTools([
        Tool::as('cache me')
            ->withProviderOptions(['cacheType' => 'ephemeral'])
    ])
    ->asText();
```

If you prefer, you can use the `AnthropicCacheType` Enum like so:

```php
use Prism\Prism\Enums\Provider;
use Prism\Prism\Providers\Anthropic\Enums\AnthropicCacheType;
use Prism\Prism\ValueObjects\Messages\UserMessage;
use Prism\Prism\ValueObjects\Media\Document;

(new UserMessage('I am a long re-usable user message.'))->withProviderOptions(['cacheType' => AnthropicCacheType::ephemeral])
```

**Important:** To enable prompt caching:

* System messages must use `withSystemPrompt()` or `withSystemPrompts()` (Anthropic does not allow SystemMessages in the messages array)
* User and Assistant messages must use `withMessages()`
* Tools use `withTools()`
* All message types support caching via `withProviderOptions(['cacheType' => 'ephemeral'])`
* You cannot use `withPrompt()` for caching as it doesn't allow adding provider options to individual messages
* Anthropic supports two TTL options: `5m` (default) or `1h`. You can leave the `cacheTtl` unset and Anthropic will use the default TTL of `5m`.

### Tool result caching

In addition to caching prompts and tool definitions, Prism supports caching tool results. This is particularly useful when making multiple tool calls where results might be referenced repeatedly.

To enable tool result caching, use the `tool_result_cache_type` provider option on your request:

```php
use Prism\Prism\Facades\Prism;

$response = Prism::text()
    ->using('anthropic', 'claude-3-5-sonnet-20241022')
    ->withMaxSteps(30)
    ->withTools([new WeatherTool()])
    ->withProviderOptions([
        'tool_result_cache_type' => 'ephemeral'
    ])
    ->withPrompt('Check the weather in New York, London, Tokyo, Paris, and Sydney')
    ->asText();
```

When multiple tool results are returned, Prism automatically applies caching to only the last result, which caches all preceding results as well. This avoids Anthropic's 4-cache-breakpoint limitation.

Please ensure you read Anthropic's [prompt caching documentation](https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching), which covers some important information on e.g. minimum cacheable tokens and message order consistency.

## Extended thinking

Claude Sonnet 3.7 supports an optional extended thinking mode, where it will reason before returning its answer. Please ensure your consider [Anthropic's own extended thinking documentation](https://docs.anthropic.com/en/docs/build-with-claude/extended-thinking) before using extended thinking with caching and/or tools, as there are some important limitations and behaviours to be aware of.

### Enabling extended thinking and setting budget

Prism supports thinking mode for text and structured with the same API:

```php
use Prism\Prism\Enums\Provider;
use Prism\Prism\Facades\Prism;

Prism::text()
    ->using('anthropic', 'claude-3-7-sonnet-latest')
    ->withPrompt('What is the meaning of life, the universe and everything in popular fiction?')
    // enable thinking
    ->withProviderOptions(['thinking' => ['enabled' => true]]) 
    ->asText();
```

By default Prism will set the thinking budget to the value set in config, or where that isn't set, the minimum allowed (1024).

You can overide the config (or its default) using `withProviderOptions`:

```php
use Prism\Prism\Enums\Provider;
use Prism\Prism\Facades\Prism;

Prism::text()
    ->using('anthropic', 'claude-3-7-sonnet-latest')
    ->withPrompt('What is the meaning of life, the universe and everything in popular fiction?')
    // Enable thinking and set a budget
    ->withProviderOptions([
        'thinking' => [
            'enabled' => true, 
            'budgetTokens' => 2048
        ]
    ]);
```

Note that thinking tokens count towards output tokens, so you will be billed for them and your token budget must be less than the max tokens you have set for the request.

If you expect a long response, you should ensure there's enough tokens left for the response - i.e. does (maxTokens - thinkingBudget) leave a sufficient remainder.

### Inspecting the thinking block

Anthropic returns the thinking block with its response.

You can access it via the additionalContent property on either the Response or the relevant step.

On the Response (easiest if not using tools):

```php
use Prism\Prism\Enums\Provider;
use Prism\Prism\Facades\Prism;

Prism::text()
    ->using('anthropic', 'claude-3-7-sonnet-latest')
    ->withPrompt('What is the meaning of life, the universe and everything in popular fiction?')
    ->withProviderOptions(['thinking' => ['enabled' => true']]) 
    ->asText();

$response->additionalContent['thinking'];
```

On the Step (necessary if using tools, as Anthropic returns the thinking block on the ToolCall step):

```php
$tools = [...];

$response = Prism::text()
    ->using('anthropic', 'claude-3-7-sonnet-latest')
    ->withTools($tools)
    ->withMaxSteps(3)
    ->withPrompt('What time is the tigers game today and should I wear a coat?')
    ->withProviderOptions(['thinking' => ['enabled' => true]])
    ->asText();

$response->steps->first()->additionalContent->thinking;
```

### Extended output mode

Claude Sonnet 3.7 also brings extended output mode which increase the output limit to 128k tokens.

This feature is currently in beta, so you will need to enable to by adding `output-128k-2025-02-19` to your Anthropic anthropic\_beta config (see [Configuration](#configuration) above).

## Streaming

Claude supports streaming responses in real-time. All the standard streaming methods work with Anthropic models:

```php
// Stream events
$stream = Prism::text()
    ->using('anthropic', 'claude-3-7-sonnet-latest')
    ->withPrompt('Write a story')
    ->asStream();

// Server-Sent Events
return Prism::text()
    ->using('anthropic', 'claude-3-7-sonnet-latest')
    ->withPrompt(request('message'))
    ->asEventStreamResponse();
```

### Streaming with Extended Thinking

When using extended thinking, the reasoning process streams separately from the final answer:

```php
use Prism\Prism\Enums\StreamEventType;

foreach ($stream as $event) {
    match ($event->type()) {
        StreamEventType::ThinkingDelta => echo "[Thinking] " . $event->delta,
        StreamEventType::TextDelta => echo $event->delta,
        default => null,
    };
}
```

For complete streaming documentation including Vercel Data Protocol and WebSocket broadcasting, see [Streaming Output](/core-concepts/streaming-output).

## Documents

Anthropic supports PDF, text and markdown documents. Note that Anthropic uses vision to process PDFs under the hood, and consequently there are some limitations detailed in their [feature documentation](https://docs.anthropic.com/en/docs/build-with-claude/pdf-support).

See the [Documents](/input-modalities/documents.html) on how to get started using them.

Anthropic also supports "custom content documents", separately documented below, which are primarily for use with citations.

### Custom content documents

Custom content documents are primarily for use with citations (see below), if you need citations to reference your own chunking strategy.

```php
use Prism\Prism\Enums\Provider;
use Prism\Prism\Facades\Prism;
use Prism\Prism\ValueObjects\Messages\UserMessage;
use Prism\Prism\ValueObjects\Media\Document;

Prism::text()
    ->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
    ->withMessages([
        new UserMessage(
            content: "Is the grass green and the sky blue?",
            additionalContent: [
                Document::fromChunks(["The grass is green.", "Flamingos are pink.", "The sky is blue."])
            ]
        )
    ])
    ->asText();
```

## Citations

Prism supports [Anthropic's citations feature](https://docs.anthropic.com/en/docs/build-with-claude/citations) for both text and structured.

Please note that citations cannot be used with native structured output mode (the default). If you need citations with structured output, use tool calling mode via `withProviderOptions(['use_tool_calling' => true, 'citations' => true])`. Note however that citations with tool calling mode can produce unreliable output, so you should implement proper error handling.

## Code execution

Anthropic offers built-in code execution capabilities that allow your AI to run code in a secure environment. This is a provider tool that executes code using Anthropic's infrastructure. For more information about the difference between custom tools and provider tools, see [Tools & Function Calling](/core-concepts/tools-function-calling#provider-tools).

To enable code execution, you will first need to enable the beta feature.

Either in prism/config.php:

```php
        'anthropic' => [
            ...
            'anthropic_beta' => 'code-execution-2025-05-22',
        ],

```

Or in your env file (assuming config/prism.php reflects the default prism setup):

```
ANTHROPIC_BETA="code-execution-2025-05-22"
```

You may then use code execution as follows:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\ValueObjects\ProviderTool;

Prism::text()
    ->using('anthropic', 'claude-3-5-haiku-latest')
    ->withPrompt('Solve the equation 3x + 10 = 14.')
    ->withProviderTools([new ProviderTool(type: 'code_execution_20250522', name: 'code_execution')])
    ->asText();

```

### Enabling citations

Anthropic require citations to be enabled on all documents in a request. To enable them, using the `withProviderOptions()` method when building your request:

```php
use Prism\Prism\Enums\Provider;
use Prism\Prism\Facades\Prism;
use Prism\Prism\ValueObjects\Messages\UserMessage;
use Prism\Prism\ValueObjects\Media\Document;

$response = Prism::text()
    ->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
    ->withMessages([
        new UserMessage(
            content: "Is the grass green and the sky blue?",
            additionalContent: [
                Document::fromChunks(
                    chunks: ["The grass is green.", "Flamingos are pink.", "The sky is blue."],
                    title: 'The colours of nature',
                    context: 'The go-to textbook on the colours found in nature!'
                )
            ]
        )
    ])
    ->withProviderOptions(['citations' => true])
    ->asText();
```

### Accessing citations

You can access the chunked output with its citations via the additionalContent property on a response, which returns an array of `MessagePartWithCitations`s.

As a rough worked example, let's assume you want to implement footnotes. You'll need to loop through those chunks and (1) re-construct the message with links to the footnotes; and (2) build an array of footnotes to loop through in your frontend.

```php
use Prism\Prism\ValueObjects\MessagePartWithCitations;
use Prism\Prism\ValueObjects\Citation;

$messageChunks = $response->additionalContent['citations'];

$text = '';
$footnotes = [];

$footnoteId = 1;

/** @var MessagePartWithCitations $messageChunk  */
foreach ($messageChunks as $messageChunk) {
    $text .= $messageChunk->outputText;
    
    /** @var Citation $citation */
    foreach ($messageChunk->citations as $citation) {
        $footnotes[] = [
            'id' => $footnoteId,
            'document_title' => $citation->sourceTitle,
            'reference_start' => $citation->sourceStartIndex,
            'reference_end' => $citation->sourceEndIndex
        ];
    
        $text .= '<sup><a href="#footnote-'.$footnoteId.'">'.$footnoteId.'</a></sup>';
    
        $footnoteId++;
    }
}
```

Note that when using streaming, Anthropic does not stream citations in the same way. Instead, of building the context as above, yield text to the browser in the usual way and pair text up with the relevant footnote using the `citationIndex` on the text chunk's additionalContent parameter.

## Considerations

### Message Order

* Message order matters. Anthropic is strict about the message order being:

1. `UserMessage`
2. `AssistantMessage`
3. `UserMessage`

### Structured Output

Prism supports two approaches for structured output with Anthropic models:

#### Native Structured Outputs (Default)

Prism uses Anthropic's native structured outputs by default. This provides guaranteed schema compliance through constrained decoding — no beta header required.

```php
use Prism\Prism\Enums\Provider;
use Prism\Prism\Facades\Prism;
use Prism\Prism\Schema\ObjectSchema;
use Prism\Prism\Schema\StringSchema;

$response = Prism::structured()
    ->withSchema(new ObjectSchema(
        'weather_report',
        'Weather forecast with recommendations',
        [
            new StringSchema('forecast', 'The weather forecast'),
            new StringSchema('recommendation', 'Clothing recommendation')
        ],
        ['forecast', 'recommendation']
    ))
    ->using(Provider::Anthropic, 'claude-sonnet-4-5-20250929')
    ->withPrompt('What\'s the weather like and what should I wear?')
    ->asStructured();
```

**Benefits of native structured outputs:**

* **Always valid JSON**: No more parsing errors or malformed responses
* **Type safe**: Guaranteed field types and required fields

**Limitations:**

* Only available on Claude Sonnet 4.5+ and Claude Opus 4.1+
* Cannot be used with citations
* Some JSON Schema features are not supported (see [Schema Limitations](#schema-limitations))

#### Tool Calling Mode

For older models that don't support native structured outputs, or when dealing with complex content or non-English text that may contain quotes, you can use tool calling mode:

```php
use Prism\Prism\Enums\Provider;
use Prism\Prism\Facades\Prism;
use Prism\Prism\Schema\ObjectSchema;
use Prism\Prism\Schema\StringSchema;

$response = Prism::structured()
    ->withSchema(new ObjectSchema(
        'weather_report',
        'Weather forecast with recommendations',
        [
            new StringSchema('forecast', 'The weather forecast'),
            new StringSchema('recommendation', 'Clothing recommendation')
        ],
        ['forecast', 'recommendation']
    ))
    ->using(Provider::Anthropic, 'claude-3-5-sonnet-latest')
    ->withPrompt('What\'s the weather like and what should I wear?')
    ->withProviderOptions(['use_tool_calling' => true])
    ->asStructured();
```

**Benefits of tool calling mode:**

* More reliable JSON parsing, especially with quotes and special characters
* Better handling of non-English content (Chinese, Japanese, etc.)
* Reduced risk of malformed JSON responses
* Compatible with thinking mode

**Limitations:**

* Cannot be used with citations (citations are not supported in tool calling mode)
* Slightly more complex under the hood but identical API usage

#### Combining Custom Tools with Structured Output

You can combine custom tools with structured output to gather data before returning a structured response. This requires tool calling mode to be enabled:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Schema\ObjectSchema;
use Prism\Prism\Schema\StringSchema;
use Prism\Prism\Tool;

$schema = new ObjectSchema(
    'weather_analysis',
    'Analysis of weather conditions',
    [
        new StringSchema('summary', 'Summary of the weather'),
        new StringSchema('recommendation', 'Recommendation based on weather'),
    ],
    ['summary', 'recommendation']
);

$weatherTool = Tool::as('get_weather')
    ->for('Get current weather for a location')
    ->withStringParameter('location', 'The city and state')
    ->using(fn (string $location): string => "Weather in {$location}: 72°F, sunny");

$response = Prism::structured()
    ->using('anthropic', 'claude-3-5-sonnet-latest')
    ->withSchema($schema)
    ->withTools([$weatherTool])
    ->withMaxSteps(3)
    ->withProviderOptions(['use_tool_calling' => true]) // Required for Anthropic
    ->withPrompt('What is the weather in San Francisco and should I wear a coat?')
    ->asStructured();

// Access structured output
dump($response->structured);

// Access tool execution details
foreach ($response->toolCalls as $toolCall) {
    echo "Called: {$toolCall->name}\n";
}
```

> \[!IMPORTANT]
> When using custom tools with structured output on Anthropic, you must:
>
> * Set `use_tool_calling: true` in provider options
> * Set `maxSteps` to at least 2

For complete documentation on combining tools with structured output, see [Structured Output - Combining with Tools](/core-concepts/structured-output#combining-structured-output-with-tools).

### Strict Tool Use

You can enable strict validation for tool inputs, which guarantees that tool parameters exactly match your schema through constrained decoding.

To enable strict mode for a tool, use the `strict` provider option:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Tool;

$weatherTool = Tool::as('get_weather')
    ->for('Get current weather for a location')
    ->withStringParameter('location', 'The city and state')
    ->withProviderOptions(['strict' => true])
    ->using(fn (string $location): string => "Weather in {$location}: 72°F, sunny");

$response = Prism::text()
    ->using('anthropic', 'claude-sonnet-4-5-20250929')
    ->withTools([$weatherTool])
    ->withPrompt('What is the weather in San Francisco?')
    ->asText();
```

**Benefits of strict tool use:**

* Functions receive correctly-typed arguments every time
* No need to validate tool inputs
* Eliminates runtime errors from type mismatches
* Production-ready agents that work consistently

### Schema Limitations

When using native structured outputs, certain JSON Schema features are not supported by Anthropic's constrained decoding:

**Not Supported:**

* Recursive schemas
* Numerical constraints (`minimum`, `maximum`, `multipleOf`)
* String constraints (`minLength`, `maxLength`)
* Complex regex patterns (lookahead/lookbehind, backreferences)
* External `$ref` definitions

**Supported:**

* All basic types (object, array, string, integer, number, boolean, null)
* `enum` for simple types (strings, numbers, booleans)
* `anyOf` and `allOf` (with limitations)
* `required` and `additionalProperties: false`
* String formats (`date-time`, `email`, `uri`, `uuid`, etc.)

If you use an unsupported feature, Anthropic will return a 400 error with details. For more information, see [Anthropic's structured outputs documentation](https://docs.anthropic.com/en/api/structured-outputs).

## Limitations

### Messages

Most providers' API include system messages in the messages array with a "system" role. Anthropic does not support the system role, and instead has a "system" property, separate from messages.

Therefore, for Anthropic we:

* Filter all `SystemMessage`s out, omitting them from messages.
* Always submit the prompt defined with `->withSystemPrompt()` at the top of the system prompts array.
* Move all `SystemMessage`s to the system prompts array in the order they were declared.

### Images

Does not support `Image::fromURL`

---

---
url: /input-modalities/audio.md
---
# Audio

Prism supports including audio files in your messages for advanced analysis with supported providers like Gemini.

See the [provider support table](/getting-started/introduction.html#provider-support) to check whether Prism supports your chosen provider.

Note however that provider support may differ by model. If you receive error messages with a provider that Prism indicates is supported, check the provider's documentation as to whether the model you are using supports audio files.

::: tip
For other input modalities like videos and images, see their respective documentation pages:

* [Video documentation](/input-modalities/video.html)
* [Images documentation](/input-modalities/images.html)
  :::

## Getting started

To add an audio file to your prompt, use the `withPrompt` method with an `Audio` value object:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\ValueObjects\Media\Audio;

// From a local path
$response = Prism::text()
    ->using(Provider::Gemini, 'gemini-1.5-flash')
    ->withPrompt(
        "What's in this audio?",
        [Audio::fromLocalPath(path: '/path/to/audio.mp3')]
    )
    ->asText();

// From a path on a storage disk
$response = Prism::text()
    ->using(Provider::Gemini, 'gemini-1.5-flash')
    ->withPrompt(
        "What's in this audio?",
        [Audio::fromStoragePath(
            path: '/path/to/audio.mp3',
            diskName: 'my-disk' // optional - omit/null for default disk
        )]
    )
    ->asText();

// From a URL
$response = Prism::text()
    ->using(Provider::Gemini, 'gemini-1.5-flash')
    ->withPrompt(
        'Analyze this audio:',
        [Audio::fromUrl(url: 'https://example.com/audio.mp3')]
    )
    ->asText();

// From base64
$response = Prism::text()
    ->using(Provider::Gemini, 'gemini-1.5-flash')
    ->withPrompt(
        'Analyze this audio:',
        [Audio::fromBase64(
            base64: base64_encode(file_get_contents('/path/to/audio.mp3')),
            mimeType: 'audio/mpeg'
        )]
    )
    ->asText();

// From raw content
$response = Prism::text()
    ->using(Provider::Gemini, 'gemini-1.5-flash')
    ->withPrompt(
        'Analyze this audio:',
        [Audio::fromRawContent(
            rawContent: file_get_contents('/path/to/audio.mp3'),
            mimeType: 'audio/mpeg'
        )]
    )
    ->asText();
```

## Alternative: Using withMessages

You can also include audio files using the message-based approach:

```php
use Prism\Prism\ValueObjects\Messages\UserMessage;
use Prism\Prism\ValueObjects\Media\Audio;

$message = new UserMessage(
    "What's in this audio?",
    [Audio::fromLocalPath(path: '/path/to/audio.mp3')]
);

$response = Prism::text()
    ->using(Provider::Gemini, 'gemini-1.5-flash')
    ->withMessages([$message])
    ->asText();
```

## Supported Audio Types

Prism supports a variety of audio formats, including:

* MP3 (audio/mpeg)
* WAV (audio/x-wav, audio/wav)
* AAC (audio/aac)
* FLAC (audio/flac)

The specific supported formats depend on the provider. Gemini is currently the main provider with comprehensive audio analysis capabilities. Check the provider's documentation for a complete list of supported formats.

## Transfer mediums

Providers are not consistent in their support of sending raw contents, base64 and/or URLs.

Prism tries to smooth over these rough edges, but its not always possible.

### Supported conversions

* Where a provider does not support URLs: Prism will fetch the URL and use base64 or rawContent.
* Where you provide a file, base64 or rawContent: Prism will switch between base64 and rawContent depending on what the provider accepts.

### Limitations

* Where a provider only supports URLs: if you provide a file path, raw contents or base64, for security reasons Prism does not create a URL for you and your request will fail.

---

---
url: /core-concepts/audio.md
---
# Audio Processing

Transform text into speech and speech into text using AI-powered audio models. Prism provides a unified API for audio processing across different providers, enabling both text-to-speech (TTS) and speech-to-text (STT) functionality.

## Getting Started

### Text-to-Speech

Convert text into natural-sounding speech:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;

$response = Prism::audio()
    ->using(Provider::OpenAI, 'tts-1')
    ->withInput('Hello, this is a test of text-to-speech functionality.')
    ->withVoice('alloy')
    ->asAudio();

$audio = $response->audio;
if ($audio->hasBase64()) {
    file_put_contents('output.mp3', base64_decode($audio->base64));
    echo "Audio saved as: output.mp3";
}
```

### Speech-to-Text

Convert audio files into text transcriptions:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\ValueObjects\Media\Audio;

$audioFile = Audio::fromPath('/path/to/audio.mp3');

$response = Prism::audio()
    ->using(Provider::OpenAI, 'whisper-1')
    ->withInput($audioFile)
    ->asText();

echo "Transcription: " . $response->text;
```

## Provider Support

Currently, Prism supports audio processing through:

* **OpenAI**: TTS-1, TTS-1-HD (text-to-speech) and Whisper-1 (speech-to-text)
* **Groq**: PlayAI TTS models (text-to-speech) and Whisper Large V3 models (speech-to-text)

Additional providers will be added in future releases as the ecosystem evolves.

## Basic Usage

### Simple Text-to-Speech

Generate speech from text input:

```php
$response = Prism::audio()
    ->using('openai', 'tts-1')
    ->withInput('Welcome to our application!')
    ->withVoice('alloy')
    ->asAudio();

$audio = $response->audio;
echo "Audio type: " . $audio->getMimeType(); // audio/mpeg
echo "Has audio data: " . ($audio->hasBase64() ? 'Yes' : 'No');
```

### Simple Speech-to-Text

Transcribe audio files to text:

```php
use Prism\Prism\ValueObjects\Media\Audio;

// From file path
$audioFile = Audio::fromPath('/path/to/recording.wav');

// From URL
$audioFile = Audio::fromUrl('https://example.com/audio.mp3');

// From base64 data
$audioFile = Audio::fromBase64($base64AudioData, 'audio/wav');

$response = Prism::audio()
    ->using('openai', 'whisper-1')
    ->withInput($audioFile)
    ->asText();

// Access the transcription
echo $response->text;
```

## Working with Audio Files

### Creating Audio Objects

The `Audio` class provides several ways to work with audio files:

```php
use Prism\Prism\ValueObjects\Media\Audio;

// From local file
$audio = Audio::fromPath('/path/to/audio.mp3');

// From remote URL
$audio = Audio::fromUrl('https://example.com/speech.wav');

// From base64 encoded data
$audio = Audio::fromBase64($base64Data, 'audio/mpeg');

// From raw binary content
$audio = Audio::fromContent($binaryData, 'audio/wav');
```

### Audio Properties

Access audio file information:

```php
$audio = Audio::fromPath('/path/to/audio.mp3');

echo "MIME type: " . $audio->mimeType();
echo "Has local path: " . ($audio->hasLocalPath() ? 'Yes' : 'No');
echo "File size: " . $audio->size() . " bytes";
```

## Working with Responses

### Text-to-Speech Responses

```php
$response = Prism::audio()
    ->using('openai', 'tts-1-hd')
    ->withInput('This is high-quality text-to-speech.')
    ->withVoice('nova')
    ->asAudio();

// Access the generated audio
$audio = $response->audio;

if ($audio->hasBase64()) {
    // Save to file
    $audioData = base64_decode($audio->base64);
    file_put_contents('speech.mp3', $audioData);
    
    // Get MIME type
    echo "Content type: " . $audio->getMimeType();
}

// Access additional response data
foreach ($response->additionalContent as $key => $value) {
    echo "{$key}: {$value}\n";
}
```

### Speech-to-Text Responses

```php
$audioFile = Audio::fromPath('/path/to/speech.mp3');

$response = Prism::audio()
    ->using('openai', 'whisper-1')
    ->withInput($audioFile)
    ->asText();

// Access the transcription
$text = $response->text;
echo "Transcription: " . $text;

// Check token usage
if ($response->usage) {
    echo "Prompt tokens: " . $response->usage->promptTokens;
    echo "Completion tokens: " . $response->usage->completionTokens;
}

// Access raw response data
print_r($response->additionalContent);
```

## Voice Selection

Prism provides a dedicated `withVoice()` method for selecting voices in text-to-speech, making voice selection a first-class citizen in the API:

```php
$response = Prism::audio()
    ->using('openai', 'tts-1')
    ->withInput('Hello, how are you today?')
    ->withVoice('alloy')  // Voice options vary by provider
    ->asAudio();
```

## Provider-Specific Options

While Prism provides a consistent API, you can access provider-specific features using the `withProviderOptions()` method.

### OpenAI Text-to-Speech Options

Customize format, speed, and other options:

```php
$response = Prism::audio()
    ->using('openai', 'tts-1')
    ->withInput('Hello, how are you today?')
    ->withVoice('nova')
    ->withProviderOptions([
        'response_format' => 'mp3',           // mp3, opus, aac, flac, wav, pcm
        'speed' => 1.0,                       // 0.25 to 4.0
    ])
    ->asAudio();
```

### OpenAI Speech-to-Text Options

Configure transcription settings:

```php
$audioFile = Audio::fromPath('/path/to/multilingual.mp3');

$response = Prism::audio()
    ->using('openai', 'whisper-1')
    ->withInput($audioFile)
    ->withProviderOptions([
        'language' => 'en',
        'prompt' => 'Previous context...'
    ])
    ->asText();
```

### Response Formats

Different response formats provide varying levels of detail:

```php
// Verbose JSON format includes timestamps and confidence scores
$response = Prism::audio()
    ->using('openai', 'whisper-1')
    ->withInput($audioFile)
    ->withProviderOptions([
        'response_format' => 'verbose_json',
    ])
    ->asText();

// Access additional metadata
$metadata = $response->additionalContent;
if (isset($metadata['segments'])) {
    foreach ($metadata['segments'] as $segment) {
        echo "Segment: " . $segment['text'] . "\n";
        echo "Start: " . $segment['start'] . "s\n";
        echo "End: " . $segment['end'] . "s\n";
    }
}
```

## Advanced Usage

Audio can be integrated into multi-modal conversations:

```php
use Prism\Prism\ValueObjects\Messages\UserMessage;
use Prism\Prism\ValueObjects\Media\Audio;
use Prism\Prism\ValueObjects\Media\Text;

$audioFile = Audio::fromPath('/path/to/question.mp3');

// First transcribe the audio
$transcription = Prism::audio()
    ->using('openai', 'whisper-1')
    ->withInput($audioFile)
    ->asText();

// Then use in a text conversation
$response = Prism::text()
    ->using('openai', 'gpt-4')
    ->withMessages([
        new UserMessage('', [
            new Text('User asked: '),
            new Text($transcription->text),
            new Text(' - Please provide a detailed response.')
        ])
    ])
    ->asText();

// Convert response back to speech
$speechResponse = Prism::audio()
    ->using('openai', 'tts-1')
    ->withInput($response->text)
    ->withVoice('alloy')
    ->asAudio();
```

## Configuration Options

### Client Configuration

Configure HTTP client options for audio processing:

```php
$response = Prism::audio()
    ->using('openai', 'tts-1')
    ->withInput('This might take a while to process.')
    ->withClientOptions([
        'timeout' => 60,           // Increase timeout for large files
        'connect_timeout' => 10,   // Connection timeout
    ])
    ->withClientRetry(3, 1000)     // Retry 3 times with 1s delay
    ->asAudio();
```

### Provider Configuration

Override provider configuration for multi-tenant applications:

```php
$customConfig = [
    'api_key' => 'user-specific-api-key',
    'organization' => 'user-org-id',
];

$response = Prism::audio()
    ->using('openai', 'whisper-1')
    ->usingProviderConfig($customConfig)
    ->withInput($audioFile)
    ->asText();
```

## Error Handling

Handle potential errors in audio processing:

```php
use Prism\Prism\Exceptions\PrismException;
use Throwable;

try {
    $response = Prism::audio()
        ->using('openai', 'tts-1')
        ->withInput('Text to convert to speech')
        ->withVoice('alloy')
        ->asAudio();
        
    // Process successful response
    file_put_contents('output.mp3', base64_decode($response->audio->base64));
    
} catch (PrismException $e) {
    Log::error('Audio processing failed:', ['error' => $e->getMessage()]);
    // Handle Prism-specific errors
} catch (Throwable $e) {
    Log::error('General error:', ['error' => $e->getMessage()]);
    // Handle any other errors
}
```

## Testing

Prism provides convenient fakes for testing audio functionality:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Testing\PrismFake;
use Prism\Prism\Audio\AudioResponse;
use Prism\Prism\Audio\TextResponse;
use Prism\Prism\ValueObjects\GeneratedAudio;

test('can generate text-to-speech', function () {
    $fakeAudio = new AudioResponse(
        audio: new GeneratedAudio(
            base64: base64_encode('fake-audio-data'),
            type: 'audio/mpeg'
        )
    );
    
    Prism::fake([$fakeAudio]);

    $response = Prism::audio()
        ->using('openai', 'tts-1')
        ->withInput('Test audio generation')
        ->withVoice('alloy')
        ->asAudio();

    expect($response->audio->hasBase64())->toBeTrue();
    expect($response->audio->getMimeType())->toBe('audio/mpeg');
});

test('can transcribe speech-to-text', function () {
    $fakeTranscription = new TextResponse(
        text: 'This is a fake transcription'
    );
    
    Prism::fake([$fakeTranscription]);

    $audioFile = Audio::fromPath('/fake/path/test.mp3');
    
    $response = Prism::audio()
        ->using('openai', 'whisper-1')
        ->withInput($audioFile)
        ->asText();

    expect($response->text)->toBe('This is a fake transcription');
});
```

---

---
url: /getting-started/configuration.md
---
# Configuration

Prism's flexible configuration allows you to easily set up and switch between different AI providers. Let's dive into how you can configure Prism to work with your preferred providers.

## Configuration File

After installation, you'll find the Prism configuration file at `config/prism.php`. If you haven't published it yet, you can do so with:

```bash
php artisan vendor:publish --tag=prism-config
```

Let's break down the key sections of this configuration file:

```php
return [
    'prism_server' => [
        'enabled' => env('PRISM_SERVER_ENABLED', false),
    ],
    'request_timeout' => env('PRISM_REQUEST_TIMEOUT', 30),
    'providers' => [
        // Provider configurations here
    ],
];
```

## Request Timeout

Prism includes a global request timeout that applies to all provider HTTP requests. By default, requests will timeout after 30 seconds. You can adjust this value to accommodate longer-running operations like complex generations or large context windows:

```php
'request_timeout' => env('PRISM_REQUEST_TIMEOUT', 30),
```

This timeout applies to both the connection and the overall request duration. If you're working with providers that need more time for complex operations, increase this value accordingly.

Request timeouts can also be set by using the `withClientOptions()` method.

```php
Prism::text()
  ->withClientOptions(['timeout' => 120]) // [!code focus]
  ->asText()
```

## Provider Configuration

Prism uses a straightforward provider configuration system that lets you set up multiple AI providers in one place. Each provider has its own section in the configuration file where you can specify:

* API credentials
* Base URLs (useful for self-hosted instances or custom endpoints)
* Other Provider-specific settings

Here's a general template for how providers are configured:

```php
'providers' => [
    'provider-name' => [
        'api_key' => env('PROVIDER_API_KEY', ''),
        'url' => env('PROVIDER_URL', 'https://api.provider.com'),
        // Other provider-specific settings
    ],
],
```

## Environment Variables

Prism follows Laravel's environment configuration best practices. All sensitive or environment-specific values should be stored in your `.env` file. Here's how it works:

1. Each provider's configuration pulls values from environment variables
2. Default values are provided as fallbacks
3. Environment variables follow a predictable naming pattern:
   * API keys: `PROVIDER_API_KEY`
   * URLs: `PROVIDER_URL`
   * Other settings: `PROVIDER_SETTING_NAME`

For example:

```shell
# Prism Server Configuration
PRISM_SERVER_ENABLED=true

# Provider Configuration
PROVIDER_API_KEY=your-api-key-here
PROVIDER_URL=https://custom-endpoint.com

```

> \[!NOTE]
> Remember to always refer to your chosen provider's documentation pages for the most up-to-date configuration options and requirements specific to that provider.

## Overriding config in your code

You can override config in your code in two ways:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;

// Via the third parameter of `using()`
$response = Prism::text()
    ->using(Provider::OpenAI, 'gpt-4o', [
        'url' => 'new-base-url'
    ])
    ->withPrompt('Explain quantum computing.')
    ->asText();

// Or via `usingProviderConfig()` (note that this will re-resolve the provider).
$response = Prism::text()
    ->using(Provider::OpenAI, 'gpt-4o')
    ->usingProviderConfig([
        'url' => 'new-base-url'
    ])
    ->withPrompt('Explain quantum computing.')
    ->asText();
```

---

---
url: /advanced/custom-providers.md
---
# Custom Providers

Want to add support for a new AI provider in Prism? This guide will walk you through creating and registering your own custom provider implementation.

## Building Your Provider

All providers must extend the `Prism\Prism\Providers\Provider` abstract class. This base class provides default implementations for all required methods, throwing exceptions for unsupported actions.

When creating your provider, you'll only need to override the methods for the features you want to support:

* `text()` - For text generation
* `structured()` - For structured output generation
* `embeddings()` - For creating embeddings
* `images()` - For image generation
* `stream()` - For streaming text responses

Here's what that looks like in practice:

```php
namespace App\Prism\Providers;

use Prism\Prism\Providers\Provider;
use Prism\Prism\Text\Request as TextRequest;
use Prism\Prism\Text\Response as TextResponse;

class MyCustomProvider extends Provider
{
    public function __construct(
        protected string $apiKey,
    ) {}

    public function text(TextRequest $request): TextResponse
    {
        // Your text generation logic here
        // Make API calls, process the response, and return a TextResponse
    }

    // Only override the methods you need!
}
```

## Registration Process

Once you've created your provider, you'll need to register it with Prism. Let's add it to a service provider:

```php
namespace App\Providers;

use App\Prism\Providers\MyCustomProvider;
use Illuminate\Support\ServiceProvider;

class AppServiceProvider extends ServiceProvider
{
    public function boot(): void
    {
        $this->app['prism-manager']->extend('my-custom-provider', function ($app, $config) {
            return new MyCustomProvider(
                apiKey: $config['api_key'] ?? '',
            );
        });
    }
}
```

Next, add your provider configuration to `config/prism.php`:

```php
return [
    'providers' => [
        // ... other providers ...
        'my-custom-provider' => [
            'api_key' => env('MY_CUSTOM_PROVIDER_API_KEY'),
        ],
    ],
];
```

That's it! You're ready to use your custom provider:

```php
use Prism\Prism\Facades\Prism;

$response = Prism::text()
    ->using('my-custom-provider', 'model-name')
    ->withPrompt('Hello, custom AI!')
    ->asText();
```

## Custom Error Handling

Your provider inherits a default `handleRequestException` method that handles common HTTP status codes. You can override this method to handle provider-specific errors or add custom logic:

```php
use Illuminate\Http\Client\RequestException;
use Prism\Prism\Exceptions\PrismException;

class MyCustomProvider extends Provider
{
    // ... other methods ...

    public function handleRequestException(string $model, RequestException $e): never
    {
        // Handle provider-specific error codes
        match ($e->response->getStatusCode()) {
            429 => throw PrismRateLimitedException::make(
                rateLimits: $this->processRateLimits($e->response),
                retryAfter: $e->response->header('retry-after') === ''
                    ? null
                    : (int) $e->response->header('retry-after'),
            ),
            default => parent::handleRequestException($model, $e),
        };
    }
}
```

The method must throw an exception (return type `never`). If you don't handle a specific status code, make sure to call the parent method to maintain the default error handling.

## Best Practices

* **Start small**: Begin by implementing just the methods you need. You don't have to support every feature right away.
* **Handle errors gracefully**: Leverage the inherited error handling or override `handleRequestException()` for provider-specific errors (see Custom Error Handling section above).
* **Test thoroughly**: Make sure to test your provider with various inputs and edge cases.
* **Document your models**: Let users know which models your provider supports and any special parameters they can use.

> \[!TIP]
> Looking at existing provider implementations in Prism's source code can give you great insights into best practices and patterns to follow.

---

---
url: /providers/deepseek.md
---
# DeepSeek

## Configuration

```php
'deepseek' => [
    'api_key' => env('DEEPSEEK_API_KEY', ''),
    'url' => env('DEEPSEEK_URL', 'https://api.deepseek.com/v1')
]
```

## Provider-specific options

## Streaming

DeepSeek supports streaming responses in real-time. All standard streaming methods are supported:

```php
return Prism::text()
    ->using('deepseek', 'deepseek-chat')
    ->withPrompt(request('message'))
    ->asEventStreamResponse();
```

For complete streaming documentation, see [Streaming Output](/core-concepts/streaming-output).

## Limitations

### Embeddings

Does not support embeddings.

### Tool Choice

Does not support tool choice.

### Images

Does not support images.

---

---
url: /input-modalities/documents.md
---
# Documents

Prism supports including documents in your messages with some providers.

See the [provider support table](/getting-started/introduction.html#provider-support) to check whether Prism supports your chosen provider.

Note however that provider support may differ by model. If you receive error messages with a provider that Prism indicates is supported, check the provider's documentation as to whether the model you are using supports documents.

## Supported file types

> \[!TIP]
> If provider interoperability is important to your app, we recommend converting documents to markdown.

Please check provider documentation for supported file/mime types, as support differs widely.

The most supported file types are pdf and text/plain (which may include markdown).

## Transfer mediums

> \[!TIP]
> If provider interoperability is important to your app, we recommend using rawContent or base64.

Providers are not consistent in their support of sending file raw contents, base64 and/or URLs.

Prism tries to smooth over these rough edges, but its not always possible.

### Supported conversions

* Where a provider does not support URLs: Prism will fetch the URL and use base64 or rawContent.
* Where you provide a file, base64 or rawContent: Prism will switch between base64 and rawContent depending on what the provider accepts.

### Limitations

* Where a provider only supports URLs: if you provide a file path, raw contents, base64 or chunks, for security reasons Prism does not create a URL for you and your request will fail.
* Chunks cannot be passed between providers, as they could be in different formats (however, currently only Anthropic supports them).

## Getting started

To add a document to your prompt, use the `withPrompt` method with a `Document` value object:

```php
use Prism\Prism\Enums\Provider;
use Prism\Prism\Facades\Prism;
use Prism\Prism\ValueObjects\Media\Document;

// From a local path
$response = Prism::text()
    ->using('my-provider', 'my-model')
    ->withPrompt(
        'Analyze this document',
        [Document::fromLocalPath(
            path: 'tests/Fixtures/test-pdf.pdf',
            title: 'My document title' // optional
        )]
    )
    ->asText();

// From a storage path
$response = Prism::text()
    ->using('my-provider', 'my-model')
    ->withPrompt(
        'Summarize this document',
        [Document::fromStoragePath(
            path: 'mystoragepath/file.pdf',
            diskName: 'my-disk', // optional - omit/null for default disk
            title: 'My document title' // optional
        )]
    )
    ->asText();

// From base64
$response = Prism::text()
    ->using('my-provider', 'my-model')
    ->withPrompt(
        'Extract key points from this document',
        [Document::fromBase64(
            base64: $baseFromDB,
            mimeType: 'optional/mimetype', // optional
            title: 'My document title' // optional
        )]
    )
    ->asText();

// From raw content
$response = Prism::text()
    ->using('my-provider', 'my-model')
    ->withPrompt(
        'Review this document',
        [Document::fromRawContent(
            rawContent: $rawContent,
            mimeType: 'optional/mimetype', // optional
            title: 'My document title' // optional
        )]
    )
    ->asText();

// From a text string
$response = Prism::text()
    ->using('my-provider', 'my-model')
    ->withPrompt(
        'Process this text document',
        [Document::fromText(
            text: 'Hello world!',
            title: 'My document title' // optional
        )]
    )
    ->asText();

// From an URL
$response = Prism::text()
    ->using('my-provider', 'my-model')
    ->withPrompt(
        'Analyze this document from URL',
        [Document::fromUrl(
            url: 'https://example.com/test-pdf.pdf',
            title: 'My document title' // optional
        )]
    )
    ->asText();

// From chunks
$response = Prism::text()
    ->using('my-provider', 'my-model')
    ->withPrompt(
        'Process this chunked document',
        [Document::fromChunks(
            chunks: [
                'chunk one',
                'chunk two'
            ],
            title: 'My document title' // optional
        )]
    )
    ->asText();

// From a provider file ID
$response = Prism::text()
    ->using('my-provider', 'my-model')
    ->withPrompt(
        'Analyze this document from provider file',
        [Document::fromFileId(
            fileId: 'my-provider-file-id'
        )]
    )
    ->asText();
```

## Alternative: Using withMessages

You can also include documents using the message-based approach:

```php
use Prism\Prism\ValueObjects\Messages\UserMessage;
use Prism\Prism\ValueObjects\Media\Document;

$message = new UserMessage(
    'Analyze this document',
    [Document::fromLocalPath(
        path: 'tests/Fixtures/test-pdf.pdf',
        title: 'My document title' // optional
    )]
);

$response = Prism::text()
    ->using('my-provider', 'my-model')
    ->withMessages([$message])
    ->asText();
```

Or, if using a provider file\_id - use fromFileId:

```php
use Prism\Prism\Enums\Provider;
use Prism\Prism\Facades\Prism;
use Prism\Prism\ValueObjects\Media\Document;

$response = Prism::text()
    ->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
    ->withPrompt(
        'Analyze this OpenAI file',
        [Document::fromFileId(
            fileId: 'file-lsfgSXyV2xEb8gw8fYjXU6'
        )]
    )
    ->asText();
```

---

---
url: /providers/elevenlabs.md
---
# ElevenLabs

## Configuration

```php
'elevenlabs' => [
    'api_key' => env('ELEVENLABS_API_KEY', ''),
    'url' => env('ELEVENLABS_URL', 'https://api.elevenlabs.io/v1/'),
]
```

## Speech-to-Text

ElevenLabs provides speech-to-text through their Scribe model with support for diarization and audio event tagging.

### Basic Usage

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\ValueObjects\Media\Audio;

$audioFile = Audio::fromPath('/path/to/recording.mp3');

$response = Prism::audio()
    ->using('elevenlabs', 'scribe_v1')
    ->withInput($audioFile)
    ->asText();
```

## Provider-specific Options

### Language Detection

```php
$response = Prism::audio()
    ->using('elevenlabs', 'scribe_v1')
    ->withInput($audioFile)
    ->withProviderOptions([
        'language_code' => 'en',
    ])
    ->asText();
```

### Speaker Diarization

```php
$response = Prism::audio()
    ->using('elevenlabs', 'scribe_v1')
    ->withInput($audioFile)
    ->withProviderOptions([
        'diarize' => true,
        'num_speakers' => 2,
    ])
    ->asText();
```

### Audio Event Tagging

```php
$response = Prism::audio()
    ->using('elevenlabs', 'scribe_v1')
    ->withInput($audioFile)
    ->withProviderOptions([
        'tag_audio_events' => true,
    ])
    ->asText();
```

## Limitations

* Text-to-speech is not yet implemented

---

---
url: /core-concepts/embeddings.md
---
# Embeddings

Transform your content into powerful vector representations! Embeddings let you add semantic search, recommendation systems, and other advanced features to your applications - whether you're working with text, images, audio, video, or documents.

## Quick Start

Here's how to generate an embedding with just a few lines of code:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;

$response = Prism::embeddings()
    ->using(Provider::OpenAI, 'text-embedding-3-large')
    ->fromInput('Your text goes here')
    ->asEmbeddings();

// Get your embeddings vector
$embeddings = $response->embeddings[0]->embedding;

// Check token usage
echo $response->usage->tokens;
```

## Generating multiple embeddings

You can generate multiple embeddings at once with providers that support batch embeddings:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;

$response = Prism::embeddings()
    ->using(Provider::OpenAI, 'text-embedding-3-large')
    // First embedding
    ->fromInput('Your text goes here')
    // Second embedding
    ->fromInput('Your second text goes here')
    // Third and fourth embeddings
    ->fromArray([
        'Third',
        'Fourth'
    ])
    ->asEmbeddings();

/** @var Embedding $embedding */
foreach ($embeddings as $embedding) {
    // Do something with your embeddings
    $embedding->embedding;
}

// Check token usage
echo $response->usage->tokens;
```

## Input Methods

You've got two convenient ways to feed text into the embeddings generator:

### Direct Text Input

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;

$response = Prism::embeddings()
    ->using(Provider::OpenAI, 'text-embedding-3-large')
    ->fromInput('Analyze this text')
    ->asEmbeddings();
```

### From File

Need to analyze a larger document? No problem:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;

$response = Prism::embeddings()
    ->using(Provider::OpenAI, 'text-embedding-3-large')
    ->fromFile('/path/to/your/document.txt')
    ->asEmbeddings();
```

> \[!NOTE]
> Make sure your file exists and is readable. The generator will throw a helpful `PrismException` if there's any issue accessing the file.

## Multimodal Embeddings

Some providers support multimodal embeddings, enabling powerful use cases like visual similarity search, cross-modal retrieval, and mixed media retrieval. Prism makes it easy to generate embeddings from images, audio, video, and documents using the same fluent API.

> \[!IMPORTANT]
> Multimodal embeddings require a provider and model that supports the input modalities you send. Check your provider's documentation to confirm support for images, audio, video, documents, and grouped content.

### Single Image

Generate an embedding from a single image:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\ValueObjects\Media\Image;

$response = Prism::embeddings()
    ->using('provider', 'model')
    ->fromImage(Image::fromLocalPath('/path/to/product.jpg'))
    ->asEmbeddings();

$embedding = $response->embeddings[0]->embedding;
```

### Multiple Images

Process multiple images in a single request:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\ValueObjects\Media\Image;

$response = Prism::embeddings()
    ->using('provider', 'model')
    ->fromImages([
        Image::fromLocalPath('/path/to/image1.jpg'),
        Image::fromUrl('https://example.com/image2.png'),
    ])
    ->asEmbeddings();

foreach ($response->embeddings as $embedding) {
    // Process each image embedding
    $vector = $embedding->embedding;
}
```

### Audio, Video, and Documents

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\ValueObjects\Media\Audio;
use Prism\Prism\ValueObjects\Media\Document;
use Prism\Prism\ValueObjects\Media\Video;

$response = Prism::embeddings()
    ->using('provider', 'model')
    ->fromAudio(Audio::fromLocalPath('/path/to/sample.mp3'))
    ->fromVideo(Video::fromLocalPath('/path/to/sample.mp4'))
    ->fromDocument(Document::fromLocalPath('/path/to/report.pdf'))
    ->asEmbeddings();
```

### Grouped Multimodal Content

Use `fromContent()` when you want a single embedding generated from multiple parts within the same content entry:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\ValueObjects\Media\Image;

$response = Prism::embeddings()
    ->using('provider', 'model')
    ->fromContent([
        'Find similar products in red',
        Image::fromBase64($productImage, 'image/png'),
    ])
    ->asEmbeddings();
```

Use `fromContents()` when you want multiple embeddings in a single request:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\ValueObjects\Media\Image;

$response = Prism::embeddings()
    ->using('provider', 'model')
    ->fromContents([
        ['The dog is cute'],
        [Image::fromLocalPath('/path/to/dog.png')],
    ])
    ->asEmbeddings();
```

You can still chain `fromInput()` and `fromImage()` in any order. Each chained call creates a separate content entry.

> \[!TIP]
> Prism media value objects support multiple input sources. See the [Images documentation](/input-modalities/images.html) and related modality guides for details.

## Common Settings

Just like with text generation, you can fine-tune your embeddings requests:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;

$response = Prism::embeddings()
    ->using(Provider::OpenAI, 'text-embedding-3-large')
    ->fromInput('Your text here')
    ->withClientOptions(['timeout' => 30]) // Adjust request timeout
    ->withClientRetry(3, 100) // Add automatic retries
    ->asEmbeddings();
```

## Response Handling

The embeddings response gives you everything you need:

```php
namespace Prism\Prism\ValueObjects\Embedding;

// Get an array of Embedding value objects
$embeddings = $response->embeddings;

// Just get first embedding
$firstVectorSet = $embeddings[0]->embedding;

// Loop over all embeddings
/** @var Embedding $embedding */
foreach ($embeddings as $embedding) {
    $vectorSet = $embedding->embedding;
}

// Check token usage
$tokenCount = $response->usage->tokens;
```

## Error Handling

Always handle potential errors gracefully:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\Exceptions\PrismException;

try {
    $response = Prism::embeddings()
        ->using(Provider::OpenAI, 'text-embedding-3-large')
        ->fromInput('Your text here')
        ->asEmbeddings();
} catch (PrismException $e) {
    Log::error('Embeddings generation failed:', [
        'error' => $e->getMessage()
    ]);
}
```

## Pro Tips

**Vector Storage**: Consider using a vector database like Milvus, Qdrant, or pgvector to store and query your embeddings efficiently.

**Text Preprocessing**: For best results, clean and normalize your text before generating embeddings. This might include:

* Removing unnecessary whitespace
* Converting to lowercase
* Removing special characters
* Handling Unicode normalization

> \[!IMPORTANT]
> Different providers and models produce vectors of different dimensions. Always check your provider's documentation for specific details about the embedding model you're using.

---

---
url: /advanced/error-handling.md
---
# Error handling

By default, Prism throws a `PrismException` for Prism errors, or a `PrismServerException` for Prism Server errors.

For production use cases, you may find yourself needing to catch Exceptions more granularly, for instance to provide more useful error messages to users or to implement failover or retry logic.

Prism has begun rolling out more specific exceptions as use cases arise.

## Provider agnostic exceptions

* `PrismStructuredDecodingException` where a provider has returned invalid JSON for a structured request.

## Exceptions based on provider feedback

Prism currently supports three exceptions based on provider feedback:

* `PrismRateLimitedException` where you have hit a rate limit or quota (see [Handling rate limits](/advanced/rate-limits.html) for more info).
* `PrismProviderOverloadedException` where the provider is unable to fulfil your request due to capacity issues.
* `PrismRequestTooLargeException` where your request is too large.

However, as providers all handle errors differently, support is being rolled out incrementally. If you'd like to make your first contribution, adding one or more of these exceptions for a provider would make a great first contribution. If you'd like to discuss, start an issue on Github, or just jump straight into a pull request.

---

---
url: /providers/gemini.md
---
# Gemini

## Configuration

```php
'gemini' => [
    'api_key' => env('GEMINI_API_KEY', ''),
    'url' => env('GEMINI_URL', 'https://generativelanguage.googleapis.com/v1beta/models'),
],
```

## Search grounding

Google Gemini offers built-in search grounding capabilities that allow your AI to search the web for real-time information. This is a provider tool that uses Google's search infrastructure. For more information about the difference between custom tools and provider tools, see [Tools & Function Calling](/core-concepts/tools-function-calling#provider-tools).

You may enable Google search grounding on text requests using withProviderTools:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\ValueObjects\ProviderTool;

$response = Prism::text()
    ->using(Provider::Gemini, 'gemini-2.0-flash')
    ->withPrompt('What is the stock price of Google right now?')
    // Enable search grounding
    ->withProviderTools([
            new ProviderTool('google_search')
        ])
    ->asText();
```

If you use search groundings, Google require you meet certain [display requirements](https://ai.google.dev/gemini-api/docs/grounding/search-suggestions).

The data you need to meet these display requirements, and to build e.g. footnote functionality will be saved to the response's `additionalContent` property.

```php
// The Google supplied and styled widget to click through to results.
$response->additionalContent['searchEntryPoint'];

// The search queries made by the model
$response->additionalContent['searchQueries'];

// The citations data is available as an array of MessagePartWithCitations
$response->additionalContent['citations'];
```

`citations` is an array of `MessagePartWithCitations`, which you can use to build up footnotes as follows:

```php
use Prism\Prism\ValueObjects\MessagePartWithCitations;
use Prism\Prism\ValueObjects\Citation;

$text = '';
$footnotes = [];

$footnoteId = 1;

/** @var MessagePartWithCitations $part */
foreach ($response->additionalContent['citations'] as $part) {
    $text .= $part->outputText;
    
    /** @var Citation $citation */
    foreach ($part->citations as $citation) {
        $footnotes[] = [
            'id' => $footnoteId,
            'title' => $citation->sourceTitle,
            'uri' => $citation->source,
        ];

        $text .= '<sup><a href="#footnote-'.$footnoteId.'">'.$footnoteId.'</a></sup>';

        $footnoteId++;
    }
}

// Pass $text and $footnotes to your frontend.
```

## Structured Output

Gemini supports structured output, allowing you to define schemas that constrain the model's responses to match your exact data structure requirements.

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\Schema\ObjectSchema;
use Prism\Prism\Schema\StringSchema;

$schema = new ObjectSchema(
    name: 'movie_review',
    description: 'A structured movie review',
    properties: [
        new StringSchema('title', 'The movie title'),
        new StringSchema('rating', 'Rating out of 5 stars'),
        new StringSchema('summary', 'Brief review summary'),
    ],
    requiredFields: ['title', 'rating', 'summary']
);

$response = Prism::structured()
    ->using(Provider::Gemini, 'gemini-2.0-flash')
    ->withSchema($schema)
    ->withPrompt('Review the movie Inception')
    ->asStructured();

// Access structured data
dump($response->structured);
```

### Flexible Types with anyOf

For fields that can match multiple types or structures, use `AnyOfSchema`. This is useful for polymorphic data or when a field might contain different shapes:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\Schema\AnyOfSchema;
use Prism\Prism\Schema\ObjectSchema;
use Prism\Prism\Schema\StringSchema;
use Prism\Prism\Schema\NumberSchema;

// Simple example: value can be string or number
$schema = new ObjectSchema(
    'response',
    'API response with flexible value',
    [
        new AnyOfSchema(
            schemas: [
                new StringSchema('text', 'Text value'),
                new NumberSchema('number', 'Numeric value'),
            ],
            name: 'value',
            description: 'Can be either text or number'
        ),
    ],
    ['value']
);

$response = Prism::structured()
    ->using(Provider::Gemini, 'gemini-2.5-flash')
    ->withSchema($schema)
    ->withPrompt('Extract the value from: "The answer is 42"')
    ->asStructured();

// $response->structured['value'] could be "42" (string) or 42 (number)
```

For complex polymorphic structures, `anyOf` can distinguish between entirely different object types:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\Schema\AnyOfSchema;
use Prism\Prism\Schema\ObjectSchema;
use Prism\Prism\Schema\StringSchema;
use Prism\Prism\Schema\NumberSchema;

$articleSchema = new ObjectSchema(
    'article',
    'A blog article',
    [
        new StringSchema('title', 'Article title'),
        new StringSchema('content', 'Full article text'),
        new StringSchema('author', 'Author name'),
    ],
    ['title', 'content']
);

$imageSchema = new ObjectSchema(
    'image',
    'An image post',
    [
        new StringSchema('url', 'Image URL'),
        new StringSchema('caption', 'Image caption'),
        new NumberSchema('width', 'Width in pixels'),
        new NumberSchema('height', 'Height in pixels'),
    ],
    ['url']
);

$schema = new ObjectSchema(
    'social_post',
    'Social media post',
    [
        new AnyOfSchema(
            schemas: [$articleSchema, $imageSchema],
            name: 'content',
            description: 'Post content - either article or image'
        ),
    ],
    ['content']
);

$response = Prism::structured()
    ->using(Provider::Gemini, 'gemini-2.5-flash')
    ->withSchema($schema)
    ->withPrompt('Analyze this post and extract its content')
    ->asStructured();

// Result will be either {title, content, author} OR {url, caption, width, height}
```

> \[!NOTE]
> The `anyOf` feature requires Gemini 2.5 or later models.

### Numeric Constraints

Constrain numeric values to specific ranges and precision using JSON Schema numeric constraints:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\Schema\ObjectSchema;
use Prism\Prism\Schema\NumberSchema;

$schema = new ObjectSchema(
    'product_rating',
    'Product rating information',
    [
        new NumberSchema(
            name: 'rating',
            description: 'User rating (1-5 stars, half-star increments)',
            minimum: 1.0,
            maximum: 5.0,
            multipleOf: 0.5
        ),
        new NumberSchema(
            name: 'price',
            description: 'Product price in USD',
            minimum: 0.01,
            exclusiveMaximum: 10000.0
        ),
        new NumberSchema(
            name: 'quantity',
            description: 'Stock quantity',
            minimum: 0
        ),
    ],
    ['rating', 'price', 'quantity']
);

$response = Prism::structured()
    ->using(Provider::Gemini, 'gemini-2.5-flash')
    ->withSchema($schema)
    ->withPrompt('Extract rating, price, and quantity from this product review')
    ->asStructured();
```

**Available Numeric Constraints:**

* `minimum` - Minimum value (inclusive)
* `maximum` - Maximum value (inclusive)
* `exclusiveMinimum` - Minimum value (exclusive)
* `exclusiveMaximum` - Maximum value (exclusive)
* `multipleOf` - Value must be a multiple of this number

### Nullable Fields

Make any field optional by marking it as nullable. The field must be present in the response, but can be `null`:

```php
use Prism\Prism\Schema\ObjectSchema;
use Prism\Prism\Schema\StringSchema;

$schema = new ObjectSchema(
    'user',
    'User profile',
    [
        new StringSchema('name', 'User name'),
        new StringSchema('email', 'Email address', nullable: true),  // Optional
    ],
    ['name', 'email']  // Both required, but email can be null
);
```

Nullable works with `anyOf` to create truly optional polymorphic fields:

```php
use Prism\Prism\Schema\AnyOfSchema;
use Prism\Prism\Schema\ObjectSchema;
use Prism\Prism\Schema\StringSchema;
use Prism\Prism\Schema\NumberSchema;

$schema = new ObjectSchema(
    'user_input',
    'User input that may be missing',
    [
        new AnyOfSchema(
            schemas: [
                new StringSchema('text', 'Text input'),
                new NumberSchema('number', 'Numeric input'),
            ],
            name: 'user_value',
            description: 'User provided value, or null if not provided',
            nullable: true  // Adds null as a valid type
        ),
    ],
    ['user_value']
);

// Result can be string, number, or null
```

### Combining Tools with Structured Output

Gemini natively supports combining custom tools with structured output. The AI can call tools to gather data, then return a structured response:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\Schema\ObjectSchema;
use Prism\Prism\Schema\StringSchema;
use Prism\Prism\Tool;

$schema = new ObjectSchema(
    name: 'weather_analysis',
    description: 'Analysis of weather conditions',
    properties: [
        new StringSchema('summary', 'Summary of the weather'),
        new StringSchema('recommendation', 'Recommendation based on weather'),
    ],
    requiredFields: ['summary', 'recommendation']
);

$weatherTool = Tool::as('get_weather')
    ->for('Get current weather for a location')
    ->withStringParameter('location', 'The city and state')
    ->using(fn (string $location): string => "Weather in {$location}: 72°F, sunny");

$response = Prism::structured()
    ->using('gemini', 'gemini-2.0-flash')
    ->withSchema($schema)
    ->withTools([$weatherTool])
    ->withMaxSteps(3)
    ->withPrompt('What is the weather in San Francisco and should I wear a coat?')
    ->asStructured();

// Access structured output
dump($response->structured);

// Access tool execution details
foreach ($response->toolCalls as $toolCall) {
    echo "Called: {$toolCall->name}\n";
}
```

> \[!IMPORTANT]
> When combining tools with structured output, set `maxSteps` to at least 2.

For complete documentation on combining tools with structured output, see [Structured Output - Combining with Tools](/core-concepts/structured-output#combining-structured-output-with-tools).

## Caching

Prism supports Gemini prompt caching, though due to Gemini requiring you first upload the cached content, it works a little differently to other providers.

To store content in the cache, use the Gemini provider cache method as follows:

```php

use Prism\Prism\Enums\Provider;
use Prism\Prism\Facades\Prism;
use Prism\Prism\Providers\Gemini\Gemini;
use Prism\Prism\ValueObjects\Media\Document;
use Prism\Prism\ValueObjects\Messages\SystemMessage;
use Prism\Prism\ValueObjects\Messages\UserMessage;

/** @var Gemini */
$provider = Prism::provider(Provider::Gemini);

$object = $provider->cache(
    model: 'gemini-1.5-flash-002',
    messages: [
        new UserMessage('', [
            Document::fromLocalPath('tests/Fixtures/long-document.pdf'),
        ]),
    ],
    systemPrompts: [
        new SystemMessage('You are a legal analyst.'),
    ],
    ttl: 60
);
```

Then reference that object's name in your request using withProviderOptions:

```php
$response = Prism::text()
    ->using(Provider::Gemini, 'gemini-1.5-flash-002')
    ->withProviderOptions(['cachedContentName' => $object->name])
    ->withPrompt('In no more than 100 words, what is the document about?')
    ->asText();
```

## Embeddings

You can customize your Gemini embeddings request with additional parameters using `->withProviderOptions()`.

### Gemini Embedding 2 Preview

Gemini's `gemini-embedding-2-preview` model supports text, images, audio, video, and PDF documents in a unified embedding space. Use Prism's content entry API to control whether you want a single aggregated embedding or multiple embeddings in one request.

### Single Modality Inputs

```php
use Prism\Prism\Enums\Provider;
use Prism\Prism\Facades\Prism;
use Prism\Prism\ValueObjects\Media\Audio;
use Prism\Prism\ValueObjects\Media\Document;
use Prism\Prism\ValueObjects\Media\Image;
use Prism\Prism\ValueObjects\Media\Video;

Prism::embeddings()
    ->using(Provider::Gemini, 'gemini-embedding-2-preview')
    ->fromImage(Image::fromLocalPath('/path/to/product.png'))
    ->fromAudio(Audio::fromLocalPath('/path/to/example.mp3'))
    ->fromVideo(Video::fromLocalPath('/path/to/example.mp4'))
    ->fromDocument(Document::fromLocalPath('/path/to/report.pdf'))
    ->asEmbeddings();
```

### Aggregated Multimodal Embeddings

Use `fromContent()` to combine multiple parts into a single embedding:

```php
use Prism\Prism\Enums\Provider;
use Prism\Prism\Facades\Prism;
use Prism\Prism\ValueObjects\Media\Image;

Prism::embeddings()
    ->using(Provider::Gemini, 'gemini-embedding-2-preview')
    ->fromContent([
        'An image of a dog',
        Image::fromLocalPath('/path/to/dog.png'),
    ])
    ->asEmbeddings();
```

### Batch Embeddings

Use `fromContents()` to generate multiple embeddings in a single request:

```php
use Prism\Prism\Enums\Provider;
use Prism\Prism\Facades\Prism;
use Prism\Prism\ValueObjects\Media\Image;

Prism::embeddings()
    ->using(Provider::Gemini, 'gemini-embedding-2-preview')
    ->fromContents([
        ['The dog is cute'],
        [Image::fromLocalPath('/path/to/dog.png')],
    ])
    ->asEmbeddings();
```

### Title

You can add a title to your embedding request. Only applicable when TaskType is `RETRIEVAL_DOCUMENT`

```php
use Prism\Prism\Enums\Provider;
use Prism\Prism\Facades\Prism;

Prism::embeddings()
    ->using(Provider::Gemini, 'text-embedding-004')
    ->fromInput('The food was delicious and the waiter...')
    ->withProviderOptions(['title' => 'Restaurant Review'])
    ->asEmbeddings();
```

### Task Type

Gemini allows you to specify the task type for your embeddings to optimize them for specific use cases:

```php
use Prism\Prism\Enums\Provider;
use Prism\Prism\Facades\Prism;

Prism::embeddings()
    ->using(Provider::Gemini, 'text-embedding-004')
    ->fromInput('The food was delicious and the waiter...')
    ->withProviderOptions(['taskType' => 'RETRIEVAL_QUERY'])
    ->asEmbeddings();
```

[Available task types](https://ai.google.dev/api/embeddings#tasktype)

### Output Dimensionality

You can control the dimensionality of your embeddings:

```php
use Prism\Prism\Enums\Provider;
use Prism\Prism\Facades\Prism;

Prism::embeddings()
    ->using(Provider::Gemini, 'text-embedding-004')
    ->fromInput('The food was delicious and the waiter...')
    ->withProviderOptions(['outputDimensionality' => 768])
    ->asEmbeddings();
```

### Thinking Mode

Gemini 2.5 series models use an internal "thinking process" during response generation. Thinking is on by default as these models have the ability to automatically decide when and how much to think based on the prompt. If you would like to customize how many tokens the model may use for thinking, or disable thinking altogether, utilize the `withProviderOptions()` method, and pass through an array with a key value pair with `thinkingBudget` and an integer representing the budget of tokens. Set this value to `0` to disable thinking.

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;

$response = Prism::text()
    ->using(Provider::Gemini, 'gemini-2.5-flash-preview')
    ->withPrompt('Explain the concept of Occam\'s Razor and provide a simple, everyday example.')
    // Set thinking budget
    ->withProviderOptions(['thinkingBudget' => 300])
    ->asText();
```

> \[!NOTE]
> Do not specify a `thinkingBudget` on 2.0 or prior series Gemini models as your request will fail.

## Streaming

Gemini supports streaming responses in real-time. All the standard streaming methods work with Gemini models:

```php
return Prism::text()
    ->using('gemini', 'gemini-2.5-flash-preview')
    ->withPrompt(request('message'))
    ->asEventStreamResponse();
```

### Streaming with Thinking

Models with thinking capabilities stream their reasoning process separately:

```php
use Prism\Prism\Enums\StreamEventType;

foreach ($stream as $event) {
    match ($event->type()) {
        StreamEventType::ThinkingDelta => echo "[Thinking] " . $event->delta,
        StreamEventType::TextDelta => echo $event->delta,
        default => null,
    };
}
```

For complete streaming documentation, see [Streaming Output](/core-concepts/streaming-output).

## Media Support

Gemini has robust support for processing multimedia content.

### Media Resolution

Gemini 3 models support the `mediaResolution` provider option to control the quality vs token usage tradeoff for images, videos, documents, and audio. Higher resolutions improve fine detail recognition but increase token consumption.

| Resolution | Image Tokens | Video Tokens (per frame) | PDF Tokens |
|------------|--------------|--------------------------|------------|
| `MEDIA_RESOLUTION_LOW` | 280 | 70 | 280 + text |
| `MEDIA_RESOLUTION_MEDIUM` | 560 | 70 | 560 + text |
| `MEDIA_RESOLUTION_HIGH` | 1120 | 280 | 1120 + text |

```php
use Prism\Prism\ValueObjects\Messages\UserMessage;
use Prism\Prism\ValueObjects\Media\Image;
use Prism\Prism\Enums\Provider;

$response = Prism::text()
    ->using(Provider::Gemini, 'gemini-3-flash-preview')
    ->withMessages([
        new UserMessage(
            'Read the fine print in this document.',
            additionalContent: [
                Image::fromLocalPath('/path/to/document.png')
                    ->withProviderOptions(['mediaResolution' => 'MEDIA_RESOLUTION_HIGH']),
            ],
        ),
    ])
    ->asText();
```

### Video Analysis

Gemini can process and analyze video content including standard video files and YouTube videos. Prism implements this through the `Video` value object which maps to Gemini's video processing capabilities.

```php
use Prism\Prism\ValueObjects\Messages\UserMessage;
use Prism\Prism\ValueObjects\Media\Video;
use Prism\Prism\Enums\Provider;

$response = Prism::text()
    ->using(Provider::Gemini, 'gemini-1.5-flash')
    ->withMessages([
        new UserMessage(
            'What is happening in this video?',
            additionalContent: [
                Video::fromUrl('https://example.com/sample-video.mp4'),
            ],
        ),
    ])
    ->asText();
```

### YouTube Integration

Gemini has special support for YouTube videos. You can easily `analyze/summarize` YouTube content by providing the URL:

```php
use Prism\Prism\ValueObjects\Messages\UserMessage;
use Prism\Prism\ValueObjects\Media\Video;
use Prism\Prism\Enums\Provider;

$response = Prism::text()
    ->using(Provider::Gemini, 'gemini-1.5-flash')
    ->withMessages([
        new UserMessage(
            'Summarize this YouTube video:',
            additionalContent: [
                Video::fromUrl('https://www.youtube.com/watch?v=dQw4w9WgXcQ'),
            ],
        ),
    ])
    ->asText();
```

### Audio Processing

Gemini can analyze audio files for various tasks like transcription, content analysis, and audio scene understanding. The implementation in Prism uses the `Audio` value object which is specifically designed for Gemini's audio processing capabilities.

```php
use Prism\Prism\ValueObjects\Messages\UserMessage;
use Prism\Prism\ValueObjects\Media\Audio;
use Prism\Prism\Enums\Provider;

$response = Prism::text()
    ->using(Provider::Gemini, 'gemini-1.5-flash')
    ->withMessages([
        new UserMessage(
            'Transcribe this audio file:',
            additionalContent: [
                Audio::fromLocalPath('/path/to/audio.mp3'),
            ],
        ),
    ])
    ->asText();
```

## Image Generation

Prism supports Gemini image generation through Imagen and Gemini models. See Gemini [image generation docs](https://ai.google.dev/gemini-api/docs/image-generation) for full usage.

### Supported Models

| Model                                       | Description                                        |
| ------------------------------------------- | -------------------------------------------------- |
| `gemini-2.0-flash-preview-image-generation` | Experimental gemini image generation model.        |
| `imagen-4.0-generate-001`                   | Latest Imagen model. Good for HD image generation. |
| `imagen-4.0-ultra-generate-001`             | Highest quality images, only one image per request |
| `imagen-4.0-fast-generate-001`              | Fastest Imagen 4 model                             |
| `imagen-3.0-generate-002`                   | Imagen 3                                           |

### Basic Usage

```php
$response = Prism::image()
    ->using(Provider::Gemini, 'gemini-2.0-flash-preview-image-generation')
    ->withPrompt('Generate an image of ducklings wearing rubber boots')
    ->generate();

file_put_contents('image.png', base64_decode($response->firstImage()->base64));

// gemini models return usage and metadata
echo $response->usage->promptTokens;
echo $response->meta->id;
```

### Image Editing with Gemini

```php
$originalImage = fopen('image/boots.png', 'r');

$response = Prism::image()
    ->using(Provider::Gemini, 'gemini-2.0-flash-preview-image-generation')
    ->withPrompt('Actually, could we make those boots red?')
    ->withProviderOptions([
        'image' => $originalImage,
        'image_mime_type' => 'image/png',
    ])
    ->generate();

file_put_contents('new-boots.png', base64_decode($response->firstImage()->base64));
```

### Image options for Imagen models

```php
$response = Prism::image()
    ->using(Provider::Gemini, 'imagen-4.0-generate-001')
    ->withPrompt('Generate an image of a magnificent building falling into the ocean')
    ->withProviderOptions([
        'n' => 3,                               // number of images to generate
        'size' => '2K',                         // 1K (default), 2K
        'aspect_ratio' => '16:9',               // 1:1 (default), 3:4, 4:3, 9:16, 16:9
        'person_generation' => 'dont_allow',    // dont_allow, allow_adult, allow_all
    ])
    ->generate();
```

Note:

* Imagen 4 Ultra can only generate 1 image at a time.
* An empty response is sent if the prompt is in violation of the person\_generation policy, causing Prism to throw an Exception.

### Response Format

All generated images are returned as base64 encoded strings.

---

---
url: /providers/groq.md
---
# Groq

## Configuration

```php
'groq' => [
    'api_key' => env('GROQ_API_KEY', ''),
    'url' => env('GROQ_URL', 'https://api.groq.com/openai/v1'),
],
```

## Streaming

Groq's ultra-fast LPU architecture provides exceptional streaming performance. All standard streaming methods are supported:

```php
return Prism::text()
    ->using('groq', 'llama-3.3-70b-versatile')
    ->withPrompt(request('message'))
    ->asEventStreamResponse();
```

For complete streaming documentation, see [Streaming Output](/core-concepts/streaming-output).

## Audio Processing

Groq provides high-performance audio processing capabilities through their ultra-fast Language Processing Unit (LPU) architecture, enabling both text-to-speech (TTS) and speech-to-text (STT) functionality with exceptional speed and quality.

### Text-to-Speech

Groq offers PlayAI TTS models that can convert text into natural-sounding speech with support for multiple languages and voices.

#### Basic TTS Usage

```php
use Prism\Prism\Facades\Prism;

$response = Prism::audio()
    ->using('groq', 'playai-tts')
    ->withInput('Hello, welcome to our application!')
    ->withVoice('Fritz-PlayAI')
    ->asAudio();

// Save the audio file
$audioData = base64_decode($response->audio->base64);
file_put_contents('welcome.wav', $audioData);
```

#### TTS Configuration Options

Control audio format and quality:

```php
$response = Prism::audio()
    ->using('groq', 'playai-tts')
    ->withInput('Testing different audio settings.')
    ->withVoice('Celeste-PlayAI')
    ->withProviderOptions([
        'response_format' => 'wav',        // wav (default)
        'speed' => 1.2,                    // Speed: 0.5 to 5.0
        'sample_rate' => 48000,            // Sample rate options: 8000, 16000, 22050, 24000, 32000, 44100, 48000
    ])
    ->asAudio();

echo "Audio type: " . $response->audio->getMimeType();
```

#### Arabic Text-to-Speech

```php
$response = Prism::audio()
    ->using('groq', 'playai-tts-arabic')
    ->withInput('مرحبا بكم في تطبيقنا')
    ->withVoice('Amira-PlayAI')
    ->asAudio();

file_put_contents('arabic_speech.wav', base64_decode($response->audio->base64));
```

### Speech-to-Text

Groq provides ultra-fast speech recognition using Whisper models, offering exceptional speed with real-time factors of up to 299x.

#### Basic STT Usage

```php
use Prism\Prism\ValueObjects\Media\Audio;

$audioFile = Audio::fromPath('/path/to/recording.mp3');

$response = Prism::audio()
    ->using('groq', 'whisper-large-v3')
    ->withInput($audioFile)
    ->asText();

echo "Transcription: " . $response->text;
```

#### Model Selection Guide

Choose the right model for your use case:

```php
$response = Prism::audio()
    ->using('groq', 'whisper-large-v3')
    ->withInput($audioFile)
    ->asText();

// For fastest English-only transcription
$response = Prism::audio()
    ->using('groq', 'distil-whisper-large-v3-en')
    ->withInput($audioFile)
    ->asText();

// For balanced speed and multilingual capability
$response = Prism::audio()
    ->using('groq', 'whisper-large-v3-turbo')
    ->withInput($audioFile)
    ->asText();
```

#### Language Detection and Specification

Whisper can automatically detect languages or you can specify them:

```php
$response = Prism::audio()
    ->using('groq', 'whisper-large-v3')
    ->withInput($audioFile)
    ->withProviderOptions([
        'language' => 'es',           // ISO-639-1 code (optional)
        'temperature' => 0.2,         // Lower for more focused results
    ])
    ->asText();
```

#### Response Formats

Get transcriptions in different formats:

```php
// Standard JSON response
$response = Prism::audio()
    ->using('groq', 'whisper-large-v3')
    ->withInput($audioFile)
    ->withProviderOptions([
        'response_format' => 'json',  // json, text, verbose_json
    ])
    ->asText();

// Verbose JSON includes timestamps and segments
$response = Prism::audio()
    ->using('groq', 'whisper-large-v3')
    ->withInput($audioFile)
    ->withProviderOptions([
        'response_format' => 'verbose_json',
        'timestamp_granularities' => ['segment'], // word, segment
    ])
    ->asText();

// Access detailed segment information
$segments = $response->additionalContent['segments'] ?? [];
foreach ($segments as $segment) {
    echo "Text: " . $segment['text'] . "\n";
    echo "Start: " . $segment['start'] . "s\n";
    echo "End: " . $segment['end'] . "s\n";
}
```

#### Context and Prompts

Improve transcription accuracy with context:

```php
$response = Prism::audio()
    ->using('groq', 'whisper-large-v3')
    ->withInput($audioFile)
    ->withProviderOptions([
        'prompt' => 'This is a technical discussion about machine learning and artificial intelligence.',
        'language' => 'en',
        'temperature' => 0.1,         // Lower temperature for technical content
    ])
    ->asText();
```

#### Creating Audio Objects

```php
use Prism\Prism\ValueObjects\Media\Audio;

// From local file path
$audio = Audio::fromPath('/path/to/audio.mp3');

// From remote URL (recommended for large files)
$audio = Audio::fromUrl('https://example.com/recording.wav');

// From base64 encoded data
$audio = Audio::fromBase64($base64AudioData, 'audio/mpeg');

// From binary content
$audioContent = file_get_contents('/path/to/audio.wav');
$audio = Audio::fromContent($audioContent, 'audio/wav');
```

---

---
url: /advanced/rate-limits.md
---
# Handling Rate Limits

Hitting issues with rate limits? We've got you covered!

In this guide we will look at handling:

* situations where you actually hit a rate limit (i.e. HTTP 429); and
* dynamic rate limiting (figuring out when you can make your next request, from a successful request).

## Provider support

Prism throws a `PrismRateLimitedException` for all providers other than DeepSeek (which does not have rate limits).

Prism provides an array of `ProviderRateLimit` value objects on the exception and on meta for all providers other than OpenAI, Gemini, xAI and VoyageAI - as they do not provide the necessary headers to do so.

## The ProviderRateLimit value object

Throughout this guide, we'll talk about the `ProviderRateLimit` value object.

Each `ProviderRateLimit` has four properties:

* name - the name given to that rate limit by the provider - e.g. "input-tokens"
* limit - the current limit set on your API key by the provider - e.g. for input-tokens, perhaps 80000
* remaining - how many you have left - e.g. for input-tokens if you have used 30000 out of your 80000 limit - this will be 50000
* resetsAt - a Carbon instance with the date and time at which remaining will reset to limit

## Handling a rate limit hit

Prism throws a `PrismRateLimitedException` when you hit a rate limit.

You can catch that exception, gracefully fail and inspect the `rateLimits` property which contains an array of `ProviderRateLimit`s.

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\ValueObjects\ProviderRateLimit;
use Prism\Prism\Exceptions\PrismRateLimitedException;

try {
    Prism::text()
        ->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
        ->withPrompt('Hello world!')
        ->asText();
}
catch (PrismRateLimitedException $e) {
    /** @var ProviderRateLimit $rate_limit */ 
    foreach ($e->rateLimits as $rate_limit) {
        // Loop through rate limits...
    }
    
    // Log, fail gracefully, etc.
}
```

### Figuring out which rate limit you have hit

In a simple world, they'd only be one rate limit.

However most providers implement various rate limits (e.g. request, input tokens, output tokens, etc.) and provide you with information on all of them on all requests, regardless of which you have hit.

For simple rate limits like "requests", the `remaining` property on `ProviderRateLimit` will be 0 if you have hit it. These are easy to find:

```php
use Prism\Prism\ValueObjects\ProviderRateLimit;
use Illuminate\Support\Arr;

try {
    // Your request
}
catch (PrismRateLimitedException $e) {
    $hit_limit = Arr::first($e->rateLimits, fn(ProviderRateLimit $rate_limit) => $rate_limit->remaining === 0);
}
```

For less simple rate limits like input tokens, the `remaining` property may not be zero. For instance, if you have 5,000 input tokens remaining and submit a request requiring 6,000 tokens, you'll be rate limited but remaining will still show 5,000.

Here, you may need to implement some logic to approximate how many tokens your request will use before sending it, and then test against that:

```php
use Prism\Prism\ValueObjects\ProviderRateLimit;
use Illuminate\Support\Arr;

try {
    // Your request
}
catch (PrismRateLimitedException $e) {
    $input_token_limit = Arr::first($e->rateLimits, fn(ProviderRateLimit $rate_limit) => $rate_limit->name === 'input-tokens');

    if ($input_token_limit < $your_token_estimate) {
        // Handle
    }
}
```

To help with approximating input token usage, we plan to implement Anthopic's token counting endpoint in a future release.

For providers that don't have a token counting endpoint, you could either roll your own token counter or use something like [tiktoken](https://github.com/openai/tiktoken) if you are comfortable calling out to Python.

Once you know which rate limit you have hit, you'll want to ensure your app does not continue making requests until after the `ProviderRateLimit` `resetsAt` property.

If you aren't sure where to start with that, check out the [What should you do with rate limit information](#what-should-you-do-with-rate-limit-information) section below.

## Dynamic rate limiting

Prism adds the same rate limit information to every successful request:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\ValueObjects\ProviderRateLimit;

$response = Prism::text()
    ->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
    ->withPrompt('Hello world!')
    ->asText();
    
/** @var ProviderRateLimit $rate_limit */ 
foreach ($response->meta->rateLimits as $rate_limit) {
    // Handle
}

```

Armed with that information, you'll probably want to [update your app's rate limiter(s)](#what-should-you-do-with-rate-limit-information).

## What should you do with rate limit information?

You'll likely want to implement a rate limiter within your app. Thankfully Laravel, as always, makes this very easy!

You should take a look at the [rate limiting](https://laravel.com/docs/11.x/rate-limiting) docs, and if you are firing requests from your queue, check out the [job middleware](https://laravel.com/docs/11.x/queues#job-middleware) docs.

You should implement a rate limiter / job middleware for each of the provider rate limits your application typically hits.

---

---
url: /core-concepts/image-generation.md
---
# Image Generation

Generate stunning images from text prompts using AI-powered models. Prism provides a clean, consistent API for image generation across different providers, starting with comprehensive OpenAI support.

## Getting Started

Creating images with Prism is as simple as describing what you want:

```php
use Prism\Prism\Facades\Prism;

$response = Prism::image()
    ->using('openai', 'dall-e-3')
    ->withPrompt('A cute baby sea otter floating on its back in calm blue water')
    ->generate();

$image = $response->firstImage();
echo $image->url; // https://oaidalleapiprodscus.blob.core.windows.net/...
```

## Provider Support

Currently, Prism supports image generation through:

* **OpenAI**: DALL-E 2, DALL-E 3, and GPT-Image-1 models
* **Gemini**: Gemini 2.0 Flash Preview Image Generation, Imagen 4, Imagen 3

Additional providers will be added in future releases as the ecosystem evolves.

## Basic Usage

### Simple Generation

The most straightforward way to generate an image:

```php
$response = Prism::image()
    ->using('openai', 'dall-e-3')
    ->withPrompt('A serene mountain landscape at sunset')
    ->generate();

// Access the generated image
$image = $response->firstImage();
if ($image->hasUrl()) {
    echo "Image URL: " . $image->url;
}
if ($image->hasBase64()) {
    echo "Base64 Image Data: " . $image->base64;
}
```

### Working with Responses

The response object provides helpful methods for accessing generated content:

```php
$response = Prism::image()
    ->using('openai', 'dall-e-2')
    ->withPrompt('Abstract geometric patterns in vibrant colors')
    ->generate();

// Check if images were generated
if ($response->hasImages()) {
    echo "Generated {$response->imageCount()} image(s)";

    // Access all images
    foreach ($response->images as $image) {
        if ($image->hasUrl()) {
            echo "Image: {$image->url}\n";
        }

        if ($image->hasBase64()) {
            echo "Base64 Image: " . substr($image->base64, 0, 50) . "...\n";
        }

        if ($image->hasRevisedPrompt()) {
            echo "Revised prompt: {$image->revisedPrompt}\n";
        }
    }

    // Or just get the first one
    $firstImage = $response->firstImage();
}

// Check usage information
echo "Prompt tokens: {$response->usage->promptTokens}";
echo "Model used: {$response->meta->model}";

// Access the raw API response data
$rawResponse = $response->raw;
```

## Provider-Specific Options

While Prism provides a consistent API, you can access provider-specific features using the `withProviderOptions()` method.

### OpenAI Options

OpenAI offers various customization options depending on the model:

#### DALL-E 3 Options

```php
$response = Prism::image()
    ->using('openai', 'dall-e-3')
    ->withPrompt('A beautiful sunset over mountains')
    ->withProviderOptions([
        'size' => '1792x1024',          // 1024x1024, 1024x1792, 1792x1024
        'quality' => 'hd',              // standard, hd
        'style' => 'vivid',             // vivid, natural
        'response_format' => 'url',     // url, b64_json
    ])
    ->generate();
```

#### GPT-Image-1 (Base64 Only)

The GPT-Image-1 model always returns base64-encoded images, regardless of the `response_format` setting:

```php
$response = Prism::image()
    ->using('openai', 'gpt-image-1')
    ->withPrompt('A cute baby sea otter floating on its back')
    ->withProviderOptions([
        'size' => '1024x1024',              // 1024x1024, 1536x1024, 1024x1536, auto
        'quality' => 'high',                // auto, high, medium, low
        'background' => 'transparent',      // transparent, opaque, auto
        'output_format' => 'png',           // png, jpeg, webp
        'output_compression' => 90,         // 0-100 (for jpeg/webp)
    ])
    ->generate();

$image = $response->firstImage();
if ($image->hasBase64()) {
    // Save the base64 image to a file
    file_put_contents('generated-image.png', base64_decode($image->base64));
    echo "Base64 image saved to generated-image.png";
}
```

#### Base64 vs URL Responses

Different models return images in different formats:

* **GPT-Image-1**: Always returns base64-encoded images in the `base64` property
* **DALL-E 2 & 3**: Return URLs by default, but can return base64 when `response_format` is set to `'b64_json'`

```php
// Request base64 format from DALL-E 3
$response = Prism::image()
    ->using('openai', 'dall-e-3')
    ->withPrompt('Abstract art')
    ->withProviderOptions([
        'response_format' => 'b64_json',
    ])
    ->generate();

$image = $response->firstImage();
if ($image->hasBase64()) {
    echo "Received base64 image data";
}
```

#### Image Editing

OpenAI's `gpt-image-1` model supports editing existing images. Pass your images as the second parameter to `withPrompt()`:

```php
use Prism\Prism\ValueObjects\Media\Image;

$originalImage = Image::fromLocalPath('photos/landscape.png');

$response = Prism::image()
    ->using('openai', 'gpt-image-1')
    ->withPrompt('Add a vaporwave sunset to the background', [$originalImage])
    ->withProviderOptions([
        'size' => '1024x1024',
        'output_format' => 'png',
        'quality' => 'high',
    ])
    ->generate();

// The edited image is returned as base64
$editedImage = $response->firstImage();
file_put_contents('edited-landscape.png', base64_decode($editedImage->base64));
```

You can edit multiple images at once:

```php
$response = Prism::image()
    ->using('openai', 'gpt-image-1')
    ->withPrompt('Make the colors more vibrant', [
        Image::fromLocalPath('photo1.png'),
        Image::fromLocalPath('photo2.png')->as('custom-name.png'),
    ])
    ->generate();
```

For precise edits, use a mask to specify which areas to modify:

```php
$response = Prism::image()
    ->using('openai', 'gpt-image-1')
    ->withPrompt('Replace the sky with a starry night', [
        Image::fromLocalPath('landscape.png'),
    ])
    ->withProviderOptions([
        'mask' => Image::fromLocalPath('sky-mask.png'), // White areas will be edited
        'size' => '1024x1024',
        'output_format' => 'png',
    ])
    ->generate();
```

> \[!NOTE]
> The mask should be a PNG image where white pixels indicate areas to edit and transparent pixels indicate areas to preserve.

### Gemini Options

Gemini offers customizations, depending on what model is selected. All Gemini image generation models return base64-encoded images only. They also return `mimeType`.

### Gemini Flash Preview Image Generation

Gemini conversational image generation provides the option to edit images by passing them as the second parameter to `withPrompt()`:

```php
use Prism\Prism\ValueObjects\Media\Image;

$originalImage = Image::fromLocalPath('image/boots.png');

$response = Prism::image()
    ->using(Provider::Gemini, 'gemini-2.0-flash-preview-image-generation')
    ->withPrompt('Actually, could we make those boots red?', [$originalImage])
    ->generate();
```

### Imagen Options

```php
$response = Prism::image()
    ->using(Provider::Gemini, 'imagen-4.0-generate-001')
    ->withPrompt('Generate an image of a magnificent building falling into the ocean')
    ->withProviderOptions([
        'n' => 3,                               // number of images to generate
        'size' => '2K',                         // 1K (default), 2K
        'aspect_ratio' => '16:9',               // 1:1 (default), 3:4, 4:3, 9:16, 16:9
        'person_generation' => 'dont_allow',    // dont_allow, allow_adult, allow_all
    ])
    ->generate();
```

## Testing

Prism provides convenient fakes for testing image generation:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Testing\PrismFake;

test('can generate images', function () {
    $fake = PrismFake::create()->image();
    Prism::fake($fake);

    $response = Prism::image()
        ->using('openai', 'dall-e-3')
        ->withPrompt('Test image')
        ->generate();

    expect($response->hasImages())->toBeTrue();
    expect($response->firstImage()->url)->toContain('fake-image-url');
});
```

Need help with a specific provider or use case? Check the [openai documentation](/providers/openai) or [gemini documentation](/providers/gemini) for detailed configuration options and examples.

---

---
url: /input-modalities/images.md
---
# Images

Prism supports including images in your messages for vision analysis for most providers.

See the [provider support table](/getting-started/introduction.html#provider-support) to check whether Prism supports your chosen provider.

Note however that provider support may differ by model. If you receive error messages with a provider that Prism indicates is supported, check the provider's documentation as to whether the model you are using supports images.

## Getting started

To add an image to your prompt, use the `withPrompt` method with an `Image` value object:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\ValueObjects\Media\Image;

// From a local path
$response = Prism::text()
    ->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
    ->withPrompt(
        "What's in this image?",
        [Image::fromLocalPath(path: '/path/to/image.jpg')]
    )
    ->asText();

// From a path on a storage disk
$response = Prism::text()
    ->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
    ->withPrompt(
        "What's in this image?",
        [Image::fromStoragePath(
            path: '/path/to/image.jpg',
            diskName: 'my-disk' // optional - omit/null for default disk
        )]
    )
    ->asText();

// From a URL
$response = Prism::text()
    ->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
    ->withPrompt(
        'Analyze this diagram:',
        [Image::fromUrl(url: 'https://example.com/diagram.png')]
    )
    ->asText();

// From base64
$response = Prism::text()
    ->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
    ->withPrompt(
        'Analyze this diagram:',
        [Image::fromBase64(base64: base64_encode(file_get_contents('/path/to/image.jpg')))]
    )
    ->asText();

// From raw content
$response = Prism::text()
    ->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
    ->withPrompt(
        'Analyze this diagram:',
        [Image::fromRawContent(rawContent: file_get_contents('/path/to/image.jpg'))]
    )
    ->asText();
```

## Alternative: Using withMessages

You can also include images using the message-based approach:

```php
use Prism\Prism\ValueObjects\Messages\UserMessage;
use Prism\Prism\ValueObjects\Media\Image;

$message = new UserMessage(
    "What's in this image?",
    [Image::fromLocalPath(path: '/path/to/image.jpg')]
);

$response = Prism::text()
    ->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
    ->withMessages([$message])
    ->asText();
```

## Customizing Image Filenames

When uploading images, you can provide a custom filename using the fluent `as()` method. This is particularly useful when working with multiple images, as it makes your API requests more readable and helps with debugging:

```php
use Prism\Prism\ValueObjects\Media\Image;

$response = Prism::image()
    ->using('openai', 'gpt-image-1')
    ->withPrompt('Edit these images', [
        Image::fromLocalPath('path/to/photo1.png')->as('original-photo.png'),
        Image::fromLocalPath('path/to/photo2.png')->as('reference-image.png'),
    ])
    ->generate();
```

Without custom filenames, images are automatically named using a default pattern. The `as()` method lets you provide meaningful names that make your code more self-documenting.

## Transfer mediums

Providers are not consistent in their support of sending raw contents, base64 and/or URLs (as noted above).

Prism tries to smooth over these rough edges, but its not always possible.

### Supported conversions

* Where a provider does not support URLs: Prism will fetch the URL and use base64 or rawContent.
* Where you provide a file, base64 or rawContent: Prism will switch between base64 and rawContent depending on what the provider accepts.

### Limitations

* Where a provider only supports URLs: if you provide a file path, raw contents or base64, for security reasons Prism does not create a URL for you and your request will fail.

---

---
url: /getting-started/installation.md
---
# Installation

Getting started with Prism is a breeze.

## Requirements

Before we dive in, make sure your project meets these requirements:

* PHP 8.2 or higher
* Laravel 11.0 or higher

## Step 1: Composer Installation

::: tip
Prism is actively evolving. To prevent unexpected issues from breaking changes, we strongly recommend pinning your installation to a specific version. Example: "prism-php/prism": "^0.3.0".
:::

First, let's add Prism to your project using Composer. Open your terminal, navigate to your project directory, and run:

```bash
composer require prism-php/prism
```

This command will download Prism and its dependencies into your project.

## Step 2: Publish the Configuration

Prism comes with a configuration file that you'll want to customize. Publish it to your config directory by running:

```bash
php artisan vendor:publish --tag=prism-config
```

This will create a new file at `config/prism.php`. We'll explore how to configure Prism in the next section.

---

---
url: /getting-started/introduction.md
---
# Introduction

Large Language Models (LLMs) have revolutionized how we interact with artificial intelligence, enabling applications to understand, generate, and manipulate human language with unprecedented sophistication. These powerful models open up exciting possibilities for developers, from creating chatbots and content generators to building complex AI-driven applications.

Prism **simplifies the process of integrating LLMs into your Laravel projects**, providing a unified interface to work with various AI providers. This allows you to focus on crafting innovative AI features for your users, rather than getting bogged down in the intricacies of different APIs and implementation details.

Here's a quick example of how you can generate text using Prism:

::: code-group

```php [Anthropic]
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;

$response = Prism::text()
    ->using(Provider::Anthropic, 'claude-3-7-sonnet-latest')
    ->withSystemPrompt(view('prompts.system'))
    ->withPrompt('Explain quantum computing to a 5-year-old.')
    ->asText();

echo $response->text;
```

```php [Mistral]
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;

$response = Prism::text()
    ->using(Provider::Mistral, 'mistral-medium')
    ->withSystemPrompt(view('prompts.system'))
    ->withPrompt('Explain quantum computing to a 5-year-old.')
    ->asText();

echo $response->text;
```

```php [Ollama]
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;

$response = Prism::text()
    ->using(Provider::Ollama, 'llama2')
    ->withSystemPrompt(view('prompts.system'))
    ->withPrompt('Explain quantum computing to a 5-year-old.')
    ->asText();

echo $response->text;
```

```php [OpenAI]
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;

$response = Prism::text()
    ->using(Provider::OpenAI, 'gpt-4')
    ->withSystemPrompt(view('prompts.system'))
    ->withPrompt('Explain quantum computing to a 5-year-old.')
    ->asText();

echo $response->text;
```

:::

Prism draws significant inspiration from the [Vercel AI SDK](https://sdk.vercel.ai/docs/ai-sdk-core), adapting its powerful concepts and developer-friendly approach to the Laravel ecosystem.

## Key Features

* **Unified Provider Interface**: Switch seamlessly between AI providers like OpenAI, Anthropic, and Ollama without changing your application code.
* **Tool System**: Extend AI capabilities by defining custom tools that can interact with your application's business logic.
* **Image Support**: Work with multi-modal models that can process both text and images.

Prism also provides a fluent `prism()` helper function to resolve the `Prism` instance from the application container.

```php
prism()
    ->text()
    ->using(Provider::OpenAI, 'gpt-4')
    ->withPrompt('Explain quantum computing to a 5-year-old.')
    ->asText();
```

## Providers

We currently offer first-party support for these leading AI providers:

* [Anthropic](/providers/anthropic.md)
* [DeepSeek](/providers/deepseek.md)
* [Groq](/providers/groq.md)
* [Mistral](/providers/mistral.md)
* [Ollama](/providers/ollama.md)
* [OpenAI](/providers/openai.md)
* [xAI](/providers/xai.md)
* [Perplexity](/providers/perplexity.md)

Each provider brings its own strengths to the table, and Prism makes it easy to use them all through a consistent, elegant interface.

## Provider Support

Make sure you check the dedicated provider pages for considerations, limitations, and options. Support may be model dependant, check with your provider for model specific features and support.

---

---
url: /providers/mistral.md
---
# Mistral

## Configuration

```php
'mistral' => [
    'api_key' => env('MISTRAL_API_KEY', ''),
    'url' => env('MISTRAL_URL', 'https://api.mistral.ai/v1'),
],
```

## Provider-specific options

## Reasoning Models

### Using Reasoning Models

Simply specify a reasoning model when making your request. The thinking process is automatically included in the response:

```php
use Prism\Prism\Facades\Prism;

$response = Prism::text()
    ->using('mistral', 'magistral-medium-latest')
    ->withPrompt('What is the capital of France?')
    ->asText();

// Access the final answer
echo $response->text;
// "The capital of France is Paris."

// Access the reasoning process
echo $response->additionalContent['thinking'];
// "Okay, the user asked about the capital of France. I know that the capital of France is Paris..."
```

### Accessing Thinking Content

You can access the thinking content via the `additionalContent` property on either the Response or the relevant Step.

On the Response (easiest when not using tools):

```php
$response = Prism::text()
    ->using('mistral', 'magistral-medium-latest')
    ->withPrompt('What is the meaning of life in popular fiction?')
    ->asText();

// Get the reasoning process
$thinking = $response->additionalContent['thinking'];

// Get the final answer
$answer = $response->text;
```

On the Step (necessary when using tools, as reasoning happens before tool calls):

```php
$tools = [
    Tool::as('search')
        ->for('Search for current information')
        ->withStringParameter('query', 'The search query')
        ->using(fn (string $query): string => 'The Tigers game is at 3pm'),
];

$response = Prism::text()
    ->using('mistral', 'magistral-medium-latest')
    ->withTools($tools)
    ->withMaxSteps(3)
    ->withPrompt('What time is the Tigers game today?')
    ->asText();

// Access thinking from the first step
$thinking = $response->steps->first()->additionalContent['thinking'];
```

### Understanding the Response Structure

Reasoning models return content in a structured format with two types of blocks:

1. **Thinking blocks** - The model's internal reasoning process (stored in `additionalContent['thinking']`)
2. **Text blocks** - The final self-contained answer (stored in `text`)

Prism automatically separates these for you, making both easily accessible.

## Streaming

Mistral supports streaming responses in real-time. All standard streaming methods are supported:

```php
return Prism::text()
    ->using('mistral', 'mistral-large-latest')
    ->withPrompt(request('message'))
    ->asEventStreamResponse();
```

For complete streaming documentation, see [Streaming Output](/core-concepts/streaming-output).

## Audio Processing

Mistral provides advanced speech-to-text capabilities through their Voxtral models, offering state-of-the-art transcription accuracy with native multilingual support and audio understanding features.

### Speech-to-Text

Mistral's Voxtral models deliver exceptional speech recognition performance, outperforming industry standards like Whisper large-v3 across multiple languages and acoustic environments.

#### Basic STT Usage

```php
use Prism\Prism\ValueObjects\Media\Audio;

$audioFile = Audio::fromPath('/path/to/recording.mp3');

$response = Prism::audio()
    ->using('mistral', 'voxtral-mini-2507')
    ->withInput($audioFile)
    ->asText();

echo "Transcription: " . $response->text;
```

#### Model Selection Guide

Choose the right Voxtral model for your use case:

```php
// For production transcription with highest accuracy
$response = Prism::audio()
    ->using('mistral', 'voxtral-small-latest')
    ->withInput($audioFile)
    ->asText();

// For efficient transcription and edge deployment
$response = Prism::audio()
    ->using('mistral', 'voxtral-mini-latest')
    ->withInput($audioFile)
    ->asText();

// For optimized transcription-only service
$response = Prism::audio()
    ->using('mistral', 'voxtral-mini-2507')
    ->withInput($audioFile)
    ->asText();
```

#### Language Detection and Specification

Voxtral automatically detects languages or you can specify them for better accuracy:

```php
$response = Prism::audio()
    ->using('mistral', 'voxtral-mini-2507')
    ->withInput($audioFile)
    ->withProviderOptions([
        'language' => 'en',           // ISO-639-1 code (optional)
        'temperature' => 0.0,         // Lower for more deterministic results
    ])
    ->asText();

// Multilingual support - single model handles multiple languages
$response = Prism::audio()
    ->using('mistral', 'voxtral-mini-2507')
    ->withInput($multilingualAudioFile)
    ->withProviderOptions([
        // Auto-detection works well for mixed-language content
        'temperature' => 0.1,
    ])
    ->asText();
```

#### Timestamps and Segmentation

Get detailed timing information with your transcriptions:

```php
$response = Prism::audio()
    ->using('mistral', 'voxtral-mini-2507')
    ->withInput($audioFile)
    ->withProviderOptions([
        'timestamp_granularities' => ['segment'], // Available: word, segment
        'response_format' => 'json',
    ])
    ->asText();

// Access segment information with timestamps
$segments = $response->additionalContent['segments'] ?? [];
foreach ($segments as $segment) {
    echo "Text: " . $segment['text'] . "\n";
    echo "Start: " . $segment['start'] . "s\n";
    echo "End: " . $segment['end'] . "s\n";
}
```

#### Context and Prompts

Improve transcription accuracy with contextual information:

```php
$response = Prism::audio()
    ->using('mistral', 'voxtral-mini-2507')
    ->withInput($audioFile)
    ->withProviderOptions([
        'prompt' => 'This is a medical consultation discussing patient symptoms and treatment options.',
        'language' => 'en',
        'temperature' => 0.0,         // Deterministic for medical content
    ])
    ->asText();
```

#### Long-form Audio Processing

Voxtral handles extended audio without chunking:

```php
// Process up to 30 minutes of audio in a single request
$longAudioFile = Audio::fromPath('/path/to/long_meeting.wav');

$response = Prism::audio()
    ->using('mistral', 'voxtral-small-latest')
    ->withInput($longAudioFile)
    ->withProviderOptions([
        'timestamp_granularities' => ['segment'],
        'language' => 'en',
    ])
    ->asText();

echo "Full transcription: " . $response->text;

// Access usage information
if ($response->usage) {
    echo "Audio duration: " . $response->usage->promptTokens . " tokens\n";
    echo "Total tokens: " . $response->usage->totalTokens . "\n";
}
```

#### Creating Audio Objects

```php
use Prism\Prism\ValueObjects\Media\Audio;

// From local file path
$audio = Audio::fromPath('/path/to/audio.mp3');

// From remote URL
$audio = Audio::fromUrl('https://example.com/recording.wav');

// From base64 encoded data
$audio = Audio::fromBase64($base64AudioData, 'audio/mpeg');

// From binary content
$audioContent = file_get_contents('/path/to/audio.wav');
$audio = Audio::fromContent($audioContent, 'audio/wav');
```

## Documents

The text generation part of the exposed Facade only allows documents to be passed in through via URL.
See the [documents](./../input-modalities/documents.md) on how to do that.

## OCR

Mistral provides an OCR endpoint which can be used to extract text from documents.
This OCR endpoint can be used like this:

```php
use Prism\Prism\Enums\Provider;
use Prism\Prism\Facades\Prism;
use Prism\Prism\Tool;
use Prism\Prism\ValueObjects\Messages\UserMessage;
use Prism\Prism\ValueObjects\Messages\SystemMessage;
use Prism\Prism\Providers\Mistral\Mistral;
use Prism\Prism\Providers\Mistral\ValueObjects\OCRResponse;

/** @var Mistral $provider */
$provider = Prism::provider(\Prism\Prism\Enums\Provider::Mistral);

/** @var OCRResponse $ocrResponse */
$ocrResponse = $provider->ocr(
    'mistral-ocr-latest',
    Document::fromUrl('https://prismphp.com/storage/prism-text-generation.pdf')
);

/**
* Just need the full text of all the pages combined? Use the toText() method.
 */
$text = $ocrResponse->toText();
```

::: tip
The OCR endpoint response time can vary depending on the size of the document. We recommend doing this in the background like a queue with a longer timeout.
:::

---

---
url: /core-concepts/moderation.md
---
# Moderation

Moderate content by checking content against AI-powered models! Moderation helps you detect potentially harmful or inappropriate content before it reaches your users or models.

## Quick Start

Here's how to check text content with just a few lines of code:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;

$response = Prism::moderation()
    ->using(Provider::OpenAI)
    ->withInput('Your text to check goes here')
    ->asModeration();

// Check if any content was flagged
if ($response->isFlagged()) {
    // Handle flagged content
    $flagged = $response->firstFlagged();
}
```

## Checking Multiple Inputs

You can check multiple text inputs at once using the unified `withInput()` method:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;

$response = Prism::moderation()
    ->using(Provider::OpenAI)
    // Multiple inputs as variadic arguments
    ->withInput('First text to check', 'Second text to check', 'Third text')
    ->asModeration();

// Or pass an array
$response = Prism::moderation()
    ->using(Provider::OpenAI)
    ->withInput(['First text', 'Second text', 'Third text'])
    ->asModeration();

// Get all flagged results
$flaggedResults = $response->flagged();

foreach ($flaggedResults as $result) {
    // Handle each flagged result
    $categories = $result->categories;
    $scores = $result->categoryScores;
}
```

## Image Moderation

You can also moderate images! This is useful for checking user-uploaded images for inappropriate content:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\ValueObjects\Media\Image;

$response = Prism::moderation()
    ->using(Provider::OpenAI, 'omni-moderation-latest')
    ->withInput(Image::fromUrl('https://example.com/image.png'))
    ->asModeration();

if ($response->isFlagged()) {
    // Handle flagged image
}
```

### Mixed Text and Image Moderation

You can check both text and images in a single request using the unified `withInput()` method:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\ValueObjects\Media\Image;

// Mix text and images as variadic arguments
$response = Prism::moderation()
    ->using(Provider::OpenAI, 'omni-moderation-latest')
    ->withInput(
        'Check this text',
        Image::fromUrl('https://example.com/image.png'),
        'Another text to check',
        Image::fromLocalPath('/path/to/image1.jpg')
    )
    ->asModeration();

// Or use arrays
$response = Prism::moderation()
    ->using(Provider::OpenAI, 'omni-moderation-latest')
    ->withInput([
        'Text 1',
        Image::fromUrl('https://example.com/image.png'),
        'Text 2',
        Image::fromLocalPath('/path/to/image2.jpg'),
    ])
    ->asModeration();
```

> \[!IMPORTANT]
> **Text as Image Context**: When mixing text and images in a single request, text inputs are treated as context/descriptions for the images, not as separate moderation inputs. This means:
>
> * Multiple text inputs alone will return multiple results (one per text input)
> * Multiple images alone will return multiple results (one per image)
> * Text + Image combinations will return one result per image, with text serving as context for the image
>
> If you need separate moderation results for text and images, make separate API calls for each type.

## Input Methods

Prism provides several methods for adding inputs to moderation requests.

### Using withInput()

The `withInput()` method is the unified way to add any type of input to moderation. It accepts strings, Image objects, or arrays of either as variadic arguments. This is the recommended method for most use cases:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\ValueObjects\Media\Image;

// Single text input
$response = Prism::moderation()
    ->using(Provider::OpenAI)
    ->withInput('Check this text for moderation')
    ->asModeration();

// Multiple text inputs
$response = Prism::moderation()
    ->using(Provider::OpenAI)
    ->withInput('Text 1', 'Text 2', 'Text 3')
    ->asModeration();

// Single image
$response = Prism::moderation()
    ->using(Provider::OpenAI, 'omni-moderation-latest')
    ->withInput(Image::fromUrl('https://example.com/image.png'))
    ->asModeration();

// Multiple images
$response = Prism::moderation()
    ->using(Provider::OpenAI, 'omni-moderation-latest')
    ->withInput([
        Image::fromUrl('https://example.com/image1.png'),
        Image::fromLocalPath('/path/to/image2.jpg'),
    ])
    ->asModeration();

// Mixed text and images
$response = Prism::moderation()
    ->using(Provider::OpenAI, 'omni-moderation-latest')
    ->withInput(
        'Text to check',
        Image::fromUrl('https://example.com/image.png'),
        'More text',
        Image::fromBase64($base64Data, 'image/jpeg')
    )
    ->asModeration();
```

### Image Sources

Images can be created from various sources:

```php
use Prism\Prism\ValueObjects\Media\Image;

// From a URL
Image::fromUrl('https://example.com/image.png')

// From a local file
Image::fromLocalPath('/path/to/image.jpg')

// From a storage disk
Image::fromStoragePath('/path/to/image.jpg', 'my-disk')

// From base64
Image::fromBase64($base64Data, 'image/jpeg')
```

> \[!NOTE]
> Image moderation requires the `omni-moderation-latest` model (or similar image-capable moderation models). Make sure to specify the correct model when using image moderation.

## Response Handling

The moderation response provides everything you need to handle flagged content:

```php
use Prism\Prism\Moderation\Response;
use Prism\Prism\ValueObjects\ModerationResult;

// Check if any content was flagged
if ($response->isFlagged()) {
    // Get the first flagged result
    $firstFlagged = $response->firstFlagged();
    
    // Or get all flagged results
    $allFlagged = $response->flagged();
}

// Access individual results
foreach ($response->results as $result) {
    /** @var ModerationResult $result */
    $isFlagged = $result->flagged;
    $categories = $result->categories; // Array of category => bool
    $categoryScores = $result->categoryScores; // Array of category => float
}

// Access response metadata
$meta = $response->meta;
$model = $meta->model; // The model used for moderation
$id = $meta->id; // Unique identifier for the moderation request
$rateLimits = $meta->rateLimits; // Rate limit information
```

### Understanding Results

Each moderation result includes:

* **`flagged`**: A boolean indicating if the content was flagged as potentially harmful
* **`categories`**: An array mapping category names to boolean values indicating if that category was detected
* **`categoryScores`**: An array mapping category names to float values indicating the confidence level

The response also includes a `meta` object with:

* **`id`**: A unique identifier for the moderation request
* **`model`**: The model used for moderation (e.g., 'omni-moderation-latest')
* **`rateLimits`**: Rate limit information from the API response

```php
$result = $response->results[0];

if ($result->flagged) {
    // Check specific categories
    if ($result->categories['hate'] ?? false) {
        // Handle hate content
    }
    
    if ($result->categories['harassment'] ?? false) {
        // Handle harassment
    }
    
    // Check scores for more nuanced handling
    $hateScore = $result->categoryScores['hate'] ?? 0.0;
    if ($hateScore > 0.5) {
        // High confidence of hate content
    }
}
```

## Common Settings

You can fine-tune your moderation requests just like other Prism features:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;

$response = Prism::moderation()
    ->using(Provider::OpenAI, 'omni-moderation-latest')
    ->withInput('Your text here')
    ->withClientOptions(['timeout' => 30]) // Adjust request timeout
    ->withClientRetry(3, 100) // Add automatic retries
    ->withProviderOptions([
        // Provider-specific options
    ])
    ->asModeration();
```

## Error Handling

Always handle potential errors gracefully:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\Exceptions\PrismException;

try {
    $response = Prism::moderation()
        ->using(Provider::OpenAI)
        ->withInput('Your text here')
        ->asModeration();
        
    if ($response->isFlagged()) {
        // Handle flagged content
    }
} catch (PrismException $e) {
    Log::error('Moderation check failed:', [
        'error' => $e->getMessage()
    ]);
}
```

## Use Cases

Moderation is useful for:

* **User-Generated Content**: Check comments, posts, or messages before displaying them
* **Content Filtering**: Filter out inappropriate content in chat applications
* **Image Moderation**: Verify user-uploaded images meet platform guidelines
* **Pre-Processing**: Check inputs before sending them to other AI models
* **Compliance**: Ensure content meets platform guidelines and policies
* **Mixed Content**: Check both text and images together in a single request

## Pro Tips

**Thresholds**: Use category scores to implement custom thresholds. Different applications may need different sensitivity levels.

**Batch Processing**: Check multiple inputs in a single request for better performance and efficiency.

**Caching**: Consider caching moderation results for repeated content to reduce API calls.

**Logging**: Always log flagged content for audit trails and to improve your filtering over time.

> \[!IMPORTANT]
> Different providers may have different category names and scoring systems. Always check your provider's documentation for specific details about available categories and score interpretations.

---

---
url: /providers/ollama.md
---
# Ollama

## Configuration

```php
'ollama' => [
    'url' => env('OLLAMA_URL', 'http://localhost:11434/v1'),
],
```

## Ollama Options

Ollama allows you to customize how the model is run via [options](https://github.com/ollama/ollama/blob/main/docs/modelfile.md#parameter). These options can be passed via the `->withProviderOptions()` method.

```php
Prism::text() // [!code focus]
  ->using(Provider::Ollama, 'gemma3:1b')
  ->withPrompt('Who are you?')
  ->withClientOptions(['timeout' => 60])
  ->withProviderOptions([ // [!code focus]
      'top_p' => 0.9, // [!code focus]
      'num_ctx' => 4096, // [!code focus]
  ]) // [!code focus]
```

> \[!NOTE]
> Using `withProviderOptions` will override settings like `topP` and `temperature`

## Streaming

Ollama supports streaming responses from your local models. All standard streaming methods are supported:

```php
return Prism::text()
    ->using('ollama', 'llama3.2')
    ->withPrompt(request('message'))
    ->withClientOptions(['timeout' => 120])
    ->asEventStreamResponse();
```

> \[!TIP]
> Remember to increase the timeout for local models to prevent premature disconnection.

For complete streaming documentation, see [Streaming Output](/core-concepts/streaming-output).

## Considerations

### Timeouts

Depending on your configuration, responses tend to time out. You may need to extend the client's timeout using `->withClientOptions(['timeout' => $seconds])`.

```php
Prism::text() // [!code focus]
  ->using(Provider::Ollama, 'gemma3:1b')
  ->withPrompt('Who are you?')
  ->withClientOptions(['timeout' => 60]) // [!code focus]
```

### Structured Output

Ollama doesn't have native JSON mode or structured output like some providers, Prism implements a robust workaround for structured output:

* We automatically append instructions to your prompt that guide the model to output valid JSON matching your schema
* If the response isn't valid JSON, Prism will raise a PrismException

## Limitations

### Image URL

Ollama does not support images using `Image::fromUrl()`.

### Tool Choice

Ollama does not currently support tool choice / required tools.

---

---
url: /providers/openai.md
---
# OpenAI

## Configuration

```php
'openai' => [
    'url' => env('OPENAI_URL', 'https://api.openai.com/v1'),
    'api_key' => env('OPENAI_API_KEY', ''),
    'organization' => env('OPENAI_ORGANIZATION', null),
]
```

## Provider-specific options

### Strict Tool Schemas

Prism supports OpenAI's [function calling with Structured Outputs](https://platform.openai.com/docs/guides/function-calling#function-calling-with-structured-outputs) via provider-specific meta.

```php
Tool::as('search') // [!code focus]
    ->for('Searching the web')
    ->withStringParameter('query', 'the detailed search query')
    ->using(fn (): string => '[Search results]')
    ->withProviderOptions([ // [!code focus]
      'strict' => true, // [!code focus]
    ]); // [!code focus]
```

### Strict Structured Output Schemas

```php
$response = Prism::structured()
    ->withProviderOptions([ // [!code focus]
        'schema' => [ // [!code focus]
            'strict' => true // [!code focus]
        ] // [!code focus]
    ]) // [!code focus]
```

> \[!WARNING]
> **All Fields Must Be Required**: When using structured outputs with OpenAI (especially in strict mode), you must include ALL fields in the `requiredFields` array. Fields that should be optional must be marked with `nullable: true` instead. This is an OpenAI API requirement and applies to all structured output requests.
>
> ```php
> new ObjectSchema(
>     name: 'user',
>     properties: [
>         new StringSchema('email', 'Email address'),
>         new StringSchema('bio', 'Optional bio', nullable: true),
>     ],
>     requiredFields: ['email', 'bio'] // ✅ All fields listed
> );
> ```
>
> For more details on required vs nullable fields, see [Schemas - Required vs Nullable Fields](/core-concepts/schemas#required-vs-nullable-fields).

### Combining Tools with Structured Output

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Schema\ObjectSchema;
use Prism\Prism\Schema\StringSchema;
use Prism\Prism\Tool;

$schema = new ObjectSchema(
    name: 'weather_analysis',
    description: 'Analysis of weather conditions',
    properties: [
        new StringSchema('summary', 'Summary of the weather'),
        new StringSchema('recommendation', 'Recommendation based on weather'),
    ],
    requiredFields: ['summary', 'recommendation']
);

$weatherTool = Tool::as('get_weather')
    ->for('Get current weather for a location')
    ->withStringParameter('location', 'The city and state')
    ->using(fn (string $location): string => "Weather in {$location}: 72°F, sunny");

$response = Prism::structured()
    ->using('openai', 'gpt-4o')
    ->withSchema($schema)
    ->withTools([$weatherTool])
    ->withMaxSteps(3)
    ->withPrompt('What is the weather in San Francisco and should I wear a coat?')
    ->asStructured();

// Access structured output
dump($response->structured);

// Access tool execution details
foreach ($response->toolCalls as $toolCall) {
    echo "Called: {$toolCall->name}\n";
}
```

> \[!IMPORTANT]
> When combining tools with structured output, set `maxSteps` to at least 2. OpenAI automatically uses the `/responses` endpoint and sets `parallel_tool_calls: false`.

For complete documentation on combining tools with structured output, see [Structured Output - Combining with Tools](/core-concepts/structured-output#combining-structured-output-with-tools).

### Metadata

```php
$response = Prism::structured()
    ->withProviderOptions([ // [!code focus]
        'metadata' => [ // [!code focus]
            'project_id' => 23 // [!code focus]
        ] // [!code focus]
    ]) // [!code focus]
```

### Previous Responses

Prism supports OpenAI's [conversation state](https://platform.openai.com/docs/guides/conversation-state#openai-apis-for-conversation-state) with the `previous_response_id` parameter.

```php
$response = Prism::structured()
    ->withProviderOptions([ // [!code focus]
        'previous_response_id' => 'response_id' // [!code focus]
    ]) // [!code focus]
```

### Truncation

```php
$response = Prism::structured()
    ->withProviderOptions([ // [!code focus]
        'truncation' => 'auto' // [!code focus]
    ]) // [!code focus]
```

### Service Tiers

Prism supports OpenAI's [Service Tier Configuration](https://platform.openai.com/docs/api-reference/chat/create#chat-create-service_tier) via provider-specific meta.

```php
$response = Prism::text()
    ->withProviderOptions([ // [!code focus]
        'service_tier' => 'priority' // [!code focus]
    ]) // [!code focus]
```

> \[!WARNING]
> **Priority Service Tiers increase Cost**: Using priority service tier may reduce response time but increases token costs.

### Reasoning Models

OpenAI's reasoning models like `gpt-5`, `gpt-5-mini`, and `gpt-5-nano` use advanced reasoning capabilities to think through complex problems before responding. These models excel at multi-step problem solving, coding, scientific reasoning, and complex analysis tasks.

#### Reasoning Effort

Control how much reasoning the model performs before generating a response using the `reasoning` parameter:

```php
$response = Prism::text()
    ->using('openai', 'gpt-5')
    ->withPrompt('Write a PHP function to implement a binary search algorithm with proper error handling')
    ->withProviderOptions([ // [!code focus]
        'reasoning' => ['effort' => 'high'] // [!code focus]
    ]) // [!code focus]
    ->asText();
```

Available reasoning effort levels:

* **`low`**: Faster responses with economical token usage, suitable for simpler tasks
* **`medium`**: Balanced approach between speed and reasoning depth (default)
* **`high`**: More thorough reasoning for complex problems requiring deep analysis

> \[!NOTE]
> Reasoning models generate internal "reasoning tokens" that help them think through problems. These tokens are included in your usage costs but aren't visible in the response.

#### Reasoning Token Usage

You can track reasoning token usage through the response's usage information:

```php
$response = Prism::text()
    ->using('openai', 'gpt-5-mini')
    ->withPrompt('Refactor this PHP code to use dependency injection')
    ->withProviderOptions([
        'reasoning' => ['effort' => 'medium']
    ])
    ->asText();

// Access reasoning token usage
$usage = $response->firstStep()->usage;
echo "Reasoning tokens: " . $usage->thoughtTokens;
echo "Total completion tokens: " . $usage->completionTokens;
```

#### Text Verbosity

```php
$response = Prism::text()
    ->using('openai', 'gpt-5')
    ->withPrompt('Explain dependency injection')
    ->withProviderOptions([ // [!code focus]
        'text_verbosity' => 'low' // low, medium, high // [!code focus]
    ]) // [!code focus]
    ->asText();
```

#### Store

```php
$response = Prism::text()
    ->using('openai', 'gpt-5')
    ->withPrompt('Give me a summary of the following legal document')
    ->withProviderOptions([ // [!code focus]
        'store' => false // true, false // [!code focus]
    ]) // [!code focus]
    ->asText();
```

## Streaming

OpenAI supports streaming responses in real-time. All the standard streaming methods work with OpenAI models:

```php
// Stream events
$stream = Prism::text()
    ->using('openai', 'gpt-4o')
    ->withPrompt('Write a story')
    ->asStream();

// Server-Sent Events
return Prism::text()
    ->using('openai', 'gpt-4o')
    ->withPrompt(request('message'))
    ->asEventStreamResponse();
```

### Streaming Reasoning Models

Reasoning models like `gpt-5` stream their thinking process separately from the final answer:

```php
use Prism\Prism\Enums\StreamEventType;

foreach ($stream as $event) {
    match ($event->type()) {
        StreamEventType::ThinkingDelta => echo "[Thinking] " . $event->delta,
        StreamEventType::TextDelta => echo $event->delta,
        default => null,
    };
}
```

### Streaming with Provider Tools

OpenAI's provider tools like `image_generation` emit streaming events during execution, letting you track progress and access results in real-time:

```php
use Prism\Prism\ValueObjects\ProviderTool;
use Prism\Prism\Streaming\Events\ProviderToolEvent;

$stream = Prism::text()
    ->using('openai', 'gpt-4o')
    ->withProviderTools([
        new ProviderTool('image_generation'),
    ])
    ->withPrompt('Generate an image of a sunset over mountains')
    ->asStream();

foreach ($stream as $event) {
    if ($event instanceof ProviderToolEvent) {
        // Check when image generation completes
        if ($event->status === 'completed' && isset($event->data['result'])) {
            $imageData = $event->data['result']; // base64 PNG
            file_put_contents('generated.png', base64_decode($imageData));
        }
    }
}
```

For complete details on handling provider tool events, see [Streaming Output](/core-concepts/streaming-output).

### Caching

Automatic caching does not currently work with JsonMode. Please ensure you use StructuredMode if you wish to utilise automatic caching.

## Provider Tools

OpenAI offers built-in provider tools that can be used alongside your custom tools. These tools are executed by OpenAI's infrastructure and provide specialized capabilities. For more information about the difference between custom tools and provider tools, see [Tools & Function Calling](/core-concepts/tools-function-calling#provider-tools).

### Code Interpreter

The OpenAI code interpreter allows your AI to execute Python code in a secure, sandboxed environment. This is particularly useful for mathematical calculations, data analysis, and code execution tasks.

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\ValueObjects\ProviderTool;

Prism::text()
    ->using('openai', 'gpt-4.1')
    ->withPrompt('Solve the equation 3x + 10 = 14.')
    ->withProviderTools([
        new ProviderTool(type: 'code_interpreter', options: ['container' => ['type' => 'auto']])
    ])
    ->asText();
```

#### Configuration Options

* **container**: Configure the execution environment
  * `type`: Set to `'auto'` for automatic environment selection

## Additional Message Attributes

Adding optional parameters to a `UserMessage` like the `name` field can be done through the `additionalAttributes` parameter.

```php
Prism::text()
    ->using('openai', 'gpt-4.1')
    ->withMessages([
        new UserMessage('Who are you?', additionalAttributes: ['name' => 'TJ']),
    ])
    ->asText()
```

## Image Generation

OpenAI provides powerful image generation capabilities through multiple models. Prism supports all of OpenAI's image generation models with their full feature sets.

### Supported Models

| Model | Description |
|-------|-------------|
| `dall-e-3` | Latest DALL-E model |
| `dall-e-2` | Previous generation |
| `gpt-image-1` | GPT-based image model |

### Basic Usage

```php
$response = Prism::image()
    ->using('openai', 'dall-e-3')
    ->withPrompt('A serene mountain landscape at sunset')
    ->generate();

$image = $response->firstImage();
echo $image->url; // Generated image URL
```

### DALL-E 3 Options

DALL-E 3 is the most advanced model with the highest quality output:

```php
$response = Prism::image()
    ->using('openai', 'dall-e-3')
    ->withPrompt('A futuristic cityscape with flying cars')
    ->withProviderOptions([
        'size' => '1792x1024',          // 1024x1024, 1024x1792, 1792x1024
        'quality' => 'hd',              // standard, hd
        'style' => 'vivid',             // vivid, natural
    ])
    ->generate();

// DALL-E 3 automatically revises prompts for better results
if ($response->firstImage()->hasRevisedPrompt()) {
    echo "Revised prompt: " . $response->firstImage()->revisedPrompt;
}
```

### DALL-E 2 Options

DALL-E 2 supports generating multiple images and is more cost-effective:

```php
$response = Prism::image()
    ->using('openai', 'dall-e-2')
    ->withPrompt('Abstract geometric patterns')
    ->withProviderOptions([
        'n' => 4,                       // Number of images (1-10)
        'size' => '1024x1024',          // 256x256, 512x512, 1024x1024
        'response_format' => 'url',     // url only
        'user' => 'user-123',           // Optional user identifier
    ])
    ->generate();

// Process multiple images
foreach ($response->images as $image) {
    echo "Image: {$image->url}\n";
}
```

### GPT-Image-1 Options

GPT-Image-1 offers advanced features including image editing and format control:

```php
$response = Prism::image()
    ->using('openai', 'gpt-image-1')
    ->withPrompt('A detailed architectural rendering of a modern house')
    ->withProviderOptions([
        'size' => '1536x1024',              // Various sizes supported
        'quality' => 'high',                // standard, high
        'output_format' => 'webp',          // png, webp, jpeg
        'output_compression' => 85,         // Compression level (0-100)
        'background' => 'transparent',      // transparent, white, black
        'moderation' => true,               // Enable content moderation
    ])
    ->generate();
```

### Image Editing with GPT-Image-1

GPT-Image-1 supports sophisticated image editing operations using the `withPrompt` method with Image value objects:

```php
use Prism\Prism\ValueObjects\Media\Image;

$response = Prism::image()
    ->using('openai', 'gpt-image-1')
    ->withPrompt('Add a vaporwave sunset to the background', [
        Image::fromLocalPath('tests/Fixtures/diamond.png'),
    ])
    ->withProviderOptions([
        'size' => '1024x1024',
        'output_format' => 'png',
        'quality' => 'high',
    ])
    ->withClientOptions(['timeout' => 9999])
    ->generate();

file_put_contents('edited-image.png', base64_decode($response->firstImage()->base64));
```

#### Using Masks for Targeted Editing

For precise control over which parts of the image to edit, use a mask image:

```php
$response = Prism::image()
    ->using('openai', 'gpt-image-1')
    ->withPrompt('Add a vaporwave sunset to the background', [
        Image::fromLocalPath('tests/Fixtures/diamond.png'),
    ])
    ->withProviderOptions([
        'mask' => Image::fromLocalPath('tests/Fixtures/diamond-mask.png'),
        'size' => '1024x1024',
        'output_format' => 'png',
        'quality' => 'high',
    ])
    ->generate();
```

#### Editing with Multiple Images

You can also edit with multiple images for more complex operations. Use the `as()` method to provide custom filenames for better readability:

```php
$response = Prism::image()
    ->using('openai', 'gpt-image-1')
    ->withPrompt('Combine these images with a futuristic theme', [
        Image::fromLocalPath('tests/Fixtures/diamond.png')->as('diamond.png'),
        Image::fromLocalPath('tests/Fixtures/sunset.png')->as('sunset-background.png'),
    ])
    ->withProviderOptions([
        'size' => '1024x1024',
        'output_format' => 'png',
        'quality' => 'high',
    ])
    ->generate();
```

### Response Format

Generated images are returned as URLs:

```php
$response = Prism::image()
    ->using('openai', 'dall-e-3')
    ->withPrompt('Digital artwork')
    ->generate();

$image = $response->firstImage();
if ($image->hasUrl()) {
    echo "<img src='{$image->url}' alt='Generated image'>";
}
```

## Audio Processing

OpenAI provides comprehensive audio processing capabilities through their TTS (Text-to-Speech) and Whisper (Speech-to-Text) models. Prism supports all of OpenAI's audio models with their full feature sets.

### Text-to-Speech

Convert text into natural-sounding speech with various voice options:

#### Basic TTS Usage

```php
use Prism\Prism\Facades\Prism;

$response = Prism::audio()
    ->using('openai', 'gpt-4o-mini-tts')
    ->withInput('Hello, welcome to our application!')
    ->withVoice('alloy')
    ->asAudio();

// Save the audio file
$audioData = base64_decode($response->audio->base64);
file_put_contents('welcome.mp3', $audioData);
```

#### High-Definition Audio

For higher quality audio output, use the model:

```php
$response = Prism::audio()
    ->using('openai', 'gpt-4o-mini-tts')
    ->withInput('This is high-quality audio generation.')
    ->withProviderOptions([
        'voice' => 'nova',
        'response_format' => 'wav',    // Higher quality format
    ])
    ->asAudio();
```

#### Audio Format Options

Control the output format and quality:

```php
$response = Prism::audio()
    ->using('openai', 'gpt-4o-mini-tts')
    ->withInput('Testing different audio formats.')
    ->withProviderOptions([
        'voice' => 'echo',
        'response_format' => 'opus',   // mp3, opus, aac, flac, wav, pcm
        'speed' => 1.25,              // Speed: 0.25 to 4.0
    ])
    ->asAudio();

echo "Audio type: " . $response->audio->getMimeType();
```

For more information on the available options, please refer to the [OpenAI API documentation](https://platform.openai.com/docs/guides/text-to-speech).

### Speech-to-Text

Convert audio files into accurate text transcriptions using Whisper:

#### Basic STT Usage

```php
use Prism\Prism\ValueObjects\Media\Audio;

$audioFile = Audio::fromPath('/path/to/recording.mp3');

$response = Prism::audio()
    ->using('openai', 'whisper-1')
    ->withInput($audioFile)
    ->asText();

echo "Transcription: " . $response->text;
```

#### Language Detection

Whisper can automatically detect the language or you can specify it:

```php
$response = Prism::audio()
    ->using('openai', 'whisper-1')
    ->withInput($audioFile)
    ->withProviderOptions([
        'language' => 'es',           // ISO-639-1 code (optional)
        'temperature' => 0.2,         // Lower temperature for more focused results
    ])
    ->asText();
```

#### Response Formats

Get transcriptions in different formats with varying detail levels:

```php
// Standard JSON response
$response = Prism::audio()
    ->using('openai', 'whisper-1')
    ->withInput($audioFile)
    ->withProviderOptions([
        'response_format' => 'json',  // json, text, srt, verbose_json, vtt
    ])
    ->asText();

// Verbose JSON includes timestamps and confidence scores
$response = Prism::audio()
    ->using('openai', 'whisper-1')
    ->withInput($audioFile)
    ->withProviderOptions([
        'response_format' => 'verbose_json',
    ])
    ->asText();

// Access detailed segment information
$segments = $response->additionalContent['segments'] ?? [];
foreach ($segments as $segment) {
    echo "Text: " . $segment['text'] . "\n";
    echo "Start: " . $segment['start'] . "s\n";
    echo "End: " . $segment['end'] . "s\n";
    echo "Confidence: " . ($segment['no_speech_prob'] ?? 'N/A') . "\n\n";
}
```

#### Subtitle Generation

Generate subtitle files directly:

```php
// SRT format subtitles
$response = Prism::audio()
    ->using('openai', 'whisper-1')
    ->withInput($audioFile)
    ->withProviderOptions([
        'response_format' => 'srt',
    ])
    ->asText();

file_put_contents('subtitles.srt', $response->text);

// VTT format subtitles
$response = Prism::audio()
    ->using('openai', 'whisper-1')
    ->withInput($audioFile)
    ->withProviderOptions([
        'response_format' => 'vtt',
    ])
    ->asText();

file_put_contents('subtitles.vtt', $response->text);
```

#### Context and Prompts

Improve transcription accuracy with context:

```php
$response = Prism::audio()
    ->using('openai', 'whisper-1')
    ->withInput($audioFile)
    ->withProviderOptions([
        'prompt' => 'This is a technical discussion about machine learning and artificial intelligence.',
        'language' => 'en',
        'temperature' => 0.1,         // Lower temperature for technical content
    ])
    ->asText();
```

### Audio File Handling

#### Creating Audio Objects

Load audio from various sources:

```php
use Prism\Prism\ValueObjects\Media\Audio;

// From local file path
$audio = Audio::fromPath('/path/to/audio.mp3');

// From remote URL
$audio = Audio::fromUrl('https://example.com/recording.wav');

// From base64 encoded data
$audio = Audio::fromBase64($base64AudioData, 'audio/mpeg');

// From binary content
$audioContent = file_get_contents('/path/to/audio.wav');
$audio = Audio::fromContent($audioContent, 'audio/wav');
```

#### File Size Considerations

Whisper has a file size limit of 25 MB. For larger files, consider:

```php
// Check file size before processing
$audio = Audio::fromPath('/path/to/large-audio.mp3');

if ($audio->size() > 25 * 1024 * 1024) { // 25 MB
    echo "File too large for processing";
} else {
    $response = Prism::audio()
        ->using('openai', 'whisper-1')
        ->withInput($audio)
        ->asText();
}
```

For more information on the available options, please refer to the [OpenAI API documentation](https://platform.openai.com/docs/guides/speech-to-text).

## Moderation

OpenAI provides powerful content moderation capabilities through their moderation API. Prism supports both text and image moderation with OpenAI.

### Supported Models

| Model | Description |
|-------|-------------|
| `omni-moderation-latest` | Latest moderation model supporting both text and images |

### Text Moderation

Check text content for potentially harmful or inappropriate material:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;

$response = Prism::moderation()
    ->using(Provider::OpenAI)
    ->withInput('Your text to check goes here')
    ->asModeration();

if ($response->isFlagged()) {
    $flagged = $response->firstFlagged();
    // Handle flagged content
}
```

### Image Moderation

Moderate images using the `omni-moderation-latest` model:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\ValueObjects\Media\Image;

$response = Prism::moderation()
    ->using(Provider::OpenAI, 'omni-moderation-latest')
    ->withInput(Image::fromUrl('https://example.com/image.png'))
    ->asModeration();

if ($response->isFlagged()) {
    // Handle flagged image
}
```

### Mixed Text and Image Moderation

You can check both text and images in a single request:

```php
$response = Prism::moderation()
    ->using(Provider::OpenAI, 'omni-moderation-latest')
    ->withInput(
        'Check this text',
        Image::fromStoragePath('uploads/user-photo.jpg', 'public'),
        'Another text to check',
        Image::fromUrl('https://example.com/image.png')
    )
    ->asModeration();
```

> \[!NOTE]
> When mixing text and images in a single request, text inputs are treated as context/descriptions for the images, not as separate moderation inputs. If you need separate moderation results for text and images, make separate API calls for each type.

### Multiple Inputs

Check multiple inputs at once:

```php
// Multiple text inputs
$response = Prism::moderation()
    ->using(Provider::OpenAI)
    ->withInput('Text 1', 'Text 2', 'Text 3')
    ->asModeration();

// Multiple images
$response = Prism::moderation()
    ->using(Provider::OpenAI, 'omni-moderation-latest')
    ->withInput([
        Image::fromUrl('https://example.com/image1.png'),
        Image::fromStoragePath('uploads/image2.jpg', 'public'),
    ])
    ->asModeration();
```

### Response Handling

Access moderation results and category information:

```php
$response = Prism::moderation()
    ->using(Provider::OpenAI, 'omni-moderation-latest')
    ->withInput('Your content here')
    ->asModeration();

// Check if any content was flagged
if ($response->isFlagged()) {
    // Get all flagged results
    $flaggedResults = $response->flagged();
    
    foreach ($flaggedResults as $result) {
        // Access categories
        $categories = $result->categories; // Array of category => bool
        $scores = $result->categoryScores; // Array of category => float
        
        // Check specific categories
        if ($result->categories['hate'] ?? false) {
            // Handle hate content
        }
    }
}
```

For complete moderation documentation, including all available options and use cases, see [Moderation](/core-concepts/moderation).

---

---
url: /providers/openrouter.md
---
# OpenRouter

OpenRouter provides access to multiple AI models through a single API. This provider allows you to use various models from different providers through OpenRouter's routing system.

## Configuration

Add your OpenRouter configuration to `config/prism.php`:

```php
'providers' => [
    'openrouter' => [
        'api_key' => env('OPENROUTER_API_KEY'),
        'url' => env('OPENROUTER_URL', 'https://openrouter.ai/api/v1'),
        'site' => [
            'http_referer' => env('OPENROUTER_SITE_HTTP_REFERER'),
            'x_title' => env('OPENROUTER_SITE_X_TITLE'),
        ],
    ],
],
```

## Environment Variables

Set your OpenRouter API key and URL in your `.env` file:

```env
OPENROUTER_API_KEY=your_api_key_here
OPENROUTER_URL=https://openrouter.ai/api/v1
OPENROUTER_SITE_HTTP_REFERER=https://your-site.example
OPENROUTER_SITE_X_TITLE="Your Site Name"
```

## Usage

### Text Generation

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;

$response = Prism::text()
    ->using(Provider::OpenRouter, 'openai/gpt-4-turbo')
    ->withPrompt('Tell me a story about AI.')
    ->generate();

echo $response->text;
```

### Structured Output

> \[!NOTE]
> OpenRouter uses OpenAI-compatible structured outputs. For strict schema validation, the root schema should be an `ObjectSchema`.

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\Schema\ObjectSchema;
use Prism\Prism\Schema\StringSchema;

$schema = new ObjectSchema('person', 'Person information', [
    new StringSchema('name', 'The person\'s name'),
    new StringSchema('occupation', 'The person\'s occupation'),
]);

$response = Prism::structured()
    ->using(Provider::OpenRouter, 'openai/gpt-4-turbo')
    ->withPrompt('Generate a person profile for John Doe.')
    ->withSchema($schema)
    ->generate();

echo $response->text;
```

### Tool Calling

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\Tool;

$weatherTool = Tool::as('get_weather')
    ->for('Get the current weather for a location')
    ->withStringParameter('location', 'The location to get weather for')
    ->using(function (string $location) {
        return "The weather in {$location} is sunny and 72°F";
    });

$response = Prism::text()
    ->using(Provider::OpenRouter, 'openai/gpt-4-turbo')
    ->withPrompt('What is the weather like in New York?')
    ->withTools([$weatherTool])
    ->generate();

echo $response->text;
```

### Multimodal Prompts

OpenRouter keeps the OpenAI content-part schema, so you can mix text and images inside a single user turn.

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\ValueObjects\Media\Image;

$response = Prism::text()
    ->using(Provider::OpenRouter, 'openai/gpt-4o-mini')
    ->withPrompt('Describe the key trends in this diagram.', [
        Image::fromLocalPath('storage/charts/retention.png'),
    ])
    ->generate();

echo $response->text;
```

> \[!TIP]
> `Image` value objects are serialized into the `image_url` entries that OpenRouter expects, so you can attach multiple images or pair them with plain text in the same message.

### Documents

OpenRouter supports sending documents (PDFs) to compatible models:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\ValueObjects\Media\Document;

$response = Prism::text()
    ->using(Provider::OpenRouter, 'anthropic/claude-sonnet-4')
    ->withPrompt('Summarize this document.', [
        Document::fromUrl('https://example.com/report.pdf', 'report.pdf'),
    ])
    ->generate();

echo $response->text;
```

> \[!TIP]
> `Document` value objects support URLs and base64-encoded content. File IDs and chunks are not supported via OpenRouter.

### Videos

OpenRouter supports sending video files to compatible models (like Gemini). Videos can be provided as URLs or base64-encoded content:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\ValueObjects\Media\Video;

$response = Prism::text()
    ->using(Provider::OpenRouter, 'google/gemini-3-flash-preview')
    ->withPrompt('Describe what happens in this video.', [
        Video::fromLocalPath('/path/to/video.mp4'),
    ])
    ->generate();

echo $response->text;
```

You can also use YouTube URLs with Gemini models via OpenRouter:

```php
$response = Prism::text()
    ->using(Provider::OpenRouter, 'google/gemini-3-flash-preview')
    ->withPrompt('Summarize this video.', [
        Video::fromUrl('https://www.youtube.com/watch?v=dQw4w9WgXcQ'),
    ])
    ->generate();
```

> \[!NOTE]
> Video support varies by model. Check [OpenRouter's models page](https://openrouter.ai/models?input_modalities=video) for models with video input support.

### Streaming

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\Enums\StreamEventType;

$stream = Prism::text()
    ->using(Provider::OpenRouter, 'openai/gpt-4-turbo')
    ->withPrompt('Tell me a long story about AI.')
    ->asStream();

foreach ($stream as $event) {
    if ($event->type() === StreamEventType::TextDelta) {
        echo $event->delta;
    }
}
```

> \[!NOTE]
> OpenRouter keeps SSE connections alive by emitting comment events such as `: OPENROUTER PROCESSING`. These lines are safe to ignore while parsing the stream.
>
> \[!WARNING]
> Mid-stream failures propagate as normal SSE payloads with `error` details and `finish_reason: "error"` while the HTTP status remains 200. Make sure to inspect each chunk for an `error` field so you can surface failures to the caller and stop reading the stream.

### Streaming with Tools

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\Tool;

$weatherTool = Tool::as('get_weather')
    ->for('Get the current weather for a location')
    ->withStringParameter('location', 'The location to get weather for')
    ->using(function (string $location) {
        return "The weather in {$location} is sunny and 72°F";
    });

$stream = Prism::text()
    ->using(Provider::OpenRouter, 'openai/gpt-4-turbo')
    ->withPrompt('What is the weather like in multiple cities?')
    ->withTools([$weatherTool])
    ->asStream();

foreach ($stream as $event) {
    match ($event->type()) {
        StreamEventType::TextDelta => echo $event->delta,
        StreamEventType::ToolCall => echo "Tool called: {$event->toolName}\n",
        StreamEventType::ToolResult => echo "Tool result: " . json_encode($event->result) . "\n",
        default => null,
    };
}
```

### Reasoning/Thinking Tokens

Some models (like OpenAI's o1 series) support reasoning tokens that show the model's thought process:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\Enums\StreamEventType;

$stream = Prism::text()
    ->using(Provider::OpenRouter, 'openai/o1-preview')
    ->withPrompt('Solve this complex math problem: What is the derivative of x^3 + 2x^2 - 5x + 1?')
    ->asStream();

foreach ($stream as $event) {
    if ($event->type() === StreamEventType::ThinkingDelta) {
        // This is the model's reasoning/thinking process
        echo "Thinking: " . $event->delta . "\n";
    } elseif ($event->type() === StreamEventType::TextDelta) {
        // This is the final answer
        echo $event->delta;
    }
}
```

#### Reasoning Effort

Control how much reasoning the model performs before generating a response using the `reasoning` parameter. The way this is structured depends on the underlying model you are calling:

```php
$response = Prism::text()
    ->using(Provider::OpenRouter, 'openai/gpt-5-mini')
    ->withPrompt('Write a PHP function to implement a binary search algorithm with proper error handling')
    ->withProviderOptions([
        'reasoning' => [
            'effort' => 'high',  // Can be "high", "medium", or "low" (OpenAI-style)
            'max_tokens' =>  2000, // Specific token limit (Gemini / Anthropic-style)
            
            // Optional: Default is false. All models support this.
            'exclude' => false, // Set to true to exclude reasoning tokens from response
            // Or enable reasoning with the default parameters:
            'enabled' => true // Default: inferred from `effort` or `max_tokens`
        ]
    ])
    ->asText();
```

### Provider Routing & Advanced Options

Use `withProviderOptions()` to forward OpenRouter-specific controls such as model preferences or sampling parameters. Prism automatically forwards the native request values for `temperature`, `top_p`, and `max_tokens`, so you can continue tuning them through the usual Prism API without duplicating them in `withProviderOptions()`. For transform pipelines, OpenRouter currently documents `"middle-out"` as the primary example—consult the parameter reference for additional context.

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;

$response = Prism::text()
    ->using(Provider::OpenRouter, 'openai/gpt-4o')
    ->withPrompt('Draft a concise product changelog entry.')
    ->withProviderOptions([
        // https://openrouter.ai/docs/model-routing
        'models' => [
            'anthropic/claude-sonnet-4.5',
            'openai/gpt-4o-mini',
        ],
        'top_k' => 40,
        // Reference: https://openrouter.ai/docs/api-reference/parameters for the full parameter list.
    ])
    ->generate();

echo $response->text;
```

> \[!IMPORTANT]
> The values you supply here are passed directly to OpenRouter. Consult the [Parameters reference](https://openrouter.ai/docs/api-reference/parameters) and [Provider Routing guide](https://openrouter.ai/docs/provider-routing) for the full list of supported keys.

The single `model` parameter and the fallback `models` array work together. When both are present, OpenRouter first tries the `model` value, then walks the `models` list in order—exactly as outlined in the [Model Routing guide](https://openrouter.ai/docs/features/model-routing). Fallbacks trigger for moderation flags, context-length errors, rate limits, or provider downtime, and the final `model` field in the response reveals which entry actually served the request (and therefore which pricing tier applies). If you prefer OpenRouter to choose the initial model, set `model` to `openrouter/auto` and still supply a `models` array for explicit overrides when needed. Because metadata is centralized, you can double-check `supported_parameters`, context length, and per-request limits via the [Models API](https://openrouter.ai/docs/overview/models) before rolling out changes.

## Available Models

OpenRouter supports many models from different providers. The [Models API](https://openrouter.ai/docs/overview/models) returns structured metadata—`supported_parameters`, context length, pricing, and more—so you can verify capabilities programmatically before issuing requests. Some popular options include:

* `x-ai/grok-code-fast-1`
* `anthropic/claude-sonnet-4.5`
* `google/gemini-2.5-flash`
* `deepseek/deepseek-chat-v3-0324`
* `z-ai/glm-4.6`
* `tngtech/deepseek-r1t2-chimera:free`
* `qwen/qwen3-coder-30b-a3b-instruct`
* `mistralai/mistral-nemo`

Visit [OpenRouter's models page](https://openrouter.ai/models) for a complete list of available models.

## Features

* ✅ Text Generation
* ✅ Structured Output
* ✅ Tool Calling
* ✅ Multiple Model Support
* ✅ Provider Routing
* ✅ Streaming
* ✅ Reasoning/Thinking Tokens (for compatible models)
* ✅ Image Support
* ✅ Video Support
* ✅ Document Support
* ❌ Embeddings (not yet implemented)
* ❌ Image Generation (not yet implemented)

## API Reference

For detailed API documentation, visit [OpenRouter's API documentation](https://openrouter.ai/docs/api-reference/chat-completion).

## Error Handling

The OpenRouter provider includes standard error handling for common issues:

* Rate limiting
* Request too large
* Provider overload
* Invalid API key

Errors are automatically mapped to appropriate Prism exceptions for consistent error handling across all providers.

---

---
url: /providers/perplexity.md
---
# Perplexity

## Configuration

```php
'perplexity' => [
    'api_key' => env('PERPLEXITY_API_KEY', ''),
    'url' => env('PERPLEXITY_URL', 'https://api.perplexity.ai'),
]
```

## Documents

Sonar models support document analysis through file uploads. You can provide files either as URLs to publicly accessible documents or as base64 encoded bytes. Ask questions about document content, get summaries, extract information, and perform detailed analysis of uploaded files in multiple formats including PDF, DOC, DOCX, TXT, and RTF.

* The maximum file size is 50MB. Files larger than this limit will not be processed
* Ensure provided HTTPS URLs are publicly accessible
  Check it out the [documentation for more details](https://docs.perplexity.ai/guides/file-attachments)

## Images

Sonar models support image analysis through direct image uploads. You can include images in your API requests to support multi-modal conversations alongside text. Images can be provided either as base64 encoded strings within a data URI or as standard HTTPS URLs.

* When using base64 encoding, the API currently only supports images up to 50 MB per image
* Supported formats for base64 encoded images: PNG (image/png), JPEG (image/jpeg), WEBP (image/webp), and GIF (image/gif)
* When using an HTTPS URL, the model will attempt to fetch the image from the provided URL. Ensure the URL is publicly accessible.

## Considerations

### Message Order

* Message order matters. Perplexity is strict about the message order being:

1. `SystemMessage`
2. `UserMessage`
3. `AssistantMessage`

### Additional fields

Perplexity outputs additional fields in the response, such as `citations`, `search_results`, and the `reasoning` that is extracted from the model response. These fields are exposed in the response object
via the property `additionalFields`. e.g `$response->additionalFields['citations']`.

### Structured Output

Perplexity supports two types of structured outputs: JSON Schema and Regex; but currently Prism only supports JSON Schema.

Here's an example of how to use JSON Schema for structured output:

```php
use Prism\Prism\Enums\Provider;
use Prism\Prism\Facades\Prism;
use Prism\Prism\Schema\ObjectSchema;
use Prism\Prism\Schema\StringSchema;

$response = Prism::structured()
    ->withSchema(new ObjectSchema(
        'weather_report',
        'Weather forecast with recommendations',
        [
            new StringSchema('forecast', 'The weather forecast'),
            new StringSchema('recommendation', 'Clothing recommendation')
        ],
        ['forecast', 'recommendation']
    ))
    ->using(Provider::Perplexity, 'sonar-pro')
    ->withPrompt('What\'s the weather like and what should I wear?')
    ->asStructured();
```

---

---
url: /core-concepts/prism-server.md
---
# Prism Server

Prism Server is a powerful feature that allows you to expose your Prism-powered AI models through a standardized API. This makes it easy to integrate your custom AI solutions into various applications, including chat interfaces and other tools that support OpenAI-compatible APIs.

## How It Works

Prism Server acts as a middleware, translating requests from OpenAI-compatible clients into Prism-specific operations. This means you can use tools like ChatGPT web UIs or any OpenAI SDK to interact with your custom Prism models.

## Setting Up Prism Server

### 1. Enable Prism Server

First, make sure Prism Server is enabled in your `config/prism.php` file:

```php
'prism_server' => [
    // The middleware that will be applied to the Prism Server routes.
    'middleware' => [],
    'enabled' => env('PRISM_SERVER_ENABLED', false),
]
```

### 2. Register Your Prisms

To make your Prism models available through the server, you need to register them. This is typically done in a service provider, such as `AppServiceProvider`:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\Facades\PrismServer;
use Illuminate\Support\ServiceProvider;

class AppServiceProvider extends ServiceProvider
{
    public function boot(): void
    {
        PrismServer::register(
            'my-custom-model',
            fn () => Prism::text()
                ->using(Provider::Anthropic, 'claude-3-5-sonnet-latest')
                ->withSystemPrompt('You are a helpful assistant.')
        );
    }
}
```

In this example, we're registering a model named `my-custom-model` that uses the Anthropic Claude 3 Sonnet model with a custom system message.

## Using Prism Server

Once set up, Prism Server exposes two main endpoints:

### Chat Completions

To generate text using your registered Prism models:

```bash
curl -X POST "http://your-app.com/prism/openai/v1/chat/completions" \
     -H "Content-Type: application/json" \
     -d '{
  "model": "my-custom-model",
  "messages": [
    {"role": "user", "content": "Hello, who are you?"}
  ]
}'
```

### List Available Models

To get a list of all registered Prism models:

```bash
curl "http://your-app.com/prism/openai/v1/models"
```

## Integration with Open WebUI

Prism Server works seamlessly with OpenAI-compatible chat interfaces like [Open WebUI](https://openwebui.com). Here's an example Docker Compose configuration:

```yaml
services:
  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    ports:
      - "3000:8080"
    environment:
      OPENAI_API_BASE_URLS: "http://laravel:8080/prism/openai/v1"
      WEBUI_SECRET_KEY: "your-secret-key"

  laravel:
    image: serversideup/php:8.3-fpm-nginx
    volumes:
      - ".:/var/www/html"
    environment:
      OPENAI_API_KEY: ${OPENAI_API_KEY}
      ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY}
    depends_on:
      - open-webui
```

With this setup, you can access your Prism models through a user-friendly chat interface at `http://localhost:3000`.

By leveraging Prism Server, you can create powerful, custom AI experiences while maintaining compatibility with a wide ecosystem of tools and libraries. Whether you're building a chatbot, a content generation tool, or something entirely new, Prism Server provides the flexibility and standardization you need to succeed.

## Adding Middleware

You can add middleware to the Prism Server routes by setting the `middleware` option in your `config/prism.php` file:

```php
'prism_server' => [
    'middleware' => ['api'],
],
```

---

---
url: /advanced/provider-interoperability.md
---
# Provider Interoperability

When working with Prism, you might need to customize requests based on which provider you're using. Different providers have unique capabilities, configuration options, and requirements that can affect how you structure your requests for optimal results.

## Using the `whenProvider` Method

The `whenProvider` method lets you easily customize your requests for specific providers while maintaining clean, readable code.

```php
$response = Prism::text()
    ->using(Provider::OpenAI, 'gpt-4o')
    ->withPrompt('Who are you?')
    ->whenProvider(
        Provider::Anthropic,
        fn ($request) => $request
            ->withProviderOptions([
                'cacheType' => 'ephemeral',
            ])
    )
    ->asText();
```

In this example, the `withProviderOptions` settings will only be applied when using Anthropic's provider. If you're using OpenAI (as specified in the `using` method), these customizations are simply skipped.

## Key Benefits

* **Cleaner Code**: Keep your provider-specific customizations encapsulated and only apply them when needed
* **Easy Provider Switching**: Swap between providers without rewriting your configuration code
* **Maintainable Applications**: Define provider-specific behaviors in one place

## Advanced Usage

You can chain multiple `whenProvider` calls to handle different provider scenarios:

```php
$response = Prism::text()
    ->using(Provider::OpenAI, 'gpt-4o')
    ->withPrompt('Generate a creative story about robots.')
    ->whenProvider(
        Provider::Anthropic,
        fn ($request) => $request
            ->withMaxTokens(4000)
            ->withProviderOptions(['cacheType' => 'ephemeral'])
    )
    ->whenProvider(
        Provider::OpenAI,
        fn ($request) => $request
            ->withMaxTokens(2000)
            ->withProviderOptions(['response_format' => ['type' => 'text']])
    )
    ->asText();
```

## Using Invokable Classes

For more complex provider-specific configurations, you can use invokable classes instead of closures:

```php
class AnthropicConfigurator
{
    public function __invoke($request)
    {
        return $request
            ->withMaxTokens(4000)
            ->withProviderOptions([
                'cacheType' => 'ephemeral',
                'citations' => true,
            ]);
    }
}

$response = Prism::text()
    ->using(Provider::Anthropic, 'claude-3-sonnet')
    ->withPrompt('Explain the theory of relativity.')
    ->whenProvider(Provider::Anthropic, new AnthropicConfigurator())
    ->asText();
```

This approach can be especially helpful when you have complex or reusable provider configurations.

> \[!TIP]
> The `whenProvider` method works with all request types in Prism including text, structured output, and embeddings requests.

## Best Practices

### Avoiding SystemMessages with Multiple Providers

When working with multiple providers, it's best to avoid using `SystemMessages` directly in your `withMessages` array. Instead, use the `withSystemPrompt` method as it offers better provider interoperability.

```php
// Avoid this when switching between providers
$response = Prism::text()
    ->using(Provider::OpenAI, 'gpt-4o')
    ->withMessages([
        new SystemMessage('You are a helpful assistant.'),
        new UserMessage('Tell me about AI'),
    ])
    ->asText();

// Prefer this instead
$response = Prism::text()
    ->using(Provider::OpenAI, 'gpt-4o')
    ->withSystemPrompt('You are a helpful assistant.')
    ->withPrompt('Tell me about AI')
    ->asText();
```

This approach allows Prism to handle the provider-specific formatting of system messages, making your code more portable across different LLM providers.

---

---
url: /core-concepts/schemas.md
---
# Schemas

Schemas are the blueprints that help you define the shape of your data in Prism. Whether you're building tool parameters or crafting structured outputs, schemas help you clearly communicate what your data should look like.

## Quick Start

Let's dive right in with a practical example:

> \[!IMPORTANT]
> **Structured Output Requirement**: When using schemas for structured output with providers like OpenAI (especially in strict mode), the root schema should be an `ObjectSchema`. Other schema types can only be used as properties within an ObjectSchema, not as the top-level schema. Different providers may have varying requirements.

```php
use Prism\Prism\Schema\ArraySchema;
use Prism\Prism\Schema\ObjectSchema;
use Prism\Prism\Schema\StringSchema;

$userSchema = new ObjectSchema(
    name: 'user',
    description: 'A user profile with their hobbies',
    properties: [
        new StringSchema('name', 'The user\'s full name'),
        new ArraySchema(
            name: 'hobbies',
            description: 'The user\'s list of hobbies',
            items: new ObjectSchema(
                name: 'hobby',
                description: 'A detailed hobby entry',
                properties: [
                    new StringSchema('name', 'The name of the hobby'),
                    new StringSchema('description', 'A brief description of the hobby'),
                ],
                requiredFields: ['name', 'description']
            )
        ),
    ],
    requiredFields: ['name', 'hobbies']
);
```

## Available Schema Types

### StringSchema

For text values of any length. Perfect for names, descriptions, or any textual data.

```php
use Prism\Prism\Schema\StringSchema;

$nameSchema = new StringSchema(
    name: 'full_name',
    description: 'The user\'s full name including first and last name'
);
```

### NumberSchema

Handles both integers and floating-point numbers. Great for ages, quantities, or measurements.

```php
use Prism\Prism\Schema\NumberSchema;

$ageSchema = new NumberSchema(
    name: 'age',
    description: 'The user\'s age in years'
);
```

### BooleanSchema

For simple true/false values. Perfect for flags and toggles.

```php
use Prism\Prism\Schema\BooleanSchema;

$activeSchema = new BooleanSchema(
    name: 'is_active',
    description: 'Whether the user account is active'
);
```

### ArraySchema

For lists of items, where each item follows a specific schema.

```php
use Prism\Prism\Schema\ArraySchema;
use Prism\Prism\Schema\StringSchema;

$tagsSchema = new ArraySchema(
    name: 'tags',
    description: 'List of tags associated with the post',
    items: new StringSchema('tag', 'A single tag')
);
```

### EnumSchema

When you need to restrict values to a specific set of options.

```php
use Prism\Prism\Schema\EnumSchema;

$statusSchema = new EnumSchema(
    name: 'status',
    description: 'The current status of the post',
    options: ['draft', 'published', 'archived']
);
```

### ObjectSchema

For complex, nested data structures. The Swiss Army knife of schemas!

> \[!NOTE]
> ObjectSchema is typically required as the root schema for structured output operations with providers like OpenAI. It's the recommended schema type to use directly with `withSchema()` in structured output requests, though different providers may have varying requirements.

```php
use Prism\Prism\Schema\ObjectSchema;
use Prism\Prism\Schema\StringSchema;
use Prism\Prism\Schema\NumberSchema;

$profileSchema = new ObjectSchema(
    name: 'profile',
    description: 'A user\'s public profile information',
    properties: [
        new StringSchema('username', 'The unique username'),
        new StringSchema('bio', 'A short biography'),
        new NumberSchema('joined_year', 'Year the user joined'),
    ],
    requiredFields: ['username']
);
```

### AnyOfSchema

For flexible data that can match one of several schemas. This is particularly useful when you need to handle different data types or structures in the same field.

> \[!IMPORTANT]
> **Provider Compatibility**: The AnyOfSchema works with OpenAI's structured outputs and Gemini's enhanced JSON Schema support (as of November 2025). Each nested schema must be a valid JSON schema according to the provider's requirements. For best results, ensure each nested schema has a proper `type` field.

```php
use Prism\Prism\Schema\AnyOfSchema;
use Prism\Prism\Schema\StringSchema;
use Prism\Prism\Schema\NumberSchema;
use Prism\Prism\Schema\ObjectSchema;

// Simple example: A value that can be either a string or number
$flexibleValueSchema = new AnyOfSchema(
    schemas: [
        new StringSchema('text', 'A text value'),
        new NumberSchema('number', 'A numeric value'),
    ],
    name: 'flexible_value',
    description: 'A value that can be either text or numeric'
);

// Complex example: Different content types
$contentSchema = new AnyOfSchema(
    schemas: [
        new ObjectSchema(
            name: 'article',
            description: 'A blog article',
            properties: [
                new StringSchema('title', 'Article title'),
                new StringSchema('content', 'Article content'),
                new StringSchema('author', 'Article author'),
            ],
            requiredFields: ['title', 'content']
        ),
        new ObjectSchema(
            name: 'image',
            description: 'An image post',
            properties: [
                new StringSchema('url', 'Image URL'),
                new StringSchema('caption', 'Image caption'),
                new NumberSchema('width', 'Image width in pixels'),
                new NumberSchema('height', 'Image height in pixels'),
            ],
            requiredFields: ['url']
        ),
    ],
    name: 'content',
    description: 'Content that can be either an article or an image'
);
```

**Key Features:**

* Accepts an array of schema objects that define the possible types
* Automatically validates nested schemas for OpenAI compatibility
* Supports nullable values through the `nullable` parameter
* Optional name and description parameters
* Removes unsupported JSON schema properties automatically

**Provider Support for AnyOfSchema:**

| Provider   | anyOf Support | Available Since | Notes |
|------------|---------------|-----------------|-------|
| OpenAI     | ✅ Full       | GPT-4 onwards   | Works with structured outputs API |
| Gemini     | ✅ Full       | Gemini 2.5+     | Enhanced JSON Schema support (Nov 2025) |
| Anthropic  | ❌ Not supported | -            | Use alternative schema design patterns |

## Nullable Fields

Sometimes, not every field is required. You can make any schema nullable by setting the `nullable` parameter to `true`:

```php
use Prism\Prism\Schema\StringSchema;

$bioSchema = new StringSchema(
    name: 'bio',
    description: 'Optional user biography',
    nullable: true
);
```

> \[!NOTE]
> When using OpenAI in strict mode, all fields must be marked as required, so optional fields must be marked as nullable.

## Required vs Nullable Fields

Understanding the difference between required fields and nullable fields is crucial when working with schemas in Prism:

### Required Fields

Required fields are specified at the object level using the `requiredFields` parameter. They indicate which properties must be present in the data structure:

```php
$userSchema = new ObjectSchema(
    name: 'user',
    description: 'User profile',
    properties: [
        new StringSchema('email', 'Primary email address'),
        new StringSchema('name', 'User\'s full name'),
        new StringSchema('bio', 'User biography', nullable: true), // bio can be null
    ],
    requiredFields: ['email', 'name', 'bio'] // all fields must be present
);
```

### Nullable Fields

Nullable fields, on the other hand, are specified at the individual field level using the `nullable` parameter. They indicate that a field can contain a `null` value:

```php
$userSchema = new ObjectSchema(
    name: 'user',
    description: 'User profile',
    properties: [
        new StringSchema('email', 'Primary email address'),
        new StringSchema('name', 'User\'s full name'),
        new StringSchema('bio', 'User biography', nullable: true), // bio can be null
    ],
    requiredFields: ['email', 'name', 'bio'] // bio must be present, but can be null
);
```

### Key Differences

1. **Required vs Present**:
   * A required field must be present in the data structure
   * A non-nullable field must contain a non-null value when present
   * A field can be required but nullable (must be present, can be null)
   * A field can be non-required and non-nullable (when present, cannot be null)

2. **Common Patterns**:

```php
// Required and Non-nullable (most strict)
new StringSchema('email', 'Primary email', nullable: false);
// requireFields: ['email']

// Required but Nullable (must be present, can be null)
new StringSchema('bio', 'User bio', nullable: true);
// requireFields: ['bio']

// Optional and Non-nullable (can be omitted, but if present cannot be null)
new StringSchema('phone', 'Phone number', nullable: false);
// requireFields: []

// Optional and Nullable (most permissive)
new StringSchema('website', 'Personal website', nullable: true);
// requireFields: []
```

### Provider Considerations

When working with providers that support strict mode (like OpenAI), you'll want to be especially careful with these settings:

```php
// For OpenAI strict mode: 
// - All fields should be required
// - Use nullable: true for optional fields
$userSchema = new ObjectSchema(
    name: 'user',
    description: 'User profile',
    properties: [
        new StringSchema('email', 'Required email address'),
        new StringSchema('bio', 'Optional biography', nullable: true),
    ],
    requiredFields: ['email', 'bio'] // Note: bio is required but nullable
);
```

> \[!TIP]
> When in doubt, be explicit about both requirements. Specify both the `nullable` status of each field AND which fields are required in your object schemas. This makes your intentions clear to both other developers and AI providers.

## Best Practices

1. **Clear Descriptions**: Write clear, concise descriptions for each field. Future you (and other developers) will thank you!
   ```php
   // ❌ Not helpful
   new StringSchema('name', 'the name');

   // ✅ Much better
   new StringSchema('name', 'The user\'s display name (2-50 characters)');
   ```

2. **Thoughtful Required Fields**: Only mark fields as required if they're truly necessary:
   ```php
   new ObjectSchema(
       name: 'user',
       description: 'User profile',
       properties: [
           new StringSchema('email', 'Primary email address'),
           new StringSchema('phone', 'Optional phone number', nullable: true),
       ],
       requiredFields: ['email']
   );
   ```

3. **Nested Organization**: Keep your schemas organized when dealing with complex structures:
   ```php
   // Define child schemas first
   $addressSchema = new ObjectSchema(/*...*/);
   $contactSchema = new ObjectSchema(/*...*/);

   // Then use them in your parent schema
   $userSchema = new ObjectSchema(
       name: 'user',
       description: 'Complete user profile',
       properties: [$addressSchema, $contactSchema]
   );
   ```

> \[!NOTE]
> Remember that while schemas help define the structure of your data, Prism doesn't currently validate the data against these schemas. Schema validation is planned for a future release!

---

---
url: /core-concepts/streaming-output.md
---
# Streaming Output

Want to show AI responses to your users in real-time? Prism provides multiple ways to handle streaming AI responses, from simple Server-Sent Events to WebSocket broadcasting for real-time applications.

> \[!WARNING]
> When using Laravel Telescope or other packages that intercept Laravel's HTTP client events, they may consume the stream before Prism can emit the stream events. This can cause streaming to appear broken or incomplete. Consider disabling such interceptors when using streaming functionality, or configure them to ignore Prism's HTTP requests.

## Quick Start

### Server-Sent Events (SSE)

The simplest way to stream AI responses to a web interface:

```php
Route::get('/chat', function () {
    return Prism::text()
        ->using('anthropic', 'claude-3-7-sonnet')
        ->withPrompt(request('message'))
        ->asEventStreamResponse();
});
```

```javascript
const eventSource = new EventSource('/chat');

eventSource.addEventListener('text_delta', (event) => {
    const data = JSON.parse(event.data);
    document.getElementById('output').textContent += data.delta;
});

eventSource.addEventListener('stream_end', (event) => {
    const data = JSON.parse(event.data);
    console.log('Stream ended:', data.finish_reason);
    eventSource.close();
});
```

### Vercel AI SDK Integration

For apps using Vercel's AI SDK, use the Data Protocol adapter which provides compatibility with the [Vercel AI SDK UI](https://ai-sdk.dev/docs/reference/ai-sdk-ui):

```php
Route::post('/api/chat', function () {
    return Prism::text()
        ->using('openai', 'gpt-4')
        ->withPrompt(request('message'))
        ->asDataStreamResponse();
});
```

Client-side with the `useChat` hook:

```javascript
import { useChat } from '@ai-sdk/react';
import { useState } from 'react';

export default function Chat() {
    // AI SDK 5.0 no longer manages input state, so we handle it ourselves
    const [input, setInput] = useState('');

    const { messages, sendMessage, status } = useChat({
        transport: {
            api: '/api/chat',
        },
    });

    const handleSubmit = (e) => {
        e.preventDefault();
        if (input.trim() && status === 'ready') {
            sendMessage(input);
            setInput('');
        }
    };

    return (
        <div>
            <div>
                {messages.map(m => (
                    <div key={m.id}>
                        <strong>{m.role}:</strong>{' '}
                        {m.parts
                            .filter(part => part.type === 'text')
                            .map(part => part.text)
                            .join('')}
                    </div>
                ))}
            </div>

            <form onSubmit={handleSubmit}>
                <input
                    value={input}
                    placeholder="Say something..."
                    onChange={(e) => setInput(e.target.value)}
                    disabled={status !== 'ready'}
                />
                <button type="submit" disabled={status !== 'ready'}>
                    {status === 'streaming' ? 'Sending...' : 'Send'}
                </button>
            </form>
        </div>
    );
}
```

> \[!NOTE]
> This example uses AI SDK 5.0, which introduced significant changes to the `useChat` hook. The hook no longer manages input state internally, and you'll need to use the `sendMessage` function directly instead of `handleSubmit`.

For more advanced usage, including tool support and custom options, see the [Vercel AI SDK UI documentation](https://ai-sdk.dev/docs/reference/ai-sdk-ui).

### WebSocket Broadcasting with Background Jobs

For real-time multi-user applications that need to process AI requests in the background:

```php
// Job Class
<?php

namespace App\Jobs;

use Illuminate\Broadcasting\Channel;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
use Prism\Prism\Facades\Prism;

class ProcessAiStreamJob implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public function __construct(
        public string $message,
        public string $channel,
        public string $model = 'claude-3-7-sonnet'
    ) {}

    public function handle(): void
    {
        Prism::text()
            ->using('anthropic', $this->model)
            ->withPrompt($this->message)
            ->asBroadcast(new Channel($this->channel));
    }
}

// Controller
Route::post('/chat-broadcast', function () {
    $sessionId = request('session_id') ?? 'session_' . uniqid();
    
    ProcessAiStreamJob::dispatch(
        request('message'),
        "chat.{$sessionId}",
        request('model', 'claude-3-7-sonnet')
    );
    
    return response()->json(['status' => 'processing', 'session_id' => $sessionId]);
});
```

Client-side with React and useEcho:

```javascript
import { useEcho } from '@/hooks/useEcho';
import { useState } from 'react';

function ChatComponent() {
    const [currentMessage, setCurrentMessage] = useState('');
    const [currentMessageId, setCurrentMessageId] = useState('');
    const [isComplete, setIsComplete] = useState(false);

    const sessionId = 'session_' + Date.now();

    // Listen for streaming events
    useEcho(`chat.${sessionId}`, {
        '.stream_start': (data) => {
            console.log('Stream started:', data);
            setCurrentMessage('');
            setIsComplete(false);
        },
        
        '.step_start': (data) => {
            console.log('Step started:', data);
            // A new generation cycle is beginning
        },
        
        '.text_start': (data) => {
            console.log('Text start event received:', data);
            setCurrentMessage('');
            setCurrentMessageId(data.message_id || Date.now().toString());
        },
        
        '.text_delta': (data) => {
            console.log('Text delta received:', data);
            setCurrentMessage(prev => prev + data.delta);
        },
        
        '.text_complete': (data) => {
            console.log('Text complete:', data);
        },
        
        '.tool_call': (data) => {
            console.log('Tool called:', data.tool_name, data.arguments);
        },
        
        '.tool_result': (data) => {
            console.log('Tool result:', data.result);
        },
        
        '.step_finish': (data) => {
            console.log('Step finished:', data);
            // Generation cycle complete, may be followed by another step
        },
        
        '.stream_end': (data) => {
            console.log('Stream ended:', data.finish_reason);
            setIsComplete(true);
        },
        
        '.error': (data) => {
            console.error('Stream error:', data.message);
        }
    });

    const sendMessage = async (message) => {
        await fetch('/chat-broadcast', {
            method: 'POST',
            headers: { 'Content-Type': 'application/json' },
            body: JSON.stringify({ 
                message, 
                session_id: sessionId,
                model: 'claude-3-7-sonnet' 
            })
        });
    };

    return (
        <div>
            <div className="message-display">
                {currentMessage}
                {!isComplete && <span className="cursor">|</span>}
            </div>
            
            <button onClick={() => sendMessage("What's the weather in Detroit?")}>
                Send Message
            </button>
        </div>
    );
}
```

## Event Types

All streaming approaches emit the same core events with consistent data structures:

### Available Events

* **`stream_start`** - Stream initialization with model and provider info
* **`step_start`** - Beginning of a generation step (emitted before each AI response cycle)
* **`text_start`** - Beginning of a text message
* **`text_delta`** - Incremental text chunks as they're generated
* **`text_complete`** - End of a complete text message
* **`thinking_start`** - Beginning of AI reasoning/thinking session
* **`thinking_delta`** - Reasoning content as it's generated
* **`thinking_complete`** - End of reasoning session
* **`tool_call`** - Tool invocation with arguments
* **`tool_result`** - Tool execution results
* **`tool_call_delta`** - Incremental tool call params chunks as they're generated
* **`artifact`** - Binary artifacts produced by tools (images, audio, files)
* **`provider_tool_event`** - Provider-specific tool events (e.g., image generation, web search)
* **`step_finish`** - End of a generation step (emitted after tool calls or before stream end)
* **`error`** - Error handling with recovery information
* **`stream_end`** - Stream completion with usage statistics

> \[!TIP]
> **Understanding Steps**: A "step" represents one cycle of AI generation. In a simple request without tools, there's typically one step. When using tools, each cycle of "AI generates → tools execute → AI continues" creates a new step. Use `step_start` and `step_finish` events to track these cycles in multi-turn tool interactions.

### Event Data Examples

Based on actual streaming output:

```javascript
// stream_start event
{
    "id": "anthropic_evt_SSrB7trNIXsLkbUB",
    "timestamp": 1756412888,
    "model": "claude-3-7-sonnet-20250219",
    "provider": "anthropic",
    "metadata": {
        "request_id": "msg_01BS7MKgXvUESY8yAEugphV2",
        "rate_limits": []
    }
}

// step_start event
{
    "id": "anthropic_evt_abc123step",
    "timestamp": 1756412888
}

// text_start event
{
    "id": "anthropic_evt_8YI9ULcftpFtHzh3",
    "timestamp": 1756412888,
    "message_id": "msg_01BS7MKgXvUESY8yAEugphV2"
}

// text_delta event
{
    "id": "anthropic_evt_NbS3LIP0QDl5whYu",
    "timestamp": 1756412888,
    "delta": "💠🌐 Well hello there! You want to know",
    "message_id": "msg_01BS7MKgXvUESY8yAEugphV2"
}

// tool_call event
{
    "id": "anthropic_evt_qXvozT6OqtmFPgkG",
    "timestamp": 1756412889,
    "tool_id": "toolu_01NAbzpjGxv2mJ8gJRX5Bb8m",
    "tool_name": "search",
    "arguments": {"query": "current date and time in Detroit Michigan"},
    "message_id": "msg_01BS7MKgXvUESY8yAEugphV2",
    "reasoning_id": null
}

// provider_tool_event (e.g., OpenAI image generation)
{
    "id": "openai_evt_abc123",
    "timestamp": 1756412890,
    "type": "provider_tool_event",
    "event_key": "provider_tool_event.image_generation_call.completed",
    "tool_type": "image_generation_call",
    "status": "completed",
    "item_id": "ig_abc123def456",
    "data": {
        "id": "ig_abc123def456",
        "type": "image_generation_call",
        "status": "completed",
        "result": "iVBORw0KGgo..." // base64 PNG data
    }
}

// artifact event (from tool output)
{
    "id": "anthropic_evt_xyz789",
    "timestamp": 1756412891,
    "tool_call_id": "toolu_01NAbzpjGxv2mJ8gJRX5Bb8m",
    "tool_name": "generate_image",
    "message_id": "msg_01BS7MKgXvUESY8yAEugphV2",
    "artifact": {
        "id": "img-abc123",
        "data": "iVBORw0KGgo...", // base64 encoded data
        "mime_type": "image/png",
        "metadata": {
            "width": 1024,
            "height": 1024
        }
    }
}

// step_finish event
{
    "id": "anthropic_evt_def456step",
    "timestamp": 1756412895
}

// stream_end event
{
    "id": "anthropic_evt_BZ3rqDYyprnywNyL",
    "timestamp": 1756412898,
    "finish_reason": "Stop",
    "usage": {
        "prompt_tokens": 3448,
        "completion_tokens": 192,
        "cache_write_input_tokens": 0,
        "cache_read_input_tokens": 0,
        "thought_tokens": 0
    }
}
```

## Handling Artifact Events

When tools produce binary artifacts (images, audio, files), they're emitted as `ArtifactEvent` through the stream. This lets your application handle binary data without it going to the LLM's context window.

### Artifact Events with SSE

Listen for artifact events alongside other stream events:

```javascript
const eventSource = new EventSource('/chat');

eventSource.addEventListener('artifact', (event) => {
    const data = JSON.parse(event.data);

    // Display an image artifact
    if (data.artifact.mime_type.startsWith('image/')) {
        const img = document.createElement('img');
        img.src = `data:${data.artifact.mime_type};base64,${data.artifact.data}`;
        document.getElementById('artifacts').appendChild(img);
    }

    // Handle other artifact types
    console.log('Artifact received:', {
        toolName: data.tool_name,
        mimeType: data.artifact.mime_type,
        metadata: data.artifact.metadata,
    });
});

eventSource.addEventListener('text_delta', (event) => {
    const data = JSON.parse(event.data);
    document.getElementById('output').textContent += data.delta;
});
```

### Artifact Events with Vercel AI SDK

When using `asDataStreamResponse()`, artifacts are sent as custom data parts with type `data-artifact`:

```javascript
import { useChat } from '@ai-sdk/react';
import { useState } from 'react';

export default function Chat() {
    const [input, setInput] = useState('');
    const [artifacts, setArtifacts] = useState([]);

    const { messages, sendMessage, status, data } = useChat({
        transport: {
            api: '/api/chat',
        },
        onData: (data) => {
            // Handle artifact data messages
            if (data.type === 'data-artifact') {
                setArtifacts(prev => [...prev, data.data.artifact]);
            }
        },
    });

    return (
        <div>
            {/* Display artifacts */}
            <div className="artifacts">
                {artifacts.map((artifact, i) => (
                    artifact.mime_type.startsWith('image/') && (
                        <img
                            key={artifact.id || i}
                            src={`data:${artifact.mime_type};base64,${artifact.data}`}
                            alt={`Generated artifact ${i + 1}`}
                        />
                    )
                ))}
            </div>

            {/* Messages display */}
            <div>
                {messages.map(m => (
                    <div key={m.id}>
                        <strong>{m.role}:</strong>{' '}
                        {m.parts
                            .filter(part => part.type === 'text')
                            .map(part => part.text)
                            .join('')}
                    </div>
                ))}
            </div>

            <form onSubmit={(e) => {
                e.preventDefault();
                if (input.trim() && status === 'ready') {
                    sendMessage(input);
                    setInput('');
                }
            }}>
                <input
                    value={input}
                    placeholder="Ask to generate an image..."
                    onChange={(e) => setInput(e.target.value)}
                />
                <button type="submit">Send</button>
            </form>
        </div>
    );
}
```

### Artifact Events with Broadcasting

When using `asBroadcast()` for WebSocket broadcasting, listen for the `.artifact` event:

```javascript
useEcho(`chat.${sessionId}`, {
    '.artifact': (data) => {
        console.log('Artifact received:', data.tool_name);

        // Store or display the artifact
        if (data.artifact.mime_type.startsWith('image/')) {
            setImages(prev => [...prev, {
                id: data.artifact.id,
                src: `data:${data.artifact.mime_type};base64,${data.artifact.data}`,
                metadata: data.artifact.metadata,
            }]);
        }
    },

    '.tool_result': (data) => {
        console.log('Tool result (text for LLM):', data.result);
    },

    // ... other event handlers
});
```

### Persisting Artifacts in Callbacks

Use streaming callbacks to save artifacts to your database or storage:

```php
use Illuminate\Support\Collection;
use Prism\Prism\Streaming\Events\ArtifactEvent;
use Prism\Prism\Streaming\Events\StreamEvent;
use Prism\Prism\Text\PendingRequest;

return Prism::text()
    ->using('anthropic', 'claude-3-7-sonnet')
    ->withTools([$imageGeneratorTool])
    ->withPrompt(request('message'))
    ->asDataStreamResponse(function (PendingRequest $request, Collection $events) use ($conversationId) {
        // Save artifacts to storage
        $events
            ->filter(fn (StreamEvent $event) => $event instanceof ArtifactEvent)
            ->each(function (ArtifactEvent $event) use ($conversationId) {
                Attachment::create([
                    'conversation_id' => $conversationId,
                    'tool_call_id' => $event->toolCallId,
                    'tool_name' => $event->toolName,
                    'mime_type' => $event->artifact->mimeType,
                    'data' => $event->artifact->rawContent(), // Decoded binary data
                    'metadata' => $event->artifact->metadata,
                ]);
            });
    });
```

For more information about creating tools that produce artifacts, see [Tool Artifacts](/core-concepts/tools-function-calling#tool-artifacts).

## Advanced Usage

### Handling Completion with Callbacks

Need to save a conversation to your database after the AI finishes responding? Pass a callback directly to your terminal method to handle the completed response. This is perfect for persisting conversations, tracking analytics, or logging AI interactions.

#### Text Generation Callbacks

For non-streaming requests, pass a callback to `asText()`:

```php
use Prism\Prism\Text\PendingRequest;
use Prism\Prism\Text\Response;

$response = Prism::text()
    ->using('anthropic', 'claude-3-7-sonnet')
    ->withPrompt(request('message'))
    ->asText(function (PendingRequest $request, Response $response) use ($conversationId) {
        // Save the response to your database
        ConversationMessage::create([
            'conversation_id' => $conversationId,
            'role' => 'assistant',
            'content' => $response->text,
            'tool_calls' => $response->toolCalls,
        ]);
    });

// The response is still returned for further use
return response()->json(['message' => $response->text]);
```

The callback receives the `PendingRequest` and the complete `Response` object, giving you access to the full response including text, tool calls, tool results, and usage statistics.

#### Streaming Response Callbacks

For streaming responses, pass a callback to receive all collected events when the stream completes:

```php
use Illuminate\Support\Collection;
use Prism\Prism\Streaming\Events\StreamEvent;
use Prism\Prism\Streaming\Events\TextDeltaEvent;
use Prism\Prism\Text\PendingRequest;

return Prism::text()
    ->using('anthropic', 'claude-3-7-sonnet')
    ->withPrompt(request('message'))
    ->asEventStreamResponse(function (PendingRequest $request, Collection $events) use ($conversationId) {
        // Reconstruct the full text from all delta events
        $fullText = $events
            ->filter(fn (StreamEvent $event) => $event instanceof TextDeltaEvent)
            ->map(fn (TextDeltaEvent $event) => $event->delta)
            ->join('');

        // Save the complete response
        ConversationMessage::create([
            'conversation_id' => $conversationId,
            'role' => 'assistant',
            'content' => $fullText,
        ]);
    });
```

The callback receives:

* `PendingRequest` - The original request configuration
* `Collection<StreamEvent>` - All events that occurred during the stream

This works with all streaming methods: `asEventStreamResponse()`, `asDataStreamResponse()`, and `asBroadcast()`.

#### Using Invokable Classes

For better organization, use invokable classes as callbacks:

```php
use Illuminate\Support\Collection;
use Prism\Prism\Streaming\Events\StreamEvent;
use Prism\Prism\Streaming\Events\TextDeltaEvent;
use Prism\Prism\Text\PendingRequest;

class SaveStreamedConversation
{
    public function __construct(
        protected string $conversationId
    ) {}

    public function __invoke(PendingRequest $request, Collection $events): void
    {
        $fullText = $events
            ->filter(fn (StreamEvent $event) => $event instanceof TextDeltaEvent)
            ->map(fn (TextDeltaEvent $event) => $event->delta)
            ->join('');

        ConversationMessage::create([
            'conversation_id' => $this->conversationId,
            'role' => 'assistant',
            'content' => $fullText,
        ]);
    }
}

// Use with streaming responses
return Prism::text()
    ->using('anthropic', 'claude-3-7-sonnet')
    ->withPrompt($message)
    ->asEventStreamResponse(new SaveStreamedConversation($conversationId));
```

### Custom Event Processing

Access raw events for complete control over handling:

```php
$events = Prism::text()
    ->using('openai', 'gpt-4')
    ->withPrompt('Explain quantum physics')
    ->asStream();

foreach ($events as $event) {
    match ($event->type()) {
        StreamEventType::TextDelta => handleTextChunk($event),
        StreamEventType::ToolCall => handleToolCall($event),
        StreamEventType::StreamEnd => handleCompletion($event),
        default => null,
    };
}
```

### Streaming with Tools

Stream responses that include tool interactions:

```php
use Prism\Prism\Facades\Tool;

$searchTool = Tool::as('search')
    ->for('Search for information')
    ->withStringParameter('query', 'Search query')
    ->using(function (string $query) {
        return "Search results for: {$query}";
    });

return Prism::text()
    ->using('anthropic', 'claude-3-7-sonnet')
    ->withTools([$searchTool])
    ->withPrompt("What's the weather in Detroit?")
    ->asEventStreamResponse();
```

### Data Protocol Output

The Vercel AI SDK format provides structured streaming data:

```
data: {"type":"start","messageId":"anthropic_evt_NPbGJs7D0oQhvz2K"}

data: {"type":"start-step"}

data: {"type":"text-start","id":"msg_013P3F8KkVG3Qasjeay3NUmY"}

data: {"type":"text-delta","id":"msg_013P3F8KkVG3Qasjeay3NUmY","delta":"Hello"}

data: {"type":"text-end","id":"msg_013P3F8KkVG3Qasjeay3NUmY"}

data: {"type":"finish-step"}

data: {"type":"finish","messageMetadata":{"finishReason":"stop","usage":{"promptTokens":1998,"completionTokens":288}}}

data: [DONE]
```

## Configuration Options

Streaming supports all the same configuration options as regular [text generation](/core-concepts/text-generation#generation-parameters), including temperature, max tokens, and provider-specific settings.

---

---
url: /core-concepts/structured-output.md
---
# Structured Output

Want your AI responses as neat and tidy as a Marie Kondo-approved closet? Structured output lets you define exactly how you want your data formatted, making it perfect for building APIs, processing forms, or any time you need data in a specific shape.

## Quick Start

Here's how to get structured data from your AI:

> \[!IMPORTANT]
> **Schema Requirement for OpenAI**: When using OpenAI's structured output (especially strict mode), the root schema must be an `ObjectSchema`. Other schema types (StringSchema, NumberSchema, etc.) can only be used as properties within an ObjectSchema, not as the top-level schema. Other providers may have different requirements.

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\Schema\ObjectSchema;
use Prism\Prism\Schema\StringSchema;

$schema = new ObjectSchema(
    name: 'movie_review',
    description: 'A structured movie review',
    properties: [
        new StringSchema('title', 'The movie title'),
        new StringSchema('rating', 'Rating out of 5 stars'),
        new StringSchema('summary', 'Brief review summary')
    ],
    requiredFields: ['title', 'rating', 'summary']
);

$response = Prism::structured()
    ->using(Provider::OpenAI, 'gpt-4o')
    ->withSchema($schema)
    ->withPrompt('Review the movie Inception')
    ->asStructured();

// Access your structured data
$review = $response->structured;
echo $review['title'];    // "Inception"
echo $review['rating'];   // "5 stars"
echo $review['summary'];  // "A mind-bending..."
```

> \[!TIP]
> This is just a basic example of schema usage. Check out our [dedicated schemas guide](/core-concepts/schemas) to learn about all available schema types, nullable fields, and best practices for structuring your data.

## Understanding Output Modes

Different AI providers handle structured output in two main ways:

1. **Structured Mode**: Some providers support strict schema validation, ensuring responses perfectly match your defined structure.
2. **JSON Mode**: Other providers simply guarantee valid JSON output that approximately matches your schema.

> \[!NOTE]
> Check your provider's documentation to understand which mode they support. Provider support can vary by model, so always verify capabilities for your specific use case.

## Provider-Specific Options

Providers may offer additional options for structured output:

### OpenAI: Strict Mode

OpenAI supports a "strict mode" for even tighter schema validation:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;

$response = Prism::structured()
    ->withProviderOptions([
        'schema' => [
            'strict' => true
        ]
    ])
    // ... rest of your configuration
```

### Anthropic

Anthropic uses native structured outputs by default (Claude Sonnet 4.5+), providing guaranteed schema compliance through constrained decoding. For older models, you can use tool calling mode as a fallback:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;

$response = Prism::structured()
    ->using(Provider::Anthropic, 'claude-3-5-sonnet-latest')
    ->withSchema($schema)
    ->withPrompt('天氣怎麼樣？應該穿什麼？') // Chinese text with potential quotes
    ->withProviderOptions(['use_tool_calling' => true])
    ->asStructured();
```

**When to use tool calling mode with Anthropic:**

* Using older models that don't support native structured outputs
* Working with non-English content that may contain quotes on older models
* When you need to combine structured output with custom tools

> \[!NOTE]
> Tool calling mode cannot be used with Anthropic's citations feature.

> \[!TIP]
> Check the provider-specific documentation pages for additional options and features that might be available for structured output.

## Response Handling

When working with structured responses, you have access to both the structured data and metadata about the generation:

```php
use Prism\Prism\Facades\Prism;

$response = Prism::structured()
    ->withSchema($schema)
    ->asStructured();

// Access the structured data as a PHP array
$data = $response->structured;

// Get the raw response text if needed
echo $response->text;

// Check why the generation stopped
echo $response->finishReason->name;

// Get token usage statistics
echo "Prompt tokens: {$response->usage->promptTokens}";
echo "Completion tokens: {$response->usage->completionTokens}";

// Access the raw API response data
$rawResponse = $response->raw;
```

> \[!TIP]
> Always validate the structured data before using it in your application:

```php
if ($response->structured === null) {
    // Handle parsing failure
}

if (!isset($response->structured['required_field'])) {
    // Handle missing required data
}
```

## Common Settings

Structured output supports several configuration options to fine-tune your generations:

### Model Configuration

* `maxTokens` - Set the maximum number of tokens to generate
* `temperature` - Control output randomness (provider-dependent)
* `topP` - Alternative to temperature for controlling randomness (provider-dependent)

### Input Methods

* `withPrompt` - Single prompt for generation
* `withMessages` - Message history for more context
* `withSystemPrompt` - System-level instructions

### Request Configuration

* `withClientOptions` - Set HTTP client options (e.g., timeouts)
* `withClientRetry` - Configure automatic retries on failures
* `usingProviderConfig` - Override provider configuration
* `withProviderOptions` - Set provider-specific options

See the [Text Generation](./text-generation.md) documentation for comparison with standard text generation capabilities.

## Combining Structured Output with Tools

You can combine structured output with tools to gather data before returning a structured response. This lets the AI call functions to fetch information, then format the results according to your schema.

### Basic Example

Here's a simple example that uses a weather tool to gather data, then returns structured output:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Schema\ObjectSchema;
use Prism\Prism\Schema\StringSchema;
use Prism\Prism\Tool;

$schema = new ObjectSchema(
    name: 'weather_analysis',
    description: 'Analysis of weather conditions',
    properties: [
        new StringSchema('summary', 'Summary of the weather'),
        new StringSchema('recommendation', 'Recommendation based on weather'),
    ],
    requiredFields: ['summary', 'recommendation']
);

$weatherTool = Tool::as('get_weather')
    ->for('Get current weather for a location')
    ->withStringParameter('location', 'The city and state')
    ->using(fn (string $location): string =>
        "Weather in {$location}: 72°F, sunny"
    );

$response = Prism::structured()
    ->using('anthropic', 'claude-3-5-sonnet-latest')
    ->withSchema($schema)
    ->withTools([$weatherTool])
    ->withMaxSteps(3)
    ->withPrompt('What is the weather in San Francisco and should I wear a coat?')
    ->asStructured();

// Access structured output
dump($response->structured);
// ['summary' => '...', 'recommendation' => '...']
```

> \[!IMPORTANT]
> When using tools with structured output, you must set `maxSteps` to at least 2. The AI needs multiple steps: one to call tools, and another to return the structured result.

### Multiple Tools

You can provide multiple tools for the AI to use:

```php
$schema = new ObjectSchema(
    name: 'game_analysis',
    description: 'Analysis of game time and weather',
    properties: [
        new StringSchema('game_time', 'The time of the game'),
        new StringSchema('weather_summary', 'Summary of weather conditions'),
        new StringSchema('recommendation', 'Recommendation on what to wear'),
    ],
    requiredFields: ['game_time', 'weather_summary', 'recommendation']
);

$tools = [
    Tool::as('get_weather')
        ->for('Get current weather for a location')
        ->withStringParameter('city', 'The city name')
        ->using(fn (string $city): string =>
            "Weather in {$city}: 45°F and cold"
        ),
    Tool::as('search_games')
        ->for('Search for game times in a city')
        ->withStringParameter('city', 'The city name')
        ->using(fn (string $city): string =>
            'The Tigers game is at 3pm in Detroit'
        ),
];

$response = Prism::structured()
    ->using('openai', 'gpt-4o')
    ->withSchema($schema)
    ->withTools($tools)
    ->withMaxSteps(5)
    ->withPrompt('What time is the Tigers game today in Detroit and should I wear a coat?')
    ->asStructured();
```

### Response Handling

When using tools with structured output, the response includes both the structured data and tool execution details:

```php
// Access final structured data
$data = $response->structured;

// Access all tool calls across all steps
foreach ($response->toolCalls as $toolCall) {
    echo "Called: {$toolCall->name}\n";
    echo "Arguments: " . json_encode($toolCall->arguments()) . "\n";
}

// Access tool results
foreach ($response->toolResults as $result) {
    echo "Tool: {$result->toolName}\n";
    echo "Result: {$result->result}\n";
}

// Inspect individual steps
foreach ($response->steps as $step) {
    echo "Step finish reason: {$step->finishReason->name}\n";

    if ($step->toolCalls) {
        echo "Tools called: " . count($step->toolCalls) . "\n";
    }

    if ($step->structured) {
        echo "Contains structured data\n";
    }
}
```

> \[!NOTE]
> Only the final step contains structured data. Intermediate steps contain tool calls and tool results, but no structured output.

For more information about tools and function calling, see the [Tools & Function Calling](./tools-function-calling.md) documentation.

> \[!IMPORTANT]
> Always validate the structured response before using it in your application, as different providers may have varying levels of schema adherence.

---

---
url: /core-concepts/testing.md
---
# Testing

Want to make sure your Prism integrations work flawlessly? Let's dive into testing! Prism provides a powerful fake implementation that makes it a breeze to test your AI‑powered features.

## Basic Test Setup

First, let's look at how to set up basic response faking:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\ValueObjects\Usage;
use Prism\Prism\Testing\TextResponseFake;

it('can generate text', function () {
    $fakeResponse = TextResponseFake::make()
        ->withText('Hello, I am Claude!')
        ->withUsage(new Usage(10, 20));

    // Set up the fake
    $fake = Prism::fake([$fakeResponse]);

    // Run your code
    $response = Prism::text()
        ->using(Provider::Anthropic, 'claude-3-5-sonnet-latest')
        ->withPrompt('Who are you?')
        ->asText();

    // Make assertions
    expect($response->text)->toBe('Hello, I am Claude!');
});
```

The response fakes create a new response with default values and let you fluently set the values you need for your test.

## Testing Multiple Responses

When testing conversations or tool usage, you might need to simulate multiple responses:

```php
use Prism\Prism\ValueObjects\Usage;
use Prism\Prism\ValueObjects\ToolCall;
use Prism\Prism\Testing\TextResponseFake;
use Prism\Prism\ValueObjects\Meta;

it('can handle tool calls', function () {
    $responses = [
        TextResponseFake::make()
            ->withToolCalls([
                new ToolCall(
                    id: 'call_1',
                    name: 'search',
                    arguments: ['query' => 'Latest news']
                )
            ])
            ->withUsage(new Usage(15, 25))
            ->withMeta(new Meta('fake-1', 'fake-model')),

        TextResponseFake::make()
            ->withText('Here are the latest news...')
            ->withUsage(new Usage(20, 30))
            ->withMeta(new Meta('fake-2', 'fake-model')),
    ];

    $fake = Prism::fake($responses);
});
```

## Using the ResponseBuilder

If you need to test a richer response object, e.g. with Steps, you may find it easier to use the `ResponseBuilder` together with the fake Step helpers.
This is especially useful when you want to test complex streamed responses.

```php
use Prism\Prism\Text\ResponseBuilder;
use Prism\Prism\Testing\TextStepFake;
use Prism\Prism\ValueObjects\Usage;
use Prism\Prism\ValueObjects\Meta;
use Prism\Prism\Enums\FinishReason;
use Prism\Prism\ValueObjects\ToolCall;
use Prism\Prism\ValueObjects\ToolResult;
use Prism\Prism\ValueObjects\Messages\{UserMessage,AssistantMessage,SystemMessage};
use Prism\Prism\ValueObjects\Media\Document;

Prism::fake([
    (new ResponseBuilder)
        ->addStep(
            TextStepFake::make()
                ->withText('Step 1 response text')
                ->withFinishReason(FinishReason::Stop)
                ->withToolCalls([/* tool calls */])
                ->withToolResults([/* tool results */])
                ->withUsage(new Usage(1000, 750))
                ->withMeta(new Meta('step1', 'test-model'))
                ->withMessages([
                    new UserMessage('Test message 1', [
                        new Document(
                            document: '',
                            mimeType: 'text/plain',
                            dataFormat: 'text',
                            documentTitle: 'Test document',
                            documentContext: 'Test context'
                        ),
                    ]),
                    new AssistantMessage('Test message 2')
                ])
                ->withSystemPrompts([
                    new SystemMessage('Test system')
                ])
                ->withAdditionalContent(['test' => 'additional'])
        )
       ->addStep(
           TextStepFake::make()
                ->withText('Step 2 response text')
                ->withFinishReason(FinishReason::Stop)
                ->withToolCalls([/* tool calls */])
                ->withToolResults([/* tool results */])
                ->withUsage(new Usage(1000, 750))
                ->withMeta(new Meta(id: 123, model: 'test-model'))
                ->withMessages([/* Second step messages */])
                ->withSystemPrompts([/* Second step system prompts */])
                ->withAdditionalContent([/* Second step additional data */])
       )
        ->toResponse()
]);
```

## Testing Tools

```php
use Prism\Prism\Enums\FinishReason;
use Prism\Prism\Enums\Provider;
use Prism\Prism\Facades\Tool;
use Prism\Prism\Facades\Prism;
use Prism\Prism\Testing\TextStepFake;
use Prism\Prism\Text\ResponseBuilder;
use Prism\Prism\ValueObjects\Meta;
use Prism\Prism\ValueObjects\ToolCall;
use Prism\Prism\ValueObjects\ToolResult;
use Prism\Prism\ValueObjects\Usage;

it('can use weather tool', function () {
    // Define the expected tool call and response sequence
    $responses = [
        (new ResponseBuilder)
            ->addStep(
                // First response: AI decides to use the weather tool
                TextStepFake::make()
                    ->withToolCalls([
                        new ToolCall(
                            id: 'call_123',
                            name: 'weather',
                            arguments: ['city' => 'Paris']
                        ),
                    ])
                    ->withFinishReason(FinishReason::ToolCalls)
                    ->withUsage(new Usage(15, 25))
                    ->withMeta(new Meta('fake-1', 'fake-model'))
            )
            ->addStep(
                // Second response: AI uses the tool result to form a response
                TextStepFake::make()
                    ->withText('Based on current conditions, the weather in Paris is sunny with a temperature of 72°F.')
                    ->withToolResults([
                        new ToolResult(
                            toolCallId: 'call_123',
                            toolName: 'weather',
                            args: ['city' => 'Paris'],
                            result: 'Sunny, 72°F'
                        ),
                    ])
                    ->withFinishReason(FinishReason::Stop)
                    ->withUsage(new Usage(20, 30))
                    ->withMeta(new Meta('fake-2', 'fake-model')),
            )
            ->toResponse(),
    ];

    // Set up the fake
    Prism::fake($responses);

    // Create the weather tool
    $weatherTool = Tool::as('weather')
        ->for('Get weather information')
        ->withStringParameter('city', 'City name')
        ->using(fn (string $city) => "The weather in {$city} is sunny with a temperature of 72°F");

    // Run the actual test
    $response = Prism::text()
        ->using(Provider::Anthropic, 'claude-3-5-sonnet-latest')
        ->withPrompt('What\'s the weather in Paris?')
        ->withTools([$weatherTool])
        ->withMaxSteps(2)
        ->asText();

    // Assert the response has the correct number of steps
    expect($response->steps)->toHaveCount(2);

    // Assert tool calls were made correctly
    expect($response->steps[0]->toolCalls)->toHaveCount(1);
    expect($response->steps[0]->toolCalls[0]->name)->toBe('weather');
    expect($response->steps[0]->toolCalls[0]->arguments())->toBe(['city' => 'Paris']);

    // Assert tool results were processed
    expect($response->toolResults)->toHaveCount(1);
    expect($response->toolResults[0]->result)
        ->toBe('Sunny, 72°F');

    // Assert final response
    expect($response->text)
        ->toBe('Based on current conditions, the weather in Paris is sunny with a temperature of 72°F.');
});
```

## Testing Streamed Responses

To test streamed responses, you can use any text response for a fake. The fake Provider will turn the text response into a fake stream of text chunks.
It will always finish with an empty chunk including your given finish reason.

```php
Prism::fake([
    TextResponseFake::make()
        ->withText('fake response text') // text to be streamed
        ->withFinishReason(FinishReason::Stop), // finish reason for final chunk
]);

$text = Prism::text()
    ->using('anthropic', 'claude-3-sonnet')
    ->withPrompt('What is the meaning of life?')
    ->asStream();

$outputText = '';
foreach ($text as $chunk) {
    $outputText .= $chunk->text; // will be ['fake ', 'respo', 'nse t', 'ext', '']; 
}

expect($outputText)->toBe('fake response text');
```

You can adjust the chunk size by using `withFakeChunkSize` on the fake.

```php
Prism::fake([
    TextResponseFake::make()->withText('fake response text'),
])->withFakeChunkSize(1);
```

Now, the text will be streamed in chunks of one character (`['f', 'a', 'k', ...]`).

### Testing Tool Calling while Streaming

When testing streamed responses with tool calls, you can use the `ResponseBuilder` to create a more complex response.
Given a text response with steps, the fake provider will not only generate text chunks, but also include chunks for tool calls and results.

```php
Prism::fake([
    (new ResponseBuilder)
        ->addStep(
            TextStepFake::make()
                ->withToolCalls(
                    [
                        new ToolCall('id-123', 'tool', ['input' => 'value']),
                    ]
                )
        )
        ->addStep(
            TextStepFake::make()
                ->withToolResults(
                    [
                        new ToolResult('id-123', 'tool', ['input' => 'value'], 'result'),
                    ]
                )
        )
        ->addStep(
            TextStepFake::make()
                ->withText('fake response text')
        )
        ->toResponse(),
]);

$text = Prism::text()
    ->using('anthropic', 'claude-3-sonnet')
    ->withPrompt('What is the meaning of life?')
    ->asStream();

$outputText = '';
$toolCalls = [];
$toolResults = [];

foreach ($text as $chunk) {
    $outputText .= $chunk->text;

    // Accumulate tool calls
    if ($chunk->toolCalls) {
        foreach ($chunk->toolCalls as $call) {
            $toolCalls[] = $call;
        }
    }

    // Accumulate tool results
    if ($chunk->toolResults) {
        foreach ($chunk->toolResults as $result) {
            $toolResults[] = $result;
        }
    }
}

expect($outputText)->toBe('fake response text')
    ->and($toolCalls)->toHaveCount(1)
    ->and($toolCalls[0])->toBeInstanceOf(ToolCall::class)
    ->and($toolCalls[0]->id)->toBe('id-123')
    ->and($toolCalls[0]->name)->toBe('tool')
    ->and($toolCalls[0]->arguments())->toBe(['input' => 'value'])
    ->and($toolResults)->toHaveCount(1)
    ->and($toolResults[0])->toBeInstanceOf(ToolResult::class)
    ->and($toolResults[0]->toolCallId)->toBe('id-123')
    ->and($toolResults[0]->toolName)->toBe('tool')
    ->and($toolResults[0]->args)->toBe(['input' => 'value'])
    ->and($toolResults[0]->result)->toBe('result');
```

## Testing Structured Output

> \[!NOTE]
> When testing OpenAI-style structured output (strict mode), the root schema should be an `ObjectSchema`.

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Testing\StructuredResponseFake;
use Prism\Prism\ValueObjects\Usage;
use Prism\Prism\ValueObjects\Meta;
use Prism\Prism\Enums\FinishReason;
use Prism\Prism\Schema\ObjectSchema;
use Prism\Prism\Schema\StringSchema;

it('can generate structured response', function () {
    $schema = new ObjectSchema(
        name: 'user',
        description: 'A user object, because we love organizing things!',
        properties: [
            new StringSchema('name', 'The user\'s name (hopefully not "test test")'),
            new StringSchema('bio', 'A brief bio (no novels, please)'),
        ],
        requiredFields: ['name', 'bio']
    );

    $fakeResponse = StructuredResponseFake::make()
        ->withText(json_encode([
            'name' => 'Alice Tester',
            'bio' => 'Professional bug hunter and code wrangler'
        ], JSON_THROW_ON_ERROR))
        ->withStructured([
            'name' => 'Alice Tester',
            'bio' => 'Professional bug hunter and code wrangler'
        ])
        ->withFinishReason(FinishReason::Stop)
        ->withUsage(new Usage(10, 20))
        ->withMeta(new Meta('fake-1', 'fake-model'));

    $fake = Prism::fake([$fakeResponse]);

    $response = Prism::structured()
        ->using('anthropic', 'claude-3-sonnet')
        ->withPrompt('Generate a user profile')
        ->withSchema($schema)
        ->asStructured();

    // Assertions
    expect($response->structured)->toBeArray();
    expect($response->structured['name'])->toBe('Alice Tester');
    expect($response->structured['bio'])->toBe('Professional bug hunter and code wrangler');
});
```

## Testing Embeddings

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\ValueObjects\Embedding;
use Prism\Prism\ValueObjects\EmbeddingsUsage;
use Prism\Prism\Testing\EmbeddingsResponseFake;
use Prism\Prism\ValueObjects\Meta;

it('can generate embeddings', function () {
    $fakeResponse = EmbeddingsResponseFake::make()
        ->withEmbeddings([Embedding::fromArray(array_fill(0, 1536, 0.1))])
        ->withUsage(new EmbeddingsUsage(10))
        ->withMeta(new Meta('fake-emb-1', 'fake-model'));

    Prism::fake([$fakeResponse]);

    $response = Prism::embeddings()
        ->using(Provider::OpenAI, 'text-embedding-3-small')
        ->fromInput('Test content for embedding generation.')
        ->asEmbeddings();

    expect($response->embeddings)->toHaveCount(1)
        ->and($response->embeddings[0]->embedding)
        ->toBeArray()
        ->toHaveCount(1536);
});
```

## Assertions

`PrismFake` provides several helpful assertion methods:

> \[!NOTE]
> When testing streamed responses, you must consume the stream before assertions will work. The `asStream()` method returns a generator, and the request is only recorded once the generator is iterated.
>
> ```php
> // Consume the stream before making assertions
> $chunks = collect($prism->asStream());
>
> // Now assertions will work
> $fake->assertCallCount(1);
> ```

```php
// Assert specific prompt was sent
$fake->assertPrompt('Who are you?');

// Assert number of calls made
$fake->assertCallCount(2);

// Assert detailed request properties
$fake->assertRequest(function ($requests) {
    expect($requests[0]->provider())->toBe('anthropic');
    expect($requests[0]->model())->toBe('claude-3-sonnet');
});

// Assert provider configuration
$fake->assertProviderConfig(['api_key' => 'sk-1234']);
```

## Using the real response classes

While the fake helpers make tests concise, you can still build responses with the real
classes if you know you will need to test against all the properties of the response:

```php
use Prism\Prism\Text\Response;
use Illuminate\Support\Collection;
use Prism\Prism\Enums\FinishReason;
use Prism\Prism\ValueObjects\Usage;
use Prism\Prism\ValueObjects\Meta;

$response = new Response(
    steps: collect([]),
    responseMessages: collect([]),
    text: 'The meaning of life is 42',
    finishReason: FinishReason::Stop,
    toolCalls: [],
    toolResults: [],
    usage: new Usage(42, 42),
    meta: new Meta('resp_1', 'real-model'),
    messages: collect([]),
    additionalContent: [],
);
```

This approach is perfectly valid—but for most tests the fake builders are shorter and
easier to read.

---

---
url: /core-concepts/text-generation.md
---
# Text Generation

Prism provides a powerful interface for generating text using Large Language Models (LLMs). This guide covers everything from basic usage to advanced features like multi-modal interactions and response handling.

## Basic Text Generation

At its simplest, you can generate text with just a few lines of code:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;

$response = Prism::text()
    ->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
    ->withPrompt('Tell me a short story about a brave knight.')
    ->asText();

echo $response->text;
```

## System Prompts and Context

System prompts help set the behavior and context for the AI. They're particularly useful for maintaining consistent responses or giving the LLM a persona:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;

$response = Prism::text()
    ->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
    ->withSystemPrompt('You are an expert mathematician who explains concepts simply.')
    ->withPrompt('Explain the Pythagorean theorem.')
    ->asText();
```

You can also use Laravel views for complex system prompts:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;

$response = Prism::text()
    ->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
    ->withSystemPrompt(view('prompts.math-tutor'))
    ->withPrompt('What is calculus?')
    ->asText();
```

You an also pass a View to the `withPrompt` method.

## Multi-Modal Input

Prism supports including images, documents, audio, and video files in your prompts for rich multi-modal analysis:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\ValueObjects\Media\Image;
use Prism\Prism\ValueObjects\Media\Document;
use Prism\Prism\ValueObjects\Media\Audio;
use Prism\Prism\ValueObjects\Media\Video;

// Analyze an image
$response = Prism::text()
    ->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
    ->withPrompt(
        'What objects do you see in this image?',
        [Image::fromLocalPath('/path/to/image.jpg')]
    )
    ->asText();

// Process a document
$response = Prism::text()
    ->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
    ->withPrompt(
        'Summarize the key points from this document',
        [Document::fromLocalPath('/path/to/document.pdf')]
    )
    ->asText();

// Analyze audio content
$response = Prism::text()
    ->using(Provider::Gemini, 'gemini-1.5-flash')
    ->withPrompt(
        'What is being discussed in this audio?',
        [Audio::fromLocalPath('/path/to/audio.mp3')]
    )
    ->asText();

// Process video content
$response = Prism::text()
    ->using(Provider::Gemini, 'gemini-1.5-flash')
    ->withPrompt(
        'Describe what happens in this video',
        [Video::fromUrl('https://www.youtube.com/watch?v=dQw4w9WgXcQ')]
    )
    ->asText();

// Multiple media types in one prompt
$response = Prism::text()
    ->using(Provider::Gemini, 'gemini-1.5-flash')
    ->withPrompt(
        'Compare this image with the information in this document',
        [
            Image::fromLocalPath('/path/to/chart.png'),
            Document::fromLocalPath('/path/to/report.pdf')
        ]
    )
    ->asText();
```

## Message Chains and Conversations

For interactive conversations, use message chains to maintain context:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\ValueObjects\Messages\UserMessage;
use Prism\Prism\ValueObjects\Messages\AssistantMessage;

$response = Prism::text()
    ->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
    ->withMessages([
        new UserMessage('What is JSON?'),
        new AssistantMessage('JSON is a lightweight data format...'),
        new UserMessage('Can you show me an example?')
    ])
    ->asText();
```

### Message Types

* `SystemMessage`
* `UserMessage`
* `AssistantMessage`
* `ToolResultMessage`

> \[!NOTE]
> Some providers, like Anthropic, do not support the `SystemMessage` type. In those cases we convert `SystemMessage` to `UserMessage`.

## Generation Parameters

Fine-tune your generations with various parameters:

`withMaxTokens`

Maximum number of tokens to generate.

`usingTemperature`

Temperature setting.

The value is passed through to the provider. The range depends on the provider and model. For most providers, 0 means almost deterministic results, and higher values mean more randomness.

> \[!TIP]
> It is recommended to set either temperature or topP, but not both.

`usingTopP`

Nucleus sampling.

The value is passed through to the provider. The range depends on the provider and model. For most providers, nucleus sampling is a number between 0 and 1. E.g. 0.1 would mean that only tokens with the top 10% probability mass are considered.

> \[!TIP]
> It is recommended to set either temperature or topP, but not both.

`withClientOptions`

Under the hood we use Laravel's [HTTP client](https://laravel.com/docs/11.x/http-client#main-content). You can use this method to pass any of Guzzles [request options](https://docs.guzzlephp.org/en/stable/request-options.html) e.g. `->withClientOptions(['timeout' => 30])`.

`withClientRetry`

Under the hood we use Laravel's [HTTP client](https://laravel.com/docs/11.x/http-client#main-content). You can use this method to set [retries](https://laravel.com/docs/11.x/http-client#retries) e.g. `->withClientRetry(3, 100)`.

`usingProviderConfig`

This allows for complete or partial override of the providers configuration. This is great for multi-tenant applications where users supply their own API keys. These values are merged with the original configuration allowing for partial or complete config override.

## Response Handling

The response object provides rich access to the generation results:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;

$response = Prism::text()
    ->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
    ->withPrompt('Explain quantum computing.')
    ->asText();

// Access the generated text
echo $response->text;

// Check why the generation stopped
echo $response->finishReason->name;

// Get token usage statistics
echo "Prompt tokens: {$response->usage->promptTokens}";
echo "Completion tokens: {$response->usage->completionTokens}";

// Access the raw API response data
$rawResponse = $response->raw;

// For multi-step generations, examine each step
foreach ($response->steps as $step) {
    echo "Step text: {$step->text}";
    echo "Step tokens: {$step->usage->completionTokens}";
    // Access raw response for individual steps
    $stepRawResponse = $step->raw;
}

// Access message history
foreach ($response->responseMessages as $message) {
    if ($message instanceof AssistantMessage) {
        echo $message->content;
    }
}
```

## Handling Completions with Callbacks

Need to perform actions after text generation completes? Pass a callback directly to `asText()` to handle the response without interrupting the return flow. This is perfect for persisting conversations, tracking analytics, or logging AI interactions.

### Basic Example

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\Text\PendingRequest;
use Prism\Prism\Text\Response;

$response = Prism::text()
    ->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
    ->withPrompt('Explain Laravel middleware')
    ->asText(function (PendingRequest $request, Response $response) {
        // Save the conversation after generation completes
        ConversationLog::create([
            'content' => $response->text,
            'role' => 'assistant',
            'tool_calls' => $response->toolCalls,
            'usage' => [
                'prompt_tokens' => $response->usage->promptTokens,
                'completion_tokens' => $response->usage->completionTokens,
            ],
        ]);
    });

// Response is still returned normally
echo $response->text;
```

The callback receives the `PendingRequest` and complete `Response` object, giving you access to the full response including text, tool calls, tool results, and usage statistics.

## Error Handling

Remember to handle potential errors in your generations:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\Exceptions\PrismException;
use Throwable;

try {
    $response = Prism::text()
        ->using(Provider::Anthropic, 'claude-3-5-sonnet-20241022')
        ->withPrompt('Generate text...')
        ->asText();
} catch (PrismException $e) {
    Log::error('Text generation failed:', ['error' => $e->getMessage()]);
} catch (Throwable $e) {
    Log::error('Generic error:', ['error' => $e->getMessage()]);
}
```

---

---
url: /core-concepts/tools-function-calling.md
---
# Tools & Function Calling

Need your AI assistant to check the weather, search a database, or call your API? Tools are here to help! They let you extend your AI's capabilities by giving it access to specific functions it can call.

## Tool Concept Overview

Think of tools as special functions that your AI assistant can use when it needs to perform specific tasks. Just like how Laravel's facades provide a clean interface to complex functionality, Prism tools give your AI a clean way to interact with external services and data sources.

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\Facades\Tool;

$weatherTool = Tool::as('weather')
    ->for('Get current weather conditions')
    ->withStringParameter('city', 'The city to get weather for')
    ->using(function (string $city): string {
        // Your weather API logic here
        return "The weather in {$city} is sunny and 72°F.";
    });

$response = Prism::text()
    ->using(Provider::Anthropic, 'claude-3-5-sonnet-latest')
    ->withMaxSteps(2)
    ->withPrompt('What is the weather like in Paris?')
    ->withTools([$weatherTool])
    ->asText();
```

## Max Steps

Prism defaults to allowing a single step. To use Tools, you'll need to increase this using `withMaxSteps`:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;

Prism::text()
    ->using(Provider::Anthropic, 'claude-3-5-sonnet-latest')
    // Increase max steps to at least 2
    ->withMaxSteps(2)
    ->withPrompt('What is the weather like in Paris?')
    ->withTools([$weatherTool])
    ->asText();
```

You should use a higher number of max steps if you expect your initial prompt to make multiple tool calls.

## Creating Basic Tools

Creating tools in Prism is straightforward and fluent. Here's how you can create a simple tool:

```php
use Prism\Prism\Facades\Tool;

$searchTool = Tool::as('search')
    ->for('Search for current information')
    ->withStringParameter('query', 'The search query')
    ->using(function (string $query): string {
        // Your search implementation
        return "Search results for: {$query}";
    });
```

Tools can take a variety of parameters, but must always return a string.

## Error Handling

By default, tools handle invalid parameters gracefully by returning error messages instead of throwing exceptions. This helps AI assistants understand and potentially correct their mistakes.

```php
$tool = Tool::as('calculate')
    ->for('Add two numbers')
    ->withNumberParameter('a', 'First number')
    ->withNumberParameter('b', 'Second number')
    ->using(fn (int $a, int $b): string => (string) ($a + $b));

// If AI provides invalid parameters, it receives:
// "Parameter validation error: Type mismatch. Expected: [a (NumberSchema, required), b (NumberSchema, required)]. Received: {"a":"five","b":10}"
```

### Opting Out

If you prefer exceptions for invalid parameters:

```php
// Per-tool
$tool->withoutErrorHandling();

// Per-request
Prism::text()->withoutToolErrorHandling();
```

**Best Practice**: Use default error handling for conversational AI. Disable it only when you need strict validation that stops execution.

## Parameter Definition

Prism offers multiple ways to define tool parameters, from simple primitives to complex objects.

### String Parameters

Perfect for text inputs:

```php
use Prism\Prism\Facades\Tool;

$tool = Tool::as('search')
    ->for('Search for information')
    ->withStringParameter('query', 'The search query')
    ->using(function (string $query): string {
        return "Search results for: {$query}";
    });
```

### Number Parameters

For integer or floating-point values:

```php
use Prism\Prism\Facades\Tool;

$tool = Tool::as('calculate')
    ->for('Perform calculations')
    ->withNumberParameter('value', 'The number to process')
    ->using(function (float $value): string {
        return "Calculated result: {$value * 2}";
    });
```

### Boolean Parameters

For true/false flags:

```php
use Prism\Prism\Facades\Tool;

$tool = Tool::as('feature_toggle')
    ->for('Toggle a feature')
    ->withBooleanParameter('enabled', 'Whether to enable the feature')
    ->using(function (bool $enabled): string {
        return "Feature is now " . ($enabled ? 'enabled' : 'disabled');
    });
```

### Array Parameters

For handling lists of items:

```php
use Prism\Prism\Facades\Tool;

$tool = Tool::as('process_tags')
    ->for('Process a list of tags')
    ->withArrayParameter(
        'tags',
        'List of tags to process',
        new StringSchema('tag', 'A single tag')
    )
    ->using(function (array $tags): string {
        return "Processing tags: " . implode(', ', $tags);
    });
```

### Enum Parameters

When you need to restrict values to a specific set:

```php
use Prism\Prism\Facades\Tool;

$tool = Tool::as('set_status')
    ->for('Set the status')
    ->withEnumParameter(
        'status',
        'The new status',
        ['draft', 'published', 'archived']
    )
    ->using(function (string $status): string {
        return "Status set to: {$status}";
    });
```

### Object Parameters

For complex objects without needing to create separate schema instances:

```php
use Prism\Prism\Facades\Tool;
use Prism\Prism\Schema\StringSchema;
use Prism\Prism\Schema\NumberSchema;

$tool = Tool::as('update_user')
    ->for('Update a user profile')
    ->withObjectParameter(
        'user',
        'The user profile data',
        [
            new StringSchema('name', 'User\'s full name'),
            new NumberSchema('age', 'User\'s age'),
            new StringSchema('email', 'User\'s email address')
        ],
        requiredFields: ['name', 'email']
    )
    ->using(function (array $user): string {
        return "Updated user profile for: {$user['name']}";
    });
```

### Schema-based Parameters

For complex, nested data structures, you can use Prism's schema system:

```php
use Prism\Prism\Facades\Tool;
use Prism\Prism\Schema\ObjectSchema;
use Prism\Prism\Schema\StringSchema;
use Prism\Prism\Schema\NumberSchema;

$tool = Tool::as('create_user')
    ->for('Create a new user profile')
    ->withParameter(new ObjectSchema(
        name: 'user',
        description: 'The user profile data',
        properties: [
            new StringSchema('name', 'User\'s full name'),
            new NumberSchema('age', 'User\'s age'),
            new StringSchema('email', 'User\'s email address')
        ],
        requiredFields: ['name', 'email']
    ))
    ->using(function (array $user): string {
        return "Created user profile for: {$user['name']}";
    });
```

> \[!TIP]
> For more complex parameter definitions, Prism provides a powerful schema system. See our [complete schemas guide](/core-concepts/schemas) to learn how to define complex nested objects, arrays, enums, and more.

## Complex Tool Implementation

For more sophisticated tools, you can create dedicated classes:

```php
namespace App\Tools;

use Prism\Prism\Tool;
use Illuminate\Support\Facades\Http;

class SearchTool extends Tool
{
    public function __construct()
    {
        $this
            ->as('search')
            ->for('useful when you need to search for current events')
            ->withStringParameter('query', 'Detailed search query. Best to search one topic at a time.')
            ->using($this);
    }

    public function __invoke(string $query): string
    {
        $response = Http::get('https://serpapi.com/search', [
            'engine' => 'google',
            'q' => $query,
            'google_domain' => 'google.com',
            'gl' => 'us',
            'hl' => 'en',
            'api_key' => config('services.serpapi.api_key'),
        ]);

        $results = collect($response->json('organic_results'));

        $results->map(function ($result) {
            return [
                'title' => $result['title'],
                'link' => $result['link'],
                'snippet' => $result['snippet'],
            ];
        })->take(4);

        return view('prompts.search-tool-results', [
            'results' => $results,
        ])->render();
    }
}
```

You can use `Tool::make($className)` if you need to resolve the dependencies:

```php
use App\Tools\SearchTool;
use Prism\Prism\Facades\Tool;

$tool = Tool::make(SearchTool::class);
```

## Concurrent Tool Execution

When the AI calls multiple tools in a single step, Prism normally executes them sequentially. For I/O-bound operations like API calls or database queries, you can enable concurrent execution to run tools in parallel, reducing total wait time.

### Marking Tools as Concurrent

Use the `concurrent()` method to mark a tool as safe for parallel execution:

```php
use Prism\Prism\Facades\Tool;

$weatherTool = Tool::as('weather')
    ->for('Get current weather conditions')
    ->withStringParameter('city', 'The city to get weather for')
    ->using(function (string $city): string {
        // API call that takes ~500ms
        return Http::get("https://api.weather.com/{$city}")->json('conditions');
    })
    ->concurrent();

$stockTool = Tool::as('stock_price')
    ->for('Get current stock price')
    ->withStringParameter('symbol', 'The stock ticker symbol')
    ->using(function (string $symbol): string {
        // Another API call that takes ~500ms
        return Http::get("https://api.stocks.com/{$symbol}")->json('price');
    })
    ->concurrent();
```

When the AI calls both tools in a single step, they'll execute in parallel instead of sequentially - taking ~500ms total instead of ~1000ms.

### How It Works

Prism uses [Laravel's Concurrency facade](https://laravel.com/docs/12.x/concurrency) to execute concurrent tools. Under the hood, tools marked as concurrent are grouped and run in parallel, while sequential tools run one at a time.

The execution flow:

1. Prism groups tool calls by their concurrency setting
2. Concurrent tools execute in parallel via `Concurrency::run()`
3. Sequential tools execute one at a time
4. Results are returned in the original order, regardless of execution order

### When to Use Concurrent Tools

**Good candidates for concurrent execution:**

* External API calls (weather, stocks, search)
* Database queries that don't depend on each other
* File reads from different sources
* Any I/O-bound operation

**Keep sequential (don't mark as concurrent):**

* Tools that modify shared state
* Tools where execution order matters
* Tools with side effects that could conflict
* CPU-bound operations (concurrency won't help)

### Mixed Execution

You can mix concurrent and sequential tools in the same request:

```php
$searchTool = Tool::as('search')
    ->for('Search the web')
    ->withStringParameter('query', 'Search query')
    ->using(fn (string $query): string => $this->search($query))
    ->concurrent(); // Safe to run in parallel

$saveResultTool = Tool::as('save_result')
    ->for('Save a result to the database')
    ->withStringParameter('data', 'Data to save')
    ->using(fn (string $data): string => $this->save($data));
    // Sequential - modifies database state
```

Prism handles the grouping automatically. Concurrent tools run in parallel, then sequential tools run in order.

### Error Handling

Errors in concurrent tools are handled the same way as sequential tools. If one concurrent tool fails, other concurrent tools still complete, and all results (including errors) are returned in the original order.

> \[!NOTE]
> Concurrent execution requires Laravel's Concurrency feature, available in Laravel 11+. Make sure you have the appropriate concurrency driver configured. See [Laravel's Concurrency documentation](https://laravel.com/docs/12.x/concurrency) for setup details.

## Using Laravel MCP Tools

You can use existing [Laravel MCP](https://github.com/laravel/mcp) Tools in Prism directly, without using the Laravel MCP Server:

```php
use App\Mcp\Tools\CurrentWeatherTool;
use Prism\Prism\Facades\Tool;

$tool = Tool::make(CurrentWeatherTool::class);
```

## Tool Choice Options

You can control how the AI uses tools with the `withToolChoice` method:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\Enums\ToolChoice;

$prism = Prism::text()
    ->using(Provider::Anthropic, 'claude-3-5-sonnet-latest')
    ->withMaxSteps(2)
    ->withPrompt('How is the weather in Paris?')
    ->withTools([$weatherTool])
    // Let the AI decide whether to use tools
    ->withToolChoice(ToolChoice::Auto)
    // Force the AI to use a tool
    ->withToolChoice(ToolChoice::Any)
    // Force the AI to use a specific tool
    ->withToolChoice('weather');
```

> \[!WARNING]
> Tool choice support varies by provider. Check your provider's documentation for specific capabilities.

## Response Handling with Tools

When your AI uses tools, you can inspect the results and see how it arrived at its answer:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;

$response = Prism::text()
    ->using(Provider::Anthropic, 'claude-3-5-sonnet-latest')
    ->withMaxSteps(2)
    ->withPrompt('What is the weather like in Paris?')
    ->withTools([$weatherTool])
    ->asText();

// Get the final answer
echo $response->text;

// ->text is empty for tool calls

// Inspect tool usage

if ($response->toolResults) {
    foreach ($response->toolResults as $toolResult) {
        echo "Tool: " . $toolResult->toolName . "\n";
        echo "Result: " . $toolResult->result . "\n";
    }
}


foreach ($response->steps as $step) {
    if ($step->toolCalls) {
        foreach ($step->toolCalls as $toolCall) {
            echo "Tool: " . $toolCall->name . "\n";
            echo "Arguments: " . json_encode($toolCall->arguments()) . "\n";
        }
    }
}
```

## Tool Artifacts

Sometimes tools need to produce binary data like images, audio, or files alongside their text response. Prism's Artifact system lets you return rich data without bloating the LLM's context window.

### The Problem with Binary Data

Normally, everything your tool returns goes to the LLM as context. This works fine for text, but for binary data like generated images:

* Base64-encoded images would waste tokens
* The LLM can't meaningfully process raw binary data
* Large payloads slow down responses

### The Solution: ToolOutput with Artifacts

Instead of returning a string, return a `ToolOutput` that separates the text result (for the LLM) from artifacts (for your application):

```php
use Prism\Prism\Facades\Tool;
use Prism\Prism\ValueObjects\Artifact;
use Prism\Prism\ValueObjects\ToolOutput;

$imageTool = Tool::as('generate_image')
    ->for('Generate an image from a prompt')
    ->withStringParameter('prompt', 'The image prompt')
    ->using(function (string $prompt): ToolOutput {
        // Your image generation logic
        $imageData = $this->imageGenerator->generate($prompt);

        return new ToolOutput(
            result: json_encode(['status' => 'success', 'description' => $prompt]),
            artifacts: [
                Artifact::fromRawContent(
                    content: $imageData,
                    mimeType: 'image/png',
                    metadata: ['width' => 1024, 'height' => 1024],
                    id: 'generated-image-001',
                ),
            ],
        );
    });
```

The `result` goes to the LLM. The `artifacts` travel through the streaming system to your application.

### Creating Artifacts

The `Artifact` class represents binary or structured data:

```php
use Prism\Prism\ValueObjects\Artifact;

// From raw content (automatically base64 encoded)
$artifact = Artifact::fromRawContent(
    content: file_get_contents('image.png'),
    mimeType: 'image/png',
    metadata: ['width' => 800, 'height' => 600],
    id: 'my-image-id',
);

// From already base64-encoded data
$artifact = new Artifact(
    data: base64_encode($rawData),
    mimeType: 'application/pdf',
    metadata: ['pages' => 5],
    id: 'report-001',
);

// Get raw content back
$rawContent = $artifact->rawContent();
```

### Handling Artifacts in Streams

Artifacts are emitted as `ArtifactEvent` through all streaming methods:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Streaming\Events\ArtifactEvent;

// Using asStream()
foreach (Prism::text()->withTools([$imageTool])->asStream() as $event) {
    if ($event instanceof ArtifactEvent) {
        $artifact = $event->artifact;
        file_put_contents(
            "output/{$event->toolName}_{$artifact->id}.png",
            $artifact->rawContent()
        );
    }
}

// Using asDataStreamResponse() with callback
Prism::text()
    ->withTools([$imageTool])
    ->asDataStreamResponse(function ($pendingRequest, $events) use ($conversationId) {
        foreach ($events as $event) {
            if ($event instanceof ArtifactEvent) {
                Attachment::create([
                    'conversation_id' => $conversationId,
                    'data' => $event->artifact->rawContent(),
                    'mime_type' => $event->artifact->mimeType,
                ]);
            }
        }
    });
```

### Non-Streaming Mode

In non-streaming mode, artifacts are available on the `ToolResult` objects:

```php
$response = Prism::text()
    ->withTools([$imageTool])
    ->withMaxSteps(3)
    ->withPrompt('Generate an image of a sunset')
    ->asText();

foreach ($response->toolResults as $result) {
    if ($result->hasArtifacts()) {
        foreach ($result->artifacts as $artifact) {
            // Process artifact
            file_put_contents(
                "output/{$artifact->id}.png",
                $artifact->rawContent()
            );
        }
    }
}
```

### Backward Compatibility

Tools returning `string` continue to work unchanged. The `ToolOutput` return type is optional:

```php
// Both are valid:
->using(fn (string $query): string => "Result: {$query}");
->using(fn (string $query): ToolOutput => new ToolOutput(result: "Result: {$query}"));
```

## Provider Tools

In addition to custom tools that you define, Prism supports **provider tools** - built-in capabilities offered directly by AI providers. These are specialized tools that leverage the provider's own infrastructure and services.

### Understanding Provider Tools vs Custom Tools

**Custom Tools** (covered above) are functions you define and implement yourself:

* You control the logic and implementation
* Called by the AI, executed by your code
* Can access your databases, APIs, and services

**Provider Tools** are built-in capabilities offered by the AI provider:

* Implemented and executed by the provider
* Access the provider's own services and infrastructure
* Enable capabilities like code execution, web search, and more

### Using Provider Tools

Provider tools are added to your requests using the `withProviderTools()` method with `ProviderTool` objects:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\ValueObjects\ProviderTool;

$response = Prism::text()
    ->using('anthropic', 'claude-3-5-sonnet-latest')
    ->withPrompt('Calculate the fibonacci sequence up to 100')
    ->withProviderTools([
        new ProviderTool(type: 'code_execution_20250522', name: 'code_execution')
    ])
    ->asText();
```

### Available Provider Tools

Each provider offers different built-in capabilities. Check the provider-specific documentation for detailed information about available tools, configuration options, and usage examples.

### ProviderTool Object

The `ProviderTool` class accepts three parameters:

```php
new ProviderTool(
    type: 'code_execution_20250522',  // Required: The provider tool identifier
    name: 'code_execution',           // Optional: Custom name for the tool
    options: []                       // Optional: Provider-specific options
)
```

* **type**: The provider-specific tool identifier (required)
* **name**: Optional custom name for the tool
* **options**: Additional provider-specific configuration options

### Combining Provider Tools and Custom Tools

You can use both provider tools and custom tools in the same request:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\ValueObjects\ProviderTool;
use Prism\Prism\Facades\Tool;

$customTool = Tool::as('database_lookup')
    ->for('Look up user information')
    ->withStringParameter('user_id', 'The user ID to look up')
    ->using(function (string $userId): string {
        // Your database lookup logic
        return "User data for ID: {$userId}";
    });

$response = Prism::text()
    ->using('anthropic', 'claude-3-5-sonnet-latest')
    ->withMaxSteps(5)
    ->withPrompt('Look up user 123 and calculate their usage statistics')
    ->withTools([$customTool])
    ->withProviderTools([
        new ProviderTool(type: 'code_execution_20250522', name: 'code_execution')
    ])
    ->asText();
```

## Using Tools with Structured Output

Tools can be combined with structured output to gather data and return formatted results in a single request. This pattern is useful when you need the AI to call functions to fetch information, then format the results according to a specific schema.

### Basic Example

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Schema\ObjectSchema;
use Prism\Prism\Schema\StringSchema;
use Prism\Prism\Facades\Tool;

$schema = new ObjectSchema(
    name: 'weather_analysis',
    description: 'Analysis of weather conditions',
    properties: [
        new StringSchema('summary', 'Summary of the weather'),
        new StringSchema('recommendation', 'Recommendation based on weather'),
    ],
    requiredFields: ['summary', 'recommendation']
);

$weatherTool = Tool::as('get_weather')
    ->for('Get current weather for a location')
    ->withStringParameter('location', 'The city and state')
    ->using(fn (string $location): string => "Weather in {$location}: 72°F, sunny");

$response = Prism::structured()
    ->using('anthropic', 'claude-3-5-sonnet-latest')
    ->withSchema($schema)
    ->withTools([$weatherTool])
    ->withMaxSteps(3)
    ->withPrompt('What is the weather in San Francisco and should I wear a coat?')
    ->asStructured();

// Response contains both structured data and tool execution details
dump($response->structured);
```

> \[!IMPORTANT]
> When combining tools with structured output, you must set `maxSteps` to at least 2. The AI needs multiple steps to call tools and then return structured output.

### Response Structure

Responses include both the structured output and tool execution details:

```php
// Final structured data
$data = $response->structured;

// All tool calls made during execution
foreach ($response->toolCalls as $toolCall) {
    echo "Called: {$toolCall->name}\n";
}

// Tool execution results
foreach ($response->toolResults as $result) {
    echo "Result: {$result->result}\n";
}
```

> \[!NOTE]
> Only the final step contains structured data. Intermediate steps contain tool calls and results, but no structured output.

For complete documentation on combining tools with structured output, see the [Structured Output](./structured-output.md#combining-structured-output-with-tools) documentation.

---

---
url: /index.md
---


---

---
url: /input-modalities/video.md
---
# Video

Prism supports including video files and YouTube videos in your messages for advanced analysis with supported providers like Gemini.

See the [provider support table](/getting-started/introduction.html#provider-support) to check whether Prism supports your chosen provider.

Note however that provider support may differ by model. If you receive error messages with a provider that Prism indicates is supported, check the provider's documentation as to whether the model you are using supports video files.

::: tip
For other input modalities like audio and images, see their respective documentation pages:

* [Audio documentation](/input-modalities/audio.html)
* [Images documentation](/input-modalities/images.html)
  :::

## Getting started

To add a video to your prompt, use the `withPrompt` method with a `Video` value object:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\ValueObjects\Media\Video;

// From a local path
$response = Prism::text()
    ->using(Provider::Gemini, 'gemini-1.5-flash')
    ->withPrompt(
        "What's in this video?",
        [Video::fromLocalPath(path: '/path/to/video.mp4')]
    )
    ->asText();

// From a path on a storage disk
$response = Prism::text()
    ->using(Provider::Gemini, 'gemini-1.5-flash')
    ->withPrompt(
        "What's in this video?",
        [Video::fromStoragePath(
            path: '/path/to/video.mp4',
            diskName: 'my-disk' // optional - omit/null for default disk
        )]
    )
    ->asText();

// From a URL
$response = Prism::text()
    ->using(Provider::Gemini, 'gemini-1.5-flash')
    ->withPrompt(
        'Analyze this video:',
        [Video::fromUrl(url: 'https://example.com/video.mp4')]
    )
    ->asText();

// From a YouTube URL (automatically extracts the video ID)
$response = Prism::text()
    ->using(Provider::Gemini, 'gemini-1.5-flash')
    ->withPrompt(
        'What is this YouTube video about?',
        [Video::fromUrl(url: 'https://www.youtube.com/watch?v=dQw4w9WgXcQ')]
    )
    ->asText();

// From shortened YouTube URL
$response = Prism::text()
    ->using(Provider::Gemini, 'gemini-1.5-flash')
    ->withPrompt(
        'What is this YouTube video about?',
        [Video::fromUrl(url: 'https://youtu.be/dQw4w9WgXcQ')]
    )
    ->asText();

// From base64
$response = Prism::text()
    ->using(Provider::Gemini, 'gemini-1.5-flash')
    ->withPrompt(
        'Analyze this video:',
        [Video::fromBase64(
            base64: base64_encode(file_get_contents('/path/to/video.mp4')),
            mimeType: 'video/mp4'
        )]
    )
    ->asText();

// From raw content
$response = Prism::text()
    ->using(Provider::Gemini, 'gemini-1.5-flash')
    ->withPrompt(
        'Analyze this video:',
        [Video::fromRawContent(
            rawContent: file_get_contents('/path/to/video.mp4'),
            mimeType: 'video/mp4'
        )]
    )
    ->asText();
```

## Alternative: Using withMessages

You can also include videos using the message-based approach:

```php
use Prism\Prism\ValueObjects\Messages\UserMessage;
use Prism\Prism\ValueObjects\Media\Video;

$message = new UserMessage(
    "What's in this video?",
    [Video::fromLocalPath(path: '/path/to/video.mp4')]
);

$response = Prism::text()
    ->using(Provider::Gemini, 'gemini-1.5-flash')
    ->withMessages([$message])
    ->asText();
```

## Supported Video Types

Prism supports a variety of video formats, including:

* MP4 (video/mp4)
* MOV (video/quicktime)
* WEBM (video/webm)
* AVI (video/x-msvideo)
* YouTube videos (via URL)

The specific supported formats depend on the provider. Gemini is currently the main provider with comprehensive video analysis capabilities. Check the provider's documentation for a complete list of supported formats.

## YouTube Video Support

Prism provides seamless support for YouTube videos. When you pass a YouTube URL to `Video::fromUrl()`, Prism automatically extracts the video ID and sends it to the provider in the appropriate format.

Supported YouTube URL formats:

* Standard: `https://www.youtube.com/watch?v=VIDEO_ID`
* Shortened: `https://youtu.be/VIDEO_ID`

Example:

```php
$response = Prism::text()
    ->using(Provider::Gemini, 'gemini-1.5-flash')
    ->withPrompt(
        'What is this YouTube video about?',
        [Video::fromUrl('https://www.youtube.com/watch?v=dQw4w9WgXcQ')]
    )
    ->asText();
```

## Transfer mediums

Providers are not consistent in their support of sending raw contents, base64 and/or URLs.

Prism tries to smooth over these rough edges, but its not always possible.

### Supported conversions

* Where a provider does not support URLs: Prism will fetch the URL and use base64 or rawContent.
* Where you provide a file, base64 or rawContent: Prism will switch between base64 and rawContent depending on what the provider accepts.

### Limitations

* Where a provider only supports URLs: if you provide a file path, raw contents or base64, for security reasons Prism does not create a URL for you and your request will fail.

---

---
url: /providers/voyageai.md
---
# Voyage AI

## Configuration

```php
'voyageai' => [
    'api_key' => env('VOYAGEAI_API_KEY', ''),
    'url' => env('VOYAGEAI_URL', 'https://api.voyageai.com/v1'),
],
```

## Provider specific options

You can change some options on your request specific to Voyage AI by using `->withProviderOptions()`.

### Input type

By default, Voyage AI generates general purpose vectors.

However, they taylor your vectors for the task they are intended for - for search ("query") or for retrieval ("document"):

For search / querying:

```php
use Prism\Prism\Enums\Provider;
use Prism\Prism\Facades\Prism;

Prism::embeddings()
    ->using(Provider::VoyageAI, 'voyage-3-lite')
    ->fromInput('The food was delicious and the waiter...')
    ->withProviderOptions(['inputType' => 'query'])
    ->asEmbeddings();
```

For document retrieval:

```php
use Prism\Prism\Enums\Provider;
use Prism\Prism\Facades\Prism;

Prism::embeddings()
    ->using(Provider::VoyageAI, 'voyage-3-lite')
    ->fromInput('The food was delicious and the waiter...')
    ->withProviderOptions(['inputType' => 'document'])
    ->asEmbeddings();
```

### Truncation

By default, Voyage AI truncates inputs that are over the context length.

You can force it to throw an error instead by setting truncation to false.

```php
use Prism\Prism\Enums\Provider;
use Prism\Prism\Facades\Prism;

Prism::embeddings()
    ->using(Provider::VoyageAI, 'voyage-3-lite')
    ->fromInput('The food was delicious and the waiter...')
    ->withProviderOptions(['truncation' => false])
    ->asEmbeddings();
```

---

---
url: /providers/xai.md
---
# xAI

## Configuration

```php
'xai' => [
    'api_key' => env('XAI_API_KEY', ''),
    'url' => env('XAI_URL', 'https://api.x.ai/v1'),
],
```

## Provider-specific options

### Extended Thinking/Reasoning

xAI's Grok models support an optional extended thinking mode, where the model will reason through problems before returning its answer. This is particularly useful for complex mathematical problems, logical reasoning, and detailed analysis tasks.

#### Enabling thinking mode

```php
use Prism\Prism\Enums\Provider;
use Prism\Prism\Facades\Prism;

$response = Prism::text()
    ->using(Provider::XAI, 'grok-4')
    ->withPrompt('Solve this complex equation: 3x² + 5x - 2 = 0')
    ->withProviderOptions([
        'thinking' => ['enabled' => true]
    ])
    ->asText();
```

Thinking content is automatically extracted when present in the response and can be accessed through streaming events.
If you prefer not to process thinking content, you can disable it. Set the `thinking` option to `false`.

#### Streaming thinking content

When using streaming, thinking content is yielded as separate events:

```php
use Prism\Prism\Enums\StreamEventType;

$stream = Prism::text()
	->using(Provider::XAI, 'grok-4')
	->withPrompt('Explain quantum entanglement in detail')
	->asStream();

foreach ($stream as $event) {
	if ($event->type() === StreamEventType::ThinkingDelta) {
		echo $event->delta . PHP_EOL; // Outputs: Thinking...
	} elseif ($event->type() === StreamEventType::TextDelta) {
		echo $event->delta;
	}
}
```

### Structured Output

xAI supports structured output through JSON schema validation. The following models support structured output:

> \[!NOTE]
> xAI uses an OpenAI-compatible API. For strict schema validation, the root schema should be an `ObjectSchema`.

* `grok-3`
* `grok-4`

```php
use Prism\Prism\Schema\ObjectSchema;
use Prism\Prism\Schema\StringSchema;
use Prism\Prism\Schema\BooleanSchema;

$schema = new ObjectSchema(
    'weather_report',
    'Weather forecast with recommendations',
    [
        new StringSchema('forecast', 'The weather forecast'),
        new StringSchema('clothing', 'Clothing recommendation'),
        new BooleanSchema('coat_required', 'Whether a coat is needed'),
    ],
    ['forecast', 'clothing', 'coat_required']
);

$response = Prism::structured()
    ->withSchema($schema)
    ->using(Provider::XAI, 'grok-4')
    ->withPrompt('What\'s the weather like in Detroit and should I wear a coat?')
    ->asStructured();

// Access structured data
echo $response->structured['forecast']; // "75° and sunny"
echo $response->structured['coat_required']; // false
```

#### Strict schema mode

Enable strict schema validation for more reliable structured output:

```php
$response = Prism::structured()
    ->withSchema($schema)
    ->using(Provider::XAI, 'grok-4')
    ->withProviderOptions([
        'schema' => ['strict' => true]
    ])
    ->withPrompt('Analyze this data')
    ->asStructured();
```

### Tool Calling

xAI supports function calling with tools. Tools can be used alongside thinking mode for complex problem-solving scenarios.

```php
use Prism\Prism\Facades\Tool;

$tools = [
    Tool::as('calculator')
        ->for('Perform mathematical calculations')
        ->withStringParameter('expression', 'Mathematical expression to calculate')
        ->using(fn (string $expression): string => "Result: " . eval("return $expression;")),
        
    Tool::as('weather')
        ->for('Get current weather information')
        ->withStringParameter('city', 'City name')
        ->using(fn (string $city): string => "Weather in {$city}: 72°F and sunny"),
];

$response = Prism::text()
    ->using(Provider::XAI, 'grok-4')
    ->withTools($tools)
    ->withMaxSteps(3)
    ->withPrompt('Calculate 15 * 23 and tell me the weather in Detroit')
    ->asText();
```

### Model Parameters

#### Temperature Control

Control the randomness of responses:

```php
$response = Prism::text()
    ->using(Provider::XAI, 'grok-4')
    ->withTemperature(0.7) // 0.0 = deterministic, 1.0 = very creative
    ->withPrompt('Write a creative story')
    ->asText();
```

#### Top-P Sampling

Control nucleus sampling:

```php
$response = Prism::text()
    ->using(Provider::XAI, 'grok-4')
    ->withTopP(0.9) // Consider top 90% probability mass
    ->withPrompt('Generate diverse responses')
    ->asText();
```

#### Token Limits

Set maximum output tokens:

```php
$response = Prism::text()
    ->using(Provider::XAI, 'grok-4')
    ->withMaxTokens(1000)
    ->withPrompt('Write a detailed explanation')
    ->asText();
```

## Advanced Examples

### Complex Analysis with Thinking

```php
$response = Prism::text()
    ->using(Provider::XAI, 'grok-4')
    ->withPrompt('
        Analyze the economic implications of implementing a universal basic income program.
        Consider both potential benefits and drawbacks, and provide specific examples.
    ')
    ->asStream();

$analysis = '';
$reasoning = '';

foreach ($response as $chunk) {
    if ($chunk->chunkType === ChunkType::Thinking) {
        $reasoning .= $chunk->text;
    } else {
        $analysis .= $chunk->text;
        echo $chunk->text; // Stream to user
    }
}

// Save the reasoning process for later review
file_put_contents('analysis_reasoning.txt', $reasoning);
```

### Structured Data Extraction

```php
use Prism\Prism\Schema\ArraySchema;
use Prism\Prism\Schema\IntegerSchema;
use Prism\Prism\Schema\NumberSchema;

$schema = new ObjectSchema(
    'financial_analysis',
    'Complete financial analysis result',
    [
        new StringSchema('summary', 'Executive summary'),
        new NumberSchema('total_revenue', 'Total revenue amount'),
        new NumberSchema('profit_margin', 'Profit margin percentage'),
        new ArraySchema('recommendations', 'List of recommendations', 
            new StringSchema('recommendation', 'Individual recommendation')
        ),
        new ObjectSchema('risk_assessment', 'Risk analysis', [
            new StringSchema('level', 'Risk level (low/medium/high)'),
            new IntegerSchema('score', 'Risk score from 1-10'),
        ], ['level', 'score']),
    ],
    ['summary', 'total_revenue', 'profit_margin', 'recommendations', 'risk_assessment']
);

$response = Prism::structured()
    ->withSchema($schema)
    ->using(Provider::XAI, 'grok-4')
    ->withPrompt('
        Analyze this financial data: 
        Q1 Revenue: $1.2M, Q1 Costs: $800K
        Q2 Revenue: $1.5M, Q2 Costs: $900K
        Provide a complete analysis with recommendations.
    ')
    ->asStructured();

$analysis = $response->structured;
echo "Revenue: $" . number_format($analysis['total_revenue']);
echo "Risk Level: " . $analysis['risk_assessment']['level'];
```

### Model Validation

Structured output is only supported on specific models. Prism will throw an exception for unsupported models:

```php
use Prism\Prism\Exceptions\PrismException;

try {
    $response = Prism::structured()
        ->withSchema($schema)
        ->using(Provider::XAI, 'unsupported-model')
        ->asStructured();
} catch (PrismException $e) {
    // Handle unsupported model error
    echo "Error: " . $e->getMessage();
}
```

## Considerations

### Thinking Content Processing

* Thinking content is automatically filtered to remove repetitive "Thinking..." patterns
* Only meaningful reasoning content is yielded in thinking chunks
* Thinking content appears before regular response content in streams
* Thinking can be disabled if not needed to reduce processing overhead

### API Compatibility

xAI uses an OpenAI-compatible API structure, which means:

* Request/response formats are similar to OpenAI
* Tool calling follows OpenAI's function calling specification
* Structured output uses JSON schema format
* Streaming follows server-sent events (SSE) format

### Token Management

* Thinking tokens count toward your total token usage
* Set appropriate `maxTokens` limits when expecting long thinking sequences
* Monitor usage through the response objects for cost tracking

---

---
url: /providers/z.md
---
# Z AI

## Configuration

```php
'z' => [
    'url' => env('Z_URL', 'https://api.z.ai/api/coding/paas/v4'),
    'api_key' => env('Z_API_KEY', ''),
]
```

## Text Generation

Generate text responses with Z AI models:

```php
$response = Prism::text()
    ->using('z', 'glm-4.6')
    ->withPrompt('Write a short story about a robot learning to love')
    ->asText();

echo $response->text;
```

## Multi-modal Support

Z AI provides comprehensive multi-modal capabilities through the `glm-4.6v` model, allowing you to work with images, documents, and videos in your AI requests.

### Images

Z AI supports image analysis through URLs using the `glm-4.6v` model:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\ValueObjects\Media\Image;
use Prism\Prism\ValueObjects\Messages\UserMessage;

$response = Prism::text()
    ->using('z', 'glm-4.6v')
    ->withMessages([
        new UserMessage(
            'What is in this image?',
            additionalContent: [
                Image::fromUrl('https://example.com/image.png'),
            ]
        ),
    ])
    ->asText();
```

### Documents

Process documents directly from URLs:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\ValueObjects\Media\Document;
use Prism\Prism\ValueObjects\Messages\UserMessage;

$response = Prism::text()
    ->using('z', 'glm-4.6v')
    ->withMessages([
        new UserMessage(
            'What does this document say about?',
            additionalContent: [
                Document::fromUrl('https://example.com/document.pdf'),
            ]
        ),
    ])
    ->asText();
```

### Videos

Z AI can analyze video content from URLs:

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\ValueObjects\Media\Video;
use Prism\Prism\ValueObjects\Messages\UserMessage;

$response = Prism::text()
    ->using('z', 'glm-4.6v')
    ->withMessages([
        new UserMessage(
            'What does this video show?',
            additionalContent: [
                Video::fromUrl('https://example.com/video.mp4'),
            ]
        ),
    ])
    ->asText();
```

### Combining Multiple Media Types

You can combine images, documents, and videos in a single request:

```php
$response = Prism::text()
    ->using('z', 'glm-4.6v')
    ->withMessages([
        new UserMessage(
            'Analyze this image, document, and video together',
            additionalContent: [
                Image::fromUrl('https://example.com/image.png'),
                Document::fromUrl('https://example.com/document.txt'),
                Video::fromUrl('https://example.com/video.mp4'),
            ]
        ),
    ])
    ->asText();
```

## Tools and Function Calling

Z AI supports function calling, allowing the model to execute your custom tools during conversation.

### Basic Tool Usage

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Tool;

$weatherTool = Tool::as('get_weather')
    ->for('Get current weather for a location')
    ->withStringParameter('city', 'The city and state')
    ->using(fn (string $city): string => "Weather in {$city}: 72°F, sunny");

$response = Prism::text()
    ->using('z', 'glm-4.6')
    ->withPrompt('What is the weather in San Francisco?')
    ->withTools([$weatherTool])
    ->asText();
```

### Multiple Tools

Z AI can use multiple tools in a single request:

```php
$tools = [
    Tool::as('get_weather')
        ->for('Get current weather for a location')
        ->withStringParameter('city', 'The city that you want the weather for')
        ->using(fn (string $city): string => 'The weather will be 45° and cold'),

    Tool::as('search_games')
        ->for('Search for current game times in a city')
        ->withStringParameter('city', 'The city that you want the game times for')
        ->using(fn (string $city): string => 'The tigers game is at 3pm in detroit'),
];

$response = Prism::text()
    ->using('z', 'glm-4.6')
    ->withTools($tools)
    ->withMaxSteps(4)
    ->withPrompt('What time is the tigers game today in Detroit and should I wear a coat?')
    ->asText();
```

### Tool Choice

Control when tools are called:

```php
use Prism\Prism\Enums\ToolChoice;

// Require at least one tool to be called
$response = Prism::text()
    ->using('z', 'glm-4.6')
    ->withPrompt('Search for information')
    ->withTools([$searchTool, $weatherTool])
    ->withToolChoice(ToolChoice::Any)
    ->asText();

// Require a specific tool to be called
$response = Prism::text()
    ->using('z', 'glm-4.6')
    ->withPrompt('Get the weather')
    ->withTools([$searchTool, $weatherTool])
    ->withToolChoice(ToolChoice::from('get_weather'))
    ->asText();

// Let the model decide (default)
$response = Prism::text()
    ->using('z', 'glm-4.6')
    ->withPrompt('What do you think?')
    ->withTools([$tools])
    ->withToolChoice(ToolChoice::Auto)
    ->asText();
```

For complete tool documentation, see [Tools & Function Calling](/core-concepts/tools-function-calling).

## Structured Output

Z AI supports structured output through schema-based JSON generation, ensuring responses match your defined structure.

### Basic Structured Output

```php
use Prism\Prism\Facades\Prism;
use Prism\Prism\Schema\ObjectSchema;
use Prism\Prism\Schema\StringSchema;
use Prism\Prism\Schema\EnumSchema;
use Prism\Prism\Schema\BooleanSchema;

$schema = new ObjectSchema(
    'interview_response',
    'Structured response from AI interviewer',
    [
        new StringSchema('message', 'The interviewer response message'),
        new EnumSchema(
            'action',
            'The next action to take',
            ['ask_question', 'ask_followup', 'complete_interview']
        ),
        new BooleanSchema('is_question', 'Whether this contains a question'),
    ],
    ['message', 'action', 'is_question']
);

$response = Prism::structured()
    ->using('z', 'glm-4.6')
    ->withSchema($schema)
    ->withPrompt('Conduct an interview')
    ->asStructured();

// Access structured data
dump($response->structured);
// [
//     'message' => '...',
//     'action' => 'ask_question',
//     'is_question' => true
// ]
```

For complete structured output documentation, see [Structured Output](/core-concepts/structured-output).

## Limitations

### Media Types

* Does not support `Image::fromPath` or `Image::fromBase64` - only `Image::fromUrl`
* Does not support `Document::fromPath` or `Document::fromBase64` - only `Document::fromUrl`
* Does not support `Video::fromPath` or `Video::fromBase64` - only `Video::fromUrl`

All media must be provided as publicly accessible URLs that Z AI can fetch and process.