pydantic_ai.models

Logic related to making requests to an LLM.

The aim here is to make a common interface for different LLMs, so that the rest of the code can be agnostic to the specific LLM being used.

Model

Bases: ABC

Abstract class for a model.

agent_model abstractmethod

agent_model(
    retrievers: Mapping[str, AbstractToolDefinition],
    allow_text_result: bool,
    result_tools: Sequence[AbstractToolDefinition] | None,
) -> AgentModel

Create an agent model.

Parameters:

Name	Type	Description	Default
retrievers	Mapping[str, AbstractToolDefinition]	The retrievers available to the agent.	required
allow_text_result	bool	Whether a plain text final response/result is permitted.	required
result_tools	Sequence[AbstractToolDefinition] | None	Tool definitions for the final result tool(s), if any.	required

Returns:

Type	Description
AgentModel	An agent model.

AgentModel

Bases: ABC

Model configured for a specific agent.

request abstractmethod async

request(
    messages: list[Message],
) -> tuple[ModelAnyResponse, Cost]

Make a request to the model.

request_stream async

request_stream(
    messages: list[Message],
) -> AsyncIterator[EitherStreamedResponse]

Make a request to the model and return a streaming response.

AbstractToolDefinition

Bases: Protocol

Abstract definition of a function/tool.

This is used for both retrievers and result tools.

name instance-attribute

name: str

The name of the tool.

description instance-attribute

description: str

The description of the tool.

json_schema instance-attribute

json_schema: ObjectJsonSchema

The JSON schema for the tool's arguments.

outer_typed_dict_key instance-attribute

outer_typed_dict_key: str | None

The key in the outer [TypedDict] that wraps a result tool.

This will only be set for result tools which don't have an object JSON schema.

StreamTextResponse

Bases: ABC

Streamed response from an LLM when returning text.

__aiter__

__aiter__() -> AsyncIterator[None]

Stream the response as an async iterable, building up the text as it goes.

This is an async iterator that yields None to avoid doing the work of validating the input and extracting the text field when it will often be thrown away.

__anext__ abstractmethod async

__anext__() -> None

Process the next chunk of the response, see above for why this returns None.

get abstractmethod

get(*, final: bool = False) -> Iterable[str]

Returns an iterable of text since the last call to get() — e.g. the text delta.

Parameters:

Name	Type	Description	Default
final	bool	If True, this is the final call, after iteration is complete, the response should be fully validated and all text extracted.	False

cost abstractmethod

cost() -> Cost

Return the cost of the request.

NOTE: this won't return the ful cost until the stream is finished.

timestamp abstractmethod

timestamp() -> datetime

Get the timestamp of the response.

StreamStructuredResponse

Bases: ABC

Streamed response from an LLM when calling a tool.

__aiter__

__aiter__() -> AsyncIterator[None]

Stream the response as an async iterable, building up the tool call as it goes.

This is an async iterator that yields None to avoid doing the work of building the final tool call when it will often be thrown away.

__anext__ abstractmethod async

__anext__() -> None

Process the next chunk of the response, see above for why this returns None.

get abstractmethod

get(*, final: bool = False) -> ModelStructuredResponse

Get the ModelStructuredResponse at this point.

The ModelStructuredResponse may or may not be complete, depending on whether the stream is finished.

Parameters:

Name	Type	Description	Default
final	bool	If True, this is the final call, after iteration is complete, the response should be fully validated.	False

cost abstractmethod

cost() -> Cost

Get the cost of the request.

NOTE: this won't return the full cost until the stream is finished.

timestamp abstractmethod

timestamp() -> datetime

Get the timestamp of the response.

ALLOW_MODEL_REQUESTS module-attribute

ALLOW_MODEL_REQUESTS = True

Whether to allow requests to models.

This global setting allows you to disable request to most models, e.g. to make sure you don't accidentally make costly requests to a model during tests.

The testing models TestModel and FunctionModel are no affected by this setting.

check_allow_model_requests

check_allow_model_requests() -> None

Check if model requests are allowed.

If you're defining your own models that have cost or latency associated with their use, you should call this in Model.agent_model.

Raises:

Type	Description
RuntimeError	If model requests are not allowed.

override_allow_model_requests

override_allow_model_requests(
    allow_model_requests: bool,
) -> Iterator[None]

Context manager to temporarily override ALLOW_MODEL_REQUESTS.

Parameters:

Name	Type	Description	Default
allow_model_requests	bool	Whether to allow model requests within the context.	required