Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Gemfile
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ end

group :development, :test do
gem "async"
gem "aws-sdk-core", "~> 3"
gem "minitest"
gem "minitest-focus"
gem "minitest-hooks"
Expand Down
14 changes: 14 additions & 0 deletions Gemfile.lock
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,18 @@ GEM
io-event (~> 1.11)
metrics (~> 0.12)
traces (~> 0.18)
aws-eventstream (1.4.0)
aws-partitions (1.1261.0)
aws-sdk-core (3.252.0)
aws-eventstream (~> 1, >= 1.3.0)
aws-partitions (~> 1, >= 1.992.0)
aws-sigv4 (~> 1.9)
base64
bigdecimal
jmespath (~> 1, >= 1.6.1)
logger
aws-sigv4 (1.12.1)
aws-eventstream (~> 1, >= 1.0.2)
base64 (0.3.0)
benchmark (0.5.0)
bigdecimal (3.3.1)
Expand Down Expand Up @@ -72,6 +84,7 @@ GEM
i18n (1.14.7)
concurrent-ruby (~> 1.0)
io-event (1.11.2)
jmespath (1.6.2)
json (2.15.2)
language_server-protocol (3.17.0.5)
lint_roller (1.1.0)
Expand Down Expand Up @@ -206,6 +219,7 @@ PLATFORMS

DEPENDENCIES
async
aws-sdk-core (~> 3)
minitest
minitest-focus
minitest-hooks
Expand Down
50 changes: 50 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,56 @@ puts(edited.data.first)

Note that you can also pass a raw `IO` descriptor, but this disables retries, as the library can't be sure if the descriptor is a file or pipe (which cannot be rewound).

## Amazon Bedrock

Use the standard client with the Bedrock provider to call OpenAI models through Amazon Bedrock's OpenAI-compatible API. Add `aws-sdk-core` to your application for AWS credential discovery and SigV4 signing:

```ruby
gem "openai"
gem "aws-sdk-core", "~> 3"
```

With your normal AWS credentials configured, only the region is required:

```ruby
require "openai"

client = OpenAI::Client.new(
provider: OpenAI::Providers.bedrock(region: "us-west-2")
)

response = client.responses.create(
model: ENV.fetch("BEDROCK_MODEL"),
input: "Say hello!"
)

puts(response.output_text)
```

The provider uses the standard AWS credential chain, including environment credentials, `~/.aws/credentials`, `~/.aws/config`, `AWS_PROFILE`, named profiles, SSO and assume-role profiles, and workload credentials. Select a profile explicitly with:

```ruby
client = OpenAI::Client.new(
provider: OpenAI::Providers.bedrock(
region: "us-west-2",
profile: "engineering"
)
)
```

`AWS_BEARER_TOKEN_BEDROCK` and explicit Bedrock bearer credentials are also supported. Bearer authentication does not load or require `aws-sdk-core`:

```ruby
client = OpenAI::Client.new(
provider: OpenAI::Providers.bedrock(
region: "us-west-2",
api_key: ENV.fetch("AWS_BEARER_TOKEN_BEDROCK")
)
)
```

See [bedrock.md](bedrock.md) for authentication precedence, static and refreshable credentials, endpoint configuration, and SigV4 request constraints.

## Workload Identity Authentication

For secure, automated environments like cloud-managed Kubernetes, Azure, and GCP, you can use workload identity authentication with short-lived tokens from cloud identity providers instead of long-lived API keys.
Expand Down
132 changes: 132 additions & 0 deletions bedrock.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
# Amazon Bedrock

Configure the standard `OpenAI::Client` with the Bedrock provider to use [Amazon Bedrock's OpenAI-compatible API](https://docs.aws.amazon.com/bedrock/latest/userguide/models-api-compatibility.html):

```ruby
require "openai"

client = OpenAI::Client.new(
provider: OpenAI::Providers.bedrock(region: "us-west-2")
)

response = client.responses.create(
model: ENV.fetch("BEDROCK_MODEL"),
input: "Say hello!"
)

puts(response.output_text)
```

The provider uses `https://bedrock-mantle.<region>.api.aws/v1` by default and exposes the normal SDK resources. AWS controls which endpoints and features are supported; unsupported calls surface as normal API errors.

The region is resolved from the explicit `region` option, `AWS_REGION`, `AWS_DEFAULT_REGION`, or the selected AWS profile. Override the endpoint with `base_url` or `AWS_BEDROCK_BASE_URL`:

```ruby
provider = OpenAI::Providers.bedrock(
region: "us-west-2",
base_url: "https://bedrock.example.com/v1"
)
```

## Authentication

The provider selects authentication in this order:

1. One explicit mode passed to `bedrock(...)`: `api_key`, `token_provider`, static AWS credentials, `profile`, or `credentials_provider`.
2. The Bedrock bearer credential in `AWS_BEARER_TOKEN_BEDROCK`.
3. The standard AWS credential chain.

Explicit bearer and AWS credential modes are mutually exclusive. Configure only one explicit AWS mode at a time.

### AWS credentials and SigV4

AWS authentication is an optional integration so applications that do not use Bedrock do not install or load AWS packages. Add the AWS SDK core gem to your application:

```ruby
gem "aws-sdk-core", "~> 3"
```

Then run `bundle install`. If you are not using Bundler, run `gem install aws-sdk-core`.

Omit explicit authentication to use environment credentials, the shared credentials and config files, SSO or assume-role profiles, web identity, ECS credentials, or EC2 instance credentials:

```ruby
client = OpenAI::Client.new(
provider: OpenAI::Providers.bedrock(region: "us-west-2")
)
```

Select a named profile with:

```ruby
client = OpenAI::Client.new(
provider: OpenAI::Providers.bedrock(
region: "us-west-2",
profile: "engineering"
)
)
```

You can omit `region` when that profile defines one in `~/.aws/config`.

Pass temporary credentials directly, including the session token:

```ruby
client = OpenAI::Client.new(
provider: OpenAI::Providers.bedrock(
region: "us-west-2",
access_key_id: ENV.fetch("AWS_ACCESS_KEY_ID"),
secret_access_key: ENV.fetch("AWS_SECRET_ACCESS_KEY"),
session_token: ENV["AWS_SESSION_TOKEN"]
)
)
```

For credentials managed by your application, pass an AWS credential provider or a callable that returns `Aws::Credentials`. It is consulted before every request attempt, including retries:

```ruby
credentials_provider = lambda do
Aws::Credentials.new(
ENV.fetch("AWS_ACCESS_KEY_ID"),
ENV.fetch("AWS_SECRET_ACCESS_KEY"),
ENV["AWS_SESSION_TOKEN"]
)
end

client = OpenAI::Client.new(
provider: OpenAI::Providers.bedrock(
region: "us-west-2",
credentials_provider: credentials_provider
)
)
```

The provider signs each finalized attempt with AWS SigV4 service name `bedrock-mantle`. Standard JSON API requests have replayable bodies and work normally. SigV4 rejects one-shot request streams before sending, and signed requests do not automatically follow redirects because a new target requires a new signature. Response streaming is unaffected.

### Bearer authentication

Pass a Bedrock bearer credential directly, set `AWS_BEARER_TOKEN_BEDROCK`, or use a callable to resolve a fresh token before every attempt:

```ruby
client = OpenAI::Client.new(
provider: OpenAI::Providers.bedrock(
region: "us-west-2",
api_key: ENV.fetch("BEDROCK_API_KEY")
)
)
```

```ruby
client = OpenAI::Client.new(
provider: OpenAI::Providers.bedrock(
region: "us-west-2",
token_provider: -> { refresh_bedrock_token }
)
)
```

Bearer authentication does not load or require `aws-sdk-core`. Passing `api_key: nil` explicitly skips `AWS_BEARER_TOKEN_BEDROCK` and selects AWS authentication.

## Security

Use Bedrock from a server-side Ruby runtime. Prefer roles, profiles, SSO, and temporary credentials over long-lived static keys. Do not send AWS access keys, session tokens, bearer credentials, or signed authorization headers to browsers or application logs.
17 changes: 17 additions & 0 deletions examples/bedrock.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
#!/usr/bin/env ruby
# frozen_string_literal: true

require_relative "../lib/openai"

client = OpenAI::Client.new(
provider: OpenAI::Providers.bedrock(
region: ENV["AWS_REGION"] || ENV["AWS_DEFAULT_REGION"]
)
)

response = client.responses.create(
model: ENV.fetch("BEDROCK_MODEL"),
input: "Say hello from Amazon Bedrock!"
)

puts(response.output_text)
3 changes: 3 additions & 0 deletions lib/openai.rb
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,9 @@
require_relative "openai/request_options"
require_relative "openai/file_part"
require_relative "openai/errors"
require_relative "openai/provider"
require_relative "openai/internal/provider"
require_relative "openai/providers/bedrock"
require_relative "openai/internal/transport/base_client"
require_relative "openai/internal/transport/pooled_net_requester"
require_relative "openai/client"
Expand Down
70 changes: 62 additions & 8 deletions lib/openai/client.rb
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,8 @@ class Client < OpenAI::Internal::Transport::BaseClient
#
# @return [Hash{String=>String}]
private def auth_headers(security:)
return {} if @provider_runtime

enabled_security = security.select { |_, enabled| enabled }
headers = {bearer_auth:, admin_api_key_auth:}.slice(*enabled_security.keys).values.reduce({}, :merge)
if headers.empty? && enabled_security.any?
Expand Down Expand Up @@ -166,6 +168,13 @@ class Client < OpenAI::Internal::Transport::BaseClient
true
end

# @api private
private def prepare_request(request, redirect_count:, retry_count:)
preparer = @provider_runtime&.prepare_request
return super unless preparer
preparer.call(request)
end

# @api private
private def send_request(request, redirect_count:, retry_count:, send_retry_header:)
return super unless @workload_identity_auth
Expand Down Expand Up @@ -218,6 +227,10 @@ class Client < OpenAI::Internal::Transport::BaseClient
#
# @param webhook_secret [String, nil] Defaults to `ENV["OPENAI_WEBHOOK_SECRET"]`
#
# @param provider [OpenAI::Provider, nil] Configure a supported
# third-party provider. Provider authentication and routing cannot be combined
# with top-level `api_key`, `admin_api_key`, `workload_identity`, or `base_url`.
#
# @param base_url [String, nil] Override the default base URL for the API, e.g.,
# `"https://api.example.com/v2/"`. Defaults to `ENV["OPENAI_BASE_URL"]`
#
Expand All @@ -229,25 +242,65 @@ class Client < OpenAI::Internal::Transport::BaseClient
#
# @param max_retry_delay [Float]
def initialize(
api_key: ENV["OPENAI_API_KEY"],
admin_api_key: ENV["OPENAI_ADMIN_KEY"],
api_key: OpenAI::Internal::OMIT,
admin_api_key: OpenAI::Internal::OMIT,
workload_identity: nil,
organization: ENV["OPENAI_ORG_ID"],
project: ENV["OPENAI_PROJECT_ID"],
webhook_secret: ENV["OPENAI_WEBHOOK_SECRET"],
base_url: ENV["OPENAI_BASE_URL"],
organization: OpenAI::Internal::OMIT,
project: OpenAI::Internal::OMIT,
webhook_secret: OpenAI::Internal::OMIT,
provider: nil,
base_url: OpenAI::Internal::OMIT,
max_retries: self.class::DEFAULT_MAX_RETRIES,
timeout: self.class::DEFAULT_TIMEOUT_IN_SECONDS,
initial_retry_delay: self.class::DEFAULT_INITIAL_RETRY_DELAY,
max_retry_delay: self.class::DEFAULT_MAX_RETRY_DELAY
)
provider_runtime = nil
unless provider.nil?
provider_name = OpenAI::Internal::Provider.name(provider)
conflicts = {
api_key: api_key,
admin_api_key: admin_api_key,
workload_identity: workload_identity,
base_url: base_url
}.filter_map do |name, value|
name unless value.equal?(OpenAI::Internal::OMIT) || value.nil?
end
unless conflicts.empty?
formatted = conflicts.map { "`#{_1}`" }.join(", ")
message =
"`provider` cannot be combined with top-level #{formatted}. Move provider " \
"authentication and routing options into `#{provider_name}(...)`."
raise ArgumentError, message
end
provider_runtime = OpenAI::Internal::Provider.configure(provider)
end

api_key = ENV["OPENAI_API_KEY"] if api_key.equal?(OpenAI::Internal::OMIT) && provider_runtime.nil?
if admin_api_key.equal?(OpenAI::Internal::OMIT) && provider_runtime.nil?
admin_api_key = ENV["OPENAI_ADMIN_KEY"]
end
if organization.equal?(OpenAI::Internal::OMIT) && provider_runtime.nil?
organization = ENV["OPENAI_ORG_ID"]
end
project = ENV["OPENAI_PROJECT_ID"] if project.equal?(OpenAI::Internal::OMIT) && provider_runtime.nil?
webhook_secret = ENV["OPENAI_WEBHOOK_SECRET"] if webhook_secret.equal?(OpenAI::Internal::OMIT)
base_url = ENV["OPENAI_BASE_URL"] if base_url.equal?(OpenAI::Internal::OMIT) && provider_runtime.nil?

api_key = nil if api_key.equal?(OpenAI::Internal::OMIT)
admin_api_key = nil if admin_api_key.equal?(OpenAI::Internal::OMIT)
organization = nil if organization.equal?(OpenAI::Internal::OMIT)
project = nil if project.equal?(OpenAI::Internal::OMIT)
webhook_secret = nil if webhook_secret.equal?(OpenAI::Internal::OMIT)
base_url = provider_runtime.base_url if provider_runtime
base_url = nil if base_url.equal?(OpenAI::Internal::OMIT)
base_url ||= "https://api.openai.com/v1"

if !api_key.nil? && !workload_identity.nil?
raise ArgumentError, "`api_key` and `workload_identity` are mutually exclusive"
end

if api_key.nil? && admin_api_key.nil? && workload_identity.nil?
if provider_runtime.nil? && api_key.nil? && admin_api_key.nil? && workload_identity.nil?
raise ArgumentError,
"Missing credentials. Please pass an `api_key`, `workload_identity`, `admin_api_key`, or set the `OPENAI_API_KEY` or `OPENAI_ADMIN_KEY` environment variable."
end
Expand All @@ -256,7 +309,7 @@ def initialize(
"openai-organization" => (@organization = organization&.to_s),
"openai-project" => (@project = project&.to_s)
}
custom_headers_env = ENV["OPENAI_CUSTOM_HEADERS"]
custom_headers_env = ENV["OPENAI_CUSTOM_HEADERS"] unless provider_runtime
unless custom_headers_env.nil?
parsed = {}
custom_headers_env.split("\n").each do |line|
Expand All @@ -280,6 +333,7 @@ def initialize(
end
@admin_api_key = admin_api_key&.to_s
@webhook_secret = webhook_secret&.to_s
@provider_runtime = provider_runtime

super(
base_url: base_url,
Expand Down
Loading