Skip to content

feat: Support Developer Edition connections - #1108

Merged
hessjcg merged 9 commits into
mainfrom
sql-data-service-public
Jun 11, 2026
Merged

hessjcg merged 9 commits into
mainfrom
sql-data-service-public

Conversation

@hessjcg

@hessjcg hessjcg commented Jun 3, 2026 •

Copy link
Copy Markdown
Member

This PR adds support for Developer Edition connections via the SqlDataService. It includes a fallback mechanism to standard IP connections if the SqlDataService is not supported by the instance edition.

@hessjcg hessjcg changed the title feat: Support AI Developer Edition connections feat: Support Developer Edition connections Jun 3, 2026
Comment thread fallback_conn.go Fixed
@hessjcg
hessjcg force-pushed the sql-data-service-public branch 2 times, most recently from 86a2d04 to 744403c Compare June 3, 2026 19:54
hessjcg added a commit to GoogleCloudPlatform/cloud-sql-jdbc-socket-factory that referenced this pull request Jun 4, 2026
This PR adds support for Developer Edition connections via the SqlDataService.
It includes a fallback mechanism to standard IP connections if the
SqlDataService is not supported by the instance edition.

See GoogleCloudPlatform/cloud-sql-go-connector#1108
hessjcg added a commit to GoogleCloudPlatform/cloud-sql-python-connector that referenced this pull request Jun 4, 2026
This PR adds support for Developer Edition connections via the SqlDataService. 
It includes a fallback mechanism to standard IP connections if the SqlDataService
is not supported by the instance edition.

See GoogleCloudPlatform/cloud-sql-go-connector#1108
hessjcg added a commit to GoogleCloudPlatform/cloud-sql-python-connector that referenced this pull request Jun 4, 2026
This PR adds support for Developer Edition connections via the SqlDataService. 
It includes a fallback mechanism to standard IP connections if the SqlDataService
is not supported by the instance edition.

See GoogleCloudPlatform/cloud-sql-go-connector#1108
hessjcg added a commit to GoogleCloudPlatform/cloud-sql-python-connector that referenced this pull request Jun 4, 2026
This PR adds support for Developer Edition connections via the SqlDataService. 
It includes a fallback mechanism to standard IP connections if the SqlDataService
is not supported by the instance edition.

See GoogleCloudPlatform/cloud-sql-go-connector#1108
hessjcg added a commit to GoogleCloudPlatform/cloud-sql-python-connector that referenced this pull request Jun 4, 2026
This PR adds support for Developer Edition connections via the SqlDataService. 
It includes a fallback mechanism to standard IP connections if the SqlDataService
is not supported by the instance edition.

See GoogleCloudPlatform/cloud-sql-go-connector#1108
@hessjcg
hessjcg marked this pull request as ready for review June 4, 2026 20:44
@hessjcg
hessjcg requested a review from a team as a code owner June 4, 2026 20:44
@hessjcg
hessjcg force-pushed the sql-data-service-public branch from 744403c to b13e992 Compare June 9, 2026 17:28
Comment thread internal/sqldata/sql_data_service.proto Outdated

@DeepeshDamodaran DeepeshDamodaran left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

had a comment about the api.

Comment thread fallback_conn.go Outdated

@panavenue panavenue left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is 1 failing unit test. Probably format too strict?

Comment thread internal/cloudsql/lazy_test.go
Comment thread internal/cloudsql/metadataexchange_test.go
Comment thread internal/cloudsql/refresh.go Outdated
Comment thread fallback_conn.go Outdated
@hessjcg
hessjcg force-pushed the sql-data-service-public branch from db4c8de to 2c41b0b Compare June 10, 2026 18:44
hessjcg added 7 commits June 10, 2026 18:52
…Auth Proxy

Cloud SQL AI Developer Edition instances allow connections through the SqlDataService.
With this change and the corresponding chagnes in the Cloud SQL Go Connector, the Auth
Proxy will allow users to connect to AIDE instances. Set the --sql-data flag to enable
this support.
ww
Review Comment from @panavenue:
"Hmm, looks like this local var is not being use anymore below, do we still need to set it to true?"

This change makes fallbackConn thread-safe:
- Coordinates the first read using sync.Once to ensure only a single goroutine executes the first read and triggers connectFallback.
- Fixes the unused local variable assignment by updating the struct's firstReadDone flag inside a mutex lock.
- Synchronizes concurrent Write calls and caching of written bytes under a write lock when firstReadDone is false, avoiding data races and silent data loss during fallback connection swap.
- Replays cached writes under the write lock to prevent concurrent modification of writebuf.
…ublic proto

Review Comment from @DeepeshDamodaran:
"we don't want to show this deprecated field in the first version of the public api. this may be more work on backend side, but i feel we should do that."

This change cleans up all deprecated fields and messages from the proto:
- Removes deprecated fields from StreamSqlDataRequest (location_id, message_type) and StreamSqlDataResponse (type).
- Deletes unused deprecated messages ConnectionSettings, InstanceId, ClientPayload, and ServerPayload.
- Regenerates Go protobuf bindings.
- Fixes build.sh in jj-go-connector to support cross-platform architecture variables and sed in-place edit syntax during local protoc setup.
…est.go

Review Comment from @hessjcg:
"Rollback. This is not a relevant change."

Rolls back TokenProvider casing changes in lazy_test.go from tokenProvider back to TokenProvider.
…hange_test.go

Review Comment from @hessjcg:
"Rollback. This is not a relevant change."

Rolls back the change from &mdx.MetadataExchangeRequest struct initialization to mdx.MetadataExchangeRequest_builder in metadataexchange_test.go.
Review Comment from @hessjcg:
"TokenProvider"

Changes tokenProvider to TokenProvider in the struct field comment of adminAPIClient.
@hessjcg
hessjcg force-pushed the sql-data-service-public branch from 2c41b0b to 5c3cd61 Compare June 10, 2026 19:05
Refines fallbackConn thread safety to fully comply with net.Conn's concurrency contract:
- Introduces a closed flag to track connection state.
- Ensures Read and Write return net.ErrClosed if Close is called.
- Closes the new secondary connection if established concurrently while the wrapper is being closed, preventing resource leaks.
- Adds TestFallbackConn_CloseDuringFallback to test and verify this exact scenario under simulation.
@hessjcg
hessjcg force-pushed the sql-data-service-public branch 2 times, most recently from c365eaa to 1252587 Compare June 10, 2026 22:06
…enerated code from CI coverage

Refactors fallbackConn and fixes CI unit test failures:
- Replaced boolean state flags with a 3-state enum: stateBeforeFirstRead, stateDuringFirstRead, and stateAfterFirstRead.
- Introduced writeMu to serialize Write() calls.
- Simplified Write() to write and cache during stateBeforeFirstRead and stateDuringFirstRead without blocking on sync.Cond.
- Reader fallback now locks writeMu during buffer replay and swap. This guarantees thread safety and resolves the concurrent read/write deadlock in TestDialerWithSqlData_LargePayload.
- Removed block-waiting from Close() to abort blocked reads and return immediately.
- Updated metadataexchange_test.go to use MetadataExchangeRequest_builder to comply with the generated opaque protobuf API.
- Updated .github/workflows/tests.yaml to generate coverprofile, filter out generated protobufs (.pb.go) and mocks (/mock/) via grep -v, and output the filtered coverage via go tool cover.
@hessjcg
hessjcg force-pushed the sql-data-service-public branch from 1252587 to 265a931 Compare June 10, 2026 23:03
@hessjcg
hessjcg merged commit 116b2e5 into main Jun 11, 2026
20 checks passed
@hessjcg
hessjcg deleted the sql-data-service-public branch June 11, 2026 04:01
hessjcg added a commit to GoogleCloudPlatform/cloud-sql-python-connector that referenced this pull request Jun 25, 2026
This PR adds support for Developer Edition connections via the SqlDataService. 
It includes a fallback mechanism to standard IP connections if the SqlDataService
is not supported by the instance edition.

See GoogleCloudPlatform/cloud-sql-go-connector#1108
panavenue added a commit that referenced this pull request Jul 7, 2026
* docs: Update documentation to describe how to use Advanced DR write endpoint. (#1042)

* docs: Add Cloud Run and Go connector samples  (#1045)

* chore(deps): bump golang.org/x/crypto from 0.43.0 to 0.45.0 (#1044)

Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.43.0 to 0.45.0.
- [Commits](golang/crypto@v0.43.0...v0.45.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-version: 0.45.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* deps: Update dependencies to latest. (#1048)

* fix: Update dependency versions. (#1049)

* chore(main): release 1.19.1 (#1050)

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

* feat: Use configured DNS name to lookup instance IP address (#1054)

When a custom DNS name is used to connect to a Cloud SQL instance, the dialer should first attempt to resolve the custom DNS name to an IP address and use that for the connection. If the lookup fails, the dialer should fall back to using the IP address from the instance metadata.

This change modifies the dialer to:
- Use the configured resolver to look up the host's IP address.
- Use the IP address from the A record of the custom DNS name if available.
- Fall back to the IP address from the instance metadata if the A record is not available.

Fixes #1053

* deps: Update project dependencies to latest (#1060)

* chore(main): release 1.20.0 (#1055)

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

* chore: Update dependencies to latest as of 2026-02-13 (#1065)

* chore(deps): bump golang.org/x/crypto in /examples/cloudrun/mysql (#1047)

Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.43.0 to 0.45.0.
- [Commits](golang/crypto@v0.43.0...v0.45.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-version: 0.45.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fix: bump golang.org/x/crypto in /examples/cloudrun/sqlserver (#1046)

Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.43.0 to 0.45.0.
- [Commits](golang/crypto@v0.43.0...v0.45.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-version: 0.45.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump golang.org/x/crypto in /examples/cloudrun/postgres (#1061)

Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.43.0 to 0.45.0.
- [Commits](golang/crypto@v0.43.0...v0.45.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-version: 0.45.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(main): release 1.20.1 (#1066)

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

* chore(deps): Update dependencies and Go version (use 1.25.7 in unit tests) (#1071)

* chore: Refactor MDX protobuf generation to use the newer opaque api. (#1072)

This updates protobuf code generator to generate the newer style opaque api. This is safer to use and more in line
with other generated Google API stubs. This does not change the API structure in any way.

* fix: update dependencies to latest (#1075)

* chore(main): release 1.20.2 (#1076)

* chore(main): release 1.20.2

* Update CHANGELOG.md

Add note about the deprecation of PGX v4 support.

---------

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
Co-authored-by: Jonathan Hess (he/him) <103529393+hessjcg@users.noreply.github.com>

* chore: update blunderbuss (#1083)

* chore: deprecate pgv4 (#1085)

* chore: deprecate pgv4

* revert back the version

* feat: upgrade example go version to match minimum (#1087)

* chore(deps): bump google.golang.org/grpc in /examples/cloudrun/sqlserver (#1086)

Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.76.0 to 1.79.3.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](grpc/grpc-go@v1.76.0...v1.79.3)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-version: 1.79.3
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump google.golang.org/grpc from 1.79.2 to 1.79.3 (#1078)

Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.79.2 to 1.79.3.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](grpc/grpc-go@v1.79.2...v1.79.3)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-version: 1.79.3
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump google.golang.org/grpc in /examples/cloudrun/mysql (#1077)

Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.76.0 to 1.79.3.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](grpc/grpc-go@v1.76.0...v1.79.3)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-version: 1.79.3
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* deps: Update Non-major dependency updates (#1074)

* deps: Update google.golang.org/genproto/googleapis/rpc digest to afd174a (#1073)

* chore(main): release 1.21.0 (#1088)

* chore(main): release 1.21.0

* add bug fixes for pgxv4 deprecation

---------

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
Co-authored-by: Jin Hou <jinphou@google.com>

* chore: Update dependencies to latest as of 2026-05-04 (#1101)

* chore(deps): bump go.opentelemetry.io/otel (#1097)

Bumps [go.opentelemetry.io/otel](https://github.com/open-telemetry/opentelemetry-go) from 1.39.0 to 1.41.0.
- [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md)
- [Commits](open-telemetry/opentelemetry-go@v1.39.0...v1.41.0)

---
updated-dependencies:
- dependency-name: go.opentelemetry.io/otel
  dependency-version: 1.41.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump go.opentelemetry.io/otel (#1096)

Bumps [go.opentelemetry.io/otel](https://github.com/open-telemetry/opentelemetry-go) from 1.38.0 to 1.41.0.
- [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md)
- [Commits](open-telemetry/opentelemetry-go@v1.38.0...v1.41.0)

---
updated-dependencies:
- dependency-name: go.opentelemetry.io/otel
  dependency-version: 1.41.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump google.golang.org/grpc in /examples/cloudrun/postgres (#1095)

Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.76.0 to 1.79.3.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](grpc/grpc-go@v1.76.0...v1.79.3)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-version: 1.79.3
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump github.com/jackc/pgx/v5 in /examples/cloudrun/postgres (#1092)

Bumps [github.com/jackc/pgx/v5](https://github.com/jackc/pgx) from 5.7.6 to 5.9.2.
- [Changelog](https://github.com/jackc/pgx/blob/master/CHANGELOG.md)
- [Commits](jackc/pgx@v5.7.6...v5.9.2)

---
updated-dependencies:
- dependency-name: github.com/jackc/pgx/v5
  dependency-version: 5.9.2
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* deps: Update google.golang.org/genproto/googleapis/rpc digest to aa98bba (#1091)

* fix: bump dependencies to latest (#1102)

* deps: Update Non-major dependency updates (#1100)

* chore(main): release 1.21.1 (#1103)

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

* deps: Update Non-major dependency updates (#1104)

* deps: update dependencies to latest (#1109)

* fix: bump dependencies to latest (#1111)

* chore(main): release 1.21.2 (#1112)

* feat: Support AI Developer Edition connections through the Cloud SQL Auth Proxy (#1108)

Cloud SQL AI Developer Edition instances allow connections through the
SqlDataService. With this change and the corresponding changes in
the Auth Proxy, this will will allow users to connect to AIDE instances.

* chore(main): release 1.22.0 (#1113)

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

* deps: Update dependencies for github (#1038)

* deps: Update dependencies to latest as of June 25, 2026. (#1116)

* fix: update dependencies to latest (#1117)

* chore(main): release 1.22.1 (#1118)

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

* System metrics setup (#1041)

* feat: Add initial set of system metircs

* Add new metrics

* Add tel_test

* fix linting

* update to add cloud.google.com/go/monitoring/apiv3/v2, following alloydb approach in GoogleCloudPlatform/alloydb-go-connector#728

* fix from comment

* fixes from comment

* update DefaultExportInterval back to var

* Add disableBuiltInMetrics to dialer config

* Add WithDisableBuiltInMetrics

* Enhance WithDisableBuiltInMetrics description

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Jonathan Hess (he/him) <103529393+hessjcg@users.noreply.github.com>
Co-authored-by: kgala2 <galakp@google.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
Co-authored-by: Mend Renovate <bot@renovateapp.com>
hessjcg added a commit to GoogleCloudPlatform/cloud-sql-python-connector that referenced this pull request Aug 12, 2026
This PR adds support for Developer Edition connections via the SqlDataService. 
It includes a fallback mechanism to standard IP connections if the SqlDataService
is not supported by the instance edition.

See GoogleCloudPlatform/cloud-sql-go-connector#1108
hessjcg added a commit to GoogleCloudPlatform/cloud-sql-python-connector that referenced this pull request Aug 13, 2026
This PR adds support for Developer Edition connections via the SqlDataService. 
It includes a fallback mechanism to standard IP connections if the SqlDataService
is not supported by the instance edition.

See GoogleCloudPlatform/cloud-sql-go-connector#1108
hessjcg added a commit to GoogleCloudPlatform/cloud-sql-python-connector that referenced this pull request Aug 15, 2026
This PR adds support for Developer Edition connections via the SqlDataService. 
It includes a fallback mechanism to standard IP connections if the SqlDataService
is not supported by the instance edition.

See GoogleCloudPlatform/cloud-sql-go-connector#1108
hessjcg added a commit to GoogleCloudPlatform/cloud-sql-jdbc-socket-factory that referenced this pull request Aug 24, 2026
Developer Edition connections allow users to connect to Cloud SQL Developer Edition instances. Cloud SQL Developer Edition instances are lightweight database instances that do not expose public or private IPv4 network addresses.

Instead of establishing direct TCP/TLS connections to instance IP addresses, connectivity is established through the Cloud SQL Data Service (SqlDataService.StreamSqlData) API. Native database client protocol traffic is encapsulated and tunneled through gRPC stream data packets. This allows developers to securely connect to Cloud SQL databases from restricted network environments using standard IAM authentication and Google Cloud credentials.

See GoogleCloudPlatform/cloud-sql-go-connector#1108
hessjcg added a commit to GoogleCloudPlatform/cloud-sql-python-connector that referenced this pull request Sep 15, 2026
This PR adds support for Developer Edition connections via the SqlDataService. 
It includes a fallback mechanism to standard IP connections if the SqlDataService
is not supported by the instance edition.

See GoogleCloudPlatform/cloud-sql-go-connector#1108
hessjcg added a commit to GoogleCloudPlatform/cloud-sql-python-connector that referenced this pull request Oct 7, 2026
This PR adds support for Developer Edition connections via the SqlDataService. 
It includes a fallback mechanism to standard IP connections if the SqlDataService
is not supported by the instance edition.

See GoogleCloudPlatform/cloud-sql-go-connector#1108
hessjcg added a commit to GoogleCloudPlatform/cloud-sql-python-connector that referenced this pull request Oct 12, 2026
This PR adds support for Developer Edition connections via the SqlDataService. 
It includes a fallback mechanism to standard IP connections if the SqlDataService
is not supported by the instance edition.

See GoogleCloudPlatform/cloud-sql-go-connector#1108
kgala2 pushed a commit to GoogleCloudPlatform/cloud-sql-python-connector that referenced this pull request Oct 13, 2026
This PR adds support for Developer Edition connections via the SqlDataService. 
It includes a fallback mechanism to standard IP connections if the SqlDataService
is not supported by the instance edition.

See GoogleCloudPlatform/cloud-sql-go-connector#1108
kgala2 pushed a commit to GoogleCloudPlatform/cloud-sql-python-connector that referenced this pull request Oct 15, 2026
This PR adds support for Developer Edition connections via the SqlDataService. 
It includes a fallback mechanism to standard IP connections if the SqlDataService
is not supported by the instance edition.

See GoogleCloudPlatform/cloud-sql-go-connector#1108
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants