> ## Documentation Index
> Fetch the complete documentation index at: https://docs.provisionr.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Credential Security Architecture

> How Provisionr authenticates to Okta and Google Workspace with keys that stay in Cloud KMS or with Google, including the key hierarchy, token flows, and IAM boundaries.

When you [connected Okta](https://docs.provisionr.io/integrations/okta/connect.md) or [Google Workspace](https://docs.provisionr.io/integrations/google/connect.md), the only things that you handled were identifiers, which were a public key for Okta and a Client ID for Google. This page explains what happens behind those steps, for the infrastructure and security engineers who need to review it. It describes which keys exist, where each one lives, what is allowed to use it, and what an attacker would and wouldn't gain at each layer.

It covers these topics, in this order:

* The goals behind the design, and the identity that each workspace runs as
* Each key in the hierarchy and the job that it does
* How Provisionr gets an access token from Okta and from Google, step by step
* How access tokens are handled once they've been issued
* What each kind of compromise would expose, and what happens when a workspace is decommissioned
* The controls, evidence, and limitations that an auditor should know about

## Design Goals

Four goals shaped how we handle integration credentials, and the rest of this page shows how each one is met.

* No long-lived private key that authenticates Provisionr to a vendor is held by a customer, stored in a database, or loaded into application memory.
* Every workspace has its own identity and its own keys, so one workspace can't use another workspace's access.
* You stay in control of what we can do, through settings in your own Okta org or Google Workspace Admin console.
* Every use of a credential leaves a record.

## Workspace Identity

Each Provisionr workspace runs as its own Cloud Run service with its own database and its own Google Cloud service account, which we call the workspace's runtime identity. The web application and any Cloud Run jobs for that workspace run as the same identity, because they all need the same database and secret access.

The runtime identity is the isolation boundary for everything on this page. Its grants are made on individual resources rather than by matching names, so the set of things that a workspace can reach can be listed exactly, and nothing matches by accident.

| Permission | Granted on | Purpose |
| - | - | - |
| Cloud SQL connection and IAM login | The project, with database access limited to the workspace's own database | Reading and writing the workspace's data |
| Secret access | Each secret individually | Reading the workspace's application key and the shared secrets that it needs |
| Encrypt and decrypt | The workspace's credential key | Protecting a credential that has to be stored |
| Create keys, generate versions, read public keys, sign, and destroy versions | The workspace's signing key ring | Managing and using its Okta signing keys |
| Sign a JWT | Its own service account | Signing Google delegation requests |
| Logging, monitoring, tracing, and error reporting | The project | Sending telemetry |

The grant on the signing key ring uses a custom role that holds only the six permissions that the workspace needs. It leaves out the permissions to change a key's access policy or to import key material, so a compromised workspace can't give anyone else lasting access to its keys.

> [!TIP]
>
> Workspace isolation is enforced by Google Cloud IAM on individual resources, rather than by application logic. The grants can be enumerated for each workspace, and the permissions that are held on each workspace's signing key ring are limited to a custom role that excludes policy changes and key import.

## Key Hierarchy

Several different keys are involved, and they do different jobs, so it helps to separate them before looking at how a token is issued.

| Key | Scope | Held in | Used for | Rotation |
| - | - | - | - | - |
| Platform key | One per Google Cloud project | Cloud KMS | Encrypts Cloud SQL, storage buckets, Secret Manager, Artifact Registry, Cloud Run revisions, and the audit log bucket at rest | Automatically every 90 days |
| Application key | One per workspace | Secret Manager | Encrypts sensitive database fields, such as names and email addresses, and cached access tokens | Manually, by replacing it |
| Credential key | One per workspace | Cloud KMS | Encrypts a credential that has to be stored and read back. Neither Okta nor Google uses one today. | Automatically every 90 days |
| Okta signing key | One per Okta integration | Cloud KMS | Signs the client assertions that authenticate Provisionr to Okta | When you run `prv okta-integration:rotate-key` |
| Google service account key | One per workspace | Google | Signs the JWTs for domain-wide delegation | Automatically by Google |
| DPoP key | One per Okta access token | The workspace's cache, encrypted with the application key | Binds an access token to Provisionr | A new key for each access token |

The difference between the credential key and the signing keys is worth understanding, because they solve different problems. Encrypting solves the problem of having to store a secret and get it back later, which applies when a vendor needs the actual secret on every request. Even with the best encryption, the plaintext has to exist in memory while it's being used. Signing solves the problem of proving who you are. A vendor that only needs a signature never needs to see the private key, so the key can stay inside Cloud KMS for its entire life. Okta and Google both support this second approach, which is why neither integration stores a secret today.

## Okta Credentials

Okta only accepts OAuth 2.0 with a signed client assertion for its management API, so Provisionr needs a private key, and that's the piece that Cloud KMS holds. Each Okta integration has one signing key, and each credential on that integration is one version of the key. Another credential on the same integration is therefore a rotation of the same key.

### Okta Key Generation

Generating a key can take a few seconds, so it happens after the API has responded, and the CLI waits for the public key to appear.

**1. Credential Creation**

Your workspace saves a staged credential with an empty public key and returns right away. Creating a credential doesn't accept any key material, so a private key can't be uploaded.

**2. Key Creation**

After the response is sent, your workspace creates the signing key in its own key ring, named after the integration's ID, if the key doesn't exist yet. Keys are RSA 4096 with SHA-256 and PKCS#1 v1.5 padding, which Okta verifies as RS256.

**3. Key Version Generation**

Your workspace asks Cloud KMS for a new version of the key. Cloud KMS generates the key material internally, and your workspace waits until the version is enabled. If generation fails, the failure is recorded and the credential is deactivated, so that the problem is visible and the credential can be created again.

**4. Public Key Storage**

Your workspace reads the public key from Cloud KMS and stores it with the credential as a JSON Web Key, along with the name of the key version. The key ID starts with `provisionr-`, followed by the start of the key's RFC 7638 thumbprint.

**5. Public Key Registration**

You add the public key to your Okta app and give Provisionr the app's Client ID. A rotation keeps the Client ID of the active credential, because it's the same Okta app.

```json Okta Public Key JWK theme={null}
{
  "kty": "RSA",
  "n": "ztw2iIbWGJfojk9L2utsKns5nr_r_Bc0efbphBnRnPVCm_ugT...",
  "e": "AQAB",
  "use": "sig",
  "alg": "RS256",
  "kid": "provisionr-3q2h9vTQx8mZk1pA"
}
```

### Okta Token Requests

The sequence below runs when Provisionr needs a new access token. A sync that can reuse a cached token doesn't sign anything.

![Sequence diagram of the workspace asking Cloud KMS to sign a client assertion and exchanging it with Okta for a DPoP-bound access token](https://mintcdn.com/provisionr/aUeSrPB5Cd0gjNLr/images/diagrams/integrations/credential-security/okta-token-request.svg?fit=max\&auto=format\&n=aUeSrPB5Cd0gjNLr\&q=85\&s=1e040dd968b83d0896d9b0acccfc5447)

How your workspace gets an Okta access token

**1. Client Assertion**

Your workspace builds a JSON Web Token with the app's Client ID as both the issuer and the subject, your Okta org's token endpoint as the audience, a unique ID so that it can't be replayed, and an expiration of 5 minutes.

**2. Signature**

Your workspace computes the SHA-256 hash of the assertion and sends only the hash to Cloud KMS, which signs it with the key version on the credential and returns the signature. The private key never leaves Cloud KMS, and each signature is recorded in your event log.

**3. Token Request**

Your workspace generates a new short-lived DPoP key, and sends the assertion to your Okta org's token endpoint with a DPoP proof signed by that key.

**4. Okta Verification**

Okta finds your registered public key by its key ID, verifies the signature and claims, checks that the requested scope is granted to the app, and returns an access token that's bound to the DPoP key.

**5. Token Cache**

The access token and its DPoP key are encrypted with the workspace's application key and cached for 59 minutes, one minute less than the token's lifetime.

```json Okta Client Assertion theme={null}
{
  "header": {
    "alg": "RS256",
    "typ": "JWT",
    "kid": "provisionr-3q2h9vTQx8mZk1pA"
  },
  "claims": {
    "iss": "0oa1b2c3d4e5f6g7h8i9",
    "sub": "0oa1b2c3d4e5f6g7h8i9",
    "aud": "https://redshirt.okta.com/oauth2/v1/token",
    "iat": 1794268800,
    "exp": 1794269100,
    "jti": "9f8c3a54-2b1e-4c7d-8e6f-1a2b3c4d5e6f"
  }
}
```

### Okta Scope Selection

We follow least privilege for each API call. When Provisionr reads a resource, it asks for the read scope first, and it only falls back to the manage scope if Okta refuses to issue a token for the read scope. An app that was only granted the manage scope keeps working, and the scope that worked is remembered on the credential, so that later syncs don't repeat a request that Okta has already refused. The fallback is recorded as `okta.credential.scope.fallback`. Scopes and the [custom admin role](https://docs.provisionr.io/integrations/okta/oauth2.md#admin-role-permissions) both apply to every request, so the admin role remains the limit on what the app can do.

### Okta Activation and Rotation

A credential can only be activated once its key exists and it has a Client ID. Before a new key replaces the current one, your workspace runs the [connection test](https://docs.provisionr.io/integrations/okta/connect.md#connection-test) with the new key. Activating a credential destroys the previous key, so this test matters, because a public key that was never added to Okta would otherwise stop the sync. If the test fails, the previous key keeps working, and the reason is returned to you.

When a credential is deactivated, or replaced by a rotation, your workspace schedules its key version for destruction, which stops the version from signing right away. Cloud KMS destroys the key material after its scheduled destruction period, which is 30 days by default. If Cloud KMS can't be reached at that moment, the credential is still deactivated, and the failure is recorded as `okta.credential.key.destroy.error`. Deleting an Okta integration deactivates all of its credentials in the same way.

## Google Credentials

Google Workspace uses domain-wide delegation, which works differently from Okta. You authorize a service account's Client ID for a list of scopes, and that service account can then act as a user in your organization, within those scopes. Provisionr uses each workspace's runtime identity as that service account, and the Client ID that you authorize belongs to your workspace alone.

### Workspace Service Account

No service account key is ever created. Google's IAM Credentials service can sign a JWT on behalf of a service account, using a key that Google holds and rotates, and the caller only needs permission to do so. Your workspace's runtime identity holds that permission on itself and on no other service account, so one workspace can't sign as another. Because there's no key file, there's nothing to store, rotate, or leak, and your Client ID doesn't change over time.

### Google Token Requests

As with Okta, this sequence runs only when a new access token is needed.

![Sequence diagram of the workspace asking Google IAM Credentials to sign delegation claims and exchanging the signed JWT for an access token](https://mintcdn.com/provisionr/aUeSrPB5Cd0gjNLr/images/diagrams/integrations/credential-security/google-token-request.svg?fit=max\&auto=format\&n=aUeSrPB5Cd0gjNLr\&q=85\&s=1626eee52d2f67116ac58dd01eb09bcd)

How your workspace gets a Google access token

**1. Delegation Claims**

Your workspace builds a set of claims with its service account as the issuer, the admin that you named on the integration as the subject, the one scope that the API call needs, Google's token endpoint as the audience, and an expiration of one hour.

**2. Signature**

Your workspace authenticates to the IAM Credentials service with a short-lived token from the Cloud Run metadata server and asks it to sign the claims as its own service account. Each signature is recorded in your event log.

**3. Token Exchange**

Your workspace exchanges the signed JWT at Google's token endpoint. Google checks that the Client ID is authorized for domain-wide delegation with that scope in your organization, and returns an access token that acts as the subject.

**4. Token Cache**

The access token is encrypted with the workspace's application key and cached until one minute before it expires. Tokens are cached separately for each scope and subject.

```json Google Delegation Claims theme={null}
{
  "iss": "lp01x7k2q9@provisionr-lp01.iam.gserviceaccount.com",
  "sub": "ian.sanford@redshirt.example",
  "scope": "https://www.googleapis.com/auth/admin.directory.user.readonly",
  "aud": "https://oauth2.googleapis.com/token",
  "iat": 1794268800,
  "exp": 1794272400
}
```

### Delegation Boundaries

The domain-wide delegation entry in your Admin console is the real limit on what Provisionr can do in your organization. A token is requested for one scope at a time, and Google refuses any scope that isn't in the entry, even though the JWT is correctly signed. Removing the Client ID from domain-wide delegation stops Provisionr from getting new tokens, without any action on our side. The [Google OAuth2 Architecture](https://docs.provisionr.io/integrations/google/oauth2.md#oauth2-scopes) page lists what each scope allows.

### Google Activation

Before a Google credential is activated, your workspace uses it to make real API calls. It reads the subject's user record and confirms that the record's customer ID matches the integration, and it then lists groups and shared drives if the integration's scopes include them. If any call fails, the credential stays staged, and you get the reason. Google applies delegation changes on its own schedule, so a new entry can take a little while to work.

## Access Token Handling

Once a token has been issued, Okta and Google differ in one important way, which is that an Okta token is bound to Provisionr, and a Google token isn't.

| | Okta | Google |
| - | - | - |
| Lifetime | 60 minutes | 60 minutes |
| Binding | Bound to a DPoP key, so a stolen token can't be replayed without that key | A bearer token, usable by anyone who has it until it expires |
| Scope of one token | One scope, for the Okta app | One scope, acting as one subject |
| Storage | Encrypted with the application key in the workspace's cache | Encrypted with the application key in the workspace's cache |
| Cached for | 59 minutes | Until one minute before it expires |
| Early revocation | Removing the public key stops new tokens, and existing tokens run until they expire | Removing the delegation entry stops new tokens, and existing tokens run until they expire |

Access tokens are only ever used by the workspace that requested them, and they aren't returned by the API or the CLI.

## Compromise Scenarios

Reviewing a design like this is mostly a matter of asking what an attacker would gain from each kind of access, so the table below answers that directly.

| Scenario | What's exposed | What limits it |
| - | - | - |
| A copy of a workspace's database or backup | Encrypted fields, public keys, and the names of key versions | There's no private key in the database, and encrypted fields can't be read without the application key. |
| A copy of the database and the application key | Personal data in encrypted fields, and any cached access tokens until they expire | Signing keys aren't exposed, and cached tokens last at most 60 minutes. |
| Code running inside a workspace | The ability to sign and call vendor APIs as that workspace, within the granted scopes and admin role | Keys can't be exported, so the access ends when the compromise ends, apart from tokens that were already issued. |
| Access to a different workspace | Nothing from this workspace | Grants are made per workspace, for the database, the secrets, the signing keys, and the service account. |
| A stolen Okta access token | Nothing by itself | The token can't be used without its DPoP key. |
| A stolen Google access token | One scope, acting as one subject, for at most 60 minutes | The token is limited to its scope, and Google's delegation entry applies when a new one is requested. |
| A principal that can deploy to a workspace | The same as code running inside that workspace | Deployment rights are held by a small set of platform identities, and each deployment is recorded. |

In any of these cases, you can cut off Provisionr without waiting for us, by removing the public key from your Okta app or removing the Client ID from domain-wide delegation.

## Workspace Decommissioning

When a workspace is decommissioned, its access is removed along with it. The workspace's service account is deleted, so the Client ID in your Admin console no longer points to an account that can sign anything. Every version of its Okta signing keys is scheduled for destruction, and its credential key is destroyed. Google Cloud doesn't allow a key ring to be deleted, so the empty ring remains, and because workspace identifiers are never reused, its name can't be taken by another workspace.

A copy of the workspace's database and its application key are retained for 90 days, so that the workspace can be restored if needed. A restored workspace doesn't get its old keys back, which means that its integrations have to be connected again.

***

## Audit and Compliance Reference

### Control Objectives

This architecture is designed to demonstrate that credentials used to access your identity provider are protected from disclosure, that their use is limited to a single workspace, that access is restricted to what you've authorized, and that each use is recorded.

### Mapping Controls to Evidence

| Framework | Control | How the design supports it | Evidence |
| - | - | - | - |
| SOC 2 | CC6.1, Logical Access Security | Private keys are held in Cloud KMS or by Google and can't be exported | `okta.credential.key.generate.success`, the public key on the credential |
| SOC 2 | CC6.3, Role-Based Access | Vendor access is limited by your scopes, admin role, and delegation entry | Your Okta app settings and Admin console delegation entry |
| SOC 2 | CC7.2, System Monitoring | Each signature and key change is recorded | `okta.credential.sign.success`, `google.credential.sign.success` |
| ISO 27001:2022 | A.8.24, Use of Cryptography | Keys are generated, used, and destroyed in Cloud KMS, with defined lifecycles | `okta.credential.key.generate.success`, `okta.credential.key.destroy.success` |
| ISO 27001:2022 | A.5.17, Authentication Information | No shared or static credential is issued to or held by a person | The credential records, which contain no private key |
| NIST SP 800-53 | SC-12, Cryptographic Key Management | One key per integration, with rotation and destruction | The rotation and destruction events |
| NIST SP 800-53 | AC-6, Least Privilege | Read scopes are requested first, and IAM grants are per workspace | `okta.credential.scope.fallback`, your granted scopes |
| NIST SP 800-53 | AU-2, Event Logging | Signing and key lifecycle events are recorded | Your [event log](https://docs.provisionr.io/concepts/workspace/logs.md) |

### Logging Credential Use

Your event log records the following events. A signature event means that a new access token was requested, so a sync that reuses a cached token doesn't produce one.

| Event | Level | Meaning |
| - | - | - |
| `okta.credential.key.generate.success` | info | A signing key version was generated for a credential |
| `okta.credential.key.generate.error` | error | Cloud KMS couldn't generate the key, and the credential was deactivated |
| `okta.credential.sign.success` | info | Cloud KMS signed a client assertion |
| `okta.credential.scope.fallback` | notice | Okta refused the read scope, and the manage scope was tried |
| `okta.credential.key.destroy.success` | notice | A key version was scheduled for destruction |
| `okta.credential.key.destroy.error` | error | A key version couldn't be scheduled for destruction |
| `google.credential.sign.success` | info | Google signed a delegation JWT |
| `google.credential.activate.success` | info | A Google credential passed its checks and was activated |

```json okta.credential.sign.success theme={null}
{
  "event_type": "okta.credential.sign.success",
  "level": "info",
  "message": "Okta Client Assertion Signed by Cloud KMS",
  "record_type": "okta_credential",
  "record_id": "okcrd_01m4k1ben6mjeen4r46m8r2aw6",
  "metadata": {
    "key_id": "provisionr-3q2h9vTQx8mZk1pA"
  }
}
```

```json google.credential.sign.success theme={null}
{
  "event_type": "google.credential.sign.success",
  "level": "info",
  "message": "Google Delegation JWT Signed by IAM",
  "record_type": "google_credential",
  "record_id": "gwcrd_01m4m2c7a9xq5p3h8w1k6d0ezt",
  "metadata": {
    "scope": "https://www.googleapis.com/auth/admin.directory.user.readonly"
  }
}
```

Separately, Google Cloud's data access logs record every Cloud KMS and IAM Credentials call, including the identity that made it. These logs are written to a storage bucket with a locked retention policy, so they can't be changed or deleted before the retention period ends. They live in our Google Cloud project, so they aren't part of your event log.

### Common Audit Questions

> [accordion] Private Key Access by Provisionr Staff
>
> A signing key can't be exported from Cloud KMS by anyone, including our engineers. Someone with the right IAM permissions could use a key to sign, but they couldn't take a copy of it, and the call would appear in Google Cloud's data access logs.

> [accordion] Revoking Provisionr's Access
>
> For Okta, remove the public key from your app, or deactivate the app. For Google, remove the Client ID from domain-wide delegation. Neither requires any action from us, and tokens that were already issued expire within 60 minutes.

> [accordion] Key Rotation Frequency
>
> An Okta signing key is rotated when you run the rotation command, and we'd ask you to run it if an incident ever affected your key. Google rotates the keys that it holds for your service account on its own schedule, without any change to your Client ID.

> [accordion] Stored Secrets
>
> Neither integration stores a private key, password, or static API token in a production workspace. The credential record holds the public key, the name of the key version, and the Client ID.

### Reviewing Credential Settings

These are the settings to check when you review the integration. [Reviewing the Okta App](https://docs.provisionr.io/integrations/okta/oauth2.md#reviewing-the-okta-app) goes through the Okta settings in detail.

* The public keys on your Okta app, which should include only the keys that Provisionr displayed
* The scopes on your Okta app and its admin role
* The Client ID and scopes in your domain-wide delegation entry, which should match the Client ID that Provisionr displayed
* The admin that you named as the subject on your Google integration

### Known Limitations

These are the boundaries to keep in mind when you evaluate this design.

* Signing keys use Cloud KMS's software protection level, rather than a hardware security module.
* The web application and the sync share one runtime identity, so anything running as the workspace can sign. IAM can't separate the two.
* A key can't be taken from a compromised workspace, but it can be used for as long as the compromise lasts.
* Anyone who can deploy to a workspace can run code as its runtime identity.
* Cached access tokens are protected by the application key rather than by Cloud KMS.
* Google access tokens are bearer tokens, so anyone who obtains one can use it until it expires.
* Okta activation confirms that a token can be issued, but it doesn't confirm that the app's admin role is correct.
* A rotation of an Okta key needs someone with the Okta `Super Administrator` role, because that's who Okta allows to change an app's keys once scopes have been granted.
* Okta key generation runs after the API has responded, when Cloud Run gives a workspace very little CPU unless another request is in progress. The CLI's polling provides that, and if generation is interrupted, running the rotation command again starts over.

***

If you want to see these flows from the setup side, the pages for each integration walk through what you do in your own Okta org or Google Workspace.

> [card] [Okta OAuth2 Architecture](https://docs.provisionr.io/integrations/okta/oauth2.md)
>
> Scopes, admin role permissions, and DPoP for Okta.

> [card] [Google OAuth2 Architecture](https://docs.provisionr.io/integrations/google/oauth2.md)
>
> Domain-wide delegation and the scopes for each feature.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.