Advanced Setup
Available in v1.20.0+

Prerequisites

Before starting, ensure you have Billing export configured to export detailed usage cost data to a BigQuery dataset, see billing export setup guide for more details

Google Cloud Cost Integration Setup

You can set up Google Cloud integration resources directly on your Google Cloud account, without using ScaleOps managed service account.

Run the script in your Google Cloud project:


curl -sL scriptshelf.scaleops.com/google/integrate.sh | bash -s -- cost \
  --cost-project-id <COST_PROJECT_ID> \
  --dataset-name <DATASET_NAME> \
  --create-service-account

Download Script / Preview

Copy the helm values provided in the output to your helm values and install ScaleOps.
Wait for ScaleOps to integrate with your Google Cloud cost data

The script configures daily cost preaggregation in the existing billing export dataset. To keep the aggregate tables somewhere else, pass --aggregate-project-id and --aggregate-dataset-name. Both datasets must be in the same BigQuery location, and the Helm values then point ScaleOps at the aggregate dataset.

ScaleOps reads those tables with BigQuery User (roles/bigquery.user) plus dataset READER. That built-in role is what lets the agent run queries and read scheduled-query status. A separate writer service account (scaleops-gke-cost-aggregate) gets dataset WRITER and BigQuery Job User so the scheduled query can refresh the tables.

Reference

Helm Parameters

Key	Description	Default
`enabled`	Enable Google Cloud cost integration	`false`
`projectId`	Google Cloud project ID	Current cluster project
`datasetName`	BigQuery dataset name for billing export	Your existing dataset
`serviceAccountCredentials`	Service account credentials JSON	Generated by script
`days`	Number of days to scan for cost data	`3`

Script parameters

Parameter	Description	Default / Notes
`-p <PROJECT_ID>`, `--project-id <PROJECT_ID>`	GCP project ID (required)
`-s <EMAIL>`, `--service-account <EMAIL>`	Service account email (required unless `--create-service-account` is used)
`-i <NAME>`, `--role-id <NAME>`	ID for the role	Default: `ScaleOpsIntegration`
`--service-account-name <NAME>`	Name for the service account	Default: `scaleops-<TIMESTAMP>`
`--create-service-account`	Create a new service account instead of using existing
`--skip-checks`	Skip organization policies validations	Default: `false`
`--cost-project-id <PROJECT_ID>`	Project ID for BigQuery cost data (required for cost integration)
`-d <DATASET>`, `--dataset-name <DATASET>`	BigQuery dataset name (required for cost integration)
`--aggregate-project-id <PROJECT_ID>`	Project for the ScaleOps aggregate tables	Default: `--cost-project-id`
`--aggregate-dataset-name <DATASET>`	Dataset for the ScaleOps aggregate tables	Default: `--dataset-name`
`-h`, `--help`	Show this help message

Example Helm Configuration


cloudBillingIntegration:
  google:
    enabled: true
    projectId: "<PROJECT_ID>"
    datasetName: "<DATASET_NAME>"
    serviceAccountCredentials: |
      {
        "type": "service_account",
        "project_id": "<PROJECT_ID>",
        "private_key_id": "...",
        "private_key": "...",
        "client_email": "scaleops-cost@<PROJECT_ID>.iam.gserviceaccount.com",
        "client_id": "...",
        "auth_uri": "https://accounts.google.com/o/oauth2/auth",
        "token_uri": "https://oauth2.googleapis.com/token"
      }

Permissions

Required Google Cloud Permissions

To run the integration script, the user running the script needs the following IAM roles:

Role	Scope	Purpose
Project Editor or Project Owner	Project	Create service accounts and manage IAM bindings
Service Account Admin	Project	Create and manage service accounts
BigQuery Admin	Project	Manage BigQuery permissions and scheduled queries
Organization Policy Viewer	Project	View organization policies (if applicable)
Billing Account Viewer	Billing Account	Access billing data and configure billing exports

Service accounts

The setup uses two accounts. Only the reader account goes into your Helm values.

ScaleOps reader (the account named in serviceAccountCredentials):

Role	Scope	Purpose
Dataset READER	Dataset in your Helm values	Read the billing export or the ScaleOps aggregate tables
BigQuery User (`roles/bigquery.user`)	Project	Run query jobs and read scheduled-query status

Aggregation writer (scaleops-gke-cost-aggregate, created for you):

Role	Scope	Purpose
Dataset READER	Billing export dataset	Read the raw billing export
Dataset WRITER	Aggregate dataset	Write the ScaleOps aggregate tables
BigQuery Job User (`roles/bigquery.jobUser`)	Aggregate project	Run the daily scheduled query

Minimal Required Permissions

IAM Permissions (to create and manage service accounts):

iam.serviceAccounts.create - Create service accounts
iam.serviceAccounts.get - Read service account information
iam.serviceAccounts.getIamPolicy - Read service account IAM policies
iam.serviceAccounts.setIamPolicy - Modify service account IAM policies

BigQuery Permissions (to manage dataset access):

bigquery.datasets.get - Read dataset information
bigquery.datasets.getIamPolicy - Read dataset IAM policies
bigquery.datasets.setIamPolicy - Modify dataset IAM policies

Resource Manager Permissions:

resourcemanager.projects.get - Read project information
resourcemanager.projects.getIamPolicy - Read project IAM policies
resourcemanager.projects.setIamPolicy - Modify project IAM policies

Billing Permissions (to access billing data and configure exports):

billing.accounts.get - Read billing account information
billing.billingAccounts.get - Access billing account details
cloudbilling.billingAccounts.get - Read Cloud Billing account data

Using External Secret

Run helm template with correct values of the Google billing integration to retrieve secret definition:


helm template ... --show-only templates/cur_settings_secret.yaml

Store the secret data from the helm template output on your secret provider, and reference it using External Secrets:


apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: scaleops-cloud-billing-settings
  namespace: scaleops-system
spec:
  secretStoreRef:
    name: SECRET_STORE_NAME # Replace with your secret store name
    kind: ClusterSecretStore # Replace with your secret store kind
  target:
    name: scaleops-cloud-billing-settings
    template:
      data:
        settings: "{{ .google_billing_config }}"
  data:
    - secretKey: google_billing_config
      remoteRef:
        key: path/to/secret # Replace with your secret path

If you use this approach, do not include cloudBillingIntegration.google values in your helm values file.

Advanced Setup Available in v1.20.0+