Google Cloud Data Ingestion

Google Cloud Data Ingestion

Collection Method

Log Type

 Ingestion label

Preferred Logging Protocol - Format

Log collection method

Data Source

Log Type

 Ingestion label

Preferred Logging Protocol - Format

Log collection method

Data Source

Cloud Audit Logs

GCP_CLOUDAUDIT

Cloud Log Stream - JSON

 C2C - Push
(Direct Ingestion)

https://cloud.google.com/chronicle/docs/ingestion/cloud/ingest-gcp-logs

https://cloud.google.com/chronicle/docs/ingestion/default-parsers/collect-security-command-center-findings?hl=en

Cloud NAT

GCP_CLOUD_NAT

Cloud Run

GCP_RUN

Cloud IDS

GCP_IDS

Cloud Load Balancing

GCP_LOADBALANCING

Cloud SQL

GCP_CLOUDSQL

Unix system

NIX_SYSTEM

Linux Sysmon

LINUX_SYSMON

Windows Event

WINEVTLOG

Zeek JSON

BRO_JSON

Kubernetes Node

KUBERNETES_NODE

Linux Auditing System (AuditD)

AUDITD

Apigee

GCP_APIGEE_X

reCAPTCHA Enterprise

GCP_RECAPTCHA_ENTERPRISE

Cloud Run

GCP_RUN

NGFW Enterprise

GCP_NGFW_ENTERPRISE

Cloud DNS

GCP_DNS

Firewall Rule Logging

GCP_FIREWALL

Cloud Storage Context

GCP_STORAGE_CONTEXT

Compute Context

GCP_COMPUTE_CONTEXT

IAM Context

GCP_IAM_CONTEXT

BigQuery

GCP_BIGQUERY_CONTEXT

Cloud IAM Analysis

GCP_IAM_ANALYSIS

Cloud Functions Context

GCP_CLOUD_FUNCTIONS_CONTEXT

Cloud SQL Context

GCP_SQL_CONTEXT

GCP_NETWORK_CONNECTIVITY

GCP_NETWORK_CONNECTIVITY_CONTEXT

Resource Manager Context

GCP_RESOURCE_MANAGER_CONTEXT

Security Command Center Threat

GCP_SECURITYCENTER_THREAT

Security Command Center Error

GCP_SECURITYCENTER_ERROR

Security Command Center Misconfiguration

GCP_SECURITYCENTER_MISCONFIGURATION

Security Command Center Observation

GCP_SECURITYCENTER_OBSERVATION

Security Command Center Vulnerability

GCP_SECURITYCENTER_VULNERABILITY

Security Command Center posture Voilation

GCP_SECURITYCENTER_POSTURE_VIOLATION

Security Command Center Toxic Combination

GCP_SECURITYCENTER_TOXIC_COMBINATION

Security Command Center Unspecified

GCP_SECURITYCENTER_UNSPECIFIED

Device Configuration

Pre-Requisite

  1. Before you enable the log ingestion, please make sure the below permissions are added at the Org level.

    1. Predefined roles: roles/logging.configWriter

    2. Permissions: chroniclesm.gcpLogFlowFilters.update

  2. Contact Adaptive MxDR onboarding engineer and obtain the one-time access code required to enable ingestion of your GCP telemetry.

  3. To access the Google Security Operations, grant the following IAM roles to the account configuring the integration:

    • Chronicle Service Admin (roles/chroniclesm.admin): IAM role for performing all activities.

    • Chronicle Service Viewer (roles/chroniclesm.viewer): IAM role to only view the state of ingestion.

    • Security Center Admin Editor (roles/securitycenter.adminEditor): Required to enable the ingestion of Cloud Asset Metadata.

      To grant IAM roles using Google Cloud console, complete the following steps:

    • Log on to the Google Cloud Organization you want to connect to and navigate to the IAM screen using Products > IAM & Admin > IAM.

    • From the IAM screen, select the user and click Edit Member.

If you are not in the Organization view of IAM, the Edit Member button is disabled, and you need to navigate to the organization's IAM screen.

  1. In the Edit Permissions screen, click Add Another Role and search for Chronicle to find the IAM roles.

    • Once you have assigned the roles, click Save.

  2. To enable Cloud Asset Metadata, you must onboard the organization to the Security Command Center. See Overview of organization-level activation for more information.

To Enable Google Cloud data ingestion

Please follow below steps to configure direct ingestion from your Google Cloud organization into your Google Security Operations instance

  1. Under Security Tab, navigate to Detections and Controls > Google SecOps.
    Select a project from the organization you wish to enable logging.

image-20241224-124531.png

2.Click the Manage organization ingestion settings button

image-20241224-124709.png
  1. If you see the message Page not viewable for projects, select an organization, then click Select.

image-20241224-154958.png
  1. Enter your one-time access code in the 1-time Chronicle access code field.

  2. Check the box labeled I consent to the terms and conditions of Chronicle's usage of my Google Cloud data.

  3. Click Connect Chronicle

image-20240308-121656.png
  1. Navigate to the Global Ingestion Settings tab for the organization, confirm that the connected Chronicle instance ID (customer-xxxxxx) matches your SIEM tenant ID (https://acn-mdr-xxxxxx*.backstory.chronicle.security/)

If there is any discrepancy, please report to Adaptive MxDR Service Delivery Lead immediately before proceeding.

  1. Enable Google Cloud Logging, Cloud Asset Metadata & Security Command Center Premium Findings

image-20250211-144643.png
  • You need to have either Security Command Center Standard or Security Command Center Premium enabled to export Google Cloud asset metadata to Google Security Operations.

  • You must have Security Command Center Premium enabled at the organization level to export your Premium findings to Google Security Operations.

Identify logs using the Log Scoping Tool

To help you identify the logs that meet your security and compliance needs, you can use the log scoping tool. This tool provides an interactive table that lists valuable security-relevant logs across Google Cloud including Cloud Audit Logs, Access Transparency logs, network logs, and several platform logs.

You can use this to enable the logs for specific services and it will give you auto-generated log filter which you can use next step to configure Export filter settings.

You can follow https://cloud.google.com/architecture/security-log-analytics page for more information about log scoping tool.

Export Filter Settings

By default, your Cloud Audit logs (admin activity and system event) and Cloud DNS logs are sent to your Chronicle account. However, you can customize the export filter to include or exclude specific types of logs. The export filter is based on the Google logging query language.

To define a custom filter for your logs, complete the following steps:

  1. Define your filter by creating a custom filter for your logs using the logging query language.
    The following documentation describes how to define this type of filter: https://cloud.google.com/logging/docs/view/logging-query-language .

  2. Navigate to Google SecOps > EXPORT FILTER SETTINGS

  3. Navigate to the Logs Explorer using the link provided on the EXPORT FILTER SETTINGS tab, copy your new query into the Query field and click Run Query to test it.

  4. Please find below export filters for your reference to export all supported Google Cloud Services.

    log_id("cloudaudit.googleapis.com/activity") OR log_id("cloudaudit.googleapis.com/system_event") OR log_id("cloudaudit.googleapis.com/policy") OR log_id("cloudaudit.googleapis.com/access_transparency") OR log_id("compute.googleapis.com/nat_flows") OR log_id("dns.googleapis.com/dns_queries") OR log_id("compute.googleapis.com/firewall") OR log_id("ids.googleapis.com/threat") OR log_id("ids.googleapis.com/traffic") OR log_id("requests") OR log_id("cloudsql.googleapis.com/mysql-general.log") OR log_id("cloudsql.googleapis.com/mysql.err") OR log_id("cloudsql.googleapis.com/postgres.log") OR log_id("cloudsql.googleapis.com/sqlagent.out") OR log_id("cloudsql.googleapis.com/sqlserver.err") OR log_id("syslog") OR log_id("authlog") OR log_id("securelog") OR log_id("winevt.raw") OR log_id("windows_event_log") OR log_id("zeek_json_streaming_conn") OR log_id("zeek_json_streaming_dhcp") OR log_id("zeek_json_streaming_dns") OR log_id("zeek_json_streaming_http") OR log_id("zeek_json_streaming_ssh") OR log_id("zeek_json_streaming_ssl") OR log_id("events") OR log_id("stdout") OR log_id("stderr") OR log_id("audit_log") OR log_id("recaptchaenterprise.googleapis.com/assessment") OR log_id("recaptchaenterprise.googleapis.com/annotation") OR log_id("run.googleapis.com/stderr") OR log_id("run.googleapis.com/stdout") OR log_id("run.googleapis.com/requests") OR log_id("run.googleapis.com/varlog/system") OR log_id("networksecurity.googleapis.com/firewall_threat") OR logName =~ "^projects/[\w\-]+/logs/apigee\.googleapis\.com[\w\-]*$"

If you want to monitor raw log telemetry data (Linux Sysmon) add log_id("sysmon.raw") filter to the above query.

Service Specific Export filters

Sr. No

GCP Cloud Service

Export Filter

Sr. No

GCP Cloud Service

Export Filter

  1.  

Cloud Audit Logs

  • log_id("cloudaudit.googleapis.com/activity")

  • log_id("cloudaudit.googleapis.com/system_event")

  • log_id("cloudaudit.googleapis.com/policy")

  • log_id("cloudaudit.googleapis.com/access_transparency")

  1.  

Cloud NAT Logs (Raw Log Telemetry)

  • log_id("compute.googleapis.com/nat_flows")

  1.  

Cloud DNS Logs

  • log_id("dns.googleapis.com/dns_queries")

  1.  

Cloud Next Generation Firewall logs

  • log_id("compute.googleapis.com/firewall")

  1.  

Cloud Intrusion Detection System

  • log_id("ids.googleapis.com/threat")

  • log_id("ids.googleapis.com/traffic")

  1.  

Cloud Load Balancing

  • log_id("requests")

  1.  

Cloud SQL

  • log_id("cloudsql.googleapis.com/mysql-general.log")

  • log_id("cloudsql.googleapis.com/mysql.err")

  • log_id("cloudsql.googleapis.com/postgres.log")

  • log_id("cloudsql.googleapis.com/sqlagent.out")

  • log_id("cloudsql.googleapis.com/sqlserver.err")

  1.  

Unix system

  • log_id("syslog")

  • log_id("authlog")

  • log_id("securelog")

  1.  

Linux Sysmon

  • log_id("sysmon.raw")

  1.  

Windows Event

  • log_id("winevt.raw")

  • log_id("windows_event_log")

  1.  

Zeek JSON

  • log_id("zeek_json_streaming_conn")

  • log_id("zeek_json_streaming_dhcp")

  • log_id("zeek_json_streaming_dns")

  • log_id("zeek_json_streaming_http")

  • log_id("zeek_json_streaming_ssh")

  • log_id("zeek_json_streaming_ssl")

  1.  

Kubernetes Node

  • log_id("events")

  • log_id("stdout")

  • log_id("stderr")

  1.  

Linux Auditing System (AuditD)

  • log_id("audit_log")

  1.  

Apigee (GCP_APIGEE_X)

  • logName =~ "^projects/[\w\-]+/logs/apigee\.googleapis\.com[\w\-]*$"

  1.  

reCAPTCHA Enterprise

  • log_id("recaptchaenterprise.googleapis.com/assessment")

  • log_id("recaptchaenterprise.googleapis.com/annotation")

  1.  

Cloud Run

  • log_id("run.googleapis.com/stderr")

  • log_id("run.googleapis.com/stdout")

  • log_id("run.googleapis.com/requests")

  • log_id("run.googleapis.com/varlog/system")

  1.  

NGFW Enterprise

  • log_id("networksecurity.googleapis.com/firewall_threat")

Google Cloud Asset Metadata Details

You can export your Google Cloud asset metadata from Cloud Asset Inventory to Google Security Operations. This asset metadata is drawn from your Cloud Asset Inventory and consists of information about your assets, resources, and identities including the following:

  • Environment

  • Location

  • Zone

  • Hardware models

  • Access control relationships between resources and identities

The following types of Google Cloud asset metadata will be exported to your Google Security Operations instance:

  • GCP_BIGQUERY_CONTEXT

  • GCP_COMPUTE_CONTEXT

  • GCP_IAM_CONTEXT

  • GCP_IAM_ANALYSIS

  • GCP_STORAGE_CONTEXT

  • GCP_CLOUD_FUNCTIONS_CONTEXT

  • GCP_SQL_CONTEXT

  • GCP_NETWORK_CONNECTIVITY_CONTEXT

  • GCP_RESOURCE_MANAGER_CONTEXT

Google Security Center Findings Details

The following types of Google Security Center Findings will be exported to your Google Security Operations instance:

  • GCP_SECURITYCENTER_ERROR

  • GCP_SECURITYCENTER_MISCONFIGURATION

  • GCP_SECURITYCENTER_OBSERVATION

  • GCP_SECURITYCENTER_THREAT

  • GCP_SECURITYCENTER_UNSPECIFIED

  • GCP_SECURITYCENTER_VULNERABILITY

  • GCP_SECURITYCENTER_POSTURE_VIOLATION

  • GCP_SECURITYCENTER_TOXIC_COMBINATION

 

About Accenture:
Accenture is a leading global professional services company that helps the world’s leading businesses, governments and other organizations build their digital core, optimize their operations, accelerate revenue growth and enhance citizen services—creating tangible value at speed and scale. We are a talent and innovation led company with 738,000 people serving clients in more than 120 countries. Technology is at the core of change today, and we are one of the world’s leaders in helping drive that change, with strong ecosystem relationships. We combine our strength in technology with unmatched industry experience, functional expertise and global delivery capability. We are uniquely able to deliver tangible outcomes because of our broad range of services, solutions and assets across Strategy & Consulting, Technology, Operations, Industry X and Accenture Song. These capabilities, together with our culture of shared success and commitment to creating 360° value, enable us to help our clients succeed and build trusted, lasting relationships. We measure our success by the 360° value we create for our clients, each other, our shareholders, partners and communities. Visit us at www.accenture.com.

About Accenture Security
Accenture Security is a leading provider of end-to-end cybersecurity services, including advanced cyber defense, applied cybersecurity solutions and managed security operations. We bring security innovation, coupled with global scale and a worldwide delivery capability through our network of Advanced Technology and Intelligent Operations centers. Helped by our team of highly skilled professionals, we enable clients to innovate safely, build cyber resilience and grow with confidence. Follow us @AccentureSecure on Twitter or visit us at www.accenture.com/security.

Legal notice: Accenture, the Accenture logo, and other trademarks, service marks, and designs are registered or unregistered trademarks of Accenture and its subsidiaries in the United States and in foreign countries. All trademarks are properties of their respective owners. This document is intended for general informational purposes only and does not take into account the reader’s specific circumstances, and may not reflect the most current developments. Accenture disclaims, to the fullest extent permitted by applicable law, any and all liability for the accuracy and completeness of the information in this presentation and for any acts or omissions made based on such information. Accenture does not provide legal, regulatory, audit, or tax advice. Readers are responsible for obtaining such advice from their own legal counsel or other licensed professionals.