Dataproc Metastore audit logging

This document describes audit logging for Dataproc Metastore. Google Cloud services generate audit logs that record administrative and access activities within your Google Cloud resources. For more information about Cloud Audit Logs, see the following:

• Types of audit logs
• Audit log entry structure
• Storing and routing audit logs
• Cloud Logging pricing summary
• Enable Data Access audit logs

Service name

Dataproc Metastore audit logs use the service name metastore.googleapis.com. Filter for this service:

    protoPayload.serviceName="metastore.googleapis.com"
  

Methods by permission type

Each IAM permission has a type property, whose value is an enum that can be one of four values: ADMIN_READ, ADMIN_WRITE, DATA_READ, or DATA_WRITE. When you call a method, Dataproc Metastore generates an audit log whose category is dependent on the type property of the permission required to perform the method. Methods that require an IAM permission with the type property value of DATA_READ, DATA_WRITE, or ADMIN_READ generate Data Access audit logs. Methods that require an IAM permission with the type property value of ADMIN_WRITE generate Admin Activity audit logs.

API interface audit logs

For information about how and which permissions are evaluated for each method, see the Identity and Access Management documentation for Dataproc Metastore.

google.cloud.metastore.v1.DataprocMetastore

The following audit logs are associated with methods belonging to google.cloud.metastore.v1.DataprocMetastore.

CreateBackup

• Method: google.cloud.metastore.v1.DataprocMetastore.CreateBackup
  
• Audit log type: Admin activity
  
• Permissions:
  • metastore.backups.create - ADMIN_WRITE
• Method is a long-running or streaming operation: Long-running operation
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1.DataprocMetastore.CreateBackup"
  

CreateMetadataImport

• Method: google.cloud.metastore.v1.DataprocMetastore.CreateMetadataImport
  
• Audit log type: Data access
  
• Permissions:
  • metastore.imports.create - DATA_WRITE
• Method is a long-running or streaming operation: Long-running operation
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1.DataprocMetastore.CreateMetadataImport"
  

CreateService

• Method: google.cloud.metastore.v1.DataprocMetastore.CreateService
  
• Audit log type: Admin activity
  
• Permissions:
  • metastore.services.create - ADMIN_WRITE
• Method is a long-running or streaming operation: Long-running operation
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1.DataprocMetastore.CreateService"
  

DeleteBackup

• Method: google.cloud.metastore.v1.DataprocMetastore.DeleteBackup
  
• Audit log type: Admin activity
  
• Permissions:
  • metastore.backups.delete - ADMIN_WRITE
• Method is a long-running or streaming operation: Long-running operation
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1.DataprocMetastore.DeleteBackup"
  

DeleteService

• Method: google.cloud.metastore.v1.DataprocMetastore.DeleteService
  
• Audit log type: Admin activity
  
• Permissions:
  • metastore.services.delete - ADMIN_WRITE
• Method is a long-running or streaming operation: Long-running operation
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1.DataprocMetastore.DeleteService"
  

ExportMetadata

• Method: google.cloud.metastore.v1.DataprocMetastore.ExportMetadata
  
• Audit log type: Data access
  
• Permissions:
  • metastore.services.export - DATA_READ
• Method is a long-running or streaming operation: Long-running operation
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1.DataprocMetastore.ExportMetadata"
  

GetBackup

• Method: google.cloud.metastore.v1.DataprocMetastore.GetBackup
  
• Audit log type: Data access
  
• Permissions:
  • metastore.backups.get - ADMIN_READ
• Method is a long-running or streaming operation: No.
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1.DataprocMetastore.GetBackup"
  

GetMetadataImport

• Method: google.cloud.metastore.v1.DataprocMetastore.GetMetadataImport
  
• Audit log type: Data access
  
• Permissions:
  • metastore.imports.get - ADMIN_READ
• Method is a long-running or streaming operation: No.
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1.DataprocMetastore.GetMetadataImport"
  

GetService

• Method: google.cloud.metastore.v1.DataprocMetastore.GetService
  
• Audit log type: Data access
  
• Permissions:
  • metastore.services.get - ADMIN_READ
• Method is a long-running or streaming operation: No.
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1.DataprocMetastore.GetService"
  

ListBackups

• Method: google.cloud.metastore.v1.DataprocMetastore.ListBackups
  
• Audit log type: Data access
  
• Permissions:
  • metastore.backups.list - ADMIN_READ
• Method is a long-running or streaming operation: No.
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1.DataprocMetastore.ListBackups"
  

ListServices

• Method: google.cloud.metastore.v1.DataprocMetastore.ListServices
  
• Audit log type: Data access
  
• Permissions:
  • metastore.services.list - ADMIN_READ
• Method is a long-running or streaming operation: No.
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1.DataprocMetastore.ListServices"
  

QueryMetadata

• Method: google.cloud.metastore.v1.DataprocMetastore.QueryMetadata
  
• Audit log type: Data access
  
• Permissions:
  • metastore.services.queryMetadata - DATA_READ
• Method is a long-running or streaming operation: Long-running operation
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1.DataprocMetastore.QueryMetadata"
  

RestoreService

• Method: google.cloud.metastore.v1.DataprocMetastore.RestoreService
  
• Audit log type: Data access
  
• Permissions:
  • metastore.backups.use - DATA_READ
  • metastore.services.restore - DATA_WRITE
• Method is a long-running or streaming operation: Long-running operation
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1.DataprocMetastore.RestoreService"
  

UpdateService

• Method: google.cloud.metastore.v1.DataprocMetastore.UpdateService
  
• Audit log type: Admin activity
  
• Permissions:
  • metastore.services.update - ADMIN_WRITE
• Method is a long-running or streaming operation: Long-running operation
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1.DataprocMetastore.UpdateService"
  

google.cloud.metastore.v1.DataprocMetastoreFederation

The following audit logs are associated with methods belonging to google.cloud.metastore.v1.DataprocMetastoreFederation.

CreateFederation

• Method: google.cloud.metastore.v1.DataprocMetastoreFederation.CreateFederation
  
• Audit log type: Admin activity
  
• Permissions:
  • metastore.federations.create - ADMIN_WRITE
• Method is a long-running or streaming operation: Long-running operation
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1.DataprocMetastoreFederation.CreateFederation"
  

DeleteFederation

• Method: google.cloud.metastore.v1.DataprocMetastoreFederation.DeleteFederation
  
• Audit log type: Admin activity
  
• Permissions:
  • metastore.federations.delete - ADMIN_WRITE
• Method is a long-running or streaming operation: Long-running operation
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1.DataprocMetastoreFederation.DeleteFederation"
  

GetFederation

• Method: google.cloud.metastore.v1.DataprocMetastoreFederation.GetFederation
  
• Audit log type: Data access
  
• Permissions:
  • metastore.federations.get - ADMIN_READ
• Method is a long-running or streaming operation: No.
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1.DataprocMetastoreFederation.GetFederation"
  

ListFederations

• Method: google.cloud.metastore.v1.DataprocMetastoreFederation.ListFederations
  
• Audit log type: Data access
  
• Permissions:
  • metastore.federations.list - ADMIN_READ
• Method is a long-running or streaming operation: No.
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1.DataprocMetastoreFederation.ListFederations"
  

UpdateFederation

• Method: google.cloud.metastore.v1.DataprocMetastoreFederation.UpdateFederation
  
• Audit log type: Admin activity
  
• Permissions:
  • metastore.federations.update - ADMIN_WRITE
• Method is a long-running or streaming operation: Long-running operation
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1.DataprocMetastoreFederation.UpdateFederation"
  

google.cloud.metastore.v1alpha.DataprocMetastore

The following audit logs are associated with methods belonging to google.cloud.metastore.v1alpha.DataprocMetastore.

CreateBackup

• Method: google.cloud.metastore.v1alpha.DataprocMetastore.CreateBackup
  
• Audit log type: Admin activity
  
• Permissions:
  • metastore.backups.create - ADMIN_WRITE
• Method is a long-running or streaming operation: Long-running operation
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1alpha.DataprocMetastore.CreateBackup"
  

CreateMetadataImport

• Method: google.cloud.metastore.v1alpha.DataprocMetastore.CreateMetadataImport
  
• Audit log type: Data access
  
• Permissions:
  • metastore.imports.create - DATA_WRITE
• Method is a long-running or streaming operation: Long-running operation
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1alpha.DataprocMetastore.CreateMetadataImport"
  

CreateService

• Method: google.cloud.metastore.v1alpha.DataprocMetastore.CreateService
  
• Audit log type: Admin activity
  
• Permissions:
  • metastore.services.create - ADMIN_WRITE
• Method is a long-running or streaming operation: Long-running operation
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1alpha.DataprocMetastore.CreateService"
  

DeleteBackup

• Method: google.cloud.metastore.v1alpha.DataprocMetastore.DeleteBackup
  
• Audit log type: Admin activity
  
• Permissions:
  • metastore.backups.delete - ADMIN_WRITE
• Method is a long-running or streaming operation: Long-running operation
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1alpha.DataprocMetastore.DeleteBackup"
  

DeleteService

• Method: google.cloud.metastore.v1alpha.DataprocMetastore.DeleteService
  
• Audit log type: Admin activity
  
• Permissions:
  • metastore.services.delete - ADMIN_WRITE
• Method is a long-running or streaming operation: Long-running operation
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1alpha.DataprocMetastore.DeleteService"
  

ExportMetadata

• Method: google.cloud.metastore.v1alpha.DataprocMetastore.ExportMetadata
  
• Audit log type: Data access
  
• Permissions:
  • metastore.services.export - DATA_READ
• Method is a long-running or streaming operation: Long-running operation
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1alpha.DataprocMetastore.ExportMetadata"
  

GetBackup

• Method: google.cloud.metastore.v1alpha.DataprocMetastore.GetBackup
  
• Audit log type: Data access
  
• Permissions:
  • metastore.backups.get - ADMIN_READ
• Method is a long-running or streaming operation: No.
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1alpha.DataprocMetastore.GetBackup"
  

GetMetadataImport

• Method: google.cloud.metastore.v1alpha.DataprocMetastore.GetMetadataImport
  
• Audit log type: Data access
  
• Permissions:
  • metastore.imports.get - ADMIN_READ
• Method is a long-running or streaming operation: No.
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1alpha.DataprocMetastore.GetMetadataImport"
  

GetService

• Method: google.cloud.metastore.v1alpha.DataprocMetastore.GetService
  
• Audit log type: Data access
  
• Permissions:
  • metastore.services.get - ADMIN_READ
• Method is a long-running or streaming operation: No.
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1alpha.DataprocMetastore.GetService"
  

ListBackups

• Method: google.cloud.metastore.v1alpha.DataprocMetastore.ListBackups
  
• Audit log type: Data access
  
• Permissions:
  • metastore.backups.list - ADMIN_READ
• Method is a long-running or streaming operation: No.
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1alpha.DataprocMetastore.ListBackups"
  

ListMetadataImports

• Method: google.cloud.metastore.v1alpha.DataprocMetastore.ListMetadataImports
  
• Audit log type: Data access
  
• Permissions:
  • metastore.imports.list - ADMIN_READ
• Method is a long-running or streaming operation: No.
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1alpha.DataprocMetastore.ListMetadataImports"
  

ListServices

• Method: google.cloud.metastore.v1alpha.DataprocMetastore.ListServices
  
• Audit log type: Data access
  
• Permissions:
  • metastore.services.list - ADMIN_READ
• Method is a long-running or streaming operation: No.
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1alpha.DataprocMetastore.ListServices"
  

RemoveIamPolicy

• Method: google.cloud.metastore.v1alpha.DataprocMetastore.RemoveIamPolicy
  
• Audit log type: Admin activity
  
• Permissions:
  • metastore.databases.setIamPolicy - ADMIN_WRITE
  • metastore.tables.setIamPolicy - ADMIN_WRITE
• Method is a long-running or streaming operation: No.
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1alpha.DataprocMetastore.RemoveIamPolicy"
  

UpdateMetadataImport

• Method: google.cloud.metastore.v1alpha.DataprocMetastore.UpdateMetadataImport
  
• Audit log type: Admin activity
  
• Permissions:
  • metastore.imports.update - ADMIN_WRITE
• Method is a long-running or streaming operation: Long-running operation
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1alpha.DataprocMetastore.UpdateMetadataImport"
  

UpdateService

• Method: google.cloud.metastore.v1alpha.DataprocMetastore.UpdateService
  
• Audit log type: Admin activity
  
• Permissions:
  • metastore.services.update - ADMIN_WRITE
• Method is a long-running or streaming operation: Long-running operation
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1alpha.DataprocMetastore.UpdateService"
  

google.cloud.metastore.v1beta.DataprocMetastore

The following audit logs are associated with methods belonging to google.cloud.metastore.v1beta.DataprocMetastore.

AlterMetadataResourceLocation

• Method: google.cloud.metastore.v1beta.DataprocMetastore.AlterMetadataResourceLocation
  
• Audit log type: Data access
  
• Permissions:
  • metastore.services.mutateMetadata - DATA_WRITE
• Method is a long-running or streaming operation: Long-running operation
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1beta.DataprocMetastore.AlterMetadataResourceLocation"
  

AlterTableProperties

• Method: google.cloud.metastore.v1beta.DataprocMetastore.AlterTableProperties
  
• Audit log type: Data access
  
• Permissions:
  • metastore.services.mutateMetadata - DATA_WRITE
• Method is a long-running or streaming operation: Long-running operation
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1beta.DataprocMetastore.AlterTableProperties"
  

CreateService

• Method: google.cloud.metastore.v1beta.DataprocMetastore.CreateService
  
• Audit log type: Admin activity
  
• Permissions:
  • metastore.services.create - ADMIN_WRITE
• Method is a long-running or streaming operation: Long-running operation
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1beta.DataprocMetastore.CreateService"
  

DeleteService

• Method: google.cloud.metastore.v1beta.DataprocMetastore.DeleteService
  
• Audit log type: Admin activity
  
• Permissions:
  • metastore.services.delete - ADMIN_WRITE
• Method is a long-running or streaming operation: Long-running operation
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1beta.DataprocMetastore.DeleteService"
  

GetService

• Method: google.cloud.metastore.v1beta.DataprocMetastore.GetService
  
• Audit log type: Data access
  
• Permissions:
  • metastore.services.get - ADMIN_READ
• Method is a long-running or streaming operation: No.
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1beta.DataprocMetastore.GetService"
  

ListBackups

• Method: google.cloud.metastore.v1beta.DataprocMetastore.ListBackups
  
• Audit log type: Data access
  
• Permissions:
  • metastore.backups.list - ADMIN_READ
• Method is a long-running or streaming operation: No.
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1beta.DataprocMetastore.ListBackups"
  

ListServices

• Method: google.cloud.metastore.v1beta.DataprocMetastore.ListServices
  
• Audit log type: Data access
  
• Permissions:
  • metastore.services.list - ADMIN_READ
• Method is a long-running or streaming operation: No.
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1beta.DataprocMetastore.ListServices"
  

MoveTableToDatabase

• Method: google.cloud.metastore.v1beta.DataprocMetastore.MoveTableToDatabase
  
• Audit log type: Data access
  
• Permissions:
  • metastore.services.mutateMetadata - DATA_WRITE
• Method is a long-running or streaming operation: Long-running operation
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1beta.DataprocMetastore.MoveTableToDatabase"
  

QueryMetadata

• Method: google.cloud.metastore.v1beta.DataprocMetastore.QueryMetadata
  
• Audit log type: Data access
  
• Permissions:
  • metastore.services.queryMetadata - DATA_READ
• Method is a long-running or streaming operation: Long-running operation
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1beta.DataprocMetastore.QueryMetadata"
  

UpdateMetadataImport

• Method: google.cloud.metastore.v1beta.DataprocMetastore.UpdateMetadataImport
  
• Audit log type: Admin activity
  
• Permissions:
  • metastore.imports.update - ADMIN_WRITE
• Method is a long-running or streaming operation: Long-running operation
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1beta.DataprocMetastore.UpdateMetadataImport"
  

UpdateService

• Method: google.cloud.metastore.v1beta.DataprocMetastore.UpdateService
  
• Audit log type: Admin activity
  
• Permissions:
  • metastore.services.update - ADMIN_WRITE
• Method is a long-running or streaming operation: Long-running operation
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1beta.DataprocMetastore.UpdateService"
  

google.cloud.metastore.v1beta.DataprocMetastoreFederation

The following audit logs are associated with methods belonging to google.cloud.metastore.v1beta.DataprocMetastoreFederation.

CreateFederation

• Method: google.cloud.metastore.v1beta.DataprocMetastoreFederation.CreateFederation
  
• Audit log type: Admin activity
  
• Permissions:
  • metastore.federations.create - ADMIN_WRITE
• Method is a long-running or streaming operation: Long-running operation
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1beta.DataprocMetastoreFederation.CreateFederation"
  

DeleteFederation

• Method: google.cloud.metastore.v1beta.DataprocMetastoreFederation.DeleteFederation
  
• Audit log type: Admin activity
  
• Permissions:
  • metastore.federations.delete - ADMIN_WRITE
• Method is a long-running or streaming operation: Long-running operation
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1beta.DataprocMetastoreFederation.DeleteFederation"
  

GetFederation

• Method: google.cloud.metastore.v1beta.DataprocMetastoreFederation.GetFederation
  
• Audit log type: Data access
  
• Permissions:
  • metastore.federations.get - ADMIN_READ
• Method is a long-running or streaming operation: No.
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1beta.DataprocMetastoreFederation.GetFederation"
  

ListFederations

• Method: google.cloud.metastore.v1beta.DataprocMetastoreFederation.ListFederations
  
• Audit log type: Data access
  
• Permissions:
  • metastore.federations.list - ADMIN_READ
• Method is a long-running or streaming operation: No.
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1beta.DataprocMetastoreFederation.ListFederations"
  

UpdateFederation

• Method: google.cloud.metastore.v1beta.DataprocMetastoreFederation.UpdateFederation
  
• Audit log type: Admin activity
  
• Permissions:
  • metastore.federations.update - ADMIN_WRITE
• Method is a long-running or streaming operation: Long-running operation
  
• Filter for this method: protoPayload.methodName="google.cloud.metastore.v1beta.DataprocMetastoreFederation.UpdateFederation"
  

google.iam.v1.IAMPolicy

The following audit logs are associated with methods belonging to google.iam.v1.IAMPolicy.

GetIamPolicy

• Method: GetIamPolicy
  
• Audit log type: Data access
  
• Permissions:
  • metastore.federations.getIamPolicy - ADMIN_READ
  • metastore.services.getIamPolicy - ADMIN_READ
• Method is a long-running or streaming operation: No.
  
• Filter for this method: protoPayload.methodName="GetIamPolicy"
  

SetIamPolicy

• Method: SetIamPolicy
  
• Audit log type: Admin activity
  
• Permissions:
  • metastore.databases.setIamPolicy - ADMIN_WRITE
  • metastore.federations.setIamPolicy - ADMIN_WRITE
  • metastore.services.setIamPolicy - ADMIN_WRITE
• Method is a long-running or streaming operation: No.
  
• Filter for this method: protoPayload.methodName="SetIamPolicy"
  

google.longrunning.Operations

The following audit logs are associated with methods belonging to google.longrunning.Operations.

DeleteOperation

• Method: google.longrunning.Operations.DeleteOperation
  
• Audit log type: Admin activity
  
• Permissions:
  • metastore.operations.delete - ADMIN_WRITE
• Method is a long-running or streaming operation: No.
  
• Filter for this method: protoPayload.methodName="google.longrunning.Operations.DeleteOperation"
  

GetOperation

• Method: google.longrunning.Operations.GetOperation
  
• Audit log type: Data access
  
• Permissions:
  • metastore.operations.get - ADMIN_READ
• Method is a long-running or streaming operation: No.
  
• Filter for this method: protoPayload.methodName="google.longrunning.Operations.GetOperation"
  

ListOperations

• Method: google.longrunning.Operations.ListOperations
  
• Audit log type: Data access
  
• Permissions:
  • metastore.operations.list - ADMIN_READ
• Method is a long-running or streaming operation: No.
  
• Filter for this method: protoPayload.methodName="google.longrunning.Operations.ListOperations"