Google Cloud Command-line Examples
Table of Contents
Google Cloud Commands
All Google Cloud commands are self-documented, most ship with Bash and Zsh auto-completion profiles.
Their official documentation is available online at https://cloud.google.com/sdk/gcloud/reference/.
gcloud
You can use gcloud to create and manage:
- Google Compute Engine virtual machine instances and other resources
- Google Cloud SQL instances
- Google Container Engine clusters
- Google Cloud Dataproc clusters and jobs
- Google Cloud DNS managed zones and record sets
- Google Cloud Deployment manager deployments
- Deploy App Engine applications
- Perform other tasks against Alpha and Beta Cloud Platform services
gsutil
You can use gsutil to:
- Create and manage Cloud Storage buckets
- Upload objects to buckets, and download and delete them
- Move, copy and rename objects
- Manage access to stored data
bq
You can use bq to manage datasets, tables and other entities in BigQuery, as well as run queries on your data.
appcfg.py
Manage App Engine instances.
dev_appserver.py
Local App Engine development server.
Configuration
Install Google Cloud SDK
curl https://sdk.cloud.google.com | bash
Warning: You can verify the SSL certificate using curl –cacert <certificate> and this root CA:
# Operating CA: Symantec (GeoTrust)
# Issuer: CN=GeoTrust Global CA O=GeoTrust Inc.
# Subject: CN=GeoTrust Global CA O=GeoTrust Inc.
# Label: "GeoTrust Global CA"
# Serial: 144470
# MD5 Fingerprint: f7:75:ab:29:fb:51:4e:b7:77:5e:ff:05:3c:99:8e:f5
# SHA1 Fingerprint: de:28:f4:a4:ff:e5:b9:2f:a3:c5:03:d1:a3:49:a7:f9:96:2a:82:12
# SHA256 Fingerprint: ff:85:6a:2d:25:1d:cd:88:d3:66:56:f4:50:12:67:98:cf:ab:aa:de:40:79:9c:72:2d:e4:d2:b5:db:36:a7:3a
-----BEGIN CERTIFICATE-----
MIIDVDCCAjygAwIBAgIDAjRWMA0GCSqGSIb3DQEBBQUAMEIxCzAJBgNVBAYTAlVT
MRYwFAYDVQQKEw1HZW9UcnVzdCBJbmMuMRswGQYDVQQDExJHZW9UcnVzdCBHbG9i
YWwgQ0EwHhcNMDIwNTIxMDQwMDAwWhcNMjIwNTIxMDQwMDAwWjBCMQswCQYDVQQG
EwJVUzEWMBQGA1UEChMNR2VvVHJ1c3QgSW5jLjEbMBkGA1UEAxMSR2VvVHJ1c3Qg
R2xvYmFsIENBMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA2swYYzD9
9BcjGlZ+W988bDjkcbd4kdS8odhM+KhDtgPpTSEHCIjaWC9mOSm9BXiLnTjoBbdq
fnGk5sRgprDvgOSJKA+eJdbtg/OtppHHmMlCGDUUna2YRpIuT8rxh0PBFpVXLVDv
iS2Aelet8u5fa9IAjbkU+BQVNdnARqN7csiRv8lVK83Qlz6cJmTM386DGXHKTubU
1XupGc1V3sjs0l44U+VcT4wt/lAjNvxm5suOpDkZALeVAjmRCw7+OC7RHQWa9k0+
bw8HHa8sHo9gOeL6NlMTOdReJivbPagUvTLrGAMoUgRx5aszPeE4uwc2hGKceeoW
MPRfwCvocWvk+QIDAQABo1MwUTAPBgNVHRMBAf8EBTADAQH/MB0GA1UdDgQWBBTA
ephojYn7qwVkDBF9qn1luMrMTjAfBgNVHSMEGDAWgBTAephojYn7qwVkDBF9qn1l
uMrMTjANBgkqhkiG9w0BAQUFAAOCAQEANeMpauUvXVSOKVCUn5kaFOSPeCpilKIn
Z57QzxpeR+nBsqTP3UEaBU6bS+5Kb1VSsyShNwrrZHYqLizz/Tt1kL/6cdjHPTfS
tQWVYrmm3ok9Nns4d0iXrKYgjy6myQzCsplFAMfOEVEiIuCl6rYVSAlk6l5PdPcF
PseKUgzbFbS9bZvlxrFUaKnjaZC2mqUPuLk/IH2uSrW4nOQdtqvmlKXBx4Ot2/Un
hw4EbNX/3aBd7YdStysVAq45pmp06drE57xNNB6pXE0zX5IJL4hmXXeXxx12E6nV
5fEWCRE11azbJHFwLJhWC9kXtNHjUStedejV0NxPNO3CBWaAocvmMw==
-----END CERTIFICATE-----
Install App Engine Python Standard Environment
gcloud components install app-engine-python
List Available Components
gcloud components list
Your current Cloud SDK version is: 123.0.0
The latest available version is: 123.0.0
┌──────────────────────────────────────────────────────────────────────────────────────────┐
│ Components │
├───────────────┬───────────────────────────────────┬──────────────────────────┬───────────┤
│ Status │ Name │ ID │ Size │
├───────────────┼───────────────────────────────────┼──────────────────────────┼───────────┤
│ Not Installed │ App Engine Go Extensions │ app-engine-go │ 46.3 MiB │
│ Not Installed │ Cloud Datastore Emulator │ cloud-datastore-emulator │ 16.1 MiB │
│ Not Installed │ Cloud Datastore Emulator (Legacy) │ gcd-emulator │ 38.1 MiB │
│ Not Installed │ Cloud Pub/Sub Emulator │ pubsub-emulator │ 16.3 MiB │
│ Not Installed │ gcloud Alpha Commands │ alpha │ < 1 MiB │
│ Not Installed │ gcloud Beta Commands │ beta │ < 1 MiB │
│ Not Installed │ gcloud app Java Extensions │ app-engine-java │ 135.6 MiB │
│ Not Installed │ gcloud app Python Extensions │ app-engine-python │ 7.2 MiB │
│ Not Installed │ kubectl │ kubectl │ 11.4 MiB │
│ Installed │ BigQuery Command Line Tool │ bq │ < 1 MiB │
│ Installed │ Cloud SDK Core Libraries │ core │ 4.1 MiB │
│ Installed │ Cloud Storage Command Line Tool │ gsutil │ 2.8 MiB │
│ Installed │ Default set of gcloud commands │ gcloud │ │
└───────────────┴───────────────────────────────────┴──────────────────────────┴───────────┘
To install or remove components at your current SDK version [123.0.0], run:
gcloud components install COMPONENT_ID
gcloud components remove COMPONENT_ID
To update your SDK installation to the latest version [123.0.0], run:
gcloud components update
Update Components
gcloud components update
Install App Engine Components
gcloud components install app-engine-python gcd-emulator pubsub-emulator cloud-datastore-emulator
Your current Cloud SDK version is: 123.0.0
Installing components from version: 123.0.0
┌──────────────────────────────────────────────────────────────┐
│ These components will be installed. │
├───────────────────────────────────┬───────────────┬──────────┤
│ Name │ Version │ Size │
├───────────────────────────────────┼───────────────┼──────────┤
│ Cloud Datastore Emulator │ 1.1.1 │ 16.1 MiB │
│ Cloud Datastore Emulator (Legacy) │ v1beta3-1.0.0 │ 38.1 MiB │
│ Cloud Pub/Sub Emulator │ 2016.08.19 │ 16.3 MiB │
│ gcloud app Python Extensions │ 1.9.40 │ 7.2 MiB │
└───────────────────────────────────┴───────────────┴──────────┘
[...]
GCloud Authentication
Project-level authentication:
gcloud auth login --project=<project ID>
Configure Default Command Line Project
gcloud config set project <project ID>
Cloud Storage
Create a Cloud Storage bucket
gsutil mb gs://<bucket name>
List files in a Cloud Storage bucket
gsutil ls gs://<bucket>
Synchronize buckets and directories files
gsutil -m rsync --help
Cloud Storage ACL with gsutil
Make bucket files world-readable:
gsutil acl ch -u AllUsers:R gs://<bucket>/<object>
HTTPS Load-balancing with Cloud CDN
Compute Engine
Copy files to/from a Compute Engine instance
gcloud compute copy-files [LOCAL_FILE_PATH] [INSTANCE_NAME]:~/
List Compute Engine Instances
gcloud compute instances list
Connect via SSH
gcloud compute ssh <instance name>
Create an instance with a GPU
gcloud beta compute instances create gpu-instance-1 \
--machine-type n1-standard-2 --zone us-east1-d \
--accelerator type=nvidia-tesla-k80,count=1 \
--image-family ubuntu-1604-lts --image-project ubuntu-os-cloud \
--maintenance-policy TERMINATE --restart-on-failure \
--metadata startup-script='#!/bin/bash
echo "Checking for CUDA and installing."
if ! dpkg-query -W cuda; then
curl -O http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/cuda-repo-ubuntu1604_8.0.61-1_amd64.deb
dpkg -i ./cuda-repo-ubuntu1604_8.0.61-1_amd64.deb
apt-get update
apt-get install cuda -y
fi'
App Engine
Deploy Application
gcloud app
appcfg.py update -A <project ID> -V <version> <src>
Run Python App Engine application locally
dev_appserver.py <src>
with application debug log level:
dev_appserver.py --log_level debug <src>
with application server debug logging:
dev_appserver.py --dev_appserver_log_level debug <src>
with both:
dev_appserver.py --log_level debug --dev_appserver_log_level debug <src>
List App Engine Instances
gcloud app instances list
Read App Engine Logs
gcloud app logs read
Stop App
gcloud compute instances stop <instance-name> <instance-name> ...
Spawn a remote Python shell
In order to have access to the remoteapi, it must first be deployed with the option enabled1:
builtins:
- remote_api: on
remote_api_shell.py -s <project-id>.appspot.com
ref: https://cloud.google.com/appengine/docs/python/tools/remoteapi
Describe a service version
gcloud app versions describe -s default v1
Stackdriver Logging
Errors within last 10 minutes
gcloud beta logging read --freshness="10m" "severity>=ERROR"
Cloud DNS
List Managed Zones
gcloud dns managed-zones list
output:
NAME DNS_NAME DESCRIPTION
example-zone example.com.
Describe Existing Zone
gcloud dns managed-zones describe <zone-name>
Add DNS Record
Here is an example of creating a CNAME record named my-cname.example.com that points to cname.value.example.org
$ gcloud dns record-sets transaction start -z <zone-name>
Transaction started [transaction.yaml]
$ gcloud dns record-sets transaction add -z <zone-name> --ttl <seconds> --type CNAME --name my-cname.example.com cname.value.example.org.
Record addition appended to transaction at [transaction.yaml]
$ gcloud dns record-sets transaction describe -z <zone-name>
additions:
- kind: dns#resourceRecordSet
name: example.com
rrdatas:
- ns-cloud-e1.googledomains.com. cloud-dns-hostmaster.google.com. 3 21600 3600 259200
<ttl seconds>
ttl: 21600
type: SOA
- kind: dns#resourceRecordSet
name: mycname.example.com
rrdatas:
- cname.value.example.org.
ttl: <ttl seconds>
type: CNAME
deletions:
- kind: dns#resourceRecordSet
name: example.com
rrdatas:
- ns-cloud-e1.googledomains.com. cloud-dns-hostmaster.google.com. 2 21600 3600 259200
<ttl seconds>
ttl: 21600
type: SOA
$ gcloud dns record-sets transaction execute -z <zone-name>
Executed transaction [transaction.yaml] for managed-zone [<zone-name>].
Created [https://www.googleapis.com/dns/v1/projects/<project id>/managedZones/<zone-name>/changes/1].
ID START_TIME STATUS
1 2016-06-16T13:10:50.051Z pending
Plain commands list:
gcloud dns record-sets transaction start -z <zone-name>
gcloud dns record-sets transaction add -z <zone-name> --name my-cname.example.com --ttl 300 --type CNAME cname.value.example.org.
gcloud dns record-sets transaction describe -z <zone-name>
gcloud dns record-sets transaction execute -z <zone-name>
BigQuery
bq help includes a lot of great info, the official documentation is available at https://cloud.google.com/bigquery/bq-command-line-tool.
Default Configuration
cat > ~/.bigqueryrc <<EOF
> project_id = <project-id>
> dataset_id = <dataset-id>
> EOF
Interactive Shell
bq shell
List common flags
bq --format=json ls -d
Comments
Comments powered by Disqus