我计划创建一个脚本来从Bigquery中提取数据,但我不知道如何设置环境变量。
下面是官方文档中的一个实例:
from google.cloud import bigquery
# Construct a BigQuery client object.
client = bigquery.Client()
query = """
SELECT name, SUM(number) as total_people
FROM `bigquery-public-data.usa_names.usa_1910_2013`
WHERE state = 'TX'
GROUP BY name, state
ORDER BY total_people DESC
LIMIT 20
"""
query_job = client.query(query) # Make an API request.
print("The query data:")
for row in query_job:
# Row values can be accessed by field name or index.
print("name={}, count={}".format(row[0], row["total_people"]))
我运行它,但返回一个错误:
DefaultCredentialsError: Could not automatically determine credentials. Please set GOOGLE_APPLICATION_CREDENTIALS or explicitly create credentials and re-run the application. For more information, please see https://cloud.google.com/docs/authentication/getting-started
我遵循官方医生,但我遇到了一个问题:第二步是设置环境变量,但它只是在Windows和Linux上提供实例。那么,如何在Colab上设置环境变量呢?
另外,我注意到实例要求我提供关键路径。在本地机器上是可以的,但是我不认为上传我的密钥文件并在我的代码在线通过它的链接是一个想法。
发布于 2022-01-19 18:02:23
与设置环境变量或直接上传到Colab不同,您可以在Google上上传您的密钥,并在那里应用必要的限制。在您的代码中,您可以将Google挂载到Colab,使用Drive位置作为关键文件路径进行身份验证。
from google.cloud import bigquery
from google.oauth2 import service_account
from google.colab import drive
import json
# Construct a BigQuery client object.
drive.mount('/content/drive/') # Mount to google drive
# Define full path from Google Drive.
# This example, key is in /MyDrive/Auth/
key_path = '/content/drive/MyDrive/Auth/your_key.json'
credentials = service_account.Credentials.from_service_account_file(
filename=key_path, scopes=["https://www.googleapis.com/auth/cloud-platform"],
)
client = bigquery.Client(credentials=credentials, project=credentials.project_id,)
query = """
SELECT name, SUM(number) as total_people
FROM `bigquery-public-data.usa_names.usa_1910_2013`
WHERE state = 'TX'
GROUP BY name, state
ORDER BY total_people DESC
LIMIT 20
"""
query_job = client.query(query) # Make an API request.
print("The query data:")
for row in query_job:
# Row values can be accessed by field name or index.
print("name={}, count={}".format(row[0], row["total_people"]))
输出:
发布于 2022-01-20 13:57:19
我问了这个问题,我认为Ricco D的解决方案可以很好地解决我的问题。
不过,我查看了谷歌官方文档,发现它提供了从BigQuery提取数据的几种方法:
实例和参数设置(参见这里 )
https://stackoverflow.com/questions/70778399
复制