BigQuery: Dataset "Not Found" on table load with REST API -


i trying load csv file bigquery using python script modelled on python sample code here: https://developers.google.com/bigquery/docs/developers_guide

but i'm running following error when try load table rest api:

{'status': '200', 'content-length': '1492', 'expires': 'fri, 01 jan 1990 00:00:00 gmt', 'server': 'http upload server built on jun 14 2012 02:12:09 (1339665129)', 'etag': '"tcivyoj9qvkabuej5memf9we85w/-mxyhudjvvydxcebr8fxi6l_5rq"', 'pragma': 'no-cache', 'cache-control': 'no-cache, no-store, must-revalidate', 'date': 'fri, 06 jul 2012 22:30:55 gmt', 'content-type': 'application/json'} { "kind": "bigquery#job", "etag": "\"tcivyoj9qvkabuej5memf9we85w/-mxyhudjvvydxcebr8fxi6l_5rq\"", "id": "firespotter.com:firespotter:job_d6b99265278b4c0da9c3033acf39d6b2", "selflink": "https://www.googleapis.com/bigquery/v2/projects/firespotter.com:firespotter/jobs/job_d6b99265278b4c0da9c3033acf39d6b2", "jobreference": { "projectid": "firespotter.com:firespotter", "jobid": "job_d6b99265278b4c0da9c3033acf39d6b2" }, "configuration": { "load": { "schema": { "fields": [ { "name": "date", "type": "string" }, { "name": "time", "type": "string" }, { "name": "call_uuid", "type": "string" }, { "name": "log_level", "type": "string" }, { "name": "file_line", "type": "string" }, { "name": "message", "type": "string" } ] }, "destinationtable": { "projectid": "385479794093", "datasetid": "telephony_logs", "tableid": "table_name" }, "createdisposition": "create_if_needed", "writedisposition": "write_truncate", "encoding": "utf-8" } }, "status": { "state": "done", "errorresult": { "reason": "notfound", "message": "not found: dataset 385479794093:telephony_logs" }, "errors": [ { "reason": "notfound", "message": "not found: dataset 385479794093:telephony_logs" } ] } } 

the projectid listed in error "385479794093" not projectid pass in, it's "project number". projectid should "firespotter.com:firespotter":

{ "kind": "bigquery#datasetlist", "etag": "\"tcivyoj9qvkabuej5memf9we85w/zma8z6lkmgwziqlwh3ti2ssss4g\"", "datasets": [ { "kind": "bigquery#dataset", "id": "firespotter.com:firespotter:telephony_logs", "datasetreference": { "datasetid": "telephony_logs", "projectid": "firespotter.com:firespotter" } } ] } 

why rest api insist on supplying own incorrect projectid, when pass correct value in 3 different places? there place need pass in or set project id?

for reference, here relevant code snippet:

project = 'firespotter.com:firespotter' dataset = 'telephony_logs' flow = oauth2webserverflow( client_id='385479794093.apps.googleusercontent.com', client_secret='<a_secret_here>', scope='https://www.googleapis.com/auth/bigquery', user_agent='firespotter-upload-script/1.0') def loadtable(http, projectid, datasetid, tableid, file_path, replace=false): url = "https://www.googleapis.com/upload/bigquery/v2/projects/" + projectid + "/jobs" # create body of request, separated boundary of xxx mime_data = ('--xxx\n' + 'content-type: application/json; charset=utf-8\n' + '\n' + '{\n' + ' "projectid": "' + projectid + '",\n' + ' "configuration": {\n' + ' "load": {\n' + ' "schema": {\n' + ' "fields": [\n' + ' {"name":"date", "type":"string"},\n' + ' {"name":"time", "type":"string"},\n' + ' {"name":"call_uuid", "type":"string"},\n' + ' {"name":"log_level", "type":"string"},\n' + ' {"name":"file_line", "type":"string"},\n' + ' {"name":"message", "type":"string"}\n' + ' ]\n' + ' },\n' + ' "destinationtable": {\n' + ' "projectid": "' + projectid + '",\n' + ' "datasetid": "' + datasetid + '",\n' + ' "tableid": "' + tableid + '"\n' + ' },\n' + ' "createdisposition": "create_if_needed",\n' + ' "writedisposition": "' + ('write_truncate' if replace else 'write_append') + '",\n' + ' "encoding": "utf-8"\n' + ' }\n' + ' }\n' + '}\n' + '--xxx\n' + 'content-type: application/octet-stream\n' + '\n') # append data specified file request body f = open(file_path, 'r') header_line = f.readline() # skip header line mime_data += f.read() # signify end of body mime_data += ('--xxx--\n') headers = {'content-type': 'multipart/related; boundary=xxx'} resp, content = http.request(url, method="post", body=mime_data, headers=headers) print str(resp) + "\n" print content # --- main ---------------------------------------------- def main(argv): csv_path = args[0] # if credentials don't exist or invalid, run native client # auth flow. storage object ensure if successful # credentials written file. storage = storage('bigquery2_credentials.dat') # choose file name store credentials. credentials = storage.get() if credentials none or credentials.invalid: credentials = run(flow, storage) # create httplib2.http object handle our http requests , authorize # our credentials. http = httplib2.http() http = credentials.authorize(http) loadtable(http, project, dataset, 'table_name', csv_path, replace=true) if __name__ == '__main__': main(sys.argv) 

did set project id firespotter.com:firespotter? if dataset created before project named, there mismatch between old project id , new. there automated system updates project ids, possible hasn't run yet or having problem (i'm on vacation right now, can't check). hopefully, if retry again time work. if not, let know.


Comments

Popular posts from this blog

javascript - backbone.js Collection.add() doesn't `construct` (`initialize`) an object -

php - Get uncommon values from two or more arrays -

Adding duplicate array rows in Php -