모듈 객체에는 속성이 없습니다. BigqueryV2 - 로컬 Apache Beam

Question

빔 제공 I/O API를 사용하여 Apache Beam과 함께 로컬에서 파이프 라인을 실행하려고합니다.

나는 가상 환경을 사용하여 내 환경을 Beam Python quickstart으로 제안했고 wordcount.py 예제를 실행할 수 있습니다. beam.Create 및 beam.ParDo으로 사용자 지정 파이프 라인을 제대로 실행할 수도 있습니다.

그러나 BigQuery I/O로 파이프 라인을 실행할 수 없습니다. 내가 뭘 잘못하고 있는지에 대한 어떤 생각?

python 스크립트는 다음과 같습니다.

import apache_beam as beam 
from apache_beam.utils.pipeline_options import PipelineOptions 
from apache_beam.io import WriteToText 


class MyDoFn(beam.DoFn): 
    def process(self, element): 
    return element 


def run(): 
    opts = { 
    'project': 'gc-project-name' 
    } 
    p = beam.Pipeline(options=PipelineOptions(**opts)) 

    input_query = "SELECT name FROM `gc-project-name.dataset_name.table_name`" 

    (p 
    | beam.io.Read(beam.io.BigQuerySource(query=input_query)) 
    | beam.ParDo(MyDoFn()) 
    | beam.io.WriteToText('output.txt') 
) 

    result = p.run() 
    result.wait_until_finish() 

if __name__ == '__main__': 
    run() 

실행할 때 다음 오류가 발생합니다.

WARNING:root:Task failed: Traceback (most recent call last): 
    File "/Users/localuser/Virtualenvs/abeam/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py", line 300, in __call__ 
result = evaluator.finish_bundle() 
    File "/Users/localuser/Virtualenvs/abeam/lib/python2.7/site-packages/apache_beam/runners/direct/transform_evaluator.py", line 208, in finish_bundle 
with self._source.reader() as reader: 
    File "/Users/localuser/Virtualenvs/abeam/lib/python2.7/site-packages/apache_beam/io/gcp/bigquery.py", line 590, in __enter__ 
self.client = BigQueryWrapper(client=self.test_bigquery_client) 
    File "/Users/localuser/Virtualenvs/abeam/lib/python2.7/site-packages/apache_beam/io/gcp/bigquery.py", line 682, in __init__ 
self.client = client or bigquery.BigqueryV2(
AttributeError: 'module' object has no attribute 'BigqueryV2' 
Traceback (most recent call last): 
    File "/Users/localuser/Virtualenvs/abeam/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py", line 300, in __call__ 
result = evaluator.finish_bundle() 
    File "/Users/localuser/Virtualenvs/abeam/lib/python2.7/site-packages/apache_beam/runners/direct/transform_evaluator.py", line 208, in finish_bundle 
with self._source.reader() as reader: 
    File "/Users/localuser/Virtualenvs/abeam/lib/python2.7/site-packages/apache_beam/io/gcp/bigquery.py", line 590, in __enter__ 
self.client = BigQueryWrapper(client=self.test_bigquery_client) 
    File "/Users/localuser/Virtualenvs/abeam/lib/python2.7/site-packages/apache_beam/io/gcp/bigquery.py", line 682, in __init__ 
self.client = client or bigquery.BigqueryV2(
AttributeError: 'module' object has no attribute 'BigqueryV2' 
WARNING:root:A task failed with exception. 
'module' object has no attribute 'BigqueryV2' 
Traceback (most recent call last): 
    File "frombigquery.py", line 54, in <module> 
run() 
    File "frombigquery.py", line 51, in run 
result.wait_until_finish() 
    File "/Users/localuser/Virtualenvs/abeam/lib/python2.7/site-packages/apache_beam/runners/direct/direct_runner.py", line 157, in wait_until_finish 
self._executor.await_completion() 
    File "/Users/localuser/Virtualenvs/abeam/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py", line 335, in await_completion 
self._executor.await_completion() 
    File "/Users/localuser/Virtualenvs/abeam/lib/python2.7/site-packages/apache_beam/runners/direct/executor.py", line 300, in __call__ 
result = evaluator.finish_bundle() 
    File "/Users/localuser/Virtualenvs/abeam/lib/python2.7/site-packages/apache_beam/runners/direct/transform_evaluator.py", line 208, in finish_bundle 
with self._source.reader() as reader: 
    File "/Users/localuser/Virtualenvs/abeam/lib/python2.7/site-packages/apache_beam/io/gcp/bigquery.py", line 590, in __enter__ 
self.client = BigQueryWrapper(client=self.test_bigquery_client) 
    File "/Users/localuser/Virtualenvs/abeam/lib/python2.7/site-packages/apache_beam/io/gcp/bigquery.py", line 682, in __init__ 
self.client = client or bigquery.BigqueryV2(
AttributeError: 'module' object has no attribute 'BigqueryV2' 

Answer 1

Apache Beam Python SDK를 설치할 때 Google Cloud Platform 관련 종속성을 사용하려면 추가 옵션을 추가해야합니다.

pip install dist/apache-beam-*.tar.gz[gcp]