가져 오기 중 imposm 중지

Question

openstreetmaps를 포함하려는 응용 프로그램을 개발하려고합니다. 나는 이것을 위해 맵 박스를 사용하기로 결정했다. osm.pbf 파일에서 데이터를 가져 오려고합니다. 멀티 프로세서 시스템은 많은 작업을 수행 할 수 있지만 가져 오기 중간에 작업을 중단하는 것처럼 보입니다. 기다려야합니까, 아니면 오류가 있습니까?

ubuntu@ip-xx-xx-xx-xx:~/Documents/mapbox-osm-bright-5f237ac/shp$ sudo imposm -U postgres -d osm -m /home/ubuntu/Documents/mapbox-osm-bright-5f237ac/imposm-mapping.py --read --write --optimize --concurrency 10 --cache-dir /mnt/cache --overwrite-cache --deploy-production-tables /mnt/cc/sweden-latest.osm.pbf 
loading /home/ubuntu/Documents/mapbox-osm-bright-5f237ac/imposm-mapping.py as mapping 
password for postgres at localhost: 
[11:30:54] ## reading /mnt/cc/sweden-latest.osm.pbf 
[11:30:54] coords: 17545k nodes: 350k ways: 2506k relations: 17k (estimated) 
[11:32:02] coords: 26272k nodes: 101k ways: 28k relations: 0k 

는 그럼, 어떤 아이디어가 지금은 2-3 %의 작업 시간이 오래

^CProcess CacheWriterProcess-3:: 101k ways: 28k relations: 0k 
Process CacheWriterProcess-5: 
Traceback (most recent call last): 
Traceback (most recent call last): 
    File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
Process ParserProgress-1: 
Traceback (most recent call last): 
    self.run() 
    self.run() 
    File "/usr/local/lib/python2.7/dist-packages/imposm/reader.py", line 123, in run 
    File "/usr/local/lib/python2.7/dist-packages/imposm/reader.py", line 123, in run 
    File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    data = self.queue.get() 
    data = self.queue.get() 
    File "/usr/lib/python2.7/multiprocessing/queues.py", line 117, in get 
    File "/usr/lib/python2.7/multiprocessing/queues.py", line 117, in get 
    res = self._recv() 
    res = self._recv() 
KeyboardInterrupt 
KeyboardInterrupt 
    self.run() 
    File "/usr/local/lib/python2.7/dist-packages/imposm/util/__init__.py", line 54, in run 
    log_statement = self.queue.get() 
    File "/usr/lib/python2.7/multiprocessing/queues.py", line 117, in get 
Process CacheWriterProcess-2: 
Traceback (most recent call last): 
    File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    res = self._recv() 
KeyboardInterrupt 
    self.run() 
    File "/usr/local/lib/python2.7/dist-packages/imposm/reader.py", line 123, in run 
    data = self.queue.get() 
    File "/usr/lib/python2.7/multiprocessing/queues.py", line 117, in get 
    res = self._recv() 
KeyboardInterrupt 
Process CacheWriterProcess-4: 
Traceback (most recent call last): 
    File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    self.run() 
    File "/usr/local/lib/python2.7/dist-packages/imposm/reader.py", line 123, in run 
Traceback (most recent call last): 
    File "/usr/local/bin/imposm", line 9, in <module> 
    data = self.queue.get() 
    File "/usr/lib/python2.7/multiprocessing/queues.py", line 117, in get 
    load_entry_point('imposm==2.5.0', 'console_scripts', 'imposm')() 
    File "/usr/local/lib/python2.7/dist-packages/imposm/app.py", line 253, in main 
    res = self._recv() 
KeyboardInterrupt 
    reader.read(arg) 
    File "/usr/local/lib/python2.7/dist-packages/imposm/reader.py", line 88, in read 
    parser.parse(filename) 
    File "/usr/local/lib/python2.7/dist-packages/imposm/parser/simple.py", line 62, in parse 
    return self.parse_pbf_file(filename) 
    File "/usr/local/lib/python2.7/dist-packages/imposm/parser/simple.py", line 73, in parse_pbf_file 
    return self._parse(filename, PBFMultiProcParser) 
    File "/usr/local/lib/python2.7/dist-packages/imposm/parser/simple.py", line 140, in _parse 
    time.sleep(0.001) 
KeyboardInterrupt 
Process Process-6: 
Traceback (most recent call last): 
    File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    self.run() 
    File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/usr/local/lib/python2.7/dist-packages/imposm/parser/simple.py", line 113, in parse_it 
    parser.parse(input) 
    File "/usr/local/lib/python2.7/dist-packages/imposm/parser/pbf/multiproc.py", line 70, in parse 
    pos_queue.put(pos) 
    File "/usr/lib/python2.7/multiprocessing/queues.py", line 311, in put 
    if not self._sem.acquire(block, timeout): 
KeyboardInterrupt 

후 Ctrl-C를을 누르고 핀란드어로 보인 적이?

Answer 1

이는 imposm의 버그입니다. 패치를 설치하려면 그것을 해결하기 위해 :

pip install https://github.com/omniscale/imposm-parser/tarball/master 

Answer 2

는 그것은 포장되지 OSM-XML 파일을 다운로드하여 우회 할 수 있습니다.

Imposm에는 특정 경우에 PBF 형식을 읽는 데 문제가 있습니다. 그러나 패키지되지 않은 OSM-XML 파일이 트릭을합니다.

덴마크에서는 pbf-file (166메가바이트) 대 osm-xml-file (2백61메가바이트, 감압 4.4 GB)

root@drake:/mnt/disk/maps# imposm -U postgres -d osm -m /usr/local/share/maps/style/osm-bright-master/imposm-mapping.py --cache-dir cache --read --write --optimize --deploy-production-tables denmark-latest.osm --overwrite-cache 
Enabling Shapely speedups. 
loading /usr/local/share/maps/style/osm-bright-master/imposm-mapping.py as mapping 

[14:53:01] ## reading denmark-latest.osm 
[14:53:01] coords: 22391k nodes: 447k ways: 3198k relations: 22k (estimated) 
[14:58:38] coords: 15171k nodes: 33k ways: 1587k relations: 6k 
[14:58:39] reading took 5 m 37s 
[14:58:39] ## dropping/creating tables 
[14:58:39] ## writing data 
[14:58:47] relations: 6k/6k 
[14:58:51] relations: total time 12s for 6783 (562/s) 
[15:08:18] ways: 1587k/1587k 
[15:08:19] ways: total time 9 m 28s for 1587006 (2793/s) 
[15:08:27] nodes: 32k/33k 
[15:08:28] nodes: total time 8 s for 33279 (3805/s) 
[15:08:28] ## creating generalized tables 
[15:08:57] generalizing tables took 28s 
[15:08:57] ## creating union views 
[15:08:57] creating views took 0 s 
[15:08:57] ## creating geometry indexes 
[15:09:18] creating indexes took 21s 
[15:09:18] writing took 10m 39s 
[15:09:18] ## optimizing tables 
Clustering table osm_new_mainroads 
Clustering table osm_new_buildings 
Clustering table osm_new_mainroads_gen1 
Clustering table osm_new_mainroads_gen0 
Clustering table osm_new_amenities 
Clustering table osm_new_waterareas_gen1 
Clustering table osm_new_waterareas_gen0 
Clustering table osm_new_motorways_gen0 
Clustering table osm_new_aeroways 
Clustering table osm_new_motorways 
Clustering table osm_new_transport_points 
Clustering table osm_new_railways_gen0 
Clustering table osm_new_railways_gen1 
Clustering table osm_new_landusages 
Clustering table osm_new_waterways 
Clustering table osm_new_railways 
Clustering table osm_new_motorways_gen1 
Clustering table osm_new_waterareas 
Clustering table osm_new_places 
Clustering table osm_new_admin 
Clustering table osm_new_barrierways 
Clustering table osm_new_minorroads 
Clustering table osm_new_barrierpoints 
Clustering table osm_new_landusages_gen1 
Clustering table osm_new_landusages_gen0 
Vacuum analyze 
[15:09:53] optimizing took 35s 
[15:09:53] imposm took 16m 52s 

이다