PySFTP 연결이 작동하지만 get()이 실패 함

Question

pysftp과 연결을 설정할 수 있지만 s.get()을 사용할 수 없습니다.

연결

잘 작동 : 나는 또한 s.chdir("/path/to/target") 작업을하고 난 s.listdir()를 통해 잡아 s.isfile("/path/to/target/file.xxx")에 대한 True을 수신 할 파일을 표시 할 수 있습니다

import pysftp 
s = pysftp.Connection(host="xxx", username="xxx", password="xxx") 

.

그러나 s.get("/path/to/target/file.xxx")을 실행하면 IOError: Folder not found: C:\some\other\folder\file.xxx이 산출됩니다. 원본 /path/to/target/ (WinSCP와 같은 SFTP 클라이언트에도 표시됨)은 이제 Windows 구문이있는 다른 폴더를 가리키는 것으로 나타납니다.

역 추적 : SFTP-3 WinSCP에 다운로드에서

[...] 
. 2014-10-09 10:39:22.146 Listing directory "/path/to/target". 
> 2014-10-09 10:39:22.146 Type: SSH_FXP_OPENDIR, Size: 77, Number: 3595 
< 2014-10-09 10:39:22.146 Type: SSH_FXP_STATUS, Size: 17, Number: 3332 
. 2014-10-09 10:39:22.146 Discarding reserved response 
< 2014-10-09 10:39:22.332 Type: SSH_FXP_HANDLE, Size: 12, Number: 3595 
> 2014-10-09 10:39:22.333 Type: SSH_FXP_READDIR, Size: 12, Number: 3852 
< 2014-10-09 10:39:22.518 Type: SSH_FXP_NAME, Size: 1043, Number: 3852 
> 2014-10-09 10:39:22.650 Type: SSH_FXP_READDIR, Size: 12, Number: 4108 
< 2014-10-09 10:39:22.785 Type: SSH_FXP_STATUS, Size: 28, Number: 4108 
< 2014-10-09 10:39:22.786 Status code: 1 
> 2014-10-09 10:39:22.786 Type: SSH_FXP_CLOSE, Size: 12, Number: 4356 
. 2014-10-09 10:39:22.786 ..;D;0;2013-11-05T06:20:22.000Z;"" [0];"" [0];r-x------;0 
[...] 
. 2014-10-09 10:39:22.786 file.xxx;-;3087870;2014-10-09T00:00:13.000Z;"" [0];"" [0];r-x------;0 
[...] 
. 2014-10-09 10:39:27.310 File: '/path/to/target/file.xxx' [2014-10-09T00:00:13.000Z] [3087870] 
. 2014-10-09 10:39:27.322 Copying "/path/to/target/file.xxx" to local directory started. 
. 2014-10-09 10:39:27.322 Binary transfer mode selected. 
. 2014-10-09 10:39:27.344 Checking existence of partially transfered file. 
. 2014-10-09 10:39:27.344 Opening remote file. 
> 2014-10-09 10:39:27.344 Type: SSH_FXP_OPEN, Size: 129, Number: 4611 
< 2014-10-09 10:39:27.344 Type: SSH_FXP_STATUS, Size: 17, Number: 4356 
. 2014-10-09 10:39:27.344 Discarding reserved response 
< 2014-10-09 10:39:27.518 Type: SSH_FXP_HANDLE, Size: 12, Number: 4611 
> 2014-10-09 10:39:27.518 Type: SSH_FXP_FSTAT, Size: 16, Number: 4872 
< 2014-10-09 10:39:27.658 Type: SSH_FXP_STATUS, Size: 112, Number: 4872 
. 2014-10-09 10:39:27.770 Confirming overwriting of file. 


. 2014-10-09 10:39:27.781 Asking user: 

[...] 

> 2014-10-09 10:39:30.736 Type: SSH_FXP_READ, Size: 24, Number: 5125 
< 2014-10-09 10:39:40.496 Status code: 1 
. 2014-10-09 10:39:40.539 222 skipped SSH_FXP_WRITE, SSH_FXP_READ, SSH_FXP_DATA and SSH_FXP_STATUS packets. 
> 2014-10-09 10:39:40.539 Type: SSH_FXP_CLOSE, Size: 12, Number: 34052 
< 2014-10-09 10:39:40.539 Type: SSH_FXP_STATUS, Size: 28, Number: 33285 
< 2014-10-09 10:39:40.632 Type: SSH_FXP_STATUS, Size: 28, Number: 33541 
< 2014-10-09 10:39:40.753 Type: SSH_FXP_STATUS, Size: 28, Number: 33797 
. 2014-10-09 10:39:40.910 Preserving timestamp [2014-10-09T00:00:13.000Z] 
. 2014-10-09 10:41:10.393 Closing connection. 
. 2014-10-09 10:41:10.393 Sending special code: 12 
. 2014-10-09 10:41:10.393 Sent EOF message 

로그 : 성공적으로 WinSCP와 다운로드에서

IOError         Traceback (most recent call last) 
<ipython-input-31-9d6fc4a6dd9d> in <module>() 
----> 1 s.get(r'/path/to/target/file.xxx') 

C:\Miniconda3\envs\py\lib\site-packages\pysftp.pyc in get(self, remotepath, localpath, callback, preserve_mtime) 
    231    sftpattrs = self._sftp.stat(remotepath) 
    232 
--> 233   self._sftp.get(remotepath, localpath, callback=callback) 
    234   if preserve_mtime: 
    235    os.utime(localpath, (sftpattrs.st_atime, sftpattrs.st_mtime)) 

C:\Miniconda3\envs\py\lib\site-packages\paramiko\sftp_client.pyc in get(self, remotepath, localpath, callback) 
    718   file_size = self.stat(remotepath).st_size 
    719   with open(localpath, 'wb') as fl: 
--> 720    size = self.getfo(remotepath, fl, callback) 
    721   s = os.stat(localpath) 
    722   if s.st_size != size: 

C:\Miniconda3\envs\py\lib\site-packages\paramiko\sftp_client.pyc in getfo(self, remotepath, fl, callback) 
    688   with self.open(remotepath, 'rb') as fr: 
    689    file_size = self.stat(remotepath).st_size 
--> 690    fr.prefetch() 
    691    size = 0 
    692    while True: 

C:\Miniconda3\envs\py\lib\site-packages\paramiko\sftp_file.pyc in prefetch(self) 
    394   .. versionadded:: 1.5.1 
    395   """ 
--> 396   size = self.stat().st_size 
    397   # queue up async reads for the rest of the file 
    398   chunks = [] 

C:\Miniconda3\envs\py\lib\site-packages\paramiko\sftp_file.pyc in stat(self) 
    237   :return: an `.SFTPAttributes` object containing attributes about this file. 
    238   """ 
--> 239   t, msg = self.sftp._request(CMD_FSTAT, self.handle) 
    240   if t != CMD_ATTRS: 
    241    raise SFTPError('Expected attributes') 

C:\Miniconda3\envs\py\lib\site-packages\paramiko\sftp_client.pyc in _request(self, t, *arg) 
    727  def _request(self, t, *arg): 
    728   num = self._async_request(type(None), t, *arg) 
--> 729   return self._read_response(num) 
    730 
    731  def _async_request(self, fileobj, t, *arg): 

C:\Miniconda3\envs\py\lib\site-packages\paramiko\sftp_client.pyc in _read_response(self, waitfor) 
    774     # synchronous 
    775     if t == CMD_STATUS: 
--> 776      self._convert_status(msg) 
    777     return t, msg 
    778    if fileobj is not type(None): 

C:\Miniconda3\envs\py\lib\site-packages\paramiko\sftp_client.pyc in _convert_status(self, msg) 
    804    raise IOError(errno.EACCES, text) 
    805   else: 
--> 806    raise IOError(text) 
    807 
    808  def _adjust_cwd(self, path): 
IOError: Folder not found: C:\some\other\folder\file.xxx 

로그인

. 2014-10-10 10:39:36.882 -------------------------------------------------------------------------- 
. 2014-10-10 10:39:36.883 WinSCP Version 5.5.1 (Build 3970) [...] 
[...] 
. 2014-10-10 10:39:36.884 SFTP Bugs: A,A 
. 2014-10-10 10:39:36.884 SFTP Server: default 
. 2014-10-10 10:39:36.884 Local directory: C:\Users, Remote directory: /path/to/target/somefolder, Update: Yes, Cache: Yes 
. 2014-10-10 10:39:36.884 Cache directory changes: Yes, Permanent: Yes 
. 2014-10-10 10:39:36.884 DST mode: 1; Timezone offset: 0h 0m 
. 2014-10-10 10:39:36.884 -------------------------------------------------------------------------- 
[...] 
. 2014-10-10 10:39:51.049 -------------------------------------------------------------------------- 
. 2014-10-10 10:39:51.050 Using SFTP protocol. 
. 2014-10-10 10:39:51.050 Doing startup conversation with host. 
> 2014-10-10 10:39:51.104 Type: SSH_FXP_INIT, Size: 5, Number: -1 
< 2014-10-10 10:39:51.256 Type: SSH_FXP_VERSION, Size: 5, Number: -1 
. 2014-10-10 10:39:51.256 SFTP version 3 negotiated. 
. 2014-10-10 10:39:51.256 We believe the server has signed timestamps bug 
. 2014-10-10 10:39:51.256 We will use UTF-8 strings when appropriate 
. 2014-10-10 10:39:51.268 Changing directory to "/path/to/target/somefolder". 
. 2014-10-10 10:39:51.268 Getting real path for '/path/to/target/somefolder' 
> 2014-10-10 10:39:51.268 Type: SSH_FXP_REALPATH, Size: 41, Number: 16 
< 2014-10-10 10:39:53.424 Type: SSH_FXP_NAME, Size: 141, Number: 16 
. 2014-10-10 10:39:53.424 Real path is '/path/to/target/somefolder' 
. 2014-10-10 10:39:53.424 Trying to open directory "/path/to/target/somefolder". 
> 2014-10-10 10:39:53.424 Type: SSH_FXP_LSTAT, Size: 41, Number: 263 
< 2014-10-10 10:39:53.594 Type: SSH_FXP_ATTRS, Size: 21, Number: 263 
. 2014-10-10 10:39:53.594 Getting current directory name. 
. 2014-10-10 10:39:53.743 Listing directory "/path/to/target/somefolder". 
> 2014-10-10 10:39:53.743 Type: SSH_FXP_OPENDIR, Size: 41, Number: 523 
< 2014-10-10 10:39:53.894 Type: SSH_FXP_HANDLE, Size: 12, Number: 523 
> 2014-10-10 10:39:53.894 Type: SSH_FXP_READDIR, Size: 12, Number: 780 
< 2014-10-10 10:39:54.403 Type: SSH_FXP_NAME, Size: 16407, Number: 780 
> 2014-10-10 10:39:54.403 Type: SSH_FXP_READDIR, Size: 12, Number: 1036 
< 2014-10-10 10:39:54.915 Type: SSH_FXP_NAME, Size: 4111, Number: 1036 
> 2014-10-10 10:39:54.916 Type: SSH_FXP_READDIR, Size: 12, Number: 1292 
< 2014-10-10 10:39:55.067 Type: SSH_FXP_STATUS, Size: 28, Number: 1292 
< 2014-10-10 10:39:55.067 Status code: 1 
> 2014-10-10 10:39:55.067 Type: SSH_FXP_CLOSE, Size: 12, Number: 1540 
[...] 
. 2014-10-10 10:39:55.164 Startup conversation with host finished. 
. 2014-10-10 10:40:23.673 Cached directory change via ".." to "/path/to/target". 
. 2014-10-10 10:40:23.730 Getting current directory name. 
. 2014-10-10 10:40:23.730 Listing directory "/path/to/target". 
> 2014-10-10 10:40:23.730 Type: SSH_FXP_OPENDIR, Size: 32, Number: 1803 
< 2014-10-10 10:40:23.730 Type: SSH_FXP_STATUS, Size: 17, Number: 1540 
. 2014-10-10 10:40:23.731 Discarding reserved response 
< 2014-10-10 10:40:23.927 Type: SSH_FXP_HANDLE, Size: 12, Number: 1803 
> 2014-10-10 10:40:24.022 Type: SSH_FXP_READDIR, Size: 12, Number: 2060 
< 2014-10-10 10:40:26.323 Type: SSH_FXP_NAME, Size: 1247, Number: 2060 
> 2014-10-10 10:40:26.381 Type: SSH_FXP_READDIR, Size: 12, Number: 2316 
< 2014-10-10 10:40:26.531 Type: SSH_FXP_STATUS, Size: 28, Number: 2316 
< 2014-10-10 10:40:26.531 Status code: 1 
> 2014-10-10 10:40:26.531 Type: SSH_FXP_CLOSE, Size: 12, Number: 2564 
[...] 
. 2014-10-10 10:40:47.716 File: '/path/to/target/otherfolder/file.xxx' [2014-10-10T00:00:10.000Z] [3087870] 
. 2014-10-10 10:40:47.742 Copying "/path/to/target/otherfolder/file.xxx" to local directory started. 
. 2014-10-10 10:40:47.742 Binary transfer mode selected. 
. 2014-10-10 10:40:47.774 Checking existence of partially transfered file. 
. 2014-10-10 10:40:47.774 Opening remote file. 
> 2014-10-10 10:40:47.774 Type: SSH_FXP_OPEN, Size: 128, Number: 3843 
< 2014-10-10 10:40:47.774 Type: SSH_FXP_STATUS, Size: 17, Number: 3588 
. 2014-10-10 10:40:47.774 Discarding reserved response 
< 2014-10-10 10:40:47.993 Type: SSH_FXP_HANDLE, Size: 12, Number: 3843 
> 2014-10-10 10:40:48.179 Type: SSH_FXP_FSTAT, Size: 12, Number: 4104 
< 2014-10-10 10:40:48.335 Type: SSH_FXP_STATUS, Size: 112, Number: 4104 
. 2014-10-10 10:40:48.336 Confirming overwriting of file. 
. 2014-10-10 10:40:48.446 Asking user: 
[...] 
< 2014-10-10 10:41:10.204 Status code: 1 
. 2014-10-10 10:41:10.241 222 skipped SSH_FXP_WRITE, SSH_FXP_READ, SSH_FXP_DATA and SSH_FXP_STATUS packets. 
> 2014-10-10 10:41:10.241 Type: SSH_FXP_CLOSE, Size: 12, Number: 33284 
< 2014-10-10 10:41:10.347 Type: SSH_FXP_STATUS, Size: 28, Number: 32517 
< 2014-10-10 10:41:10.347 Type: SSH_FXP_STATUS, Size: 28, Number: 32773 
< 2014-10-10 10:41:10.669 Type: SSH_FXP_STATUS, Size: 28, Number: 33029 
. 2014-10-10 10:41:10.812 Preserving timestamp [2014-10-10T00:00:10.000Z] 
. 2014-10-10 10:42:26.145 Closing connection. 
. 2014-10-10 10:42:26.195 Sending special code: 12 
. 2014-10-10 10:42:26.196 Sent EOF message 

어떤 아이디어?

Answer 1

편집 : 나는 paramiko 코드를 원숭이 패치에 의존하는 더 높은 성능의 해결 방법을 발견했습니다. 다시 말하지만, 이것은 큰 수정은 아니지만, 필요한 것을 얻을 수 있습니다. 이 문제를 좀 더 일반적인 방법으로 수정 한 다음 paramiko 소스로 다시 전달하는 방법을 생각하려고합니다.

코드를 표시하기 전에 paramiko 라이브러리를 원숭이 패치해야하므로 SFTPFile.stat을 호출 할 때마다 fake_stat로 패치 된 마지막 파일 이름에 대한 통계 결과가 항상 반환됩니다. 만 사용 본

는 당신은 한 번에 하나의 파일을 다운로드하는 간단한 스크립트 작동하는 것 같다 동안

을하고있는 것을 알게 된 경우, 나는 더 이상 복잡한 사용 사례 위해 작동 할 것을 보장하지 않습니다.

import paramiko.sftp_file 

with pysftp.Connection(host, username=self.user, password=self.password) as sftp: 
     try: 
      with sftp.cd(location): 
       def fake_stat(sftpFile):                                      
        return sftpFile.sftp.stat(latest_file.filename)                                

       paramiko.sftp_file.SFTPFile.stat = fake_stat                                 

       sftp.get(filename, out_filename)                             
      return self.filename  
     except Exception as e: 
      logging.fatal(e.message) 
      raise 

---

나는 마틴의 분석을 확인할 수 있습니다 : 그것은 오류가 발생 paramiko를 통해서만 기본 스탯 호출입니다. 내가 찾은 한 가지 해결 방법은 수동으로 직접 복사본을 수행하는 것이 었습니다. 물론 이것은 프리 페치 호출이 각 블록의 왕복 대기 시간을 줄이는 데 정말로 도움이되므로 실제로 느린 단점이 있습니다.

MB = 1024 
BLOCK_SIZE = 4 * MB 


with pysftp.Connection(host, username=self.user, password=self.password) as sftp: 
     try: 
      with sftp.cd(location): 
       with sftp.open(remote_filename) as remote, open(local_filename), 'w') as local: 
        result = chardet.detect(remote.read(BLOCK_SIZE)) 
        encoding = result['encoding'] 
        remote.seek(0) 

        if encoding == 'ascii': 
         # Ascii is the most gimped encoding there is.                              
         # Latin1 is strictly better, because at least its byte-for-byte.                         
         # Apologies to purists.                                    
         print 'using encoding latin1' 
         encoding = 'latin1' 

        while True: 
         buf = remote.read(BLOCK_SIZE) 
         if not buf: 
          break 

         line = codecs.decode(buf, encoding, 'replace') 
         local.write(codecs.encode(line, 'utf-8')) 

         if len(buf) < BLOCK_SIZE: 
          break 
     except Exception as e: 
      logging.fatal(e.message) 
      raise