fs.sshfs not threadsafe?

[dhirschfeld]

When I try to make multiple calls to fs.open in a ThreadPoolExecutor my program hangs :(

Is fs.sshfs expected to be threadsafe? Is this a known issue?

[althonos]

I expect it to be, since paramiko.SFTPClient is supposed to be as well. I tried to test against the new workers tests added by Will but unfortunately it always reaches a point where SSHFile.read hangs undefinitely, and I can't understand why yet.

[dhirschfeld]

Yeah, my app hung when I tried to use multiple threads. I've currently worked around it by using a threadlocal copy.

[abichinger]

I expect it to be, since paramiko.SFTPClient is supposed to be as well.

I think paramiko is not thread safe
This StackOverflow answer states that paramiko is not thread safe.
Searching "thread" on paramiko.org shows no results related to thread safty.

According to the PyFilesystem documentation

All Filesystems should be thread-safe

I would appreciate it if fs.sshfs was also thread safe.
I'm going to look at it, maybe I can provide a solution.

Some things I have noticed so far:

• fs.sshfs/fs/sshfs/sshfs.py

  Line 332 in 0209946

  _sftp = self._client.open_sftp()

  
  I think the underlying channel never gets closed
• Making concurrent calls on SSHFS._sftp can cause a deadlock.
  For example, the following code snipped hangs on my system

  sftp = client.open_sftp()
  
  def stat(path):
    return sftp.stat(path)
  
  with ThreadPoolExecutor(5) as e:
    futures = [e.submit(stat, "/home/root/.bashrc") for i in range(20)]
    for future in futures:
        res = future.result()
        print(res)