vfs - Virtual file system for loading models from zip files

RealFS
ZipFS Opens a zip file as the root file system.
popfs
pushfs
vfs_init Call this very early in your program so that various filesystem functions will be redirected even if they are expressed as “from module import fn”

Redirect calls to open, etc. to a virtual file system.

Use this to mount a zip file as a file system, and then all subsequent calls to chdir, open, etc. will reference files in the zip file instead of the disk.

This will only work for packages which do all their I/O in python, and not those which use direct calls to the C library, so for example, it will not work with h5py. XML parsing from a zip file is also unlikely to work since expat uses the C library directly for parsing.

Usage:

# Do this before importing any other modules!  It sets up hooks for
# redirecting filesystem access even if the module imports the symbol
# directly as "from os import getcwd".
import vfs
vfs.vfs_init()
...
with vfs.ZipFS('data.zip'):
    data = np.loadtxt('file1.dat')

Filesystems available:

Calls redirected:

__builtin__.open (python 2 only)
builtins.open (python 2 and 3)
io.open (python 2 and 3)
os, nt, posix:
    chdir, getcwd, listdir
os.path, ntpath, posixpath:
    exists, isfile, isir, abspath, realpath

You can also use the file systems directly without using the vfs_init() hook or the with statement. Just call fs.chdir, etc. on the file system object.

file in python 2.x is a type as well as constructor, so a simple redirect to a replacement constructor will not work. Don’t try to support it since it is gone in python 3.

Works with numpy.loadtxt and pandas parsers.

For pandas, either need to specify engine=”python” or pass an open call to the reader; if you just pass a filename, then it will try opening it with the libc open function and fail. Could potentially monkeypatch pandas to pre-open the file.

class bumps.vfs.RealFS[source]

Bases: object

abspath(path)[source]
chdir(path)[source]
exists(path)[source]
getcwd()[source]
isdir(path)[source]
isfile(path)[source]
listdir(path=None)[source]
open(*args, **kw)[source]
py2_open(*args, **kw)[source]
realpath(path)[source]
class bumps.vfs.ZipFS(path)[source]

Bases: object

Opens a zip file as the root file system.

abspath(path)[source]
chdir(path)[source]
exists(path)[source]
isdir(s)[source]
isfile(path)[source]
listdir(path=None)[source]
open(file, mode='r', buffering=-1, encoding=None, errors=None, newline=None, **kw)[source]
py2_open(name, mode='r', buffering=-1)[source]
realpath(filename)[source]
bumps.vfs.popfs()[source]
bumps.vfs.pushfs(fs)[source]
bumps.vfs.vfs_init()[source]

Call this very early in your program so that various filesystem functions will be redirected even if they are expressed as “from module import fn”