Groups are the container mechanism by which HDF5 files are organized. From a Python perspective, they operate somewhat like dictionaries. In this case the “keys” are the names of group members, and the “values” are the members themselves (Group and Dataset) objects.
Group objects also contain most of the machinery which makes HDF5 useful. The File object does double duty as the HDF5 root group, and serves as your entry point into the file:
>>> f = h5py.File('foo.hdf5','w')
>>> f.name
u'/'
>>> f.keys()
[]
Names of all objects in the file are all text strings (unicode on Py2, str on Py3). These will be encoded with the HDF5-approved UTF-8 encoding before being passed to the HDF5 C library. Objects may also be retrieved using byte strings, which will be passed on to HDF5 as-is.
New groups are easy to create:
>>> grp = f.create_group("bar")
>>> grp.name
'/bar'
>>> subgrp = grp.create_group("baz")
>>> subgrp.name
'/bar/baz'
Multiple intermediate groups can also be created implicitly:
>>> grp2 = f.create_group("/some/long/path")
>>> grp2.name
'/some/long/path'
>>> grp3 = f['/some/long']
>>> grp3.name
'/some/long'
Groups implement a subset of the Python dictionary convention. They have methods like keys(), values() and support iteration. Most importantly, they support the indexing syntax, and standard exceptions:
>>> myds = subgrp["MyDS"]
>>> missing = subgrp["missing"]
KeyError: "Name doesn't exist (Symbol table: Object not found)"
Objects can be deleted from the file using the standard syntax:
>>> del subgroup["MyDataset"]
Note
When using h5py from Python 3, the keys(), values() and items() methods will return view-like objects instead of lists. These objects support containership testing and iteration, but can’t be sliced like lists.
What happens when assigning an object to a name in the group? It depends on the type of object being assigned. For NumPy arrays or other data, the default is to create an HDF5 datasets:
>>> grp["name"] = 42
>>> out = grp["name"]
>>> out
<HDF5 dataset "name": shape (), type "<i8">
When the object being stored is an existing Group or Dataset, a new link is made to the object:
>>> grp["other name"] = out
>>> grp["other name"]
<HDF5 dataset "other name": shape (), type "<i8">
Note that this is not a copy of the dataset! Like hard links in a UNIX file system, objects in an HDF5 file can be stored in multiple groups:
>>> f["other name"] == f["name"]
True
Also like a UNIX filesystem, HDF5 groups can contain “soft” or symbolic links, which contain a text path instead of a pointer to the object itself. You can easily create these in h5py by using h5py.SoftLink:
>>> myfile = h5py.File('foo.hdf5','w')
>>> group = myfile.create_group("somegroup")
>>> myfile["alias"] = h5py.SoftLink('/somegroup')
If the target is removed, they will “dangle”:
>>> del myfile['somegroup']
>>> print myfile['alias']
KeyError: 'Component not found (Symbol table: Object not found)'
New in HDF5 1.8, external links are “soft links plus”, which allow you to specify the name of the file as well as the path to the desired object. You can refer to objects in any file you wish. Use similar syntax as for soft links:
>>> myfile = h5py.File('foo.hdf5','w')
>>> myfile['ext link'] = h5py.ExternalLink("otherfile.hdf5", "/path/to/resource")
When the link is accessed, the file “otherfile.hdf5” is opened, and object at “/path/to/resource” is returned.
Since the object retrieved is in a different file, its ”.file” and ”.parent” properties will refer to objects in that file, not the file in which the link resides.
Note
Currently, you can’t access an external link if the file it points to is already open. This is related to how HDF5 manages file permissions internally.
Generally Group objects are created by opening objects in the file, or by the method Group.create_group(). Call the constructor with a GroupID instance to create a new Group bound to an existing low-level identifier.
Iterate over the names of objects directly attached to the group. Use Group.visit() or Group.visititems() for recursive access to group members.
Dict-like containership testing. name may be a relative or absolute path.
Retrieve an object. name may be a relative or absolute path, or an object or region reference. See Dict interface and links.
Create a new link, or automatically create a dataset. See Dict interface and links.
Get the names of directly attached group members. On Py2, this is a list. On Py3, it’s a set-like object. Use Group.visit() or Group.visititems() for recursive access to group members.
Get the objects contained in the group (Group and Dataset instances). Broken soft or external links show up as None. On Py2, this is a list. On Py3, it’s a collection or bag-like object.
Get (name, value) pairs for object directly attached to this group. Values for broken soft or external links show up as None. On Py2, this is a list. On Py3, it’s a set-like object.
(Py2 only) Get an iterator over key names. Exactly equivalent to iter(group). Use Group.visit() or Group.visititems() for recursive access to group members.
(Py2 only) Get an iterator over objects attached to the group. Broken soft and external links will show up as None.
(Py2 only) Get an iterator over (name, value) pairs for objects directly attached to the group. Broken soft and external link values show up as None.
Retrieve an item, or information about an item. name and default work like the standard Python dict.get.
Parameters: |
|
---|
Recursively visit all objects in this group and subgroups. You supply a callable with the signature:
callable(name) -> None or return value
name will be the name of the object relative to the current group. Return None to continue visiting until all objects are exhausted. Returning anything else will immediately stop visiting and return that value from visit:
>>> def find_foo(name):
... """ Find first object with 'foo' anywhere in the name """
... if 'foo' in name:
... return name
>>> group.visit(find_foo)
u'some/subgroup/foo'
Recursively visit all objects in this group and subgroups. Like Group.visit(), except your callable should have the signature:
callable(name, object) -> None or return value
Move an object or link in the file. If source is a hard link, this effectively renames the object. If a soft or external link, the link itself is moved.
Parameters: |
|
---|
Copy an object or group. The source and destination need not be in the same file. If the source is a Group object, by default all objects within that group will be copied recursively.
Parameters: |
|
---|
Create and return a new group in the file.
Parameters: | name (String or None) – Name of group to create. May be an absolute or relative path. Provide None to create an anonymous group, to be linked into the file later. |
---|---|
Returns: | The new Group object. |
Open a group in the file, creating it if it doesn’t exist. TypeError is raised if a conflicting object already exists. Parameters as in Group.create_group().
Create a new dataset. Options are explained in Creating datasets.
Parameters: |
|
---|
Open a dataset, creating it if it doesn’t exist.
If keyword “exact” is False (default), an existing dataset must have the same shape and a conversion-compatible dtype to be returned. If True, the shape and dtype must match exactly.
Other dataset keywords (see create_dataset) may be provided, but are only used if a new dataset is to be created.
Raises TypeError if an incompatible object already exists, or if the shape or dtype don’t match according to the above rules.
Parameters: | exact – Require shape and type to match exactly (T/F) |
---|
HDF5 Attributes for this group.
The groups’s low-level identifer; an instance of GroupID.
An HDF5 object reference pointing to this group. See Using object references.
A proxy object allowing you to interrogate region references. See Using region references.
String giving the full path to this group.
Exists only to support Group.get(). Has no state and provides no properties or methods.
Exists to allow creation of soft links in the file. See Soft links. These only serve as containers for a path; they are not related in any way to a particular file.
Parameters: | path (String) – Value of the soft link. |
---|
Value of the soft link
Like SoftLink, only they specify a filename in addition to a path. See External links.
Parameters: |
|
---|
Name of the external file
Path to the object in the external file