Indexing and selecting data¶
Xarray offers extremely flexible indexing routines that combine the best features of NumPy and pandas for data selection.
The most basic way to access elements of a DataArray
object is to use Python’s []
syntax, such as array[i, j]
, where
i
and j
are both integers.
As xarray objects can store coordinates corresponding to each dimension of an
array, label-based indexing similar to pandas.DataFrame.loc
is also possible.
In label-based indexing, the element position i
is automatically
looked-up from the coordinate values.
Dimensions of xarray objects have names, so you can also lookup the dimensions by name, instead of remembering their positional order.
Quick overview¶
In total, xarray supports four different kinds of indexing, as described below and summarized in this table:
Dimension lookup |
Index lookup |
|
|
---|---|---|---|
Positional |
By integer |
|
not available |
Positional |
By label |
|
not available |
By name |
By integer |
|
|
By name |
By label |
|
|
More advanced indexing is also possible for all the methods by
supplying DataArray
objects as indexer.
See Vectorized Indexing for the details.
Positional indexing¶
Indexing a DataArray
directly works (mostly) just like it
does for numpy arrays, except that the returned object is always another
DataArray:
In [1]: da = xr.DataArray(
...: np.random.rand(4, 3),
...: [
...: ("time", pd.date_range("2000-01-01", periods=4)),
...: ("space", ["IA", "IL", "IN"]),
...: ],
...: )
...:
In [2]: da[:2]
Out[2]:
<xarray.DataArray (time: 2, space: 3)>
array([[0.127, 0.967, 0.26 ],
[0.897, 0.377, 0.336]])
Coordinates:
* time (time) datetime64[ns] 2000-01-01 2000-01-02
* space (space) <U2 'IA' 'IL' 'IN'
In [3]: da[0, 0]
Out[3]:
<xarray.DataArray ()>
array(0.127)
Coordinates:
time datetime64[ns] 2000-01-01
space <U2 'IA'
In [4]: da[:, [2, 1]]
Out[4]:
<xarray.DataArray (time: 4, space: 2)>
array([[0.26 , 0.967],
[0.336, 0.377],
[0.123, 0.84 ],
[0.448, 0.373]])
Coordinates:
* time (time) datetime64[ns] 2000-01-01 2000-01-02 2000-01-03 2000-01-04
* space (space) <U2 'IN' 'IL'
Attributes are persisted in all indexing operations.
Warning
Positional indexing deviates from the NumPy when indexing with multiple
arrays like da[[0, 1], [0, 1]]
, as described in
Vectorized Indexing.
Xarray also supports label-based indexing, just like pandas. Because
we use a pandas.Index
under the hood, label based indexing is very
fast. To do label based indexing, use the loc
attribute:
In [5]: da.loc["2000-01-01":"2000-01-02", "IA"]
Out[5]:
<xarray.DataArray (time: 2)>
array([0.127, 0.897])
Coordinates:
* time (time) datetime64[ns] 2000-01-01 2000-01-02
space <U2 'IA'
In this example, the selected is a subpart of the array in the range ‘2000-01-01’:’2000-01-02’ along the first coordinate time and with ‘IA’ value from the second coordinate space.
You can perform any of the label indexing operations supported by pandas, including indexing with individual, slices and lists/arrays of labels, as well as indexing with boolean arrays. Like pandas, label based indexing in xarray is inclusive of both the start and stop bounds.
Setting values with label based indexing is also supported:
In [6]: da.loc["2000-01-01", ["IL", "IN"]] = -10
In [7]: da
Out[7]:
<xarray.DataArray (time: 4, space: 3)>
array([[ 0.127, -10. , -10. ],
[ 0.897, 0.377, 0.336],
[ 0.451, 0.84 , 0.123],
[ 0.543, 0.373, 0.448]])
Coordinates:
* time (time) datetime64[ns] 2000-01-01 2000-01-02 2000-01-03 2000-01-04
* space (space) <U2 'IA' 'IL' 'IN'
Indexing with dimension names¶
With the dimension names, we do not have to rely on dimension order and can use them explicitly to slice data. There are two ways to do this:
Use the
sel()
andisel()
convenience methods:# index by integer array indices In [8]: da.isel(space=0, time=slice(None, 2)) Out[8]: <xarray.DataArray (time: 2)> array([0.127, 0.897]) Coordinates: * time (time) datetime64[ns] 2000-01-01 2000-01-02 space <U2 'IA' # index by dimension coordinate labels In [9]: da.sel(time=slice("2000-01-01", "2000-01-02")) Out[9]: <xarray.DataArray (time: 2, space: 3)> array([[ 0.127, -10. , -10. ], [ 0.897, 0.377, 0.336]]) Coordinates: * time (time) datetime64[ns] 2000-01-01 2000-01-02 * space (space) <U2 'IA' 'IL' 'IN'
Use a dictionary as the argument for array positional or label based array indexing:
# index by integer array indices In [10]: da[dict(space=0, time=slice(None, 2))] Out[10]: <xarray.DataArray (time: 2)> array([0.127, 0.897]) Coordinates: * time (time) datetime64[ns] 2000-01-01 2000-01-02 space <U2 'IA' # index by dimension coordinate labels In [11]: da.loc[dict(time=slice("2000-01-01", "2000-01-02"))] Out[11]: <xarray.DataArray (time: 2, space: 3)> array([[ 0.127, -10. , -10. ], [ 0.897, 0.377, 0.336]]) Coordinates: * time (time) datetime64[ns] 2000-01-01 2000-01-02 * space (space) <U2 'IA' 'IL' 'IN'
The arguments to these methods can be any objects that could index the array
along the dimension given by the keyword, e.g., labels for an individual value,
Python slice
objects or 1-dimensional arrays.
Note
We would love to be able to do indexing with labeled dimension names inside
brackets, but unfortunately, Python does yet not support indexing with
keyword arguments like da[space=0]
Nearest neighbor lookups¶
The label based selection methods sel()
,
reindex()
and reindex_like()
all
support method
and tolerance
keyword argument. The method parameter allows for
enabling nearest neighbor (inexact) lookups by use of the methods 'pad'
,
'backfill'
or 'nearest'
:
In [12]: da = xr.DataArray([1, 2, 3], [("x", [0, 1, 2])])
In [13]: da.sel(x=[1.1, 1.9], method="nearest")
Out[13]:
<xarray.DataArray (x: 2)>
array([2, 3])
Coordinates:
* x (x) int64 1 2
In [14]: da.sel(x=0.1, method="backfill")
Out[14]:
<xarray.DataArray ()>
array(2)
Coordinates:
x int64 1
In [15]: da.reindex(x=[0.5, 1, 1.5, 2, 2.5], method="pad")
Out[15]:
<xarray.DataArray (x: 5)>
array([1, 2, 2, 3, 3])
Coordinates:
* x (x) float64 0.5 1.0 1.5 2.0 2.5
Tolerance limits the maximum distance for valid matches with an inexact lookup:
In [16]: da.reindex(x=[1.1, 1.5], method="nearest", tolerance=0.2)
Out[16]:
<xarray.DataArray (x: 2)>
array([ 2., nan])
Coordinates:
* x (x) float64 1.1 1.5
The method parameter is not yet supported if any of the arguments
to .sel()
is a slice
object:
In [17]: da.sel(x=slice(1, 3), method="nearest")
NotImplementedError
However, you don’t need to use method
to do inexact slicing. Slicing
already returns all values inside the range (inclusive), as long as the index
labels are monotonic increasing:
In [18]: da.sel(x=slice(0.9, 3.1))
Out[18]:
<xarray.DataArray (x: 2)>
array([2, 3])
Coordinates:
* x (x) int64 1 2
Indexing axes with monotonic decreasing labels also works, as long as the
slice
or .loc
arguments are also decreasing:
In [19]: reversed_da = da[::-1]
In [20]: reversed_da.loc[3.1:0.9]
Out[20]:
<xarray.DataArray (x: 2)>
array([3, 2])
Coordinates:
* x (x) int64 2 1
Note
If you want to interpolate along coordinates rather than looking up the
nearest neighbors, use interp()
and
interp_like()
.
See interpolation for the details.
Dataset indexing¶
We can also use these methods to index all variables in a dataset simultaneously, returning a new dataset:
In [21]: da = xr.DataArray(
....: np.random.rand(4, 3),
....: [
....: ("time", pd.date_range("2000-01-01", periods=4)),
....: ("space", ["IA", "IL", "IN"]),
....: ],
....: )
....:
In [22]: ds = da.to_dataset(name="foo")
In [23]: ds.isel(space=[0], time=[0])
Out[23]:
<xarray.Dataset>
Dimensions: (time: 1, space: 1)
Coordinates:
* time (time) datetime64[ns] 2000-01-01
* space (space) <U2 'IA'
Data variables:
foo (time, space) float64 0.1294
In [24]: ds.sel(time="2000-01-01")
Out[24]:
<xarray.Dataset>
Dimensions: (space: 3)
Coordinates:
time datetime64[ns] 2000-01-01
* space (space) <U2 'IA' 'IL' 'IN'
Data variables:
foo (space) float64 0.1294 0.8599 0.8204
Positional indexing on a dataset is not supported because the ordering of dimensions in a dataset is somewhat ambiguous (it can vary between different arrays). However, you can do normal indexing with dimension names:
In [25]: ds[dict(space=[0], time=[0])]
Out[25]:
<xarray.Dataset>
Dimensions: (time: 1, space: 1)
Coordinates:
* time (time) datetime64[ns] 2000-01-01
* space (space) <U2 'IA'
Data variables:
foo (time, space) float64 0.1294
In [26]: ds.loc[dict(time="2000-01-01")]
Out[26]:
<xarray.Dataset>
Dimensions: (space: 3)
Coordinates:
time datetime64[ns] 2000-01-01
* space (space) <U2 'IA' 'IL' 'IN'
Data variables:
foo (space) float64 0.1294 0.8599 0.8204
Dropping labels and dimensions¶
The drop_sel()
method returns a new object with the listed
index labels along a dimension dropped:
In [27]: ds.drop_sel(space=["IN", "IL"])
Out[27]:
<xarray.Dataset>
Dimensions: (time: 4, space: 1)
Coordinates:
* time (time) datetime64[ns] 2000-01-01 2000-01-02 2000-01-03 2000-01-04
* space (space) <U2 'IA'
Data variables:
foo (time, space) float64 0.1294 0.3521 0.5948 0.2355
drop_sel
is both a Dataset
and DataArray
method.
Use drop_dims()
to drop a full dimension from a Dataset.
Any variables with these dimensions are also dropped:
In [28]: ds.drop_dims("time")
Out[28]:
<xarray.Dataset>
Dimensions: (space: 3)
Coordinates:
* space (space) <U2 'IA' 'IL' 'IN'
Data variables:
*empty*
Masking with where
¶
Indexing methods on xarray objects generally return a subset of the original data.
However, it is sometimes useful to select an object with the same shape as the
original data, but with some elements masked. To do this type of selection in
xarray, use where()
:
In [29]: da = xr.DataArray(np.arange(16).reshape(4, 4), dims=["x", "y"])
In [30]: da.where(da.x + da.y < 4)
Out[30]:
<xarray.DataArray (x: 4, y: 4)>
array([[ 0., 1., 2., 3.],
[ 4., 5., 6., nan],
[ 8., 9., nan, nan],
[12., nan, nan, nan]])
Dimensions without coordinates: x, y
This is particularly useful for ragged indexing of multi-dimensional data,
e.g., to apply a 2D mask to an image. Note that where
follows all the
usual xarray broadcasting and alignment rules for binary operations (e.g.,
+
) between the object being indexed and the condition, as described in
Computation:
In [31]: da.where(da.y < 2)
Out[31]:
<xarray.DataArray (x: 4, y: 4)>
array([[ 0., 1., nan, nan],
[ 4., 5., nan, nan],
[ 8., 9., nan, nan],
[12., 13., nan, nan]])
Dimensions without coordinates: x, y
By default where
maintains the original size of the data. For cases
where the selected data size is much smaller than the original data,
use of the option drop=True
clips coordinate
elements that are fully masked:
In [32]: da.where(da.y < 2, drop=True)
Out[32]:
<xarray.DataArray (x: 4, y: 2)>
array([[ 0., 1.],
[ 4., 5.],
[ 8., 9.],
[12., 13.]])
Dimensions without coordinates: x, y
Selecting values with isin
¶
To check whether elements of an xarray object contain a single object, you can
compare with the equality operator ==
(e.g., arr == 3
). To check
multiple values, use isin()
:
In [33]: da = xr.DataArray([1, 2, 3, 4, 5], dims=["x"])
In [34]: da.isin([2, 4])
Out[34]:
<xarray.DataArray (x: 5)>
array([False, True, False, True, False])
Dimensions without coordinates: x
isin()
works particularly well with
where()
to support indexing by arrays that are not
already labels of an array:
In [35]: lookup = xr.DataArray([-1, -2, -3, -4, -5], dims=["x"])
In [36]: da.where(lookup.isin([-2, -4]), drop=True)
Out[36]:
<xarray.DataArray (x: 2)>
array([2., 4.])
Dimensions without coordinates: x
However, some caution is in order: when done repeatedly, this type of indexing
is significantly slower than using sel()
.
Vectorized Indexing¶
Like numpy and pandas, xarray supports indexing many array elements at once in a vectorized manner.
If you only provide integers, slices, or unlabeled arrays (array without
dimension names, such as np.ndarray
, list
, but not
DataArray()
or Variable()
) indexing can be
understood as orthogonally. Each indexer component selects independently along
the corresponding dimension, similar to how vector indexing works in Fortran or
MATLAB, or after using the numpy.ix_()
helper:
In [37]: da = xr.DataArray(
....: np.arange(12).reshape((3, 4)),
....: dims=["x", "y"],
....: coords={"x": [0, 1, 2], "y": ["a", "b", "c", "d"]},
....: )
....:
In [38]: da
Out[38]:
<xarray.DataArray (x: 3, y: 4)>
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11]])
Coordinates:
* x (x) int64 0 1 2
* y (y) <U1 'a' 'b' 'c' 'd'
In [39]: da[[0, 2, 2], [1, 3]]
Out[39]:
<xarray.DataArray (x: 3, y: 2)>
array([[ 1, 3],
[ 9, 11],
[ 9, 11]])
Coordinates:
* x (x) int64 0 2 2
* y (y) <U1 'b' 'd'
For more flexibility, you can supply DataArray()
objects
as indexers.
Dimensions on resultant arrays are given by the ordered union of the indexers’
dimensions:
In [40]: ind_x = xr.DataArray([0, 1], dims=["x"])
In [41]: ind_y = xr.DataArray([0, 1], dims=["y"])
In [42]: da[ind_x, ind_y] # orthogonal indexing
Out[42]:
<xarray.DataArray (x: 2, y: 2)>
array([[0, 1],
[4, 5]])
Coordinates:
* x (x) int64 0 1
* y (y) <U1 'a' 'b'
In [43]: da[ind_x, ind_x] # vectorized indexing
Out[43]:
<xarray.DataArray (x: 2)>
array([0, 5])
Coordinates:
* x (x) int64 0 1
y (x) <U1 'a' 'b'
Slices or sequences/arrays without named-dimensions are treated as if they have the same dimension which is indexed along:
# Because [0, 1] is used to index along dimension 'x',
# it is assumed to have dimension 'x'
In [44]: da[[0, 1], ind_x]
Out[44]:
<xarray.DataArray (x: 2)>
array([0, 5])
Coordinates:
* x (x) int64 0 1
y (x) <U1 'a' 'b'
Furthermore, you can use multi-dimensional DataArray()
as indexers, where the resultant array dimension is also determined by
indexers’ dimension:
In [45]: ind = xr.DataArray([[0, 1], [0, 1]], dims=["a", "b"])
In [46]: da[ind]
Out[46]:
<xarray.DataArray (a: 2, b: 2, y: 4)>
array([[[0, 1, 2, 3],
[4, 5, 6, 7]],
[[0, 1, 2, 3],
[4, 5, 6, 7]]])
Coordinates:
x (a, b) int64 0 1 0 1
* y (y) <U1 'a' 'b' 'c' 'd'
Dimensions without coordinates: a, b
Similar to how NumPy’s advanced indexing works, vectorized indexing for xarray is based on our broadcasting rules. See Indexing rules for the complete specification.
Vectorized indexing also works with isel
, loc
, and sel
:
In [47]: ind = xr.DataArray([[0, 1], [0, 1]], dims=["a", "b"])
In [48]: da.isel(y=ind) # same as da[:, ind]
Out[48]:
<xarray.DataArray (x: 3, a: 2, b: 2)>
array([[[0, 1],
[0, 1]],
[[4, 5],
[4, 5]],
[[8, 9],
[8, 9]]])
Coordinates:
* x (x) int64 0 1 2
y (a, b) <U1 'a' 'b' 'a' 'b'
Dimensions without coordinates: a, b
In [49]: ind = xr.DataArray([["a", "b"], ["b", "a"]], dims=["a", "b"])
In [50]: da.loc[:, ind] # same as da.sel(y=ind)
Out[50]:
<xarray.DataArray (x: 3, a: 2, b: 2)>
array([[[0, 1],
[1, 0]],
[[4, 5],
[5, 4]],
[[8, 9],
[9, 8]]])
Coordinates:
* x (x) int64 0 1 2
y (a, b) <U1 'a' 'b' 'b' 'a'
Dimensions without coordinates: a, b
These methods may also be applied to Dataset
objects
In [51]: ds = da.to_dataset(name="bar")
In [52]: ds.isel(x=xr.DataArray([0, 1, 2], dims=["points"]))
Out[52]:
<xarray.Dataset>
Dimensions: (points: 3, y: 4)
Coordinates:
x (points) int64 0 1 2
* y (y) <U1 'a' 'b' 'c' 'd'
Dimensions without coordinates: points
Data variables:
bar (points, y) int64 0 1 2 3 4 5 6 7 8 9 10 11
Vectorized indexing may be used to extract information from the nearest grid cells of interest, for example, the nearest climate model grid cells to a collection specified weather station latitudes and longitudes.
In [53]: ds = xr.tutorial.open_dataset("air_temperature")
---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/tutorial.py:131, in open_dataset(name, cache, cache_dir, engine, **kws)
130 try:
--> 131 import pooch
132 except ImportError as e:
ModuleNotFoundError: No module named 'pooch'
The above exception was the direct cause of the following exception:
ImportError Traceback (most recent call last)
Cell In [53], line 1
----> 1 ds = xr.tutorial.open_dataset("air_temperature")
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/tutorial.py:133, in open_dataset(name, cache, cache_dir, engine, **kws)
131 import pooch
132 except ImportError as e:
--> 133 raise ImportError(
134 "tutorial.open_dataset depends on pooch to download and manage datasets."
135 " To proceed please install pooch."
136 ) from e
138 logger = pooch.get_logger()
139 logger.setLevel("WARNING")
ImportError: tutorial.open_dataset depends on pooch to download and manage datasets. To proceed please install pooch.
# Define target latitude and longitude (where weather stations might be)
In [54]: target_lon = xr.DataArray([200, 201, 202, 205], dims="points")
In [55]: target_lat = xr.DataArray([31, 41, 42, 42], dims="points")
# Retrieve data at the grid cells nearest to the target latitudes and longitudes
In [56]: da = ds["air"].sel(lon=target_lon, lat=target_lat, method="nearest")
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:1336, in Dataset._construct_dataarray(self, name)
1335 try:
-> 1336 variable = self._variables[name]
1337 except KeyError:
KeyError: 'air'
During handling of the above exception, another exception occurred:
KeyError Traceback (most recent call last)
Cell In [56], line 1
----> 1 da = ds["air"].sel(lon=target_lon, lat=target_lat, method="nearest")
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:1427, in Dataset.__getitem__(self, key)
1425 return self.isel(**key)
1426 if utils.hashable(key):
-> 1427 return self._construct_dataarray(key)
1428 if utils.iterable_of_hashable(key):
1429 return self._copy_listed(key)
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:1338, in Dataset._construct_dataarray(self, name)
1336 variable = self._variables[name]
1337 except KeyError:
-> 1338 _, name, variable = _get_virtual_variable(self._variables, name, self.dims)
1340 needed_dims = set(variable.dims)
1342 coords: dict[Hashable, Variable] = {}
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:177, in _get_virtual_variable(variables, key, dim_sizes)
175 split_key = key.split(".", 1)
176 if len(split_key) != 2:
--> 177 raise KeyError(key)
179 ref_name, var_name = split_key
180 ref_var = variables[ref_name]
KeyError: 'air'
In [57]: da
Out[57]:
<xarray.DataArray (x: 3, y: 4)>
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11]])
Coordinates:
* x (x) int64 0 1 2
* y (y) <U1 'a' 'b' 'c' 'd'
Tip
If you are lazily loading your data from disk, not every form of vectorized
indexing is supported (or if supported, may not be supported efficiently).
You may find increased performance by loading your data into memory first,
e.g., with load()
.
Note
If an indexer is a DataArray()
, its coordinates should not
conflict with the selected subpart of the target array (except for the
explicitly indexed dimensions with .loc
/.sel
).
Otherwise, IndexError
will be raised.
Assigning values with indexing¶
To select and assign values to a portion of a DataArray()
you
can use indexing with .loc
:
In [58]: ds = xr.tutorial.open_dataset("air_temperature")
---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/tutorial.py:131, in open_dataset(name, cache, cache_dir, engine, **kws)
130 try:
--> 131 import pooch
132 except ImportError as e:
ModuleNotFoundError: No module named 'pooch'
The above exception was the direct cause of the following exception:
ImportError Traceback (most recent call last)
Cell In [58], line 1
----> 1 ds = xr.tutorial.open_dataset("air_temperature")
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/tutorial.py:133, in open_dataset(name, cache, cache_dir, engine, **kws)
131 import pooch
132 except ImportError as e:
--> 133 raise ImportError(
134 "tutorial.open_dataset depends on pooch to download and manage datasets."
135 " To proceed please install pooch."
136 ) from e
138 logger = pooch.get_logger()
139 logger.setLevel("WARNING")
ImportError: tutorial.open_dataset depends on pooch to download and manage datasets. To proceed please install pooch.
# add an empty 2D dataarray
In [59]: ds["empty"] = xr.full_like(ds.air.mean("time"), fill_value=0)
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
Cell In [59], line 1
----> 1 ds["empty"] = xr.full_like(ds.air.mean("time"), fill_value=0)
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/common.py:272, in AttrAccessMixin.__getattr__(self, name)
270 with suppress(KeyError):
271 return source[name]
--> 272 raise AttributeError(
273 f"{type(self).__name__!r} object has no attribute {name!r}"
274 )
AttributeError: 'Dataset' object has no attribute 'air'
# modify one grid point using loc()
In [60]: ds["empty"].loc[dict(lon=260, lat=30)] = 100
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:1336, in Dataset._construct_dataarray(self, name)
1335 try:
-> 1336 variable = self._variables[name]
1337 except KeyError:
KeyError: 'empty'
During handling of the above exception, another exception occurred:
KeyError Traceback (most recent call last)
Cell In [60], line 1
----> 1 ds["empty"].loc[dict(lon=260, lat=30)] = 100
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:1427, in Dataset.__getitem__(self, key)
1425 return self.isel(**key)
1426 if utils.hashable(key):
-> 1427 return self._construct_dataarray(key)
1428 if utils.iterable_of_hashable(key):
1429 return self._copy_listed(key)
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:1338, in Dataset._construct_dataarray(self, name)
1336 variable = self._variables[name]
1337 except KeyError:
-> 1338 _, name, variable = _get_virtual_variable(self._variables, name, self.dims)
1340 needed_dims = set(variable.dims)
1342 coords: dict[Hashable, Variable] = {}
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:177, in _get_virtual_variable(variables, key, dim_sizes)
175 split_key = key.split(".", 1)
176 if len(split_key) != 2:
--> 177 raise KeyError(key)
179 ref_name, var_name = split_key
180 ref_var = variables[ref_name]
KeyError: 'empty'
# modify a 2D region using loc()
In [61]: lc = ds.coords["lon"]
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:1336, in Dataset._construct_dataarray(self, name)
1335 try:
-> 1336 variable = self._variables[name]
1337 except KeyError:
KeyError: 'lon'
During handling of the above exception, another exception occurred:
KeyError Traceback (most recent call last)
Cell In [61], line 1
----> 1 lc = ds.coords["lon"]
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/coordinates.py:281, in DatasetCoordinates.__getitem__(self, key)
279 if key in self._data.data_vars:
280 raise KeyError(key)
--> 281 return cast("DataArray", self._data[key])
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:1427, in Dataset.__getitem__(self, key)
1425 return self.isel(**key)
1426 if utils.hashable(key):
-> 1427 return self._construct_dataarray(key)
1428 if utils.iterable_of_hashable(key):
1429 return self._copy_listed(key)
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:1338, in Dataset._construct_dataarray(self, name)
1336 variable = self._variables[name]
1337 except KeyError:
-> 1338 _, name, variable = _get_virtual_variable(self._variables, name, self.dims)
1340 needed_dims = set(variable.dims)
1342 coords: dict[Hashable, Variable] = {}
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:177, in _get_virtual_variable(variables, key, dim_sizes)
175 split_key = key.split(".", 1)
176 if len(split_key) != 2:
--> 177 raise KeyError(key)
179 ref_name, var_name = split_key
180 ref_var = variables[ref_name]
KeyError: 'lon'
In [62]: la = ds.coords["lat"]
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:1336, in Dataset._construct_dataarray(self, name)
1335 try:
-> 1336 variable = self._variables[name]
1337 except KeyError:
KeyError: 'lat'
During handling of the above exception, another exception occurred:
KeyError Traceback (most recent call last)
Cell In [62], line 1
----> 1 la = ds.coords["lat"]
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/coordinates.py:281, in DatasetCoordinates.__getitem__(self, key)
279 if key in self._data.data_vars:
280 raise KeyError(key)
--> 281 return cast("DataArray", self._data[key])
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:1427, in Dataset.__getitem__(self, key)
1425 return self.isel(**key)
1426 if utils.hashable(key):
-> 1427 return self._construct_dataarray(key)
1428 if utils.iterable_of_hashable(key):
1429 return self._copy_listed(key)
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:1338, in Dataset._construct_dataarray(self, name)
1336 variable = self._variables[name]
1337 except KeyError:
-> 1338 _, name, variable = _get_virtual_variable(self._variables, name, self.dims)
1340 needed_dims = set(variable.dims)
1342 coords: dict[Hashable, Variable] = {}
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:177, in _get_virtual_variable(variables, key, dim_sizes)
175 split_key = key.split(".", 1)
176 if len(split_key) != 2:
--> 177 raise KeyError(key)
179 ref_name, var_name = split_key
180 ref_var = variables[ref_name]
KeyError: 'lat'
In [63]: ds["empty"].loc[
....: dict(lon=lc[(lc > 220) & (lc < 260)], lat=la[(la > 20) & (la < 60)])
....: ] = 100
....:
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:1336, in Dataset._construct_dataarray(self, name)
1335 try:
-> 1336 variable = self._variables[name]
1337 except KeyError:
KeyError: 'empty'
During handling of the above exception, another exception occurred:
KeyError Traceback (most recent call last)
Cell In [63], line 1
----> 1 ds["empty"].loc[
2 dict(lon=lc[(lc > 220) & (lc < 260)], lat=la[(la > 20) & (la < 60)])
3 ] = 100
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:1427, in Dataset.__getitem__(self, key)
1425 return self.isel(**key)
1426 if utils.hashable(key):
-> 1427 return self._construct_dataarray(key)
1428 if utils.iterable_of_hashable(key):
1429 return self._copy_listed(key)
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:1338, in Dataset._construct_dataarray(self, name)
1336 variable = self._variables[name]
1337 except KeyError:
-> 1338 _, name, variable = _get_virtual_variable(self._variables, name, self.dims)
1340 needed_dims = set(variable.dims)
1342 coords: dict[Hashable, Variable] = {}
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:177, in _get_virtual_variable(variables, key, dim_sizes)
175 split_key = key.split(".", 1)
176 if len(split_key) != 2:
--> 177 raise KeyError(key)
179 ref_name, var_name = split_key
180 ref_var = variables[ref_name]
KeyError: 'empty'
or where()
:
# modify one grid point using xr.where()
In [64]: ds["empty"] = xr.where(
....: (ds.coords["lat"] == 20) & (ds.coords["lon"] == 260), 100, ds["empty"]
....: )
....:
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:1336, in Dataset._construct_dataarray(self, name)
1335 try:
-> 1336 variable = self._variables[name]
1337 except KeyError:
KeyError: 'lat'
During handling of the above exception, another exception occurred:
KeyError Traceback (most recent call last)
Cell In [64], line 2
1 ds["empty"] = xr.where(
----> 2 (ds.coords["lat"] == 20) & (ds.coords["lon"] == 260), 100, ds["empty"]
3 )
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/coordinates.py:281, in DatasetCoordinates.__getitem__(self, key)
279 if key in self._data.data_vars:
280 raise KeyError(key)
--> 281 return cast("DataArray", self._data[key])
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:1427, in Dataset.__getitem__(self, key)
1425 return self.isel(**key)
1426 if utils.hashable(key):
-> 1427 return self._construct_dataarray(key)
1428 if utils.iterable_of_hashable(key):
1429 return self._copy_listed(key)
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:1338, in Dataset._construct_dataarray(self, name)
1336 variable = self._variables[name]
1337 except KeyError:
-> 1338 _, name, variable = _get_virtual_variable(self._variables, name, self.dims)
1340 needed_dims = set(variable.dims)
1342 coords: dict[Hashable, Variable] = {}
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:177, in _get_virtual_variable(variables, key, dim_sizes)
175 split_key = key.split(".", 1)
176 if len(split_key) != 2:
--> 177 raise KeyError(key)
179 ref_name, var_name = split_key
180 ref_var = variables[ref_name]
KeyError: 'lat'
# or modify a 2D region using xr.where()
In [65]: mask = (
....: (ds.coords["lat"] > 20)
....: & (ds.coords["lat"] < 60)
....: & (ds.coords["lon"] > 220)
....: & (ds.coords["lon"] < 260)
....: )
....:
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:1336, in Dataset._construct_dataarray(self, name)
1335 try:
-> 1336 variable = self._variables[name]
1337 except KeyError:
KeyError: 'lat'
During handling of the above exception, another exception occurred:
KeyError Traceback (most recent call last)
Cell In [65], line 2
1 mask = (
----> 2 (ds.coords["lat"] > 20)
3 & (ds.coords["lat"] < 60)
4 & (ds.coords["lon"] > 220)
5 & (ds.coords["lon"] < 260)
6 )
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/coordinates.py:281, in DatasetCoordinates.__getitem__(self, key)
279 if key in self._data.data_vars:
280 raise KeyError(key)
--> 281 return cast("DataArray", self._data[key])
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:1427, in Dataset.__getitem__(self, key)
1425 return self.isel(**key)
1426 if utils.hashable(key):
-> 1427 return self._construct_dataarray(key)
1428 if utils.iterable_of_hashable(key):
1429 return self._copy_listed(key)
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:1338, in Dataset._construct_dataarray(self, name)
1336 variable = self._variables[name]
1337 except KeyError:
-> 1338 _, name, variable = _get_virtual_variable(self._variables, name, self.dims)
1340 needed_dims = set(variable.dims)
1342 coords: dict[Hashable, Variable] = {}
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:177, in _get_virtual_variable(variables, key, dim_sizes)
175 split_key = key.split(".", 1)
176 if len(split_key) != 2:
--> 177 raise KeyError(key)
179 ref_name, var_name = split_key
180 ref_var = variables[ref_name]
KeyError: 'lat'
In [66]: ds["empty"] = xr.where(mask, 100, ds["empty"])
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Cell In [66], line 1
----> 1 ds["empty"] = xr.where(mask, 100, ds["empty"])
NameError: name 'mask' is not defined
Vectorized indexing can also be used to assign values to xarray object.
In [67]: da = xr.DataArray(
....: np.arange(12).reshape((3, 4)),
....: dims=["x", "y"],
....: coords={"x": [0, 1, 2], "y": ["a", "b", "c", "d"]},
....: )
....:
In [68]: da
Out[68]:
<xarray.DataArray (x: 3, y: 4)>
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11]])
Coordinates:
* x (x) int64 0 1 2
* y (y) <U1 'a' 'b' 'c' 'd'
In [69]: da[0] = -1 # assignment with broadcasting
In [70]: da
Out[70]:
<xarray.DataArray (x: 3, y: 4)>
array([[-1, -1, -1, -1],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11]])
Coordinates:
* x (x) int64 0 1 2
* y (y) <U1 'a' 'b' 'c' 'd'
In [71]: ind_x = xr.DataArray([0, 1], dims=["x"])
In [72]: ind_y = xr.DataArray([0, 1], dims=["y"])
In [73]: da[ind_x, ind_y] = -2 # assign -2 to (ix, iy) = (0, 0) and (1, 1)
In [74]: da
Out[74]:
<xarray.DataArray (x: 3, y: 4)>
array([[-2, -2, -1, -1],
[-2, -2, 6, 7],
[ 8, 9, 10, 11]])
Coordinates:
* x (x) int64 0 1 2
* y (y) <U1 'a' 'b' 'c' 'd'
In [75]: da[ind_x, ind_y] += 100 # increment is also possible
In [76]: da
Out[76]:
<xarray.DataArray (x: 3, y: 4)>
array([[98, 98, -1, -1],
[98, 98, 6, 7],
[ 8, 9, 10, 11]])
Coordinates:
* x (x) int64 0 1 2
* y (y) <U1 'a' 'b' 'c' 'd'
Like numpy.ndarray
, value assignment sometimes works differently from what one may expect.
In [77]: da = xr.DataArray([0, 1, 2, 3], dims=["x"])
In [78]: ind = xr.DataArray([0, 0, 0], dims=["x"])
In [79]: da[ind] -= 1
In [80]: da
Out[80]:
<xarray.DataArray (x: 4)>
array([-1, 1, 2, 3])
Dimensions without coordinates: x
Where the 0th element will be subtracted 1 only once.
This is because v[0] = v[0] - 1
is called three times, rather than
v[0] = v[0] - 1 - 1 - 1
.
See Assigning values to indexed arrays for the details.
Note
Dask array does not support value assignment (see Parallel computing with Dask for the details).
Note
Coordinates in both the left- and right-hand-side arrays should not
conflict with each other.
Otherwise, IndexError
will be raised.
Warning
Do not try to assign values when using any of the indexing methods isel
or sel
:
# DO NOT do this
da.isel(space=0) = 0
Instead, values can be assigned using dictionary-based indexing:
da[dict(space=0)] = 0
Assigning values with the chained indexing using .sel
or .isel
fails silently.
In [81]: da = xr.DataArray([0, 1, 2, 3], dims=["x"])
# DO NOT do this
In [82]: da.isel(x=[0, 1, 2])[1] = -1
In [83]: da
Out[83]:
<xarray.DataArray (x: 4)>
array([0, 1, 2, 3])
Dimensions without coordinates: x
You can also assign values to all variables of a Dataset
at once:
In [84]: ds_org = xr.tutorial.open_dataset("eraint_uvz").isel(
....: latitude=slice(56, 59), longitude=slice(255, 258), level=0
....: )
....:
---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/tutorial.py:131, in open_dataset(name, cache, cache_dir, engine, **kws)
130 try:
--> 131 import pooch
132 except ImportError as e:
ModuleNotFoundError: No module named 'pooch'
The above exception was the direct cause of the following exception:
ImportError Traceback (most recent call last)
Cell In [84], line 1
----> 1 ds_org = xr.tutorial.open_dataset("eraint_uvz").isel(
2 latitude=slice(56, 59), longitude=slice(255, 258), level=0
3 )
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/tutorial.py:133, in open_dataset(name, cache, cache_dir, engine, **kws)
131 import pooch
132 except ImportError as e:
--> 133 raise ImportError(
134 "tutorial.open_dataset depends on pooch to download and manage datasets."
135 " To proceed please install pooch."
136 ) from e
138 logger = pooch.get_logger()
139 logger.setLevel("WARNING")
ImportError: tutorial.open_dataset depends on pooch to download and manage datasets. To proceed please install pooch.
# set all values to 0
In [85]: ds = xr.zeros_like(ds_org)
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Cell In [85], line 1
----> 1 ds = xr.zeros_like(ds_org)
NameError: name 'ds_org' is not defined
In [86]: ds
Out[86]:
<xarray.Dataset>
Dimensions: (x: 3, y: 4)
Coordinates:
* x (x) int64 0 1 2
* y (y) <U1 'a' 'b' 'c' 'd'
Data variables:
bar (x, y) int64 0 1 2 3 4 5 6 7 8 9 10 11
# by integer
In [87]: ds[dict(latitude=2, longitude=2)] = 1
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:1532, in Dataset._setitem_check(self, key, value)
1531 try:
-> 1532 var_k = var[key]
1533 except Exception as e:
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataarray.py:815, in DataArray.__getitem__(self, key)
813 else:
814 # xarray-style array indexing
--> 815 return self.isel(indexers=self._item_key_to_dict(key))
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataarray.py:1394, in DataArray.isel(self, indexers, drop, missing_dims, **indexers_kwargs)
1391 # Much faster algorithm for when all indexers are ints, slices, one-dimensional
1392 # lists, or zero or one-dimensional np.ndarray's
-> 1394 variable = self._variable.isel(indexers, missing_dims=missing_dims)
1395 indexes, index_variables = isel_indexes(self.xindexes, indexers)
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/variable.py:1271, in Variable.isel(self, indexers, missing_dims, **indexers_kwargs)
1269 indexers = either_dict_or_kwargs(indexers, indexers_kwargs, "isel")
-> 1271 indexers = drop_dims_from_indexers(indexers, self.dims, missing_dims)
1273 key = tuple(indexers.get(dim, slice(None)) for dim in self.dims)
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/utils.py:823, in drop_dims_from_indexers(indexers, dims, missing_dims)
822 if invalid:
--> 823 raise ValueError(
824 f"Dimensions {invalid} do not exist. Expected one or more of {dims}"
825 )
827 return indexers
ValueError: Dimensions {'longitude', 'latitude'} do not exist. Expected one or more of ('x', 'y')
The above exception was the direct cause of the following exception:
ValueError Traceback (most recent call last)
Cell In [87], line 1
----> 1 ds[dict(latitude=2, longitude=2)] = 1
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:1457, in Dataset.__setitem__(self, key, value)
1453 from .dataarray import DataArray
1455 if utils.is_dict_like(key):
1456 # check for consistency and convert value to dataset
-> 1457 value = self._setitem_check(key, value)
1458 # loop over dataset variables and set new values
1459 processed = []
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:1534, in Dataset._setitem_check(self, key, value)
1532 var_k = var[key]
1533 except Exception as e:
-> 1534 raise ValueError(
1535 f"Variable '{name}': indexer {key} not available"
1536 ) from e
1538 if isinstance(value, Dataset):
1539 val = value[name]
ValueError: Variable 'bar': indexer {'latitude': 2, 'longitude': 2} not available
In [88]: ds["u"]
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:1336, in Dataset._construct_dataarray(self, name)
1335 try:
-> 1336 variable = self._variables[name]
1337 except KeyError:
KeyError: 'u'
During handling of the above exception, another exception occurred:
KeyError Traceback (most recent call last)
Cell In [88], line 1
----> 1 ds["u"]
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:1427, in Dataset.__getitem__(self, key)
1425 return self.isel(**key)
1426 if utils.hashable(key):
-> 1427 return self._construct_dataarray(key)
1428 if utils.iterable_of_hashable(key):
1429 return self._copy_listed(key)
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:1338, in Dataset._construct_dataarray(self, name)
1336 variable = self._variables[name]
1337 except KeyError:
-> 1338 _, name, variable = _get_virtual_variable(self._variables, name, self.dims)
1340 needed_dims = set(variable.dims)
1342 coords: dict[Hashable, Variable] = {}
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:177, in _get_virtual_variable(variables, key, dim_sizes)
175 split_key = key.split(".", 1)
176 if len(split_key) != 2:
--> 177 raise KeyError(key)
179 ref_name, var_name = split_key
180 ref_var = variables[ref_name]
KeyError: 'u'
In [89]: ds["v"]
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:1336, in Dataset._construct_dataarray(self, name)
1335 try:
-> 1336 variable = self._variables[name]
1337 except KeyError:
KeyError: 'v'
During handling of the above exception, another exception occurred:
KeyError Traceback (most recent call last)
Cell In [89], line 1
----> 1 ds["v"]
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:1427, in Dataset.__getitem__(self, key)
1425 return self.isel(**key)
1426 if utils.hashable(key):
-> 1427 return self._construct_dataarray(key)
1428 if utils.iterable_of_hashable(key):
1429 return self._copy_listed(key)
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:1338, in Dataset._construct_dataarray(self, name)
1336 variable = self._variables[name]
1337 except KeyError:
-> 1338 _, name, variable = _get_virtual_variable(self._variables, name, self.dims)
1340 needed_dims = set(variable.dims)
1342 coords: dict[Hashable, Variable] = {}
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:177, in _get_virtual_variable(variables, key, dim_sizes)
175 split_key = key.split(".", 1)
176 if len(split_key) != 2:
--> 177 raise KeyError(key)
179 ref_name, var_name = split_key
180 ref_var = variables[ref_name]
KeyError: 'v'
# by label
In [90]: ds.loc[dict(latitude=47.25, longitude=[11.25, 12])] = 100
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
Cell In [90], line 1
----> 1 ds.loc[dict(latitude=47.25, longitude=[11.25, 12])] = 100
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:427, in _LocIndexer.__setitem__(self, key, value)
421 raise TypeError(
422 "can only set locations defined by dictionaries from Dataset.loc."
423 f" Got: {key}"
424 )
426 # set new values
--> 427 dim_indexers = map_index_queries(self.dataset, key).dim_indexers
428 self.dataset[dim_indexers] = value
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/indexing.py:175, in map_index_queries(obj, indexers, method, tolerance, **indexers_kwargs)
172 options = {"method": method, "tolerance": tolerance}
174 indexers = either_dict_or_kwargs(indexers, indexers_kwargs, "map_index_queries")
--> 175 grouped_indexers = group_indexers_by_index(obj, indexers, options)
177 results = []
178 for index, labels in grouped_indexers:
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/indexing.py:139, in group_indexers_by_index(obj, indexers, options)
137 raise KeyError(f"no index found for coordinate {key!r}")
138 elif key not in obj.dims:
--> 139 raise KeyError(f"{key!r} is not a valid dimension or coordinate")
140 elif len(options):
141 raise ValueError(
142 f"cannot supply selection options {options!r} for dimension {key!r}"
143 "that has no associated coordinate or index"
144 )
KeyError: "'latitude' is not a valid dimension or coordinate"
In [91]: ds["u"]
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:1336, in Dataset._construct_dataarray(self, name)
1335 try:
-> 1336 variable = self._variables[name]
1337 except KeyError:
KeyError: 'u'
During handling of the above exception, another exception occurred:
KeyError Traceback (most recent call last)
Cell In [91], line 1
----> 1 ds["u"]
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:1427, in Dataset.__getitem__(self, key)
1425 return self.isel(**key)
1426 if utils.hashable(key):
-> 1427 return self._construct_dataarray(key)
1428 if utils.iterable_of_hashable(key):
1429 return self._copy_listed(key)
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:1338, in Dataset._construct_dataarray(self, name)
1336 variable = self._variables[name]
1337 except KeyError:
-> 1338 _, name, variable = _get_virtual_variable(self._variables, name, self.dims)
1340 needed_dims = set(variable.dims)
1342 coords: dict[Hashable, Variable] = {}
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:177, in _get_virtual_variable(variables, key, dim_sizes)
175 split_key = key.split(".", 1)
176 if len(split_key) != 2:
--> 177 raise KeyError(key)
179 ref_name, var_name = split_key
180 ref_var = variables[ref_name]
KeyError: 'u'
# dataset as new values
In [92]: new_dat = ds_org.loc[dict(latitude=48, longitude=[11.25, 12])]
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Cell In [92], line 1
----> 1 new_dat = ds_org.loc[dict(latitude=48, longitude=[11.25, 12])]
NameError: name 'ds_org' is not defined
In [93]: new_dat
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Cell In [93], line 1
----> 1 new_dat
NameError: name 'new_dat' is not defined
In [94]: ds.loc[dict(latitude=47.25, longitude=[11.25, 12])] = new_dat
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Cell In [94], line 1
----> 1 ds.loc[dict(latitude=47.25, longitude=[11.25, 12])] = new_dat
NameError: name 'new_dat' is not defined
In [95]: ds["u"]
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:1336, in Dataset._construct_dataarray(self, name)
1335 try:
-> 1336 variable = self._variables[name]
1337 except KeyError:
KeyError: 'u'
During handling of the above exception, another exception occurred:
KeyError Traceback (most recent call last)
Cell In [95], line 1
----> 1 ds["u"]
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:1427, in Dataset.__getitem__(self, key)
1425 return self.isel(**key)
1426 if utils.hashable(key):
-> 1427 return self._construct_dataarray(key)
1428 if utils.iterable_of_hashable(key):
1429 return self._copy_listed(key)
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:1338, in Dataset._construct_dataarray(self, name)
1336 variable = self._variables[name]
1337 except KeyError:
-> 1338 _, name, variable = _get_virtual_variable(self._variables, name, self.dims)
1340 needed_dims = set(variable.dims)
1342 coords: dict[Hashable, Variable] = {}
File /build/python-xarray-iFXG89/python-xarray-2022.10.0/xarray/core/dataset.py:177, in _get_virtual_variable(variables, key, dim_sizes)
175 split_key = key.split(".", 1)
176 if len(split_key) != 2:
--> 177 raise KeyError(key)
179 ref_name, var_name = split_key
180 ref_var = variables[ref_name]
KeyError: 'u'
The dimensions can differ between the variables in the dataset, but all variables need to have at least the dimensions specified in the indexer dictionary.
The new values must be either a scalar, a DataArray
or a Dataset
itself that contains all variables that also appear in the dataset to be modified.
More advanced indexing¶
The use of DataArray()
objects as indexers enables very
flexible indexing. The following is an example of the pointwise indexing:
In [96]: da = xr.DataArray(np.arange(56).reshape((7, 8)), dims=["x", "y"])
In [97]: da
Out[97]:
<xarray.DataArray (x: 7, y: 8)>
array([[ 0, 1, 2, ..., 5, 6, 7],
[ 8, 9, 10, ..., 13, 14, 15],
[16, 17, 18, ..., 21, 22, 23],
...,
[32, 33, 34, ..., 37, 38, 39],
[40, 41, 42, ..., 45, 46, 47],
[48, 49, 50, ..., 53, 54, 55]])
Dimensions without coordinates: x, y
In [98]: da.isel(x=xr.DataArray([0, 1, 6], dims="z"), y=xr.DataArray([0, 1, 0], dims="z"))
Out[98]:
<xarray.DataArray (z: 3)>
array([ 0, 9, 48])
Dimensions without coordinates: z
where three elements at (ix, iy) = ((0, 0), (1, 1), (6, 0))
are selected
and mapped along a new dimension z
.
If you want to add a coordinate to the new dimension z
,
you can supply a DataArray
with a coordinate,
In [99]: da.isel(
....: x=xr.DataArray([0, 1, 6], dims="z", coords={"z": ["a", "b", "c"]}),
....: y=xr.DataArray([0, 1, 0], dims="z"),
....: )
....:
Out[99]:
<xarray.DataArray (z: 3)>
array([ 0, 9, 48])
Coordinates:
* z (z) <U1 'a' 'b' 'c'
Analogously, label-based pointwise-indexing is also possible by the .sel
method:
In [100]: da = xr.DataArray(
.....: np.random.rand(4, 3),
.....: [
.....: ("time", pd.date_range("2000-01-01", periods=4)),
.....: ("space", ["IA", "IL", "IN"]),
.....: ],
.....: )
.....:
In [101]: times = xr.DataArray(
.....: pd.to_datetime(["2000-01-03", "2000-01-02", "2000-01-01"]), dims="new_time"
.....: )
.....:
In [102]: da.sel(space=xr.DataArray(["IA", "IL", "IN"], dims=["new_time"]), time=times)
Out[102]:
<xarray.DataArray (new_time: 3)>
array([0.92, 0.34, 0.59])
Coordinates:
time (new_time) datetime64[ns] 2000-01-03 2000-01-02 2000-01-01
space (new_time) <U2 'IA' 'IL' 'IN'
* new_time (new_time) datetime64[ns] 2000-01-03 2000-01-02 2000-01-01
Align and reindex¶
Xarray’s reindex
, reindex_like
and align
impose a DataArray
or
Dataset
onto a new set of coordinates corresponding to dimensions. The
original values are subset to the index labels still found in the new labels,
and values corresponding to new labels not found in the original object are
in-filled with NaN.
Xarray operations that combine multiple objects generally automatically align their arguments to share the same indexes. However, manual alignment can be useful for greater control and for increased performance.
To reindex a particular dimension, use reindex()
:
In [103]: da.reindex(space=["IA", "CA"])
Out[103]:
<xarray.DataArray (time: 4, space: 2)>
array([[0.574, nan],
[0.245, nan],
[0.92 , nan],
[0.754, nan]])
Coordinates:
* time (time) datetime64[ns] 2000-01-01 2000-01-02 2000-01-03 2000-01-04
* space (space) <U2 'IA' 'CA'
The reindex_like()
method is a useful shortcut.
To demonstrate, we will make a subset DataArray with new values:
In [104]: foo = da.rename("foo")
In [105]: baz = (10 * da[:2, :2]).rename("baz")
In [106]: baz
Out[106]:
<xarray.DataArray 'baz' (time: 2, space: 2)>
array([[5.74 , 0.613],
[2.453, 3.404]])
Coordinates:
* time (time) datetime64[ns] 2000-01-01 2000-01-02
* space (space) <U2 'IA' 'IL'
Reindexing foo
with baz
selects out the first two values along each
dimension:
In [107]: foo.reindex_like(baz)
Out[107]:
<xarray.DataArray 'foo' (time: 2, space: 2)>
array([[0.574, 0.061],
[0.245, 0.34 ]])
Coordinates:
* time (time) datetime64[ns] 2000-01-01 2000-01-02
* space (space) <U2 'IA' 'IL'
The opposite operation asks us to reindex to a larger shape, so we fill in the missing values with NaN:
In [108]: baz.reindex_like(foo)
Out[108]:
<xarray.DataArray 'baz' (time: 4, space: 3)>
array([[5.74 , 0.613, nan],
[2.453, 3.404, nan],
[ nan, nan, nan],
[ nan, nan, nan]])
Coordinates:
* time (time) datetime64[ns] 2000-01-01 2000-01-02 2000-01-03 2000-01-04
* space (space) <U2 'IA' 'IL' 'IN'
The align()
function lets us perform more flexible database-like
'inner'
, 'outer'
, 'left'
and 'right'
joins:
In [109]: xr.align(foo, baz, join="inner")
Out[109]:
(<xarray.DataArray 'foo' (time: 2, space: 2)>
array([[0.574, 0.061],
[0.245, 0.34 ]])
Coordinates:
* time (time) datetime64[ns] 2000-01-01 2000-01-02
* space (space) <U2 'IA' 'IL',
<xarray.DataArray 'baz' (time: 2, space: 2)>
array([[5.74 , 0.613],
[2.453, 3.404]])
Coordinates:
* time (time) datetime64[ns] 2000-01-01 2000-01-02
* space (space) <U2 'IA' 'IL')
In [110]: xr.align(foo, baz, join="outer")
Out[110]:
(<xarray.DataArray 'foo' (time: 4, space: 3)>
array([[0.574, 0.061, 0.59 ],
[0.245, 0.34 , 0.985],
[0.92 , 0.038, 0.862],
[0.754, 0.405, 0.344]])
Coordinates:
* time (time) datetime64[ns] 2000-01-01 2000-01-02 2000-01-03 2000-01-04
* space (space) <U2 'IA' 'IL' 'IN',
<xarray.DataArray 'baz' (time: 4, space: 3)>
array([[5.74 , 0.613, nan],
[2.453, 3.404, nan],
[ nan, nan, nan],
[ nan, nan, nan]])
Coordinates:
* time (time) datetime64[ns] 2000-01-01 2000-01-02 2000-01-03 2000-01-04
* space (space) <U2 'IA' 'IL' 'IN')
Both reindex_like
and align
work interchangeably between
DataArray
and Dataset
objects, and with any number of matching dimension names:
In [111]: ds
Out[111]:
<xarray.Dataset>
Dimensions: (x: 3, y: 4)
Coordinates:
* x (x) int64 0 1 2
* y (y) <U1 'a' 'b' 'c' 'd'
Data variables:
bar (x, y) int64 0 1 2 3 4 5 6 7 8 9 10 11
In [112]: ds.reindex_like(baz)
Out[112]:
<xarray.Dataset>
Dimensions: (x: 3, y: 4)
Coordinates:
* x (x) int64 0 1 2
* y (y) <U1 'a' 'b' 'c' 'd'
Data variables:
bar (x, y) int64 0 1 2 3 4 5 6 7 8 9 10 11
In [113]: other = xr.DataArray(["a", "b", "c"], dims="other")
# this is a no-op, because there are no shared dimension names
In [114]: ds.reindex_like(other)
Out[114]:
<xarray.Dataset>
Dimensions: (x: 3, y: 4)
Coordinates:
* x (x) int64 0 1 2
* y (y) <U1 'a' 'b' 'c' 'd'
Data variables:
bar (x, y) int64 0 1 2 3 4 5 6 7 8 9 10 11
Missing coordinate labels¶
Coordinate labels for each dimension are optional (as of xarray v0.9). Label
based indexing with .sel
and .loc
uses standard positional,
integer-based indexing as a fallback for dimensions without a coordinate label:
In [115]: da = xr.DataArray([1, 2, 3], dims="x")
In [116]: da.sel(x=[0, -1])
Out[116]:
<xarray.DataArray (x: 2)>
array([1, 3])
Dimensions without coordinates: x
Alignment between xarray objects where one or both do not have coordinate labels succeeds only if all dimensions of the same name have the same length. Otherwise, it raises an informative error:
In [117]: xr.align(da, da[:2])
ValueError: arguments without labels along dimension 'x' cannot be aligned because they have different dimension sizes: {2, 3}
Underlying Indexes¶
Xarray uses the pandas.Index
internally to perform indexing
operations. If you need to access the underlying indexes, they are available
through the indexes
attribute.
In [118]: da = xr.DataArray(
.....: np.random.rand(4, 3),
.....: [
.....: ("time", pd.date_range("2000-01-01", periods=4)),
.....: ("space", ["IA", "IL", "IN"]),
.....: ],
.....: )
.....:
In [119]: da
Out[119]:
<xarray.DataArray (time: 4, space: 3)>
array([[0.171, 0.395, 0.642],
[0.275, 0.462, 0.871],
[0.401, 0.611, 0.118],
[0.702, 0.414, 0.342]])
Coordinates:
* time (time) datetime64[ns] 2000-01-01 2000-01-02 2000-01-03 2000-01-04
* space (space) <U2 'IA' 'IL' 'IN'
In [120]: da.indexes
Out[120]:
Indexes:
time DatetimeIndex(['2000-01-01', '2000-01-02', '2000-01-03', '2000-01-04'], dtype='datetime64[ns]', name='time', freq='D')
space Index(['IA', 'IL', 'IN'], dtype='object', name='space')
In [121]: da.indexes["time"]
Out[121]: DatetimeIndex(['2000-01-01', '2000-01-02', '2000-01-03', '2000-01-04'], dtype='datetime64[ns]', name='time', freq='D')
Use get_index()
to get an index for a dimension,
falling back to a default pandas.RangeIndex
if it has no coordinate
labels:
In [122]: da = xr.DataArray([1, 2, 3], dims="x")
In [123]: da
Out[123]:
<xarray.DataArray (x: 3)>
array([1, 2, 3])
Dimensions without coordinates: x
In [124]: da.get_index("x")
Out[124]: RangeIndex(start=0, stop=3, step=1, name='x')
Copies vs. Views¶
Whether array indexing returns a view or a copy of the underlying data depends on the nature of the labels.
For positional (integer) indexing, xarray follows the same rules as NumPy:
Positional indexing with only integers and slices returns a view.
Positional indexing with arrays or lists returns a copy.
The rules for label based indexing are more complex:
Label-based indexing with only slices returns a view.
Label-based indexing with arrays returns a copy.
Label-based indexing with scalars returns a view or a copy, depending upon if the corresponding positional indexer can be represented as an integer or a slice object. The exact rules are determined by pandas.
Whether data is a copy or a view is more predictable in xarray than in pandas, so unlike pandas, xarray does not produce SettingWithCopy warnings. However, you should still avoid assignment with chained indexing.
Multi-level indexing¶
Just like pandas, advanced indexing on multi-level indexes is possible with
loc
and sel
. You can slice a multi-index by providing multiple indexers,
i.e., a tuple of slices, labels, list of labels, or any selector allowed by
pandas:
In [125]: midx = pd.MultiIndex.from_product([list("abc"), [0, 1]], names=("one", "two"))
In [126]: mda = xr.DataArray(np.random.rand(6, 3), [("x", midx), ("y", range(3))])
In [127]: mda
Out[127]:
<xarray.DataArray (x: 6, y: 3)>
array([[0.596, 0.2 , 0.1 ],
[0.735, 0.017, 0.481],
[0.096, 0.497, 0.839],
[0.897, 0.733, 0.759],
[0.561, 0.471, 0.139],
[0.094, 0.942, 0.134]])
Coordinates:
* x (x) object MultiIndex
* one (x) object 'a' 'a' 'b' 'b' 'c' 'c'
* two (x) int64 0 1 0 1 0 1
* y (y) int64 0 1 2
In [128]: mda.sel(x=(list("ab"), [0]))
Out[128]:
<xarray.DataArray (x: 2, y: 3)>
array([[0.596, 0.2 , 0.1 ],
[0.096, 0.497, 0.839]])
Coordinates:
* x (x) object MultiIndex
* one (x) object 'a' 'b'
* two (x) int64 0 0
* y (y) int64 0 1 2
You can also select multiple elements by providing a list of labels or tuples or a slice of tuples:
In [129]: mda.sel(x=[("a", 0), ("b", 1)])
Out[129]:
<xarray.DataArray (x: 2, y: 3)>
array([[0.596, 0.2 , 0.1 ],
[0.897, 0.733, 0.759]])
Coordinates:
* x (x) object MultiIndex
* one (x) object 'a' 'b'
* two (x) int64 0 1
* y (y) int64 0 1 2
Additionally, xarray supports dictionaries:
In [130]: mda.sel(x={"one": "a", "two": 0})
Out[130]:
<xarray.DataArray (y: 3)>
array([0.596, 0.2 , 0.1 ])
Coordinates:
x object ('a', 0)
one <U1 'a'
two int64 0
* y (y) int64 0 1 2
For convenience, sel
also accepts multi-index levels directly
as keyword arguments:
In [131]: mda.sel(one="a", two=0)
Out[131]:
<xarray.DataArray (y: 3)>
array([0.596, 0.2 , 0.1 ])
Coordinates:
x object ('a', 0)
one <U1 'a'
two int64 0
* y (y) int64 0 1 2
Note that using sel
it is not possible to mix a dimension
indexer with level indexers for that dimension
(e.g., mda.sel(x={'one': 'a'}, two=0)
will raise a ValueError
).
Like pandas, xarray handles partial selection on multi-index (level drop). As shown below, it also renames the dimension / coordinate when the multi-index is reduced to a single index.
In [132]: mda.loc[{"one": "a"}, ...]
Out[132]:
<xarray.DataArray (two: 2, y: 3)>
array([[0.596, 0.2 , 0.1 ],
[0.735, 0.017, 0.481]])
Coordinates:
* two (two) int64 0 1
* y (y) int64 0 1 2
one <U1 'a'
Unlike pandas, xarray does not guess whether you provide index levels or
dimensions when using loc
in some ambiguous cases. For example, for
mda.loc[{'one': 'a', 'two': 0}]
and mda.loc['a', 0]
xarray
always interprets (‘one’, ‘two’) and (‘a’, 0) as the names and
labels of the 1st and 2nd dimension, respectively. You must specify all
dimensions or use the ellipsis in the loc
specifier, e.g. in the example
above, mda.loc[{'one': 'a', 'two': 0}, :]
or mda.loc[('a', 0), ...]
.
Indexing rules¶
Here we describe the full rules xarray uses for vectorized indexing. Note that this is for the purposes of explanation: for the sake of efficiency and to support various backends, the actual implementation is different.
(Only for label based indexing.) Look up positional indexes along each dimension from the corresponding
pandas.Index
.A full slice object
:
is inserted for each dimension without an indexer.slice
objects are converted into arrays, given bynp.arange(*slice.indices(...))
.Assume dimension names for array indexers without dimensions, such as
np.ndarray
andlist
, from the dimensions to be indexed along. For example,v.isel(x=[0, 1])
is understood asv.isel(x=xr.DataArray([0, 1], dims=['x']))
.For each variable in a
Dataset
orDataArray
(the array and its coordinates):Broadcast all relevant indexers based on their dimension names (see Broadcasting by dimension name for full details).
Index the underling array by the broadcast indexers, using NumPy’s advanced indexing rules.
If any indexer DataArray has coordinates and no coordinate with the same name exists, attach them to the indexed object.
Note
Only 1-dimensional boolean arrays can be used as indexers.