casacore
PagedArray.h
Go to the documentation of this file.
1 //# PagedArray.h: templated Lattice, paged from disk to memory on demand
2 //# Copyright (C) 1994,1995,1996,1997,1998,1999,2000,2001,2002,2003
3 //# Associated Universities, Inc. Washington DC, USA.
4 //#
5 //# This library is free software; you can redistribute it and/or modify it
6 //# under the terms of the GNU Library General Public License as published by
7 //# the Free Software Foundation; either version 2 of the License, or (at your
8 //# option) any later version.
9 //#
10 //# This library is distributed in the hope that it will be useful, but WITHOUT
11 //# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
12 //# FITNESS FOR A PARTICULAR PURPOSE. See the GNU Library General Public
13 //# License for more details.
14 //#
15 //# You should have received a copy of the GNU Library General Public License
16 //# along with this library; if not, write to the Free Software Foundation,
17 //# Inc., 675 Massachusetts Ave, Cambridge, MA 02139, USA.
18 //#
19 //# Correspondence concerning AIPS++ should be addressed as follows:
20 //# Internet email: aips2-request@nrao.edu.
21 //# Postal address: AIPS++ Project Office
22 //# National Radio Astronomy Observatory
23 //# 520 Edgemont Road
24 //# Charlottesville, VA 22903-2475 USA
25 //#
26 //# $Id$
27 
28 #ifndef LATTICES_PAGEDARRAY_H
29 #define LATTICES_PAGEDARRAY_H
30 
31 //# Includes
32 #include <casacore/casa/aips.h>
33 #include <casacore/lattices/Lattices/Lattice.h>
34 #include <casacore/lattices/Lattices/TiledShape.h>
35 #include <casacore/tables/Tables/ArrayColumn.h>
36 #include <casacore/tables/Tables/Table.h>
37 #include <casacore/tables/DataMan/TiledStManAccessor.h>
38 #include <casacore/casa/BasicSL/String.h>
39 
40 
41 namespace casacore { //# NAMESPACE CASACORE - BEGIN
42 
43 // <summary>
44 // A Lattice that is read from or written to disk.
45 // </summary>
46 
47 // <use visibility=export>
48 
49 // <reviewed reviewer="Peter Barnes" date="1999/10/30" tests="tPagedArray.cc" demos="dPagedArray.cc">
50 // </reviewed>
51 
52 // <prerequisite>
53 // <li> <linkto class="Lattice">Lattice</linkto>
54 // <li> <linkto class="TiledShape">TiledShape</linkto>
55 // </prerequisite>
56 
57 // <etymology>
58 // "Demand paging" is a technique used to implement virtual memory in
59 // computer operating systems. In this scheme, code or data are read from
60 // disk to memory only as needed by a process, and are read in fixed-sized
61 // chunks called "pages". PagedArrays are somewhat the same -- though
62 // without the automatic features found in virtual memory demand paging.
63 // However PagedArrays do allow the user to access chunks of the disk in a
64 // flexible way, that can match the requirements of many algorithms.
65 // </etymology>
66 
67 // <synopsis>
68 // At the time of writing, typical scientific computers provide sufficient
69 // memory for storing and manipulating 2-dimensional astronomical images,
70 // which have average size of around 8 MBytes. Astronomy is increasingly
71 // using three or higher dimensional arrays, which can be larger by one or
72 // two orders of magnitude. PagedArrays provide a convenient way of
73 // accessing these large arrays without requiring all the data to be read
74 // into real or virtual memory.
75 // <p>
76 // When you construct a PagedArray you do not read any data into
77 // memory. Instead a disk file (ie. a Table) is created, in a place you
78 // specify, to hold the data. This means you need to have enough disk space
79 // to hold the array. Constructing a PagedArray is equivalent to opening a
80 // file.
81 // <p>
82 // Because the data is stored on disk it can be saved after the program,
83 // function, or task that created the PagedArray has finished. This saved
84 // array can then be read again at a later stage.
85 // <p>
86 // So there are two reasons for using a PagedArray:
87 // <ol>
88 // <li> To provide for arrays that are too large for the computer's memory.
89 // <li> To provide a way of saving arrays to disk for later access.
90 // </ol>
91 //
92 // To access the data in a PagedArray you can either:
93 // <ol>
94 // <li> Use a <linkto class=LatticeIterator>LatticeIterator</linkto>
95 // <li> Use the getSlice and putSlice member functions
96 // <li> Use the parenthesis operator or getAt and putAt functions
97 // </ol>
98 // These access methods are given in order of preference. Some examples of
99 // these access methods are in the documentation for the
100 // <linkto class=Lattice>Lattice</linkto> class as well as below.
101 // <p>
102 // In nearly all cases you access the PagedArray by reading a "slice" of the
103 // PagedArray into a Casacore <linkto class=Array>Array</linkto>. Because the
104 // slice is stored in memory it is important that the slice you read is not
105 // too big compared to the physical memory on your computer. Otherwise your
106 // computer will page excessively and performance will be poor.
107 // <p>
108 // To overcome this you may be tempted to access the PagedArray a pixel at a
109 // time. This will use little memory but the overhead of accessing a large
110 // data set by separately reading each pixel from disk will also lead to poor
111 // performance.
112 // <p>
113 // In general the best way to access the data in PagedArrays is to use a
114 // LatticeIterator with a cursor size that "fits" nicely into memory. Not
115 // only do the LaticeIterator classes provide a relatively simple way to
116 // read/write all the data but they optimally set up the cache that is
117 // associated with each PagedArray.
118 // <p>
119 // If the LatticeIterator classes do not access the data the way you want
120 // you can use the getSlice and putSlice member functions. These functions
121 // do not set up the cache for you and improved performance may be obtained
122 // by tweaking the cache using the setCacheSizeFromPath member frunction.
123 //
124 // <ANCHOR NAME="PagedArray:Advanced"><h3>More Details</h3></ANCHOR>
125 // In order to utilise PagedArrays fully and understand many of the member
126 // functions and data access methods in this class, you need to be familiar
127 // with some of the concepts involved in the implementation of PagedArrays.
128 // <p>
129 // Each PagedArray is stored in one cell of a Table as an indirect Array
130 // (see the documentation for the <linkto module="Tables">Tables</linkto>
131 // module for more information). This means that multiple PagedArrays can be
132 // stored in one Table. To specify which PagedArray you are referring to in
133 // a given Table you need to specify the cell using its column name and row
134 // number during construction. If a cell is not specified the default column
135 // name (as given by the defaultColumnName function) and row number (as
136 // given by the defaultRowNumber function) are used. This ability to store
137 // multiple PagedArrays's is used in the PagedImage class where the image is
138 // stored in one cell and a mask is optionally stored in a another column in
139 // the same row.
140 // <p>
141 // There are currently a number of limitations when storing multiple
142 // PagedArrays in the same Table.
143 // <ul>
144 // <li> All the PagedArrays in the same column MUST have the same number of
145 // dimensions. The dimension used for any particular column is set when the
146 // first PagedArray in that column is constructed. If you want to put a
147 // say two-dimensional PagedArray into another row of a column that
148 // already contains a four-dimensional PagedArray you need to add two
149 // degenerate axes. In principle you could use the resize function, but see
150 // below for why this is not recommended. It is better to just ensure that
151 // all the PagedArrays have the same number of dimensions.
152 // <li> All the cells in a column that contains PagedArrays must have their
153 // shape defined. This becomes important if you are creating a PagedArray in
154 // say row five of a Table that currently only has one row. The PagedArray
155 // constructor will add another four rows to the Table, and put your
156 // PagedArray (with the shape you specify) in row five. For the three
157 // rows for which no shape was specified, the constructor will construct
158 // PagedArrays with only one element (and of an appropriate
159 // dimensionality). As you cannot resize these single element PagedArrays
160 // without difficulty (see below), it is recommended that you add
161 // PagedArrays to rows in your Table sequentially. It is necessary to have
162 // the constructor define the shape of all cells in the Table as it is an
163 // error to write a Table to disk with undefined cell shapes.
164 // </ul>
165 //
166 // Each PagedArray is stored on disk using the tiled cell storage manager
167 // (<linkto class=TiledCellStMan>TiledCellStMan</linkto>). This stores the
168 // data in tiles which are regular subsections of the PagedArray. For
169 // example a PagedArray of shape [1024,1024,4,128] may have a tile shape of
170 // [32,16,4,16]. The data in each tile is stored as a unit on the disk. This
171 // means that there is no preferred axis when accessing multi-dimensional
172 // data.
173 // <br>
174 // The tile shape can be specified when constructing a new PagedArray but
175 // not when reading an old one as it is intrinsic to the way the data is
176 // stored on disk. It is NOT recommended that you specify the tile shape
177 // unless you can control the lifetime of the PagedArray (this includes the
178 // time it spends on disk), or can guarantee the access pattern. For example
179 // if you know that a PagedArray of shape [512,512,4,32] will always be
180 // sliced plane by plane you may prefer to specify a tile shape of
181 // [512,64,1,1] rather than the default of [32,16,4,16].
182 // <br>
183 // Tiles can be cached by the tile storage manager so that it does not need
184 // to read the data from disk every time you are accessing the a pixel in a
185 // different tile. In order to cache the correct tiles you should tell the
186 // storage manager what section of the PagedArray you will be
187 // accessing. This is done using the setCacheSizeFromPath member
188 // function. Alternatively you can set the size of the cache using the
189 // setCacheSizeInTiles member function.
190 // <br>
191 // By default there is no limit on how much memory the tile cache can
192 // consume. This can be changed using the setMaximumCacheSize member
193 // function. The tiled storage manager always tries to cache enough tiles to
194 // ensure that each tile is read from disk only once, so setting the maximum
195 // cache size will trade off memory usage for disk I/O. Setting the cache
196 // size is illustrated in example 5 below.
197 // <br>
198 // The showCacheStatistics member function is provided to allow you to
199 // evaluate the performance of the tile cache.
200 // </synopsis>
201 
202 // <example>
203 // All the examples in this section are available in dPagedArray.cc
204 //
205 // <h4>Example 1:</h4>
206 // Create a PagedArray of Floats of shape [1024,1024,4,256] in a file
207 // called "myData_tmp.array" and initialize it to zero. This will create a
208 // directory on disk called "myData_tmp.array" that contains files that
209 // exceed 1024*1024*4*256*4 (= 4 GBytes) in size.
210 // <srcblock>
211 // const IPosition arrayShape(4,1024,1024,4,256);
212 // const String filename("myData_tmp.array");
213 // PagedArray<Float> diskArray(arrayShape, filename);
214 // cout << "Created a PagedArray of shape " << diskArray.shape()
215 // << " (" << diskArray.shape().product()/1024/1024*sizeof(Float)
216 // << " MBytes)" << endl
217 // << "in the table called " << diskArray.tableName() << endl;
218 // diskArray.set(0.0f);
219 // // Using the set function is an efficient way to initialize the PagedArray
220 // // as it uses a PagedArrIter internally. Note that the set function is
221 // // defined in the Lattice class that PagedArray is derived from.
222 // </srcblock>
223 //
224 // <h4>Example 2:</h4>
225 // Read the PagedArray produced in Example 1 and put a Gaussian profile into
226 // each spectral channel.
227 // <srcblock>
228 // PagedArray<Float> diskArray("myData_tmp.array");
229 // IPosition shape = diskArray.shape();
230 // // Construct a Gaussian Profile to be 10 channels wide and centred on
231 // // channel 16. Its height is 1.0.
232 // Gaussian1D<Float> g(1.0f, 16.0f, 10.0f);
233 // // Create a vector to cache a sampled version of this profile.
234 // Vector<Float> profile(shape(3));
235 // indgen(profile);
236 // profile.apply(g);
237 // // Now put this profile into every spectral channel in the paged array. This
238 // // is best done using an iterator.
239 // LatticeIterator<Float> iter(diskArray,
240 // TiledLineStepper(shape, diskArray.tileShape(), 3));
241 // for (iter.reset(); !iter.atEnd(); iter++) {
242 // iter.woCursor() = profile;
243 // }
244 // </srcblock>
245 //
246 // <h4>Example 3:</h4>
247 // Now multiply the I-polarization data by 10.0 in this PagedArray. The
248 // I-polarization data occupies 1 GByte of RAM which is too big to read
249 // into the memory of most computers. So an iterator is used to get suitable
250 // sized chunks.
251 // <srcblock>
252 // Table t("myData_tmp.array", Table::Update);
253 // PagedArray<Float> da(t);
254 // const IPosition latticeShape = da.shape();
255 // const nx = latticeShape(0);
256 // const ny = latticeShape(1);
257 // const npol = latticeShape(2);
258 // const nchan = latticeShape(3);
259 // IPosition cursorShape = da.niceCursorShape();
260 // cursorShape(2) = 1;
261 // LatticeStepper step(latticeShape, cursorShape);
262 // step.subSection(IPosition(4,0), IPosition(4,nx-1,ny-1,0,nchan-1));
263 // LatticeIterator<Float> iter(da, step);
264 // for (iter.reset(); !iter.atEnd(); iter++) {
265 // iter.rwCursor() *= 10.0f;
266 // }
267 // </srcblock>
268 //
269 // <h4>Example 4:</h4>
270 // Use a direct call to getSlice to access a small central region of the
271 // V-polarization in spectral channel 0 only. The region is small enough
272 // to not warrant constructing iterators and setting up
273 // LatticeNavigators. In this example the call to the getSlice function
274 // is unnecessary but is done for illustration purposes anyway.
275 // <srcblock>
276 // SetupNewTable maskSetup("mask_tmp.array", TableDesc(), Table::New);
277 // Table maskTable(maskSetup);
278 // PagedArray<Bool> maskArray(IPosition(4,1024,1024,4,256), maskTable);
279 // maskArray.set(False);
280 // COWPtr<Array<Bool> > maskPtr;
281 // maskArray.getSlice(maskPtr, IPosition(4,240,240,3,0),
282 // IPosition(4,32,32,1,1), IPosition(4,1));
283 // maskPtr.rwRef() = True;
284 // maskArray.putSlice(*maskPtr, IPosition(4,240,240,3,1));
285 // </srcblock>
286 //
287 // <h4>Example 5:</h4>
288 // In this example the data in the PagedArray will be accessed a row at
289 // a time while setting the cache size to different values. The comments
290 // illustrate the results when running on an Ultra 1/140 with 64MBytes
291 // of memory.
292 // <srcblock>
293 // PagedArray<Float> pa(IPosition(4,128,128,4,32));
294 // const IPosition latticeShape = pa.shape();
295 // cout << "The tile shape is:" << pa.tileShape() << endl;
296 // // The tile shape is:[32, 16, 4, 16]
297 //
298 // // Setup to access the PagedArray a row at a time
299 // const IPosition sliceShape(4,latticeShape(0), 1, 1, 1);
300 // const IPosition stride(4,1);
301 // Array<Float> row(sliceShape);
302 // IPosition start(4, 0);
303 //
304 // // Set the cache size to enough pixels for one tile only. This uses
305 // // 128kBytes of cache memory and takes 125 secs.
306 // pa.setCacheSizeInTiles (1);
307 // Timer clock;
308 // for (start(3) = 0; start(3) < latticeShape(3); start(3)++) {
309 // for (start(2) = 0; start(2) < latticeShape(2); start(2)++) {
310 // for (start(1) = 0; start(1) < latticeShape(1); start(1)++) {
311 // pa.getSlice(row, start, sliceShape, stride);
312 // }
313 // }
314 // }
315 // clock.show();
316 // pa.showCacheStatistics(cout);
317 // pa.clearCache();
318 //
319 // // Set the cache size to enough pixels for one row of tiles (ie. 4).
320 // // This uses 512 kBytes of cache memory and takes 10 secs.
321 // pa.setCacheSizeInTiles (4);
322 // clock.mark();
323 // for (start(3) = 0; start(3) < latticeShape(3); start(3)++) {
324 // for (start(2) = 0; start(2) < latticeShape(2); start(2)++) {
325 // for (start(1) = 0; start(1) < latticeShape(1); start(1)++) {
326 // pa.getSlice(row, start, sliceShape, stride);
327 // }
328 // }
329 // }
330 // clock.show();
331 // pa.showCacheStatistics(cout);
332 // pa.clearCache();
333 //
334 // // Set the cache size to enough pixels for one plane of tiles
335 // // (ie. 4*8). This uses 4 MBytes of cache memory and takes 2 secs.
336 // pa.setCacheSizeInTiles (4*8);
337 // clock.mark();
338 // for (start(3) = 0; start(3) < latticeShape(3); start(3)++) {
339 // for (start(2) = 0; start(2) < latticeShape(2); start(2)++) {
340 // for (start(1) = 0; start(1) < latticeShape(1); start(1)++) {
341 // pa.getSlice(row, start, sliceShape, stride);
342 // }
343 // }
344 // }
345 // clock.show();
346 // pa.showCacheStatistics(cout);
347 // pa.clearCache();
348 // </srcblock>
349 // </example>
350 
351 // <motivation>
352 // Arrays of data are sometimes much too large to hold in random access memory.
353 // PagedArrays, especially in combination with LatticeIterator,
354 // provide convenient access to such large data sets.
355 // </motivation>
356 
357 // <templating arg=T>
358 // <li> Due to storage in Tables, the templated type must be able to be
359 // stored in a Casacore Table. This restricts the template argument to all
360 // the common types Bool, Float, Double, Complex, String etc.) More details
361 // can be found in the RetypedArrayEngine class.
362 // </templating>
363 
364 // <todo asof="1997/04/14">
365 // <li> A better way of resizing PagedArrays
366 // </todo>
367 
368 // <linkfrom anchor="PagedArray" classes="Lattice ArrayLattice">
369 // <here>PagedArray</here> - a disk based Lattice.
370 // </linkfrom>
371 
372 
373 template <class T> class PagedArray : public Lattice<T>
374 {
375  //# Make members of parent class known.
376 public:
377  using Lattice<T>::ndim;
378 
379 public:
380  // The default constructor creates a PagedArray that is useless for just
381  // about everything, except that it can be assigned to with the assignment
382  // operator.
383  PagedArray();
384 
385  // Construct a new PagedArray with the specified shape. A new Table with
386  // the specified filename is constructed to hold the array. The Table will
387  // remain on disk after the PagedArray goes out of scope or is deleted.
388  PagedArray (const TiledShape& shape, const String& filename);
389 
390  // Construct a new PagedArray with the specified shape. A scratch Table is
391  // created in the current working directory to hold the array. This Table
392  // will be deleted automatically when the PagedArray goes out of scope or
393  // is deleted.
394  explicit PagedArray (const TiledShape& shape);
395 
396  // Construct a new PagedArray, with the specified shape, in the default
397  // row and column of the supplied Table.
398  PagedArray (const TiledShape& shape, Table& file);
399 
400  // Construct a new PagedArray, with the specified shape, in the specified
401  // row and column of the supplied Table.
402  PagedArray (const TiledShape& shape, Table& file,
403  const String& columnName, uInt rowNum);
404 
405  // Reconstruct from a pre-existing PagedArray in the default row and
406  // column of the supplied Table with the supplied filename.
407  explicit PagedArray (const String& filename);
408 
409  // Reconstruct from a pre-existing PagedArray in the default row and
410  // column of the supplied Table.
411  explicit PagedArray (Table& file);
412 
413  // Reconstruct from a pre-existing PagedArray in the specified row and
414  // column of the supplied Table.
415  PagedArray (Table& file, const String& columnName, uInt rowNum);
416 
417  // The copy constructor which uses reference semantics. Copying by value
418  // doesn't make sense, because it would require the creation of a
419  // temporary (but possibly huge) file on disk.
420  PagedArray (const PagedArray<T>& other);
421 
422  // The destructor flushes the PagedArrays contents to disk.
423  ~PagedArray();
424 
425  // The assignment operator with reference semantics. As with the copy
426  // constructor assigning by value does not make sense.
427  PagedArray<T>& operator= (const PagedArray<T>& other);
428 
429  // Make a copy of the object (reference semantics).
430  virtual Lattice<T>* clone() const;
431 
432  // A PagedArray is always persistent.
433  virtual Bool isPersistent() const;
434 
435  // A PagedArray is always paged to disk.
436  virtual Bool isPaged() const;
437 
438  // Is the PagedArray writable?
439  virtual Bool isWritable() const;
440 
441  // Returns the shape of the PagedArray.
442  virtual IPosition shape() const;
443 
444  // Return the current Table name. By default this includes the full path.
445  // The path preceeding the file name can be stripped off on request.
446  virtual String name (Bool stripPath=False) const;
447 
448  // Functions to resize the PagedArray. The old contents are lost. Usage of
449  // this function is NOT currently recommended (see the <linkto
450  // class="PagedArray:Advanced">More Details</linkto> section above).
451  void resize (const TiledShape& newShape);
452 
453  // Returns the current table name (ie. filename) of this PagedArray.
454  const String& tableName() const;
455 
456  // Return the current table object.
457  // <group>
458  Table& table();
459  const Table& table() const;
460  // </group>
461 
462  // Returns the current Table column name of this PagedArray.
463  const String& columnName() const;
464 
465  // Returns the default TableColumn name for a PagedArray.
466  static String defaultColumn();
467 
468  // Returns an accessor to the tiled storage manager.
469  const ROTiledStManAccessor& accessor() const;
470 
471  // Returns the current row number of this PagedArray.
472  uInt rowNumber() const;
473 
474  // Returns the default row number for a PagedArray.
475  static uInt defaultRow();
476 
477  // Returns the current tile shape for this PagedArray.
478  IPosition tileShape() const;
479 
480  // Returns the maximum recommended number of pixels for a cursor. This is
481  // the number of pixels in a tile.
482  virtual uInt advisedMaxPixels() const;
483 
484  // Set the maximum allowed cache size for all Arrays in this column of the
485  // Table. The actual value used may be smaller. A value of zero means
486  // that there is no maximum.
487  virtual void setMaximumCacheSize (uInt howManyPixels);
488 
489  // Return the maximum allowed cache size (in pixels) for all Arrays in
490  // this column of the Table. The actual cache size may be smaller. A
491  // value of zero means that no maximum is currently defined.
492  virtual uInt maximumCacheSize() const;
493 
494  // Set the actual cache size for this Array to be big enough for the
495  // indicated number of tiles. This cache is not shared with PagedArrays
496  // in other rows and is always clipped to be less than the maximum value
497  // set using the setMaximumCacheSize member function.
498  // Tiles are cached using a first in first out algorithm.
499  virtual void setCacheSizeInTiles (uInt howManyTiles);
500 
501  // Set the actual cache size for this Array to "fit" the indicated
502  // path. This cache is not shared with PagedArrays in other rows and is
503  // always less than the maximum value. The sliceShape is the cursor or
504  // slice that you will be requiring (with each call to
505  // {get,put}Slice). The windowStart and windowLength delimit the range of
506  // pixels that will ultimatly be accessed. The AxisPath is described in
507  // the documentation for the LatticeStepper class.
508  virtual void setCacheSizeFromPath (const IPosition& sliceShape,
509  const IPosition& windowStart,
510  const IPosition& windowLength,
511  const IPosition& axisPath);
512 
513  // Clears and frees up the tile cache. The maximum allowed cache size is
514  // unchanged from when <src>setMaximumCacheSize</src> was last called.
515  virtual void clearCache();
516 
517  // Generate a report on how the cache is doing. This is reset every
518  // time <src>clearCache</src> is called.
519  virtual void showCacheStatistics (ostream& os) const;
520 
521  // Return the value of the single element located at the argument
522  // IPosition.
523  // Note that <src>Lattice::operator()</src> can also be used.
524  virtual T getAt (const IPosition& where) const;
525 
526  // Put the value of a single element.
527  virtual void putAt (const T& value, const IPosition& where);
528 
529  // A function which checks for internal consistency. Returns False if
530  // something nasty has happened to the PagedArray. In that case
531  // it also throws an exception.
532  virtual Bool ok() const;
533 
534  // This function is used by the LatticeIterator class to generate an
535  // iterator of the correct type for a specified Lattice. Not recommended
536  // for general use.
537  virtual LatticeIterInterface<T>* makeIter (const LatticeNavigator& navigator,
538  Bool useRef) const;
539 
540  // Do the actual getting of an array of values.
541  virtual Bool doGetSlice (Array<T>& buffer, const Slicer& section);
542 
543  // Do the actual getting of an array of values.
544  virtual void doPutSlice (const Array<T>& sourceBuffer,
545  const IPosition& where,
546  const IPosition& stride);
547 
548  // Get the best cursor shape.
549  virtual IPosition doNiceCursorShape (uInt maxPixels) const;
550 
551  // Handle the (un)locking.
552  // <group>
553  virtual Bool lock (FileLocker::LockType, uInt nattempts);
554  virtual void unlock();
555  virtual Bool hasLock (FileLocker::LockType) const;
556  // </group>
557 
558  // Resynchronize the PagedArray object with the lattice file.
559  // This function is only useful if no read-locking is used, ie.
560  // if the table lock option is UserNoReadLocking or AutoNoReadLocking.
561  // In that cases the table system does not acquire a read-lock, thus
562  // does not synchronize itself automatically.
563  virtual void resync();
564 
565  // Flush the data (but do not unlock).
566  virtual void flush();
567 
568  // Temporarily close the lattice.
569  // It will be reopened automatically on the next access.
570  virtual void tempClose();
571 
572  // Explicitly reopen the temporarily closed lattice.
573  virtual void reopen();
574 
575 private:
576  // Set the data in the TableInfo file
577  void setTableType();
578  // make the ArrayColumn
579  void makeArray (const TiledShape& shape);
580  // Make a Table to hold this PagedArray
581  void makeTable (const String& filename, Table::TableOption option);
582  // The default comment for PagedArray Colums
583  static String defaultComment();
584  // Get the writable ArrayColumn object.
585  // It reopens the table for write if needed.
587  // Do the reopen of the table (if not open already).
588  // <group>
589  void doReopen() const;
590  void tempReopen() const;
591  // </group>
592 
593  mutable Table itsTable;
596  mutable Bool itsIsClosed;
603 };
604 
605 
606 template<class T>
608 {
609  if (itsIsClosed) {
610  doReopen();
611  }
612  if (!itsWritable) {
613  itsTable.reopenRW();
614  itsWritable = True;
615  }
616  return itsArray;
617 }
618 
619 template<class T>
621 {
622  doReopen();
623  return itsTable;
624 }
625 template<class T>
626 inline const Table& PagedArray<T>::table() const
627 {
628  doReopen();
629  return itsTable;
630 }
631 
632 template<class T>
633 inline const String& PagedArray<T>::columnName() const
634 {
635  return itsColumnName;
636 }
637 
638 template<class T>
640 {
641  return "PagedArray";
642 }
643 
644 template<class T>
646 {
647  return itsAccessor;
648 }
649 
650 template<class T>
652 {
653  return itsRowNumber;
654 }
655 
656 template<class T>
658 {
659  return 0;
660 }
661 
662 template<class T>
664 {
665  if (itsIsClosed) {
666  tempReopen();
667  }
668 }
669 
670 //# Declare extern templates for often used types.
671  extern template class PagedArray<Float>;
672  extern template class PagedArray<Complex>;
673 
674 
675 } //# NAMESPACE CASACORE - END
676 
677 #ifndef CASACORE_NO_AUTO_TEMPLATES
678 #include <casacore/lattices/Lattices/PagedArray.tcc>
679 #endif //# CASACORE_NO_AUTO_TEMPLATES
680 #endif
casacore::PagedArray::tempClose
virtual void tempClose()
Temporarily close the lattice.
casacore::PagedArray::doReopen
void doReopen() const
Do the reopen of the table (if not open already).
Definition: PagedArray.h:663
casacore::Slicer
Specify which elements to extract from an n-dimensional array.
Definition: Slicer.h:289
casacore::PagedArray::PagedArray
PagedArray()
The default constructor creates a PagedArray that is useless for just about everything,...
casacore::PagedArray::makeTable
void makeTable(const String &filename, Table::TableOption option)
Make a Table to hold this PagedArray.
casacore::PagedArray::itsColumnName
String itsColumnName
Definition: PagedArray.h:594
casacore::PagedArray::doGetSlice
virtual Bool doGetSlice(Array< T > &buffer, const Slicer &section)
Do the actual getting of an array of values.
casacore::IPosition
A Vector of integers, for indexing into Array<T> objects.
Definition: IPosition.h:119
casacore::PagedArray::tempReopen
void tempReopen() const
casacore::PagedArray::defaultComment
static String defaultComment()
The default comment for PagedArray Colums.
casacore::PagedArray::columnName
const String & columnName() const
Returns the current Table column name of this PagedArray.
Definition: PagedArray.h:633
casacore::PagedArray::clone
virtual Lattice< T > * clone() const
Make a copy of the object (reference semantics).
casacore::PagedArray::getAt
virtual T getAt(const IPosition &where) const
Return the value of the single element located at the argument IPosition.
casacore::PagedArray::itsTable
Table itsTable
Definition: PagedArray.h:593
casacore::PagedArray::itsWritable
Bool itsWritable
Definition: PagedArray.h:599
casacore::PagedArray::tileShape
IPosition tileShape() const
Returns the current tile shape for this PagedArray.
casacore::PagedArray::shape
virtual IPosition shape() const
Returns the shape of the PagedArray.
casacore::PagedArray::defaultRow
static uInt defaultRow()
Returns the default row number for a PagedArray.
Definition: PagedArray.h:657
casacore::PagedArray::itsArray
ArrayColumn< T > itsArray
Definition: PagedArray.h:601
casacore::PagedArray::clearCache
virtual void clearCache()
Clears and frees up the tile cache.
casacore::FileLocker::LockType
LockType
Define the possible lock types.
Definition: FileLocker.h:95
casacore::PagedArray::itsRowNumber
uInt itsRowNumber
Definition: PagedArray.h:595
casacore::PagedArray::flush
virtual void flush()
Flush the data (but do not unlock).
casacore::PagedArray::doPutSlice
virtual void doPutSlice(const Array< T > &sourceBuffer, const IPosition &where, const IPosition &stride)
Do the actual getting of an array of values.
casacore::ArrayColumn
Read and write access to an array table column with arbitrary data type.
Definition: CopyRecord.h:48
casacore::PagedArray::setTableType
void setTableType()
Set the data in the TableInfo file.
casacore::PagedArray::isWritable
virtual Bool isWritable() const
Is the PagedArray writable?
casacore::PagedArray::operator=
PagedArray< T > & operator=(const PagedArray< T > &other)
The assignment operator with reference semantics.
casacore::PagedArray::doNiceCursorShape
virtual IPosition doNiceCursorShape(uInt maxPixels) const
Get the best cursor shape.
casacore::PagedArray::itsTableName
String itsTableName
Definition: PagedArray.h:598
casacore::PagedArray::defaultColumn
static String defaultColumn()
Returns the default TableColumn name for a PagedArray.
Definition: PagedArray.h:639
casacore::PagedArray::lock
virtual Bool lock(FileLocker::LockType, uInt nattempts)
Handle the (un)locking.
casacore::PagedArray::maximumCacheSize
virtual uInt maximumCacheSize() const
Return the maximum allowed cache size (in pixels) for all Arrays in this column of the Table.
casacore::ROTiledStManAccessor
Give access to some TiledStMan functions.
Definition: TiledStManAccessor.h:156
casacore::PagedArray::reopen
virtual void reopen()
Explicitly reopen the temporarily closed lattice.
casacore::PagedArray::getRWArray
ArrayColumn< T > & getRWArray()
Get the writable ArrayColumn object.
Definition: PagedArray.h:607
casacore::PagedArray::putAt
virtual void putAt(const T &value, const IPosition &where)
Put the value of a single element.
casacore::PagedArray::itsMarkDelete
Bool itsMarkDelete
Definition: PagedArray.h:597
casacore::PagedArray::itsLockOpt
TableLock itsLockOpt
Definition: PagedArray.h:600
casacore::Table
Main interface class to a read/write table.
Definition: Table.h:153
casacore::PagedArray::unlock
virtual void unlock()
casacore::PagedArray::itsIsClosed
Bool itsIsClosed
Definition: PagedArray.h:596
casacore::value
LatticeExprNode value(const LatticeExprNode &expr)
This function returns the value of the expression without a mask.
casacore::LatticeNavigator
Abstract base class to steer lattice iterators.
Definition: LatticeNavigator.h:182
casacore::PagedArray::hasLock
virtual Bool hasLock(FileLocker::LockType) const
casacore::PagedArray::tableName
const String & tableName() const
Returns the current table name (ie.
casacore::False
const Bool False
Definition: aipstype.h:44
casacore::uInt
unsigned int uInt
Definition: aipstype.h:51
casacore::PagedArray::resync
virtual void resync()
Resynchronize the PagedArray object with the lattice file.
casacore::PagedArray::~PagedArray
~PagedArray()
The destructor flushes the PagedArrays contents to disk.
casacore::PagedArray
A Lattice that is read from or written to disk.
Definition: PagedArray.h:373
casacore::PagedArray::name
virtual String name(Bool stripPath=False) const
Return the current Table name.
casacore::PagedArray::rowNumber
uInt rowNumber() const
Returns the current row number of this PagedArray.
Definition: PagedArray.h:651
casacore::PagedArray::itsAccessor
ROTiledStManAccessor itsAccessor
Definition: PagedArray.h:602
casacore::TableLock
Class to hold table lock options.
Definition: TableLock.h:68
casacore
this file contains all the compiler specific defines
Definition: mainpage.dox:28
casacore::PagedArray::advisedMaxPixels
virtual uInt advisedMaxPixels() const
Returns the maximum recommended number of pixels for a cursor.
casacore::True
const Bool True
Definition: aipstype.h:43
casacore::PagedArray::resize
void resize(const TiledShape &newShape)
Functions to resize the PagedArray.
casacore::PagedArray::accessor
const ROTiledStManAccessor & accessor() const
Returns an accessor to the tiled storage manager.
Definition: PagedArray.h:645
casacore::PagedArray::table
Table & table()
Return the current table object.
Definition: PagedArray.h:620
casacore::PagedArray::isPersistent
virtual Bool isPersistent() const
A PagedArray is always persistent.
casacore::PagedArray::setCacheSizeInTiles
virtual void setCacheSizeInTiles(uInt howManyTiles)
Set the actual cache size for this Array to be big enough for the indicated number of tiles.
casacore::PagedArray::ok
virtual Bool ok() const
A function which checks for internal consistency.
casacore::PagedArray::makeIter
virtual LatticeIterInterface< T > * makeIter(const LatticeNavigator &navigator, Bool useRef) const
This function is used by the LatticeIterator class to generate an iterator of the correct type for a ...
casacore::PagedArray::showCacheStatistics
virtual void showCacheStatistics(ostream &os) const
Generate a report on how the cache is doing.
casacore::Lattice
A templated, abstract base class for array-like objects.
Definition: Functional.h:37
casacore::PagedArray::setMaximumCacheSize
virtual void setMaximumCacheSize(uInt howManyPixels)
Set the maximum allowed cache size for all Arrays in this column of the Table.
casacore::PagedArray::makeArray
void makeArray(const TiledShape &shape)
make the ArrayColumn
casacore::PagedArray::setCacheSizeFromPath
virtual void setCacheSizeFromPath(const IPosition &sliceShape, const IPosition &windowStart, const IPosition &windowLength, const IPosition &axisPath)
Set the actual cache size for this Array to "fit" the indicated path.
casacore::Array
template <class T, class U> class vector;
Definition: Array.h:166
casacore::String
String: the storage and methods of handling collections of characters.
Definition: String.h:223
casacore::Bool
bool Bool
Define the standard types used by Casacore.
Definition: aipstype.h:42
casacore::PagedArray::isPaged
virtual Bool isPaged() const
A PagedArray is always paged to disk.
casacore::Table::TableOption
TableOption
Define the possible options how a table can be opened.
Definition: Table.h:168
casacore::TiledShape
Define the shape and tile shape.
Definition: TiledShape.h:99
casacore::LatticeIterInterface
A base class for Lattice iterators.
Definition: ImageExpr.h:47