evefile.entities.data module

Entities representing an eveH5 file on the data level.

Contents

Data are organised in “datasets” within HDF5, and the evefile.entities.data module provides the relevant entities to describe these datasets.

Please note that in contrast to the evedata package, the evefile package has a somewhat reduced data model, e.g. not considering individual data points for average and interval channels (that are currently not available from the underlying data files, anyway). Whether the corresponding module in the evedata package will become a true superset of this module remains to be seen.

Overview

A first overview of the classes implemented in this module and their hierarchy is given in the UML diagram below, Fig. 11. The first distinction is made between MonitorData and MeasureData, with the former having timestamps (in milliseconds) as their quantisation axis, and the latter individual positions (integer values). MeasureData can further be separated into AxisData, ChannelData, and DeviceData. The TimestampData class is somewhat special, as it (only) gets used to map timestamps to positions and does not correspond to any physical device or option of such device. Generally, each data class comes with its corresponding metadata class implemented in the evefile.entities.metadata module.

Fig. 11 Class hierarchy of the evefile.entities.data module. Each class has a corresponding metadata class in the evefile.entities.metadata module. While in this diagram, some child classes seem to be identical, they have a different type of metadata (see the evefile.entities.metadata module). Generally, having different types serves to discriminate where necessary between detector channels and motor axes. You may click on the image for a larger view.

Individual classes

The following is not a strict inheritance hierarchy, but rather a grouped hierarchical list of classes for quick access to their individual API documentation:

• Data

  • MonitorData

  • MeasureData

    • DeviceData

    • TimestampData

    • AxisData

    • ChannelData

      • SinglePointChannelData

        • SinglePointNormalizedChannelData

      • AverageChannelData

        • AverageNormalizedChannelData

      • IntervalChannelData

        • IntervalNormalizedChannelData

Special aspects

There is a number of special aspects that need to be taken into account when reading data. These are detailed below.

Sorting non-monotonic positions

For MeasureData, positions (“PosCounts”) need not be monotonically increasing. This is due to the way the engine handles the different scan modules and writes data. However, this will usually be a problem for the analysis. Therefore, positions need to be sorted monotonically, and this is done during data import.

Handling duplicate positions

Although technically speaking a bug, some (older) measurement files contain duplicate positions (“PosCounts”). Here, the handling is different for AxisData and ChannelData, but in both cases taken care of during data import:

• For AxisData, only the last position is taken.

• For ChannelData, only the first position is taken.

Module documentation

class evefile.entities.data.Data

Bases: object

Data recorded from the devices involved in a measurement.

This is the base class for all data(sets) and not meant to be used directly. Rather, one of the individual subclasses should actually be used.

When subclassing, make sure to create the corresponding metadata class in the evefile.entities.metadata module as well.

Data are read from HDF5 files, and to save time and resources, actual data are only read upon request.

metadata

Relevant metadata for the individual device.

Type:

evefile.entities.metadata.Metadata

options

(Variable) options of the device.

Devices can have options. Generally, there are two types of options: those whose values are not changing within a given scan module, and those whose values can potentially change for every individual position (count). The latter are stored here as key–value pairs with the key corresponding to the option name. The former are stored in the evefile.entities.metadata.Metadata.options attribute.

Type:

dict

importer

Importer objects for the data and possibly (variable) options.

Each item is a DataImporter object.

Data are loaded on demand, not already when initially loading the eveH5 file. Hence, the need for a mechanism to provide the relevant information where to get the relevant data from and how. Different versions of the underlying eveH5 schema differ even in whether all data belonging to one Data object are located in one HDF5 dataset or spread over multiple HDF5 datasets. In the latter case, individual importers are necessary for the separate HDF5 datasets. Hence, the list of importers.

Type:

list

Examples

The Data class is not meant to be used directly, as any entities, but rather indirectly by means of the respective facades in the boundaries technical layer of the evefile package. Hence, for the time being, there are no dedicated examples how to use this class. Of course, you can instantiate an object as usual.

property data

Actual data recorded from the device.

Data are loaded only on demand. Hence, upon the first access of the data property, the get_data() method will be called, calling out to the respective importers.

Returns:

data – Actual data recorded from the device.

The actual data type (numpy.dtype) depends on the specific dataset loaded.

Return type:

numpy.ndarray

get_data()

Load data (and variable option data) using the respective importer.

Data are loaded only on demand. Hence, upon the first access of the data property, this method will be called, calling out to the respective importers.

As Data objects may contain (variable) options that are themselves data, but loading these data is only triggered when accessing the data property, you can either once access the data property or call this method.

Data may be spread over several HDF5 datasets, depending on the version of the eveH5 file read. Hence, there may be several importers, and they are dealt with sequentially.

Furthermore, for each importer type, there is a special private method _import_from_<importer-type>, with <importer-type> being the lowercase class name. Those classes using additional importers beyond HDF5DataImporter need to implement additional private methods to handle the special importer classes. A typical use case is the AreaChannelData class dealing with image data stored mostly in separate files.

copy_attributes_from(source=None)

Obtain attributes from another Data object.

Sometimes, it is useful to obtain the (public) attributes from another Data object. Note that only public attributes are copied. Furthermore, a (true) copy of the attributes is obtained, hence the properties of source and target are actually different objects.

Parameters:

source (Data) –

Object to copy attributes from.

Should typically be of the same (super)type.

Raises:

ValueError – Raised if no source is provided to copy attributes from.

show_info()

Print basic information regarding the contents of a data object.

Often, it is convenient to get a brief overview of the contents of a data object. The output of this method currently contains the following sections:

• metadata

• options (if present)

• fields

The output could look similar to the following:

METADATA
name: jane

OPTIONS
some_option: value

FIELDS
data

Here, the METADATA block simply outputs what you would get with

print(data.metadata)

If options are present, then the keys of the options dict are returned in the OPTIONS block. Finally, the FIELDS block provides an overview of all the attributes containing some kind of data. This will differ depending on the type of data you are looking at.

get_dataframe()

Retrieve Pandas DataFrame with data as column.

Important

While working with a Pandas DataFrame may seem convenient, you’re loosing basically all the relevant metadata of the datasets. Hence, this method is rather a convenience method to be backwards-compatible to older interfaces, but it is explicitly not suggested for extensive use.

Returns:

dataframe – Pandas DataFrame containing data as column.

Return type:

pandas.DataFrame

class evefile.entities.data.MonitorData

Bases: Data

Data from devices monitored, but not controlled by the eve engine.

Monitors are a concept stemming from the underlying EPICS layer and are closely related to telemetry in general. In short: You register a certain “device”, record an initial value, and from then on only changes to this value (together with a timestamp). This allows you to record any relevant changes in your setup with minimal overhead and data storage.

In contrast to MeasureData, MonitorData do not have a position as primary axis, but a timestamp in milliseconds, i.e., the milliseconds attribute. This means that without further ado, you cannot plot monitor data against other data.

metadata

Relevant metadata for the individual device.

Type:

evefile.entities.metadata.MonitorMetadata

milliseconds

Time in milliseconds since start of the scan.

Type:

numpy.ndarray

Examples

The MonitorData class is not meant to be used directly, as any entities, but rather indirectly by means of the respective facades in the boundaries technical layer of the evefile package. Hence, for the time being, there are no dedicated examples how to use this class. Of course, you can instantiate an object as usual.

get_dataframe()

Retrieve Pandas DataFrame with data as column.

The index is named “milliseconds” and contains the values of the milliseconds attribute of the data object.

Important

While working with a Pandas DataFrame may seem convenient, you’re loosing basically all the relevant metadata of the datasets. Hence, this method is rather a convenience method to be backwards-compatible to older interfaces, but it is explicitly not suggested for extensive use.

Returns:

dataframe – Pandas DataFrame containing data as column.

Return type:

pandas.DataFrame

copy_attributes_from(source=None)

Obtain attributes from another Data object.

Sometimes, it is useful to obtain the (public) attributes from another Data object. Note that only public attributes are copied. Furthermore, a (true) copy of the attributes is obtained, hence the properties of source and target are actually different objects.

Parameters:

source (Data) –

Object to copy attributes from.

Should typically be of the same (super)type.

Raises:

ValueError – Raised if no source is provided to copy attributes from.

property data

Actual data recorded from the device.

Data are loaded only on demand. Hence, upon the first access of the data property, the get_data() method will be called, calling out to the respective importers.

Returns:

data – Actual data recorded from the device.

The actual data type (numpy.dtype) depends on the specific dataset loaded.

Return type:

numpy.ndarray

get_data()

Load data (and variable option data) using the respective importer.

Data are loaded only on demand. Hence, upon the first access of the data property, this method will be called, calling out to the respective importers.

As Data objects may contain (variable) options that are themselves data, but loading these data is only triggered when accessing the data property, you can either once access the data property or call this method.

Data may be spread over several HDF5 datasets, depending on the version of the eveH5 file read. Hence, there may be several importers, and they are dealt with sequentially.

Furthermore, for each importer type, there is a special private method _import_from_<importer-type>, with <importer-type> being the lowercase class name. Those classes using additional importers beyond HDF5DataImporter need to implement additional private methods to handle the special importer classes. A typical use case is the AreaChannelData class dealing with image data stored mostly in separate files.

show_info()

Print basic information regarding the contents of a data object.

Often, it is convenient to get a brief overview of the contents of a data object. The output of this method currently contains the following sections:

• metadata

• options (if present)

• fields

The output could look similar to the following:

METADATA
name: jane

OPTIONS
some_option: value

FIELDS
data

Here, the METADATA block simply outputs what you would get with

print(data.metadata)

If options are present, then the keys of the options dict are returned in the OPTIONS block. Finally, the FIELDS block provides an overview of all the attributes containing some kind of data. This will differ depending on the type of data you are looking at.

class evefile.entities.data.MeasureData

Bases: Data

Base class for data from devices actually controlled by the eve engine.

In contrast to MonitorData, MeasureData have a position as primary axis rather than a timestamp in milliseconds, i.e., the positions attribute.

Note

Positions and (all) corresponding data are sorted upon load. For the handling of duplicate positions in AxisData and ChannelData. see there.

metadata

Relevant metadata for the individual device.

Type:

evefile.entities.metadata.MeasureMetadata

Examples

The MeasureData class is not meant to be used directly, as any entities, but rather indirectly by means of the respective facades in the boundaries technical layer of the evefile package. Hence, for the time being, there are no dedicated examples how to use this class. Of course, you can instantiate an object as usual.

property position_counts

Position counts data are recorded for.

Each data “point” corresponds to an overall position of all actuators (motor axes) of the setup and is assigned an individual “position count” (an integer number).

Actual data recorded from the device.

Data are loaded only on demand. Hence, upon the first access of the positions property, the get_data() method will be called, calling out to the respective importers.

Returns:

positions – Position values data are recorded for.

The actual data type (numpy.dtype) is usually int.

Return type:

numpy.ndarray

get_dataframe()

Retrieve Pandas DataFrame with data as column.

The index is named “positions” and contains the values of the position_counts attribute of the data object.

Important

While working with a Pandas DataFrame may seem convenient, you’re loosing basically all the relevant metadata of the datasets. Hence, this method is rather a convenience method to be backwards-compatible to older interfaces, but it is explicitly not suggested for extensive use.

Returns:

dataframe – Pandas DataFrame containing data as column.

Return type:

pandas.DataFrame

join(positions=None)

Perform a left join of the data on the provided list of positions.

The main “quantisation” axis of the values for a device and the common reference is the list of positions. To sensibly compare the data of different devices or plot different device data against each other, the data need to be harmonised, i.e. share a common set of positions as indices.

If positions are not present in the original data, by default, the corresponding entries will be masked and the data attribute converted into a numpy.ma.MaskedArray.

The reason for not using “NaN” (not a number) is, in short, that “NaN” is only defined for floating point numbers, but neither integers nor non-numeric values. Data, however, could generally contain values that are not floating point numbers. For a more detailed discussion, see the evefile.controllers.joining module.

Note

The method will alter the data and positions of the underlying MeasureData object. Hence, make sure to make a copy if this is not your intended use case.

Parameters:

positions (numpy.ndarray) – Array with positions the data should be mapped to.

Raises:

ValueError – Raised if no positions are provided

copy_attributes_from(source=None)

Obtain attributes from another Data object.

Sometimes, it is useful to obtain the (public) attributes from another Data object. Note that only public attributes are copied. Furthermore, a (true) copy of the attributes is obtained, hence the properties of source and target are actually different objects.

Parameters:

source (Data) –

Object to copy attributes from.

Should typically be of the same (super)type.

Raises:

ValueError – Raised if no source is provided to copy attributes from.

property data

Actual data recorded from the device.

Data are loaded only on demand. Hence, upon the first access of the data property, the get_data() method will be called, calling out to the respective importers.

Returns:

data – Actual data recorded from the device.

The actual data type (numpy.dtype) depends on the specific dataset loaded.

Return type:

numpy.ndarray

get_data()

Load data (and variable option data) using the respective importer.

Data are loaded only on demand. Hence, upon the first access of the data property, this method will be called, calling out to the respective importers.

As Data objects may contain (variable) options that are themselves data, but loading these data is only triggered when accessing the data property, you can either once access the data property or call this method.

Data may be spread over several HDF5 datasets, depending on the version of the eveH5 file read. Hence, there may be several importers, and they are dealt with sequentially.

Furthermore, for each importer type, there is a special private method _import_from_<importer-type>, with <importer-type> being the lowercase class name. Those classes using additional importers beyond HDF5DataImporter need to implement additional private methods to handle the special importer classes. A typical use case is the AreaChannelData class dealing with image data stored mostly in separate files.

show_info()

Print basic information regarding the contents of a data object.

Often, it is convenient to get a brief overview of the contents of a data object. The output of this method currently contains the following sections:

• metadata

• options (if present)

• fields

The output could look similar to the following:

METADATA
name: jane

OPTIONS
some_option: value

FIELDS
data

Here, the METADATA block simply outputs what you would get with

print(data.metadata)

If options are present, then the keys of the options dict are returned in the OPTIONS block. Finally, the FIELDS block provides an overview of all the attributes containing some kind of data. This will differ depending on the type of data you are looking at.

class evefile.entities.data.DeviceData

Bases: MeasureData

Data from (dumb) devices.

Three types of devices are distinguished by the eve measurement program: (dumb) devices, motor axes, and detector channels.

metadata

Relevant metadata for the individual device.

Type:

evefile.entities.metadata.DeviceMetadata

Examples

The DeviceData class is not meant to be used directly, as any entities, but rather indirectly by means of the respective facades in the boundaries technical layer of the evefile package. Hence, for the time being, there are no dedicated examples how to use this class. Of course, you can instantiate an object as usual.

copy_attributes_from(source=None)

Obtain attributes from another Data object.

Sometimes, it is useful to obtain the (public) attributes from another Data object. Note that only public attributes are copied. Furthermore, a (true) copy of the attributes is obtained, hence the properties of source and target are actually different objects.

Parameters:

source (Data) –

Object to copy attributes from.

Should typically be of the same (super)type.

Raises:

ValueError – Raised if no source is provided to copy attributes from.

property data

Actual data recorded from the device.

Data are loaded only on demand. Hence, upon the first access of the data property, the get_data() method will be called, calling out to the respective importers.

Returns:

data – Actual data recorded from the device.

The actual data type (numpy.dtype) depends on the specific dataset loaded.

Return type:

numpy.ndarray

get_data()

Load data (and variable option data) using the respective importer.

Data are loaded only on demand. Hence, upon the first access of the data property, this method will be called, calling out to the respective importers.

As Data objects may contain (variable) options that are themselves data, but loading these data is only triggered when accessing the data property, you can either once access the data property or call this method.

Data may be spread over several HDF5 datasets, depending on the version of the eveH5 file read. Hence, there may be several importers, and they are dealt with sequentially.

Furthermore, for each importer type, there is a special private method _import_from_<importer-type>, with <importer-type> being the lowercase class name. Those classes using additional importers beyond HDF5DataImporter need to implement additional private methods to handle the special importer classes. A typical use case is the AreaChannelData class dealing with image data stored mostly in separate files.

get_dataframe()

Retrieve Pandas DataFrame with data as column.

The index is named “positions” and contains the values of the position_counts attribute of the data object.

Important

While working with a Pandas DataFrame may seem convenient, you’re loosing basically all the relevant metadata of the datasets. Hence, this method is rather a convenience method to be backwards-compatible to older interfaces, but it is explicitly not suggested for extensive use.

Returns:

dataframe – Pandas DataFrame containing data as column.

Return type:

pandas.DataFrame

join(positions=None)

Perform a left join of the data on the provided list of positions.

The main “quantisation” axis of the values for a device and the common reference is the list of positions. To sensibly compare the data of different devices or plot different device data against each other, the data need to be harmonised, i.e. share a common set of positions as indices.

If positions are not present in the original data, by default, the corresponding entries will be masked and the data attribute converted into a numpy.ma.MaskedArray.

The reason for not using “NaN” (not a number) is, in short, that “NaN” is only defined for floating point numbers, but neither integers nor non-numeric values. Data, however, could generally contain values that are not floating point numbers. For a more detailed discussion, see the evefile.controllers.joining module.

Note

The method will alter the data and positions of the underlying MeasureData object. Hence, make sure to make a copy if this is not your intended use case.

Parameters:

positions (numpy.ndarray) – Array with positions the data should be mapped to.

Raises:

ValueError – Raised if no positions are provided

property position_counts

Position counts data are recorded for.

Each data “point” corresponds to an overall position of all actuators (motor axes) of the setup and is assigned an individual “position count” (an integer number).

Actual data recorded from the device.

Data are loaded only on demand. Hence, upon the first access of the positions property, the get_data() method will be called, calling out to the respective importers.

Returns:

positions – Position values data are recorded for.

The actual data type (numpy.dtype) is usually int.

Return type:

numpy.ndarray

show_info()

Print basic information regarding the contents of a data object.

Often, it is convenient to get a brief overview of the contents of a data object. The output of this method currently contains the following sections:

• metadata

• options (if present)

• fields

The output could look similar to the following:

METADATA
name: jane

OPTIONS
some_option: value

FIELDS
data

Here, the METADATA block simply outputs what you would get with

print(data.metadata)

If options are present, then the keys of the options dict are returned in the OPTIONS block. Finally, the FIELDS block provides an overview of all the attributes containing some kind of data. This will differ depending on the type of data you are looking at.

class evefile.entities.data.AxisData

Bases: MeasureData

Data from motor axes.

Three types of devices are distinguished by the eve measurement program: (dumb) devices, motor axes, and detector channels.

Note

Positions and (all) corresponding data are sorted upon load. In case of duplicate positions, only the last position is retained.

metadata

Relevant metadata for the individual device.

Type:

evefile.entities.metadata.AxisMetadata

set_values

Values the axis should have been set to.

While the Data.data attribute contains the actual positions of the axis, here, the originally intended positions are stored. This allows for easily checking whether the axis has been positioned within a scan as intended.

Type:

None

Examples

The AxisData class is not meant to be used directly, as any entities, but rather indirectly by means of the respective facades in the boundaries technical layer of the evefile package. Hence, for the time being, there are no dedicated examples how to use this class. Of course, you can instantiate an object as usual.

join(positions=None, fill=False, snapshot=None)

Perform a left join of the data on the provided list of positions.

The main “quantisation” axis of the values for a device and the common reference is the list of positions. To sensibly compare the data of different devices or plot different device data against each other, the data need to be harmonised, i.e. share a common set of positions as indices.

If positions are not present in the original data, by default, the corresponding entries will be masked and the data attribute converted into a numpy.ma.MaskedArray.

The reason for not using “NaN” (not a number) is, in short, that “NaN” is only defined for floating point numbers, but neither integers nor non-numeric values. Data, however, could generally contain values that are not floating point numbers. For a more detailed discussion, see the evefile.controllers.joining module.

Note

The method will alter the data and positions of the underlying MeasureData object. Hence, make sure to make a copy if this is not your intended use case.

Parameters:

• positions (numpy.ndarray) – Array with positions the data should be mapped to.

• fill (bool) –

  Whether to fill missing positions with previous values.

  Only in case a previous value exists for a given position (or a snapshot containing a previous value is provided as additional parameter), filling for the position will be performed. Otherwise, the position is masked.

• snapshot (AxisData) – Snapshot data corresponding to the original AxisData object.

Raises:

ValueError – Raised if no positions are provided

copy_attributes_from(source=None)

Obtain attributes from another Data object.

Sometimes, it is useful to obtain the (public) attributes from another Data object. Note that only public attributes are copied. Furthermore, a (true) copy of the attributes is obtained, hence the properties of source and target are actually different objects.

Parameters:

source (Data) –

Object to copy attributes from.

Should typically be of the same (super)type.

Raises:

ValueError – Raised if no source is provided to copy attributes from.

property data

Actual data recorded from the device.

Data are loaded only on demand. Hence, upon the first access of the data property, the get_data() method will be called, calling out to the respective importers.

Returns:

data – Actual data recorded from the device.

The actual data type (numpy.dtype) depends on the specific dataset loaded.

Return type:

numpy.ndarray

get_data()

Load data (and variable option data) using the respective importer.

Data are loaded only on demand. Hence, upon the first access of the data property, this method will be called, calling out to the respective importers.

As Data objects may contain (variable) options that are themselves data, but loading these data is only triggered when accessing the data property, you can either once access the data property or call this method.

Data may be spread over several HDF5 datasets, depending on the version of the eveH5 file read. Hence, there may be several importers, and they are dealt with sequentially.

Furthermore, for each importer type, there is a special private method _import_from_<importer-type>, with <importer-type> being the lowercase class name. Those classes using additional importers beyond HDF5DataImporter need to implement additional private methods to handle the special importer classes. A typical use case is the AreaChannelData class dealing with image data stored mostly in separate files.

get_dataframe()

Retrieve Pandas DataFrame with data as column.

The index is named “positions” and contains the values of the position_counts attribute of the data object.

Important

While working with a Pandas DataFrame may seem convenient, you’re loosing basically all the relevant metadata of the datasets. Hence, this method is rather a convenience method to be backwards-compatible to older interfaces, but it is explicitly not suggested for extensive use.

Returns:

dataframe – Pandas DataFrame containing data as column.

Return type:

pandas.DataFrame

property position_counts

Position counts data are recorded for.

Each data “point” corresponds to an overall position of all actuators (motor axes) of the setup and is assigned an individual “position count” (an integer number).

Actual data recorded from the device.

Data are loaded only on demand. Hence, upon the first access of the positions property, the get_data() method will be called, calling out to the respective importers.

Returns:

positions – Position values data are recorded for.

The actual data type (numpy.dtype) is usually int.

Return type:

numpy.ndarray

show_info()

Print basic information regarding the contents of a data object.

Often, it is convenient to get a brief overview of the contents of a data object. The output of this method currently contains the following sections:

• metadata

• options (if present)

• fields

The output could look similar to the following:

METADATA
name: jane

OPTIONS
some_option: value

FIELDS
data

Here, the METADATA block simply outputs what you would get with

print(data.metadata)

If options are present, then the keys of the options dict are returned in the OPTIONS block. Finally, the FIELDS block provides an overview of all the attributes containing some kind of data. This will differ depending on the type of data you are looking at.

class evefile.entities.data.ChannelData

Bases: MeasureData

Data from detector channels.

Three types of devices are distinguished by the eve measurement program: (dumb) devices, motor axes, and detector channels.

Note

Positions and (all) corresponding data are sorted upon load. In case of duplicate positions, only the first position is retained.

metadata

Relevant metadata for the individual device.

Type:

evefile.entities.metadata.ChannelMetadata

Examples

The ChannelData class is not meant to be used directly, as any entities, but rather indirectly by means of the respective facades in the boundaries technical layer of the evefile package. Hence, for the time being, there are no dedicated examples how to use this class. Of course, you can instantiate an object as usual.

copy_attributes_from(source=None)

Obtain attributes from another Data object.

Sometimes, it is useful to obtain the (public) attributes from another Data object. Note that only public attributes are copied. Furthermore, a (true) copy of the attributes is obtained, hence the properties of source and target are actually different objects.

Parameters:

source (Data) –

Object to copy attributes from.

Should typically be of the same (super)type.

Raises:

ValueError – Raised if no source is provided to copy attributes from.

property data

Actual data recorded from the device.

Data are loaded only on demand. Hence, upon the first access of the data property, the get_data() method will be called, calling out to the respective importers.

Returns:

data – Actual data recorded from the device.

The actual data type (numpy.dtype) depends on the specific dataset loaded.

Return type:

numpy.ndarray

get_data()

Load data (and variable option data) using the respective importer.

Data are loaded only on demand. Hence, upon the first access of the data property, this method will be called, calling out to the respective importers.

As Data objects may contain (variable) options that are themselves data, but loading these data is only triggered when accessing the data property, you can either once access the data property or call this method.

Data may be spread over several HDF5 datasets, depending on the version of the eveH5 file read. Hence, there may be several importers, and they are dealt with sequentially.

Furthermore, for each importer type, there is a special private method _import_from_<importer-type>, with <importer-type> being the lowercase class name. Those classes using additional importers beyond HDF5DataImporter need to implement additional private methods to handle the special importer classes. A typical use case is the AreaChannelData class dealing with image data stored mostly in separate files.

get_dataframe()

Retrieve Pandas DataFrame with data as column.

The index is named “positions” and contains the values of the position_counts attribute of the data object.

Important

While working with a Pandas DataFrame may seem convenient, you’re loosing basically all the relevant metadata of the datasets. Hence, this method is rather a convenience method to be backwards-compatible to older interfaces, but it is explicitly not suggested for extensive use.

Returns:

dataframe – Pandas DataFrame containing data as column.

Return type:

pandas.DataFrame

join(positions=None)

Perform a left join of the data on the provided list of positions.

The main “quantisation” axis of the values for a device and the common reference is the list of positions. To sensibly compare the data of different devices or plot different device data against each other, the data need to be harmonised, i.e. share a common set of positions as indices.

If positions are not present in the original data, by default, the corresponding entries will be masked and the data attribute converted into a numpy.ma.MaskedArray.

The reason for not using “NaN” (not a number) is, in short, that “NaN” is only defined for floating point numbers, but neither integers nor non-numeric values. Data, however, could generally contain values that are not floating point numbers. For a more detailed discussion, see the evefile.controllers.joining module.

Note

The method will alter the data and positions of the underlying MeasureData object. Hence, make sure to make a copy if this is not your intended use case.

Parameters:

positions (numpy.ndarray) – Array with positions the data should be mapped to.

Raises:

ValueError – Raised if no positions are provided

property position_counts

Position counts data are recorded for.

Each data “point” corresponds to an overall position of all actuators (motor axes) of the setup and is assigned an individual “position count” (an integer number).

Actual data recorded from the device.

Data are loaded only on demand. Hence, upon the first access of the positions property, the get_data() method will be called, calling out to the respective importers.

Returns:

positions – Position values data are recorded for.

The actual data type (numpy.dtype) is usually int.

Return type:

numpy.ndarray

show_info()

Print basic information regarding the contents of a data object.

Often, it is convenient to get a brief overview of the contents of a data object. The output of this method currently contains the following sections:

• metadata

• options (if present)

• fields

The output could look similar to the following:

METADATA
name: jane

OPTIONS
some_option: value

FIELDS
data

Here, the METADATA block simply outputs what you would get with

print(data.metadata)

If options are present, then the keys of the options dict are returned in the OPTIONS block. Finally, the FIELDS block provides an overview of all the attributes containing some kind of data. This will differ depending on the type of data you are looking at.

class evefile.entities.data.TimestampData

Bases: MeasureData

Data correlating the positions to the time used for monitors.

There are generally two different types of devices: those directly controlled by the eve engine, and those who are monitored. The former are instances of the MeasureData class, the latter of the MonitorData class.

The TimestampData class allows to map the time stamps (in milliseconds) of the MonitorData data to positions of MeasureData data. This is a necessary prerequisite to correlate monitored values to the data from controlled devices, such as motor axes and detector channels.

metadata

Relevant metadata for the individual device.

Type:

evefile.entities.metadata.TimestampMetadata

Examples

The TimestampData class is not meant to be used directly, as any entities, but rather indirectly by means of the respective facades in the boundaries technical layer of the evefile package. Hence, for the time being, there are no dedicated examples how to use this class. Of course, you can instantiate an object as usual.

get_position(time=-1)

Get position for a given (list of) time(stamp)s.

This method is used to map monitor timestamps to positions.

Note

Due to not being able to distinguish between axes and channels up to eveH5 v7, timestamps are generally mapped to the previous position.

Parameters:

time (int | list | numpy.ndarray) – Time(s) a position is requested for.

Returns:

positions – Position(s) corresponding to the timestamp(s) given.

Return type:

numpy.ndarray

copy_attributes_from(source=None)

Obtain attributes from another Data object.

Sometimes, it is useful to obtain the (public) attributes from another Data object. Note that only public attributes are copied. Furthermore, a (true) copy of the attributes is obtained, hence the properties of source and target are actually different objects.

Parameters:

source (Data) –

Object to copy attributes from.

Should typically be of the same (super)type.

Raises:

ValueError – Raised if no source is provided to copy attributes from.

property data

Actual data recorded from the device.

Data are loaded only on demand. Hence, upon the first access of the data property, the get_data() method will be called, calling out to the respective importers.

Returns:

data – Actual data recorded from the device.

The actual data type (numpy.dtype) depends on the specific dataset loaded.

Return type:

numpy.ndarray

get_data()

Load data (and variable option data) using the respective importer.

Data are loaded only on demand. Hence, upon the first access of the data property, this method will be called, calling out to the respective importers.

As Data objects may contain (variable) options that are themselves data, but loading these data is only triggered when accessing the data property, you can either once access the data property or call this method.

Data may be spread over several HDF5 datasets, depending on the version of the eveH5 file read. Hence, there may be several importers, and they are dealt with sequentially.

Furthermore, for each importer type, there is a special private method _import_from_<importer-type>, with <importer-type> being the lowercase class name. Those classes using additional importers beyond HDF5DataImporter need to implement additional private methods to handle the special importer classes. A typical use case is the AreaChannelData class dealing with image data stored mostly in separate files.

get_dataframe()

Retrieve Pandas DataFrame with data as column.

The index is named “positions” and contains the values of the position_counts attribute of the data object.

Important

While working with a Pandas DataFrame may seem convenient, you’re loosing basically all the relevant metadata of the datasets. Hence, this method is rather a convenience method to be backwards-compatible to older interfaces, but it is explicitly not suggested for extensive use.

Returns:

dataframe – Pandas DataFrame containing data as column.

Return type:

pandas.DataFrame

join(positions=None)

Perform a left join of the data on the provided list of positions.

The main “quantisation” axis of the values for a device and the common reference is the list of positions. To sensibly compare the data of different devices or plot different device data against each other, the data need to be harmonised, i.e. share a common set of positions as indices.

If positions are not present in the original data, by default, the corresponding entries will be masked and the data attribute converted into a numpy.ma.MaskedArray.

The reason for not using “NaN” (not a number) is, in short, that “NaN” is only defined for floating point numbers, but neither integers nor non-numeric values. Data, however, could generally contain values that are not floating point numbers. For a more detailed discussion, see the evefile.controllers.joining module.

Note

The method will alter the data and positions of the underlying MeasureData object. Hence, make sure to make a copy if this is not your intended use case.

Parameters:

positions (numpy.ndarray) – Array with positions the data should be mapped to.

Raises:

ValueError – Raised if no positions are provided

property position_counts

Position counts data are recorded for.

Each data “point” corresponds to an overall position of all actuators (motor axes) of the setup and is assigned an individual “position count” (an integer number).

Actual data recorded from the device.

Data are loaded only on demand. Hence, upon the first access of the positions property, the get_data() method will be called, calling out to the respective importers.

Returns:

positions – Position values data are recorded for.

The actual data type (numpy.dtype) is usually int.

Return type:

numpy.ndarray

show_info()

Print basic information regarding the contents of a data object.

Often, it is convenient to get a brief overview of the contents of a data object. The output of this method currently contains the following sections:

• metadata

• options (if present)

• fields

The output could look similar to the following:

METADATA
name: jane

OPTIONS
some_option: value

FIELDS
data

Here, the METADATA block simply outputs what you would get with

print(data.metadata)

If options are present, then the keys of the options dict are returned in the OPTIONS block. Finally, the FIELDS block provides an overview of all the attributes containing some kind of data. This will differ depending on the type of data you are looking at.

class evefile.entities.data.SinglePointChannelData

Bases: ChannelData

Data for channels with numeric 0D data.

Detector channels can be distinguished by the dimension of their data:

0D

scalar values per position, including average and interval channels

1D

array values, i.e. vectors, per position

2D

area values, i.e. images, per position

This class represents 0D, scalar values.

metadata

Relevant metadata for the individual device.

Type:

evefile.entities.metadata.SinglePointChannelMetadata

Examples

The SinglePointChannelData class is not meant to be used directly, as any entities, but rather indirectly by means of the respective facades in the boundaries technical layer of the evefile package. Hence, for the time being, there are no dedicated examples how to use this class. Of course, you can instantiate an object as usual.

copy_attributes_from(source=None)

Obtain attributes from another Data object.

Sometimes, it is useful to obtain the (public) attributes from another Data object. Note that only public attributes are copied. Furthermore, a (true) copy of the attributes is obtained, hence the properties of source and target are actually different objects.

Parameters:

source (Data) –

Object to copy attributes from.

Should typically be of the same (super)type.

Raises:

ValueError – Raised if no source is provided to copy attributes from.

property data

Actual data recorded from the device.

Data are loaded only on demand. Hence, upon the first access of the data property, the get_data() method will be called, calling out to the respective importers.

Returns:

data – Actual data recorded from the device.

The actual data type (numpy.dtype) depends on the specific dataset loaded.

Return type:

numpy.ndarray

get_data()

Load data (and variable option data) using the respective importer.

Data are loaded only on demand. Hence, upon the first access of the data property, this method will be called, calling out to the respective importers.

As Data objects may contain (variable) options that are themselves data, but loading these data is only triggered when accessing the data property, you can either once access the data property or call this method.

Data may be spread over several HDF5 datasets, depending on the version of the eveH5 file read. Hence, there may be several importers, and they are dealt with sequentially.

Furthermore, for each importer type, there is a special private method _import_from_<importer-type>, with <importer-type> being the lowercase class name. Those classes using additional importers beyond HDF5DataImporter need to implement additional private methods to handle the special importer classes. A typical use case is the AreaChannelData class dealing with image data stored mostly in separate files.

get_dataframe()

Retrieve Pandas DataFrame with data as column.

The index is named “positions” and contains the values of the position_counts attribute of the data object.

Important

While working with a Pandas DataFrame may seem convenient, you’re loosing basically all the relevant metadata of the datasets. Hence, this method is rather a convenience method to be backwards-compatible to older interfaces, but it is explicitly not suggested for extensive use.

Returns:

dataframe – Pandas DataFrame containing data as column.

Return type:

pandas.DataFrame

join(positions=None)

Perform a left join of the data on the provided list of positions.

The main “quantisation” axis of the values for a device and the common reference is the list of positions. To sensibly compare the data of different devices or plot different device data against each other, the data need to be harmonised, i.e. share a common set of positions as indices.

If positions are not present in the original data, by default, the corresponding entries will be masked and the data attribute converted into a numpy.ma.MaskedArray.

The reason for not using “NaN” (not a number) is, in short, that “NaN” is only defined for floating point numbers, but neither integers nor non-numeric values. Data, however, could generally contain values that are not floating point numbers. For a more detailed discussion, see the evefile.controllers.joining module.

Note

The method will alter the data and positions of the underlying MeasureData object. Hence, make sure to make a copy if this is not your intended use case.

Parameters:

positions (numpy.ndarray) – Array with positions the data should be mapped to.

Raises:

ValueError – Raised if no positions are provided

property position_counts

Position counts data are recorded for.

Each data “point” corresponds to an overall position of all actuators (motor axes) of the setup and is assigned an individual “position count” (an integer number).

Actual data recorded from the device.

Data are loaded only on demand. Hence, upon the first access of the positions property, the get_data() method will be called, calling out to the respective importers.

Returns:

positions – Position values data are recorded for.

The actual data type (numpy.dtype) is usually int.

Return type:

numpy.ndarray

show_info()

Print basic information regarding the contents of a data object.

Often, it is convenient to get a brief overview of the contents of a data object. The output of this method currently contains the following sections:

• metadata

• options (if present)

• fields

The output could look similar to the following:

METADATA
name: jane

OPTIONS
some_option: value

FIELDS
data

Here, the METADATA block simply outputs what you would get with

print(data.metadata)

If options are present, then the keys of the options dict are returned in the OPTIONS block. Finally, the FIELDS block provides an overview of all the attributes containing some kind of data. This will differ depending on the type of data you are looking at.

class evefile.entities.data.AverageChannelData

Bases: ChannelData

Data for channels with averaged numeric 0D data.

Detector channels can be distinguished by the dimension of their data:

0D

scalar values per position, including average and interval channels

1D

array values, i.e. vectors, per position

2D

area values, i.e. images, per position

This class represents 0D, scalar values that are averaged.

metadata

Relevant metadata for the individual device.

Type:

evefile.entities.metadata.AverageChannelMetadata

Examples

The AverageChannelData class is not meant to be used directly, as any entities, but rather indirectly by means of the respective facades in the boundaries technical layer of the evefile package. Hence, for the time being, there are no dedicated examples how to use this class. Of course, you can instantiate an object as usual.

property attempts

Number of attempts needed until final data were recorded.

Returns:

attempts – Number of attempts

Return type:

numpy.ndarray

property mean

Mean values for channel data.

Returns:

mean – The mean of the values recorded.

As at least up to eveH5 v7.x only the averaged values are stored in the HDF5 file, this simply returns the values stored in data.

Return type:

numpy.ndarray

copy_attributes_from(source=None)

Obtain attributes from another Data object.

Sometimes, it is useful to obtain the (public) attributes from another Data object. Note that only public attributes are copied. Furthermore, a (true) copy of the attributes is obtained, hence the properties of source and target are actually different objects.

Parameters:

source (Data) –

Object to copy attributes from.

Should typically be of the same (super)type.

Raises:

ValueError – Raised if no source is provided to copy attributes from.

property data

Actual data recorded from the device.

Data are loaded only on demand. Hence, upon the first access of the data property, the get_data() method will be called, calling out to the respective importers.

Returns:

data – Actual data recorded from the device.

The actual data type (numpy.dtype) depends on the specific dataset loaded.

Return type:

numpy.ndarray

get_data()

Load data (and variable option data) using the respective importer.

Data are loaded only on demand. Hence, upon the first access of the data property, this method will be called, calling out to the respective importers.

As Data objects may contain (variable) options that are themselves data, but loading these data is only triggered when accessing the data property, you can either once access the data property or call this method.

Data may be spread over several HDF5 datasets, depending on the version of the eveH5 file read. Hence, there may be several importers, and they are dealt with sequentially.

Furthermore, for each importer type, there is a special private method _import_from_<importer-type>, with <importer-type> being the lowercase class name. Those classes using additional importers beyond HDF5DataImporter need to implement additional private methods to handle the special importer classes. A typical use case is the AreaChannelData class dealing with image data stored mostly in separate files.

get_dataframe()

Retrieve Pandas DataFrame with data as column.

The index is named “positions” and contains the values of the position_counts attribute of the data object.

Important

While working with a Pandas DataFrame may seem convenient, you’re loosing basically all the relevant metadata of the datasets. Hence, this method is rather a convenience method to be backwards-compatible to older interfaces, but it is explicitly not suggested for extensive use.

Returns:

dataframe – Pandas DataFrame containing data as column.

Return type:

pandas.DataFrame

join(positions=None)

Perform a left join of the data on the provided list of positions.

The main “quantisation” axis of the values for a device and the common reference is the list of positions. To sensibly compare the data of different devices or plot different device data against each other, the data need to be harmonised, i.e. share a common set of positions as indices.

If positions are not present in the original data, by default, the corresponding entries will be masked and the data attribute converted into a numpy.ma.MaskedArray.

The reason for not using “NaN” (not a number) is, in short, that “NaN” is only defined for floating point numbers, but neither integers nor non-numeric values. Data, however, could generally contain values that are not floating point numbers. For a more detailed discussion, see the evefile.controllers.joining module.

Note

The method will alter the data and positions of the underlying MeasureData object. Hence, make sure to make a copy if this is not your intended use case.

Parameters:

positions (numpy.ndarray) – Array with positions the data should be mapped to.

Raises:

ValueError – Raised if no positions are provided

property position_counts

Position counts data are recorded for.

Each data “point” corresponds to an overall position of all actuators (motor axes) of the setup and is assigned an individual “position count” (an integer number).

Actual data recorded from the device.

Data are loaded only on demand. Hence, upon the first access of the positions property, the get_data() method will be called, calling out to the respective importers.

Returns:

positions – Position values data are recorded for.

The actual data type (numpy.dtype) is usually int.

Return type:

numpy.ndarray

show_info()

Print basic information regarding the contents of a data object.

Often, it is convenient to get a brief overview of the contents of a data object. The output of this method currently contains the following sections:

• metadata

• options (if present)

• fields

The output could look similar to the following:

METADATA
name: jane

OPTIONS
some_option: value

FIELDS
data

Here, the METADATA block simply outputs what you would get with

print(data.metadata)

If options are present, then the keys of the options dict are returned in the OPTIONS block. Finally, the FIELDS block provides an overview of all the attributes containing some kind of data. This will differ depending on the type of data you are looking at.

class evefile.entities.data.IntervalChannelData

Bases: ChannelData

Data for channels with numeric 0D data measured in a time interval.

Detector channels can be distinguished by the dimension of their data:

0D

scalar values per position, including average and interval channels

1D

array values, i.e. vectors, per position

2D

area values, i.e. images, per position

This class represents 0D, scalar values that are measured in a time interval.

metadata

Relevant metadata for the individual device.

Type:

evefile.entities.metadata.IntervalChannelMetadata

Examples

The IntervalChannelData class is not meant to be used directly, as any entities, but rather indirectly by means of the respective facades in the boundaries technical layer of the evefile package. Hence, for the time being, there are no dedicated examples how to use this class. Of course, you can instantiate an object as usual.

property counts

The number of values measured in the given time interval.

Returns:

counts – Number of values measured in the given time interval

Return type:

numpy.ndarray

property std

Standard deviation values for channel data.

Returns:

std – Standard deviation values for channel data.

Return type:

numpy.ndarray

property mean

Mean values for channel data.

Returns:

mean – The mean of the values measured in the given time interval.

As at least up to eveH5 v7.x only the averaged values are stored in the HDF5 file, this simply returns the values stored in data.

Return type:

numpy.ndarray

copy_attributes_from(source=None)

Obtain attributes from another Data object.

Sometimes, it is useful to obtain the (public) attributes from another Data object. Note that only public attributes are copied. Furthermore, a (true) copy of the attributes is obtained, hence the properties of source and target are actually different objects.

Parameters:

source (Data) –

Object to copy attributes from.

Should typically be of the same (super)type.

Raises:

ValueError – Raised if no source is provided to copy attributes from.

property data

Actual data recorded from the device.

Data are loaded only on demand. Hence, upon the first access of the data property, the get_data() method will be called, calling out to the respective importers.

Returns:

data – Actual data recorded from the device.

The actual data type (numpy.dtype) depends on the specific dataset loaded.

Return type:

numpy.ndarray

get_data()

Load data (and variable option data) using the respective importer.

Data are loaded only on demand. Hence, upon the first access of the data property, this method will be called, calling out to the respective importers.

As Data objects may contain (variable) options that are themselves data, but loading these data is only triggered when accessing the data property, you can either once access the data property or call this method.

Data may be spread over several HDF5 datasets, depending on the version of the eveH5 file read. Hence, there may be several importers, and they are dealt with sequentially.

Furthermore, for each importer type, there is a special private method _import_from_<importer-type>, with <importer-type> being the lowercase class name. Those classes using additional importers beyond HDF5DataImporter need to implement additional private methods to handle the special importer classes. A typical use case is the AreaChannelData class dealing with image data stored mostly in separate files.

get_dataframe()

Retrieve Pandas DataFrame with data as column.

The index is named “positions” and contains the values of the position_counts attribute of the data object.

Important

While working with a Pandas DataFrame may seem convenient, you’re loosing basically all the relevant metadata of the datasets. Hence, this method is rather a convenience method to be backwards-compatible to older interfaces, but it is explicitly not suggested for extensive use.

Returns:

dataframe – Pandas DataFrame containing data as column.

Return type:

pandas.DataFrame

join(positions=None)

Perform a left join of the data on the provided list of positions.

The main “quantisation” axis of the values for a device and the common reference is the list of positions. To sensibly compare the data of different devices or plot different device data against each other, the data need to be harmonised, i.e. share a common set of positions as indices.

If positions are not present in the original data, by default, the corresponding entries will be masked and the data attribute converted into a numpy.ma.MaskedArray.

The reason for not using “NaN” (not a number) is, in short, that “NaN” is only defined for floating point numbers, but neither integers nor non-numeric values. Data, however, could generally contain values that are not floating point numbers. For a more detailed discussion, see the evefile.controllers.joining module.

Note

The method will alter the data and positions of the underlying MeasureData object. Hence, make sure to make a copy if this is not your intended use case.

Parameters:

positions (numpy.ndarray) – Array with positions the data should be mapped to.

Raises:

ValueError – Raised if no positions are provided

property position_counts

Position counts data are recorded for.

Each data “point” corresponds to an overall position of all actuators (motor axes) of the setup and is assigned an individual “position count” (an integer number).

Actual data recorded from the device.

Data are loaded only on demand. Hence, upon the first access of the positions property, the get_data() method will be called, calling out to the respective importers.

Returns:

positions – Position values data are recorded for.

The actual data type (numpy.dtype) is usually int.

Return type:

numpy.ndarray

show_info()

Print basic information regarding the contents of a data object.

Often, it is convenient to get a brief overview of the contents of a data object. The output of this method currently contains the following sections:

• metadata

• options (if present)

• fields

The output could look similar to the following:

METADATA
name: jane

OPTIONS
some_option: value

FIELDS
data

Here, the METADATA block simply outputs what you would get with

print(data.metadata)

If options are present, then the keys of the options dict are returned in the OPTIONS block. Finally, the FIELDS block provides an overview of all the attributes containing some kind of data. This will differ depending on the type of data you are looking at.

class evefile.entities.data.NormalizedChannelData

Bases: object

Mixin class (interface) for normalized channel data.

0D channels can be normalized by the data of another 0D channel, i.e. by dividing its values by the values of the normalizing channel.

metadata

Relevant metadata for normalization.

Type:

evefile.entities.metadata.AreaChannelMetadata

Raises:

exception – Short description when and why raised

Examples

The AreaChannelData class is not meant to be used directly, as any entities, but rather indirectly by means of the respective facades in the boundaries technical layer of the evefile package. Hence, for the time being, there are no dedicated examples how to use this class. Of course, you can instantiate an object as usual.

property normalized_data

Data that have been normalized.

Normalization takes place by dividing by the values of the normalizing channel.

Returns:

normalized_data – Data that have been normalized.

Return type:

Any

property normalizing_data

Data used for normalization.

Returns:

normalized_data – Data used for normalization.

Return type:

Any

class evefile.entities.data.SinglePointNormalizedChannelData

Bases: SinglePointChannelData, NormalizedChannelData

Data for channels with normalized numeric 0D data.

Detector channels can be distinguished by the dimension of their data:

0D

scalar values per position, including average and interval channels

1D

array values, i.e. vectors, per position

2D

area values, i.e. images, per position

This class represents 0D, scalar values that are normalized by the data of another 0D channel.

metadata

Relevant metadata for the individual normalized device.

Type:

evefile.entities.metadata.SinglePointNormalizedChannelMetadata

Examples

The SinglePointNormalizedChannelData class is not meant to be used directly, as any entities, but rather indirectly by means of the respective facades in the boundaries technical layer of the evefile package. Hence, for the time being, there are no dedicated examples how to use this class. Of course, you can instantiate an object as usual.

property normalized_data

Data that have been normalized.

Normalization takes place by dividing by the values of the normalizing channel.

Returns:

normalized_data – Data that have been normalized.

Return type:

Any

property normalizing_data

Data used for normalization.

Returns:

normalized_data – Data used for normalization.

Return type:

Any

copy_attributes_from(source=None)

Obtain attributes from another Data object.

Sometimes, it is useful to obtain the (public) attributes from another Data object. Note that only public attributes are copied. Furthermore, a (true) copy of the attributes is obtained, hence the properties of source and target are actually different objects.

Parameters:

source (Data) –

Object to copy attributes from.

Should typically be of the same (super)type.

Raises:

ValueError – Raised if no source is provided to copy attributes from.

property data

Actual data recorded from the device.

Data are loaded only on demand. Hence, upon the first access of the data property, the get_data() method will be called, calling out to the respective importers.

Returns:

data – Actual data recorded from the device.

The actual data type (numpy.dtype) depends on the specific dataset loaded.

Return type:

numpy.ndarray

get_data()

Load data (and variable option data) using the respective importer.

Data are loaded only on demand. Hence, upon the first access of the data property, this method will be called, calling out to the respective importers.

As Data objects may contain (variable) options that are themselves data, but loading these data is only triggered when accessing the data property, you can either once access the data property or call this method.

Data may be spread over several HDF5 datasets, depending on the version of the eveH5 file read. Hence, there may be several importers, and they are dealt with sequentially.

Furthermore, for each importer type, there is a special private method _import_from_<importer-type>, with <importer-type> being the lowercase class name. Those classes using additional importers beyond HDF5DataImporter need to implement additional private methods to handle the special importer classes. A typical use case is the AreaChannelData class dealing with image data stored mostly in separate files.

get_dataframe()

Retrieve Pandas DataFrame with data as column.

The index is named “positions” and contains the values of the position_counts attribute of the data object.

Important

While working with a Pandas DataFrame may seem convenient, you’re loosing basically all the relevant metadata of the datasets. Hence, this method is rather a convenience method to be backwards-compatible to older interfaces, but it is explicitly not suggested for extensive use.

Returns:

dataframe – Pandas DataFrame containing data as column.

Return type:

pandas.DataFrame

join(positions=None)

Perform a left join of the data on the provided list of positions.

The main “quantisation” axis of the values for a device and the common reference is the list of positions. To sensibly compare the data of different devices or plot different device data against each other, the data need to be harmonised, i.e. share a common set of positions as indices.

If positions are not present in the original data, by default, the corresponding entries will be masked and the data attribute converted into a numpy.ma.MaskedArray.

The reason for not using “NaN” (not a number) is, in short, that “NaN” is only defined for floating point numbers, but neither integers nor non-numeric values. Data, however, could generally contain values that are not floating point numbers. For a more detailed discussion, see the evefile.controllers.joining module.

Note

The method will alter the data and positions of the underlying MeasureData object. Hence, make sure to make a copy if this is not your intended use case.

Parameters:

positions (numpy.ndarray) – Array with positions the data should be mapped to.

Raises:

ValueError – Raised if no positions are provided

property position_counts

Position counts data are recorded for.

Each data “point” corresponds to an overall position of all actuators (motor axes) of the setup and is assigned an individual “position count” (an integer number).

Actual data recorded from the device.

Data are loaded only on demand. Hence, upon the first access of the positions property, the get_data() method will be called, calling out to the respective importers.

Returns:

positions – Position values data are recorded for.

The actual data type (numpy.dtype) is usually int.

Return type:

numpy.ndarray

show_info()

Print basic information regarding the contents of a data object.

Often, it is convenient to get a brief overview of the contents of a data object. The output of this method currently contains the following sections:

• metadata

• options (if present)

• fields

The output could look similar to the following:

METADATA
name: jane

OPTIONS
some_option: value

FIELDS
data

Here, the METADATA block simply outputs what you would get with

print(data.metadata)

If options are present, then the keys of the options dict are returned in the OPTIONS block. Finally, the FIELDS block provides an overview of all the attributes containing some kind of data. This will differ depending on the type of data you are looking at.

class evefile.entities.data.AverageNormalizedChannelData

Bases: AverageChannelData, NormalizedChannelData

Data for channels with normalized averaged numeric 0D data.

Detector channels can be distinguished by the dimension of their data:

0D

scalar values per position, including average and interval channels

1D

array values, i.e. vectors, per position

2D

area values, i.e. images, per position

This class represents 0D, scalar values that are averaged and normalized by the data of another 0D channel.

metadata

Relevant metadata for the individual normalized device.

Type:

evefile.entities.metadata.AverageNormalizedChannelMetadata

Examples

The AverageNormalizedChannelData class is not meant to be used directly, as any entities, but rather indirectly by means of the respective facades in the boundaries technical layer of the evefile package. Hence, for the time being, there are no dedicated examples how to use this class. Of course, you can instantiate an object as usual.

property normalized_data

Data that have been normalized.

Normalization takes place by dividing by the values of the normalizing channel.

Returns:

normalized_data – Data that have been normalized.

Return type:

Any

property normalizing_data

Data used for normalization.

Returns:

normalized_data – Data used for normalization.

Return type:

Any

property attempts

Number of attempts needed until final data were recorded.

Returns:

attempts – Number of attempts

Return type:

numpy.ndarray

copy_attributes_from(source=None)

Obtain attributes from another Data object.

Sometimes, it is useful to obtain the (public) attributes from another Data object. Note that only public attributes are copied. Furthermore, a (true) copy of the attributes is obtained, hence the properties of source and target are actually different objects.

Parameters:

source (Data) –

Object to copy attributes from.

Should typically be of the same (super)type.

Raises:

ValueError – Raised if no source is provided to copy attributes from.

property data

Actual data recorded from the device.

Data are loaded only on demand. Hence, upon the first access of the data property, the get_data() method will be called, calling out to the respective importers.

Returns:

data – Actual data recorded from the device.

The actual data type (numpy.dtype) depends on the specific dataset loaded.

Return type:

numpy.ndarray

get_data()

Load data (and variable option data) using the respective importer.

Data are loaded only on demand. Hence, upon the first access of the data property, this method will be called, calling out to the respective importers.

As Data objects may contain (variable) options that are themselves data, but loading these data is only triggered when accessing the data property, you can either once access the data property or call this method.

Data may be spread over several HDF5 datasets, depending on the version of the eveH5 file read. Hence, there may be several importers, and they are dealt with sequentially.

Furthermore, for each importer type, there is a special private method _import_from_<importer-type>, with <importer-type> being the lowercase class name. Those classes using additional importers beyond HDF5DataImporter need to implement additional private methods to handle the special importer classes. A typical use case is the AreaChannelData class dealing with image data stored mostly in separate files.

get_dataframe()

Retrieve Pandas DataFrame with data as column.

The index is named “positions” and contains the values of the position_counts attribute of the data object.

Important

While working with a Pandas DataFrame may seem convenient, you’re loosing basically all the relevant metadata of the datasets. Hence, this method is rather a convenience method to be backwards-compatible to older interfaces, but it is explicitly not suggested for extensive use.

Returns:

dataframe – Pandas DataFrame containing data as column.

Return type:

pandas.DataFrame

join(positions=None)

Perform a left join of the data on the provided list of positions.

The main “quantisation” axis of the values for a device and the common reference is the list of positions. To sensibly compare the data of different devices or plot different device data against each other, the data need to be harmonised, i.e. share a common set of positions as indices.

If positions are not present in the original data, by default, the corresponding entries will be masked and the data attribute converted into a numpy.ma.MaskedArray.

The reason for not using “NaN” (not a number) is, in short, that “NaN” is only defined for floating point numbers, but neither integers nor non-numeric values. Data, however, could generally contain values that are not floating point numbers. For a more detailed discussion, see the evefile.controllers.joining module.

Note

The method will alter the data and positions of the underlying MeasureData object. Hence, make sure to make a copy if this is not your intended use case.

Parameters:

positions (numpy.ndarray) – Array with positions the data should be mapped to.

Raises:

ValueError – Raised if no positions are provided

property mean

Mean values for channel data.

Returns:

mean – The mean of the values recorded.

As at least up to eveH5 v7.x only the averaged values are stored in the HDF5 file, this simply returns the values stored in data.

Return type:

numpy.ndarray

property position_counts

Position counts data are recorded for.

Each data “point” corresponds to an overall position of all actuators (motor axes) of the setup and is assigned an individual “position count” (an integer number).

Actual data recorded from the device.

Data are loaded only on demand. Hence, upon the first access of the positions property, the get_data() method will be called, calling out to the respective importers.

Returns:

positions – Position values data are recorded for.

The actual data type (numpy.dtype) is usually int.

Return type:

numpy.ndarray

show_info()

Print basic information regarding the contents of a data object.

Often, it is convenient to get a brief overview of the contents of a data object. The output of this method currently contains the following sections:

• metadata

• options (if present)

• fields

The output could look similar to the following:

METADATA
name: jane

OPTIONS
some_option: value

FIELDS
data

Here, the METADATA block simply outputs what you would get with

print(data.metadata)

If options are present, then the keys of the options dict are returned in the OPTIONS block. Finally, the FIELDS block provides an overview of all the attributes containing some kind of data. This will differ depending on the type of data you are looking at.

class evefile.entities.data.IntervalNormalizedChannelData

Bases: IntervalChannelData, NormalizedChannelData

Data for channels with normalized interval-measured numeric 0D data.

Detector channels can be distinguished by the dimension of their data:

0D

scalar values per position, including average and interval channels

1D

array values, i.e. vectors, per position

2D

area values, i.e. images, per position

This class represents 0D, scalar values that are measured in a time interval and normalized by the data of another 0D channel.

metadata

Relevant metadata for the individual normalized device.

Type:

evefile.entities.metadata.IntervalNormalizedChannelMetadata

Examples

The IntervalNormalizedChannelData class is not meant to be used directly, as any entities, but rather indirectly by means of the respective facades in the boundaries technical layer of the evefile package. Hence, for the time being, there are no dedicated examples how to use this class. Of course, you can instantiate an object as usual.

property normalized_data

Data that have been normalized.

Normalization takes place by dividing by the values of the normalizing channel.

Returns:

normalized_data – Data that have been normalized.

Return type:

Any

property normalizing_data

Data used for normalization.

Returns:

normalized_data – Data used for normalization.

Return type:

Any

copy_attributes_from(source=None)

Obtain attributes from another Data object.

Sometimes, it is useful to obtain the (public) attributes from another Data object. Note that only public attributes are copied. Furthermore, a (true) copy of the attributes is obtained, hence the properties of source and target are actually different objects.

Parameters:

source (Data) –

Object to copy attributes from.

Should typically be of the same (super)type.

Raises:

ValueError – Raised if no source is provided to copy attributes from.

property counts

The number of values measured in the given time interval.

Returns:

counts – Number of values measured in the given time interval

Return type:

numpy.ndarray

property data

Actual data recorded from the device.

Data are loaded only on demand. Hence, upon the first access of the data property, the get_data() method will be called, calling out to the respective importers.

Returns:

data – Actual data recorded from the device.

The actual data type (numpy.dtype) depends on the specific dataset loaded.

Return type:

numpy.ndarray

get_data()

Load data (and variable option data) using the respective importer.

Data are loaded only on demand. Hence, upon the first access of the data property, this method will be called, calling out to the respective importers.

As Data objects may contain (variable) options that are themselves data, but loading these data is only triggered when accessing the data property, you can either once access the data property or call this method.

Data may be spread over several HDF5 datasets, depending on the version of the eveH5 file read. Hence, there may be several importers, and they are dealt with sequentially.

Furthermore, for each importer type, there is a special private method _import_from_<importer-type>, with <importer-type> being the lowercase class name. Those classes using additional importers beyond HDF5DataImporter need to implement additional private methods to handle the special importer classes. A typical use case is the AreaChannelData class dealing with image data stored mostly in separate files.

get_dataframe()

Retrieve Pandas DataFrame with data as column.

The index is named “positions” and contains the values of the position_counts attribute of the data object.

Important

While working with a Pandas DataFrame may seem convenient, you’re loosing basically all the relevant metadata of the datasets. Hence, this method is rather a convenience method to be backwards-compatible to older interfaces, but it is explicitly not suggested for extensive use.

Returns:

dataframe – Pandas DataFrame containing data as column.

Return type:

pandas.DataFrame

join(positions=None)

Perform a left join of the data on the provided list of positions.

The main “quantisation” axis of the values for a device and the common reference is the list of positions. To sensibly compare the data of different devices or plot different device data against each other, the data need to be harmonised, i.e. share a common set of positions as indices.

If positions are not present in the original data, by default, the corresponding entries will be masked and the data attribute converted into a numpy.ma.MaskedArray.

The reason for not using “NaN” (not a number) is, in short, that “NaN” is only defined for floating point numbers, but neither integers nor non-numeric values. Data, however, could generally contain values that are not floating point numbers. For a more detailed discussion, see the evefile.controllers.joining module.

Note

The method will alter the data and positions of the underlying MeasureData object. Hence, make sure to make a copy if this is not your intended use case.

Parameters:

positions (numpy.ndarray) – Array with positions the data should be mapped to.

Raises:

ValueError – Raised if no positions are provided

property mean

Mean values for channel data.

Returns:

mean – The mean of the values measured in the given time interval.

As at least up to eveH5 v7.x only the averaged values are stored in the HDF5 file, this simply returns the values stored in data.

Return type:

numpy.ndarray

property position_counts

Position counts data are recorded for.

Each data “point” corresponds to an overall position of all actuators (motor axes) of the setup and is assigned an individual “position count” (an integer number).

Actual data recorded from the device.

Data are loaded only on demand. Hence, upon the first access of the positions property, the get_data() method will be called, calling out to the respective importers.

Returns:

positions – Position values data are recorded for.

The actual data type (numpy.dtype) is usually int.

Return type:

numpy.ndarray

show_info()

Print basic information regarding the contents of a data object.

Often, it is convenient to get a brief overview of the contents of a data object. The output of this method currently contains the following sections:

• metadata

• options (if present)

• fields

The output could look similar to the following:

METADATA
name: jane

OPTIONS
some_option: value

FIELDS
data

Here, the METADATA block simply outputs what you would get with

print(data.metadata)

If options are present, then the keys of the options dict are returned in the OPTIONS block. Finally, the FIELDS block provides an overview of all the attributes containing some kind of data. This will differ depending on the type of data you are looking at.

property std

Standard deviation values for channel data.

Returns:

std – Standard deviation values for channel data.

Return type:

numpy.ndarray

class evefile.entities.data.DataImporter(source='')

Bases: object

Base class for data importer.

Data need to be imported from somewhere. And usually, data should only be imported once they are requested, to save time and resources.

Actual importer classes inherit from this base class and implement the private method _load(). This method simply returns the loaded data.

Optionally, preprocessing will be applied to the data loaded, if the list preprocessing is not empty.

source

Source the data should be loaded from.

Typically, a file name.

Type:

str

preprocessing

Preprocessing steps applied after loading the original data.

Each entry in the list is an object of type ImporterPreprocessingStep.

Type:

list

Raises:

ValueError – Raised upon load if no source is provided.

Examples

While this base class is not intended to be used directly, the general usage is the same for all descendants:

importer = DataImporter()
data = importer.import(source="myfilename")

For convenience, you can set the source when instantiating the object. This makes actually importing simpler, not having to worry anymore about the source:

importer = DataImporter(source="myfilename")
data = importer.import()

load(source='')

Load data from source.

The method first checks for the source to be present, and afterwards calls out to the private method _load() that does the actual business. Child classes hence need to implement this private method. Make sure to return the loaded data from this method.

Once loaded, the data are preprocessed with each of the preprocessing steps defined in preprocessing.

Parameters:

source (str) –

Source the data should be loaded from.

Typically, a file name.

Raises:

ValueError – Raised if no source is provided.

Returns:

data – Data loaded from the source.

The actual type of data depends on the source and importer type.

Return type:

any

class evefile.entities.data.HDF5DataImporter(source='')

Bases: DataImporter

Load data from HDF5 dataset.

HDF5 files are organised hierarchically, with groups as nodes and datasets as leafs. Data can (only) be contained in datasets, and this is what this importer is concerned about.

Note

Perhaps it is possible to move this class to the boundary technical layer, by means of creating an (abstract) DataImporterFactory in the entities layer and a concrete factory in the boundary layer. The only complication currently: the controller technical layer needs to access the concrete DataImporterFactory.

source

Source the data should be loaded from.

Name of an HDF5 file.

Type:

str

item

The dataset within the HDF5 file.

Datasets are addressed by a path-like string, with slashes separating the hierarchy levels in the file.

Type:

str

mapping

Mapping table for table columns to Data object attributes.

HDF5 datasets in eveH5 files usually consist of at least two columns for their data, the first either the position or the time since start of the measurement in milliseconds. Besides this, there can be more than one additional column for the actual data. As the structure of the datasets changed and will change, there is a need for a mapping table that gets filled properly by the VersionMapper class.

Furthermore, storing this mapping information is relevant as data are usually only loaded upon request, not preliminary, to save time and resources.

Type:

dict

data

Data loaded from the HDF5 dataset.

The actual data type (numpy.dtype) depends on the specific dataset loaded.

Type:

numpy.ndarray

Raises:

ValueError – Raised upon load if either source or item are not provided.

Examples

To import data from an HDF5 dataset located in an HDF5 file, you need to provide both, file name (source) and dataset name (item):

importer = HDF5DataImporter()
importer.source = "test.h5"
importer.item = "/c1/main/test"
data = importer.load()

You can, for convenience, provide both, source and item upon instantiating the importer object:

importer = HDF5DataImporter(source="test.h5", item="/c1/main/test")
data = importer.load()

load(source='', item='')

Load data from source.

The method first checks for the source to be present, and afterwards calls out to the private method _load() that does the actual business. Child classes hence need to implement this private method. Make sure to return the loaded data from this method.

Besides returning the data (for convenience), they are set to the data attribute for later access.

Parameters:

• source (str) –

  Source the data should be loaded from.

  Name of an HDF5 file.

• item (str) –

  The dataset within the HDF5 file.

  Datasets are addressed by a path-like string, with slashes separating the hierarchy levels in the file.

Raises:

ValueError – Raised if either source or item are not provided.

Returns:

data – Data loaded from the HDF5 dataset.

The actual data type (numpy.dtype) depends on the specific dataset loaded.

Return type:

numpy.ndarray