qudi.util.datastorage

This file contains data storage utilities for qudi.

Functions

get_timestamp_filename(timestamp[, nametag])	Returns a qudi standard filename used for saving measurement data to a file, not including any file extension.
format_column_headers(column_headers[, ...])
format_header(timestamp[, number_format, ...])
metadata_to_str_dict(metadata)
str_dict_to_metadata(str_dict)
get_header_from_file(file_path)
get_info_from_header(header)
create_dir_for_file(file_path)	Helper method to create the directory (recursively) for a given file path.

Classes

CsvDataStorage(*[, file_extension])	Helper class to store (measurement)data on disk as CSV file.
DataStorageBase(*[, root_dir, ...])	Base helper class to store/load (measurement)data to/from disk.
ImageFormat(value)	Image format to use for saving data thumbnails.
NpyDataStorage(*, root_dir, **kwargs)	Helper class to store (measurement)data on disk as binary .npy file.
TextDataStorage(*, root_dir[, comments, ...])	Helper class to store (measurement)data on disk as text file.

class qudi.util.datastorage.CsvDataStorage(*, file_extension='.csv', **kwargs)

Bases: qudi.util.datastorage.TextDataStorage

Helper class to store (measurement)data on disk as CSV file. This is a specialized sub-class of TextDataStorage that uses hard-coded commas as delimiter and includes column headers uncommented in the first row of data. This is the standard format for importing a table into e.g. MS Excel.

__init__(*, file_extension='.csv', **kwargs)

See: qudi.util.datastorage.TextDataStorage

create_header(timestamp=None, metadata=None, notes=None, column_headers=None, column_dtypes=None)

Include column_headers without line comment specifier. for more information see: qudi.util.datastorage.TextDataStorage.create_header()

property delimiter

!! processed by numpydoc !!

static load_data(file_path)

See load_data() for more information.

Parameters:

file_pathstr, optional

Path to the file to load data from.

class qudi.util.datastorage.DataStorageBase(*, root_dir=None, include_global_metadata=True, image_format=ImageFormat.PNG)

Bases: builtins.object

Base helper class to store/load (measurement)data to/from disk. Subclasses handle saving and loading of measurement data (including metadata) for specific file formats. Metadata is represented as dictionary (key-value pairs). It is also possible to set so called “global metadata” using this or any subclass of this class. Global metadata is shared and accessible throughout all instances of these storage objects within the Python process.

If the storage type is file based and root_dir is not initialized, each call to save_data must provide the full save path information and not just a file name or name tag.

__init__(*, root_dir=None, include_global_metadata=True, image_format=ImageFormat.PNG)

Parameters:

root_dirstr, optional

Root directory for this storage instance to work in.

include_global_metadatabool, optional

Flag indicating whether to save global metadata.

image_formatImageFormat, optional

Image file format Enum for saving thumbnails.

classmethod add_global_metadata(name, value=None, *, overwrite=False)

Set a single global metadata key-value pair or alternatively multiple ones as dict. Metadata added this way will persist for all data storage instances in this process until being selectively removed by calls to “remove_global_metadata”.

classmethod get_global_metadata()

Return a copy of the global metadata dict.

get_unified_metadata(local_metadata=None)

Helper method to return a dictionary containing provided local metadata as well as global metadata depending on the include_global_metadata flag.

Parameters:

local_metadatadict

Metadata to include in addition to global metadata.

include_global_metadatabool, optional

Flag indicating whether to include global metadata.

Returns:

dict

New dictionary containing local metadata and global metadata.

abstract load_data(*args, **kwargs)

This method must be implemented in a subclass. It should provide the facility to load a saved data set including the metadata/experiment parameters and column headers (if possible). Many storage classes can even implement this method as a static method.

For file-based storage objects, the only parameter should be file_path.

Parameters:

file_pathstr

Path to the file to be loaded.

Returns:

np.ndarray

Data as a numpy array.

dict

User metadata.

dict

General header data.

classmethod remove_global_metadata(names)

Remove a global metadata key-value pair by key. Does not raise an error if the key is not found.

abstract save_data(data, *, metadata=None, notes=None, nametag=None, timestamp=None, **kwargs)

This method must be implemented in a subclass. It should provide the facility to save an entire measurement as a whole along with experiment metadata (to include e.g. in the file header). The user can either specify an explicit filename or a generic one will be created. If optional nametag and/or timestamp is provided, this will be used to create the generic filename (only if the filename parameter is omitted).

Parameters:

datanumpy.ndarray

Data array to be saved (must be 1D or 2D for text files).

notesstr, optional

String that is included in the metadata “as-is” without a key.

metadatadict, optional

Named metadata to be saved in the data header / metadata.

nametagstr, optional

Nametag to include in the generic filename.

timestampdatetime.datetime, optional

Timestamp to construct a generic filename from.

filenamestr, optional

Explicit filename to use for saving the data.

Returns:

tuple

Full file path (str), timestamp used (datetime.datetime), saved data shape (tuple).

save_thumbnail(mpl_figure, file_path)

Save a matplotlib figure visualizing the saved data in the configured image format. It is recommended to use the same file path as the corresponding data file (if applicable) and exclude the file extension (which will be added according to the image format).

Parameters:

mpl_figurematplotlib.figure.Figure

The matplotlib figure object to save as an image.

file_pathstr

Full file path to use without the file extension.

Returns:

str

Full absolute path of the saved image.

class qudi.util.datastorage.ImageFormat(value)

Bases: enum.Enum

Image format to use for saving data thumbnails.

PDF = '.pdf'

PNG = '.png'

class qudi.util.datastorage.NpyDataStorage(*, root_dir, **kwargs)

Bases: qudi.util.datastorage.DataStorageBase

Helper class to store (measurement)data on disk as binary .npy file.

create_header(timestamp, dtype, metadata=None, notes=None, column_headers=None)

property file_extension

!! processed by numpydoc !!

static load_data(file_path)

See load_data() for more information.

Parameters:

file_pathstr

Path to the file to load data from.

save_data(data, *, metadata=None, notes=None, nametag=None, timestamp=None, column_headers=None, filename=None)

Saves a binary file containing the data array. Also saves alongside a text file containing the notes, (global) metadata, and column headers for this data set. The filename of the text file will be the same as for the binary file appended by “_metadata”.

For more information, see save_data().

Parameters:

column_headersstr or list, optional

Data column header strings or a single string.

class qudi.util.datastorage.TextDataStorage(*, root_dir, comments='# ', delimiter='\t', file_extension='.dat', column_formats=None, **kwargs)

Bases: qudi.util.datastorage.DataStorageBase

Helper class to store (measurement)data on disk as text file. Data will always be saved in a tabular format with column headers. Single/Multiple rows are appendable.

__init__(*, root_dir, comments='# ', delimiter='\t', file_extension='.dat', column_formats=None, **kwargs)

Initialize storage configuration for saving files.

Parameters:

root_dirstr, optional

Root directory for this storage instance to save files into.

commentsstr, optional

String to put at the beginning of comment and header lines.

delimiterstr, optional

Column delimiter used in text files.

file_extensionstr, optional

File extension to use for text files.

column_formatsstr or sequence, optional

Value format specifier (mini-language) for each column. If a single string is provided, it will be used for all columns.

column_headersstr or sequence, optional

Sequence of strings containing column headers. If a single string is given, write it to the file header without formatting.

column_dtypestype or str or sequence, optional

The column data types to expect.

**kwargs

Additional keyword arguments for further customization.

Notes

For additional keyword arguments, see the documentation for __init__.

append_file(data, file_path)

Append single or multiple rows to an existing data file.

Parameters:

datanumpy.ndarray

Data array to be appended. For 1D arrays, it represents a single row. For 2D arrays, it represents multiple rows.

file_pathstr

File path to append to.

Returns:

tuple

Number of rows written (int). Number of columns written (int).

create_header(timestamp=None, metadata=None, notes=None, column_headers=None, column_dtypes=None)

property delimiter

!! processed by numpydoc !!

property file_extension

!! processed by numpydoc !!

static load_data(file_path)

See load_data() for more information.

Parameters:

file_pathstr, optional

Path to the file to load data from.

new_file(*, timestamp=None, metadata=None, notes=None, nametag=None, column_headers=None, column_dtypes=None, filename=None)

Create a new data file on disk and write header string to it. Will overwrite old files silently if they have the same path.

Parameters:

metadatadict, optional

Named metadata values to be saved in the data header.

notesstr, optional

String that is included in the file header “as-is”.

nametagstr, optional

Nametag to include in the generic filename.

timestampdatetime.datetime, optional

Timestamp to use. Will create one if missing.

filenamestr, optional

Custom filename to use (nametag, timestamp, and configured file_extension will not be included for file naming).

Returns:

tuple

Full file path (str), timestamp used (datetime.datetime).

save_data(data, *, timestamp=None, metadata=None, notes=None, nametag=None, column_headers=None, column_dtypes=None, filename=None)

See save_data() for more information.

Parameters:

column_headersstr or list, optional

Data column header strings or a single string.

qudi.util.datastorage.create_dir_for_file(file_path)

Helper method to create the directory (recursively) for a given file path. Will NOT raise an error if the directory already exists.

Parameters:

file_pathstr

File path to create the directory for.

qudi.util.datastorage.format_column_headers(column_headers, delimiter=';;')

qudi.util.datastorage.format_header(timestamp, number_format=None, metadata=None, notes=None, column_dtypes=None, column_headers=None, comments=None, delimiter=None)

qudi.util.datastorage.get_header_from_file(file_path)

qudi.util.datastorage.get_info_from_header(header)

qudi.util.datastorage.get_timestamp_filename(timestamp, nametag=None)

Returns a qudi standard filename used for saving measurement data to a file, not including any file extension.

Parameters:

timestampdatetime.datetime

Timestamp used to create the filename from.

nametagstr, optional

Additional string to include in the filename.

Returns:

str

Generated filename without file extension.

qudi.util.datastorage.metadata_to_str_dict(metadata)

qudi.util.datastorage.str_dict_to_metadata(str_dict)