Exa Analytics Framework ¶

An ecosystem for data processing, analytics, and visualization.

This package provides a framework for building packages tailored to specific industry or academic applications. Data objects, such as arrays and matrices are organized into “containers”. Containers provide methods for manipulating, processing, organizing, analyzing, and visualizing their data. The package is built with minimal external dependencies, on top of established open-source software, and uses the Jupyter notebook environment for interactive data exploration.

Exa¶

This package creates a systematic infrastructure for an ecosystem of packages, tailored to specific industry or academic displines, for organizing, processing, analyzing, and visualizing data. It is built with minimal dependencies, leverages established open-source packages, is itself extensible, and is targeted at both industry and academic applications.

At a high level, data objects such as series or dataframes (i.e. pandas like objects) are organized into containers which track relationships between these objects and provide methods for computation, conversion to other formats, analysis, and visualization within the Jupyter notebook environment.

Executables¶

Exa provides two executables; “exa” and “exw”. For the graphical user interface, built on top of the Jupyter notebook environment, run “exa” on the command line.

Installation¶

Python’s external libraries are maintained as packages in repositories. There are two main repositories, pypi and anaconda and two corresponding Python applications that interact with them (pip and conda respectively).

This project recommends using conda because it is both a package manager and a Python virtual environment manager. Anaconda also provides better cross platform support especially for Python packages that require compiled external dependences.

Anaconda¶

Using anaconda or miniconda...

conda install numba exa

Pypi¶

Using pip...

# sudo apt-get install llvm-3.7 or sudo yum install ... etc.
sudo pip install numba exa

CUDA¶

If working on a system with CUDA supported Nvidia GPUs...

conda install cudatoolkit     # or via apt-get or yum etc.

Repository¶

Manually...

# install llvm, numba, cudatoolkit, and CPython3.x
git clone https://github.com/exa-analytics/exa
cd exa
pip install .

What’s Next?¶

Users should check out the User Introduction
Contributors should check out the Developer Introduction
The API contains usage and extension examples, and developer notes

API¶

Configuration¶

This module generates the “~/.exa” directory where all databases, logs, notebooks, and data reside. This configuration may be altered by editing the items below:

paths:

data: Path to the data directory (default ~/.exa/data)
notebooks: Path to the notebooks directory (default ~/.exa/notebooks)

log:

nlogs: Number of log files to rotate
nbytes: Max log file size (in bytes)
syslog: System log file path
dblog: Database log file path (if necessary)
level: Logging level, 0: normal, 1: extra info, 2: debug

db:

uri: String URI for database connection
update: If 1, refresh static database data (e.g. unit conversions)

js:

update: If 1, update JavaScript notebook extensions

Warning

The configuration file (~/.exa/config) should only be altered when no exa notebooks are running (i.e. exa or any of its related packages are imported).

exa._config.save()[source]¶: Save the configuration file to disk on exit, resetting update flags.

Warning

This is a bit unsafe because we are not guarenteed to hit the updating function during execution (that is what well written tests are for - use mock), but it is advantageous in the case that multiple packages that use exa are running simultaneously.

exa._config.set_update()[source]¶: Set update flags.

exa._config.del_update()[source]¶: Reset update flags.

Exceptions¶

All base level exceptions are defined here.

exception exa.error.ExaException(msg)[source]¶: Exception with support for logging.

exception exa.error.RequiredIndexError(missing, clsname)[source]¶: DataFrame index name error.

exception exa.error.RequiredColumnError(missing, clsname)[source]¶: DataFrame column error.

Logging¶

There are two log files that exa writes to, the system log and the database log. The database log is used when the database backend does not provide its own logging solution.

Levels:

0: default (>= WARNING messages are logged)
1: info (>= INFO messages are logged)
2: debug (all messages are logged)

class exa.log.LogFormat(fmt=None, datefmt=None, style='%')[source]¶

Custom log format used by all loggers.

format(record)[source]¶: Overwrites the built-in format function (called when sending a message to the logger) with the specific format defined by this class.

exa.log.create_logger(name)[source]¶: Create a logger with a given name.

Editor¶

Text-editor-like functionality for programatically manipulating raw text input and output data and converting this data into container objects. This class does not behave like a fully fledged text editor but does have some basic find, replace, insert, etc. functionality.

class exa.editor.Editor(path_stream_or_string, as_interned=False, nprint=30, name=None, description=None, meta={})[source]¶

An editor is a representation of a text file on disk that can be programmatically manipulated.

Text lines are stored in memory; no files remain open. This class does not strive to be a fully fledged text editor rather a base class for converting input and output data from text on disk to some type of (exa framework) container object (and vice versa).

>>> template = "Hello World!\nHello {user}"
>>> editor = Editor(template)
>>> editor[0]
'Hello World!'
>>> len(editor)
2
>>> del editor[0]
>>> len(editor)
1
>>> editor.write(fullpath=None, user='Alice')
Hello Alice

Tip:: Editor line numbers use a 0 base index. To increase the number of lines displayed by the repr, increase the value of the nprint attribute.

Warning

For large text with repeating strings be sure to use the as_interned argument.

name¶

str

Data/file/misc name

description¶

str

Data/file/misc description

meta¶

dict

Additional metadata as key, value pairs

nrpint¶

int

Number of lines to display when printing

cursor¶

int

Line number position of the cusor (see find_next_any() and find_next_string())

write(path=None, *args, **kwargs)[source]¶

Perform formatting and write the formatted string to a file or stdout.

Optional arguments can be used to format the editor’s contents. If no file path is given, prints to standard output.

Parameters:	path (str) – Full file path (default None, prints to stdout) args – Positional arguments to format the editor with *kwargs – Keyword arguments to format the editor with

format(*args, inplace=False, **kwargs)[source]¶

Format the string representation of the editor.

Parameters:	inplace (bool) – If True, overwrite editor’s contents with formatted contents

head(n=10)[source]¶

Display the top of the file.

Parameters:	n (int) – Number of lines to display

tail(n=10)[source]¶

Display the bottom of the file.

Parameters:	n (int) – Number of lines to display

append(lines)[source]¶

Parameters:	lines (list) – List of line strings to append to the end of the editor

prepend(lines)[source]¶

Parameters:	lines (list) – List of line strings to insert at the beginning of the editor

insert(lines={})[source]¶

Insert lines into the editor.

Note

To insert before the first line, use preappend() (or key 0); to insert after the last line use append().

Parameters:	lines (dict) – Dictionary of lines of form (lineno, string) pairs

remove_blank_lines()[source]¶: Remove all blank lines (blank lines are those with zero characters).

delete_lines(lines)[source]¶

Delete all lines with given line numbers.

Parameters:	lines (list) – List of integers corresponding to line numbers to delete

find(*strings)[source]¶

Search the entire editor for lines that match the string.

Parameters:	*strings – Any number of strings to search for
Returns:	results (dict) – Dictionary of string key, line values.

find_next(string)[source]¶

From the editor’s current cursor position find the next instance of the given string.

Parameters:	string (str) – String to search for from the current cursor position. reverse (bool) – Search in reverse (default false)
Returns:	tup (tuple) – Tuple of cursor position and line or None if not found

Note

This function cycles the entire editor (i.e. cursor to length of editor to zero and back to cursor position).

regex(*patterns, line=False)[source]¶

Search the editor for lines matching the regular expression.

Parameters:	patterns – Regular expressions to search each line for line* (bool) – Return the whole line or the matched groups (groups default)
Returns:	results (dict) – Dictionary of pattern keys, line values (or groups - default)

replace(pattern, replacement)[source]¶

Replace all instances of a pattern with a replacement.

Parameters:	pattern (str) – Pattern to replace replacement (str) – Text to insert

pandas_dataframe(start, stop, ncol)[source]¶

Returns the result of tab-separated pandas.read_csv on a subset of the file.

Parameters:	start (int) – line number where structured data starts stop (int) – line number where structured data stops ncol (int or list) – the number of columns in the structured data or a list of that length with column names
Returns:	pd.DataFrame – structured data

variables¶

Display a list of templatable variables present in the file.

Templating is accomplished by creating a bracketed object in the same way that Python performs string formatting. The editor is able to replace the placeholder value of the template. Integer templates are positional arguments.

classmethod from_file(path, **kwargs)[source]¶: Create an editor instance from a file on disk.

classmethod from_stream(f, **kwargs)[source]¶: Create an editor instance from a file stream.

classmethod from_string(string, **kwargs)[source]¶: Create an editor instance from a string template.

exa.editor.lines_from_file(path, as_interned=False)[source]¶

Create a list of file lines from a given filepath.

Parameters:	path (str) – File path as_interned (bool) – List of “interned” strings (default False)
Returns:	strings (list) – File line list

exa.editor.lines_from_stream(f, as_interned=False)[source]¶

Create a list of file lines from a given file stream.

Parameters:	( (f) – class:`~io.TextIOWrapper): File stream as_interned (bool) – List of “interned” strings (default False)
Returns:	strings (list) – File line list

exa.editor.lines_from_string(string, as_interned=False)[source]¶

Create a list of file lines from a given string.

Parameters:	string (str) – File string as_interned (bool) – List of “interned” strings (default False)
Returns:	strings (list) – File line list

Trait Supporting Data Objects¶

The DataFrame is an extension of the DataFrame object. It provides additional methods for creating traits.

Note

For further information on traits, see widget.

Additionally, DataFrame and related objects (e.g Field) provide attributes for defining their index and column names. This has the effect of creating relationships between different dataframes. They can be grouped into three types:

index name (df1) matches index name (df2)
index name (df1) matches column name (df2)
column name (df1) matches column name (df2)

Note

These types correspond to one to one, one to many, and many to many relational types, respectively.

Finally, the objects contained in this module provide convenience methods for handling categorical data.

See also

For information about traits and how they allow enable dynamic visualizations see widget. For usage of numerical objects see container.

class exa.numerical.Numerical[source]¶

Base class for Series, DataFrame, and Field objects, providing default trait functionality, shortened string representation, and in memory copying support.

copy(*args, **kwargs)[source]¶: Create a copy without mangling the (class) type.

class exa.numerical.Series(*args, **kwargs)[source]¶

Trait supporting analogue of Series.

import numpy as np

class MySeries(Series):
    """Example usage of the exa.Series object."""
    _sname = 'data'
    _iname = 'data_index'
    _precision = 2

s = MySeries(np.random.rand(10**5))
traits = s._update_traits()
type(traits)    # dict containing "myseries_values" as a Unicode trait

class exa.numerical.DataFrame(*args, **kwargs)[source]¶: Trait supporting analogue of DataFrame.

class exa.numerical.Field(*args, field_values=None, **kwargs)[source]¶

A field is composed of the field definition (“field_data”) and values ( “field_values”). Field data define the shape and discretization of a field. Field values are scalar magnitudes (or vectors) that describe the field at each discretized point in space.

copy(*args, **kwargs)[source]¶: Copy the field dataframe, including the field values

memory_usage()[source]¶: Get the combined memory usage of the field data and field values.

class exa.numerical.Field3D(*args, field_values=None, **kwargs)[source]¶

Dataframe for storing dimensions of a scalar or vector field of 3D space.

Column	Type	Description
nx	int	number of grid points in x
ny	int	number of grid points in y
nz	int	number of grid points in z
ox	float	field origin point in x
oy	float	field origin point in y
oz	float	field origin point in z
xi	float	First component in x
xj	float	Second component in x
xk	float	Third component in x
yi	float	First component in y
yj	float	Second component in y
yk	float	Third component in y
zi	float	First component in z
zj	float	Second component in z
zk	float	Third component in z

Note

Each field should be flattened into an N x 1 (scalar) or N x 3 (vector) series or dataframe respectively. The orientation of the flattening should have x as the outer loop and z values as the inner loop (for both cases). This is sometimes called C-major order, C-style order, and has the last index changing the fastest and the first index changing the slowest.

See also

Field

class exa.numerical.SparseSeries(data=None, index=None, sparse_index=None, kind='block', fill_value=None, name=None, dtype=None, copy=False, fastpath=False)[source]¶: Trait supporting sparse series.

class exa.numerical.SparseDataFrame(*args, **kwargs)[source]¶: Trait supporting sparse dataframe.

Container¶

The BaseContainer class is the primary object for data processing, analysis, and visualization. Containers are composed of n-dimensional spreadsheet-like (see numerical) objects whose columns contain data for 2D and 3D visualization.

The BaseContainer is akin to a HDFStore in that it is a container for dataframes (and saves to an HDF5 file). It is different in that it provides visualization tools access to the data contained via automated JSON strings, transferrable between languages.

See also

container and widget

class exa.container.Container(name=None, description=None, meta=None, **kwargs)[source]¶

Container class responsible for all features related to data management.

copy(**kwargs)[source]¶: Create a copy of the current object.

concat(*args, **kwargs)[source]¶: Concatenate any number of container objects with the current object into a single container object.

See also

For argument description, see concat().

slice_by_indices(key)[source]¶: Slice the container by series or dataframe index.

Warning

Does not make a copy, must call the .copy() method on the resulting container if a copy is needed.

slice_by_cardinal_axis(key)[source]¶: Slice the container according to its cardinal axis.

See also

Note the warning in slice_by_indices().

info()[source]¶: Display information about the container’s objects.

Note

Sizes are reported in bytes.

memory_usage()[source]¶: Estimate the memory usage of the entire container.

network()[source]¶: Display information about the container’s object relationships.

Note

Due to quirks of plotting, rerunning this command until a “pleasing” visual is generated may be useful.

save(path)[source]¶

Save the container as an HDF5 archive.

Parameters:	path (str) – Path where to save the container

classmethod load(pkid_or_path=None)[source]¶

Load a container object from a persistent location or file path.

Parameters:	pkid_or_path – Integer pkid corresponding to the container table or file path
Returns:	container – The saved container object

class exa.container.TypedMeta[source]¶

This metaclass creates statically typed class attributes using the property framework.

class TestMeta(TypedMeta):
    attr1 = (int, float)
    attr2 = DataFrame

class TestClass(metaclass=TestMeta):
    def __init__(self, attr1, attr2):
        self.attr1 = attr1
        self.attr2 = attr2

The above code dynamically creates code that looks like the following:

class TestClass:
    @property
    def attr1(self):
        return self._attr1

    @attr1.setter
    def attr1(self, obj):
        if not isinstance(obj, (int, float)):
            raise TypeError('attr1 must be int')
        self._attr1 = obj

    @attr1.deleter
    def attr1(self):
        del self._attr1

    @property
    def attr2(self):
        return self._attr2

    @attr2.setter
    def attr2(self, obj):
        if not isinstance(obj, DataFrame):
            raise TypeError('attr2 must be DataFrame')
        self._attr2 = obj

    @attr2.deleter
    def attr2(self):
        del self._attr2

    def __init__(self, attr1, attr2):
        self.attr1 = attr1
        self.attr2 = attr2

static create_property(name, ptype)[source]¶: Creates a custom property with a getter that performs computing functionality (if available) and raise a type error if setting with the wrong type.

Note

By default, the setter attempts to convert the object to the correct type; a type error is raised if this fails.

Widget¶

Functionality for using Jupyter notebook extensions to visualize data speficic containers. This module requires the infrastructure provided by the traitlets and ipywidgets packages.

class exa.widget.Widget(*pargs, **kwargs)[source]¶: Base widget class for Jupyter notebook widgets provided by exa. Standardizes bidirectional communication handling between notebook extensions’ frontend JavaScript and backend Python.

class exa.widget.ContainerWidget(container, *args, **kwargs)[source]¶: Jupyter notebook widget representation of an exa Container. The widget accepts a (reference to a) container and parameters and creates a suitable display. By default a container displays an interactive graphic containing information provided by info() and network().

See also

container, container

exa.widget.install_notebook_widgets(pkg_nbext, sys_nbext, verbose=False)[source]¶

Convenience wrapper around install_nbextension() that organizes notebook extensions for exa and related packages in a systematic fashion.

Parameters:	pkg_nbext (str) – Path to the “_nbextension” directory in the source sys_nbext (str) – Path to the system “nbextensions” directory verbose (bool) – Verbose installation

Matplotlib Utilities¶

TeX Utilities¶

Utilities¶

Commonly used functions (primarily for convenience and repetition reduction).

exa.utility.datetime_header(title='')[source]¶: Creates a simple header string containing the current date/time stamp delimited using “=”.

exa.utility.mkp(*args, mk=False)[source]¶

Generate a directory path, and create it if requested.

filepath = mkp('base', 'folder', 'file')
dirpath = mkp('root', 'path', 'folder', mk=True)

Parameters:	args – File or directory path segments to be concatenated mk (bool*) – Make the directory (if it doesn’t exist)
Returns:	path (str) – File or directory path

exa.utility.convert_bytes(value)[source]¶

Reduces bytes to more convenient units (i.e. KiB, GiB, TiB, etc.).

Parameters:	values (int) – Value in Bytes
Returns:	tup (tuple) – Tuple of value, unit (e.g. (10, ‘MiB’))

exa.utility.get_internal_modules(key='exa')[source]¶

Get a list of modules belonging to the given package.

Parameters:	key (str) – Package or library name (e.g. “exa”)

Tester¶

Custom tester class for running interactive (i.e. within the Jupyter notebook environment) tests.

class exa.test.tester.UnitTester(methodName='runTest')[source]¶

The custom tester class which provides an alternative test runner.

classmethod run_interactively(log=False)[source]¶

Run a test suite in a Jupyter notebook environment or shell.

Parameters:	log (bool) – Write output to a log file instead of to stdout

exa.test.tester.run_doctests(log=False)[source]¶

Run all docstring tests.

Parameters:	log (bool) – Write test results to system log (default false)

exa.test.tester.run_unittests(log=False)[source]¶

Perform (interactive) unit testing logging the results.

Parameters:	log (bool) – Send results to system log (default False)

Basic Tests¶

Tests for _config, log, and the suite of modules in the test subpackage, tester, etc.

class exa.test.test_base.TestConfig(methodName='runTest')[source]¶

Tests for _config.

test_type()[source]¶: Check that the config is a ConfigParser object.

class exa.test.test_base.TestLog(methodName='runTest')[source]¶

Tests for log.

test_logger()[source]¶: Check that at least dblog and syslog are present in the loggers.

class exa.test.test_base.TestTester(methodName='runTest')[source]¶

Tests for tester.

test_type()[source]¶: Ensure that the tester is an instance of a test case.

test_runners_present()[source]¶: Test that the runners have been imported.

Relational¶

This (sub)package is provides the content management framework for container objects and a collection of static data for reference and unit conversions.

exa.relational.__init__.load_isotope_data()[source]¶: Load isotope data into the database (replacing existing).

exa.relational.__init__.load_unit_data()[source]¶: Load unit conversions into the database (replacing existing).

Note

This function actually computes (prior to bulk inserting data) conversion factors.

exa.relational.__init__.load_constant_data()[source]¶: Load constants into the database (replacing existing).

Database Engine Configuration¶

This module provides the base classes and metaclasses for relational tables created by exa. It also provides the database engine configuration and session class factory.

exa.relational.base.generate_hexuid()[source]¶: Create a unique, random, hex string id.

exa.relational.base.cleanup_engine()[source]¶: At exit, cleanup connection pool.

exa.relational.base.scoped_session(*args, **kwargs)[source]¶: Safely commit relational objects.

class exa.relational.base.BaseMeta(classname, bases, dict_)[source]¶

This is the base metaclass for all relational tables. It provides convient lookup methods, bulk insert methods, and conversions to other formats.

get_by_pkid(pkid)[source]¶: Select an object by pkid.

get_by_name(name)[source]¶: Select objects by name.

get_by_uid(uid)[source]¶: Select an object by hexuid (as string)

bulk_insert(mappings)[source]¶

Perform a bulk insert into a specific table.

mappings = [{'column1': 'foo', 'column2': 42, 'column3': 'bar'},
            {'column1': 'fop', 'column2': 43, 'column3': 'baz'}]
Table.bulk_insert(mappings)

to_frame()[source]¶: Dump the table to a DataFrame object.

Warning

If performing this action on a very large table, may raise a memory error. It is almost always more effective to query the table for the specific records of interest.

class exa.relational.base.Name[source]¶: Name and description fields.

class exa.relational.base.HexUID[source]¶: Hex-based unique id (uid) field.

class exa.relational.base.Time[source]¶: Timestamp fields.

class exa.relational.base.Size[source]¶: Approximate size (on disk) and file count fields.

Project Table¶

A project represents a continuous or finite study of a subject matter. It is the highest level of categorical organization for the content management system.

See also

job and file

class exa.relational.project.Project(**kwargs)[source]¶: Continuous or finite study of a certain subject with a specific goal.

Job Table¶

A job is a single experiment, typically resulting in a number of “raw” data files (inputs and outputs) that can be represented in memory by a single container. Since this is not always the case, jobs have a many to many relationship with container files.

class exa.relational.job.Job(**kwargs)[source]¶: Specific task in a Program or Project.

File Tables¶

There are two types of file tables, one for “raw” data files (those coming from third party software for example) and container objects on disk (stored as HDF5 files). Because a container is typically comprised of multiple raw data files, there is a many to many relationship between raw data files and container files.

class exa.relational.file.DataFile(**kwargs)[source]¶: Representation of a non exa container file on disk. Provides content management for “raw” data files.

class exa.relational.file.ContainerFile(**kwargs)[source]¶: Representation of an exa container object on disk. Containers are often composed from multiple raw data files.

Table of Isotopes¶

This module provides an interface for interacting with isotopes of atoms; the extended periodic table. For convenience, functions are provided for obtaining traditionally used elements. This module also provides mappers for commonly used dataframe manipulations.

class exa.relational.isotope.Meta(classname, bases, dict_)[source]¶

Provides lookup methods for Isotope.

Isotope['1H']     # Returns

get_by_strid(strid)[source]¶: Get an isotope using a string id.

get_by_symbol(symbol)[source]¶: Get an isotope using a string id.

get_element(name_or_symbol)[source]¶: Get (i.e. compute) the element with the given name or symbol (an element’s data is given as an average over isotopic composition).

class exa.relational.isotope.Isotope(**kwargs)[source]¶

A variant of a chemical element with a specific proton and neutron count.

>>> h = Isotope['1H']
>>> h.A
1
>>> h.Z
1
>>> h.mass
1.0078250321
>>> Isotope['C']
[8C, 9C, 10C, 11C, 12C, 13C, 14C, 15C, 16C, 17C, 18C, 19C, 20C, 21C, 22C]
>>> Isotope['13C'].szuid
175
>>> c = Isotope[57]
>>> c.A
13
>>> c.Z
6
>>> c.strid
'13C'

exa.relational.isotope.symbol_to_z()[source]¶

Create a “mapper” (Series) from element symbol to proton number (“Z”). This object can be used to quickly transform element symbols to proton number via:

mapper = symbol_to_z()
z_series = symbol_series.map(mapper)

exa.relational.isotope.z_to_symbol()[source]¶: Create a mapper from proton number to element symbol.

See also

Opposite mapper of symbol_to_z().

exa.relational.isotope.symbols_to_radii()[source]¶: Mapper from symbol pairs to sum of covalent radii.

exa.relational.isotope.symbol_to_element_mass()[source]¶: Mapper from symbol to element mass.

exa.relational.isotope.symbol_to_radius()[source]¶: Mapper from isotope symbol to covalent radius.

exa.relational.isotope.symbol_to_color()[source]¶: Mapper from isotope symbol to color.

Physical Constants¶

Table of reference physical constants in SI units.

class exa.relational.constant.Meta(classname, bases, dict_)[source]¶

Metaclass for Constant.

get_by_symbol(symbol)[source]¶: Get the value of a constant with the given symbol.

class exa.relational.constant.Constant(**kwargs)[source]¶

Physical constants and their values in SI units.

>>> hartree = Constant['Eh']
>>> hartree
4.35974434e-18

Note

Available constants can be inspected by calling:

Constant.to_frame()

Unit Conversions¶

This module provides relational classes for unit conversion tables.

class exa.relational.unit.Meta(classname, bases, dict_)[source]¶: Special metaclass for unit objects supporting aliases. Aliases are alternative names for standard symbols for units.

class exa.relational.unit.Dimension[source]¶

Descriptive class for units.

from_unit¶

str

Unit to convert from

to_unit¶

str

Unit to convert to

factor¶

float

Conversion factor

class exa.relational.unit.Length(**kwargs)[source]¶

>>> Length['angstrom', 'au']
1.88971616463
>>> Length['A', 'au']
1.88971616463
>>> Length['A', 'a0']
1.88971616463

class exa.relational.unit.Mass(**kwargs)[source]¶

>>> Mass['kg', 'lb']
2.2046226218
>>> Mass['Da', 'kg']
1.660538921000011e-27
>>> Mass['u', 'kg']
1.660538921000011e-27

class exa.relational.unit.Time(**kwargs)[source]¶

>>> Time['min', 's']
60.0000000000024
>>> Time['hr', 's']
3599.999712000023
>>> Time['weeks', 'days']
6.999999999955003

class exa.relational.unit.Current(**kwargs)[source]¶

>>> Current['A', 'C_s']
1.0
>>> Current['A', 'Bi']
0.1

class exa.relational.unit.Amount(**kwargs)[source]¶

>>> Amount['gmol', 'mol']
1.0
>>> Amount['lbmol', 'mol']
453.5923744952991

class exa.relational.unit.Luminosity(**kwargs)[source]¶

>>> Luminosity['cp', 'cd']
0.9810000000433602

class exa.relational.unit.Dose(**kwargs)[source]¶

>>> Dose['Gy', 'rd']
100.0
>>> Dose['J_kg', 'rd']
100.0

class exa.relational.unit.Acceleration(**kwargs)[source]¶

>>> Acceleration['m_s2', 'cm_s2']
100.0
>>> Acceleration['m_s2', 'stdgrav']
0.10197162130000001

class exa.relational.unit.Charge(**kwargs)[source]¶

>>> Charge['e', 'C']
1.6021765649999947e-19

class exa.relational.unit.Dipole(**kwargs)[source]¶

>>> Dipole['yCm', 'D']
299792.45817809016

class exa.relational.unit.Energy(**kwargs)[source]¶

>>> Energy['J', 'cal']
0.2388458966
>>> Energy['kcal', 'Btu']
3.9683205782473134

class exa.relational.unit.Force(**kwargs)[source]¶

>>> Force['N', 'lbf']
0.22480894310000002

class exa.relational.unit.Frequency(**kwargs)[source]¶

>>> Frequency['1_s', 'Hz']
1.0

class exa.relational.unit.MolarMass(**kwargs)[source]¶

>>> MolarMass['g_mol', 'kg_mol']
0.001

Tests for `isotope`¶

class exa.relational.test.test_isotope.TestIsotope(methodName='runTest')[source]¶

Check isotope table

test_table()[source]¶: Check that the table can be converted to a DataFrame.

test_selections()[source]¶: Check that isotopes can be selected in any manner.

test_data()[source]¶: Test the integrity of the database data.

Tests for `constant`¶

Tests the table of physical constants.

class exa.relational.test.test_constant.TestConstant(methodName='runTest')[source]¶

Check the physical constants table.

test_table()[source]¶: Check that the table can be converted to a DataFrame.

test_value()[source]¶: Check data values.

Tests for `unit`¶

Test availablility of unit conversions.

class exa.relational.test.test_unit.TestUnit(methodName='runTest')[source]¶

Check unit conversion tables.

test_table()[source]¶: Check that the table can be converted to a DataFrame.

test_factor()[source]¶: Ensure that conversion factors can be selected from the tables.

Distributed¶

Provides the Workflow class which systematizes remote and distributed computing.

Workflow¶

Math¶

Discrete and symbolic mathematical functions and algorithms. Many of the modules in this sub-package attempt to compile Python source code down to machine code (for CPU and GPU instruction as available).

Note

If a module compiles its functions to machine code, access to original (typically numpy supporting) functions is done via the same function name with a leading underscore. CUDA compiled functions have a trailing “_cuda”.

See also

__init__

Warning

Performance is guarenteed if numba is installed. If it is missing, performance will be affected and distributed/parallel computing features will not be available.

Tests for Math Modules¶

Cartesian Vector Operations¶

This module provides common operations for vectors in cartesian space:

\[\vec{r} = (x, y, z)\]

exa.math.vector.cartesian.magnitude(v)[source]¶

exa.math.vector.cartesian.magnitude_xyz(x, y, z)[source]¶

exa.math.vector.cartesian.magnitude_squared(v)[source]¶

exa.math.vector.cartesian.magnitude_squared_xyz(x, y, z)[source]¶

Tests for `cartesian`¶

Repeat¶

Functions for repeating arrays of varying dimensions.

exa.math.misc.repeat.repeat_count(array, n)[source]¶: Repeat each element of an array n times.

exa.math.misc.repeat.repeat_counts_f8_1d(array, counts)[source]¶: Repeat each element of an array n times (with variable n).

exa.math.misc.repeat.repeat_counts_f8_2d(array, counts)[source]¶: Repeat each element of an array n times (with variable n).

Summations¶

Fast algorithms for various types of (commonly used) summations.

exa.math.misc.summation.sum_product_pair(x, y)[source]¶: Sum each pair of elements coming from product().

exa.math.misc.summation.sum_product_pair_f8(x, y)[source]¶

Sum each pair of elements from two 1 dimensional arrays.

x = np.array([1., 2., 3.])
y = np.array([4., 1., 2.])
sum_product_pair_f8(x, y)    # array([5.0, 2.0, 3.0, 6.0, 3.0, 4.0, 7.0, 4.0, 5.0])

Tests for `summation`¶

class exa.math.misc.test.test_summation.TestSumProductPair(methodName='runTest')[source]¶

Tests for all functions name sum_product_pair*

setUp()[source]¶: Generate a couple of attributes for testing.

test_sum_product_pair()[source]¶: Test sum_product_pair(). This function should work regardless of types.

test_sum_product_pair_f8()[source]¶: Test sum_product_pair_f8(). This function only works on 64-bit floating point numbers.

Filetypes¶

Editors for common file types.

Generic CSV Support¶

Provides an editor with convenience methods tailored specifically for comma separated value (CSV) files.

class exa.filetypes.csv.CSV(path_stream_or_string, as_interned=False, nprint=30, name=None, description=None, meta={})[source]¶

clean()[source]¶: Remove extra whitespace, and replace commonly used null characters with nothing.

to_frame()[source]¶: Create a DataFrame from this file.

Frontend Container¶

================= container.js ================= JavaScript “frontend” counterpart of exa’s Container object for use within the Jupyter notebook interface. This “module” standardizes bidirectional communication logic for all container widget views.

The structure of the frontend is to generate an HTML widget (“container” - see create_container) and then populate its canvas with an application (“app”) appropriate to the type of container. If the (backend) container is empty, then populate the HTML widget with the test application. If the container is not empty but doesn’t have a dedicated application, the info application (info.js) is used.

ContainerView¶

Base view for creating data specific container widgets used within the Jupyter notebook. All logic related to communication (between Python and JavaScript) should be located here. This class provides a number of commonly used functions for such logic.

Warning:: Do not override the DOMWidgetView constructor (“initialize”).

render¶

Main entry point (called immediately after initialize) for (ipywidgets) DOMWidgetView objects.

Note:: This function can be overwritten by container specific code, but it is more common to overwrite the init function.
See Also:: init()

init¶

Container view classes that extend this class can overwrite this method to customize the behavior of their data specific view.

get_trait¶

Wrapper around the DOMWidgetView (Backbone.js) “model.get” function, that attempts to convert JSON strings to objects.

set_trait¶

Wrapper around the DOMWidgetView “model.set” function to correctly set json strings.

if_empty¶

If the (exa) container object is empty, render the test application widget.

default_listeners¶

Set up listeners for basic variables related to the window dimensions and system settings.

create_container¶

Create a resizable container.

create_canvas¶

Create a canvas for WebGL.

Applications¶

================ info.js ================ The default container view; displays container information (number of data objects, size, and relationships) in a dynamic fashion.

============= test.js ============= A test application called when an empty container widget is rendered in a Jupyter notebook environment.

TestContainer¶

A test application for the container

create_gui¶

Creates the gui

Third Party Library Support¶

3D Visualization¶

App3D¶

A 3D visualization application built on top of threejs

test_mesh¶

Example of a render

render¶

Render the 3D application

animate¶

Start the animation.

resize¶

Resizing of the renderer and controls

remove_meshes¶

Iterates over the meshes and removes each from the scene.

add_points¶

Create a point cloud from x, y, z coordinates and colors and radii (optional).

Args:: x (array-like): Array like object of x values y (array-like): Array like object of y values z (array-like): Array like object of z values colors (object): List like colors corresponding to every object radii (object): List like radii corresponding to every object
Returns:: points (THREE.Points): Reference to added points object
Tip:: By tracking the returned object, one can create animations.
Warning:: On a modern machine attempting to render >5 million (approximately) objects can cause a slowdown of the browser and framerate of the application.

add_spheres¶

Create a point cloud from x, y, z coordinates and colors and radii (optional).

Args:: x (array-like): Array like object of x values y (array-like): Array like object of y values z (array-like): Array like object of z values colors (object): List like colors corresponding to every object radii (object): List like radii corresponding to every object
Returns:: spheres (list): List of THREE.Mesh objects

add_lines¶

Add lines between pairs of points.

Args:: v0 (array): Array of first vertex in pair v1 (array): Array of second vertex x (array): Position in x of vertices y (array): Position in y of vertices z (array): Position in z of vertices colors (array): Colors of vertices
Returns:: linesegs (THREE.LineSegments): Line segment objects

add_lines¶

Add lines between pairs of points.

Args:: v0 (array): Array of first vertex in pair v1 (array): Array of second vertex x (array): Position in x of vertices y (array): Position in y of vertices z (array): Position in z of vertices colors (array): Colors of vertices
Returns:: linesegs (THREE.LineSegments): Line segment objects

add_wireframe¶

Create a wireframe object

flatten_color¶

set_camera¶

Set the camera in the default position and have it look at the origin.

Args:: kwargs: {‘x’: x, ‘y’: y, ..., ‘ox’: ox, ...}

set_camera_from_scene¶

add_scalar_field¶

Create an isosurface of a scalar field.

When given a scalar field, creating a surface requires selecting a set of vertices that intersect the provided field magnitude (isovalue). There are a couple of algorithms that do this.

add_unit_axis¶

Adds a unit length coordinate axis at the origin

march_cubes1¶

Run the marching cubes algorithm finding the volumetric shell that is smaller than the isovalue.

The marching cubes algorithm takes a scalar field and for each field vertex looks at the nearest indices (in an evenly space field this forms a cube), determines along what edges the scalar field is less than the isovalue, and creates new vertices along the edges of the field’s cube. The at each point in the field, a cube is created with vertices numbered:

4——-5

/ | / |

7——-6 | | 0—-|–1 | / | / 3——-2

Field values are given for each field vertex. Edges are labeled as follows (see the lookup table below).

4

o——-o o——-o

/ | / | 7 / | 6 / | 5

o——-o | o——-o | | o—-|–o | o—-|–o

3 | / 0 | / 1 | / | /

o——-o o——-o

2

Edges 8, 9, 10, and 11 wrap around (clockwise looking from the top as drawn) the vertical edges of the cube, with 8 being the vertical edge between vertex 0 and 4 (see above).

Note:: Scalar fields are assumed to be in row major order (also known C style, and implies that the last index is changing the fastest).
See Also:: field.js

march_cubes2¶

Similar to the above but for finding positive and negative surfaces.

============ gui.js ============ Basic gui for container views.

ContainerGUI ===============

Misc.¶

field.js¶

This module provides infrastructure for storing and manipulating 3D fields. By standardizing how field data is stored, marching cubes (or other surface computing algorithms) are easier to work with.

Field¶

Base class for dealing with scalar and vector field data

Args:

dimensions: {‘ox’: ox, ‘nx’: nx, ‘dxi’: dxi, ‘dxj’: dxj, ‘dxk’: dxk,: ‘oy’: oy, ‘ny’: ny, ‘dyi’: dyi, ‘dyj’: dyj, ‘dyk’: dyk, ‘oz’: oz, ‘nz’: nz, ‘dzi’: dzi, ‘dzj’: dzj, ‘dzk’: dzk}

Note:

The dimensions argument can alternatively be {‘x’: xarray, ‘y’: yarray, ‘z’: zarray} if they have already been constructed but in this case the arrays should form cubic discrete points

update¶

Updates the field after establishing new x, y, z arrays

ScalarField¶

Representation of a scalar field.

num.js¶

Numerical utilities

fac2¶

hstack¶

Horizontally concatenate a list of arrays.

minspace¶

Creates a linearly spaced array with knowledge of the length, origin and spacing of the array.

Args:: min (number): origin space (number): spacing n (number): length of array

linspace¶

Create a linearly spaced array of the form [min, max] with n linearly spaced elements.

Args:: min (number): Starting number max (number): Ending number (inclusive) n (number): Number of elements
Returns:: array (array): Array of values

arange¶

meshgrid3d¶

From three discrete dimensions, create a set of 3d gridpoints

ellipsoid¶

sphere¶

torus¶

compute_field¶

gen_array¶

Generates discrete spatial points in space. Used to generate x, y, z spatial values for the cube field. In most cases, for the x basis vector, dy and dz are zero (“cube-like”).

================ utility.js ================ Helper functions used by custom notebook JS.

flatten_to_array ================== Flattens an array-like object into a 1-D Float32Array object.

Args:: obj (object): Array like object to be flattened

create_float_array_xyz ========================== Create a 1D array from 3D data.

repeat_float ================== Repeat a value n times.

repeat_object ============== Repeat an object n time.

mapper ——–

User Introduction¶

Developer Introduction¶

Code Organization¶

Python Style Suggestions¶

JavaScript Style Suggestions¶

Info¶

License

Exa Analytics Framework¶

Exa¶

Executables¶

Installation¶

Anaconda¶

Pypi¶

CUDA¶

Repository¶

What’s Next?¶

API¶

Configuration¶

Exceptions¶

Logging¶

Editor¶

Trait Supporting Data Objects¶

Container¶

Widget¶

Matplotlib Utilities¶

TeX Utilities¶

Utilities¶

Tester¶

Basic Tests¶

Relational¶

Database Engine Configuration¶

Project Table¶

Job Table¶

File Tables¶

Table of Isotopes¶

Physical Constants¶

Unit Conversions¶

Tests for isotope¶

Tests for constant¶

Tests for unit¶

Distributed¶

Workflow¶

Math¶

Tests for Math Modules¶

Cartesian Vector Operations¶

Tests for cartesian¶

Repeat¶

Summations¶

Tests for summation¶

Filetypes¶

Generic CSV Support¶

Frontend Container¶

ContainerView¶

render¶

init¶

get_trait¶

set_trait¶

if_empty¶

default_listeners¶

create_container¶

create_canvas¶

Applications¶

TestContainer¶

create_gui¶

Third Party Library Support¶

3D Visualization¶

App3D¶

test_mesh¶

render¶

animate¶

resize¶

remove_meshes¶

add_points¶

add_spheres¶

add_lines¶

add_lines¶

add_wireframe¶

flatten_color¶

set_camera¶

set_camera_from_scene¶

add_scalar_field¶

add_unit_axis¶

march_cubes1¶

march_cubes2¶

Misc.¶

field.js¶

Field¶

Exa Analytics Framework ¶

Tests for `isotope`¶

Tests for `constant`¶

Tests for `unit`¶

Tests for `cartesian`¶

Tests for `summation`¶