btrdb.conn

Connection related objects for the BTrDB library

class btrdb.conn.BTrDB(endpoint)

The primary server connection object for communicating with a BTrDB server.

Methods

collection_metadata(prefix[, auto_retry, ...])	Gives statistics about metadata for collections that match a prefix.
create(uuid, collection[, tags, ...])	Tells BTrDB to create a new stream with UUID uuid in collection with specified tags and annotations.
info()	Returns information about the platform proxy server.
list_collections([starts_with])	Returns a list of collection paths using the starts_with argument for filtering.
list_unique_annotations([collection])	Returns a list of annotation keys used in a given collection prefix.
list_unique_names([collection])	Returns a list of names used in a given collection prefix.
list_unique_units([collection])	Returns a list of units used in a given collection prefix.
query(stmt[, params, auto_retry, retries, ...])	Performs a SQL query on the database metadata and returns a list of dictionaries from the resulting cursor.
stream_from_uuid(uuid)	Creates a stream handle to the BTrDB stream with the UUID uuid.
streams(*identifiers[, versions, ...])	Returns a StreamSet object with BTrDB streams from the supplied identifiers.
streams_in_collection(*collection[, ...])	Search for streams matching given parameters

collection_metadata(prefix, auto_retry=False, retries=5, retry_delay=3, retry_backoff=4)

Gives statistics about metadata for collections that match a prefix.

Parameters:

• prefix (str, required) – A prefix of the collection names to look at

• auto_retry (bool, default: False) – Whether to retry this request in the event of an error

• retries (int, default: 5) – Number of times to retry this request if there is an error. Will be ignored if auto_retry is False

• retry_delay (int, default: 3) – initial time to wait before retrying function call if there is an error. Will be ignored if auto_retry is False

• retry_backoff (int, default: 4) – Exponential factor by which the backoff increases between retries. Will be ignored if auto_retry is False

Returns:

A tuple of dictionaries containing metadata on the streams in the provided collection.

Return type:

tuple

Examples

>>> conn.collection_metadata("sunshine/PMU1")
({'name': 0, 'unit': 0, 'ingress': 0, 'distiller': 0},
.. {'foo': 1, 'impedance': 12, 'location': 12})

>>> conn.collection_metadata("sunshine/")
({'name': 0, 'unit': 0, 'ingress': 0, 'distiller': 0},
.. {'foo': 1, 'impedance': 72, 'location': 72})

create(uuid, collection, tags=None, annotations=None, auto_retry=False, retries=5, retry_delay=3, retry_backoff=4)

Tells BTrDB to create a new stream with UUID uuid in collection with specified tags and annotations.

Parameters:

• uuid (UUID, required) – The uuid of the requested stream.

• collection (str, required) – The collection string prefix that the stream will belong to.

• tags (dict, required) – The tags-level metadata key:value pairs.

• annotations (dict, optional) – The mutable metadata of the stream, key:value pairs

• auto_retry (bool, default: False) – Whether to retry this request in the event of an error

• retries (int, default: 5) – Number of times to retry this request if there is an error. Will be ignored if auto_retry is False

• retry_delay (int, default: 3) – initial time to wait before retrying function call if there is an error. Will be ignored if auto_retry is False

• retry_backoff (int, default: 4) – Exponential factor by which the backoff increases between retries. Will be ignored if auto_retry is False

Returns:

instance of Stream class

Return type:

Stream

Examples

>>> import btrdb
>>> from uuid import uuid4 # this generates a random uuid
>>> conn = btrdb.connect()
>>> collection = "new/stream/collection"
>>> tags = {"name":"foo", "unit":"V"}
>>> annotations = {"bar": "baz"}
>>> s = conn.create(uuid=uuid4(), tags=tags, annotations=annotations, collection=collection)
<Stream collection=new/stream/collection name=foo>

info()

Returns information about the platform proxy server.

Returns:

Proxy server connection and status information

Return type:

dict

Examples

>>> conn = btrdb.connect()
>>> conn.info()
{
..        'majorVersion': 5,
..        'minorVersion': 8,
..        'build': ...,
..        'proxy': ...,
}

list_collections(starts_with='')

Returns a list of collection paths using the starts_with argument for filtering.

Parameters:

starts_with (str, optional, default: '') – Filter collections that start with the string provided, if none passed, will list all collections.

Returns:

collections

Return type:

List[str]

Examples

Assuming we have the following collections in the platform: foo, bar, foo/baz, bar/baz

>>> conn = btrdb.connect()
>>> conn.list_collections().sort()
["bar", "bar/baz", "foo", "foo/bar"]

>>> conn.list_collections(starts_with="foo")
["foo", "foo/bar"]

list_unique_annotations(collection=None)

Returns a list of annotation keys used in a given collection prefix.

Parameters:

collection (str) – Prefix of the collection to filter.

Returns:

annotations

Return type:

list[str]

Notes

This query treats the collection string as a prefix, so collection="foo" will match with the following wildcard syntax foo%. If you only want to filter for a single collection, you will need to provide the full collection, if there are other collections that match the foo% pattern, you might need to use a custom SQL query using conn.query.

Examples

>>> conn.list_unique_annotations(collection="sunshine/PMU1")
['foo', 'location', 'impedance']

list_unique_names(collection=None)

Returns a list of names used in a given collection prefix.

Parameters:

collection (str) – Prefix of the collection to filter.

Returns:

names

Return type:

list[str]

Examples

Can specify a full collection name.

>>> conn.list_unique_names(collection="sunshine/PMU1")
['C1ANG', 'C1MAG', 'C2ANG', 'C2MAG', 'C3ANG', 'C3MAG', 'L1ANG', 'L1MAG', 'L2ANG', 'L2MAG', 'L3ANG', 'L3MAG', 'LSTATE']

And also provide a collection prefix.

>>> conn.list_unique_names(collection="sunshine/")
['C1ANG', 'C1MAG', 'C2ANG', 'C2MAG', 'C3ANG', 'C3MAG', 'L1ANG', 'L1MAG', 'L2ANG', 'L2MAG', 'L3ANG', 'L3MAG', 'LSTATE']

list_unique_units(collection=None)

Returns a list of units used in a given collection prefix.

Parameters:

collection (str) – Prefix of the collection to filter.

Returns:

units

Return type:

list[str]

Examples

>>> conn.list_unique_units(collection="sunshine/PMU1")
['amps', 'deg', 'mask', 'volts']

query(stmt: str, params: Tuple[str] | List[str] = None, auto_retry=False, retries=5, retry_delay=3, retry_backoff=4)

Performs a SQL query on the database metadata and returns a list of dictionaries from the resulting cursor.

Parameters:

• stmt (str) – a SQL statement to be executed on the BTrDB metadata. Available tables are noted below. To sanitize inputs use a $1 style parameter such as select * from streams where name = $1 or name = $2.

• params (list or tuple) – a list of parameter values to be sanitized and interpolated into the SQL statement. Using parameters forces value/type checking and is considered a best practice at the very least.

• auto_retry (bool, default: False) – Whether to retry this request in the event of an error

• retries (int, default: 5) – Number of times to retry this request if there is an error. Will be ignored if auto_retry is False

• retry_delay (int, default: 3) – initial time to wait before retrying function call if there is an error. Will be ignored if auto_retry is False

• retry_backoff (int, default: 4) – Exponential factor by which the backoff increases between retries. Will be ignored if auto_retry is False

Returns:

a list of dictionary object representing the cursor results.

Return type:

list

Notes

Parameters will be inserted into the SQL statement as noted by the parameter number such as $1, $2, or $3. The streams table is available for SELECT statements only.

See https://btrdb.readthedocs.io/en/latest/ for more info.

The following are the queryable columns in the postgres streams table.

Column	Type	Nullable
uuid	uuid	not null
collection	character varying(256)	not null
name	character varying(256)	not null
unit	character varying(256)	not null
ingress	character varying(256)	not null
property_version	bigint	not null
annotations	hstore

Examples

Count all streams in the platform.

>>> conn = btrdb.connect()
>>> conn.query("SELECT COUNT(uuid) FROM streams")
[{'count': ...}]

Count all streams in the collection foo/bar by passing in the variable as a parameter.

>>> conn.query("SELECT COUNT(uuid) FROM streams WHERE collection=$1::text", params=["foo/bar"])
[{'count': ...}]

Count all streams in the platform that has a non-null entry for the metadata annotation foo.

>>> conn.query("SELECT COUNT(uuid) FROM streams WHERE annotations->$1::text IS NOT NULL", params=["foo"])
[{'count': ...}]

stream_from_uuid(uuid)

Creates a stream handle to the BTrDB stream with the UUID uuid. This method does not check whether a stream with the specified UUID exists. It is always good form to check whether the stream existed using stream.exists().

Parameters:

uuid (UUID) – The uuid of the requested stream.

Returns:

instance of Stream class or None

Return type:

Stream

Examples

>>> import btrdb
>>> conn = btrdb.connect()
>>> uuid = "f98f4b4e-9fab-46b5-8a80-f282059d69b1"
>>> stream = conn.stream_from_uuid(uuid)
>>> stream
<Stream collection=foo/test name=test_stream>

streams(*identifiers, versions=None, is_collection_prefix=False)

Returns a StreamSet object with BTrDB streams from the supplied identifiers. If any streams cannot be found matching the identifier then a StreamNotFoundError will be returned.

Parameters:

• identifiers (str or UUID) – a single item or iterable of items which can be used to query for streams. Identifiers are expected to be UUID as string, UUID as UUID, or collection/name string.

• versions (list[int]) – a single or iterable of version numbers to match the identifiers

• is_collection_prefix (bool, default=False) – If providing a collection string, is that string just a prefix, or the entire collection name? This will impact how many streams are returned.

Returns:

Collection of streams.

Return type:

StreamSet

Examples

With a sequence of uuids.

>>> conn = btrdb.connect()
>>> conn.streams(identifiers=list_of_uuids)
<btrdb.stream.StreamSet at 0x...>

With a sequence of uuids and version numbers. Here we are using version 0 to use the latest data points.

>>> conn.streams(identifiers=list_of_uuids, versions=[0 for _ in list_of_uuids])
<btrdb.stream.StreamSet at 0x...>

Filtering by collection prefix "foo" where multiple collections exist like the following: foo/bar, foo/baz, foo/bar/new, and foo. If we set is_collection_prefix` to True, this will return all streams that exist in the collections defined above. It is similar to a regex pattern ^foo.* for matching purposes.

>>> conn.streams(identifiers="foo", is_collection_prefix=True)
<btrdb.stream.StreamSet at 0x...>

If you set is_collection_prefix to False, this will assume that the string identifier you provide is the full collection name. Matching like the regex here: ^foo

>>> conn.streams(identifiers="foo", is_collection_prefix=False)
<btrdb.stream.StreamSet at 0x...>

streams_in_collection(*collection, is_collection_prefix=True, tags=None, annotations=None, auto_retry=False, retries=5, retry_delay=3, retry_backoff=4)

Search for streams matching given parameters

This function allows for searching

Parameters:

• collection (str) – collections to use when searching for streams, case sensitive.

• is_collection_prefix (bool) – Whether the collection is a prefix.

• tags (Dict[str, str]) – The tags to identify the stream.

• annotations (Dict[str, str]) – The annotations to identify the stream.

• auto_retry (bool, default: False) – Whether to retry this request in the event of an error

• retries (int, default: 5) – Number of times to retry this request if there is an error. Will be ignored if auto_retry is False

• retry_delay (int, default: 3) – initial time to wait before retrying function call if there is an error. Will be ignored if auto_retry is False

• retry_backoff (int, default: 4) – Exponential factor by which the backoff increases between retries. Will be ignored if auto_retry is False

Returns:

A list of Stream objects found with the provided search arguments.

Return type:

list[Stream]

Note

In a future release, the default return value of this function will be a StreamSet

Examples

>>> conn = btrdb.connect()
>>> conn.streams_in_collection(collection="foo", is_collection_prefix=True)
[<Stream collection=foo name=test1>, <Stream collection=foo name=test2,
... <Stream collection=foo/bar, name=testX>, <Stream collection=foo/baz/bar name=testY>]

>>> conn.streams_in_collection(collection="foo", is_collection_prefix=False)
[<Stream collection=foo, name=test1>, <Stream collection=foo, name=test2>]

>>> conn.streams_in_collection(collection="foo",
...  is_collection_prefix=False, tags={"unit":"Volts"})
[<Stream collection=foo, name=test1>]

>>> conn.streams_in_collection(collection="foo",
...  is_collection_prefix=False, tags={"unit":"UNKNOWN"})
[]