Gen3 Metadata Class

class gen3.metadata.Gen3Metadata(endpoint=None, auth_provider=None, service_location='mds', admin_endpoint_suffix='-admin')[source]

Bases: object

A class for interacting with the Gen3 Metadata services.

Examples

This generates the Gen3Metadata class pointed at the sandbox commons while using the credentials.json downloaded from the commons profile page.

>>> auth = Gen3Auth(refresh_file="credentials.json")
... metadata = Gen3Metadata(auth)
endpoint

public endpoint for reading/querying metadata - only necessary if auth_provider not provided

Type:

str

auth_provider

auth manager

Type:

Gen3Auth

async async_create(guid, metadata, aliases=None, overwrite=False, _ssl=None, **kwargs)[source]

Asynchronous function to create metadata

Parameters:

  • guid (str) – guid to use

  • metadata (Dict) – dictionary representing what will end up a JSON blob attached to the provided GUID as metadata

  • overwrite (bool, optional) – whether or not to overwrite existing data

  • _ssl (None, optional) – whether or not to use ssl

async async_create_aliases(guid, aliases, _ssl=None, **kwargs)[source]

Asyncronously create Aliases for the given guid

Parameters:

  • guid (TYPE) – Globally unique ID for the metadata blob

  • aliases (list[str]) – Aliases to set for the guid

  • _ssl (None, optional) – whether or not to use ssl

  • **kwargs – additional query params

Returns:

response from the request to create aliases

Return type:

requests.Response

async async_delete_alias(guid, alias, _ssl=None, **kwargs)[source]

Asyncronously delete single Aliases for the given guid

Parameters:

  • guid (str) – Globally unique ID for the metadata blob

  • alias (str) – alternative identifier (alias) to delete

  • _ssl (None, optional) – whether or not to use ssl

  • **kwargs – additional query params

Returns:

response from the request to delete aliases

Return type:

requests.Response

async async_delete_aliases(guid, _ssl=None, **kwargs)[source]

Asyncronously delete all Aliases for the given guid

Parameters:

  • guid (TYPE) – Globally unique ID for the metadata blob

  • _ssl (None, optional) – whether or not to use ssl

  • **kwargs – additional query params

Returns:

response from the request to delete aliases

Return type:

requests.Response

async async_get(guid, _ssl=None, **kwargs)[source]

Asynchronous function to get metadata

Parameters:

  • guid (str) – guid to use

  • _ssl (None, optional) – whether or not to use ssl

Returns:

metadata for given guid

Return type:

Dict

async async_get_aliases(guid, _ssl=None, **kwargs)[source]

Asyncronously get Aliases for the given guid

Parameters:

  • guid (TYPE) – Globally unique ID for the metadata blob

  • _ssl (None, optional) – whether or not to use ssl

  • **kwargs – additional query params

Returns:

response from the request to get aliases

Return type:

requests.Response

async async_update(guid, metadata, aliases=None, merge=False, _ssl=None, **kwargs)[source]

Asynchronous function to update metadata

Parameters:

  • guid (str) – guid to use

  • metadata (Dict) – dictionary representing what will end up a JSON blob attached to the provided GUID as metadata

  • aliases (list[str], optional) – List of aliases to update the GUID with

  • merge (bool, optional) – Whether or not to merge metadata AND aliases with existing values

  • _ssl (None, optional) – whether or not to use ssl

  • **kwargs – Description

async async_update_aliases(guid, aliases, merge=False, _ssl=None, **kwargs)[source]

Asyncronously update Aliases for the given guid

Parameters:

  • guid (TYPE) – Globally unique ID for the metadata blob

  • aliases (list[str]) – Aliases to set for the guid

  • merge (bool, optional) – Whether or not to aliases with existing values

  • _ssl (None, optional) – whether or not to use ssl

  • **kwargs – additional query params

Returns:

response from the request to update aliases

Return type:

requests.Response

batch_create(metadata_list, overwrite=True, **kwargs)[source]

Create the list of metadata associated with the list of guids

Parameters:

  • (List[Dict{"guid" (metadata_list) – “”, “data”: {}}]): list of metadata objects in a specific format. Expects a dict with “guid” and “data” fields where “data” is another JSON blob to add to the mds

  • overwrite (bool, optional) – whether or not to overwrite existing data

create(guid, metadata, aliases=None, overwrite=False, **kwargs)[source]

Create the metadata associated with the guid

Parameters:

  • guid (str) – guid to use

  • metadata (Dict) – dictionary representing what will end up a JSON blob attached to the provided GUID as metadata

  • overwrite (bool, optional) – whether or not to overwrite existing data

create_aliases(guid, aliases, **kwargs)[source]

Create Aliases for the given guid

Parameters:

  • guid (TYPE) – Globally unique ID for the metadata blob

  • aliases (list[str]) – Aliases to set for the guid

  • **kwargs – additional query params

Returns:

response from the request to create aliases

Return type:

requests.Response

create_index_key_path(path)[source]

Create a metadata key path indexed in the database.

Parameters:

path (str) – metadata key path

delete(guid, **kwargs)[source]

Delete the metadata associated with the guid

Parameters:

guid (str) – guid to use

delete_alias(guid, alias, **kwargs)[source]

Delete single Alias for the given guid

Parameters:

  • guid (TYPE) – Globally unique ID for the metadata blob

  • alias (str) – alternative identifier (alias) to delete

  • **kwargs – additional query params

Returns:

response from the request to delete aliases

Return type:

requests.Response

delete_aliases(guid, **kwargs)[source]

Delete all Aliases for the given guid

Parameters:

  • guid (TYPE) – Globally unique ID for the metadata blob

  • **kwargs – additional query params

Returns:

response from the request to delete aliases

Return type:

requests.Response

delete_index_key_path(path)[source]

List all the metadata key paths indexed in the database.

Parameters:

path (str) – metadata key path

get(guid, **kwargs)[source]

Get the metadata associated with the guid :param guid: guid to use :type guid: str

Returns:

metadata for given guid

Return type:

Dict

get_aliases(guid, **kwargs)[source]

Get Aliases for the given guid

Parameters:

  • guid (TYPE) – Globally unique ID for the metadata blob

  • **kwargs – additional query params

Returns:

response from the request to get aliases

Return type:

requests.Response

get_index_key_paths()[source]

List all the metadata key paths indexed in the database.

Returns:

list of metadata key paths

Return type:

List

get_version()[source]

Return the version

Returns:

the version

Return type:

str

is_healthy()[source]

Return if is healthy or not

Returns:

True if healthy

Return type:

bool

query(query, return_full_metadata=False, limit=10, offset=0, use_agg_mds=False, **kwargs)[source]

Query the metadata given a query.

Query format is based off the logic used in the service:

‘’’ Without filters, this will return all data. Add filters as query strings like this:

GET /metadata?a=1&b=2

This will match all records that have metadata containing all of:

{“a”: 1, “b”: 2} The values are always treated as strings for filtering. Nesting is supported:

GET /metadata?a.b.c=3

Matching records containing:

{“a”: {“b”: {“c”: 3}}} Providing the same key with more than one value filters records whose value of the given key matches any of the given values. But values of different keys must all match. For example:

GET /metadata?a.b.c=3&a.b.c=33&a.b.d=4

Matches these:

{“a”: {“b”: {“c”: 3, “d”: 4}}} {“a”: {“b”: {“c”: 33, “d”: 4}}} {“a”: {“b”: {“c”: “3”, “d”: 4, “e”: 5}}} But won’t match these:

{“a”: {“b”: {“c”: 3}}} {“a”: {“b”: {“c”: 3, “d”: 5}}} {“a”: {“b”: {“d”: 5}}} {“a”: {“b”: {“c”: “333”, “d”: 4}}} ‘’’

Parameters:

  • query (str) – mds query as defined by the metadata api

  • return_full_metadata (bool, optional) – if False will just return a list of guids

  • limit (int, optional) – max num records to return

  • offset (int, optional) – offset for output

Returns:

list of guids matching query

OR if return_full_metadata=True

Dict{guid: {metadata}}: Dictionary with GUIDs as keys and associated

metadata JSON blobs as values

Return type:

List

update(guid, metadata, aliases=None, merge=False, **kwargs)[source]

Update the metadata associated with the guid

Parameters:

  • guid (str) – guid to use

  • metadata (Dict) – dictionary representing what will end up a JSON blob attached to the provided GUID as metadata

update_aliases(guid, aliases, merge=False, **kwargs)[source]

Update Aliases for the given guid

Parameters:

  • guid (TYPE) – Globally unique ID for the metadata blob

  • aliases (list[str]) – Aliases to set for the guid

  • merge (bool, optional) – Whether or not to aliases with existing values

  • **kwargs – additional query params

Returns:

response from the request to update aliases

Return type:

requests.Response