Skip to content

Objects Glossary

Jump to bottom
Viktor Gsteiger edited this page Apr 30, 2021 · 5 revisions

Both REST & WebSocket API uses various JSON Objects for query & results. This page lists all shared objects model and their field descriptions for reference. A link to the object description on this page will be available in the whole API Docs wherever that particular object is being used.

Credentials

credentials of an added user in cineast

Credentials : Object
{
  "username": String,
  "password": String
}
  • username is the username of the added user in cineast.
  • password is the corresponding password of the added user in cineast.

ExtractionContainerMessage

A message containing the items to be extracted.

ExtractionContainerMessage : Object
{
  "items": Array<ExtractionItemContainer>
}

ExtractionItemContainer

An ExtractionItemContainer contains all the information for ONE item which is supposed to be extracted. A container MUST contain a path linking to the item to be extracted. The corresponding object MUST contain the mediaType so an item can be handed out to different extraction handlers.

ExtractionItemContainer : Object
{
  "object": MultimediaObjectDescriptor,
  "metadata": Array<MultimediaMetadataDescriptor>
}
  • object contains details about the item to be extracted.
  • metadata to be associated with the object.

Hints

hints used in QueryConfig for tuning the query.

Hints : Enum {
  exact, inexact, lsh, ecp, mi, pq, sh, va, vaf, vav, sequential, empirical
}

IdList

Contains a list of ids that can be used as a payload in request.

IdList : Object
{
  "ids": Array<String>
}
  • ids is an array of String which represents an id.

MediaType

An Enum, the value of which tells the type of media an object is

MediaType : Enum
{
  VIDEO, IMAGE, AUDIO, MODEL3D, UNKNOWN
}

MetadataLookup

This object represents a MetadataLookup message, i.e. a request for a metadata lookup.

MetadataLookup : Object
{
  "objectids": Array<String>,
  "domains": Array<String>,
  "messageType": "M_LOOKUP"
}
  • objectids is a list of object IDs for which metadata should be looked up.
  • domains is a list of metadata domains that should be considered. If empty, all domains are considered!
  • messageType is always M_LOOKUP.

MetadataQueryResult

This object contains list of MultimediaMetadataDescriptors & is received as a part of MetadataLookup results.

MetadataQueryResult : Object
{
  "content": Array<MultimediaMetadataDescriptors>,
  "queryId": String,
  "messageType": "QR_METADATA"
}

MoreLikeThisQuery

This object represents a MoreLikeThisQuery message, i.e. a request for search more like the provided segment.

MoreLikeThisQuery : Object
{
  "segmentId": String,
  "categories": Array,
  "config": QueryConfig,
  "messageType": "Q_MLT"
}
  • segmentId is the id of the segment for which more-like-this query is to be performed.
  • categories is the list of named feature categories that should be considered when doing More-Like-This.
  • config attribute is optional and can be supplied for additional query configurations.
  • messageType is always Q_MLT.

MultimediaMetadataDescriptor

Contains metadata about the object.

MultimediaMetadataDescriptor : Object
{
  "objectId": String,
  "domain": String,
  "key": String,
  "value": String
}
  • objectId is the id of the object this metadata belongs to.
  • domain tells the metadata domain (e.g. EXIF, IPTC, DC...).
  • key is the key of the metadata.
  • value is the value corresponding to the key.

MultimediaObjectDescriptor

Contains details about a particular multimedia object.

MultimediaObjectDescriptor : Object
{
  "objectId": String,
  "name": String,
  "path": String,
  "mediatype": MediaType,
  "contentURL": String
}
  • objectId uniquely identifies a given MultimediaObjectDescriptor object.
  • name is the name of the multimedia object.
  • path is the path of a multimedia object known to Cineast.
  • mediaType is the type of multimedia the object is.
  • contentURL is the absolute path of the multimedia object.

NeighboringSegmentQuery

This object represents a NeighboringSegmentQuery message, i.e. a request for a neighbor-search.

NeighboringSegmentQuery : Object
{
  "segmentId": String,
  "count": Int,
  "config": QueryConfig,
  "messageType": "Q_NESEG"
}
  • segmentId is the id of the segment for which neighbors should be retrieved.
  • count is the Number of neighbors that should be retrieved.
  • config attribute is optional and can be supplied for additional query configurations.
  • messageType is always <a href="Message-Type-Glossary#q_neseg">Q_NESEG</a>.

ObjectQueryResult

This object contains a list of MultimediaObjectDescriptors & is received as a part of query results.

ObjectQueryResult : Object
{
  "content": Array<MultimediaObjectDescriptor>,
  "contentType": String,
  "queryId": String,
  "messageType": "QR_OBJECT"
}
  • content will be a list of MultimediaObjectDescriptors.
  • contentType will be org.vitrivr.cineast.core.data.entities.MediaObjectDescriptor.
  • queryId will be same as what was in the request if specified or else will be randomly generated.
  • messageType will always be QR_OBJECT.

PingRequest

An object to be sent to ping the server

PingRequest : Object
{
  "messageType": "PING"
}
  • messageType is always PING to specify it's a ping request.

PingResponse

An object received contains the ping status of the server.

PingResponse : Object
{
  "status": PingStatus,
  "messageType": "PING"
}
  • status tells the ping result of the server.
  • messageType is always PING.

PingStatus

An enum that specifies all the ping status of the server.

PingStatus : Enum
{
  UNKNOWN, OK, ERROR
}

TemporalQueryComponent

It represents a query container with multiple QueryComponent.

QueryComponent : Object
{
  "stages": Array<QueryComponent>
}

QueryComponent

It represents a query container with multiple QueryTerms.

QueryComponent : Object
{
  "terms": Array<QueryTerm>
}

QueryConfig

A configuration object which is used to tune query.

QueryConfig : Object
{
  "queryId": String
  "hints": Array<Hints>
}
  • queryId will be same as what was in the request if specified or else will be randomly generated.
  • hints is a list of Hints supplied to tune query.

QueryEnd

This object is received at the end of query results when using WebSockets.

QueryEnd : Object
{
  "queryId": String,
  "messageType": "QR_END"
}
  • queryId will be the same as what was in the request if specified or else will be randomly generated.
  • messageType will always be QR_END.

QueryError

This object is received if an error occurs during retrieval.

QueryError : Object
{
  "queryId": String,
  "message": String,
  "messageType": "QR_ERROR"
}
  • queryId will be the same as what was in the request if specified or else will be randomly generated.
  • message is the error message.
  • messageType is always QR_ERROR.

QueryTerm

It represents a particular query term in a QueryComponent.

QueryTerm : Object
{
  "type": QueryTermType,
  "data": String,
  "categories": Array<String>
}
  • type is a QueryTermType which tells the type of query term.
  • QueryTerm Object data format:
QueryTermType data
IMAGE
  • Must be a valid Data URI.
  • Format : "data:image/png;base64,base64String".
  • base64String to be replaced with actual image base64 data.
AUDIO
  • Must be a valid Data URI.
  • Format : "data:audio/wav;base64,base64String".
  • base64String to be replaced with actual audio3 base64 data.
  • Only wav audio is currently supported.
  • Audio settings must be 22050Hz sample rate, Mono channel, 16bit PCM.
MOTION
  • Must be a valid Data URI.
  • Format : "data:application/json;base64,base64String".
  • base64String to be replaced with actual motion base64 data which is a JSON object of model MotionDataModel described below.
MotionDataModel : Object 
{
  "foreground" : Array<Path>,
  "background" : Array<Path>
}

Path : Object 
{
  "path": Array<Point>
}

Point : Object
{
  "x": Float
  "y": Float
}
  • x is relative X position ranging from 0 for extreme left to 1 for extreme right.
  • y is relative Y position ranging from 0 for extreme top to 1 for extreme bottom.
    MODEL3D
    • Supports two types of data - either the JSV4 JSON format for Meshes OR a valid image (for 2D sketch to 3D model lookup).
      1. JSV4 JSON format
        • Must be a valid Data URI.
        • Format : "data:application/3d-json;base64,base64String".
        • base64String to be replaced with valid 3D Model data. The base64String gets parsed into 3D Mesh and treated as Geometry JSON used by the Three.js JavaScript library.
      2. 2D sketch to 3D model lookup
        • Must be a valid Data URI.
        • Format : "data:image/png;base64,base64String".
        • base64String to be replaced with actual image base64 data.
    LOCATION
    • data is a JSON object with model described below
    • Latitude value must lie between [-90,90]
    • Longitude value must lie between [-180,180)
    LocationDataModel : Object 
    {
      "latitude" : Float,
      "longitude" : Float
    }
    TIME
    • The data must be a string which represents a valid instant in UTC. It is parsed using Instant#parse.
    TEXT
    • The data must be a string, the text to search for.
    TAG
    • Must be a valid Data URI.
    • Format : "data:application/json;base64,base64String".
    • base64String to be replaced with actual base64 data which is an array of JSON object of model TagModel described below.
    TagModel : Object 
    {
      "id" : String,
      "name": String,
      "description": String,
      "weight": Float
    }
    • id attribute is required but others can be omitted.
    • default value of weight if not supplied is 1.0
      • QueryTerm Object categories:

      The categories are the collection of features to search for. They can be found here. All the nodes in features block in retriever block are categories with subnodes being the features belonging to that particular category.

      QueryTermType

      An Enum tells the type of query term.

      QueryTermType : Enum
{
  IMAGE, AUDIO, MOTION, MODEL3D, LOCATION, TIME, TEXT, TAG
}

      QueryStart

      This object is received at the start of query results when using WebSockets.

      QueryStart : Object
{
  "queryId": String,
  "messageType": "QR_START"
}
      • queryId will be the same as what was in the request if specified or else will be randomly generated.
      • messageType will always be QR_START.

      SegmentDescriptor

      Contains details about a particular multimedia object's segment.

      SegmentDescriptor : Object
{
  "segmentId": String,
  "objectId": String,
  "start": Int,
  "end": Int,
  "startabs": Float,
  "endabs": Float,
  "sequenceNumber": Int,
  "count": Int
}
      • segmentId uniquely identifies a given SegmentDescriptor object.
      • objectId is the id of the segment's object.
      • start tells the start frame of this segment if applicable else is 0.
      • end tells the end frame of this segment if applicable else is 0.
      • startabs tells the absolute starting time (in s) of this segment if applicable else is 0.
      • endabs tells the absolute ending time (in s) of this segment if applicable else is 0.
      • sequenceNumber is the segment's sequence number. All segments of an object have incremental sequence number starting from 1.
      • count is the total number of frames in this segment.

      SegmentQueryResult

      This object contains a list of SegmentDescriptors & is received as a part of query results.

      SegmentQueryResult : Object
{
  "content": Array<SegmentDescriptor>,
  "contentType": String,
  "queryId": String,
  "messageType": "QR_SEGMENT"
}
      • content will be a list of SegmentDescriptors.
      • contentType will be org.vitrivr.cineast.core.data.entities.MediaSegmentDescriptor
      • queryId will be same as what was in the request if specified or else will be randomly generated.
      • messageType will always be QR_SEGMENT.

      SegmentWeight

      It contains a segment's matched value for a query.

      SegmentWeight : Object
{
  "key": String,
  "value": Float
}
      • key is the segment id for which this object contains matched value.
      • value is the matched value for the query.

      SessionState

      Contains the state of a session

      SessionState : Object
{
  "id": String,
  "validUntil": Long,
  "type": SessionType
}
      • id is the session id.
      • validUntil tells till when the session is valid.
      • type tells the type of session.

      SessionType

      An enum that tells the type of the session.

      SessionType : Enum
{
  UNAUTHENTICATED, USER, ADMIN
}

      TemporalQuery

      This object represents a temporal-query message, i.e. a request for temporal chaining of similarity-searches.

      TemporalQuery : Object
{
  "queries": Array<TemporalQueryComponent>,
  "config": QueryConfig,
  "messageType": "Q_SIM"
}
      • queries is a list of TemporalQueryComponents.
      • config attribute is optional and can be supplied for additional query configurations.

      SimilarityQuery

      This object represents a similarity-query message, i.e. a request for a similarity-search.

      SimilarityQuery : Object
{
  "containers": Array<QueryComponent>,
  "config": QueryConfig,
  "messageType": "Q_SIM"
}
      • containers is a list of QueryComponents.
      • config attribute is optional and can be supplied for additional query configurations.

      SimilarityQueryResult

      This object contains a list of SegmentWeights & is received as a part of query results.

      SimilarityQueryResult : Object
{
  "content": Array<SegmentWeight>,
  "contentType": String,
  "queryId": String,
  "category": String,
  "containerId" Int,
  "messageType": "QR_SIMILARITY"
}
      • content will be a list of SegmentWeight.
      • contentType will be org.vitrivr.cineast.core.data.StringDoublePair.
      • queryId will be the same as what was in the request if specified or else will be randomly generated.
      • category is the category for which this query result is.
      • containerId is 0 if it is a simple similarity query or any integer of 0 and above when it is part of a TemporalQuery.
      • messageType will always be QR_SIMILARITY.

      SimilarityResult

      Contains similarity query results.

      SimilarityResult : Object
{
  "categories": Array<String>,
  "results": Array<SimilarityQueryResult>
}
      • categories will contain all the categories queried.
      • results is a list of SimilarityQueryResults containing results of similarity query.

      StartSessionMessage

      Message to be sent when starting a session.

      StartSessionMessage : Object
{
  "credentials": Credentials
}
      • credentials are the credentials of an added user in cineast.

      Tag

      A tag associated with an object.

      Tag : Object
{
  "id": String,
  "name": String,
  "description": String
}
      • id is the id of the object this tag belongs to.
      • name: name of the tag.
      • description: description of the tag.
      Clone this wiki locally