Skip to content

storage.md

Latest commit

 

History

History
293 lines (240 loc) · 9.18 KB

storage.md

File metadata and controls

293 lines (240 loc) · 9.18 KB

AMO Storage Service

Although it is possible to use AMO blockchain as a data trade medium for any data stored in arbitrary storage, it is a good idea to follow the AMO storage specification to utilize a granting scheme used in AMO blockchain. Any storage service which meets the following requirements is called an AMO storage service.

NOTE: There is a default AMO storage service, and its availability is guaranteed by AMO Labs. See Default AMO Storage Service for more information.

General Requirements

Object storage

AMO blockchain intermediates data trades with the unit called a data parcel. Since there is no restriction in the internal structure of the data, a data parcel can be anything. However, a data parcel is identified by a parcel id without any hierarchical structure. So, it is natural to use an object storage as a physical medium. A hierarchical data storage such as directory-based file system can be transformed into an object storage by using a dedicated directory. In that case, the absolute path to the dedicated directory would be the fixed prefix of a parcel id.

High availability

An AMO storage service needs to provide high availability.

Long-term persistent storage

We do not consider any volatile storages. A data parcel may reside in a storage indefinitely.

Updatable data body

Body of a data parcel may be updated along the course of the lifetime.

Data Parcel

A data parcel is a generic term used to call any data item for sale in AMO infrastructure. Each data parcel should have a globally unique identifier. A data parcel has no intrinsic properties related to internal data structure or contextual meaning. It can be a single byte or a tera byte-sized multi-media data. One requirement is that anyone shall get an identical data from an AMO storage service for the same data parcel identifier and for a given time.

For a convention, empty data body means there is no data parcel registered with a given data parcel ID.

Identifiers

Storage Service ID

An AMO storage service ID is used to identify a specific storage service and its access point. It is a natural requirement that this ID should be globally unique. For this matter, AMO blockchain provide a method to register a storage service by transmitting register transaction. According to AMO blockchain protocol spec, this id is a 32-bit unsigned integer.

Data Parcel ID

A data parcel ID is used to identify a single data parcel in AMO infrastructure. A storage service may introduce its own internal ID to identify a data item or object within the storage. But since each storage service is run independently, there is no guarantee that an internal ID used in a storage service is also unique in whole AMO ecosystem. To solve this problem, a data parcel ID in AMO infrastructure is composed of a storage service ID and an internal ID or index. In a human readable form or in a protocol message, this data parcel ID is represented by concatenation of an 8-digit hexadecimal sequence and a 64-digit hexadecimal sequence, which is a hexadecimal representation of 36-byte binary sequence.

One possible strategy to generate an internal ID for a data parcel is to take a SHA-256 hash of the (metadata, payload), or just the payload. When the storage id is an integer 1, a data parcel ID would have the form like the following (every zero-byte(0x00) counts):

000000010123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF

Protocol Messages

User Identity

"_account_address_"
  • address is a hex-encoded 20-byte binary sequence. address is used to uniquely identify a user account in AMO ecosystem. See AMO Blockchain Protocol Specification for more information.

Signature Appendix

{
	"pubkey": "_HEX-encoded_ECDSA_public_key_",
	"sig_bytes": "_HEX-encoded_ECDSA_signature_"
}
  • pubkey is a hexadecimal representation of a compressed elliptic curve point. See AMO Blockchain Protocol Specification for more information.
  • sig_bytes is a hexadecimal representation of concatenated ECDSA signature bytes.

Data Parcel Identifier

"_hex-encoded_binary_sequence_"

Data Parcel

{
	"id": _data_parcel_ID_,
	"owner": _user_identity_,
	"metadata": _metadata_,
	"data": _hex-encoded_binary_sequence_
}
  • id may be omitted when uploading a new data to a storage service. If this is the case, a new data parcel id is generated by the storage service and then returned to the client.
  • metadata is any JSON object with a mandatory field owner.

Initiation Message Type 1

{
	"user": _user_identity_,
	"operation": _operation_description_,
	"signature": _signature_appendix_
}
  • signing bytes for the signature is the initiation message itself without signature field.

Initiation Message Type 2

{
	"user": _user_identity_,
	"operation": _operation_description_
}

Auth Challenge Message

{
	"challenge": "_hex-encoded_256-bit_sequence_"
}
  • challenge is a hexadecimal representation of a 256-bit binary sequence

Auth Response Message

{
	"signature": _signature_appendix_
}
  • signing bytes for the signature is the concatenation of initiation message and challenge message.

Result Message

{
	"code": _integer_,
	"data": _operation_specific_result_data_
}

API Operations

There three tiers in API operations

  • tier 0: download-only without authentication
  • tier 1: upload, inspect, download, remove opertions with authentication
  • tier 2: append, replace operations plus all the tier 1 operations with authentication

Tier 0 operations should only be used in a simple type of storage. Since it does not support authentication or access control, such a service may not be considered as fully compatible AMO storage. However, this type of storage may be essential in the early stages of the AMO business.

Tier 0 operations are processed with a simple HTTP GET request and response. Unlike tier 1 and 2 operations, the request is sent to the URL pointing to the data parcel in question.

All of tier 1 and tier 2 API operations are processed with a series of protocol messages. It is assumed that all protocol messages are sent to the same API endpoint. Each protocol step is distinguished by the operation description transferred by the first client message, and the order of subsequent messages.

(Tier 0) Simple download

Operation steps are as follows:

  • client ⇒ storage: HTTP GET with no request body
  • client ⇐ storage: data parcel body or fail

The tier 0 simple download could be supported by any standard web server. This operation can be considered as a wide-accepted RESTful API. So the URL pointing to each data parcel should look like the following form:

https://serveraddress:port/prefix/to/path/<data parcel id in hexadecimal form>

In this case, all of the data parcels should be accessible under the same URL prefix.

(Tier 1) Upload

Operation description is as follows:

{
	"name": "upload",
	"hash": "_hex-encoded_256-bit_hash_"
}
  • hash is SHA256 output of the data body.

Operation steps are as follows:

  • client ⇒ storage: initiation message type 1
  • client ⇐ storage: continue or fail
  • client ⇒ storage: data parcel
  • client ⇐ storage: result message with no data field

Upon receiving initiation message, storage service shall perform signature verification and response to the client with either continue or fail. No access control than this.

Result data is a data parcel ID. need discussion

(Tier 1) Inspect

Operation description is as follows:

{
	"name": "inspect",
	"id": _data_parcel_ID_
}

Operation steps are as follows:

  • client ⇒ storage: initiation message type 2
  • client ⇐ storage: result message

No access control at this operation.

Result data is a data parcel metadata.

(Tier 1) Download

Operation description is as follows:

{
	"name": "download",
	"id": _data_parcel_ID_
}

Operation steps are as follows:

  • client ⇒ storage: initiation message type 2
  • client ⇐ storage: auth challenge
  • client ⇒ storage: auth response
  • client ⇐ storage: result message

Upon receiving auth response, storage service shall access AMO blockchain to perform AMO-based access control.

Result data is either empty or a data parcel depending on the access control result.

(Tier 1) Remove

Operation description is as follows:

{
	"name": "remove",
	"id": _data_parcel_ID_
}

Operation steps are as follows:

  • client ⇒ storage: initiation message type 2
  • client ⇐ storage: auth challenge
  • client ⇒ storage: auth response
  • client ⇐ storage: result message

Upon receiving auth response, storege service shall perform local identity matching.

Result data is empty.

(Tier 2) Append

TBA

optional. tier 2 oepration

(Tier 2) Replace

TBA

optional. tier 2 oepration

Access Control

AMO-Based Access Control

TODO

See Usage query for how to query via RPC channel.

Local Identity Matching

It's a simple account address matching. Identity from the user is reliable enough, and if it exactly matches the owner field in the saved data parcel the access control result is success. Fail otherwise.