diff --git a/fatips/0.md b/fatips/0.md index 54d7b68..5d81955 100644 --- a/fatips/0.md +++ b/fatips/0.md @@ -186,6 +186,11 @@ and so are ignored to prevent replay attacks. | --------- | ------ | ------------------------------------- | ------------------------------------------------------------ | -------- | | `inputs` | object | The inputs of the transaction | Mapping of Public Factoid Address => Amount. Amount must be am integer greater than or equal to zero. May not be empty or contain duplicate Addresses. | Y | | `outputs` | object | The outputs of the transaction | Mapping of Public Factoid Address => Amount. Amount must be am integer greater than or equal to zero. May not be empty or contain duplicate Addresses. May overlap with addresses found in `inputs` to form a send-to-self transaction. Sum of the Amounts must equal the sum of the `inputs` Amounts. | Y | +| | | | | | +| `contract` | string | The contract publication chain to use to establish a contract at output address 0 | Valid Factom chain ID. Valid [FATIP-104](104.md) contract datastore. May not be specified alongside `func` or `args`. See [Contract Publication Transactions](#Contract-Publication-Transactions). | N | +| `func` | string | The full function name to call | Function must exist in the ABI of the contract at `contract`. Must be specified with `args`. See [Contract Call Transactions](#Contract-Call-Transactions). | N | +| `args` | array | The array of arguments to call `func` with | Arguments must comply with the function signature argument types specified in the ABI at `contract`. Must be specified with `func`. See [Contract Call Transactions](#Contract-Call-Transactions). | N | +| | | | | | | `metadata` | any | Optional metadata defined by user | This may be any valid JSON type. | N | For a Transaction to be well-formed it must follow the above defined structure @@ -246,6 +251,22 @@ described above, but with the Coinbase Address as the sole input in the output addresses may not intersect. Thus a Coinbase Transaction may not directly burn the tokens it issues. +### Contract Publication Transactions + +Contract Publication Transactions are transactions that establish a WASM based smart contract at a given Factoid address. Establishing a contract at an address removes the address from human control and delegates control of the address to the contracts code. + +Contract publication transactions are identified by a zero amount input-output transaction from the desired contract address to itself, while specifying the `contract` key value pair in the Transaction JSON. The `contract` field specifies the [FATIP-104](104.md) contract chain to use at the address. + +An address may hold a balance of FAT-0 tokens before having a contract established, allowing contracts to have a starting balance. + +### Contract Call Transactions + +Contract Call Transactions are transactions that call a specific function with given arguments inside of a contract established at an address. The field `func` and `args` are used to carry the intended function to call and the arguments to call it with. + +To call a contract, a single-input single-output transaction is constructed with the output being the contract Factoid address and `func` & `args` in the main JSON body. A nonzero amount of FAT-0 tokens may be sent in the transaction to denote a fee or other use of tokens. + +Normal transactions may be sent to a contract address without `func` & `args` to deposit tokens to the contract address. + #### Coinbase Signing Set A signature from the current key establised by the Issuer's Identity key is @@ -291,7 +312,7 @@ Normal transactions must meet all T.x and N.x requirements. [FATIP-103](103.md). No additional RCD/Signature pairs beyond those that correspond with an input may be included. -### C.x Requirements for Coinbase distribution transactions +### C.x Requirements for Coinbase transactions Coinbase transactions must meet all T.x and C.x requirements. @@ -302,17 +323,38 @@ Issuance entry (if not unlimited). - C.3.1: The entry must be signed by the Issuer's currently established key in the Identity chain according to [FATIP-101](101.md) and [FATIP-103](103.md). +### CP.x Requirements for Contract Publication Transactions + +Contract Publication Transactions must meet all T.x and CP.x requirements. + +- The target address must be the only input +- The target address must be the only output +- The sum of all amounts must equal zero +- The target address may not already have a contract established on it +- The contract at `contract` must pass all validation, `func` and `args` may not be specified + +### CC.x Requirements for Contract Call Transactions + +Contract Publication Transactions must meet all T.x and CC.x requirements. + +- The calling address must be the only input +- The target address must be the only output +- The target address must already have a contract established on it +- The `func` and `args` field must be present and pass validation. The `func` and `args` submitted must match and comply with the function signature of `func`'s `args` specified in the contract ABI. + + + ## Computing the Current State Implementations must maintain the state of the balances of all addresses in order to evaluate the validity of a transaction. The current state can be built by iterating through all entries in the token chain in chronological order and updating the state for any valid transaction. +Additionally, implementations must maintain the state of contracts and their calls, which are conducted by FAT transactions and may affect the balances of their host token. Contracts have the ability to send, receive, and burn their host FAT tokens. Each contract call results in a return value from the function and arguments submitted, which is then persisted in the state of the contract and linked to the transaction. Thus, the result of a contract call is obtained and identified by it's transaction entryhash. + The following pseudo code describes how to compute the current state of all balances. A transaction must be applied entirely or not at all. Entries that -are not valid transactions are simply ignored. Transactions must be evaluated -in the order that they appear in the token chain. This assumes the token has -already been properly initialized. +are not valid transactions are simply ignored. Contract call results, including errors stemming from Contract Call Transactions are independent of transaction validity, and are persisted in the implementation for later retrieval. Transactions must be evaluated in the order that they appear in the token chain. This assumes the token has already been properly initialized. ``` for entry in token_chain.entries: @@ -320,8 +362,14 @@ for entry in token_chain.entries: if !entry.is_coinbase_transaction(): for input in entry.inputs: balances[input.address] -= input.amount - for output in entry.outputs: - balances[output.address] += output.amount + for output in entry.outputs: + balances[output.address] += output.amount + else balances[entry.outputs[0]] += output.amount + + if entry.is_contract_call(): + transaction.result = token_chain.contracts[entry.inputs[0]][func](args...) + else if entry.is_contract_publication(): + token_chain.contracts[entry.inputs[0]] = entry.contract ``` # Implementation diff --git a/fatips/104.md b/fatips/104.md new file mode 100644 index 0000000..1865500 --- /dev/null +++ b/fatips/104.md @@ -0,0 +1,124 @@ +| FATIP | Title | Status | Category | Author | Created | +| ----- | ----------------------------- | ------ | -------- | --------------------------------- | --------- | +| 104 | Contract Publication Standard | WIP | Core | Devon Katz\<\> | 9-16-2019 | + + + +# Summary + +This standard describes a type of FATIP-107 Factom Data Store with specialized validation used to publish WebAssembly(WASM) contract bytecode to a Factom chain. This standard is used in conjunction with FATIP-0 to establish contracts at Factoid addresses, which then can be interacted with by users of FAT via transactions. + +An Application Binary Interface(ABI) object structure is defined in this standard. The ABI lists the functions and arguments are available to be called in the contract, as well return types of those functions and is critical for interacting with contracts. + +# Motivation + +Our [proof of +concept](https://github.com/Factom-Asset-Tokens/wasm-contract-poc) successfully +demonstrated that raw WASM bytecode could successfully be stored published and +interpreted from the External Ids of Factom entry. While effective, this simple +system created a hard limit of 10KB in contract size as it used a single entry. + +This standard uses the robust FATIP-107 datastore standard to publish WASM contract binaries to the Factom blockchain. FATIP-107 gives contract code a dedicated chain, allowing the reuse of established +and vetted contracts by chain Id. For example, an escrow contract established by a law firm +could be implemented directly by multiple different FAT tokens & contracts at +their addresses by specifying the chain Id of the contracts compiled source +code. + +# Specification + +This standard uses a FATIP-107 Data Store to host contract code and specify a contract ABI. Therefore, all functionality defined in this standard is built on top of the content, metadata, and name Ids of a FATIP-107 Data Store. No additional functionality is added to FATIP-107 as a result of this standard. + +## Data Block Entries + +The FATIP-107 Data Store data block entries shall be filled with the complete raw byte buffer of the compiled WASM bytecode. + +The total size FATIP-107 content shall be limited to 100KB or less in final on-chain size, regardless of compression. Contract bytecode may be compressed as per FATIP-107 if desired. + +## Content Metadata Object + +The FATIP-107 Data Store metadata residing in the content object shall be used to define the contract's Application Binary Interface. + +There is no limit on the number of functions or their respective arguments; however, the datastore content object must necessarily be less than 10KB in size as per Factom's per-entry size limit. + +### Application Binary Interface + +The ABI is a datastructure that defines how the higher level smart contract +platform interfaces with the low level binary WASM code. At it's heart a WASM +VM is a very low level machine which needs help understanding how to interpret +more complex types like strings from the host environment, and vice versa. + +The ABI is defined in the root of the `metadata` key in the Content Metadata Object. All fields are inheritied and comply to FATIP-107's Content Metadata Object requirements. + +### Content Metadata Object Example + +```json +{ + "data-store": "1.0", + "size": 999, + "dbi-start": "4da6c4ba7c9f3d01a52df43ae5d67d7d8b3909babf6817463b8ceff3c74065ee", + "metadata": { + "_add":{ + "args":[ + "i32", + "i32" + ], + "returns":"i32" + } + } +} +``` + +### Content Metadata Object Field Summary & Validation + +Please note validation of fields inherited from FATIP-107 is not covered in this table. + +| Name | Type | Description | Validation | Required | +| --------------------- | ------ | ------------------------------------------------------------ | ------------------------------------------------------------ | -------- | +| `metadata[*]` | string | The full function name | | Y | +| `metadata[*].args` | array | The ordered array of argument types that the function signature accepts. | All elements must be strings. Must be one of the [Supported Argument And Return Types](https://github.com/Factom-Asset-Tokens/FAT/blob/FATIP-S-Smart-Contracts-And-Supporting-Standards/fatips/104.md#Supported-Arguments-and-Return-Types) | Y | +| `metadata[*].returns` | string | The type of return value to expect from the function | Must be a string. Must be one of the [Supported Argument And Return Types](https://github.com/Factom-Asset-Tokens/FAT/blob/FATIP-S-Smart-Contracts-And-Supporting-Standards/fatips/104.md#Supported-Arguments-and-Return-Types) | Y | +| | | | | | + +##### Supported Arguments and Return Types + +- `i32` - 32 Bit integer +- `i64` - 64 Bit integer +- `string` - `i32` pointer to an array of null terminated character bytes in + linear memory +- `array[i32]` - `i32` pointer to an array of `i32` bytes in linear memory +- `array[i64]` - `i32` pointer to an array of `i64` bytes in linear memory + +##### Functions + +For a contract function to be callable externally it must exist in the ABI +object. Typically C compilers and linkers will append an underscore onto the +function name in code when making an exported function available. For example, +in C `add(int x)` will be exported as `_add `. + +Note that it is only possible to derive a function's name, not it's signature +from a WebAssembly binary. The publisher must to declare the proper function +argument types, counts, and ordering in the ABI to avoid undefined behavior. + + + +## Contract Validation + +The contract residing in the FATIP-107 Data Store must be validated to be considered a valid FAT compatible contract: + +- The resulting data bust be a valid WASM binary (See this standard's [Implementation](#Implementation) section for tooling notes) +- All functions defined inside the ABI must be present in the compiled contract and externally callable by the host environment. However, it is not required that all internal or external functions in the contract be defined in the ABI so as to restrict host access to certain functions within the contract. + + + +# Implementation + +The official [WebAssembly Toolkit +(WABT)](https://github.com/WebAssembly/wabt)'s +[wasm-validate](https://webassembly.github.io/wabt/doc/wasm-validate.1.html) +command line tool is a very popular way of validating WASM binaries. + + +# Copyright + +Copyright and related rights waived via +[CC0](https://creativecommons.org/publicdomain/zero/1.0/). diff --git a/fatips/105.md b/fatips/105.md new file mode 100644 index 0000000..cc6a69c --- /dev/null +++ b/fatips/105.md @@ -0,0 +1,72 @@ +| FATIP | Title | Status | Category | Author | Created | +| ----- | ----------------------- | ------ | -------- | --------------------------------- | --------- | +| 105 | Host Interface Standard | WIP | Core | Devon Katz\<\> | 9-17-2019 | + + + +# Summary + +This standard describes the basic host interface that a WASM based contract has access to as a FAT token contract. The interface is used to allow contracts access to critical context information from the host environment, which would ordinarily be completely restricted. This specification includes a basic mechanism for storing data persistently across contract calls, allowing stateful behavior for FAT contracts. + +# Motivation + +FAT smart contracts have the ability to securely execute trustless logic; However, with this security comes seclusion and isolation. Contracts may need access to more context info than a single contract call's `func` and `args` can contain to do their job. + +For example: + +- What is the calling transaction's entryhash? +- How many tokens were sent in the transaction? +- Who sent the transaction? +- What is the current balance at the contract? +- What is the current timestamp or block height? +- What value was placed in addressable persistent storage from the last call? + +For even basic contract applications, this vital information is required. This standard seeks to define a standard namespace and set of functions & constants WASM contracs can use to conduct their operations + + + +# Specification + +To access the host's functions, they must be declared as external functions in the WASM code. For example, in C, this is how the contract would declare the function to return the current blockheight from the host: + +```c +extern int getHeight(void); +``` + +The functions mentioned in this document are defined identically with a root namespace, albeit given different argument and return types. + +Since the interface is only is used by the host, functions may be omitted if not required by the contract. + + + +## Supported Functions + +| Name | Description | Arguments (C) | Return (C) | Declaration Example (C) | +| -------------- | ------------------------------------------------------------ | ------------------------------------------------------------ | -------------------------------------------------- | ----------------------------------- | +| `getSender` | Get the calling tx's input Factoid address | `void` | `char *` | ```extern char * getInput(void);``` | +| `getAmount` | Get the calling tx's output amount | `void` | `int` | | +| `getEntryhash` | Get the calling tx's Factom Entryhash | `void` | `char *` | | +| `getTimestamp` | Get the unix timestamp of the calling tx | `void` | `int` | | +| `getHeight` | Get the Factom blockheight of the calling tx | `void` | `int` | | +| `getAddress` | Get the Factoid address of this contract | `void` | `char *` | | +| | | | | | +| `getStorageAt` | Get a 256 bit value from the contract's persistent storage | `int` (256 Bit) | `int` (256 Bit) | | +| `setStorageAt` | Set a 256 bit value in the contract's persistent storage | `int` ( 256 Bit) | `int` (256 Bit) | | +| | | | | | +| `getPrecision` | Get the decimal precision of the host token | `void` | `int` | | +| `getBalance` | Get the FAT-0 balance of a Factoid address on the host token | `char *` - The Factoid address string | `int` | | +| `send` | Send FAT-0 tokens from the contracts balance | `char *` - The Factoid address string destination, `int` - The amount of tokens to send in base units | `int` - The boolean success value of the operation | | +| `burn` | Burn the specified amount of tokens from the contracts balance | `int` - The amount of tokens to burn | `void` | | +| | | | | | +| `revert` | Revert the current contract calls state changes and abort the call. Will still charge the input amount | `void` | `void` | | +| `invalidate` | Invalidate the calling transaction and abort state changes. Refunds input amount to caller | | | | +| `selfDestruct` | Terminate the current contract, liquidating the FAT-0 balance to a Factoid address | `char *` - the liquidation destination Factoid address | `void` | | + + + +# Implementation + +# Copyright + +Copyright and related rights waived via +[CC0](https://creativecommons.org/publicdomain/zero/1.0/). \ No newline at end of file diff --git a/fatips/107.md b/fatips/107.md new file mode 100644 index 0000000..a6e36c0 --- /dev/null +++ b/fatips/107.md @@ -0,0 +1,230 @@ +| FATIP | Title | Status | Category | Author | Created | +| ----- | -------------------------- | -------- | -------- | ------------------------------------------ | -------- | +| 107 | Data Storage Chain | WIP | Core | Adam S Levy \ | 2019-11-07 | + + +# Factom Data Store + +A robust protocol for storing data of nearly arbitrary size on the Factom +Blockchain. + +## Features +- Secure - ChainID uniquely identifies the exact data within an optional + application defined Namespace. +- Extensible - Applications may define any arbitrary metadata to attach to the + data store. +- DOS proof - The data blocks may appear in any order, on any chain, and may be + interspersed with unrelated or garbage entries. +- Efficient - Clients only need to download the first modestly sized entry in + the chain to discover the total data size, and exact number of entries that +they will need to download. All required Data Block Entries can be discovered +and then downloaded concurrently. +- Arbitrary size - There is no theoretical limit to the size of data that may + be stored. However most clients will likely enforce sane practical limits +before downloading a file. +- Censorship resistent - Commit all entries before revealing any to ensure that + a data store cannot be censored. +- Optional compression - Use gzip, zlib, or none. + +## Specification + +#### Abstract + +The Factom Blockchain has a maximum Entry size of 10240 bytes. Applications +that wish to store contiguous data that exceed this limit must break up the +data across many Entries. This standard defines a generic Data Store Chain +protocol for storing data of arbitrary size, while allowing efficient lookup of +the data by Chain ID, which can be derived using the data's hash and an +optional application defined Namespace data. + +#### Summary + +Raw Data of arbitrary size is broken up into Data Blocks and stored in Data +Block Entries. The Data Block Entry Hashes are recorded in their proper order +in the Data Block Index. The Data Block Index (DBI) is stored in a linked list +of DBI Entries. The First Entry of a Data Store Chain records the first DBI +Entry Hash of the linked list, along with size, compression, and any +application defined Metadata. The Chain ID of the Data Store Chain is derived +from the sha256d data hash, along with any optional application defined +namespace data. Thus the Chain ID uniquely identifies a piece of data within an +application defined Namedspace. + + +### First Chain Entry + +The First Chain Entry establishes the sha256d hash of the data, its size, +compression details, the first Data Block Index Entry Hash, and any optional +application defined Namespace data or Metadata. + + +#### NameIDs/ExtIDs + +The sha256d hash and the Namespace data are used to compute the Chain ID. This +allows for applications to lookup or deduplicate data within their own +Namespace by Chain ID. + +##### Namespace + +A Data Store may be defined within an optional application defined Namespace. +The Namespace is an arbitrary set of ExtIDs that are appended to the NameIDs of +a Data Store Chain. Thus the Namespace must be known by a client in order to +compute the Chain ID for a Data Store. For this reason it is not recommended to +use the Namespace to describe the data. Use the Application Metadata which is +stored in the Content of the First Entry instead for metadata that describes +the data. + +| i | Type | Description | +|- |- |- | +| 0 | string | "data-store", A human readable marker that defines this data protocol. | +| 1 | Bytes32 | The sha256d hash of the data. | +| ... | (any) | Optional application defined Namespace IDs... | + +#### Content + +The Content is a Metadata JSON Object which includes the information required +to reconstruct the data along with any application defined metadata. This +includes the first DBI Entry, data size, and any optional compression details. + +Note: Normally DBI Entries and Data Blocks will exist on the same Data Store +Chain that references them, but this is not a requirement. The First Entry may +reference any valid DBI Entry on any Chain. This allows Data Stores to be +created that reference pre-existing DBI Entries from a different Namespace. + +##### Metadata Object +| Name | Type| Description | +|-|-|-| +| "data-store" | string | Protocol version, currently "1.0" | +| "size" | uint64 | Total data size | +| "dbi-start" | Bytes32 | The hash of the first DBI Entry as a hex string | +| "compression" | Compression Object | Optional compression details, omit if no compression is used | +| "metadata" | (any) | Optional application defined Metadata | + +##### Compression Object + +Data may optionally be compressed before it is stored on chain. Currently this +standard defines the use of the following compression formats: zlib, gzip. +Compression details are stored in a JSON Compression Object with the following +fields. + +| Name | Type| Description | +|-|-|-| +| "format" | string | "zlib" or "gzip" | +| "size" | uint64 | Total compressed data size | + +### Data Block Index Entry + +A Data Block Index Entry contains all or part of the Data Block Index (DBI) +which defines all the Data Block Entry Hashes in the proper order to +reconstruct the data, or the compressed data. + +If the DBI does not fit into a single Entry, the DBI is split into the shortest +possible linked list of DBI Entries. The linked list is formed by the first +ExtID referencing the Hash of the next DBI Entry, if it exists. + +#### ExtIDs +| i | Type | Description | +|-|-|-| +| 0 | Bytes32 | The Hash of the next DBI Entry, if it exists. | + +#### Content + +The Data Block Index is a binary data structure which is simply the ordered +concatenation of all raw 32 byte Data Block Entry Hashes. The Content of a DBI +Entry is as many complete Data Block Entry Hashes that may fit in the remaining +space of the Entry. Only the final DBI Entry in a linked list may be not full. + +An Entry contains 10240 bytes. Thus a maximum of `10240/32 = 320` Entry Hashes +may fit in a single DBI entry. However, if more than 320 hashes are required, +an ExtID with the next DBI Entry Hash must be included, which is `2+32 = 34` +bytes. This means that only `10240-34 = 10206` bytes remain, which only leaves +space for `10206/32 = 318` DBI Entry Hashes. + +Note: While it would be possible to parse a DBI Entry Linked List with DBI +Entries that did not use all of their available space, this opens up the +possibility for the creation of Data Stores that cause a client to download +more Entries than strictly necessary. Such Data Stores should be considered +invalid. + +### Data Block Entry + +A Data Block Entry contains a piece of the Data referenced by the DBI. The Data +is split into as few Data Block Entries as possible. Thus only the last Data +Block in a DBI may be not full. + +Note: While it would be possible to parse a set of Data Blocks that did not use +all of their available space, this opens up the possibility for the creation of +Data Stores that cause a client to download more Entries than strictly +necessary. Such Data Stores should be considered invalid. + +#### ExtIDs + +A Data Block Entry may not have any ExtIDs. + +#### Content + +A block of the raw data. The Content must include as much raw data as possible. +Thus only the last Data Block in a DBI may be not full. + +### Writing a data store + +1. Compute the Chain ID +- Compute the hash of the data. `sha256d(data)` +- Construct the ExtIDs of the First Entry with any application defined + Namespace data. +- Compute the ChainID + `sha256(sha256(ExtID[0])|sha256(ExtID[1])|...|sha256(ExtID[n]))` + +2. Build the Data Block Entries +- Optionally compress the data using zlib or gzip. +- Construct `len(compressData)/10240` Entries (`+1` if + `len(compressedData)%10240 > 0`). +- Sequentially fill the Content of the Entries as much of the data as possible, + populate the ChainID, and compute the Entry Hashes. + +3. Build the Data Block Index Entries +- Construct Data Block Index Entries as follows until no more Data Block Hashes + remain. + a. If the number of remaining Data Block Entry Hashes are less than or +equal to 320, put all remaining hashes in this DBI Entry. + b. Else, put 318 hashes in the content, and create another DBI Entry. +- Compute the DBI Entry Hashes from last to first, placing the Entry Hash of + the "next" (logical order, not iteration order) DBI Entry in the first ExtID +of each "preceding" DBI Entry. + +4. Construct the Data Store Chain First Entry +- Set the NameIDs +- Construct the Content Metadata JSON Object with data size, any compression + details, and any application defined metadata. + +5. Publish the data store +- Commit the first entry, and then commit all DBI Entries and Data Block + entries. +- Wait for ACK for all Commits. +- Reveal all Entries. + +### Reading a data store + +1. Compute the Chain ID, if not already known, using the data hash and optional + application Namespace data. +2. Download and validate the First Entry. +- Validate ExtID structure. +- Validate JSON content structure. +- Confirm Data Store version. +- Confirm that the declared file size, and compressed file size if compressed, + are sane values. +- Validate any application Metadata. +3. Download the DBI. +- Download all DBI Entries by traversing the linked list. Ensure that only the + last DBI Entry has fewer than 318 Data Block Entry Hashes. +4. Download the Data Blocks and reconstruct the, possibly compressed, data. +- Data Blocks may be downloaded concurrently since they are all known from the + DBI. +- Order the data according to the DBI. +- Decompress the data, if compressed. +- Verify the sha256d data hash. + + +# Copyright + +Copyright and related rights waived via +[CC0](https://creativecommons.org/publicdomain/zero/1.0/).