OSIS

Object Storage and Indexing System A system for storing root objects efficiently and indexed by certain fields.

OSIS comprises of 2 elements:

• Indexer: responsible for indexing and identifying objects
• Storer: responsible of storing in different databases, with varying encodings, and encryption.

Indexer

The indexers primary duty is to be able to create and query sql tables for a given base object specification and it's indices. For instance: I specify a Base Object called Pet, and I specify Pet so that (more on writing specifications here) it's breed tag is indexable.

struct Pet {
    breed string @[index]
}

Given this specification, the indexer is expected to create an sql table with the breed field as a column. This allows the backend to filter and search base objects by their fields. Note that, the object isn't stored on the table, but just it's id. Object storage and modification is handled by the

Getting started

Generic Code

The solution provided by this module is to create a backend interface with generic CRUD + list + filter methods for root objects that different backends can implement.

This allows for a single generated actor code to use different backends, without having to generate separate code for each. Having less generated code is less prone to errors, and using the same backend methods for each actor makes it easier modify, fix and add features to the backends. Using the same data manipulation methods in generated code also makes it easier to generate code for the actor as the implementations don't differ for different root objects.

Creating a backend

fn new(config OSISConfig) !OSIS

fn new_indexer(config IndexerConfig) !Indexer

fn new_storer(config StorerConfig) !Storer

fn reset(path string) !

deletes an indexer table belonging to a base object

fn root_object[T](object T) RootObject

return the description of a given generic

fn root_object_from_json(json string) !RootObject

enum FieldType {
	number
	text
}

struct FieldDescription {
pub mut:
	name        string // name of field
	typ         FieldType
	value       string // value of field
	is_secret   bool   // whether field should be encrypted upon storage
	is_index    bool   // whether object is searchable by field
	fts_enabled bool   // whether full text search on field is enabled
}

fn (field FieldDescription) sql_type() string

returns the sql type name of the field

struct FilterParams {
	// indices     map[string]string // map of index values that are being filtered by, in order of priority.
	limit       int  // limit to the number of values to be returned, in order of priority
	fuzzy       bool // if fuzzy matching is enabled in matching indices
	matches_all bool // if results should match all indices or any
}

from and to for int f64 time etc.

struct Indexer {
	db sqlite.DB
}

fn (mut backend Indexer) delete(id string, obj RootObject) !

save the session to redis & mem

fn (mut backend Indexer) filter(filter RootObject, params FilterParams) ![]string

filter lists root objects of type T that match provided index parameters and params.

fn (mut indexer Indexer) generic_delete[T](id u32) !

fn (mut indexer Indexer) generic_filter[T, D](filter D, params FilterParams) ![]string

filter lists root objects of type T that match provided index parameters and params.

fn (mut indexer Indexer) generic_get[T](id u32) !T

fn (mut indexer Indexer) generic_list[T]() ![]u32

fn (mut indexer Indexer) generic_new[T](obj T) !u32

new creates a new root object entry in the root_objects table, and the table belonging to the type of root object with columns for index fields

fn (mut indexer Indexer) generic_set[T](obj T) !

fn (mut backend Indexer) get(id string, obj RootObject) !RootObject

fn (i Indexer) get_id(id u32) !string

fn (mut backend Indexer) get_json(id string, obj RootObject) !string

fn (mut i Indexer) init() !

fn (mut backend Indexer) list(obj RootObject) ![]u32

fn (mut backend Indexer) new(object RootObject) !u32

new creates a new root object entry in the root_objects table, and the table belonging to the type of root object with columns for index fields

fn (mut i Indexer) new_id(object string) !u32

fn (mut backend Indexer) set(obj RootObject) !

save the session to redis & mem

struct IndexerConfig {
	db_path string
	reset   bool
}

struct OSIS {
pub mut:
	indexer Indexer // storing indeces
	storer  Storer
}

fn (mut o OSIS) generic_delete[T](id u32) !

fn (mut o OSIS) generic_filter[T, D](filter D, params FilterParams) ![]T

fn (mut o OSIS) generic_get[T](id u32) !T

fn (mut o OSIS) generic_list[T]() ![]T

fn (mut o OSIS) generic_new[T](obj T) !u32

fn (mut o OSIS) generic_reset[T]() !

fn (mut o OSIS) generic_set[T](obj T) !

fn (mut backend OSIS) reset_all() !

struct OSISConfig {
pub:
	directory string
	name      string
	secret    string
	reset     bool
}

struct RootObject {
pub mut:
	id     string
	name   string // Story
	fields []FieldDescription
}

describes a root object

fn (obj RootObject) to_json() string

fn (obj RootObject) sql_indices_values() ([]string, []string)

returns the lists of the indices of a root objects db table, and corresponding values

fn (object RootObject) to_generic[T]() T

decodes root object into generic struct T

struct StorageParams {
	// database_type DatabaseType
	encrypted bool
}

struct Storer {
pub:
	directory     string
	db_filesystem dbfs.DBCollection
	db_sqlite     sqlite.DB
}

fn (mut storer Storer) delete(id u32, params StorageParams) !

fn (mut storer Storer) generic_delete[T](id u32) !

fn (mut storer Storer) generic_get[T](id u32) !T

fn (mut storer Storer) generic_list[T](ids []u32) ![]T

fn (mut storer Storer) generic_new[T](obj T) !u32

new creates a new root object entry in the root_objects table, and the table belonging to the type of root object with columns for index fields

fn (mut storer Storer) generic_set[T](obj T) !

fn (mut storer Storer) get(id u32, params StorageParams) !RootObject

fn (mut storer Storer) list(ids []u32, params StorageParams) ![]RootObject

fn (mut storer Storer) new(object RootObject, params StorageParams) !u32

fn (mut storer Storer) set(object RootObject, params StorageParams) !

struct StorerConfig {
	context_id u32
	secret     string
	directory  string ='${os.home_dir()}/hero/baobab/storer' // Directory of the storer
}