dbfs

a key value stor on filesystem, can work with id's as well as with keys.

The algo's used have been optimized for scalability and human readability, the idea is that the data files need to be small, efficient and be well usable on a e.g. git based storage system.

• dbcollection, can linked to a context of hero (can be a circle or another area worth remembering things for)
• db, there can be more than 1 db per dbcollection
• the secret is specified per dbcollection
• each subdb inherits the secret from the dbcollection but needs to be configured as encrypted

> TODO: fix, we refactored

import freeflowuniverse.crystallib.core.dbfs

mut dbcollection := get(context: 'test', secret: '123456')!

mut db := dbcollection.get_encrypted('db_a')!

db.set('a', 'bbbb')!
assert 'bbbb' == db.get('a')!

dbname

DBName has functionality to efficiently store millions of names and generate a unique id for it, each name gets a unique id, and based on the id the name can be found back easily.

Some string based data can be attached to one name so it becomes a highly efficient key value stor, can be used for e.g. having DB of pubkeys, for a nameserver, ...

dbfs examples

Each session has such a DB attached to it, data is stored on filesystem,

e.g. ideal for config sessions (which are done on context level)

> TODO: fix, we refactored

import freeflowuniverse.crystallib.core.dbfs

mut dbcollection := get(context: 'test', secret: '123456')!

mut db := dbcollection.get_encrypted('db_a')!


>> TODO: need to be updated, is now based on key and id

//get the value
fn (mut db DB) get(name_ string) !string {

//set the key/value will go to filesystem, is organzed per context and each db has a name
fn (mut db DB) set(name_ string, data_ string) !

//check if entry exists based on keyname
fn (mut db DB) exists(name_ string) bool

//delete an entry
fn (mut db DB) delete(name_ string) !

//get all keys of the db (e.g. per session)
fn (mut db DB) keys(prefix string) ![]string

// delete all data
fn (mut db DB) destroy() !

fn db_encrypted(name_ string, secret string) !DB

will use default connection and get database with name as specified, if not specified then name=core is not encrypted

fn db_get(name_ string) !DB

will use default connection and get database with name as specified is not encrypted

fn get(args_ CollectionGetArgs) !DBCollection

will return the dbcollection for a specific context

fn namedb_new(path string) !NameDB

if key and ok to hash, then we can generated unique id out of the hashed key

fn sid2int(sid string) u32

get u32 from sid as string

struct CollectionGetArgs {
pub mut:
	dbpath    string
	secret    string
	contextid u32
}

struct DB {
mut:
	config DBConfig
pub mut:
	path   pathlib.Path
	parent &DBCollection @[skip; str: skip]
	namedb ?NameDB // optional namedb which is for hashed keys
}

fn (mut db DB) get(args_ GetArgs) !string

get the value, if it doesn't exist then return empty string

fn (mut db DB) set(args_ SetArgs) !u32

set the key/value will go to filesystem, is organzed per context and each db has a name

fn (mut db DB) exists(args_ GetArgs) !bool

check if entry exists based on keyname

fn (mut db DB) delete(args_ GetArgs) !

delete an entry

fn (mut db DB) destroy() !

delete the db, will not be able to use it any longer

fn (mut db DB) keys(prefix_ string) ![]string

get all keys of the db (e.g. per session) can be with a prefix

fn (mut db DB) ids() ![]u32

get all keys of the db (e.g. per session) can be with a prefix

fn (mut db DB) empty() !

delete all data

fn (mut db DB) encrypt() !

will mark db for encryption . will go over all existing keys and encrypt

fn (db DB) is_encrypted() bool

struct DBCollection {
pub mut:
	path           pathlib.Path
	contextid      u32
	secret         string
	memberid       u16 // needed to do autoincrement of the DB, when user logged in we need memberid, memberid is unique per circle
	member_pubkeys map[int]string
	// redis redisclient.Redis
}

fn (mut dbcollection DBCollection) incr() !u32

fn (mut dbcollection DBCollection) db_create(args_ DBCreateArgs) !DB

create the dabase (init), cannot use unless this is done

 name string
 encrypted bool
 withkeys bool //if set means we will use keys in stead of only u32
 keyshashed bool //if its ok to hash the keys, which will generate id out of these keys and its more scalable

fn (mut dbcollection DBCollection) db_get_create(args_ DBCreateArgs) !DB

fn (mut dbcollection DBCollection) db_get(name_ string) !DB

get a DB from the dbcollection

fn (mut dbcollection DBCollection) get_encrypted(name_ string) !DB

fn (mut collection DBCollection) exists(name_ string) bool

fn (mut collection DBCollection) delete(name_ string) !

fn (mut collection DBCollection) list() ![]string

fn (mut collection DBCollection) prefix(prefix string) ![]string

fn (mut collection DBCollection) destroy() !

delete all data in the dbcollection (be careful)

struct DBConfig {
mut:
	encrypted bool
pub:
	name       string
	withkeys   bool   // if set means we will use keys in stead of only u32
	keyshashed bool   // if its ok to hash the keys, which will generate id out of these keys and its more scalable
	ext        string // extension if we want to use it in DB e.g. 'json'
	// base64 bool //if binary data will be base encoded, not used now
}

struct DBCreateArgs {
pub mut:
	name       string
	encrypted  bool
	withkeys   bool // if set means we will use keys in stead of only u32
	keyshashed bool // if its ok to hash the keys, which will generate id out of these keys and its more scalable
}

struct GetArgs {
pub mut:
	key string
	id  u32
}

struct NameDB {
pub mut:
	path   pathlib.Path
	config NameDBConfig
}

fn (mut db NameDB) save() !

fn (mut db NameDB) set(key string, data string) !u32

will store in a place where it can easily be found back and it returns a unique u32

fn (mut db NameDB) delete(key string) !

fn (mut db NameDB) get(key string) !(u32, string)

will store in a place where it can easily be found back and it returns a unique u32

fn (mut db NameDB) exists(key string) !bool

fn (mut db NameDB) get_from_id(myid u32) !(string, string)

struct NameDBConfig {
pub mut:
	levels int = 1
}

struct SetArgs {
pub mut:
	key    string
	id     u32
	value  string
	valueb []u8 // as bytes
}