Operating System Abstraction Layer (OSAL) - Core Module

The lib/osal/core module provides a comprehensive, platform-independent abstraction layer for common operating system functionalities in V. It simplifies interactions with the underlying system, offering robust tools for process management, network operations, file system manipulation, environment variable handling, and more.

Capabilities & Usage

This module encapsulates essential OS-level features, making system programming more consistent and reliable across different environments.

1. Process Execution (exec.v)

Execute shell commands with fine-grained control and robust error handling.

• osal.exec(cmd: Command): Executes a command with extensive options.
• Command struct fields:
• cmd (string): The command string.
• timeout (int): Max execution time in seconds (default: 3600).
• retry (int): Number of retries on failure.
• work_folder (string): Working directory for the command.
• environment (map[string]string): Environment variables for the command.
• stdout (bool): Show command output (default: true).
• raise_error (bool): Raise V error on failure (default: true).
• ignore_error (bool): Do not raise error, just report.
• debug (bool): Enable debug output.
• shell (bool): Execute in interactive shell.
• async (bool): Run command asynchronously.
• runtime (RunTime): Specify runtime (e.g., .bash, .python).
• Returns Job struct with status, output, error, exit_code, etc.
• osal.execute_silent(cmd string): Executes a command silently, returns output.
• osal.execute_debug(cmd string): Executes a command with debug output, returns output.
• osal.execute_stdout(cmd string): Executes a command and prints output to stdout, returns output.
• osal.execute_interactive(cmd string): Executes a command in an interactive shell.
• osal.cmd_exists(cmd string): Checks if a command exists in the system's PATH.

Example: Flexible Command Execution

import freeflowuniverse.herolib.osal.core as osal

// Simple command execution
job := osal.exec(cmd: 'ls -la')!
println(job.output)

// Execute with error handling and custom options
job := osal.exec(osal.Command{
    cmd: 'complex_command_that_might_fail'
    timeout: 60 // seconds
    retry: 2
    work_folder: '/tmp'
    environment: {
        'MY_VAR': 'some_value'
    }
    stdout: true
    raise_error: true
})!

// Check job status and handle specific errors
if job.status == .done {
    println('Command executed successfully!')
} else if job.status == .error_timeout {
    println('Command timed out.')
} else {
    println('Command failed with exit code: ${job.exit_code}, Error: ${job.error}')
}

// Example of error handling with match
job_result := osal.exec(cmd: 'non_existent_command')
if job_result.is_err() {
    err := job_result.err
    match err.error_type {
        .exec { println('Execution error: ${err.msg()}') }
        .timeout { println('Command timed out: ${err.msg()}') }
        .args { println('Invalid arguments: ${err.msg()}') }
        else { println('An unexpected error occurred: ${err.msg()}') }
    }
}

2. Network Utilities (net.v)

Tools for network diagnostics and information.

• osal.ping(args: PingArgs): Checks host reachability.
• PingArgs struct fields: address (string, required), count (u8), timeout (u16), retry (u8).
• Returns PingResult enum: .ok, .timeout, .unknownhost.
• osal.tcp_port_test(args: TcpPortTestArgs): Tests if a TCP port is open.
• TcpPortTestArgs struct fields: address (string, required), port (int), timeout (u16 in milliseconds).
• Returns bool.
• osal.ipaddr_pub_get(): Retrieves the public IP address. Returns string.
• osal.is_ip_on_local_interface(ip string): Checks if an IP is on a local interface. Returns bool.

3. File System Operations (file.v)

Functions for managing files and directories.

• osal.file_write(path string, text string): Writes text to a file.
• osal.file_read(path string): Reads content from a file. Returns string.
• osal.dir_ensure(path string): Ensures a directory exists, creates if not.
• osal.dir_delete(path string): Deletes a directory if it exists.
• osal.dir_reset(path string): Deletes and recreates a directory.
• osal.rm(todelete string): Removes files/directories (supports ~, comma/newline separated paths, sudo).

4. Environment Variables (env.v)

Manage system environment variables.

• osal.env_set(args: EnvSet): Sets an environment variable.
• EnvSet struct fields: key (string, required), value (string, required), overwrite (bool).
• osal.env_unset(key string): Unsets an environment variable.
• osal.env_unset_all(): Unsets all environment variables.
• osal.env_set_all(args: EnvSetAll): Sets multiple environment variables.
• EnvSetAll struct fields: env (map[string]string), clear_before_set (bool), overwrite_if_exists (bool).
• osal.env_get(key string): Retrieves an environment variable's value. Returns string.
• osal.env_exists(key string): Checks if an environment variable exists. Returns bool.
• osal.env_get_default(key string, def string): Gets an environment variable or a default value. Returns string.
• osal.load_env_file(file_path string): Loads environment variables from a file.

5. Command & Profile Management (cmds.v)

Manage system commands and shell profile paths.

• osal.cmd_add(args: CmdAddArgs): Adds a binary to system paths and updates profiles.
• CmdAddArgs struct fields: cmdname (string), source (string, required), symlink (bool), reset (bool).
• osal.profile_path_add_hero(): Adds ~/hero/bin to profile. Returns string (path).
• osal.bin_path(): Returns the preferred binary installation path. Returns string.
• osal.hero_path(): Returns the ~/hero directory path. Returns string.
• osal.usr_local_path(): Returns /usr/local (Linux) or ~/hero (macOS). Returns string.
• osal.profile_path_source(): Returns a source command for the preferred profile. Returns string.
• osal.profile_path_source_and(): Returns a source command followed by &&. Returns string.
• osal.profile_path_add_remove(args: ProfilePathAddRemoveArgs): Adds/removes paths from profiles.
• ProfilePathAddRemoveArgs struct fields: paths_profile (string), paths2add (string), paths2delete (string), allprofiles (bool).
• osal.cmd_path(cmd string): Returns the full path of an executable command. Returns string.
• osal.cmd_delete(cmd string): Deletes commands from their locations.
• osal.profile_paths_all(): Lists all possible profile file paths. Returns []string.
• osal.profile_paths_preferred(): Lists preferred profile file paths for the OS. Returns []string.
• osal.profile_path(): Returns the most preferred profile file path. Returns string.

6. System Information & Utilities (ps_tool.v, sleep.v, downloader.v, users.v, etc.)

Miscellaneous system functionalities.

• osal.processmap_get(): Gets a map of all running processes. Returns ProcessMap.
• osal.processinfo_get(pid int): Gets info for a specific process. Returns ProcessInfo.
• osal.processinfo_get_byname(name string): Gets info for processes by name. Returns []ProcessInfo.
• osal.process_exists(pid int): Checks if a process exists by PID. Returns bool.
• osal.processinfo_with_children(pid int): Gets a process and its children. Returns ProcessMap.
• osal.processinfo_children(pid int): Gets children of a process. Returns ProcessMap.
• osal.process_kill_recursive(args: ProcessKillArgs): Kills a process and its children.
• ProcessKillArgs struct fields: name (string), pid (int).
• osal.whoami(): Returns the current username. Returns string.
• osal.sleep(duration int): Pauses execution for duration seconds.
• osal.download(args: DownloadArgs): Downloads a file from a URL.
• DownloadArgs struct fields: url (string), name (string), reset (bool), hash (string), dest (string), timeout (int), retry (int), minsize_kb (u32), maxsize_kb (u32), expand_dir (string), expand_file (string).
• Returns pathlib.Path.
• osal.user_exists(username string): Checks if a user exists. Returns bool.
• osal.user_id_get(username string): Gets user ID. Returns int.
• osal.user_add(args: UserArgs): Adds a user.
• UserArgs struct fields: name (string, required). Returns int (user ID).

Notes on the CMD Job Execution

• Commands are executed from temporary scripts in /tmp/execscripts.
• Failed script executions are preserved for debugging.
• Successful script executions are automatically cleaned up.
• Platform-specific behavior is automatically handled.
• Timeout and retry mechanisms are available for robust execution.
• Environment variables and working directories can be specified per command.
• Interactive and non-interactive modes are supported.

fn bin_path() !string

fn cmd_add(args_ CmdAddArgs) !

copy a binary to the right location on the local computer . e.g. is /usr/local/bin on linux . e.g. is ~/hero/bin on osx . will also add the bin location to the path of .zprofile and .zshrc (different per platform)

fn cmd_delete(cmd string) !

delete cmds from found locations can be one command of multiple

fn cmd_exists(cmd string) bool

fn cmd_exists_profile(cmd string) bool

fn cmd_path(cmd string) !string

is same as executing which in OS returns path or error

fn cmd_to_script_path(cmd Command) !string

will return temporary path which then can be executed, is a helper function for making script out of command

fn dir_delete(path string) !

remove all if it exists

fn dir_ensure(path string) !

remove all if it exists

fn dir_reset(path string) !

remove all if it exists and then (re-)create

fn done_delete(key string) !

fn done_exists(key string) bool

fn done_get(key string) ?string

fn done_get_int(key string) int

fn done_get_str(key string) string

fn done_print() !

fn done_reset() !

fn done_set(key string, val string) !

fn download(args_ DownloadArgs) !pathlib.Path

if name is not specified, then will be the filename part if the last ends in an extension like .md .txt .log .text ... the file will be downloaded

fn env_exists(key string) !bool

fn env_get(key string) !string

Returns the requested environment variable if it exists or throws an error if it does not

fn env_get_all() map[string]string

Returns all existing environment variables

fn env_get_default(key string, def string) string

Returns the requested environment variable if it exists or returns the provided default value if it does not

fn env_set(args EnvSet)

Sets an environment if it was not set before, it overwrites the enviroment variable if it exists and if overwrite was set to true (default)

fn env_set_all(args EnvSetAll)

Allows to set multiple enviroment variables in one go, if clear_before_set is true all existing environment variables will be unset before the operation, if overwrite_if_exists is set to true it will overwrite all existing enviromnent variables

fn env_unset(key string)

Unsets an environment variable

fn env_unset_all()

Unsets all environment variables

fn exec(cmd_ Command) !Job

cmd is the cmd to execute can use ' ' and spaces . if \n in cmd it will write it to ext and then execute with bash . if die==false then will just return returncode,out but not return error . if stdout will show stderr and stdout . . if cmd starts with find or ls, will give to bash -c so it can execute . if cmd has no path, path will be found . . Command argument: .

 name                             string // to give a name to your command, good to see logs...
 cmd                              string
 description                      string
 timeout                          int  = 3600 // timeout in sec
 stdout                           bool = true
 stdout_log                       bool = true
 raise_error                      bool = true // if false, will not raise an error but still error report
 ignore_error                     bool // means if error will just exit and not raise, there will be no error reporting
 work_folder                      string // location where cmd will be executed
 environment                      map[string]string // env variables
 ignore_error_codes               []int
 scriptpath                       string // is the path where the script will be put which is executed
 scriptkeep                       bool   // means we don't remove the script
 debug                            bool   // if debug will put +ex in the script which is being executed and will make sure script stays
 shell                            bool   // means we will execute it in a shell interactive
 retry                            int
 interactive 					 	bool = true // make sure we run on non interactive way
 async							 bool
 runtime							 RunTime (.bash, .python)

 returns Job:
 start  time.Time
 end    time.Time
 cmd    Command
 output []string
 error    []string
 exit_code int
 status JobStatus
 process os.Process

return Job .

fn exec_string(cmd Command) !string

cmd is the cmd to execute can use ' ' and spaces if \n in cmd it will write it to ext and then execute with bash if die==false then will just return returncode,out but not return error if stdout will show stderr and stdout

if cmd starts with find or ls, will give to bash -c so it can execute if cmd has no path, path will be found $... are remplaced by environment arguments TODO:implement

Command argument: cmd string timeout int = 600 stdout bool = true die bool = true debug bool

return what needs to be executed can give it to bash -c ...

fn execute_debug(cmd string) !string

fn execute_interactive(cmd string) !

shortcut to execute a job interactive means in shell

fn execute_ok(cmd string) bool

executes a cmd, if not error return true

fn execute_silent(cmd string) !string

shortcut to execute a job silent

fn execute_stdout(cmd string) !string

shortcut to execute a job to stdout

fn file_read(path string) !string

fn file_write(path string, text string) !

fn get_ssh_key(key_name string, config SSHConfig) ?SSHKey

Returns a specific SSH key with the given name from the default SSH directory (~/.ssh)

fn hero_path() !string

fn ipaddr_pub_get() !string

Returns the public IP address as known on the public side Uses resolver4.opendns.com to fetch the IP address

fn ipaddr_pub_get_check() !string

also check the address is on local interface

fn is_ip_on_local_interface(public_ip string) !bool

Check if the public IP matches any of the local network interfaces

fn load_env_file(file_path string) !

fn new_ssh_key(key_name string, config SSHConfig) !SSHKey

Creates a new SSH key pair to the specified directory

fn package_install(name_ string) !

install a package using the right commands per platform

fn package_refresh() !

update the package list

fn package_remove(name_ string) !

remove a package using the right commands per platform

fn ping(args PingArgs) !PingResult

if reached in timout result will be True address is e.g. 8.8.8.8 ping means we check if the destination responds

fn process_exists(pid int) bool

fn process_exists_byname(name string) !bool

fn process_kill_recursive(args ProcessKillArgs) !

kill process and all the ones underneith

fn processinfo_children(pid int) !ProcessMap

get all children of 1 process

fn processinfo_get(pid int) !ProcessInfo

get process info from 1 specific process returns

 pub struct ProcessInfo {
 pub mut:
    cpu_perc	f32
    mem_perc	f32
    cmd 		string
    pid 		int
    ppid 		int
    //resident memory
    rss			int
 }

fn processinfo_get_byname(name string) ![]ProcessInfo

fn processinfo_with_children(pid int) !ProcessMap

return the process and its children

fn processmap_get() !ProcessMap

make sure to use new first, so that the connection has been initted then you can get it everywhere

fn profile_path() !string

fn profile_path_add_hero() !string

fn profile_path_add_remove(args_ ProfilePathAddRemoveArgs) !

add and/or remove paths from profiles if paths_profile not specified it will walk over all of them

fn profile_path_source() !string

return the source statement if the profile exists

fn profile_path_source_and() !string

return source $path && . or empty if it doesn't exist

fn profile_paths_all() ![]string

return possible profile paths in OS

fn profile_paths_preferred() ![]string

fn rm(todelete_ string) !

can be list of dirs, files ~ supported can be \n or , separated

fn sleep(duration int)

sleep in seconds

fn tcp_port_test(args TcpPortTestArgs) bool

test if a tcp port answers

 address string //192.168.8.8
 port int = 22
 timeout u16 = 2000 // total time in milliseconds to keep on trying

fn user_add(args UserArgs) !int

add's a user if the user does not exist yet

fn user_exists(username string) bool

fn user_id_get(username string) !int

fn usr_local_path() !string

/usr/local on linux, ${os.home_dir()}/hero on osx

fn whoami() !string

enum ErrorType {
	exec
	timeout
	args
}

enum JobStatus {
	init
	running
	error_exec
	error_timeout
	error_args
	done
}

enum PMState {
	init
	ok
	old
}

enum PingResult {
	ok
	timeout     // timeout from ping
	unknownhost // means we don't know the hostname its a dns issue
}

enum RunTime {
	bash
	python
	heroscript
	herocmd
	v
}

struct CmdAddArgs {
pub mut:
	cmdname string
	source  string @[required] // path where the binary is
	symlink bool // if rather than copy do a symlink
	reset   bool = true // if existing cmd will delete
	// bin_repo_url string = 'https://github.com/freeflowuniverse/freeflow_binary' // binary where we put the results
}

struct Command {
pub mut:
	name               string // to give a name to your command, good to see logs...
	cmd                string
	description        string
	timeout            int  = 3600 // timeout in sec
	stdout             bool = true
	stdout_log         bool = true
	raise_error        bool = true // if false, will not raise an error but still error report
	ignore_error       bool              // means if error will just exit and not raise, there will be no error reporting
	work_folder        string            // location where cmd will be executed
	environment        map[string]string // env variables
	ignore_error_codes []int
	scriptpath         string // is the path where the script will be put which is executed
	scriptkeep         bool   // means we don't remove the script
	debug              bool   // if debug will put +ex in the script which is being executed and will make sure script stays
	shell              bool   // means we will execute it in a shell interactive
	retry              int
	interactive        bool = true
	async              bool
	runtime            RunTime
}

struct DownloadArgs {
pub mut:
	name        string // optional (otherwise derived out of filename)
	url         string
	reset       bool   // will remove
	hash        string // if hash is known, will verify what hash is
	dest        string // if specified will copy to that destination	
	timeout     int = 180
	retry       int = 3
	minsize_kb  u32 = 10 // is always in kb
	maxsize_kb  u32
	expand_dir  string
	expand_file string
}

struct EnvSet {
pub mut:
	key       string @[required]
	value     string @[required]
	overwrite bool = true
}

struct EnvSetAll {
pub mut:
	env                 map[string]string
	clear_before_set    bool
	overwrite_if_exists bool = true
}

struct Job {
pub mut:
	start     time.Time
	end       time.Time
	cmd       Command
	output    string
	error     string
	exit_code int
	status    JobStatus
	process   ?&os.Process @[skip; str: skip]
	runnr     int // nr of time it runs, is for retry
}

fn (mut job Job) execute_retry() !

execute the job and wait on result will retry as specified

fn (mut job Job) execute() !

execute the job, start process, process will not be closed . important you need to close the process later by job.close()! otherwise we get zombie processes

fn (mut job Job) wait() !

wait till the job finishes or goes in error

fn (mut job Job) process() !

process (read std.err and std.out of process)

fn (mut job Job) close() !

will wait & close

struct JobError {
	Error
pub mut:
	job        Job
	error_type ErrorType
}

struct PingArgs {
pub mut:
	address string @[required]
	count   u8  = 1 // the ping is successful if it got count amount of replies from the other side
	timeout u16 = 1 // the time in which the other side should respond in seconds
	retry   u8
}

struct ProcessInfo {
pub mut:
	cpu_perc f32
	mem_perc f32
	cmd      string
	pid      int
	ppid     int // parentpid
	// resident memory
	rss int
}

fn (mut p ProcessInfo) str() string

struct ProcessKillArgs {
pub mut:
	name string
	pid  int
}

struct ProcessMap {
pub mut:
	processes []ProcessInfo
	lastscan  time.Time
	state     PMState
	pids      []int
}

struct ProfilePathAddRemoveArgs {
pub mut:
	paths_profile string
	paths2add     string
	paths2delete  string
	allprofiles   bool
}

struct SSHConfig {
pub:
	directory string = os.join_path(os.home_dir(), '.ssh')
}

struct SSHKey {
pub:
	name      string
	directory string
}

fn (key SSHKey) public_key_path() !pathlib.Path

returns the public ssh key's path of the keypair

fn (key SSHKey) private_key_path() !pathlib.Path

returns the private ssh key's path of the keypair

fn (key SSHKey) public_key() !string

returns the public ssh key of the keypair

fn (key SSHKey) private_key() !string

returns the private ssh key of the keypair

struct TcpPortTestArgs {
pub mut:
	address string @[required] // 192.168.8.8
	port    int = 22
	timeout u16 = 2000 // total time in milliseconds to keep on trying
}

struct UserArgs {
pub mut:
	name string @[required]
}