HeroLib Cache System

A high-performance, generic in-memory caching system for V with support for TTL, size limits, and LRU eviction.

Features

• Generic type support (can cache any type)
• Configurable maximum entries and memory size limits
• Time-To-Live (TTL) support
• Least Recently Used (LRU) eviction policy
• Memory-aware caching with size-based eviction
• Thread-safe operations
• Optional persistence support (configurable)

Configuration

The cache system is highly configurable through the CacheConfig struct:

pub struct CacheConfig {
pub mut:
    max_entries    u32 = 1000    // Maximum number of entries
    max_size_mb    f64 = 100.0   // Maximum cache size in MB
    ttl_seconds    i64 = 3600    // Time-to-live in seconds (0 = no TTL)
    eviction_ratio f64 = 0.05    // Percentage of entries to evict when full (5%)
    persist        bool          // Whether to persist cache to disk
}

Basic Usage

Here's a simple example of using the cache:

import freeflowuniverse.herolib.data.cache

// Define your struct type
@[heap]
struct User {
    id   u32
    name string
    age  int
}

fn main() {
    // Create a cache with default configuration
    mut user_cache := cache.new_cache[User]()

    // Create a user
    user := &User{
        id: 1
        name: 'Alice'
        age: 30
    }

    // Add to cache
    user_cache.set(user.id, user)

    // Retrieve from cache
    if cached_user := user_cache.get(1) {
        println('Found user: ${cached_user.name}')
    }
}

Advanced Usage

Custom Configuration

mut user_cache := cache.new_cache[User](
    max_entries: 1000      // Maximum number of entries
    max_size_mb: 10.0      // Maximum cache size in MB
    ttl_seconds: 300       // Items expire after 5 minutes
    eviction_ratio: 0.2    // Evict 20% of entries when full 
)

Memory Management

The cache automatically manages memory using two mechanisms:

1. Entry Count Limit: When max_entries is reached, least recently used items are evicted.
2. Memory Size Limit: When max_size_mb is reached, items are evicted based on the eviction_ratio.

// Create a cache with strict memory limits
config := cache.CacheConfig{
    max_entries: 100       // Only keep 100 entries maximum
    max_size_mb: 1.0      // Limit cache to 1MB
    eviction_ratio: 0.1   // Remove 10% of entries when full
}

Cache Operations

mut cache := cache.new_cache[User](cache.CacheConfig{})

// Add/update items
cache.set(1, user1)
cache.set(2, user2)

// Get items
if user := cache.get(1) {
    // Use cached user
}

// Check cache size
println('Cache entries: ${cache.len()}')

// Clear the cache
cache.clear()

Best Practices

1. Choose Appropriate TTL: Set TTL based on how frequently your data changes and how critical freshness is.

2. Memory Management:

• Set reasonable max_entries and max_size_mb limits based on your application's memory constraints
• Monitor cache size using len()
• Use appropriate eviction_ratio (typically 0.05-0.2) to balance performance and memory usage

3. Type Safety:

• Always use @[heap] attribute for structs stored in cache
• Ensure cached types are properly memory managed

4. Error Handling:

• Always use option types when retrieving items (if value := cache.get(key) {)
• Handle cache misses gracefully

5. Performance:

• Consider the trade-off between cache size and hit rate
• Monitor and adjust TTL and eviction settings based on usage patterns

Thread Safety

The cache implementation is thread-safe for concurrent access. However, when using the cache in a multi-threaded environment, ensure proper synchronization when accessing cached objects.

fn new_cache[T](config CacheConfig) &Cache[T]

new_cache creates a new cache instance with the given configuration

fn (mut c Cache[T]) get(id u32) ?&T

get retrieves an object from the cache if it exists

fn (mut c Cache[T]) set(id u32, obj &T)

set adds or updates an object in the cache

fn (mut c Cache[T]) remove(id u32)

remove deletes a single entry from the cache

fn (mut c Cache[T]) clear()

clear empties the cache

fn (c &Cache[T]) len() int

len returns the number of entries in the cache

struct Cache[T] {
mut:
	entries    map[u32]&CacheEntry[T] // Map of object ID to cache entry
	config     CacheConfig            // Cache configuration
	access_log []u32                  // Ordered list of object IDs by access time
	total_size u64                    // Total size of cached entries in bytes
}

Cache manages the in-memory caching of objects

struct CacheConfig {
pub mut:
	max_entries    u32 = 1000  // Maximum number of entries
	max_size_mb    f64 = 100.0 // Maximum cache size in MB
	ttl_seconds    i64 = 3600  // Time-to-live in seconds (0 = no TTL)
	eviction_ratio f64 = 0.05  // Percentage of entries to evict when full (5%)
}

CacheConfig holds cache configuration parameters