slipstream.caching

Slipstream caching.

Attributes

MB
MINUTES

Classes

Cache

Create a RocksDB database in the specified folder.

Module Contents

slipstream.caching.MB = 1048576

slipstream.caching.MINUTES = 60

class slipstream.caching.Cache(path: str, options: rocksdict.Options | None = None, column_families: dict[str, rocksdict.Options] | None = None, access_type: rocksdict.AccessType | None = None, target_table_size: int = 25 * MB, number_of_locks: int = 16)

Bases: slipstream.interfaces.ICache

Create a RocksDB database in the specified folder.

>>> cache = Cache('db/mycache')  

The cache instance acts as a callable to store data:

>>> cache('key', {'msg': 'Hello World!'})  
>>> cache['key']  
{'msg': 'Hello World!'}

name

db

async transaction(key: slipstream.interfaces.Key) → collections.abc.AsyncGenerator[Cache, None]

Lock the db entry while using the context manager.

>>> async with cache.transaction('fish'):  
...     cache['fish'] = '🐟'

• This works for asynchronous code (not multi-threading/processing)

• While locked, other transactions on the same key will block

• Actions outside of transaction blocks ignore ongoing transactions

• Reads aren’t limited by ongoing transactions

set_dumps(dumps: collections.abc.Callable[[Any], bytes]) → None

Set custom dumps function.

set_loads(dumps: collections.abc.Callable[[bytes], Any]) → None

Set custom loads function.

set_read_options(read_opt: rocksdict.ReadOptions) → None

Set custom read options.

set_write_options(write_opt: rocksdict.WriteOptions) → None

Set custom write options.

put(key: slipstream.interfaces.Key, value: Any, write_opt: rocksdict.WriteOptions | None = None) → None

Put item in database using key.

get(key: slipstream.interfaces.Key | list[slipstream.interfaces.Key], default: T = None, read_opt: rocksdict.ReadOptions | None = None) → Any | T

Get item from database by key.

put_entity(key: slipstream.interfaces.Key, names: list[Any], values: list[Any], write_opt: rocksdict.WriteOptions | None = None) → None

Put wide-column in database using key.

>>> cache.put_entity('key', ['a', 'b'], [1, 2])  

get_entity(key: slipstream.interfaces.Key | list[slipstream.interfaces.Key], default: Any = None, read_opt: rocksdict.ReadOptions | None = None) → list[tuple[Any, Any]] | None

Get wide-column from database by key.

>>> cache.get_entity('key')  
[('a', 1), ('b', 2)]

delete(key: slipstream.interfaces.Key, write_opt: rocksdict.WriteOptions | None = None) → None

Delete item from database.

key_may_exist(key: slipstream.interfaces.Key, fetch: bool = False, read_opt: rocksdict.ReadOptions | None = None) → bool | tuple[bool, Any]

Check if a key exist without performing IO operations.

iter(read_opt: rocksdict.ReadOptions | None = None) → rocksdict.RdictIter

Get iterable.

items(backwards: bool = False, from_key: slipstream.interfaces.Key | None = None, read_opt: rocksdict.ReadOptions | None = None, prefix: slipstream.interfaces.Key | None = None) → collections.abc.Iterator[tuple[slipstream.interfaces.Key, Any]]

Get tuples of key-value pairs.

keys(backwards: bool = False, from_key: slipstream.interfaces.Key | None = None, read_opt: rocksdict.ReadOptions | None = None, prefix: slipstream.interfaces.Key | None = None) → collections.abc.Iterator[slipstream.interfaces.Key]

Get keys.

values(backwards: bool = False, from_key: slipstream.interfaces.Key | None = None, read_opt: rocksdict.ReadOptions | None = None, prefix: slipstream.interfaces.Key | None = None) → collections.abc.Iterator[Any]

Get values.

columns(backwards: bool = False, from_key: slipstream.interfaces.Key | None = None, read_opt: rocksdict.ReadOptions | None = None, prefix: slipstream.interfaces.Key | None = None) → collections.abc.Iterator[list[tuple[Any, Any]]]

Get values as widecolumns.

entities(backwards: bool = False, from_key: slipstream.interfaces.Key | None = None, read_opt: rocksdict.ReadOptions | None = None, prefix: slipstream.interfaces.Key | None = None) → collections.abc.Iterator[tuple[slipstream.interfaces.Key, list[tuple[Any, Any]]]]

Get keys and entities.

ingest_external_file(paths: list[str], opts: rocksdict.IngestExternalFileOptions | None = None) → None

Load list of SST files into current column family.

get_column_family(name: str) → rocksdict.Rdict

Get column family by name.

get_column_family_handle(name: str) → rocksdict.ColumnFamily

Get column family handle by name.

drop_column_family(name: str) → None

Drop column family by name.

create_column_family(name: str, options: rocksdict.Options | None = None) → rocksdict.Rdict

Craete column family.

delete_range(begin: slipstream.interfaces.Key, end: slipstream.interfaces.Key, write_opt: rocksdict.WriteOptions | None = None) → None

Delete database items, excluding end.

snapshot() → rocksdict.Snapshot

Create snapshot of current column family.

path() → str

Get current database path.

set_options(options: dict[str, str]) → None

Set options for current column family.

property_value(name: str) → str | None

Get property by name from current column family.

property_int_value(name: str) → int | None

Get property as int by name from current column family.

latest_sequence_number() → int

Get sequence number of the most recent transaction.

try_catch_up_with_primary() → None

Try to catch up with the primary by reading log files.

cancel_all_background(wait: bool = True) → None

Request stopping background work.

list_cf(path: str, options: rocksdict.Options | None = None) → list[str]

List column families.

live_files() → list[dict[str, Any]]

Get list of all table files with their level, start/end key.

compact_range(begin: slipstream.interfaces.Key | None, end: slipstream.interfaces.Key | None, compact_opt: rocksdict.CompactOptions | None = None) → None

Run manual compaction on range for the current column family.

close() → None

Flush memory to disk, and drop the current column family.

write(write_batch: rocksdict.WriteBatch, write_opt: rocksdict.WriteOptions | None = None) → None

Write a batch.

flush(wait: bool = True) → None

Manually flush the current column family.

flush_wal(sync: bool = True) → None

Manually flush the WAL buffer.

repair(path: str, options: rocksdict.Options | None = None) → None

Repair the database.

destroy(options: rocksdict.Options | None = None) → None

Delete the database.