pythonwrench.disk_cache module¶
-
pythonwrench.disk_cache.disk_cache_call(fn: Callable[[...], T], *args, cache_dpath: str | Path | None =
None, cache_force: bool =False, cache_verbose: int =0, cache_checksum_fn: Callable[[tuple[Callable[[P], T], tuple, dict[str, Any]]], int] =checksum_any, cache_saving_backend: 'custom', cache_fname_fmt: str | Callable[[...], str] ='{fn_name}_{checksum_hex}{suffix}', cache_fname_fmt_args: Iterable[str] | None =None, cache_dump_fn: Callable[[Any, Path], Any], cache_load_fn: Callable[[Path], Any], cache_enable: bool =True, cache_store_mode: 'outputs_only' | 'outputs_metadata' | 'outputs_metadata_inputs', **kwargs) T[source]¶ -
pythonwrench.disk_cache.disk_cache_call(fn: Callable[[...], T], *args, cache_dpath: str | Path | None =
None, cache_force: bool =False, cache_verbose: int =0, cache_checksum_fn: Callable[[tuple[Callable[[P], T], tuple, dict[str, Any]]], int] =checksum_any, cache_saving_backend: 'csv' | 'json' | 'pickle', cache_fname_fmt: str | Callable[[...], str] ='{fn_name}_{checksum_hex}{suffix}', cache_fname_fmt_args: Iterable[str] | None =None, cache_dump_fn: None =None, cache_load_fn: None =None, cache_enable: bool =True, cache_store_mode: 'outputs_only' | 'outputs_metadata' | 'outputs_metadata_inputs' =_DEFAULT_CACHE_STORE_MODE, **kwargs) T -
pythonwrench.disk_cache.disk_cache_call(fn: Callable[[...], T], *args, cache_dpath: str | Path | None =
None, cache_force: bool =False, cache_verbose: int =0, cache_checksum_fn: Callable[[tuple[Callable[[P], T], tuple, dict[str, Any]]], int] =checksum_any, cache_saving_backend: 'csv' | 'json' | 'pickle' | 'custom' | 'auto' ='auto', cache_fname_fmt: str | Callable[[...], str] ='{fn_name}_{checksum_hex}{suffix}', cache_fname_fmt_args: Iterable[str] | None =None, cache_dump_fn: Callable[[Any, Path], Any] | None =None, cache_load_fn: Callable[[Path], Any] | None =None, cache_enable: bool =True, cache_store_mode: 'outputs_only' | 'outputs_metadata' | 'outputs_metadata_inputs' =_DEFAULT_CACHE_STORE_MODE, **kwargs) T Call function and store output in a cache file.
Cache file is identified by the checksum of the function arguments, and stored by default in ‘~/.cache/disk_cache/<Function_name>/’ directory.
Example¶
>>> import pythonwrench as pw >>> def heavy_processing(): >>> # Lot of stuff here >>> ... >>> outputs = pw.disk_cache_call(heavy_processing) # first time function is called >>> outputs = pw.disk_cache_call(heavy_processing) # second time outputs is loaded from disk- Args:
fn: Function to store its output. By default, it must be a callable that returns a pickable object. cache_dpath: Cache directory path. defaults to ‘~/.cache/disk_cache’. cache_force: Force function call and overwrite cache. defaults to False. cache_verbose: Set verbose logging level. Higher means more verbose. defaults to 0. cache_checksum_fn: Checksum function to identify input arguments. defaults to
pythonwrench.checksum_any. cache_saving_backend: Optional saving backend. Can be one of (‘csv’, ‘json’, ‘pickle’, ‘custom’, ‘auto’). defaults to ‘auto’. cache_fname_fmt: Cache filename format. defaults to ‘{fn_name}_{checksum_hex}{suffix}’. cache_dump_fn: Dump/save function to store outputs and overwrite saving backend. defaults to None. cache_load_fn: Load function to store outputs and overwrite saving backend. defaults to None. cache_enable: Enable disk cache. If False, the function has no effect. defaults to True. cache_store_mode: Disk cache storage mode. By default, it store function output and saved date into the cache file. defaults to ‘outputs_metadata’. *args: Positional arguments passed to the function. **kwargs: Keywords arguments passed to the function.
-
pythonwrench.disk_cache.disk_cache_decorator(fn: None =
None, *, cache_dpath: str | Path | None =None, cache_force: bool =False, cache_verbose: int =0, cache_checksum_fn: Callable[[tuple[Callable[[P], T], tuple, dict[str, Any]]], int] =checksum_any, cache_saving_backend: 'custom', cache_fname_fmt: str | Callable[[...], str] ='{fn_name}_{checksum_hex}{suffix}', cache_fname_fmt_args: Iterable[str] | None =None, cache_dump_fn: Callable[[Any, Path], Any], cache_load_fn: Callable[[Path], Any], cache_enable: bool =True, cache_store_mode: 'outputs_only' | 'outputs_metadata' | 'outputs_metadata_inputs') Callable[[Callable[[P], T]], Callable[[P], T]][source]¶ -
pythonwrench.disk_cache.disk_cache_decorator(fn: None =
None, *, cache_dpath: str | Path | None =None, cache_force: bool =False, cache_verbose: int =0, cache_checksum_fn: Callable[[tuple[Callable[[P], T], tuple, dict[str, Any]]], int] =checksum_any, cache_saving_backend: 'csv' | 'json' | 'pickle', cache_fname_fmt: str | Callable[[...], str] ='{fn_name}_{checksum_hex}{suffix}', cache_fname_fmt_args: Iterable[str] | None =None, cache_dump_fn: None =None, cache_load_fn: None =None, cache_enable: bool =True, cache_store_mode: 'outputs_only' | 'outputs_metadata' | 'outputs_metadata_inputs' =_DEFAULT_CACHE_STORE_MODE) Callable[[Callable[[P], T]], Callable[[P], T]] -
pythonwrench.disk_cache.disk_cache_decorator(fn: None =
None, *, cache_dpath: str | Path | None =None, cache_force: bool =False, cache_verbose: int =0, cache_checksum_fn: Callable[[tuple[Callable[[P], T], tuple, dict[str, Any]]], int] =checksum_any, cache_saving_backend: 'csv' | 'json' | 'pickle' | 'custom' | 'auto' ='auto', cache_fname_fmt: str | Callable[[...], str] ='{fn_name}_{checksum_hex}{suffix}', cache_fname_fmt_args: Iterable[str] | None =None, cache_dump_fn: Callable[[Any, Path], Any] | None =None, cache_load_fn: Callable[[Path], Any] | None =None, cache_enable: bool =True, cache_store_mode: 'outputs_only' | 'outputs_metadata' | 'outputs_metadata_inputs' =_DEFAULT_CACHE_STORE_MODE) Callable[[Callable[[P], T]], Callable[[P], T]] -
pythonwrench.disk_cache.disk_cache_decorator(fn: Callable[[P], T], *, cache_dpath: str | Path | None =
None, cache_force: bool =False, cache_verbose: int =0, cache_checksum_fn: Callable[[tuple[Callable[[P], T], tuple, dict[str, Any]]], int] =checksum_any, cache_saving_backend: 'custom', cache_fname_fmt: str | Callable[[...], str] ='{fn_name}_{checksum_hex}{suffix}', cache_fname_fmt_args: Iterable[str] | None =None, cache_dump_fn: Callable[[Any, Path], Any], cache_load_fn: Callable[[Path], Any], cache_enable: bool =True, cache_store_mode: 'outputs_only' | 'outputs_metadata' | 'outputs_metadata_inputs' =_DEFAULT_CACHE_STORE_MODE) Callable[[P], T] -
pythonwrench.disk_cache.disk_cache_decorator(fn: Callable[[P], T], *, cache_dpath: str | Path | None =
None, cache_force: bool =False, cache_verbose: int =0, cache_checksum_fn: Callable[[tuple[Callable[[P], T], tuple, dict[str, Any]]], int] =checksum_any, cache_saving_backend: 'csv' | 'json' | 'pickle' | 'custom' | 'auto' ='auto', cache_fname_fmt: str | Callable[[...], str] ='{fn_name}_{checksum_hex}{suffix}', cache_fname_fmt_args: Iterable[str] | None =None, cache_dump_fn: Callable[[Any, Path], Any] | None =None, cache_load_fn: Callable[[Path], Any] | None =None, cache_enable: bool =True, cache_store_mode: 'outputs_only' | 'outputs_metadata' | 'outputs_metadata_inputs' =_DEFAULT_CACHE_STORE_MODE) Callable[[P], T] Decorator to store function output in a cache file.
Cache file is identified by the checksum of the function arguments, and stored by default in “~/.cache/disk_cache/<Function_name>/” directory.
Example¶
>>> import pythonwrench as pw >>> @pw.disk_cache_decorator >>> def heavy_processing(): >>> # Lot of stuff here >>> ... >>> outputs = heavy_processing() # first time function is called >>> outputs = heavy_processing() # second time outputs is loaded from disk- Args:
fn: Function to store its output. By default, it must be a callable that returns a pickable object. cache_dpath: Cache directory path. defaults to “~/.cache/disk_cache”. cache_force: Force function call and overwrite cache. defaults to False. cache_verbose: Set verbose logging level. Higher means more verbose. defaults to 0. cache_checksum_fn: Checksum function to identify input arguments. defaults to
pythonwrench.checksum_any. cache_saving_backend: Optional saving backend. Can be one of (‘csv’, ‘json’, ‘pickle’, ‘custom’, ‘auto’). defaults to ‘auto’. cache_fname_fmt: Cache filename format. defaults to “{fn_name}_{checksum_hex}{suffix}”. cache_dump_fn: Dump/save function to store outputs and overwrite saving backend. defaults to None. cache_load_fn: Load function to store outputs and overwrite saving backend. defaults to None. cache_enable: Enable disk cache. If False, the function has no effect. defaults to True. cache_store_mode: Disk cache storage mode. By default, it store function output and saved date into the cache file. defaults to ‘outputs_metadata’.