`archivebox.logging_util`

Module Contents

Classes

`RuntimeStats`	mutable stats counter for logging archiving timing info to CLI output
`SmartFormatter`	Patched formatter that prints newlines in argparse help strings
`TimedProgress`	Show a progress bar and measure elapsed time until .end() is called

Functions

`debug_dict_summary`
`get_fd_info`
`reject_stdin`	Tell the user they passed stdin to a command that doesn’t accept it
`accept_stdin`	accept any standard input and return it as a string or None
`progress_bar`	show timer in the form of progress bar, with percentage and seconds remaining
`log_cli_command`
`log_importing_started`
`log_source_saved`
`log_parsing_finished`
`log_deduping_finished`
`log_crawl_started`
`log_indexing_process_started`
`log_indexing_process_finished`
`log_indexing_started`
`log_indexing_finished`
`log_archiving_started`
`log_archiving_paused`
`log_archiving_finished`
`log_link_archiving_started`
`log_link_archiving_finished`
`log_archive_method_started`
`log_archive_method_finished`	quote the argument with whitespace in a command so the user can copy-paste the outputted string directly to run the cmd
`log_list_started`
`log_list_finished`
`log_removal_started`
`log_removal_finished`
`pretty_path`	convert paths like …/ArchiveBox/archivebox/../output/abc into output/abc
`printable_filesize`
`printable_folders`
`printable_config`
`printable_folder_status`
`printable_dependency_version`

Data

_LAST_RUN_STATS

API

class archivebox.logging_util.RuntimeStats[source]

mutable stats counter for logging archiving timing info to CLI output

skipped: int[source]: 0

succeeded: int[source]: 0

failed: int[source]: 0

parse_start_ts: Optional[datetime.datetime][source]: None

parse_end_ts: Optional[datetime.datetime][source]: None

index_start_ts: Optional[datetime.datetime][source]: None

index_end_ts: Optional[datetime.datetime][source]: None

archiving_start_ts: Optional[datetime.datetime][source]: None

archiving_end_ts: Optional[datetime.datetime][source]: None

archivebox.logging_util._LAST_RUN_STATS[source]: ‘RuntimeStats(…)’

archivebox.logging_util.debug_dict_summary(obj: Dict[Any, Any]) → None[source]

archivebox.logging_util.get_fd_info(fd) → Dict[str, Any][source]

class archivebox.logging_util.SmartFormatter[source]

Bases: django.core.management.base.DjangoHelpFormatter, rich_argparse.RichHelpFormatter

Patched formatter that prints newlines in argparse help strings

_split_lines(text, width)[source]

archivebox.logging_util.reject_stdin(caller: str, stdin: Optional[IO] = sys.stdin) → None[source]: Tell the user they passed stdin to a command that doesn’t accept it

archivebox.logging_util.accept_stdin(stdin: Optional[IO] = sys.stdin) → Optional[str][source]: accept any standard input and return it as a string or None

class archivebox.logging_util.TimedProgress(seconds, prefix='')[source]

Show a progress bar and measure elapsed time until .end() is called

Initialization

end()[source]: immediately end progress, clear the progressbar line, and save end_ts

archivebox.logging_util.progress_bar(seconds: int, prefix: str = '', ANSI: Dict[str, str] = ANSI) → None[source]: show timer in the form of progress bar, with percentage and seconds remaining

archivebox.logging_util.log_cli_command(subcommand: str, subcommand_args: List[str], stdin: Optional[str | IO], pwd: str = '.')[source]

archivebox.logging_util.log_importing_started(urls: Union[str, List[str]], depth: int, index_only: bool)[source]

archivebox.logging_util.log_source_saved(source_file: str)[source]

archivebox.logging_util.log_parsing_finished(num_parsed: int, parser_name: str)[source]

archivebox.logging_util.log_deduping_finished(num_new_links: int)[source]

archivebox.logging_util.log_crawl_started(new_links)[source]

archivebox.logging_util.log_indexing_process_started(num_links: int)[source]

archivebox.logging_util.log_indexing_process_finished()[source]

archivebox.logging_util.log_indexing_started(out_path: str)[source]

archivebox.logging_util.log_indexing_finished(out_path: str)[source]

archivebox.logging_util.log_archiving_started(num_links: int, resume: Optional[float] = None)[source]

archivebox.logging_util.log_archiving_paused(num_links: int, idx: int, timestamp: str)[source]

archivebox.logging_util.log_archiving_finished(num_links: int)[source]

archivebox.logging_util.log_link_archiving_started(link: archivebox.index.schema.Link, link_dir: str, is_new: bool)[source]

archivebox.logging_util.log_link_archiving_finished(link: archivebox.index.schema.Link, link_dir: str, is_new: bool, stats: dict, start_ts: datetime.datetime)[source]

archivebox.logging_util.log_archive_method_started(method: str)[source]

archivebox.logging_util.log_archive_method_finished(result: archivebox.index.schema.ArchiveResult)[source]: quote the argument with whitespace in a command so the user can copy-paste the outputted string directly to run the cmd

archivebox.logging_util.log_list_started(filter_patterns: Optional[List[str]], filter_type: str)[source]

archivebox.logging_util.log_list_finished(links)[source]

archivebox.logging_util.log_removal_started(links: List[archivebox.index.schema.Link], yes: bool, delete: bool)[source]

archivebox.logging_util.log_removal_finished(all_links: int, to_remove: int)[source]

archivebox.logging_util.pretty_path(path: Union[pathlib.Path, str], pwd: Union[pathlib.Path, str] = DATA_DIR, color: bool = True) → str[source]: convert paths like …/ArchiveBox/archivebox/../output/abc into output/abc

archivebox.logging_util.printable_filesize(num_bytes: Union[int, float]) → str[source]

archivebox.logging_util.printable_folders(folders: Dict[str, Optional[archivebox.index.schema.Link]], with_headers: bool = False) → str[source]

archivebox.logging_util.printable_config(config: dict, prefix: str = '') → str[source]

archivebox.logging_util.printable_folder_status(name: str, folder: Dict) → str[source]

archivebox.logging_util.printable_dependency_version(name: str, dependency: Dict) → str[source]

archivebox.logging_util