class documentation

class SiteCrawler:

View In Hierarchy

A sitecrawler is a generator that yields a tuple of the form (PreParser, file) Its goal is to create a list of preparsers and files to be parsed based on the files and directories in the initial path. This generator is to be used in a for loop to parse all the files and produce the final website.

Method __init__ :param path: The path to the directory to be crawled :param destination: The path to the directory where the website will be created :param _env: The environment object :type path: str :type destination: str :type _env: environment...
Method __iter__ This method is used to iterate over the preparsers and files. :yield: A tuple of the form (PreParser, file) :ytype: (preparser.PreParser, str)
Method copy_unparsable_files This method is used to copy all the files that could not be parsed.
Method create_all_paths This method is used to create all the directories in the destination path.
Method create_file This method is used to create a file. :raises FileExistsError: If the file already exists and the force_rewrite option is False :param path: The path to the file to be created :type path: str
Method get_all_paths This method is used to get all the paths to be crawled. Its goal is to get all the paths to be crawled and to store them in the self.directories and self.files, self.files_to_copy variables.
Method list_recursively Recursive method to get all the paths to be crawled. :param path: The current path to be crawled :param root: The root path (initial path) :type path: str :type root: str :return: A tuple of the form (files, directories) :rtype: (list[(str, str)], list[(str, str)], list[(str, str)])...
Method set_all_preparsers This method is used to set all the preparsers and initialize them. The preparsers are stored in the self.preparsers variable. :return: self.preparsers :rtype: list[preparser.PreParser]
Instance Variable authorised_extensions Undocumented
Instance Variable destination_path Undocumented
Instance Variable directories Undocumented
Instance Variable files Undocumented
Instance Variable files_to_copy Undocumented
Instance Variable forbidden_folders Undocumented
Instance Variable force_rewrite Undocumented
Instance Variable global_dict_of_imports Undocumented
Instance Variable initial_path Undocumented
Instance Variable preparsers Undocumented
Instance Variable _env Undocumented
def __init__(self, path, destination, _env):

:param path: The path to the directory to be crawled :param destination: The path to the directory where the website will be created :param _env: The environment object :type path: str :type destination: str :type _env: environment.Environment

def __iter__(self):

This method is used to iterate over the preparsers and files. :yield: A tuple of the form (PreParser, file) :ytype: (preparser.PreParser, str)

def copy_unparsable_files(self):

This method is used to copy all the files that could not be parsed.

def create_all_paths(self):

This method is used to create all the directories in the destination path.

def create_file(self, path):

This method is used to create a file. :raises FileExistsError: If the file already exists and the force_rewrite option is False :param path: The path to the file to be created :type path: str

def get_all_paths(self):

This method is used to get all the paths to be crawled. Its goal is to get all the paths to be crawled and to store them in the self.directories and self.files, self.files_to_copy variables.

def list_recursively(self, path, root):

Recursive method to get all the paths to be crawled. :param path: The current path to be crawled :param root: The root path (initial path) :type path: str :type root: str :return: A tuple of the form (files, directories) :rtype: (list[(str, str)], list[(str, str)], list[(str, str)])

def set_all_preparsers(self):

This method is used to set all the preparsers and initialize them. The preparsers are stored in the self.preparsers variable. :return: self.preparsers :rtype: list[preparser.PreParser]

authorised_extensions: list[str] =

Undocumented

destination_path =

Undocumented

directories =

Undocumented

files =

Undocumented

files_to_copy =

Undocumented

forbidden_folders: list[str] =

Undocumented

force_rewrite =

Undocumented

global_dict_of_imports: dict =

Undocumented

initial_path =

Undocumented

preparsers: list =

Undocumented

_env =

Undocumented