module documentation
Module for file and directories repartition Usage:
- from bootstraparse.modules import SiteCrawler
- crawler = SiteCrawler(origin_path, dest_path, _env)
- crawler.set_all_preparsers()
- crawler.copy_unparsable_files()
- for element, destination in crawler:
- - save(preparse_parse(element), destination, _env)
| Class | |
A sitecrawler is a generator that yields a tuple of the form (PreParser, file) Its goal is to create a list of preparsers and files to be parsed based on the files and directories in the initial path. ... |