module documentation

Module for file and directories repartition Usage:

  • from bootstraparse.modules import SiteCrawler
  • crawler = SiteCrawler(origin_path, dest_path, _env)
  • crawler.set_all_preparsers()
  • crawler.copy_unparsable_files()
  • for element, destination in crawler:
  • - save(preparse_parse(element), destination, _env)
Class SiteCrawler A sitecrawler is a generator that yields a tuple of the form (PreParser, file) Its goal is to create a list of preparsers and files to be parsed based on the files and directories in the initial path. ...