Compare commits

..

No commits in common. "main" and "V1.0.1" have entirely different histories.
main ... V1.0.1

29 changed files with 288 additions and 398 deletions

View File

@ -5,9 +5,9 @@ name: Python Deployment
on: on:
push: push:
branches: [ main, test*] branches: [ main ]
pull_request: pull_request:
branches: [ main, test* ] branches: [ main ]
jobs: jobs:

View File

@ -5,9 +5,9 @@ name: Python Tests and Lint
on: on:
push: push:
branches: [ main, release, releases/*, test*] branches: [ main, release, releases/*]
pull_request: pull_request:
branches: [ main, release, releases/*, test* ] branches: [ main, release, releases/* ]
jobs: jobs:

3
.gitignore vendored
View File

@ -156,5 +156,4 @@ cython_debug/
example_userfiles/output/ example_userfiles/output/
example_userfiles/output/* example_userfiles/output/*
dev_outputs/* dev_outputs/*
example_output/* /example_output/*
docs/

View File

@ -6,11 +6,10 @@ bootstraparse aims to be customisable in that regard, but its first iteration an
You can refer to the `example_userfiles` for an idea of what the parser expects you to feed it and find an example of our automated generation at https://idle-org.github.io/bootstraparse/ for this particular folder. Or you can run the program yourself for the same result! You can refer to the `example_userfiles` for an idea of what the parser expects you to feed it and find an example of our automated generation at https://idle-org.github.io/bootstraparse/ for this particular folder. Or you can run the program yourself for the same result!
- Main - Main
- [![Python Tests and Lint](https://github.com/idle-org/bootstraparse/actions/workflows/python-tests.yml/badge.svg?branch=main)](https://github.com/idle-org/bootstraparse/actions/workflows/python-tests.yml) - [![Python Main](https://github.com/idle-org/bootstraparse/actions/workflows/python-app.yml/badge.svg?branch=main)](https://github.com/idle-org/bootstraparse/actions/workflows/python-app.yml)
- Releases - Releases
- [![Python Tests and Lint](https://github.com/idle-org/bootstraparse/actions/workflows/python-tests.yml/badge.svg?branch=develop)](https://github.com/idle-org/bootstraparse/actions/workflows/python-tests.yml) - [![Python Release](https://github.com/idle-org/bootstraparse/actions/workflows/python-app.yml/badge.svg?branch=release)](https://github.com/idle-org/bootstraparse/actions/workflows/python-app.yml)
- Deploy
- [![Python Deployment](https://github.com/idle-org/bootstraparse/actions/workflows/python-deploy.yml/badge.svg?branch=main)](https://github.com/idle-org/bootstraparse/actions/workflows/python-deploy.yml)
--- ---
## Release Notes ## Release Notes
### V1.0.1 ### V1.0.1
@ -21,29 +20,11 @@ You can refer to the `example_userfiles` for an idea of what the parser expects
- Copy un-parsable files to destination folder without modifications - Copy un-parsable files to destination folder without modifications
- Many bug fixes and improvements - Many bug fixes and improvements
### V1.0.2
- Put out many fires
### V1.0.3
- Put out many more fires
- Fixed continuous integration
- Added doc generation
### V1.0.4
- Made the default example site more useful
- Wrote a barebone documentation for the application
## Roadmap ## Roadmap
### V1.0.5 ### V1.0.2
- Add more context to the warning messages ☐
- Test configs more in-depth ☐
- Test overall site generation ☐
- Move the WARNING-level output to INFO for config overwrite ☐
### V1.0.6
- Option to ignore br behavior ☐
- Make the default example site way more useful ☐ - Make the default example site way more useful ☐
- Write a documentation for the application (use specs.yaml) ☐
- Move the WARNING-level output to INFO for config overwrite ☐
### V1.1 ### V1.1
- Functioning `table` Token ☐ - Functioning `table` Token ☐
@ -51,24 +32,18 @@ You can refer to the `example_userfiles` for an idea of what the parser expects
- Establish a list of all configurable parameters to implement in the future, and update the roadmap with them. ☐ - Establish a list of all configurable parameters to implement in the future, and update the roadmap with them. ☐
- Decide level of logic to be implemented, and whether it should be configurable ☐ - Decide level of logic to be implemented, and whether it should be configurable ☐
- Update the uses of remaining files ☐ - Update the uses of remaining files ☐
- Add a global line count and a line tracker to the parser ☐
### V1.2 ### V1.2
- Check error messages and add a real debug level to parameters ☐ - Check error messages and add a real debug level to parameters ☐
- Thouroughly test logging, and error messages for the user ☐
- Functioning `blockquote` Token ☐ - Functioning `blockquote` Token ☐
### V1.3 ### V1.3
- Add advanced lookahead logic for `*` - Add advanced lookahead logic for `*` ☐
- Functioning `lead` Token
### V1.4 ### V1.4
- Add html indentation for human readability of the output. ☐ - Add html indentation for human readability of the output. ☐
- Write thourough documentation for the application (use specs.yaml) ☐
### V2 ### V2
- Add a functioning post-context enhancer able to generate menus and elements from arbitrary logic. - Add a functioning post-context enhancer able to generate menus and elements from arbitrary logic. ☐
- Achieve perfect markdown compatibility with appropriate config parameters - Achieve perfect markdown compatibility with appropriate config parameters ☐
### V3
- Add Template customisation.

View File

@ -10,7 +10,7 @@ shortcuts:
tab_bnw: 'tab_bnw_imported' tab_bnw: 'tab_bnw_imported'
tab_color: 'tab_colour_imported' tab_color: 'tab_colour_imported'
custom_permutation: 'custom_permutated' custom_permutation: 'custom_permutated'
empty: ''
start: | start: |
<!doctype html> <!doctype html>
<html lang="en"> <html lang="en">

View File

@ -1,48 +0,0 @@
Structural Elements:
<<div
This is a div
div>>{class='blue', 123}{{cinsertion}}[var='test', number=11]
<<article
This is an article
article>>{class='blue', 123}{{cinsertion}}[var='test', number=11]
<<aside
This is an aside
aside>>{class='blue', 123}{{cinsertion}}[var='test', number=11]
<<section
This is a section
section>>{class='blue', 123}{{cinsertion}}[var='test', number=11]
<<div
<<article
<<aside
<<section
Text
section>>{class='blue', 123}{{cinsertion}}[var='test', number=11]
aside>>{class='blue', 123}{{cinsertion}}[var='test', number=11]
article>>{class='blue', 123}{{cinsertion}}[var='test', number=11]
div>>{class='blue', 123}{{cinsertion}}[var='test', number=11]
# Header 1 #{class='blue', 123}{{cinsertion}}[var='test', number=11]
! Display 1 !{class='blue', 123}{{cinsertion}}[var='test', number=11]
Inline Element:
[Link text]('http://this.is.a.link')
*This is in italic*
**This is strong**
~~This is stricken through~~
__This is underlined__
(#1)This is a custom span(#1)
Oneline elements:
- This is an unordered list item
- This is an unordered list item
- This is an unordered list item{class='blue', 123}{{cinsertion}}[var='test', number=11]
#. This is an ordered list item
#. This is an ordered list item
#. This is an ordered list item{class='blue', 123}{{cinsertion}}[var='test', number=11]
This is a long text segment. This is a long text segment. This is a long text segment. This is a long text segment. This is a long text segment.
This is still the same long text segment. This is still the same long text segment. This is still the same long text segment.
Use a double linebreak to force linebreaks.

View File

@ -1,28 +1,51 @@
@[start]['Bootstraparse'] @[start]['bootstrap']
<<div
<<header
<<div
<<div
Bootstraparse
div>>{{blog-header-logo text-dark text-center border-bottom}}
div>>{{row flex-nowrap justify-content-between align-items-center}}
header>>{{blog-header py-3}}
<<div
# What is bootstraparse ? #
bootstraparse is a tool that allows you to bootstrap your own project. Structural Elements:
In a very small number of steps, you can create a new project with the <<div
following features: This is a div
- Imports, with the <code> ::&lt; filename > </code>syntax you can import another file and have it's contents dumped into your file. div>>{class='blue', 123}{{cinsertion}}[var='test', number=11]
- Markup, a subset of markdown is already supported and the rest is incomming ! <<article
- Aliases : With the <code> @&#91;alias_name&#93; </code> syntax you can create an alias for a string of text, with replacement capabilities. This is an article
- Images: With the <code> @&#123;imagename&#125; </code> syntax you can specify an image template to import, with replacement capabilities. article>>{class='blue', 123}{{cinsertion}}[var='test', number=11]
<<aside
And much much more ! This is an aside
aside>>{class='blue', 123}{{cinsertion}}[var='test', number=11]
<<section
This is a section
section>>{class='blue', 123}{{cinsertion}}[var='test', number=11]
<<div
<<article
<<aside
<<section
Text
section>>{class='blue', 123}{{cinsertion}}[var='test', number=11]
aside>>{class='blue', 123}{{cinsertion}}[var='test', number=11]
article>>{class='blue', 123}{{cinsertion}}[var='test', number=11]
div>>{class='blue', 123}{{cinsertion}}[var='test', number=11]
# Header 1 #{class='blue', 123}{{cinsertion}}[var='test', number=11]
! Display 1 !{class='blue', 123}{{cinsertion}}[var='test', number=11]
We encourage you to read the [documentation]("apidoc/index.html") to learn more about bootstraparse Inline Element:
or explore the [source code]("https://github.com/idle-org/bootstraparse") [Link text]('http://this.is.a.link')
div>>{{nav-scroller py-1 mb-2}} *This is in italic*
div>>{{container}} **This is strong**
~~This is stricken through~~
__This is underlined__
(#1)This is a custom span(#1)
Oneline elements:
- This is an unordered list item
- This is an unordered list item
- This is an unordered list item{class='blue', 123}{{cinsertion}}[var='test', number=11]
#. This is an ordered list item
#. This is an ordered list item
#. This is an ordered list item{class='blue', 123}{{cinsertion}}[var='test', number=11]
This is a long text segment. This is a long text segment. This is a long text segment. This is a long text segment. This is a long text segment.
This is still the same long text segment. This is still the same long text segment. This is still the same long text segment.
Use a double linebreak to force linebreaks.
@[end] @[end]

View File

@ -2,14 +2,15 @@
name = bootstraparse name = bootstraparse
description = A custom markdown parser for Bootstrap & Python description = A custom markdown parser for Bootstrap & Python
author = Vkyfox author = Vkyfox
version = 1.0.4 version = 1.0.1
license = Mozilla Public License Version 2.0 license = Mozilla Public License Version 2.0
license_file = LICENSE license_file = LICENSE
platforms = unix, linux, osx, cygwin, win32 platforms = unix, linux, osx, cygwin, win32
classifiers = classifiers =
Programming Language :: Python :: 3 Programming Language :: Python :: 3
Programming Language :: Python :: 3 :: Only Programming Language :: Python :: 3 :: Only
Programming Language :: Python :: 3.10 Programming Language :: Python :: 3.7
Programming Language :: Python :: 3.8
Programming Language :: Python :: 3.9 Programming Language :: Python :: 3.9
[options] [options]
@ -18,7 +19,7 @@ packages =
bootstraparse.modules bootstraparse.modules
install_requires = install_requires =
pyyaml>=6.0 pyyaml>=6.0
python_requires = >=3.9 python_requires = >=3.6
package_dir = package_dir =
=src =src

View File

@ -1,5 +0,0 @@
"""
Bootstraparse is a tool to parse a folder and create a website from it.
After installing it with pip, you can use it by running the following command:
- bootstraparse <origin> <destination>
"""

View File

@ -1,8 +1,8 @@
#!/usr/bin/python3 #!/usr/bin/python3
""" Main program, use this to start parsing""" # Main program, use this to start parsing
from bootstraparse.modules import sitecreator, error_mngr from bootstraparse.modules import sitecreator
import argparse import argparse
import sys import sys
@ -21,6 +21,5 @@ def parse(_args):
if __name__ == "__main__": # pragma: no cover if __name__ == "__main__": # pragma: no cover
args = parse(sys.argv[1:]) args = parse(sys.argv[1:])
error_mngr.init_logging(filename=None, loglevel="DEBUG", filemode='w', handler=None)
if sitecreator.create_website(args.origin, args.destination) == 0: if sitecreator.create_website(args.origin, args.destination) == 0:
print("Bootstraparse run successful!") print("Bootstraparse run successful!")

View File

@ -2,7 +2,7 @@ global:
template: "bootstrap.yml" template: "bootstrap.yml"
output: "example_site/" output: "example_site/"
bootstrap_version: "3.3.7" bootstrap_version: "3.3.7"
version: "1.0.4" version: "1.0.0"
bootstrap_location: "" bootstrap_location: ""
logging: logging:
@ -21,4 +21,4 @@ export:
intermediate_files: true intermediate_files: true
type: "html" type: "html"
force_rewrite: true force_rewrite: true
copy_unparsable_files: copy copy_unparsable_files: "copy" # "copy" or "ignore"

View File

@ -1,3 +0,0 @@
"""
The modules in this folder are used to parse the website.
"""

View File

@ -1,11 +1,11 @@
"""Interprets config files # Interprets config files
The Config will be loaded by the environment at runtime and should be accessed that way # The Config will be loaded by the environment at runtime and should be accessed that way
Usage: # Usage:
- from bootstraparse.modules.config import ConfigLoader # from bootstraparse.modules.config import ConfigLoader
- config = ConfigLoader([list of config files], extensions=[list of extensions]) # config = ConfigLoader([list of config files], extensions=[list of extensions])
- config['file']['key'] # config['file']['key']
The ConfigLoader class has load_from_file and load_from_folder methods and __getitem__ for accessing loaded elements # The ConfigLoader class has load_from_file and load_from_folder methods and __getitem__ for accessing loaded elements
"""
import os import os
import rich import rich
@ -74,16 +74,11 @@ class ConfigLoader:
if name not in self.loaded_conf: if name not in self.loaded_conf:
self.loaded_conf[name] = yaml.safe_load(f) self.loaded_conf[name] = yaml.safe_load(f)
else: else:
new_config = yaml.safe_load(f) self.loaded_conf[name].update(yaml.safe_load(f))
for key in new_config: error_mngr.log_message(f"Warning: {name} is already in {self.loaded_conf}", level='INFO')
if key in self.loaded_conf[name]:
self.loaded_conf[name][key].update(new_config[key])
else:
self.loaded_conf[name][key] = new_config[key]
error_mngr.log_message(f"Warning: {name} is already in {self.loaded_conf}", level='CRITICAL')
except BaseException as e: except BaseException as e:
error_mngr.log_message(f'Error parsing in file {basename} at {filepath}.', level='CRITICAL') error_mngr.log_message(f'Error parsing in file {basename} at {filepath}.', level='CRITICAL')
error_mngr.log_exception(e, level='CRITICAL') error_mngr.log_exception(str(e), level='CRITICAL')
def load_from_folder(self, folder): def load_from_folder(self, folder):
""" """

View File

@ -1,16 +1,16 @@
"""Interprets where the parser is located and resolves errors # Interprets where the parser is located and resolves errors
You're probably here for the ContextManager class. # You're probably here for the ContextManager class.
Containers have class_name, debug_map and validate methods for debug, # Containers have class_name, debug_map and validate methods for debug,
- self.content: accessible via __getitem__, __setitem__, add, __len__, __iter__ and __getslice__. # self.content: accessible via __getitem__, __setitem__, add, __len__, __iter__ and __getslice__.
- self.optionals: accessible via __rshift__ (remapped to return self.map[other]()) # self.optionals: accessible via __rshift__ (remapped to return self.map[other]())
- and self.map: accessible via __invert__ (remapped to return self.optionals) # and self.map: accessible via __invert__ (remapped to return self.optionals)
Usage: # Usage:
- from bootstraparse.modules.context_mngr import ContextManager # from bootstraparse.modules.context_mngr import ContextManager
- ctx = ContextManager() # ctx = ContextManager()
- container = BaseContainer() # container = BaseContainer()
- container[number] -> The number element in the content # container[number] -> The number element in the content
- "class_insert" >> container[number] -> Get an element from one of the mapped methods # "class_insert" >> container[number] -> Get an element from one of the mapped methods
"""
from bootstraparse.modules import syntax, error_mngr, export from bootstraparse.modules import syntax, error_mngr, export
from bootstraparse.modules.error_mngr import MismatchedContainerError, log_exception, log_message, LonelyOptionalError # noqa from bootstraparse.modules.error_mngr import MismatchedContainerError, log_exception, log_message, LonelyOptionalError # noqa
@ -208,7 +208,7 @@ class BaseContainer:
:rtype : str :rtype : str
""" """
return '{} ({})'.format(", ".join(map(str, self.content)), return '{} ({})'.format(", ".join(map(str, self.content)),
", ".join(map(str, self.get_optionals().content)) if self.get_optionals() else "") ", ".join(map(str, self.get_optionals())) if self.get_optionals() else "")
def __repr__(self): def __repr__(self):
""" """
@ -335,7 +335,7 @@ class SeContainer(BaseContainer):
type = "structural_elements" type = "structural_elements"
def export(self, exm): def export(self, exm):
self.subtype = "se_"+self[0].content[0] self.subtype = self[0].content[0]
return super().export(exm) return super().export(exm)
@ -436,12 +436,14 @@ class ContextManager:
def __init__(self, parsed_list, name=None, ident=0): def __init__(self, parsed_list, name=None, ident=0):
""" """
Takes a list of parsed tokens. Takes a list of parsed tokens.
:type parsed_list : list[syntax.SemanticType] Parameters
:param parsed_list: List of parsed tokens output by our parser. ----------
:type name: str parsed_list : list[syntax.SemanticType]
:param name: Name of the file being parsed. List of parsed tokens output by our parser.
:type ident: int name : str
:param ident: Number of spaces to indent the output. Name of the file being parsed.
ident : int
Number of spaces to indent the output.
""" """
self.output = [] self.output = []
self.parsed_list = parsed_list self.parsed_list = parsed_list
@ -462,12 +464,15 @@ class ContextManager:
""" """
Method to encapsulate a number of tokens together as a final container object Method to encapsulate a number of tokens together as a final container object
and replace all elements in the pile with None, except the first one which becomes the container. and replace all elements in the pile with None, except the first one which becomes the container.
:type start : int Parameters
:param start: Index of the pile where the encapsulation must begin. ----------
:type end: int start : int
:param end: Index of the pile where the encapsulation must end. Index of the pile where the encapsulation must begin.
:rtype: list[(BaseContainer|None)] end : int
:return: List of containers and None objects. Index of the pile where the encapsulation must end.
Returns
-------
list[(BaseContainer|None)]
""" """
# Enabling this will allow us to debug the pile # Enabling this will allow us to debug the pile
# print(f"Step : [{end}] - encapsulation:") # print(f"Step : [{end}] - encapsulation:")
@ -509,16 +514,18 @@ class ContextManager:
container.add(self.pile[end]) container.add(self.pile[end])
self.pile[end] = None self.pile[end] = None
self.pile[start] = container self.pile[start] = container
return self.pile # ? I think this is the right way to return the pile
# print(" <<< Pile:", self.pile) # print(" <<< Pile:", self.pile)
def _add_matched(self, label, index): def _add_matched(self, label, index):
""" """
Method to add a matched element to the dictionary of matched elements. Method to add a matched element to the dictionary of matched elements.
:type label : str Parameters
:param label: Label of the matched element. ----------
:type index: int label : str
:param index: Index of the matched element in the pile. Label of the matched element.
index : int
Index of the matched element in the pile.
""" """
if label not in self.matched_elements: if label not in self.matched_elements:
self.matched_elements[label] = [] self.matched_elements[label] = []
@ -527,10 +534,13 @@ class ContextManager:
def _get_matched(self, label): def _get_matched(self, label):
""" """
Method to get the index of the last matched element of a given label. Method to get the index of the last matched element of a given label.
:type label: str Parameters
:param label: Label of the matched element. ----------
:rtype: int label : str
:return: Index of the last matched element Label of the matched element.
Returns
-------
int : Index of the last matched element of the given label.
""" """
return self.matched_elements[label].pop() return self.matched_elements[label].pop()
@ -538,9 +548,14 @@ class ContextManager:
""" """
Interprets the list of tokens provided and deduces context, encapsulating them to their closest neighbour and Interprets the list of tokens provided and deduces context, encapsulating them to their closest neighbour and
containing all tokens inbetween. containing all tokens inbetween.
:rtype: list[BaseContainer] Returns
:return: Returns the pile entirely processed as a list of containers. -------
:raises: MismatchedContainerError if a container is not matched to its corresponding token. list[BaseContainer]
Returns the pile entirely processed as a list of containers.
Raise
-----
MismatchedContainerError
Raised when a container is not matched to its corresponding token.
""" """
# self.pile = self.parsed_list.copy() # self.pile = self.parsed_list.copy()
if self.contextualised: if self.contextualised:
@ -551,7 +566,6 @@ class ContextManager:
while index < len(self.parsed_list): while index < len(self.parsed_list):
token = self.parsed_list[index] token = self.parsed_list[index]
token.line_number = line_number token.line_number = line_number
token.index = index
token.file_name = self.name token.file_name = self.name
token.ident = self.ident token.ident = self.ident
self.pile.append(token) self.pile.append(token)
@ -584,17 +598,16 @@ class ContextManager:
# Error if closing token does not have a start # Error if closing token does not have a start
elif isinstance(token, syntax.ClosedSemanticType): elif isinstance(token, syntax.ClosedSemanticType):
raise MismatchedContainerError(token, self.parsed_list) raise MismatchedContainerError(token)
# Starting token by default (can cause unintended behaviours on bad implementations) # Starting token by default (can cause unintended behaviours on bad implementations)
elif isinstance(token, syntax.TokensToMatch): elif isinstance(token, syntax.TokensToMatch):
self._add_matched(token.label, index) self._add_matched(token.label, index)
else: else:
raise MismatchedContainerError(token, self.parsed_list) raise MismatchedContainerError(token)
except MismatchedContainerError as e: except MismatchedContainerError as e:
error_mngr.log_message(f"In context: {e.get_context()}", level="CRITICAL")
error_mngr.log_exception(e, level="CRITICAL") # FUTURE: Try to guess some hints. error_mngr.log_exception(e, level="CRITICAL") # FUTURE: Try to guess some hints.
index += 1 index += 1
@ -642,12 +655,15 @@ class ContextManager:
def lookahead(self, token, index): def lookahead(self, token, index):
""" """
Iterates the pile beginning from index and looks for all tokens matching labels with token. Iterates the pile beginning from index and looks for all tokens matching labels with token.
:type token : syntax.BaseContainer Parameters
:param token: Token to look for. ----------
:type index : int token : syntax.BaseContainer
:param index: Index of the token in the pile. Token to look for.
:return: (number of tokens matched, number of linebreaks) index : int
:rtype: (int, int) Index of the token in the pile.
Returns
-------
(int, int) : (number of tokens matched, number of linebreaks)
""" """
range_to_encapsulate = 0 range_to_encapsulate = 0
line_skipped = 0 line_skipped = 0
@ -675,21 +691,29 @@ class ContextManager:
def recontext(self, token): def recontext(self, token):
""" """
Function to recontextualise the content of a token. Function to recontextualise the content of a token.
:param token: Token to recontextualise. Parameters
:type token: syntax.SemanticType ----------
:return: Token to recontextualise. token : syntax.SemanticType
:rtype: syntax.SemanticType Token to recontextualise.
""" """
token.content = ContextManager(token.content, name=self.name)() token.content = ContextManager(token.content, name=self.name)()
def get_last_container_in_pile(self, index): def get_last_container_in_pile(self, index):
""" """
Returns the last element in the pile if it is a Container, raises an error otherwise. Returns the last element in the pile if it is a Container, raises an error otherwise.
:type index : int Parameters
:param index: Index of the element to check. ----------
:return: Last element in the pile if it is a Container, raises an error otherwise. index : int
:rtype: BaseContainer Index of the element to check.
:raise: error_mngr.LonelyOptionalError when an element is found in the pile that is not encapsulated in a container.
Returns
-------
BaseContainer
Raises
------
error_mngr.LonelyOptionalError
Raised when an element is found in the pile that is not encapsulated in a container.
""" """
i = index-1 # skip last token as it is self i = index-1 # skip last token as it is self
while i >= 0: while i >= 0:

View File

@ -1,20 +1,19 @@
"""Module for containing super important variables # Module for containing super important variables
Environment has self._mParams for mandatory parameters, self._sParams for secondary parameters and # Environment has self._mParams for mandatory parameters, self._sParams for secondary parameters and
- self.wasInitialised and integrity_check for debug. # self.wasInitialised and integrity_check for debug.
- __getattr__ and __setattr__ check for existing super, s or m parameters, returns error if not found. # __getattr__ and __setattr__ check for existing super, s or m parameters, returns error if not found.
Usage: # Usage:
- from bootstraparse.modules.environment import Environment # from bootstraparse.modules.environment import Environment
- env = Environment() # env = Environment()
- env.integrity_check() -> Checks if all mandatory parameters are set. # env.integrity_check() -> Checks if all mandatory parameters are set.
- env["config"] -> returns config object # env["config"] -> returns config object
- env["template"] -> returns template object # env["template"] -> returns template object
- env["export_mngr"] -> returns export manager object # env["export_mngr"] -> returns export manager object
- env["global_path"] -> returns global path ??? # env["global_path"] -> returns global path ???
- env["user_path"] -> returns user path ??? # env["user_path"] -> returns user path ???
- env["site_path"] -> returns site path # env["site_path"] -> returns site path
- env["export"] -> returns export object ??? # env["export"] -> returns export object ???
- env["site_crawler"] -> returns site crawler object # env["site_crawler"] -> returns site crawler object
"""
from bootstraparse.modules import error_mngr from bootstraparse.modules import error_mngr

View File

@ -1,14 +1,12 @@
""" # Dedicated module for defining and interpreting errors as well as generating error messages
Dedicated module for defining and interpreting errors as well as generating error messages # You're probably here for log_message or log_exception.
You're probably here for log_message or log_exception. # They take: message, level=("ERROR, "INFO", "WARNING", "DEBUG", "CRITICAL")
- They take: message, level=("ERROR, "INFO", "WARNING", "DEBUG", "CRITICAL") # There's also a dict_check(dic, *args) that returns true if all args are keys in the dict.
There's also a dict_check(dic, *args) that returns true if all args are keys in the dict. # Usage:
Usage: # from bootstraparse.modules.error_mngr import log_message, log_exception
- from bootstraparse.modules.error_mngr import log_message, log_exception # log_message("This is a message", level="INFO") level=("ERROR, "INFO", "WARNING", "DEBUG", "CRITICAL")
- log_message("This is a message", level="INFO") level=("ERROR, "INFO", "WARNING", "DEBUG", "CRITICAL") # log_exception(Exception("This is an exception"), level="ERROR")
- log_exception(Exception("This is an exception"), level="ERROR") # dict_check({"a": 1, "b": 2}, "a", "b") # returns [True, True]
- dict_check({"a": 1, "b": 2}, "a", "b") # returns [True, True]
"""
import logging import logging
import traceback import traceback
@ -95,54 +93,6 @@ class BootstraparseError(Exception):
pass pass
class BootstraparseTokenError(BootstraparseError):
"""
Error on a token, provides if possible the line and column number of the error,
the file name and the message of the error
and if possible a context for the error
"""
def __init__(self, token, pile, message):
self.token = token
self.pile = pile
if token is not None:
self.line = token.line_number
self.index = token.index
self.label = token.label
self.name = token.file_name
else:
self.line = None
self.index = None
self.label = None
self.name = None
self.context = self.get_context()
super().__init__(message+"\n"+self.context_multiline())
def get_context(self, context_size=5):
"""
Returns the context of the error
:return: The context of the error
:rtype: str
"""
if self.pile:
st = max(0, self.index - context_size)
en = min(len(self.pile), self.index + context_size)
return self.pile[st:en]
return []
def context_multiline(self, context_size=10):
"""
Returns the context of the error as a multiline string
:return: The context of the error
:rtype: str
"""
if self.pile:
return "\n".join([
f"{' ' if i!=context_size else '> '}{i-context_size:2d}: {j}"
for i, j in enumerate(self.get_context(context_size)) if j.label != "linebreak"
])
return ""
class ParsingError(Exception): class ParsingError(Exception):
""" """
Exception class for parsing errors Exception class for parsing errors
@ -181,18 +131,17 @@ class ParsingError(Exception):
return str(self.message) return str(self.message)
class MismatchedContainerError(BootstraparseTokenError): class MismatchedContainerError(BootstraparseError):
""" """
The token is not final and cannot be contained. Indicative of a mismatched token. The token is not final and cannot be contained. Indicative of a mismatched token.
""" """
def __init__(self, token, pile=None): def __init__(self, token):
""" """
Initializes the MismatchedContainerError class with the token that was not final Initializes the MismatchedContainerError class with the token that was not final
:param token: The token that is not final :param token: The token that is not final
:type token: syntax.SemanticType :type token: syntax.SemanticType
""" """
self.token = token self.token = token
self.pile = pile
if token: if token:
self.line = token.line_number self.line = token.line_number
self.label = token.label self.label = token.label
@ -201,7 +150,7 @@ class MismatchedContainerError(BootstraparseTokenError):
self.line = None self.line = None
self.label = None self.label = None
self.name = None self.name = None
super().__init__(token, pile, f"Could not process {self.label} at line {self.line} in file {self.name}.") super().__init__(f"Could not process {self.label} at line {self.line} in file {self.name}.")
class LonelyOptionalError(BootstraparseError): class LonelyOptionalError(BootstraparseError):

View File

@ -1,12 +1,10 @@
""" # Module for final generation of the translated website
Module for final generation of the translated website # Usage:
Usage: # from bootstraparse.modules.export import ExportManager, ExportRequest, ExportResponse
- from bootstraparse.modules.export import ExportManager, ExportRequest, ExportResponse # rqs = ExportRequest("strucural_elements", "div", "optionals", {"id": "my_id", "class": "my_class"})
- rqs = ExportRequest("strucural_elements", "div", "optionals", {"id": "my_id", "class": "my_class"}) # rsp = ExportResponse("start_string", "end_string")
- rsp = ExportResponse("start_string", "end_string") # em = ExportManager(config_file, template_file)
- em = ExportManager(config_file, template_file) # em(ExportRequest()) -> ExportResponse()
- em(ExportRequest()) -> ExportResponse()
"""
from io import StringIO from io import StringIO
from bootstraparse.modules import config, pathresolver, error_mngr, context_mngr from bootstraparse.modules import config, pathresolver, error_mngr, context_mngr
@ -96,10 +94,18 @@ class ExportManager:
""" """
Function for initializing other transform functions. Function for initializing other transform functions.
:param export_request: ExportRequest tuples Parameters:
:type export_request: ExportRequest ----------
:return: start, end, optionals export_request: ExportRequest tuples
:rtype: str, str, str
Returns:
-------
start: str
Start string of the markup element
end: str
End string of the markup element
optionals: syntax.Optionals
Optionals object containing html_insert and class_insert
""" """
start, end = None, None start, end = None, None
@ -200,12 +206,14 @@ class ContextConverter:
def __init__(self, pile, exporter, destination): def __init__(self, pile, exporter, destination):
""" """
Takes a pile and the exporter object Takes a pile and the exporter object
:type pile : list[context_mngr.BaseContainer] Parameters
:param pile: output from the context manager. ----------
:type exporter : ExportManager pile : list[context_mngr.BaseContainer]
:param exporter: Our ExportManager pile output from the context manager.
:type destination : str exporter : ExportManager
:param destination: Destination of the output file. Our ExportManager
destination : str
Destination of the output file.
""" """
self.io_output = StringIO() self.io_output = StringIO()
self.pile = pile self.pile = pile

View File

@ -1,10 +1,9 @@
""" # Main parser, gets config files and parses them for the context manager.
Main parser, gets config files and parses them for the context manager. # parse_line takes a io and outputs it all as a list of parsed elements
parse_line takes a io and outputs it all as a list of parsed elements # Usage:
Usage: # from bootstraparse.modules.parser import parse_line
- from bootstraparse.modules.parser import parse_line # parse_line(io) -> [element, element, element]
- parse_line(io) -> [element, element, element]
"""
from io import StringIO from io import StringIO
import bootstraparse.modules.syntax as syntax import bootstraparse.modules.syntax as syntax
@ -13,10 +12,10 @@ import bootstraparse.modules.syntax as syntax
def parse_line(io): def parse_line(io):
""" """
Takes an io string and returns the parsed output. Takes an io string and returns the parsed output.
:param io: The io string to parse. Parameters:
:type io: StringIO io (StringIO): The io string to parse.
:return: The parsed output. Returns:
:rtype: list[syntax.SemanticType] list[syntax.SemanticType]: The parsed output.
""" """
output = [] output = []
for line in io.readlines(): for line in io.readlines():

View File

@ -1,12 +1,11 @@
""" # Module for resolving relative path problems
Module for resolving relative path problems # PathResolver takes a relative path in the module and outputs the absolute path,
PathResolver takes a relative path in the module and outputs the absolute path, # BoostraPath can deal with relative paths in the installation folder.
BoostraPath can deal with relative paths in the installation folder. # Usage:
Usage: # from bootstraparse.modules.pathresolver import PathResolver, b_path
- from bootstraparse.modules.pathresolver import PathResolver, b_path # b_path("../relative/path/to/file") # returns absolute path to file from the bootstraparse folder
- b_path("../relative/path/to/file") # returns absolute path to file from the bootstraparse folder # path= PathResolver("../relative/path/to/file") # Make a PathResolver object pointing at the first path given
- path= PathResolver("../relative/path/to/file") # Make a PathResolver object pointing at the first path given
"""
import os import os

View File

@ -1,12 +1,11 @@
"""Module for pre-parsing user files in preparation for the parser # Module for pre-parsing user files in preparation for the parser
Usage: # Usage:
- from bootstraparse.modules.preparser import preparser # from bootstraparse.modules.preparser import preparser
- pp = preparser(file, enviroment) # pp = preparser(file, enviroment)
- pp.do_import() # imports all the modules and adds them to the file, do the same for all the files that are to be imported - # noqa # pp.do_import() # imports all the modules and adds them to the file, do the same for all the files that are to be imported # noqa
- pp.do_replacements() # replaces all images and shortcuts in the file # pp.do_replacements() # replaces all images and shortcuts in the file
- pp.readlines() # returns the lines of ORIGINAL file # pp.readlines() # returns the lines of ORIGINAL file
- pp.get_all_lines() # returns the lines of the file after replacements and imports # pp.get_all_lines() # returns the lines of the file after replacements and imports
"""
import os import os

View File

@ -1,14 +1,4 @@
""" # Module for file and directories repartition
Module for file and directories repartition
Usage:
- from bootstraparse.modules import SiteCrawler
- crawler = SiteCrawler(origin_path, dest_path, _env)
- crawler.set_all_preparsers()
- crawler.copy_unparsable_files()
- for element, destination in crawler:
- - save(preparse_parse(element), destination, _env)
"""
import os import os
import shutil import shutil
from bootstraparse.modules import pathresolver, preparser, error_mngr, environment, export from bootstraparse.modules import pathresolver, preparser, error_mngr, environment, export

View File

@ -1,9 +1,4 @@
""" # Module sequencing the successive actions necessary for website building
Module sequencing the successive actions necessary for website building
Usage:
- create_website(origin, destination)
"""
import os import os
from bootstraparse.modules import pathresolver, sitecrawler, environment, config, export, parser, context_mngr from bootstraparse.modules import pathresolver, sitecrawler, environment, config, export, parser, context_mngr

View File

@ -1,4 +1,3 @@
"""
# Dedicated module for the syntax of all the parsing # Dedicated module for the syntax of all the parsing
# All tokens inherit a SemanticType among SemanticType, ExplicitSemanticType and EmptySemanticType # All tokens inherit a SemanticType among SemanticType, ExplicitSemanticType and EmptySemanticType
# All tokens have a label, and __eq__ and __ne__ methods. # All tokens have a label, and __eq__ and __ne__ methods.
@ -9,7 +8,6 @@
# line_to_replace.parse_line('string') # returns a List of tokens parsed for replacements # line_to_replace.parse_line('string') # returns a List of tokens parsed for replacements
# imports.parse_line('string', True) # returns a List of tokens parsed for imports # imports.parse_line('string', True) # returns a List of tokens parsed for imports
# any_token.create_diagram("filename") # Debugging # any_token.create_diagram("filename") # Debugging
"""
import os import os
from itertools import zip_longest from itertools import zip_longest
@ -100,13 +98,21 @@ class SemanticType:
def to_container(self, filter_func=None): def to_container(self, filter_func=None):
""" """
Function called by the context manager to know how to deal with the encapsulation of the token. Function called by the context manager to know how to deal with the encapsulation of the token.
:param filter_func: A function that will be used to filter the content of the token.
:type filter_func: (callable| None)
:return: Self if containable, the expected replacement token if it can, raises an error otherwise.
:rtype: SemanticType | context_mngr.BaseContainer
:raises CannotBeContainedError: Raises error if the token we were trying to encapsulate cannot be (probably mismatched).
"""
Parameters
----------
filter_func: (function | None)
A function that takes a token and returns a boolean. If the token is not valid, it should return False.
Returns
-------
SemanticType | context_mngr.BaseContainer
Self if containable, the expected replacement token if it can, raises an error otherwise.
Raises
------
CannotBeContainedError
Raises error if the token we were trying to encapsulate cannot be (probably mismatched).
"""
raise MismatchedContainerError(self) raise MismatchedContainerError(self)
@ -557,9 +563,10 @@ alias = alias_element + pp.Opt(optional)
# Syntax elements # Syntax elements
line_to_replace = pp.OneOrMore( line_to_replace = pp.OneOrMore(
pp.SkipTo(image)('text').add_parse_action(of_type(TextToken)) + image.add_parse_action(of_type(ImageToken)) | pp.SkipTo(image ^ alias)('text').add_parse_action(of_type(TextToken))
pp.SkipTo(alias)('text').add_parse_action(of_type(TextToken)) + alias.add_parse_action(of_type(AliasToken)) ^ image.add_parse_action(of_type(ImageToken))
) | pp.rest_of_line('text').add_parse_action(of_type(TextToken)) ^ alias.add_parse_action(of_type(AliasToken))
) ^ pp.rest_of_line('text').add_parse_action(of_type(TextToken))
############################################################################## ##############################################################################

View File

@ -1,3 +1 @@
""" # Module for interpreting default or custom templates.
Module for interpreting default or custom templates.
"""

View File

@ -1,13 +1,11 @@
""" ###############################################################################
List of tools for testing # List of tools for testing
========================= ###############################################################################
# Usage:
Usage: # from bootstraparse.modules.tools import __GLk, __GL, find_*_in_file
- from bootstraparse.modules.tools import __GLk, __GL, find_*_in_file # __GLk(n) # Returns a file link n traceback objects back
- __GLk(n) # Returns a file link n traceback objects back # __GL() # Returns a line number at current position
- __GL() # Returns a line number at current position # find_*_in_file(file) # Try to find (class, function or variable) in file and return its line number
- find_*_in_file(file) # Try to find (class, function or variable) in file and return its line number
"""
import inspect import inspect
import os import os

View File

@ -1,11 +1,8 @@
structural_elements: structural_elements:
se_div: ["<div{optionals}>", "</div>"] div: ["<div{optionals}>", "</div>"]
se_article: ["<article{optionals}>", "</article>"] article: ["<article{optionals}>", "</article>"]
se_aside: ["<aside{optionals}>", "</aside>"] aside: ["<aside{optionals}>", "</aside>"]
se_section: ["<section{optionals}>", "</section>"] section: ["<section{optionals}>", "</section>"]
se_nav: ["<nav{optionals}>", "</nav>"]
se_header: ["<header{optionals}>", "</header>"]
se_body: ["<body{optionals}>", "</body>"]
header: ["<h{header_level}{optionals}>", "</h{header_level}>"] header: ["<h{header_level}{optionals}>", "</h{header_level}>"]
display: ["<p class=\"display-{display_level}\"{optionals}>", "</p>"] display: ["<p class=\"display-{display_level}\"{optionals}>", "</p>"]

View File

@ -1,7 +1,5 @@
import os import os
from yaml.scanner import ScannerError
import bootstraparse.modules.config as config import bootstraparse.modules.config as config
import pytest import pytest
import tempfile import tempfile
@ -12,8 +10,7 @@ configs = {
"app/configs/parser_config.yaml": '{"parser": {"name": "test_parser", "type": "test_parser"}}', "app/configs/parser_config.yaml": '{"parser": {"name": "test_parser", "type": "test_parser"}}',
"app/configs/aliases.yaml": '{"parser": {"name": "test_parser", "type": "test_parser"}}', "app/configs/aliases.yaml": '{"parser": {"name": "test_parser", "type": "test_parser"}}',
"user/config/aliases.yaml": '{"aliases": {"test_alias": "test_parser"}}', "user/config/aliases.yaml": '{"aliases": {"test_alias": "test_parser"}}',
"app/configs/glossary.yaml": '{"glossary": {"test_glossary": "test_parser"}}', "user/config/glossary.yaml": '{"glossary": {"test_glossary": "test_parser"}}',
"user/config/glossary.yaml": '{"glossary": {"test_glossary": "second_parser"}}',
"user/config/custom_template.yaml": '{"custom_template": {"test_template": "test_parser"}}', "user/config/custom_template.yaml": '{"custom_template": {"test_template": "test_parser"}}',
} }
bad_configs = { bad_configs = {
@ -39,9 +36,7 @@ def make_false_configs():
def test_config_load(): def test_config_load():
usr_c = config.ConfigLoader(user_conf) usr_c = config.ConfigLoader(user_conf)
assert usr_c["aliases"] == {"aliases": {"test_alias": "test_parser"}} assert usr_c["aliases"] == {"aliases": {"test_alias": "test_parser"}}
assert usr_c["glossary"] == {"glossary": {"test_glossary": "second_parser"}} assert usr_c["glossary"] == {"glossary": {"test_glossary": "test_parser"}}
assert usr_c["glossary"]["glossary"] == {"test_glossary": "second_parser"}
assert usr_c["glossary"]["glossary"]["test_glossary"] == "second_parser"
assert usr_c["aliases"]["aliases"] == {"test_alias": "test_parser"} assert usr_c["aliases"]["aliases"] == {"test_alias": "test_parser"}
assert usr_c["aliases"]["aliases"]["test_alias"] == "test_parser" assert usr_c["aliases"]["aliases"]["test_alias"] == "test_parser"
with pytest.raises(KeyError): with pytest.raises(KeyError):
@ -59,6 +54,7 @@ def test_add_to_config():
assert from_list["aliases"] == {"aliases": {"test_alias": "test_parser"}, assert from_list["aliases"] == {"aliases": {"test_alias": "test_parser"},
"parser": {"name": "test_parser", "type": "test_parser"}} "parser": {"name": "test_parser", "type": "test_parser"}}
assert from_list["parser_config"] == {"parser": {"name": "test_parser", "type": "test_parser"}} assert from_list["parser_config"] == {"parser": {"name": "test_parser", "type": "test_parser"}}
with pytest.raises(KeyError): # Not FileNotFoundError with pytest.raises(KeyError): # Not FileNotFoundError
assert from_empty["aliases"] assert from_empty["aliases"]
from_empty.add_folder(user_conf) from_empty.add_folder(user_conf)
@ -68,7 +64,7 @@ def test_add_to_config():
assert from_empty["aliases"] == {"aliases": {"test_alias": "test_parser"}, assert from_empty["aliases"] == {"aliases": {"test_alias": "test_parser"},
"parser": {"name": "test_parser", "type": "test_parser"}} "parser": {"name": "test_parser", "type": "test_parser"}}
with pytest.raises(ScannerError): with pytest.raises(TypeError):
from_empty.load_from_file(os.path.join(bad_conf, "bad_config.yaml")) from_empty.load_from_file(os.path.join(bad_conf, "bad_config.yaml"))

View File

@ -12,14 +12,12 @@ __templates = config.ConfigLoader(pathresolver.b_path("templates/"))
__XF = pytest.mark.xfail __XF = pytest.mark.xfail
_all_templates = { _all_templates = {
"structural_elements": [ "structural_elements": [
["se_div", __GLk(1)], ["div", __GLk(1)],
["se_article", __GLk(1)], ["article", __GLk(1)],
["se_aside", __GLk(1)], ["aside", __GLk(1)],
["se_section", __GLk(1)], ["section", __GLk(1)],
["header", __GLk(1)], ["header", __GLk(1)],
["display", __GLk(1)], ["display", __GLk(1)],
["se_body", __GLk(1)],
["se_nav", __GLk(1)],
], ],
"inline_elements": [ "inline_elements": [
["link", __GLk(1)], ["link", __GLk(1)],
@ -88,7 +86,7 @@ def test_export_response():
("table", "t_head"), ("table", "t_head"),
("table", "t_row"), ("table", "t_row"),
("table", "t_cell"), ("table", "t_cell"),
("structural_elements", "se_div"), ("structural_elements", "div"),
]) ])
def test_transform(export_type, export_subtype): def test_transform(export_type, export_subtype):
""" """
@ -120,7 +118,7 @@ def test_get_template_error():
def test_with_optionnals(): def test_with_optionnals():
em = export.ExportManager(__config, __templates) em = export.ExportManager(__config, __templates)
em(export.ExportRequest("structural_elements", "se_div", _opts)) # noqa E741 em(export.ExportRequest("structural_elements", "div", _opts)) # noqa E741
def test_format_optionnals(): def test_format_optionnals():
@ -149,7 +147,6 @@ def test_bad_header():
@pytest.mark.parametrize("export_type, export_subtype, line", zipped_templates) @pytest.mark.parametrize("export_type, export_subtype, line", zipped_templates)
def test_get_templates(export_type, export_subtype, line): def test_get_templates(export_type, export_subtype, line):
print(line)
em = export.ExportManager(__config, __templates) em = export.ExportManager(__config, __templates)
# assert type(em.get_templates()) is dict # assert type(em.get_templates()) is dict
print(export_type, export_subtype) print(export_type, export_subtype)

View File

@ -47,7 +47,6 @@ def test_counterpart_container(type_n, expected_counterpart, expected_container,
try: try:
token = type_n(["test"]) # noqa token = type_n(["test"]) # noqa
token.line_number = 1 token.line_number = 1
token.index = 1
assert token.counterpart() == expected_counterpart assert token.counterpart() == expected_counterpart
if expected_container == "error": if expected_container == "error":
with pytest.raises(error_mngr.MismatchedContainerError): with pytest.raises(error_mngr.MismatchedContainerError):