Merge pull request #21 from idle-org/release

Release
This commit is contained in:
Maximilien Rigaut 2022-06-15 20:24:14 +02:00 committed by GitHub
commit d7af2e213a
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
25 changed files with 691 additions and 278 deletions

37
.github/workflows/python-deploy.yml vendored Normal file
View File

@ -0,0 +1,37 @@
# This workflow will install Python dependencies, run tests and lint with a single version of Python
# For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions
name: Python Deployment
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
deployment:
runs-on: ubuntu-latest
environment: production
steps:
- uses: actions/checkout@v2
- name: Set up Python 3.10
uses: actions/setup-python@v2
with:
python-version: "3.10"
- name: Build the application
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
pip install -e .
pip install pydoctor
python -m bootstraparse "example_userfiles" "docs"
- name: Build the documentation
run: |
pydoctor --make-html --html-output=docs/apidoc src/bootstraparse/
- name: deploy
uses: JamesIves/github-pages-deploy-action@v4.3.3
with:
branch: gh-pages # The branch the action should deploy to.
folder: docs

View File

@ -1,13 +1,13 @@
# This workflow will install Python dependencies, run tests and lint with a single version of Python
# For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions
name: Python application
name: Python Tests and Lint
on:
push:
branches: [ main, test ]
branches: [ main, release, releases/*]
pull_request:
branches: [ main, test ]
branches: [ main, release, releases/* ]
jobs:
@ -40,23 +40,3 @@ jobs:
- name: Test with pytest
run: |
pytest --cov-fail-under=100 -o xfail_strict=true --cov-report=term-missing .
deployment:
runs-on: ubuntu-latest
environment: production
steps:
- uses: actions/checkout@v2
- name: Set up Python 3.10
uses: actions/setup-python@v2
with:
python-version: "3.10"
- name: Build the application
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
pip install -e .
python -m bootstraparse "example_userfiles" "docs"
- name: deploy
uses: JamesIves/github-pages-deploy-action@v4.3.3
with:
branch: gh-pages # The branch the action should deploy to.
folder: docs

View File

@ -5,42 +5,49 @@ bootstraparse aims to be customisable in that regard, but its first iteration an
You can refer to the `example_userfiles` for an idea of what the parser expects you to feed it and find an example of our automated generation at https://idle-org.github.io/bootstraparse/ for this particular folder. Or you can run the program yourself for the same result!
[![Python application](https://github.com/idle-org/bootstraparse/actions/workflows/python-app.yml/badge.svg?branch=main)](https://github.com/idle-org/bootstraparse/actions/workflows/python-app.yml)
- Main
- [![Python Main](https://github.com/idle-org/bootstraparse/actions/workflows/python-app.yml/badge.svg?branch=main)](https://github.com/idle-org/bootstraparse/actions/workflows/python-app.yml)
- Releases
- [![Python Release](https://github.com/idle-org/bootstraparse/actions/workflows/python-app.yml/badge.svg?branch=release)](https://github.com/idle-org/bootstraparse/actions/workflows/python-app.yml)
---
## Roadmap
## Release Notes
### V1.0.1
- Make the default example site way more useful
- Remove old regex usage
- Add copy behaviour to config file
- Add the type and return types of all functions
- Add documentation to remaining lone functions
- Write a documentation for the application (use specs.yaml)
### V1.0.2
- Move the WARNING-level output to INFO for config overwrite
### V1.0.3
- Copy un-parsable files to destination folder without modifications
- Add copy behaviour to config file
- Many bug fixes and improvements
## Roadmap
### V1.0.2
- Make the default example site way more useful ☐
- Write a documentation for the application (use specs.yaml) ☐
- Move the WARNING-level output to INFO for config overwrite ☐
### V1.1
- Functioning `table` Token
- Functioning `code` Token
- Establish a list of all configurable parameters to implement in the future, and update the roadmap with them.
- Decide level of logic to be implemented, and whether it should be configurable
- Update the uses of remaining files
- Functioning `table` Token ☐
- Functioning `code` Token ☐
- Establish a list of all configurable parameters to implement in the future, and update the roadmap with them. ☐
- Decide level of logic to be implemented, and whether it should be configurable ☐
- Update the uses of remaining files ☐
### V1.2
- Check error messages and add a real debug level to parameters
- Functioning `blockquote` Token
- Remove old regex usage
- Check error messages and add a real debug level to parameters ☐
- Functioning `blockquote` Token ☐
### V1.3
- Add advanced lookahead logic for `*`
- Functioning `lead` Token
### V1.4
- Add html indentation for human readability of the output.
- Add html indentation for human readability of the output. ☐
### V2
- Add a functioning post-context enhancer able to generate menus and elements from arbitrary logic.
- Achieve perfect markdown compatibility with appropriate config parameters
### V3
- Add Template customisation.

View File

@ -3,27 +3,27 @@
Structural Elements:
<<div
This is a div
div>>[class='blue', 123]{var='test', number=11}{{cinsertion}}
div>>{class='blue', 123}{{cinsertion}}[var='test', number=11]
<<article
This is an article
article>>[class='blue', 123]{var='test', number=11}{{cinsertion}}
article>>{class='blue', 123}{{cinsertion}}[var='test', number=11]
<<aside
This is an aside
aside>>[class='blue', 123]{var='test', number=11}{{cinsertion}}
aside>>{class='blue', 123}{{cinsertion}}[var='test', number=11]
<<section
This is a section
section>>[class='blue', 123]{var='test', number=11}{{cinsertion}}
section>>{class='blue', 123}{{cinsertion}}[var='test', number=11]
<<div
<<article
<<aside
<<section
Text
section>>[class='blue', 123]{var='test', number=11}{{cinsertion}}
aside>>[class='blue', 123]{var='test', number=11}{{cinsertion}}
article>>[class='blue', 123]{var='test', number=11}{{cinsertion}}
div>>[class='blue', 123]{var='test', number=11}{{cinsertion}}
# Header 1 #[class='blue', 123]{var='test', number=11}{{cinsertion}}
! Display 1 ![class='blue', 123]{var='test', number=11}{{cinsertion}}
section>>{class='blue', 123}{{cinsertion}}[var='test', number=11]
aside>>{class='blue', 123}{{cinsertion}}[var='test', number=11]
article>>{class='blue', 123}{{cinsertion}}[var='test', number=11]
div>>{class='blue', 123}{{cinsertion}}[var='test', number=11]
# Header 1 #{class='blue', 123}{{cinsertion}}[var='test', number=11]
! Display 1 !{class='blue', 123}{{cinsertion}}[var='test', number=11]
Inline Element:
@ -38,12 +38,14 @@ __This is underlined__
Oneline elements:
- This is an unordered list item
- This is an unordered list item
- This is an unordered list item[class='blue', 123]{var='test', number=11}{{cinsertion}}
- This is an unordered list item{class='blue', 123}{{cinsertion}}[var='test', number=11]
#. This is an ordered list item
#. This is an ordered list item
#. This is an ordered list item[class='blue', 123]{var='test', number=11}{{cinsertion}}
#. This is an ordered list item{class='blue', 123}{{cinsertion}}[var='test', number=11]
This is a long text segment. This is a long text segment. This is a long text segment. This is a long text segment. This is a long text segment. This is a long text segment. This is a long text segment. This is a long text segment.
I wish there could be linebreaks.
This is a long text segment. This is a long text segment. This is a long text segment. This is a long text segment. This is a long text segment.
This is still the same long text segment. This is still the same long text segment. This is still the same long text segment.
@[end]
Use a double linebreak to force linebreaks.
@[end]

View File

@ -1,7 +1,5 @@
pyyaml==6.0
rich==12.0.1
regex==2022.3.15
pyparse~=1.1.7
pyparsing[diagrams]~=3.0.7
pytest==7.1.1

View File

@ -5,4 +5,3 @@ pytest-cov==3.0.0
mypy==0.941
pyyaml==6.0
rich==12.0.1
regex==2022.3.15

View File

@ -2,7 +2,7 @@
name = bootstraparse
description = A custom markdown parser for Bootstrap & Python
author = Vkyfox
version = 1.0
version = 1.0.1
license = Mozilla Public License Version 2.0
license_file = LICENSE
platforms = unix, linux, osx, cygwin, win32

View File

View File

@ -21,3 +21,4 @@ export:
intermediate_files: true
type: "html"
force_rewrite: true
copy_unparsable_files: "copy" # "copy" or "ignore"

View File

@ -17,16 +17,16 @@ class ConfigLoader:
"""
Reads all config files in a folder
Behaves like a dictionary of all config files
return FileNotFound Error on Error
:param config_folder: The folder to read
"""
def __init__(self, config_folder=None, extensions=("yaml", "yml")):
"""
Defines the config folder and loads all configs
:param config_folder: path to config file
:type config_folder: (str | list[str])
:param extensions: file extensions to load
:return: None
:type extensions: list[str]
:raise TypeError: if config_folder is not a string or a list of strings
"""
if config_folder is None:
self.config_folders = []
@ -54,6 +54,7 @@ class ConfigLoader:
"""
Add a Folder to the list of configs
:param folder: path to folder
:type folder: str
:return: None
"""
self.config_folders.append(folder)
@ -63,6 +64,7 @@ class ConfigLoader:
"""
Loads a config file
:param filepath: path to config file
:type filepath: str
:return: None
"""
basename = os.path.basename(filepath)
@ -81,6 +83,8 @@ class ConfigLoader:
def load_from_folder(self, folder):
"""
Loads all configs in the config folder
:param folder: path to folder
:type folder: str
:return: None
"""
for filename in os.listdir(folder):
@ -92,6 +96,8 @@ class ConfigLoader:
"""
Returns a yaml config object if in self.loaded_conf
:param item: config key
:type item: str
:raise KeyError: if item is not in self.loaded_conf
:return: config value
"""
try:
@ -102,36 +108,21 @@ class ConfigLoader:
def __contains__(self, item):
"""
Same as a dictionary.
:param item: config key
:type item: str
:return: True if item is in self.loaded_conf
"""
return item in self.loaded_conf
def __repr__(self):
"""
Returns config as string
:return: config as string
:return: the dict of configs and their values
:rtype: str
"""
return self.loaded_conf.__repr__()
return str(self.loaded_conf.__repr__())
# class UserConfig(ConfigLoader):
# """
# Reads user config file.
# :param config_folder: path to config file
# :return: None
# """
# def __init__(self, config_folder):
# super().__init__(config_folder)
#
#
# class GlobalConfig(ConfigLoader):
# """
# Reads global config file.
# :param config_folder: path to config file
# :return: None
# """
# def __init__(self, config_folder):
# super().__init__(config_folder)
if __name__ == '__main__': # pragma: no cover
conf = ConfigLoader("../configs/")
conf.add_folder("../../../example_userfiles/config/")

View File

@ -7,7 +7,6 @@
# Usage:
# from bootstraparse.modules.context_mngr import ContextManager
# ctx = ContextManager()
# ctx.???
# container = BaseContainer()
# container[number] -> The number element in the content
# "class_insert" >> container[number] -> Get an element from one of the mapped methods
@ -24,6 +23,14 @@ class BaseContainer:
subtype = None
def __init__(self, content=None, optionals=None, others=None):
"""
:param content: List of tokens to be added to the container
:param optionals: List of optional tokens to be added to the container (from the parser)
:param others: List of values to add to the .format() method
:type content: (list[(syntax.SemanticType | BaseContainer)])
:type optionals: (syntax.SemanticType)
:type others: (dict[str, str])
"""
if content is None:
content = []
if others is None:
@ -34,24 +41,41 @@ class BaseContainer:
self.indentation_level = 0 # future: add an indentation level to every token for human readability of the final output
def get_content(self, exm, arbitrary_list=None):
"""
Get the content of any BaseContainer object (default self.content).
:param exm: ExportManager to use for exporting
:param arbitrary_list: List to get the content from
:type exm: export.ExportManager
:type arbitrary_list: list[BaseContainer]
:rtype : str
"""
output = ""
if not arbitrary_list:
arbitrary_list = self.content
# rich.inspect(arbitrary_list)
for element in arbitrary_list:
if isinstance(element, BaseContainer):
output += element.export(exm) + " "
# rich.inspect(output[:-1])
return output[:-1]
def get_optionals(self):
"""
Get the optionals of the container.
:rtype : (syntax.SemanticType | BaseContainer)
:return: The optionals of the container
"""
return self.optionals
def get_others(self):
"""
Get the other values of the container.
:rtype : dict[str, str]
:return: The other values of the container
"""
return self.others
def export(self, exm):
"""
Export the container to a string.
:type exm: export.ExportManager
:rtype : str
"""
@ -62,15 +86,27 @@ class BaseContainer:
return output
def add(self, other):
"""
Adds a list of elements to the container.
:param other: List of elements to add
:type other: syntax.SemanticType
:return: None
"""
self.content.append(other)
def class_name(self):
"""
A string representation of the class name.
:rtype : str
"""
return self.__class__.__name__
def validate(self, other):
"""
Takes a list as input and checks if every element is in map.
:return: Bool
:param other: List of elements to check
:type other: list[(syntax.SemanticType | BaseContainer)]
:return: True if every element is in map, False otherwise
"""
for o in other:
if o not in self.map:
@ -78,30 +114,68 @@ class BaseContainer:
return True
def debug_map(self):
"""
Prints the map of the container.
:return: None
"""
print(f'Debug for {self.class_name()} <{id(self)}>')
for k, v in self.map.items():
print(f'{k} = {v}')
def __len__(self):
"""
:rtype : int
"""
return len(self.content)
def __iter__(self):
"""
:ytype : iter[syntax.SemanticType]
"""
for content in self.content:
yield content
def __getitem__(self, item):
"""
:param item: Index of the element to get
:type item: int
:rtype : (syntax.SemanticType | BaseContainer)
"""
return self.content[item]
def __setitem__(self, key, value):
"""
:param key: Index of the element to set
:param value: Value to set
:type key: int
:type value: (syntax.SemanticType | BaseContainer)
"""
self.content[key] = value
def __getslice__(self, start, end):
"""
:param start: Start index of the slice
:param end: End index of the slice
:type start: int
:type end: int
:rtype : list[(syntax.SemanticType | BaseContainer)]
"""
return self.content[start:end]
def __rshift__(self, other):
"""
:param other: The name of the map element to get
:type other: str
:rtype : (syntax.SemanticType | BaseContainer)
"""
return self.map[other]()
def __eq__(self, other):
"""
:param other: The other object to compare to
:type other: BaseContainer | list[(syntax.SemanticType | BaseContainer)] | syntax.SemanticType
:rtype : bool
"""
if not hasattr(other, "__len__"):
return False
if len(self) != len(other):
@ -123,13 +197,23 @@ class BaseContainer:
return True
def __invert__(self):
"""
Returns the inverted optionals of the container.
:rtype : BaseContainer
"""
return self.get_optionals()
def __str__(self):
"""
:rtype : str
"""
return '{} ({})'.format(", ".join(map(str, self.content)),
", ".join(map(str, self.get_optionals())) if self.get_optionals() else "")
def __repr__(self):
"""
:rtype : str
"""
representation = {}
for content in self.content:
if type(content) in representation:
@ -146,6 +230,9 @@ class BaseContainer:
)
def print_all(self, ident=''):
"""
Prints the container and all its content.
"""
print(ident + self.class_name())
for e in self:
if isinstance(e, BaseContainer):
@ -154,8 +241,12 @@ class BaseContainer:
print(ident + "> " + str(e))
print("")
def to_container(self, filter_func=None): # idea: additional parameters for encapsulation
return self
def to_container(self, filter_func=None): # noqa : C901
"""
Returns a container with the same content as the current container.
Used for compatibility with the syntax for encapsulation.
"""
return self # idea: additional parameters for encapsulation
# Define containers all the Enhanced text elements, divs, headers, list and any element that can be a container
@ -194,7 +285,7 @@ class EtCustomSpanContainer(BaseContainer):
type = "inline_elements"
def export(self, exm):
self.subtype = "custom_" + self.content[0].content[0]
self.subtype = "custom_" + self.content[0].content[0] # noqa F821 (self.content[0] is a token, by definition
return super().export(exm)
@ -253,7 +344,7 @@ class HeaderContainer(BaseContainer):
subtype = "header"
def export(self, exm):
self.others = {}
self.others = {} # noqa F821
self.others["header_level"] = len(self.content[0].content[0])
return super().export(exm)
@ -349,6 +440,10 @@ class ContextManager:
----------
parsed_list : list[syntax.SemanticType]
List of parsed tokens output by our parser.
name : str
Name of the file being parsed.
ident : int
Number of spaces to indent the output.
"""
self.output = []
self.parsed_list = parsed_list
@ -423,11 +518,30 @@ class ContextManager:
# print(" <<< Pile:", self.pile)
def _add_matched(self, label, index):
"""
Method to add a matched element to the dictionary of matched elements.
Parameters
----------
label : str
Label of the matched element.
index : int
Index of the matched element in the pile.
"""
if label not in self.matched_elements:
self.matched_elements[label] = []
self.matched_elements[label] += [index]
def _get_matched(self, label):
"""
Method to get the index of the last matched element of a given label.
Parameters
----------
label : str
Label of the matched element.
Returns
-------
int : Index of the last matched element of the given label.
"""
return self.matched_elements[label].pop()
def __call__(self):
@ -438,6 +552,10 @@ class ContextManager:
-------
list[BaseContainer]
Returns the pile entirely processed as a list of containers.
Raise
-----
MismatchedContainerError
Raised when a container is not matched to its corresponding token.
"""
# self.pile = self.parsed_list.copy()
if self.contextualised:
@ -497,6 +615,10 @@ class ContextManager:
return self.finalize_pile()
def __iter__(self):
"""
Iterator over the pile.
:ytype: BaseContainer
"""
for e in self.pile:
if e:
yield e
@ -505,6 +627,7 @@ class ContextManager:
"""
Function for cleaning up of the pile after full contextualisation.
Removes Nones and checks for any errors or illogical containers.
:return: list[BaseContainer]
"""
final_pile = []
for p in self.pile:
@ -532,6 +655,15 @@ class ContextManager:
def lookahead(self, token, index):
"""
Iterates the pile beginning from index and looks for all tokens matching labels with token.
Parameters
----------
token : syntax.BaseContainer
Token to look for.
index : int
Index of the token in the pile.
Returns
-------
(int, int) : (number of tokens matched, number of linebreaks)
"""
range_to_encapsulate = 0
line_skipped = 0
@ -557,18 +689,31 @@ class ContextManager:
return range_to_encapsulate, line_skipped
def recontext(self, token):
"""
Function to recontextualise the content of a token.
Parameters
----------
token : syntax.SemanticType
Token to recontextualise.
"""
token.content = ContextManager(token.content, name=self.name)()
def get_last_container_in_pile(self, index):
"""
Returns the last element in the pile if it is a Container, raises an error otherwise.
Return
------
Parameters
----------
index : int
Index of the element to check.
Returns
-------
BaseContainer
Raises
------
error_mngr.LonelyOptionalError
Raised when an element is found in the pile that is not encapsulated in a container.
"""
i = index-1 # skip last token as it is self
while i >= 0:
@ -581,6 +726,9 @@ class ContextManager:
log_exception(LonelyOptionalError(self.pile[index], None), level="CRITICAL")
def print_all(self):
"""
Prints the pile.
"""
for e in self:
if isinstance(e, BaseContainer):
e.print_all()

View File

@ -26,6 +26,10 @@ class Environment:
"""
def __init__(self):
"""
Initialises the environment, all parameters are set to None.
Set _wasInitialised to False for all parameters.
"""
# mandatoryParameters
self._mParams = {
'config': None,
@ -60,8 +64,10 @@ class Environment:
def __getattr__(self, attribute):
"""
Getter for all parameters.
Getter for all parameters, looks for super if the parameter is prefaced with an underscore.
Looks for mandatory parameters first, then secondary parameters.
:param attribute: The name of the parameter to get.
:type attribute: str
:return: The value of the parameter.
"""
if attribute[0] == "_":
@ -80,8 +86,9 @@ class Environment:
def __setattr__(self, attribute, value):
"""
Setter for all parameters.
Setter for all parameters, looks for super if the parameter is prefaced with an underscore.
:param attribute: The name of the parameter to set.
:type attribute: str
"""
if attribute[0] == "_":
super().__setattr__(attribute, value)

View File

@ -15,17 +15,19 @@ from bootstraparse.modules.tools import __GLk # __GFi, __GFu, __GL
# Define the error codes
_ERRORS = ["ParsingError", "MismatchedContainerError"]
__all__ = _ERRORS+[]
def init_logging(filename=None, loglevel="ERROR", filemode='w', handler=None):
"""
Initializes logging
:param filename: The name of the log file
:param loglevel: The level of logging
:type loglevel: str
:param filemode: The mode of the log file
:param filename: The path for the log file
:param loglevel: The level of logging: "ERROR", "INFO", "WARNING", "DEBUG", "CRITICAL"
:param filemode: The mode of the log file: "w", "a"
:param handler: The handler to use
:type filename: str
:type loglevel: str
:type filemode: str
:type handler: logging.Handler
:return: None
"""
loglevel = loglevel.upper()
@ -50,6 +52,7 @@ def log_message(message, level="ERROR"):
:param message: The message to log
:param level: The level of the message
:type level: str
:type message: str
:return: None
"""
level = level.lower()
@ -84,24 +87,30 @@ def log_exception(exception, level="ERROR"):
class BootstraparseError(Exception):
"""
Base class for all Bootstraparse errors
"""
pass
class ParsingError(Exception):
"""
Exception class for parsing errors
:param message: The message to display
:type message: str
:param line: The line number of the error
:param column: The column number of the error
:param file: The name of the file
:return: None
"""
def __init__(self, message=None, line=None, column=None, file=None):
"""
Initializes the ParsingError class with the filename and a line and column number
based on the current position in the file (if available)
:param message: The message to display
:type message: str
:param line: The line number of the error
:type line: int
:param column: The column number of the error
:type column: int
:param file: The name of the file
:type file: str
:return: None
"""
super().__init__(message)
self.line = line
@ -127,6 +136,11 @@ class MismatchedContainerError(BootstraparseError):
The token is not final and cannot be contained. Indicative of a mismatched token.
"""
def __init__(self, token):
"""
Initializes the MismatchedContainerError class with the token that was not final
:param token: The token that is not final
:type token: syntax.SemanticType
"""
self.token = token
if token:
self.line = token.line_number
@ -144,6 +158,14 @@ class LonelyOptionalError(BootstraparseError):
Optional token could not be matched with last element in pile (not a container).
"""
def __init__(self, token, last_in_pile):
"""
Initializes the LonelyOptionalError class with the token found and the last element in the pile
Used when a token is found that is optional and cannot be matched with the last element in the pile (not a container)
:param token: The token that is not final
:type token: syntax.SemanticType
:param last_in_pile: The last element in the pile
:type last_in_pile: syntax.SemanticType
"""
self.token = token
if token:
self.line = token.line_number

View File

@ -5,8 +5,6 @@
# rsp = ExportResponse("start_string", "end_string")
# em = ExportManager(config_file, template_file)
# em(ExportRequest()) -> ExportResponse()
# import rich
import rich
from io import StringIO
from bootstraparse.modules import config, pathresolver, error_mngr, context_mngr
@ -48,7 +46,14 @@ class ExportManager:
"""
Transforms ExportRequest tuples to ExportResponse tuples with the config-provided appropriate markup.
"""
def __init__(self, cnoifg, templates): # noqa
def __init__(self, cnoifg, templates):
"""
Initialization function for the ExportManager.
:param cnoifg: Config file
:type cnoifg: config.ConfigLoader
:param templates: Templates file
:type templates: config.ConfigLoader
"""
self.config = cnoifg
self.templates = templates
self.advanced_export = {
@ -62,6 +67,12 @@ class ExportManager:
}
def __call__(self, export_request):
"""
Callable function for ExportManager, transforming ExportRequest tuples into ExportResponse tuples.
:param export_request: ExportRequest tuples
:type export_request: ExportRequest
:return: ExportResponse tuples
"""
return self.transform(export_request)
def transform(self, export_request):
@ -103,7 +114,6 @@ class ExportManager:
except KeyError:
log_entries = ["bootstrap", export_request.type, export_request.subtype]
log_ = tools.dict_check(self.templates, *log_entries)
print()
error_mngr.log_exception(
KeyError(
f'Template "bootstrap"/{export_request.type}/{export_request.subtype} could not be found.\n' +
@ -152,6 +162,7 @@ class ExportManager:
"""
Specific template for display.
:type export_request: ExportRequest
:rtype: ExportResponse
"""
start, end, optionals = self._get_template(export_request)
start = start.format(optionals=optionals, display_level=export_request.others["display_level"])
@ -161,6 +172,7 @@ class ExportManager:
"""
Specific template for link.
:type export_request: ExportRequest
:rtype: ExportResponse
"""
start, end, _ = self._get_template(export_request)
start = start.format(url=export_request.others["url"])
@ -170,6 +182,7 @@ class ExportManager:
"""
Specific template for table content.
:type export_request: ExportRequest
:rtype: ExportResponse
"""
start, end, _ = self._get_template(export_request)
start = start.format(col_span=export_request.others["col_span"])
@ -179,6 +192,7 @@ class ExportManager:
"""
Specific template for images.
:type export_request: ExportRequest
:rtype: ExportResponse
"""
start, end, optionals = self._get_template(export_request)
end = end.format(optionals=optionals)
@ -198,6 +212,8 @@ class ContextConverter:
pile output from the context manager.
exporter : ExportManager
Our ExportManager
destination : str
Destination of the output file.
"""
self.io_output = StringIO()
self.pile = pile
@ -207,7 +223,8 @@ class ContextConverter:
def process_pile(self):
"""
Main method processing the given pile.
Processes the pile and writes the output to the io_output object.
:rtype: StringIO
"""
for container in self.pile:
self.io_output.write(container.export(self.exporter))
@ -225,6 +242,7 @@ class ContextConverter:
def readlines(self):
"""
Yields the converted pile line after line
:rtype: list[str]
"""
return self.io_output.readlines()
@ -233,6 +251,9 @@ class ContextConverter:
f"Status: {'initialized' if self.io_initialized else 'uninitialized'}"
def print_all(self):
"""
Prints the entirety of the converted pile
"""
print(self)
def __eq__(self, other):
@ -240,10 +261,11 @@ class ContextConverter:
if __name__ == '__main__': # pragma: no cover
import rich
from bootstraparse.modules import parser
io_string = StringIO(
"""[this is a link]('http://te.st')"""
"""! This is a test ! {{fgd}}"""
)
test = parser.parse_line(io_string)
__config = config.ConfigLoader(pathresolver.b_path("configs/"))

View File

@ -12,16 +12,21 @@ import os
class PathResolver:
"""
Initialised with a relative path and returns the translated absolute path
:param base_path: Relative path to be resolved
"""
def __init__(self, base_path):
"""
Initialises the PathResolver with the base path
:param base_path: Base path
:type base_path: str
"""
self.base_path = os.path.normpath(os.path.join(os.getcwd(), os.path.dirname(base_path)))
def __call__(self, relative_path='.'):
"""
Returns the absolute path of the relative path
:param relative_path: Relative path to be resolved
:type relative_path: str
:return: Absolute path
"""
return self.give_absolute(relative_path)
@ -30,6 +35,7 @@ class PathResolver:
"""
Returns the absolute path of the relative path
:param relative_path: Relative path to be resolved
:type relative_path: str
:return: Absolute path
"""
return os.path.normpath(os.path.join(self.base_path, relative_path))

View File

@ -9,13 +9,12 @@
import os
import regex # future: remove regex
from io import StringIO
from bootstraparse.modules import pathresolver as pr
from bootstraparse.modules import environment
from bootstraparse.modules import syntax
from bootstraparse.modules import error_mngr # noqa # pylint: disable=unused-import
from bootstraparse.modules import error_mngr
from bootstraparse.modules import export
import rich
@ -37,6 +36,10 @@ class PreParser:
:param _env: the environment object
:param list_of_paths: the list of files that have been imported in this branch of the import tree
:param dict_of_imports: Dictionary of all imports made to avoid duplicate file opening / pre-parsing
:type file_path: str
:type _env: environment.Environment
:type list_of_paths: list[str]
:type dict_of_imports: dict[str, PreParser]
"""
if list_of_paths is None:
list_of_paths = []
@ -75,7 +78,6 @@ class PreParser:
def make_temporary_files(self):
"""
Creates temporary files for the import list and the replacement.
:return: None
"""
self.file_with_all_imports = StringIO()
self.file_with_all_replacements = StringIO()
@ -86,13 +88,14 @@ class PreParser:
"""
Make new temporary files, if the old one are not referenced,
the garbage collector will delete them.
:return: None
"""
self.make_temporary_files()
def do_imports(self):
"""
Execute all actions needed to do the imports and setup for the next step
:return: The file descriptor of the file with all imports
:rtype: StringIO
"""
self.parse_import_list()
self.make_import_list()
@ -103,6 +106,8 @@ class PreParser:
def do_replacements(self):
"""
Execute all actions needed to do the replacements.
:return: The file descriptor of the file with all replacements
:rtype: StringIO
"""
self.do_imports()
self.parse_shortcuts_and_images()
@ -112,7 +117,8 @@ class PreParser:
def readlines(self):
"""
Reads the original file and returns a list of lines.
:return: A list of lines
:return: a list of lines
:rtype: list[str]
"""
with open(self.relative_path_resolver(self.name), 'r') as f:
return f.readlines()
@ -121,6 +127,7 @@ class PreParser:
"""
Get the lines from the file on the step you are in
:return: A list of lines
:rtype: list[str]
"""
if self.current_origin_for_read is None:
return self.readlines()
@ -133,6 +140,7 @@ class PreParser:
Makes sure that the files are not already imported through a previous import statement.
Recursively build a list of PreParser object for each file to be imported.
:return: the dictionary of all files to be imported (key: file name, value: PreParser object)
:rtype: dict[str, PreParser]
"""
import_list = self.parse_import_list()
if self.is_global_dict_of_imports_initialized:
@ -144,8 +152,7 @@ class PreParser:
RecursionError(f"Error: {e} was imported earlier in {self.list_of_paths}"), level='CRITICAL'
)
if e in self.global_dict_of_imports:
pp = self.global_dict_of_imports[e]
self.local_dict_of_imports[e] = pp
self.local_dict_of_imports[e] = self.global_dict_of_imports[e]
else:
try:
pp = PreParser(e, self._env, self.list_of_paths.copy(), self.global_dict_of_imports)
@ -162,7 +169,8 @@ class PreParser:
def parse_import_list(self):
"""
Parses the import list of the file.
:return: a list of files to be imported
:return: a list of files to be imported, and the line number of the import statement
:rtype: list[(str, int)]
"""
if self.saved_import_list:
return self.saved_import_list
@ -170,10 +178,10 @@ class PreParser:
line_count = 0
for line in self.readlines():
results = regex.match(_rgx_import_file, line)
results = _rgx_import_file.searchString(line)
if results:
for e in results.captures('file_name'):
import_list += [(e, line_count)]
for e in results[0]:
import_list += [(e.rstrip(), line_count)]
line_count += 1
# converts relative paths to absolute and returns a table
self.saved_import_list = [(self.relative_path_resolver(p), l) for p, l in import_list]
@ -183,6 +191,7 @@ class PreParser:
"""
Return the file object with all file imports done
:return: a filelike object with all file imports done
:rtype: StringIO
"""
self.make_import_list()
@ -202,9 +211,9 @@ class PreParser:
import_list = self.parse_import_list()
source_lines = self.readlines()
for import_path, import_line in import_list:
source_lines[import_line] = ""
temp_file.writelines(source_lines[source_line_count:import_line])
source_line_count = import_line
source_lines[import_line] = "" # remove the line where the import was
temp_file.writelines(source_lines[source_line_count:import_line]) # copy origin to destination
source_line_count = import_line # update origin for next import
import_file = self.global_dict_of_imports[import_path].export_with_imports()
temp_file.writelines(import_file.readlines())
temp_file.writelines(source_lines[source_line_count:])
@ -217,7 +226,8 @@ class PreParser:
"""
Parses through the output files from export_with_imports
and replaces shortcuts and images calls with appropriate html
:return: a table of occurrences and their lines
:return: The file descriptor of the file with all replacements
:rtype: StringIO
"""
temp_file = self.file_with_all_imports
temp_text = ''
@ -238,7 +248,10 @@ class PreParser:
def get_element_from_config(self, *list_keys):
"""
Fetches an element from the config
Fetches an element from the config (a nested dictionary) going through the list of keys
Raises an error if the element is not found
:param list_keys: the list of keys to go through the config
:type list_keys: str
"""
sub_dict = self._env.config.loaded_conf
validated_elements = []
@ -258,6 +271,12 @@ class PreParser:
def make_replacements(message, *var_list, **var_dict):
"""
Tries to format the message or give an error message and return the original message
:param message: the message to format
:type message: str
:param var_list: the list of variables to replace in the message
:type var_list: str
:keyword var_dict: the dictionary of variables to replace in the message
:type var_dict: dict[str, str]
"""
try:
return message.format(*var_list, **var_dict)
@ -271,10 +290,13 @@ class PreParser:
def get_alias_from_config(self, shortcut, optionals):
"""
Fetches shortcut paths from aliases.yaml
:return: the html to insert as a string
Returns the alias from the config and makes the replacements with the options provided
:param shortcut: id of the shortcut to fetch
:type shortcut: str
:param optionals: optional parameters along with alias
:type optionals: syntax.Optional
:return: the html to insert as a string
:rtype: str
"""
output = self.get_element_from_config('aliases', 'shortcuts', shortcut)
_, _, var_list, var_dict = syntax.split_optionals(optionals)
@ -283,10 +305,12 @@ class PreParser:
def get_image_from_config(self, shortcut, optionals):
"""
Fetches image paths from aliases.yaml
:return: html to insert as a string
:param shortcut: the id of the picture to fetch
Returns the image from the config and makes the replacements with the options provided
:param shortcut: id of the image to fetch
:type shortcut: str
:param optionals: optional parameters along with image
:type optionals: syntax.Optional
:return: the html to insert as a string
"""
shortcut_s = self.get_element_from_config('aliases', 'images', shortcut)
@ -299,7 +323,6 @@ class PreParser:
def __repr__(self):
"""
Returns a string representation of the PreParser object.
:return: a string representation of the PreParser object
"""
return "PreParser[{}](path={}, name={}, base_path={}, " \
"relative_path_resolver={}, list_of_paths={}, " \
@ -311,7 +334,6 @@ class PreParser:
def __str__(self):
"""
Returns a string representation of the PreParser object.
:return: a string representation of the PreParser object
"""
return self.__repr__()
@ -334,6 +356,7 @@ class PreParser:
"""
Checks if the PreParser object is not equal to another PreParser object.
:param other: the other PreParser object
:type other: PreParser
:return: True if the PreParser objects are not equal, False otherwise
"""
return not self.__eq__(other)
@ -341,6 +364,14 @@ class PreParser:
def rich_tree(self, prefix="", suffix="", force=True, strip_prefix=""):
"""
Returns a rich representation of the PreParser object.
:param prefix: the prefix to add to the representation
:type prefix: str
:param suffix: the suffix to add to the representation
:type suffix: str
:param force: if True, the representation is always computed, even if it has already been done
:type force: bool
:param strip_prefix: the prefix to remove from the path
:type strip_prefix: str
:return: a rich representation of the PreParser object
"""
unparsed = False

View File

@ -1,10 +1,26 @@
# Module for file and directories repartition
import os
import shutil
from bootstraparse.modules import pathresolver, preparser, error_mngr, environment, export
class SiteCrawler:
"""
A sitecrawler is a generator that yields a tuple of the form (PreParser, file)
Its goal is to create a list of preparsers and files to be parsed based on
the files and directories in the initial path.
This generator is to be used in a for loop to parse all the files and produce
the final website.
"""
def __init__(self, path, destination, _env):
"""
:param path: The path to the directory to be crawled
:param destination: The path to the directory where the website will be created
:param _env: The environment object
:type path: str
:type destination: str
:type _env: environment.Environment
"""
if not os.path.exists(path):
error_mngr.log_exception(
FileNotFoundError(f'The specified path "{path}" could not be found.'),
@ -20,6 +36,7 @@ class SiteCrawler:
self.force_rewrite = self._env.config["parser_config"]["export"]["force_rewrite"]
self.directories = []
self.files = []
self.files_to_copy = []
self.preparsers = []
self.global_dict_of_imports = {}
@ -32,26 +49,48 @@ class SiteCrawler:
self.create_all_paths()
def get_all_paths(self):
self.files, self.directories = self.list_recursively(self.initial_path, self.initial_path)
"""
This method is used to get all the paths to be crawled.
Its goal is to get all the paths to be crawled and to store them in
the self.directories and self.files, self.files_to_copy variables.
"""
self.files, self.files_to_copy, self.directories = self.list_recursively(self.initial_path, self.initial_path)
def list_recursively(self, path, root):
"""
Recursive method to get all the paths to be crawled.
:param path: The current path to be crawled
:param root: The root path (initial path)
:type path: str
:type root: str
:return: A tuple of the form (files, directories)
:rtype: (list[(str, str)], list[(str, str)], list[(str, str)])
"""
files = []
directories = []
files_to_copy = []
for element in os.listdir(path):
element_fpath = os.path.join(path, element)
element_rpath = os.path.relpath(path, root)
if os.path.isdir(element_fpath):
if element not in self.forbidden_folders:
directories.append((element_rpath, element))
f, d = self.list_recursively(element_fpath, root)
f, fc, d = self.list_recursively(element_fpath, root)
files += f
directories += d
files_to_copy += fc
else:
if os.path.splitext(element)[1] in self.authorised_extensions and element[0] != "_":
files.append((element_rpath, element))
return files, directories
if element[0] != "_":
if os.path.splitext(element)[1] in self.authorised_extensions:
files.append((element_rpath, element))
else:
files_to_copy.append((element_rpath, element))
return files, files_to_copy, directories
def create_all_paths(self):
"""
This method is used to create all the directories in the destination path.
"""
if not os.path.exists(self.destination_path):
os.mkdir(self.destination_path)
for root, di in self.directories:
@ -60,6 +99,12 @@ class SiteCrawler:
os.mkdir(temp_path)
def set_all_preparsers(self):
"""
This method is used to set all the preparsers and initialize them.
The preparsers are stored in the self.preparsers variable.
:return: self.preparsers
:rtype: list[preparser.PreParser]
"""
for root, file in self.files:
preparser_path = os.path.join(self.initial_path, root, file)
pp = preparser.PreParser(preparser_path, self._env, dict_of_imports=self.global_dict_of_imports)
@ -69,7 +114,21 @@ class SiteCrawler:
return self.preparsers
def copy_unparsable_files(self):
"""
This method is used to copy all the files that could not be parsed.
"""
if self._env.config["parser_config"]["export"]["copy_unparsable_files"].lower() == "copy":
for root, file in self.files_to_copy:
shutil.copy(os.path.join(self.initial_path, root, file), os.path.join(self.destination_path, root, file))
def create_file(self, path):
"""
This method is used to create a file.
:raises FileExistsError: If the file already exists and the force_rewrite option is False
:param path: The path to the file to be created
:type path: str
"""
if os.path.exists(path):
if not self.force_rewrite:
error_mngr.log_exception(
@ -81,6 +140,11 @@ class SiteCrawler:
return path
def __iter__(self):
"""
This method is used to iterate over the preparsers and files.
:yield: A tuple of the form (PreParser, file)
:ytype: (preparser.PreParser, str)
"""
for pre in self.preparsers:
yield pre

View File

@ -8,10 +8,16 @@ def create_website(origin, destination):
"""
First function called by bparse.py,
calls all other modules in the right order.
:param origin: The path of the website to be built.
:param destination: The destination path of the built website.
:type origin: str
:type destination: str
:return: 0 if everything went well, 1 otherwise.
"""
env = create_environment(origin, destination)
crwlr = create_crawler(origin, destination, env)
crwlr.set_all_preparsers()
crwlr.copy_unparsable_files()
for element, destination in crwlr:
save(preparse_parse(element), destination, env)
@ -22,6 +28,12 @@ def create_environment(origin, destination):
"""
Returns parserEnvironment as an object containing
everything needed for app execution.
:param origin: The path of the website to be built.
:param destination: The destination path of the built website.
:type origin: str
:type destination: str
:return: Environment object.
:rtype: environment.Environment
"""
env = environment.Environment()
env.config = config.ConfigLoader(pathresolver.b_path("configs"))
@ -42,12 +54,26 @@ def create_environment(origin, destination):
def create_crawler(origin, destination, _env):
"""
Returns crawler as an object for navigation in the user files.
rtype: sitecrawler.SiteCrawler
:param origin: The path of the website to be built.
:param destination: The destination path of the built website.
:param _env: The environment object.
:type origin: str
:type destination: str
:type _env: environment.Environment
:return: Crawler object.
:rtype: sitecrawler.SiteCrawler
"""
return sitecrawler.SiteCrawler(origin, destination, _env)
def preparse_parse(preparser):
"""
Returns a list of containers from a preparser.
:param preparser: The preparser object.
:type preparser: parser.Preparser
:return: List of containers.
:rtype: list
"""
io = preparser.do_replacements()
parsed_list = parser.parse_line(io)
output = context_mngr.ContextManager(parsed_list, name=preparser.name)()
@ -55,6 +81,15 @@ def preparse_parse(preparser):
def save(list_of_containers, destination, env):
"""
Saves the list of containers in the destination path.
:param list_of_containers: The list of containers to be saved.
:param destination: The destination path.
:param env: The environment object.
:type list_of_containers: list
:type destination: str
:type env: environment.Environment
"""
with open(destination, "w") as output_file:
output_file.write(export.ContextConverter(list_of_containers, env.export_mngr, destination).process_pile().read())

View File

@ -16,7 +16,6 @@ from collections import namedtuple
from bootstraparse.modules.error_mngr import MismatchedContainerError
from bootstraparse.modules import context_mngr as cm
import pyparsing as pp
import regex # future: remove regex
pps = pp.Suppress
@ -29,6 +28,11 @@ class SemanticType:
label = None
def __init__(self, content):
"""
Initialises the token.
:param content: The content of the token.
:type content: list[pp.ParseResults]
"""
self.content = content
self.line_number = "Undefined"
self.file_name = "Undefined"
@ -37,6 +41,8 @@ class SemanticType:
def to_markup(self):
"""
Function used for testing and readability purposes. Replaces matched markup with html-like tags.
:return: The markup of the token.
:rtype: str
"""
if self.content:
return_list = []
@ -51,6 +57,8 @@ class SemanticType:
def to_original(self):
"""
Return the original string of the token.
:return: The original string of the token.
:rtype: str
"""
return " ".join(self.content)
@ -79,6 +87,12 @@ class SemanticType:
return not self.__eq__(other)
def counterpart(self): # noqa # FUTURE: Check for static counterpart implementation
"""
Function called by the context manager to know how to deal with the encapsulation of the token.
Will return the counterpart of the token (the previous token to look for).
:return: The counterpart of the token.
:rtype: str | None
"""
return None
def to_container(self, filter_func=None):
@ -92,7 +106,7 @@ class SemanticType:
Returns
-------
SemanticType
SemanticType | context_mngr.BaseContainer
Self if containable, the expected replacement token if it can, raises an error otherwise.
Raises
------
@ -102,13 +116,19 @@ class SemanticType:
raise MismatchedContainerError(self)
class TokensToMatch:
"""Class used for verification in the context manager.
Classes inheriting this should be added to matched_elements."""
class TokensToMatch(SemanticType):
"""
Class used for verification in the context manager.
Classes inheriting this should be added to matched_elements.
"""
pass
class AddFirstElementToLabel(SemanticType):
"""
Class used for verification in the context manager.
Adds the first element of the content to the label (for matching purposes).
"""
def __init__(self, content):
super().__init__(content)
self.label += ":" + content[0]
@ -247,12 +267,12 @@ class DisplayToken(FinalSemanticType):
class StructuralElementStartToken(AddFirstElementToLabel, OpenedSemanticType, TokensToMatch):
"""<<div|article|section|aside"""
"""<<div|article|section|aside|header|body|nav"""
label = 'se:start'
class StructuralElementEndToken(AddFirstElementToLabel, ClosedSemanticType):
"""div|article|section|aside>>"""
"""div|article|section|aside|header|body|nav>>"""
label = "se:end"
def counterpart(self):
@ -330,7 +350,9 @@ def of_type(token_class):
"""
Function creating a custom function for generating the given Token type.
:param token_class: SemanticType class
:type token_class: type
:return: returns a function creating an instance of the given class
:rtype: function
"""
def _of_type(_, __, content):
if len(content) == 0: # Drop Empty token
@ -344,7 +366,9 @@ def reparse(parse_element):
"""
Creates a function which reparses given match with specified parsing element.
:param parse_element: pp object defining desired match
:type parse_element: pp.ParserElement
:return: returns a function parsing given match with specified parsing element
:rtype: function
"""
def _reparse(__, _, tokens):
return parse_element.parseString(tokens[0]) # Future: document this behaviour more in-depth
@ -392,6 +416,7 @@ def readable_markup(list_of_tokens):
"""
Function used for testing and readability purposes. Replaces matched markup with html-like tags.
:param list_of_tokens: list of matched markup tokens in analysed text.
:type list_of_tokens: list
:return: returns readable string.
:rtype: string
"""
@ -408,8 +433,7 @@ def readable_markup(list_of_tokens):
# Pre-parser expressions
rgx_import_file = regex.compile(r'::( ?\< ?(?P<file_name>[\w\-._/]+) ?\>[ \s]*)+')
rgx_import_file = pps("::") + pp.OneOrMore(pps("<") + pp.SkipTo(">").set_name("file_name")("file_name") + pps(">"))
# Base elements
quotes = pp.Word(r""""'""")
@ -446,7 +470,10 @@ structural_elements = (
pp.CaselessLiteral('div') |
pp.CaselessLiteral('article') |
pp.CaselessLiteral('aside') |
pp.CaselessLiteral('section')
pp.CaselessLiteral('section') |
pp.CaselessLiteral('header') |
pp.CaselessLiteral('body') |
pp.CaselessLiteral('nav')
)('structural_element')
header_element = pp.Word('#')
display_element = pp.Word('!')
@ -501,22 +528,20 @@ multi_line = se | table | quotation
one_header = (
header_element +
pp.SkipTo(pp.match_previous_literal(header_element)) +
pps(pp.match_previous_literal(header_element)) +
pp.Opt(optional)
).add_parse_action(of_type(HeaderToken))
pps(pp.match_previous_literal(header_element))
).add_parse_action(of_type(HeaderToken)) + pp.Opt(optional)
one_display = (
display_element +
pp.SkipTo(pp.match_previous_literal(display_element)) +
pps(pp.match_previous_literal(display_element)) +
pp.Opt(optional)
).add_parse_action(of_type(DisplayToken))
pps(pp.match_previous_literal(display_element))
).add_parse_action(of_type(DisplayToken)) + pp.Opt(optional)
one_olist = pp.line_start + (
pps(pp.Literal('#.')) + (
pps(pp.Combine(pp.line_start + pp.Literal('#.'))) + (
(pp.SkipTo(optional)('text').add_parse_action(reparse(enhanced_text)) + optional) |
enhanced_text)
).add_parse_action(of_type(EtOlistToken))
one_ulist = pp.line_start + (
pps(pp.Literal('-')) + (
pps(pp.Combine(pp.line_start + pp.Literal('-'))) + (
(pp.SkipTo(optional)('text').add_parse_action(reparse(enhanced_text)) + optional) |
enhanced_text)
).add_parse_action(of_type(EtUlistToken))

View File

@ -15,11 +15,14 @@ import re
###############################################################################
# Dictionary checks
########################################
def dict_check(dic, *args):
"""
Helper function to test existence within a dictionary.
Helper function to test existence of a list of succesive keys within a nested dictionary.
:param dic: The dictionary of dictionaries to check
:param args: Any number of keys to check the dictionaries for
:type dic: dict | config.ConfigLoader
:type args: str
:return: A table of Booleans for every key checked.
"""
output = [False for _ in args]
@ -34,8 +37,15 @@ def dict_check(dic, *args):
def str_dict_check(dic, *args):
"""
Helper function to test existence within a dictionary and return a string of the values.
Helper function to test existence within a dictionary and return a string of the values of the keys checked
and wether they exist.
'arg[0]' : True or False
:param dic: The nested dictionary to check
:param args: Any number of keys to check successively
:type dic: dict
:type args: str
:return: A string of the values of the keys checked and wether they exist.
:rtype: str
"""
out = dict_check(dic, *args)
return "\n".join([f"{args[i]}: {out[i]}" for i in range(len(out))])

View File

@ -64,6 +64,21 @@ _token_list_with_expected_result = [
],
__GLk(1),
],
[
[
sy.StructuralElementStartToken(["header"]),
sy.TextToken(["1"]),
sy.StructuralElementEndToken(["header"]),
],
[
context_mngr.SeContainer([
sy.StructuralElementStartToken(["header"]),
context_mngr.TextContainer([sy.TextToken(["1"])]),
sy.StructuralElementEndToken(["header"])
]),
],
__GLk(1),
],
[
[
sy.EtEmToken(["*"]),

View File

@ -246,7 +246,7 @@ def test_make_import_list(filename, content, capsys):
])
def test_preparser_content(filename, content):
"""
Test the preparser content
Test the PreParser content
"""
testfile = temp_name(os.path.join(_BASE_PATH_GIVEN, filename))
assert os.path.exists(testfile)
@ -258,7 +258,7 @@ def test_preparser_content(filename, content):
def test_subfolder_parsing():
"""
Test the preparser content
Test the PreParser content
"""
testfile = temp_name(os.path.join(_BASE_PATH_GIVEN, "superimports.bpr"))
assert os.path.exists(testfile)

View File

@ -19,12 +19,21 @@ files = [
("base/config/test6.bpr", "", None),
("base/config/test7.yml", "", None),
("base/templates/test8.yml", "", None),
("base/unparsable.php", "", ""),
]
def make_new_file(path, content="", mode="w+"):
"""
Make a new file
Make a new file and all the directories in the path
:param path: the path to the file
:type path: str
:param content: the content of the file
:type content: str
:param mode: the mode of the file
:type mode: str
:return: the path to the file
:rtype: str
"""
name = os.path.join(_TEMP_DIRECTORY.name, path)
os.makedirs(os.path.dirname(name), exist_ok=True)
@ -88,6 +97,7 @@ def test_result_crawler(list_files, env):
"""
crw = sitecrawler.SiteCrawler(_BASE, _DEST, env)
crw.set_all_preparsers()
crw.copy_unparsable_files()
for pp, dest in crw:
assert os.path.exists(dest)
assert os.path.isfile(dest)

View File

@ -4,7 +4,7 @@ import pyparsing
import pytest
import bootstraparse.modules.syntax as sy
from bootstraparse.modules.tools import __GL, __module_path, find_variables_in_file
from bootstraparse.modules.tools import __GL, __GLk, __module_path, find_variables_in_file
ptp = pytest.param
__XF = pytest.mark.xfail
@ -111,33 +111,33 @@ dict_advanced_syntax_input_and_expected_output = {
# Enhanced text
"et_em": [
# Single em element, should only match the first "*"
("*", (sy.EtEmToken(["*"]),), __GL()),
("*", (sy.EtEmToken(["*"]),), __GLk()),
],
"et_strong": [
# Single strong element, should only match the first "**"
("**", (sy.EtStrongToken(["**"]),), __GL()),
("**", (sy.EtStrongToken(["**"]),), __GLk()),
],
"et_underline": [
# Single underline element, should only match the first "__"
("__", (sy.EtUnderlineToken(["__"]),), __GL()),
("__", (sy.EtUnderlineToken(["__"]),), __GLk()),
],
"et_strikethrough": [
# Single strikethrough element, should only match the first "~~"
("~~", (sy.EtStrikethroughToken(["~~"]),), __GL()),
("~~", (sy.EtStrikethroughToken(["~~"]),), __GLk()),
],
"et_custom_span": [
# Single custom span element, should only match the first "(#number)"
("(#12345)", (sy.EtCustomSpanToken(["12345"]),), __GL()),
("(#12345)", (sy.EtCustomSpanToken(["12345"]),), __GLk()),
],
"il_link": [
# Single link element, should only match the first "[link_name]('link')"
("[text_link]('text://www.website.com/link.html')",
[sy.HyperlinkToken(["[text_link]('text://www.website.com/link.html')"])], __GL())
[sy.HyperlinkToken(["[text_link]('text://www.website.com/link.html')"])], __GLk())
],
"assignation": [
# Single assignation element, should only match the first "a=1"
("a=1", [sy.BeAssignToken([["a", 1]]), ], __GL()),
("a='str'", [sy.BeAssignToken([["a", 'str']]), ], __GL()),
("a=1", [sy.BeAssignToken([["a", 1]]), ], __GLk()),
("a='str'", [sy.BeAssignToken([["a", 'str']]), ], __GLk()),
],
"var": [
# List of variables or assignations, should match "[a=12, b=13, 'c']"
@ -145,13 +145,13 @@ dict_advanced_syntax_input_and_expected_output = {
sy.BeAssignToken([["a_12", 12]]),
sy.BeAssignToken([["b", 13]]),
sy.BeValueToken(["c"]),
], __GL()),
], __GLk()),
("[a=12, b=13, 'c', d=14]", [
sy.BeAssignToken([["a", 12]]),
sy.BeAssignToken([["b", 13]]),
sy.BeValueToken(["c"]),
sy.BeAssignToken([["d", 14]]),
], __GL()),
], __GLk()),
],
"optional": [
# Optional element, should match any optional element
@ -163,7 +163,7 @@ dict_advanced_syntax_input_and_expected_output = {
sy.BeValueToken(["c"]),
]),
]),
], __GL()),
], __GLk()),
("[a=12, b=13, 'c', d=14]", [
sy.OptionalToken([
sy.OptionalVarToken([
@ -173,7 +173,7 @@ dict_advanced_syntax_input_and_expected_output = {
sy.BeAssignToken([["d", 14]]),
]),
]),
], __GL()),
], __GLk()),
("[a=12]{aze,/.}", (
sy.OptionalToken([
sy.OptionalVarToken([
@ -183,7 +183,7 @@ dict_advanced_syntax_input_and_expected_output = {
"aze,/.",
]),
]),
), __GL()),
), __GLk()),
("{aze,/.__}[a=12]", (
sy.OptionalToken([
sy.OptionalInsertToken([
@ -193,21 +193,21 @@ dict_advanced_syntax_input_and_expected_output = {
sy.BeAssignToken([["a", 12]]),
]),
]),
), __GL()),
), __GLk()),
("{aze!=}", (
sy.OptionalToken([
sy.OptionalInsertToken([
"aze!=",
]),
]),
), __GL()),
), __GLk()),
("{{aze!=}}", (
sy.OptionalToken([
sy.OptionalClassToken([
"aze!=",
]),
]),
), __GL()),
), __GLk()),
("{{aze!=}}{azett!=}", (
sy.OptionalToken([
sy.OptionalClassToken([
@ -217,7 +217,7 @@ dict_advanced_syntax_input_and_expected_output = {
"azett!=",
]),
]),
), __GL()),
), __GLk()),
("{{aze!=}}{azett!=}[a_32=12]", (
sy.OptionalToken([
sy.OptionalClassToken([
@ -230,7 +230,7 @@ dict_advanced_syntax_input_and_expected_output = {
sy.BeAssignToken([["a_32", 12]]),
]),
]),
), __GL()),
), __GLk()),
("[_a=12]{aze,/.}{{azer!=}}", (
sy.OptionalToken([
sy.OptionalVarToken([
@ -243,36 +243,36 @@ dict_advanced_syntax_input_and_expected_output = {
"azer!=",
]),
]),
), __GL()),
), __GLk()),
],
# Enhanced Text
"enhanced_text": [
# Matches a line of text, with or without inline elements
("*test*", (sy.EtEmToken(["*"]), sy.TextToken(["test"]), sy.EtEmToken(["*"])), __GL()),
("**test**", (sy.EtStrongToken(["**"]), sy.TextToken(["test"]), sy.EtStrongToken(["**"])), __GL()),
("__test__", (sy.EtUnderlineToken(["__"]), sy.TextToken(["test"]), sy.EtUnderlineToken(["__"])), __GL()),
("~~test~~", (sy.EtStrikethroughToken(["~~"]), sy.TextToken(["test"]), sy.EtStrikethroughToken(["~~"])), __GL()),
("(#12345)", (sy.EtCustomSpanToken(["12345"]),), __GL()),
("*test*", (sy.EtEmToken(["*"]), sy.TextToken(["test"]), sy.EtEmToken(["*"])), __GLk()),
("**test**", (sy.EtStrongToken(["**"]), sy.TextToken(["test"]), sy.EtStrongToken(["**"])), __GLk()),
("__test__", (sy.EtUnderlineToken(["__"]), sy.TextToken(["test"]), sy.EtUnderlineToken(["__"])), __GLk()),
("~~test~~", (sy.EtStrikethroughToken(["~~"]), sy.TextToken(["test"]), sy.EtStrikethroughToken(["~~"])), __GLk()),
("(#12345)", (sy.EtCustomSpanToken(["12345"]),), __GLk()),
("*test*test*", (sy.EtEmToken(["*"]), sy.TextToken(["test"]), sy.EtEmToken(["*"]), sy.TextToken(["test"]),
sy.EtEmToken(["*"])), __GL()),
sy.EtEmToken(["*"])), __GLk()),
("**test**test**",
(sy.EtStrongToken(["**"]), sy.TextToken(["test"]), sy.EtStrongToken(["**"]), sy.TextToken(["test"]),
sy.EtStrongToken(["**"]),), __GL()),
sy.EtStrongToken(["**"]),), __GLk()),
("__test__test__", (sy.EtUnderlineToken(["__"]), sy.TextToken(["test"]), sy.EtUnderlineToken(["__"]),
sy.TextToken(["test"]), sy.EtUnderlineToken(["__"]),), __GL()),
sy.TextToken(["test"]), sy.EtUnderlineToken(["__"]),), __GLk()),
("~~test~~test~~", (sy.EtStrikethroughToken(["~~"]), sy.TextToken(["test"]), sy.EtStrikethroughToken(["~~"]),
sy.TextToken(["test"]), sy.EtStrikethroughToken(["~~"]),), __GL()),
sy.TextToken(["test"]), sy.EtStrikethroughToken(["~~"]),), __GLk()),
("(#12345)test(#12345)", (sy.EtCustomSpanToken(["12345"]), sy.TextToken(["test"]),
sy.EtCustomSpanToken(["12345"]),), __GL()),
sy.EtCustomSpanToken(["12345"]),), __GLk()),
("__test(#12)test**test__test~~", (sy.EtUnderlineToken(["__"]), sy.TextToken(["test"]),
sy.EtCustomSpanToken(["12"]), sy.TextToken(["test"]),
sy.EtStrongToken(["**"]),
sy.TextToken(["test"]), sy.EtUnderlineToken(["__"]),
sy.TextToken(["test"]), sy.EtStrikethroughToken(["~~"]),), __GL()),
sy.TextToken(["test"]), sy.EtStrikethroughToken(["~~"]),), __GLk()),
("Test text with a link [link_name]('link')", (
sy.TextToken(["Test text with a link"]),
sy.HyperlinkToken(["[link_name]('link')"]),
), __GL()),
), __GLk()),
("Test text with a link [link_name](\"link\") and a *bold* text", (
sy.TextToken(["Test text with a link"]),
sy.HyperlinkToken(['[link_name]("link")']),
@ -280,39 +280,39 @@ dict_advanced_syntax_input_and_expected_output = {
sy.TextToken(["bold"]),
sy.EtEmToken(["*"]),
sy.TextToken(["text"]),
), __GL()),
), __GLk()),
("Reverse order (#123) Span __ Underline ~~ Strikethrough", (
sy.TextToken(["Reverse order"]),
sy.EtCustomSpanToken(["123"]), sy.TextToken(["Span"]),
sy.EtUnderlineToken(["__"]), sy.TextToken(["Underline"]),
sy.EtStrikethroughToken(["~~"]), sy.TextToken(["Strikethrough"]),
), __GL()),
), __GLk()),
],
"html_insert": [
# Match an HTML insert
("{azer='\"}", (
"azer='\"",
), __GL()),
), __GLk()),
],
"class_insert": [
# Match a class insert
("{{azer='\"}}", (
"azer='\"",
), __GL()),
), __GLk()),
],
# Structural Elements
"se_start": [
# Matches a start of a structural element
("<<div", (sy.StructuralElementStartToken(["div"]),), __GL()),
("<<aside", (sy.StructuralElementStartToken(["aside"]),), __GL()),
("<<article", (sy.StructuralElementStartToken(["article"]),), __GL()),
("<<section", (sy.StructuralElementStartToken(["section"]),), __GL()),
("<<div", (sy.StructuralElementStartToken(["div"]),), __GLk()),
("<<aside", (sy.StructuralElementStartToken(["aside"]),), __GLk()),
("<<article", (sy.StructuralElementStartToken(["article"]),), __GLk()),
("<<section", (sy.StructuralElementStartToken(["section"]),), __GLk()),
],
"se_end": [
("div>>", (sy.StructuralElementEndToken(["div"]),), __GL()),
("aside>>", (sy.StructuralElementEndToken(["aside"]),), __GL()),
("article>>", (sy.StructuralElementEndToken(["article"]),), __GL()),
("section>>", (sy.StructuralElementEndToken(["section"]),), __GL()),
("div>>", (sy.StructuralElementEndToken(["div"]),), __GLk()),
("aside>>", (sy.StructuralElementEndToken(["aside"]),), __GLk()),
("article>>", (sy.StructuralElementEndToken(["article"]),), __GLk()),
("section>>", (sy.StructuralElementEndToken(["section"]),), __GLk()),
],
"se": [
# Matches a div element, must be at the beginning of the line, the closing div can be with arguments
@ -325,7 +325,7 @@ dict_advanced_syntax_input_and_expected_output = {
]),
sy.OptionalInsertToken(["var='test', number=11"]),
]),
), __GL(),),
), __GLk(),),
("div>> [class='blue', 123]", (
sy.StructuralElementEndToken(["div"]),
sy.OptionalToken([
@ -334,56 +334,56 @@ dict_advanced_syntax_input_and_expected_output = {
sy.BeValueToken([123.0]),
]),
]),
), __GL(),),
), __GLk(),),
("div>> {var='test', number=11}", (
sy.StructuralElementEndToken(["div"]),
sy.OptionalToken([
sy.OptionalInsertToken(["var='test', number=11"]),
]),
), __GL(),),
), __GLk(),),
],
# List Elements
"one_olist": [
# Matches a one-level ordered list element, must be at the beginning of the line
# Drop the "." from the match
("#. Text", (sy.EtOlistToken([sy.TextToken(["Text"])]),), __GL()),
("#. Text", (sy.EtOlistToken([sy.TextToken(["Text"])]),), __GLk()),
],
"one_ulist": [
# Matches a one-level unordered list element, must be at the beginning of the line
# Drop the "-" from the match
("- Text", (sy.EtUlistToken([sy.TextToken(["Text"])]),), __GL()),
("- Text", (sy.EtUlistToken([sy.TextToken(["Text"])]),), __GLk()),
],
# Headers
"one_header": [
# Matches a one-level header element, must be at the beginning of the line
("# Text1 #", (sy.HeaderToken(["#", "Text1 "]),), __GL()),
("## Text2 ##", (sy.HeaderToken(["##", "Text2 "]),), __GL()),
("### Text3 ###", (sy.HeaderToken(["###", "Text3 "]),), __GL()),
("#### Text4 ####", (sy.HeaderToken(["####", "Text4 "]),), __GL()),
("##### Text5 #####", (sy.HeaderToken(["#####", "Text5 "]),), __GL()),
("###### Text6 ######", (sy.HeaderToken(["######", "Text6 "]),), __GL()),
("# Text1 #", (sy.HeaderToken(["#", "Text1 "]),), __GLk()),
("## Text2 ##", (sy.HeaderToken(["##", "Text2 "]),), __GLk()),
("### Text3 ###", (sy.HeaderToken(["###", "Text3 "]),), __GLk()),
("#### Text4 ####", (sy.HeaderToken(["####", "Text4 "]),), __GLk()),
("##### Text5 #####", (sy.HeaderToken(["#####", "Text5 "]),), __GLk()),
("###### Text6 ######", (sy.HeaderToken(["######", "Text6 "]),), __GLk()),
],
# One-line display elements
"one_display": [
# Matches a one-level display element, must be at the beginning of the line
("! Display1 !", (sy.DisplayToken(["!", "Display1 "]),), __GL()),
("!! Display2 !!", (sy.DisplayToken(["!!", "Display2 "]),), __GL()),
("!!! Display3 !!!", (sy.DisplayToken(["!!!", "Display3 "]),), __GL()),
("! Display1 !", (sy.DisplayToken(["!", "Display1 "]),), __GLk()),
("!! Display2 !!", (sy.DisplayToken(["!!", "Display2 "]),), __GLk()),
("!!! Display3 !!!", (sy.DisplayToken(["!!!", "Display3 "]),), __GLk()),
],
# One-line elements
"one_line": [
# Matches a one-line element, must be at the beginning of the line
("- Text1", (sy.EtUlistToken([sy.TextToken(["Text1"])]),), __GL()),
("#. Text2", (sy.EtOlistToken([sy.TextToken(["Text2"])]),), __GL()),
("! Display3 !", (sy.DisplayToken(["!", "Display3 "]),), __GL()),
("!! Display4 !!", (sy.DisplayToken(["!!", "Display4 "]),), __GL()),
("### Text5 ###", (sy.HeaderToken(["###", "Text5 "]),), __GL()),
("#### Text6 ####", (sy.HeaderToken(["####", "Text6 "]),), __GL()),
("- Text1", (sy.EtUlistToken([sy.TextToken(["Text1"])]),), __GLk()),
("#. Text2", (sy.EtOlistToken([sy.TextToken(["Text2"])]),), __GLk()),
("! Display3 !", (sy.DisplayToken(["!", "Display3 "]),), __GLk()),
("!! Display4 !!", (sy.DisplayToken(["!!", "Display4 "]),), __GLk()),
("### Text5 ###", (sy.HeaderToken(["###", "Text5 "]),), __GLk()),
("#### Text6 ####", (sy.HeaderToken(["####", "Text6 "]),), __GLk()),
("- Text7 [class='blue', 123]{var='test', number=11}", (
sy.EtUlistToken([
sy.TextToken(["Text7"]),
@ -395,7 +395,7 @@ dict_advanced_syntax_input_and_expected_output = {
sy.OptionalInsertToken(["var='test', number=11"]),
]),
]),
), __GL()),
), __GLk()),
("#. Text8 [class='blue', 123]{var='test', number=11}", (
sy.EtOlistToken([
sy.TextToken(["Text8"]),
@ -407,33 +407,33 @@ dict_advanced_syntax_input_and_expected_output = {
sy.OptionalInsertToken(["var='test', number=11"]),
]),
]),
), __GL()),
), __GLk()),
("! Display9 ! [class='blue', 123]{var='test', number=11}", (
sy.DisplayToken([
"!",
"Display9 ",
sy.OptionalToken([
sy.OptionalVarToken([
sy.BeAssignToken([["class", "blue"]]),
sy.BeValueToken([123.0]),
]),
sy.OptionalInsertToken(["var='test', number=11"]),
]),
]),
), __GL()), # Bad optional implementation
sy.OptionalToken([
sy.OptionalVarToken([
sy.BeAssignToken([["class", "blue"]]),
sy.BeValueToken([123.0]),
]),
sy.OptionalInsertToken(["var='test', number=11"]),
]),
), __GLk(1)), # Bad optional implementation
("## Text10 ## [class='blue', 123]{var='test', number=11}", (
sy.HeaderToken([
"##",
"Text10 ",
sy.OptionalToken([
sy.OptionalVarToken([
sy.BeAssignToken([["class", "blue"]]),
sy.BeValueToken([123.0]),
]),
sy.OptionalInsertToken(["var='test', number=11"]),
]),
]),
), __GL()), # Bad optional implementation
sy.OptionalToken([
sy.OptionalVarToken([
sy.BeAssignToken([["class", "blue"]]),
sy.BeValueToken([123.0]),
]),
sy.OptionalInsertToken(["var='test', number=11"]),
]),
), __GLk(1)), # Bad optional implementation
],
# Tables
@ -442,27 +442,27 @@ dict_advanced_syntax_input_and_expected_output = {
("| Text1 | Text2 |", (sy.TableRowToken([
sy.TableCellToken([sy.TextToken(["Text1 "])]),
sy.TableCellToken([sy.TextToken(["Text2 "])]),
]), ), __GL()),
]), ), __GLk()),
("| Text1 | Text2 | Text3 |", (
sy.TableRowToken([
sy.TableCellToken([sy.TextToken(["Text1 "])]),
sy.TableCellToken([sy.TextToken(["Text2 "])]),
sy.TableCellToken([sy.TextToken(["Text3 "])]),
]),
), __GL()),
), __GLk()),
("| 2 Text1 | Text2 |", (
sy.TableRowToken([
sy.TableCellToken([sy.TextToken(["2 Text1 "])]),
sy.TableCellToken([sy.TextToken(["Text2 "])]),
]),
), __GL()),
), __GLk()),
("|2 Text1 | Text2 |", (
sy.TableRowToken([
'2',
sy.TableCellToken([sy.TextToken(["Text1 "])]),
sy.TableCellToken([sy.TextToken(["Text2 "])]),
]),
), __GL()),
), __GLk()),
("|2 Text1 |3 Text2 |", (
sy.TableRowToken([
'2',
@ -470,7 +470,7 @@ dict_advanced_syntax_input_and_expected_output = {
'3',
sy.TableCellToken([sy.TextToken(["Text2 "])]),
]),
), __GL()),
), __GLk()),
("|2 Text1 |3 Text2 |4 Text3 |", (
sy.TableRowToken([
'2',
@ -480,7 +480,7 @@ dict_advanced_syntax_input_and_expected_output = {
'4',
sy.TableCellToken([sy.TextToken(["Text3 "])]),
]),
), __GL()),
), __GLk()),
("|3 Text1 | Text2 | {var='test', number=11}", (
sy.TableRowToken([
'3',
@ -492,19 +492,19 @@ dict_advanced_syntax_input_and_expected_output = {
"var='test', number=11",
]),
]),
), __GL()),
), __GLk()),
],
"table_separator": [
# Matches a table element, must be at the beginning of the line
("|---|---|", (sy.TableSeparatorToken(["---", "---"]),), __GL()),
("|---|---|---|", (sy.TableSeparatorToken(["---", "---", "---"]),), __GL()),
("|:--|-:-|--:|--:|", (sy.TableSeparatorToken([":--", "-:-", "--:", "--:"]),), __GL()),
("|---|---|", (sy.TableSeparatorToken(["---", "---"]),), __GLk()),
("|---|---|---|", (sy.TableSeparatorToken(["---", "---", "---"]),), __GLk()),
("|:--|-:-|--:|--:|", (sy.TableSeparatorToken([":--", "-:-", "--:", "--:"]),), __GLk()),
],
"line": [
# Match any line parsed by the parser (can match header, list table etc...) this is the main syntax element
("# Text1 #", (sy.HeaderToken(["#", "Text1 "]),), __GL()),
("# Text1 #", (sy.HeaderToken(["#", "Text1 "]),), __GLk()),
("Text *em __underline__ still em*", (
sy.TextToken(["Text"]),
sy.EtEmToken(["*"]),
@ -514,17 +514,19 @@ dict_advanced_syntax_input_and_expected_output = {
sy.EtUnderlineToken(["__"]),
sy.TextToken(["still em"]),
sy.EtEmToken(["*"]),
), __GL(),),
), __GLk(),),
("|2 Text1 | Text2 |", (
sy.TableRowToken([
"2",
sy.TableCellToken([sy.TextToken(["Text1 "])]),
sy.TableCellToken([sy.TextToken(["Text2 "])]),
]),
), __GL()),
("|---|---|", (sy.TableSeparatorToken(["---", "---"]),), __GL()),
("- Text", (sy.EtUlistToken([sy.TextToken(["Text"])]),), __GL()),
("div>>", (sy.StructuralElementEndToken(["div"]),), __GL()),
), __GLk()),
("|---|---|", (sy.TableSeparatorToken(["---", "---"]),), __GLk()),
("- Text", (sy.EtUlistToken([sy.TextToken(["Text"])]),), __GLk()),
(" - Non ulist", (sy.TextToken(["- Non ulist"]),), __GLk()),
("div>>", (sy.StructuralElementEndToken(["div"]),), __GLk()),
("body>>", (sy.StructuralElementEndToken(["body"]),), __GLk()),
("#. Text with *em* and __underline__ and {arguments}{{super-arguments}}[a=1, c=2,'ui']", (
sy.EtOlistToken([
sy.TextToken(["Text with"]),
@ -550,11 +552,11 @@ dict_advanced_syntax_input_and_expected_output = {
]),
]),
]),
), __GL(),),
("```Code```", (sy.CodeToken(["```"]), sy.TextToken(["Code"]), sy.CodeToken(["```"])), __GL()),
), __GLk(),),
("```Code```", (sy.CodeToken(["```"]), sy.TextToken(["Code"]), sy.CodeToken(["```"])), __GLk()),
],
"blockquote": [
("> Text", (sy.BlockQuoteToken([sy.TextToken(["Text"])]),), __GL()),
("> Text", (sy.BlockQuoteToken([sy.TextToken(["Text"])]),), __GLk()),
("> Text with *em* and __underline__", (
sy.BlockQuoteToken([
sy.TextToken(["Text with"]),
@ -566,15 +568,15 @@ dict_advanced_syntax_input_and_expected_output = {
sy.TextToken(["underline"]),
sy.EtUnderlineToken(["__"]),
]),
), __GL()),
), __GLk()),
],
"blockquote_author": [
("> -- Author Name", (
sy.BlockQuoteAuthorToken(["Author Name"]),
), __GL()),
), __GLk()),
],
"quotation": [
("> Text", (sy.BlockQuoteToken([sy.TextToken(["Text"])]),), __GL()),
("> Text", (sy.BlockQuoteToken([sy.TextToken(["Text"])]),), __GLk()),
("> Text with *em* and __underline__", (
sy.BlockQuoteToken([
sy.TextToken(["Text with"]),
@ -586,14 +588,14 @@ dict_advanced_syntax_input_and_expected_output = {
sy.TextToken(["underline"]),
sy.EtUnderlineToken(["__"]),
]),
), __GL()),
), __GLk()),
("> -- Author Name", (
sy.BlockQuoteAuthorToken(["Author Name"]),
), __GL()),
), __GLk()),
],
"code": [
("```\nText\n```", (sy.CodeToken(["```"]),), __GL()),
("```\nText\nText\n```", (sy.CodeToken(["```"]),), __GL()),
("```\nText\n```", (sy.CodeToken(["```"]),), __GLk()),
("```\nText\nText\n```", (sy.CodeToken(["```"]),), __GLk()),
],
}
@ -666,6 +668,7 @@ list_of_text_input_and_readable_output = [
# Other Mostly for testing and coverage
("'", "quotes", "'"),
(" -", "line", "-"),
]
@ -824,7 +827,7 @@ def test_advanced_expression_and_token_creation(markup_element, to_parse, expect
result = expr.parse_string(to_parse)
print()
print(f"Parsing string: '{to_parse}'")
print("Defined at %(filename)s:%(lineno)d" % {'filename': __file__, 'lineno': line_test})
print(line_test)
print(f"With expression: '{markup_element}'")
print("Defined at %(filename)s:%(lineno)d" % {'filename': __syntax_file,
'lineno': __definition_of_syntax_elements[markup_element]})

View File

@ -1,7 +1,7 @@
import pytest
from bootstraparse.modules import syntax, error_mngr, tools, context_mngr
from bootstraparse.modules.tools import __GL, __module_path, find_variables_in_file
from bootstraparse.modules import syntax # , error_mngr, tools, context_mngr
# from bootstraparse.modules.tools import __GL, __module_path, find_variables_in_file
__XF = pytest.mark.xfail