alonbl patch series (#1017)

* Add linesep configuration option

Platform independent line ends are important to avoid conflicts.

Resolves: #1010
Signed-off-by: Alon Bar-Lev <alon.barlev@gmail.com>

* Perform atomic write to files

Do not overwrite file as we may corrupt the file if something fails, instead
write into temporary location and move the result into the final destination.

Signed-off-by: Alon Bar-Lev <alon.barlev@gmail.com>

* Synchronize README command-line usage

Signed-off-by: Alon Bar-Lev <alon.barlev@gmail.com>

* Support files as positional argument

Tools such as pre-commit and other tools expects the tools to accept the
input files as positional argument and not as options.

Let's support the two variants.

Signed-off-by: Alon Bar-Lev <alon.barlev@gmail.com>

* Support shallow git clone without tags

pre-commit uses shallow clone without tags, as this is not used to publish the
artifact, let's assume version '0' in this case.

Signed-off-by: Alon Bar-Lev <alon.barlev@gmail.com>

* Add pre-commit hooks support

Signed-off-by: Alon Bar-Lev <alon.barlev@gmail.com>

---------

Signed-off-by: Alon Bar-Lev <alon.barlev@gmail.com>
This commit is contained in:
Alon Bar-Lev 2023-11-05 15:24:50 +02:00 committed by GitHub
parent 941a661e19
commit aeeeed6c27
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
17 changed files with 190 additions and 36 deletions

8
.pre-commit-hooks.yaml Normal file
View File

@ -0,0 +1,8 @@
- id: vsg
name: fix VHDL style
description: VHDL Style Guide
entry: vsg --fix --jobs=1
language: python
types:
- file
files: \.(vhd|vhdl)$

View File

@ -158,12 +158,16 @@ The command line tool can be invoked with:
$ vsg
usage: VHDL Style Guide (VSG) [-h] [-f FILENAME [FILENAME ...]] [-lr LOCAL_RULES] [-c CONFIGURATION [CONFIGURATION ...]] [--fix]
[-fp FIX_PHASE] [-j JUNIT] [-js JSON] [-of {vsg,syntastic,summary}] [-b] [-oc OUTPUT_CONFIGURATION]
[-rc RULE_CONFIGURATION] [--style {indent_only,jcl}] [-v] [-ap] [--fix_only FIX_ONLY] [-p JOBS]
[--debug]
[-rc RULE_CONFIGURATION] [--style {indent_only,jcl}] [-v] [-ap] [--fix_only FIX_ONLY] [--stdin]
[--quality_report QUALITY_REPORT] [-p JOBS] [--debug]
[FILENAME ...]
Analyzes VHDL files for style guide violations. Reference documentation is located at: http://vhdl-style-guide.readthedocs.io/en/latest/index.html
optional arguments:
positional arguments:
FILENAME File to analyze
options:
-h, --help show this help message and exit
-f FILENAME [FILENAME ...], --filename FILENAME [FILENAME ...]
File to analyze
@ -190,6 +194,9 @@ The command line tool can be invoked with:
-v, --version Displays version information
-ap, --all_phases Do not stop when a violation is detected.
--fix_only FIX_ONLY Restrict fixing via JSON file.
--stdin Read VHDL input from stdin, disables all other file selections, disables multiprocessing
--quality_report QUALITY_REPORT
Create code quality report for GitLab
-p JOBS, --jobs JOBS number of parallel jobs to use, default is the number of cpu cores
--debug Displays verbose debug information
@ -197,6 +204,19 @@ Here is an example output running against a test file:
.. image:: https://github.com/jeremiah-c-leary/vhdl-style-guide/blob/master/docs/img/fixing_single_file.gif
pre-commit Integration
----------------------
Here is an example of ``.pre-commit-config.yaml`` file:
.. code-block:: yaml
repos:
- repo: https://github.com/jeremiah-c-leary/vhdl-style-guide
rev: v3.18.0
hooks:
- id: vsg
Documentation
-------------

View File

@ -97,6 +97,17 @@ The file_rules option allows for configuration of individual rules per file.
Any file listed under this option will have the configuration applied if it is being analyzed.
.. The file_rules is exactly the same as file_list except that it will not add the file to the scan list.
linesep
-------
The linesep is an optional settings for line separator.
Default is platform specific.
Logical values may be "\n" or "\r\n".
.. code-block:: yaml
linesep: "\n"
local_rules
-----------

View File

@ -20,7 +20,7 @@ Welcome to vhdl-style-guide's documentation!
code_tags
continuous_integration_servers
editor_integration/editor_integration
tool_integration
tool_integration/tool_integration
pragmas
localizing
phases

View File

@ -1,5 +1,5 @@
Tool Integration
----------------
Generic Tool Integration
------------------------
VSG supports integration with other tools via several command line options.

View File

@ -0,0 +1,25 @@
pre-commit Integration
----------------------
VSG supports integration with pre-commit using the following ``.pre-commit-config.yaml`` file:
.. code-block:: yaml
repos:
- repo: https://github.com/jeremiah-c-leary/vhdl-style-guide
rev: v3.18.0
hooks:
- id: vsg
You may customize VSG by using the ``arg`` node, for example:
.. code-block:: yaml
repos:
- repo: https://github.com/alonbl/vhdl-style-guide
rev: v3.18.0
hooks:
- id: vsg
args:
- --configuration=.vsg.yaml
- --output_format=syntastic

View File

@ -0,0 +1,8 @@
Tool Integration
------------------
.. toctree::
:maxdepth: 2
generic.rst
pre-commit.rst

View File

@ -14,14 +14,18 @@ The command line tool can be invoked with:
[-of {vsg,syntastic,summary}] [-b]
[-oc OUTPUT_CONFIGURATION]
[-rc RULE_CONFIGURATION]
[--style {jcl,indent_only}] [-v] [-ap]
[--style {indent_only,jcl}] [-v] [-ap]
[--fix_only FIX_ONLY] [--stdin]
[--quality_report QUALITY_REPORT] [-p JOBS]
[--debug]
[FILENAME ...]
Analyzes VHDL files for style guide violations. Reference documentation is
located at: http://vhdl-style-guide.readthedocs.io/en/latest/index.html
positional arguments:
FILENAME File to analyze
options:
-h, --help show this help message and exit
-f FILENAME [FILENAME ...], --filename FILENAME [FILENAME ...]
@ -45,7 +49,7 @@ The command line tool can be invoked with:
Write configuration to file name.
-rc RULE_CONFIGURATION, --rule_configuration RULE_CONFIGURATION
Display configuration of a rule
--style {jcl,indent_only}
--style {indent_only,jcl}
Use predefined style
-v, --version Displays version information
-ap, --all_phases Do not stop when a violation is detected.

View File

@ -125,7 +125,7 @@ def apply_rules(commandLineArguments, oConfig, tIndexFileName):
oRules.fix(
commandLineArguments.fix_phase, commandLineArguments.skip_phase, fix_only
)
write_vhdl_file(oVhdlFile)
write_vhdl_file(oVhdlFile, oConfig.dConfig)
oRules.clear_violations()
oRules.check_rules(
@ -149,13 +149,20 @@ def apply_rules(commandLineArguments, oConfig, tIndexFileName):
return fExitStatus, testCase, dJsonEntry, sOutputStd, sOutputErr, bKeepProcessingFiles
def write_vhdl_file(oVhdlFile):
def write_vhdl_file(oVhdlFile, dConfig):
tmpfile = f"{oVhdlFile.filename}.tmp"
try:
with open(oVhdlFile.filename, 'w', encoding='utf-8') as oFile:
with open(tmpfile, 'w', encoding='utf-8', newline=dConfig.get("linesep")) as oFile:
for sLine in oVhdlFile.get_lines()[1:]:
oFile.write(sLine + '\n')
os.replace(tmpfile, oVhdlFile.filename)
except PermissionError as err:
print (err, "Could not write fixes back to file.")
finally:
try:
os.remove(tmpfile)
except FileNotFoundError:
pass
def create_junit_testcase(sVhdlFileName, oException):

View File

@ -48,7 +48,7 @@ def parse_command_line_arguments():
Reference documentation is located at:
http://vhdl-style-guide.readthedocs.io/en/latest/index.html''')
parser.add_argument('-f', '--filename', type=__is_valid_file, nargs='+', help='File to analyze')
parser.add_argument('-f', '--filename', type=__is_valid_file, nargs='+', default=[], help='File to analyze')
parser.add_argument('-lr', '--local_rules', help='Path to local rules')
parser.add_argument('-c', '--configuration', type=__is_valid_file, nargs='+', help='JSON or YAML configuration file(s)')
parser.add_argument('--fix', default=False, action='store_true', help='Fix issues found')
@ -82,6 +82,8 @@ def parse_command_line_arguments():
help="number of parallel jobs to use, default is the number of cpu cores",
)
parser.add_argument('--debug', default=False, action='store_true', help='Displays verbose debug information')
parser.add_argument('filename_args', metavar='FILENAME', type=__is_valid_file, nargs='*', default=[],
help='File to analyze')
args_ = parser.parse_args()
@ -117,7 +119,7 @@ def get_predefined_styles():
with open(os.path.join(sStylePath, sStyle)) as yaml_file:
tempConfiguration = yaml.safe_load(yaml_file)
lReturn.append(tempConfiguration['name'])
return lReturn
return sorted(lReturn)
def validate_backup_argument(args_):
@ -147,12 +149,12 @@ def add_quality_report_argument(parser):
def fix_filename_argument(args_):
if args_.filename is None:
return None
lUpdate = []
for lFilenames in args_.filename:
lUpdate.extend(lFilenames)
args_.filename = lUpdate
for lFilenames in args_.filename_args:
lUpdate.extend(lFilenames)
args_.filename = lUpdate or None
def fix_configuration_argument(args_):

View File

@ -48,6 +48,32 @@ class test(unittest.TestCase):
self.assertEqual(lActual, lExpected)
def test_filename_w_two_files_positional(self):
sys.argv = ['vsg']
sys.argv.extend([sAFile, sBFile])
lExpected = [sAFile, sBFile]
lExpected.sort()
oActual = cmd_line_args.parse_command_line_arguments()
lActual = oActual.filename
lActual.sort()
self.assertEqual(lActual, lExpected)
def test_filename_w_two_files_positional_merge(self):
sys.argv = ['vsg']
sys.argv.extend(['-f', sAFile, sBFile])
lExpected = [sAFile, sBFile]
lExpected.sort()
oActual = cmd_line_args.parse_command_line_arguments()
lActual = oActual.filename
lActual.sort()
self.assertEqual(lActual, lExpected)
def test_valid_glob(self):
sys.argv = ['vsg']
sys.argv.extend(['-f', sGlobFile])

View File

@ -0,0 +1,5 @@
--------------------------------------------------------------------------------
0 |
--------------------------------------------------------------------------------
1 | -- ANSI "€‚" comment
<class 'vsg.parser.comment'>

View File

@ -0,0 +1 @@
-- ANSI "€<>‚" comment

View File

@ -0,0 +1,28 @@
import os
import unittest
from vsg.tests import utils
from vsg import vhdlFile
sLrmUnit = utils.extract_lrm_unit_name(__name__)
lFile, eError =vhdlFile.utils.read_vhdlfile(os.path.join(os.path.dirname(__file__), sLrmUnit,'classification_test_input.vhd'))
oFile = vhdlFile.vhdlFile(lFile)
class test_token(unittest.TestCase):
def test_classification(self):
sTestDir = os.path.join(os.path.dirname(__file__), sLrmUnit)
lExpected = []
utils.read_file(os.path.join(sTestDir, 'classification_results.txt'), lExpected, False)
lActual = []
for oObject in utils.extract_objects(oFile, True):
lActual.append(str(oObject))
self.assertEqual(lExpected, lActual)

View File

@ -520,3 +520,13 @@ class testVsg(unittest.TestCase):
iExitStatus = e.returncode
self.assertEqual(lActual, lExpected)
def test_file_as_stdin(self):
with open('vsg/tests/vsg/entity1.vhd') as file1:
lExpected = []
lExpected.append('')
lActual = subprocess.check_output(['bin/vsg','--configuration','vsg/tests/vsg/config_3.yaml','--output_format','syntastic','--stdin'], stdin=file1)
lActual = str(lActual.decode('utf-8')).split('\n')
self.assertEqual(lActual, lExpected)

View File

@ -33,13 +33,14 @@ def get_version_info():
sReturnPath = os.getcwd()
sPath = os.path.dirname(__file__)
os.chdir(sPath)
lActual = subprocess.check_output(['git', 'describe', '--tags'])
lActual = str(lActual.decode('utf-8')).split('\n')
lVersion = lActual[0].split('-')
lVersion = "0" # version if no tags (shallow checkout)
try:
lActual = subprocess.check_output(['git', 'describe', '--tags'])
lActual = str(lActual.decode('utf-8')).split('\n')
lVersion = lActual[0].split('-')
sVersion = str(lVersion[0]) + '.dev' + str(lVersion[1])
sShaNum = str(lVersion[-1][1:])
except IndexError:
except (IndexError, subprocess.CalledProcessError):
sVersion = str(lVersion[0])
lActual = subprocess.check_output(['git', 'rev-parse', '--short', 'HEAD'])
lActual = str(lActual.decode('utf-8')).split('\n')

View File

@ -757,23 +757,21 @@ def is_whitespace(oObject):
def read_vhdlfile(sFileName):
def _read(oFile):
lLines = []
for sLine in oFile:
lLines.append(sLine.rstrip('\r\n'))
return lLines
if sFileName == 'stdin':
lLines = []
for line in sys.stdin:
lLines.append(line[:-1])
return lLines, None
return _read(sys.stdin), None
try:
lLines = []
with open(sFileName, encoding='utf-8') as oFile:
for sLine in oFile:
lLines.append(sLine)
return lLines, None
return _read(oFile), None
except UnicodeDecodeError:
lLines = []
with open(sFileName, encoding="ISO-8859-1") as oFile:
for sLine in oFile:
lLines.append(sLine)
return lLines, None
return _read(oFile), None
except OSError as e:
return [], e