Objective
Pylint is a Python static code analysis tool which looks for programming errors, helps enforcing a coding standard, sniffs for code smells and offers simple refactoring suggestions.
Setting up pylint
setting up pylint in pycharm
https://www.jetbrains.com/help/pycharm/configuring-third-party-tools.html
follow the above documentation to add pylint plugin to pycharm and to run it within the IDE as an external tool. On the top left hand corner of the documentation you can select the version of the pycharm IDE you are using. steps involved may slightly vary due to the version of the IDE.
or else you can intall pylint via
pip install pylint
create the pylintrc file
pylint --generate-rcfile > .pylintrc
this results in the creation of .pylintrc file in the root directory of your codebase.
Pylint checks for many things. Some of them may be more important than others for your application. You can create a file that allows you to tell Pylint to ignore certain checks. This file is called .pylintrc.
you can enable or disable rules via .pylintrc file, further you can set limits assigned to parameters such as line lenght , max statements allowed inside a file or a method using this file.
running pylint
you can run pylint on all files or within a certain directory via the commandline
pylint src
run pylint only within the src directory and will produce a result as below
************* Module src.pa.analyse_load_wait
src/pa/analyse_load_wait.py:45:0: R0914: Too many local variables (25/15) (too-many-locals)
src/pa/analyse_load_wait.py:50:4: C0103: Variable name "step_functionArn" doesn't conform to snake_case naming style (invalid-name)
src/pa/analyse_load_wait.py:55:4: C0103: Variable name "ALLOWED_CONCURRENT_EXECUTIONS" doesn't conform to snake_case naming style (invalid-name)
src/pa/analyse_load_wait.py:77:4: C0103: Variable name "shouldWait" doesn't conform to snake_case naming style (invalid-name)
src/pa/analyse_load_wait.py:104:20: C0103: Variable name "shouldWait" doesn't conform to snake_case naming style (invalid-name)
src/pa/analyse_load_wait.py:111:16: C0103: Variable name "shouldWait" doesn't conform to snake_case naming style (invalid-name)
src/pa/analyse_load_wait.py:45:26: W0613: Unused argument 'context' (unused-argument)
************* Module src.TakeBackupDecisionLambda.index
src/TakeBackupDecisionLambda/index.py:161:9: W0511: TODO: handle this (fixme)
src/TakeBackupDecisionLambda/index.py:80:0: R0914: Too many local variables (29/15) (too-many-locals)
src/TakeBackupDecisionLambda/index.py:159:11: W0703: Catching too general exception Exception (broad-except)
src/TakeBackupDecisionLambda/index.py:80:26: W0613: Unused argument 'context' (unused-argument)
************* Module src.TakeBackupDecisionLambda
src/TakeBackupDecisionLambda/__init__.py:1:0: C0103: Module name "TakeBackupDecisionLambda" doesn't conform to snake_case naming style (invalid-name)
------------------------------------------------------------------
Your code has been rated at 8.92/10 (previous run: 8.92/10, +0.00)the result will show the current issues within the code as well as the lines in which the said issues are found , it will also provide the current score out of 10.
Your code has been rated at 9.13/10 (previous run: 8.78/10, +0.35)Disabling pylint rules
there are two possible ways to achieve this
- disable the rule at the project level
- disable the rule only for a specific file or a place
to disable the rule at project level you will need to add it to the .pylintrc file disable section, ones tdone lint issues arising from that rule will not be counted for the final lint score
disabling at file level we can add the below line to the top of the file
#pylint: disable=unused-argument
in this case unused argument rule is disabled in the given file, you can add more rules in the same row with comma separation.
Adding a Pre-commit hook
To avoid bad code with lint issues being committed this precommit hook was introduced .follow the below documentation on how to set up precommit in your project
Install pre commit
pip install pre-commit
Add a precommit configuration
add the .pre-commit-config.yml file to your root direcotry
and add the following code lines to it
exclude: test/
repos:
- repo: local
hooks:
- id: pylint
name: pylint
entry: pylint src
language: system
types: [python]Here repo is set to local , meaning rules are deifined in a local file and referred internally.
further with this file we can exclude the folders on top of which pylint might run
Install the githook scripts
$ pre-commit install
pre-commit installed at .git/hooks/pre-commitRun pre commit
you can rull pre commit against all files or a given directory
pre-commit run --files src/*
pre-commit run --all-filespre commit hook will not allow you to commit until the score is above a given value we defined
Ignore commit check
$git commit -m -n ”commit message”
addition of -n flag lets you commit the changes eventhough the desired lint check score is not met.
Setting up pylint check during the Pull Request initiation
we have a jenkns job that triggers upon creation of any pull request
this job will update the sonar stats according to the current status of the project and will pass or fail the job accordingly
another job was created with the same template to do the lint check and return the result to the pull request .
Create a clone of the curremt pull request trigger jenkins job and name it as pull request trigger lint check - this can be done via create new option in jenkis and selecing the projects like)
In the windows shell execution remove all sonar related commands and add the below commands
pip install pylint
pylint src
pylint src >>pylint.log
pylint-fail-under --fail_under 9.0 src
these commands will install and run pylint against the project on the fly and log the issues and the score.
issues can be seen in the jenkins dashboard with the rest of the execution logs
and also fail the job when lint score is below 9.0
please refer the below job to see the set configurations
Further add the violations plugin to jenkins (already done)
set up a post build job to read the pylint.log created during the build , filter the violations
and display any violations in any given build visually in the jenkins dashboard
finally in a case where the lint check fails pull request will be notified with the message Jenkins CI lint check failed
Github actions ( An alternative way to lint check )
Lint checking can also be done via github actions
https://medium.com/swlh/enhancing-code-quality-with-github-actions-67561c6f7063
please refer the below repo for sample code
https://github.com/fnote/pylint-test-repo/actions
GitHub actions allow us to define a workflow to run the lint check
you need to create a .github folder within root and create a workflows directory inside. Within workflows you can have multiple files for each workflow.below scenario has 2 workflows defined.one yaml to check the python version and other for the lint check
Below are the result once .github is pushed to your repo
these workflows will be reflected under the actions
