Pylint Integration

5 min readAug 19, 2021

Objective

Pylint is a Python static code analysis tool which looks for programming errors, helps enforcing a coding standard, sniffs for code smells and offers simple refactoring suggestions.

Setting up pylint

setting up pylint in pycharm

https://www.jetbrains.com/help/pycharm/configuring-third-party-tools.html

follow the above documentation to add pylint plugin to pycharm and to run it within the IDE as an external tool. On the top left hand corner of the documentation you can select the version of the pycharm IDE you are using. steps involved may slightly vary due to the version of the IDE.

or else you can intall pylint via

pip install pylint

create the pylintrc file

pylint --generate-rcfile > .pylintrc

this results in the creation of .pylintrc file in the root directory of your codebase.

Pylint checks for many things. Some of them may be more important than others for your application. You can create a file that allows you to tell Pylint to ignore certain checks. This file is called .pylintrc.

you can enable or disable rules via .pylintrc file, further you can set limits assigned to parameters such as line lenght , max statements allowed inside a file or a method using this file.

running pylint

you can run pylint on all files or within a certain directory via the commandline

pylint src

run pylint only within the src directory and will produce a result as below

************* Module src.pa.analyse_load_wait
src/pa/analyse_load_wait.py:45:0: R0914: Too many local variables (25/15) (too-many-locals)
src/pa/analyse_load_wait.py:50:4: C0103: Variable name "step_functionArn" doesn't conform to snake_case naming style (invalid-name)
src/pa/analyse_load_wait.py:55:4: C0103: Variable name "ALLOWED_CONCURRENT_EXECUTIONS" doesn't conform to snake_case naming style (invalid-name)
src/pa/analyse_load_wait.py:77:4: C0103: Variable name "shouldWait" doesn't conform to snake_case naming style (invalid-name)
src/pa/analyse_load_wait.py:104:20: C0103: Variable name "shouldWait" doesn't conform to snake_case naming style (invalid-name)
src/pa/analyse_load_wait.py:111:16: C0103: Variable name "shouldWait" doesn't conform to snake_case naming style (invalid-name)
src/pa/analyse_load_wait.py:45:26: W0613: Unused argument 'context' (unused-argument)
************* Module src.TakeBackupDecisionLambda.index
src/TakeBackupDecisionLambda/index.py:161:9: W0511: TODO: handle this (fixme)
src/TakeBackupDecisionLambda/index.py:80:0: R0914: Too many local variables (29/15) (too-many-locals)
src/TakeBackupDecisionLambda/index.py:159:11: W0703: Catching too general exception Exception (broad-except)
src/TakeBackupDecisionLambda/index.py:80:26: W0613: Unused argument 'context' (unused-argument)
************* Module src.TakeBackupDecisionLambda
src/TakeBackupDecisionLambda/__init__.py:1:0: C0103: Module name "TakeBackupDecisionLambda" doesn't conform to snake_case naming style (invalid-name)

------------------------------------------------------------------
Your code has been rated at 8.92/10 (previous run: 8.92/10, +0.00)

the result will show the current issues within the code as well as the lines in which the said issues are found , it will also provide the current score out of 10.

Your code has been rated at 9.13/10 (previous run: 8.78/10, +0.35)

Disabling pylint rules

there are two possible ways to achieve this

disable the rule at the project level
disable the rule only for a specific file or a place

to disable the rule at project level you will need to add it to the .pylintrc file disable section, ones tdone lint issues arising from that rule will not be counted for the final lint score

disabling at file level we can add the below line to the top of the file

#pylint: disable=unused-argument

in this case unused argument rule is disabled in the given file, you can add more rules in the same row with comma separation.

Adding a Pre-commit hook

To avoid bad code with lint issues being committed this precommit hook was introduced .follow the below documentation on how to set up precommit in your project

https://pre-commit.com/

Install pre commit

pip install pre-commit

Add a precommit configuration

add the .pre-commit-config.yml file to your root direcotry

and add the following code lines to it

exclude: test/
repos:
-   repo: local
    hooks:
    -   id: pylint
        name: pylint
        entry: pylint src
        language: system
        types: [python]

Here repo is set to local , meaning rules are deifined in a local file and referred internally.

further with this file we can exclude the folders on top of which pylint might run

Install the githook scripts

$ pre-commit install
pre-commit installed at .git/hooks/pre-commit

Run pre commit

you can rull pre commit against all files or a given directory

pre-commit run --files src/*

pre-commit run --all-files

pre commit hook will not allow you to commit until the score is above a given value we defined

Ignore commit check

$git commit -m -n ”commit message”

addition of -n flag lets you commit the changes eventhough the desired lint check score is not met.

Setting up pylint check during the Pull Request initiation

we have a jenkns job that triggers upon creation of any pull request

this job will update the sonar stats according to the current status of the project and will pass or fail the job accordingly

another job was created with the same template to do the lint check and return the result to the pull request .

Create a clone of the curremt pull request trigger jenkins job and name it as pull request trigger lint check - this can be done via create new option in jenkis and selecing the projects like)

In the windows shell execution remove all sonar related commands and add the below commands

pip install pylint

pylint src

pylint src >>pylint.log

pylint-fail-under --fail_under 9.0 src

these commands will install and run pylint against the project on the fly and log the issues and the score.

issues can be seen in the jenkins dashboard with the rest of the execution logs

and also fail the job when lint score is below 9.0

please refer the below job to see the set configurations

http://jenkins-dev.prcp-np.us-east-1.aws.sysco.net:8080/job/Pull_Request_Triggering_Reference_Price_Service_ETL_Lint_Check/

Further add the violations plugin to jenkins (already done)

set up a post build job to read the pylint.log created during the build , filter the violations

and display any violations in any given build visually in the jenkins dashboard

finally in a case where the lint check fails pull request will be notified with the message Jenkins CI lint check failed

Github actions ( An alternative way to lint check )

Lint checking can also be done via github actions

https://medium.com/swlh/enhancing-code-quality-with-github-actions-67561c6f7063

please refer the below repo for sample code

https://github.com/fnote/pylint-test-repo/actions

GitHub actions allow us to define a workflow to run the lint check

you need to create a .github folder within root and create a workflows directory inside. Within workflows you can have multiple files for each workflow.below scenario has 2 workflows defined.one yaml to check the python version and other for the lint check

Below are the result once .github is pushed to your repo

these workflows will be reflected under the actions