I started in my new position 1 year ago as a Quality Engineer at Red Hat. I was struggling to find a better way to design, create, and run tests, so I decided to create my own setup from scratch. Here you'll check my own development workflow for pytest on Visual Studio Code.
Before we start, a big thank you to Lucas Aoki, who teaches me most of what you'll see in this post.
Why pytest and vscode?
I'm working on a project that has a big amount of code created using the pytest framework, so that's the reason why I'm using pytest. I don't have experience yet to define a testing framework or think about migrating to another one.
Now, about the vscode, it's mostly because is the most flexible development tool that I ever used, allowing me to mix and match my extensions to work with Vagrant, Podman, Docker, Kubernetes, etc. Unfortunately is not entirely open source and I have hopes that we will have some alternative in the future, but in the end, I don't think this matters too much, considering the alternative I can imagine would be JetBrain's PyCharm, an even closer and harder to extend IDE.
Let's create an application example
You can check the code that I created here (if you want to just clone it):
. ├── src │ ├── __init__.py │ ├── main.py
__init__.py file will be e mpty. This file is required anyway. It will be used later on to allow the FastAPI
app object to be imported by the pytest because this empty file makes the
src/ dir be interpreted as a class.
main.py file is our application. It's a simple web application created using the
Our dependency file and virtual env
I created two dependency files, one that contains the application dependencies and the other that contains all the development and testing dependencies, but also import the first one as well.
. ├── requirements.txt ├── requirements-dev.txt
The pytest files
Maybe you are new in the QA world just like, so I'll try to give you a little more explanation about the test files.
. ├── pytest.ini ├── tests │ ├── __init__.py │ ├── conftest.py
pytest.ini is the main file for pytest settings. You have different ways to configure a pytest run, you can check all those options and priorities over each other at official docs:
I already comment below on the options that I used, but I think it's important to mention the
addopts option to avoid pass all the pytest arguments for each run. For example the
--strict-markers that will make sure you and all your team will be forced to use only the
markers defined in this config file.
conftest.py is a dir-scoped file for your
fixtures, allows you to load
external plugins and specify
hooks (e.g. setup and teardown hooks).
You can learn more about
conftest.py at this Stack Overflow thread:
Everything defined in this file will be available for the
test_*.py files in this directory and all subdirectories.
My test files
My application is a REST API server, so I have two major types of endpoint:
resource based. The static is a simple returned text (e.g. version endpoint). The resource-based endpoint is the REST API itself, you have all the possible interactions with the resource. For example, considering we have a resource type called
user, then we can have an endpoint to add, remove, list, and change a user.
- This Thoghtworks post is a good introduction
I'm talking about REST API in this pytest post because the test files and how it'll be structured is tightly related to the application itself. That means you will need to get some abstractions from the application architecture to architect your test suite.
. ├── tests │ ├── test_static_endpoint.py │ ├── test_user.py
test_user.py file there's an important pytest feature, the
parametrize. This allows you to run the same function multiple times using different parameters. A simple and easy to ready approach for input validation, for example.
Visual Studio Code files
Now, we will finally check the vscode features. You'll need only a single configuration file. This one, depending on how your team works, could even be committed to a Git repository and shared across the team.
.json file doesn't support comments, I'll describe each option in the following list:
python.testing.pytestEnabled: enable all the pytest features if the
pythonplugin is installed.
python.defaultInterpreterPath: default Python interpreter to be used. Very useful to define a Python binary from a
python.linting.pycodestyleEnabled: enables the code style features. It shows alerts when you have something that does not respect the defined code style.
python.linting.pycodestyleArgs: define a code style to be respected by the vscode. The arg
--max-line-lenght 80it's used by the linter whenever you have lines longer than 80 characters.
editor.rulers: Creates a vertical line after each number in the list. Guides you to not create longer lines.
. ├── .vscode │ ├── settings.json
Visual Studio Code dev tools
There is one plugin that I enjoy too much about the vscode, the Remote development using the SSH plugin. This is always my main approach to create a controlled environment to do my development.
I create a VM using Vagrant, install everything that I need using the Ansible provisioner, and jump into the VM using this plugin. Doesn't even matter my host OS, I always have my fresh new Linux waiting for me.
With that plugin, I'm even able to forward the development machine to my
localhost port, allowing me to easily reach web development servers, for example, using a secure and encrypted channel.
WIP PUT A VIDEO HERE
Visual Studio Code test workflow
- pytest docs - Configuration
- Stack Overflow - In pytest, what is the use of conftest.py files?
- Thoughtworks - REST API Design - Resource Modeling