🐍 Python Development Guidelines¶

Welcome to the Python coding guidelines for the CLOE Toolbox! These guidelines will help you write clean, readable, and maintainable Python code that integrates seamlessly with our ecosystem.

General Guidelines¶

Project Structure¶

Maintain a clear and consistent project structure. Here’s a basic layout:

project_name/
│
├── src/
│   └── your_package/
│       ├── __init__.py
│       ├── module1.py
│       └── module2.py
│
├── tests/
│   ├── __init__.py
│   └── test_module1.py
│
├── pyproject.toml
├── README.md
└── docs/
    └── index.md

src/: Contains the source code.
tests/: Holds test files.
docs/: Documentation files for MKDocs.

Dependency Management with UV¶

UV is a fast, reliable Python package manager written in Rust. Here's how to manage your project's dependencies in pyproject.toml:

Add a Package:

uv add "<package_name>~=1.4.5"  # Compatible release: >=1.4.5, ==1.4.*
uv add "<package_name>~=2.2"    # Compatible release: >=2.2, ==2.*
uv add <package_name>==1.2.3    # Pins to exact version

Add Development Dependencies:

uv add --dev <package_name>     # Add to [project.dependency-groups.dev]

Remove a Package:

uv remove <package_name>

Sync Dependencies:

uv sync                         # Install all dependencies from pyproject.toml

Version Constraint Best Practices:

Use ~= (compatible release) for most packages to get compatible updates
- ~=2.2 allows any version 2.2 and up, but less than 3.0
- ~=1.4.5 allows any version 1.4.5 and up, but less than 1.5.0
Only pin exact versions (==) when specifically required
Review your dependencies periodically for available updates
Use tools like dependabot or renovate to automate security update PRs

Common PEP 440 Version Specifiers:

~=2.2 equals >=2.2, ==2.*
~=1.4.5 equals >=1.4.5, ==1.4.*
==1.2.3 - Exact version
Note: ~=1 is not valid according to PEP 440

Using Devcontainers with VSCode¶

We use Devcontainers to ensure a consistent development environment across all projects. This helps avoid "it works on my machine" issues.

Setting Up Development Environment with VSCode, WSL, and Devcontainers¶

Set up WSL
Enable WSL in Windows Features or run wsl --install in an admin PowerShell
Install Ubuntu (recommended version: 24.04) using either:
- wsl --install -d Ubuntu-24.04 in PowerShell
- Microsoft Store installation
Verify that you have only one Ubuntu installed, e.g. by typing "Ubuntu" in the Windows search. If you have multiple, deinstall the wrong one. Otherwise VSCode might not use the right one.
After installation, launch Ubuntu and create your user account.
Connect VSCode to WSL
Important: Don't try to connect to WSL before creating your user account. Otherwise, VSCode gets confused and uses the root directory, not your user directory.
Click the blue button in the bottom-left corner of VSCode (it will display "Open a Remote Window" if you hover over it with your mouse)
Select "Connect to WSL" from the command palette
VSCode will connect to your WSL instance
Working with Repositories in WSL
Using the terminal:
- Open your WSL terminal in VSCode using Ctrl + ` (or Strg + ö on a german Key Board)
- Navigate to your preferred working directory
- Clone your repository
```
git clone https://github.com/your-repo.git
cd your-repo
```
Remember to enter you password when prompted. In VSCode, the input will be done at the top of the window.
Using the inbuilt VSCode version control:
- Open the command palette using Ctrl + Shift + P
- Type Git: Clone
- Enter the URL of the repo
- Enter you password (for a repo in Azure DevOps, you can use a Personal Access Token as password)
If the repository has a .devcontainer folder, VSCode will prompt you to "Reopen in Container"
If Docker is not installed, VSCode will prompt you to install it automatically
Using Devcontainers
Install the VSCode extensions pack Remote Development
When opening a project with a .devcontainer configuration:
- VSCode will detect it and prompt to "Reopen in Container"
- Click "Reopen in Container" to build and start the development container
- IF VSCode did not detect it resp. the message did not show, you can do it manually. Open the command palette using Ctrl + Shift + P, type Reopen in Container and hit Enter
The container will be built using the WSL Docker engine
All terminal sessions will now run inside the container
Troubleshooting
If you cannot create your user account or you get an logon error when launching Ubuntu, open an admin Power Shell and run Get-Service vmcompute | Restart-Service. Afterwards, try again to create your user account.
If VSCode cannot connect to WSL, try to open Ubuntu directly. You might get an Logon failure, see failure above.
If VSCode does not connect to WSL:Ubuntu-24.04, check whether you have two Ubuntus installed. If yes, deinstall the wrong one. Afterwards, try to cennect to WSL again. If it still connects to the wrong Ubuntu, deinstall all Ubuntus and run wsl --unregister ubuntu resp. wsl --unregister ubuntu-24.4 in an Admin PowerShell. Afterwards, reinstall Ubuntu-24.04 as described above.
If the container doesn't open with an error containing .devcontainer/devcontainer.json, reinstall the VSCode extension package Remote Development. If this does not help, try restarting your machine.

If Docker commands fail, ensure the Docker daemon is running:

sudo service docker status
sudo service docker start

If you can't connect to WSL, try:
```
wsl --shutdown
wsl --start
```
For permission issues with Docker:
```
sudo chmod 666 /var/run/docker.sock
```

Benefits of Using Devcontainers¶

Consistency: Ensures all team members are using the same development environment.
Isolation: Keeps your local environment clean by isolating project dependencies in the container.
Portability: The configuration can be shared across different machines and platforms, ensuring the project works the same everywhere.

Python Style¶

Emphasize Readability¶

Python code should be easy to read and understand. Use meaningful variable names, keep functions small and focused, and write clear comments where necessary.

# Good
def calculate_total(price: float, tax_rate: float) -> float:
    """Calculate the total price including tax."""
    return price * (1 + tax_rate)

# Bad
def ct(p, t):
    return p * (1 + t)

Use Docstrings¶

Include docstrings in your functions and classes to describe their purpose and usage.

def add_user(name: str, age: int) -> None:
    """
    Add a user to the database.

    Args:
        name (str): The name of the user.
        age (int): The age of the user.
    """
    pass  # Implementation goes here

Typing and Type Hints¶

Python 3.11 introduced simplified syntax for type hints. Use these wherever possible for better readability:

def fetch_data(url: str, timeout: int) -> dict:
    # Implementation goes here

For complex data structures:

def process_data(data: list[dict[str, str]]) -> None:
    # Implementation goes here

For optional types, use the new syntax:

def get_username(user_id: int) -> str | None:
    # Implementation goes here

Use Logging¶

Use the built-in logging module for logging messages. This allows you to control the log level and format and should be used instead of print.

import logging

logging.basicConfig(level=logging.INFO, format="%(asctime)s - %(message)s")
logging.info("This is an info message")

When working with classes, add a logger as an instance variable:

import logging

class MyClass:
    def __init__(self):
        self.logger = logging.getLogger(__name__)

    def do_something(self):
        self.logger.info("Doing something")

Loggers allow you to define dynamic targets, so called handlers. The CLOE toolbox defines a standardized logging module. This module currently supports handlers for Snowflake, Databricks and Log Analytics. Check the DevOps Repository.

Using Pydantic for Data Validation¶

Pydantic is ideal for validating and parsing data, especially when dealing with external sources like APIs or config files.

from pydantic import BaseModel

class User(BaseModel):
    name: str
    age: int

# Usage
user_data = {"name": "John", "age": 30}
user = User.model_validate(user_data)

Pydantic will automatically validate the data types and raise errors for invalid data.

Conclusion¶

For more detailed documentation, refer to the official Python documentation.