24 views
# Python Setup Guide for Carpentries and HIDA Courses This document contains drafts in Markdown mode for the Python setup intructions for the Software Carpentries course "Plotting and Programming in Python" ([URL](https://swcarpentry.github.io/python-novice-gapminder)) and the HIDA course "Kickstart Python" based on it (see HIDA Course Catalogue, [URL](https://www.helmholtz-hida.de/course-catalog/en/)). For use in the course website using the Carpentries Workshop Template ([URL](https://github.com/carpentries/workshop-template)), they need to be converted to the proper format (HTML / Liquid, for instance, with AI chatbot), and inserted into the config file `_includes/install_instructions/python.html`. The reason for this is that until start of 2025, the workshop-template used the Anaconda Python distribution as default. However, as of 2025, some entities (such as HIDA) cannot use this Python distribution anymore because of licensing changes (background [here](https://www.fz-juelich.de/en/rse/the_latest/the-anaconda-is-squeezing-us)). Proposed updates to the workshop template (see pull requests, [here](https://github.com/carpentries/workshop-template/pulls)) were not yet implemented, or inadequate for HIDA purposes. # Table of contents - [Version v1.1](#Version-v11), date 2025-03-10. By Johannes Wasmer. - Default setup: python.org Python distribution plus JupyterLab. Virtual environment setup part of setup instructions. - Alternative setups: Google Colab, Visual Studio Code. - Not used. Reasons: - Virtual environment setup instructions are too long and complex for beginners to do it on their own. Do it in the course instead. - [Version v1.0](#Version-v10), date 2025-03-10. By Johannes Wasmer. - Default setup: python.org Python distribution plus JupyterLab or Visual Studio Code. Virtual environment setup NOT part of setup instructions. - Alternative setups: Google Colab. - Used in course editions: - 2025-03-27 HIDA Kickstart Python - [Course website](https://irratzo.github.io/2025-03-27-hida-kickstart-python-online/) - [Course website repository](https://github.com/Irratzo/2025-03-27-hida-kickstart-python-online) - [Version v0.1](#Version-v01), date 2024-10-10. By Johannes Wasmer. - First attempt. Not used. - Extracted from reference: Claude AI. 2024-10-10. Setting up Python Environment for Carpentries Course. [URL](https://claude.ai/chat/3c01d12a-e1ea-4944-a2f1-0aad371a3617). # Version v1.1 We offer two options. A local setup on your own computer, and a setup-free browser-based solution. We recommend the local setup for setting you up for a continued growth of your future programming skills. But if you don't have time for the setup, go for the browser-based solution. That is fine. ## Browser-based setup If you don't want to do or can't do the local setup, we offer a setup-free, browser-based solution. We will use Google Colab. This is a free service by Google for browser-based Python programming. You will need: a compatible web browser (Chromium, Firefox or Safari; Chromium is recommended), and a personal Google account. If you don't have one, crate one here: https://accounts.google.com/signup. Then try to access [Google Colab](https://colab.research.google.com/). If that works, you are done. We will distribute a Colab notebook template for you to use in the course. If you don't want to use Google Colab, while we acknowledge that there alternatives to Google Colab that do not require a Google account, we kindly ask you to attempt the local setup, instead. This course relies on all instructors and students to have the same setup. We lack personnel to be able to offer support for more than two setups at a time. ## Local setup (recommended) For the local setup, we need to install two things: the official python.org Python distribution and an editor for the programming. If at any point during the installation, you fail to proceed because of missing administrator rights, please contact your local system administrator or tech support, and ask them to install the software for you. Please ask them to stick to the software and extensions mentioned here. This course relies on all instructors and students have the same local setup, to minimize differing setup issues. (An aside note. Editions prior to 2025 of this course used the Anaconda Python distribution. However, due to change in licensing, this option is not viable anymore for research organizations like the Helmholtz Society. If you would like more information on this, see [here](https://www.fz-juelich.de/en/rse/the_latest/the-anaconda-is-squeezing-us).) ### Install Python Check if Python is already installed on your computer. Open a terminal/command prompt. - **Windows**: Press `Windows key + R`, type `cmd` and press Enter - **Mac**: Press `Command key + Space`, type `term`, select Terminal and press Enter - **Linux**: Press `Ctrl + Alt + T` or find Terminal in your applications menu Check for Python. Type one of these commands and press Enter: ``` python --version ``` If that doesn't work, try: ``` python3 --version ``` Windows users can also try: ``` py --version ``` If any command returns a version number 3.9 or higher (e.g., `Python 3.11.5`), Python is installed and you can proceed to the VS Code installation. If Python is not found or version is too old, we will install it now. 1. Visit [python.org/downloads](https://python.org/downloads) 2. Download the latest version for your operating system 3. Run the installer - **Windows users**: Check the box "Add Python to PATH" during installation - **Mac/Linux users**: Follow the default installation options After installation, open a new terminal window and check again, using the same method as before, to confirm Python is now installed correctly. You may need to open a new terminal tab or window for the update to the system to be recognized. ### Set up the project In Python programming, it is custom to create a self-contained folder for each new project. The folder will hold all data and code belonging to the project. It will furthermore contain an isolated Python runtime that we will use only for this project. This custom ensures reproducibility. **1. Create a Project Folder** Create a folder named `hida-kickstart-python` in suitable location. We recommend to choose a location where you intend to keep similar Python projects in the future. The Python runtime that we will create will be hard-coded to that location. More on that below. We further recommend to not use whitespaces in file and folder names for programming projects. This complicates command-line commands **2. Download and Extract the Course Data** In this course, we will use a dataset from the [Gapminder Foundation](https://www.gapminder.org/). 1. Download the course data file from [this link](https://swcarpentry.github.io/python-novice-gapminder/files/python-novice-gapminder-data.zip) 2. Save the zip file to your new `hida-kickstart-python` project folder 3. Extract the contents of the zip file in that same folder - On Windows: Right-click the file and select "Extract All" - On Mac: Double-click the file - On Linux: Right-click and select "Extract Here" or use the archive manager After extraction, you should see a `data` folder inside your `hida-kickstart-python` project folder. **3. Set Up a Python Virtual Environment** First, open a terminal as described in the "Install Python" section above. Now, we'll navigate to your project folder and set up a virtual environment. *Navigate to Your Project Folder* For all operating systems: 1. In the terminal, type `cd` followed by a space in your terminal 2. Drag and drop your `hida-kickstart-python` project folder from your file manager into the terminal. Now the path to the folder should appear there, for instance like so: `cd /Users/alice/courses/hida-kickstart-python`. Mind the space between `cd` and the path. The path to the folder may look different, depending on your operating system. 3. Press Enter If drag and drop doesn't work: On Windows, you can copy the full path from File Explorer (click in the address bar and press Ctrl+C), then paste it in the terminal (right-click or Ctrl+V). On Linux, you can usually also right-click inside your project folder and select "Open in Terminal". *Create and Activate a Virtual Environment* In Python, an isolated Python runtime is called a "virtual environment". The corresponding Python command to create such an environment is called `venv`. A virtual environment is just another folder that contains a fully functional Python runtime that is completely separate from you system Python (the one that we installed above). The reason for this is because in this project, we will install additional libraries that we will only use for this project. By isolating this process from the system Python, we ensure that any issues that might arise will not compromise the system Python installation. If it breaks, which can happen, we can just delete the environment folder and create a new one. This step is a custom practice in Python development and should never be skipped when starting a new project. We will now create our project's Python environment. While inside your project folder in your terminal, copy-paste or type the following code into your terminal and press Enter. ``` python -m venv .venv ``` This command tells the system Python to create a new self-contained Python runtime, a virtual environment, in the folder `.venv` inside our project folder. The `.venv` name is a convention for a Python project environment. Check that the folder has been created by typing in the terminal `ls -la` (Linux/Mac) or `dir/a` (Windows). Your file manager probably hides "hidden files" (the ones beginning with a dot) by default, so you won't see the new folder there. You can show it there like so. - Windows: In File Explorer, click on the "View" tab, then check the "Hidden items" box - Mac: In Finder, press `Command+Shift+.` (period) to toggle hidden files - Linux: In most file managers like Nautilus (Ubuntu), press `Ctrl+H` or look for "Show Hidden Files" in the View menu Next, we need to access the environment in order to install the required libraries. This is called "activating" the environment. In your terminal, execute this command. ``` # Mac / Linux source .venv/bin/activate # Windows .venv\Scripts\activate ``` You should see the terminal prompt change to include `(.venv)` at the beginning, indicating that the virtual environment is now active in this terminal session. *Verify Python Environment* To verify that you're using the Python from your virtual environment, execute: ``` # Mac / Linux which python # Windows where python ``` The output should show a path that includes your project folder and `.venv`. *Install Required Packages* With your virtual environment activated, we will now install the required libraries, which in Python are usually called "packages". For that, we use Python's built-in package manager `pip`. ``` pip install matplotlib pandas jupyterlab ``` The installation may take a few minutes. When successful, you should see a message like: ``` Successfully installed matplotlib-3.7.2 pandas-2.0.3 jupyterlab-4.0.5 [... and more packages] ``` *Deactivate the Environment* Once the installation is complete, you can deactivate the virtual environment: ``` deactivate ``` The `(.venv)` prefix should disappear from your terminal prompt. Later in the course, we will use this environment to start the editor JupyterLab, in which we will do the programming. ### Install Visual Studio Code (optional) If you already have VSCode installed, or if you want to use it instead of JupyterLab, you can also install it now. Visual Studio Code is a popular, free code editor by Microsoft for Python and other programming languages. The editor itself only has basic programming features. But the rich ecosystem of community-driven extensions makes it adaptable to most programming tasks. Download it from [code.visualstudio.com](https://code.visualstudio.com/). (Be aware that Microsoft's VS Code is open source, but their installable version "collects telemetry data, which is used to help understand how to improve the product" ([reference](https://code.visualstudio.com/docs/editor/telemetry)). The large majority of programmers (including the course instructors) does not see this as an issue and use the Microsoft version. However, if you do not want telemetry, there is a community-driven version that does not collect telemetry data. You can download that version [here](https://vscodium.com/). Please be aware that the instructors cannot guarantee support for possible issues compared to the Microsoft version, although their occurrence is unlikely.) Now we need to install VSCode extensions needed for this course. 1. Open VSCode 2. Click on the Extensions icon in the left sidebar (looks like four squares) 3. Search these extension and click "Install": - [Python (by Microsoft)](https://marketplace.visualstudio.com/items?itemName=ms-python.python) - [Jupyter (by Microsoft)](https://marketplace.visualstudio.com/items?itemName=ms-toolsai.jupyter) After the extension installations are complete, close the editor. # Version v1.0 We offer two options. A local setup on your own computer, and a setup-free browser-based solution. We recommend the local setup for setting you up for a continued growth of your future programming skills. But if you don't have time for the setup, go for the browser-based solution. That is fine. In all of the offered solutions, we will use [Jupyter](https://jupyter.org/) notebooks for programming, a popular approach in data science. The interfaces in all solutions are nearly identical. ## Browser-based setup If you don't want to do or can't do the local setup, we offer a setup-free, browser-based solution. We will use Google Colab. This is a free service by Google for browser-based Python programming. You will need: a compatible web browser (Chromium, Firefox or Safari; Chromium is recommended), and a personal Google account. If you don't have one, crate one here: https://accounts.google.com/signup. Then try to access [Google Colab](https://colab.research.google.com/). If that works, you are done. We will distribute a Colab notebook template for you to use in the course. If you don't want to use Google Colab, while we acknowledge that there alternatives to Google Colab that do not require a Google account, we kindly ask you to attempt the local setup, instead. This course relies on all instructors and students to have the same setup. We lack personnel to be able to offer support for more than two setups at a time. You are now done with the browser-based setup. ## Local setup (recommended) For the local setup, we need to install two things: the official python.org Python distribution and an editor for the programming. If at any point during the installation, you fail to proceed because of missing administrator rights, please contact your local system administrator or tech support, and ask them to install the software for you. Please ask them to stick to the software and extensions mentioned here. This course relies on all instructors and students have a similar setup, to minimize differing setup issues. (An aside note. Editions prior to 2025 of this course used the Anaconda Python distribution. However, due to change in licensing, this option is not viable anymore for research organizations like the Helmholtz Society. If you would like more information on this, see [here](https://www.fz-juelich.de/en/rse/the_latest/the-anaconda-is-squeezing-us).) ### Install Python Check if Python is already installed on your computer. Open a terminal/command prompt. - **Windows**: Press `Windows key + R`, type `cmd` and press Enter - **Mac**: Press `Command key + Space`, type `term`, select Terminal and press Enter - **Linux**: Press `Ctrl + Alt + T` or find Terminal in your applications menu Check for Python. Type one of these commands and press Enter: ``` python --version ``` If that doesn't work, try: ``` python3 --version ``` Windows users can also try: ``` py --version ``` If any command returns a version number 3.9 or higher (e.g., `Python 3.11.5`), Python is installed and you can proceed to the VS Code installation. If Python is not found or version is too old, we will install it now. 1. Visit [python.org/downloads](https://python.org/downloads) 2. Download the latest version for your operating system 3. Run the installer - **Windows users**: Check the box "Add Python to PATH" during installation - **Mac/Linux users**: Follow the default installation options After installation, open a new terminal window and check again, using the same method as before, to confirm Python is now installed correctly. You may need to open a new terminal tab or window for the update to the system to be recognized. The next thing to do is to set up a project folder and a Python virtual environment to work with data science libraries in Jupyter notebooks. However, we will do this all together during the course. ### Install Visual Studio Code (optional) At this point, you can either choose to stop, or proceed to installing Visual Studio Code. If you choose to stop, you will work with [JupyterLab](https://jupyter.org/). This comes together with the virtual environment that we'll set up together, so you don't need to do that now. We give you either option, because the interfaces of all these options (Google Colab, JupyterLab, VSCode) are very similar. For data science beginners, JupyterLab is absolutely fine. If you are already curious, proceed to VSCode. Visual Studio Code is a popular, free code editor by Microsoft for Python and other programming languages. The editor itself only has basic programming features. But the rich ecosystem of community-driven extensions makes it adaptable to most programming tasks. Download it from [code.visualstudio.com](https://code.visualstudio.com/). We want to make you aware that while Microsoft's VS Code is open source, it collects anonymized telemetry data for product improvement ([reference](https://code.visualstudio.com/docs/editor/telemetry)). If you do not want that, there is a community-driven version that does not collect telemetry data, [here](https://vscodium.com/). In the course, we will only be able to give support for issues with the Microsoft version. Now we need to install VSCode extensions needed for this course. 1. Open VSCode 2. Click on the Extensions icon in the left sidebar (looks like four squares) 3. Search these extension and click "Install": - [Python (by Microsoft)](https://marketplace.visualstudio.com/items?itemName=ms-python.python) - [Jupyter (by Microsoft)](https://marketplace.visualstudio.com/items?itemName=ms-toolsai.jupyter) After the extension installations are complete, close the editor. You are now done with the local setup. # Version v0.1 This guide will help you set up your Python environment for the Carpentries course using Visual Studio Code (VSCode). Follow these steps regardless of your operating system (Windows, macOS, or Linux). ## 1. Set Up Visual Studio Code 0. Install Python First make sure that you have Python not already installed. Open a terminal and try any of these commands. To check for Python: ``` python --version python3 --version ``` If any of these commands returns a version number, Python is installed. If not, follow these instructions to install Python. - Windows https://docs.python.org/3/using/windows.html - Mac https://docs.python.org/3/using/mac.html - Linux (usually Python is pre-installed) https://docs.python.org/3/using/unix.html 1. Ensure you have Visual Studio Code installed. If not, download it from [code.visualstudio.com](https://code.visualstudio.com/). 2. Install the following VSCode extensions: - Python (by Microsoft) - Jupyter (by Microsoft) To install extensions: 1. Open VSCode 2. Click on the Extensions icon in the left sidebar (looks like four squares) 3. Search for each extension and click "Install" ## 2. Check for Conda or Pip Before creating a virtual environment, let's check whether you have conda or pip available: 1. Open VSCode 2. Open a new terminal in VSCode (Terminal > New Terminal) 3. In the terminal, try the following commands: To check for conda: ``` conda --version ``` If this returns a version number, you have conda installed. You might have a conda derivative installed. If you suspect this, replace "conda" in the command above with either name of the derivate (miniconda, mamba, micromamba). If that is the case, replace all following "conda" in all statements below with the name of the derivative. They work exactly the same. To check for pip: ``` pip --version ``` If this returns a version number, you have pip installed. Note which one (or both) is available on your system, as this will determine how you create your virtual environment and install packages. ## 3. Create a Virtual Environment (Optional but Recommended) Creating a virtual environment helps isolate your course dependencies. In your VSCode terminal: If you have conda: ``` conda create -n hida python=3.12 conda activate hida ``` If you don't have conda (which is fine), use venv instead (built-in Python module): ``` python -m venv hida ``` To activate the venv environment: - On Windows: `hida\Scripts\activate` - On macOS/Linux: `source hida/bin/activate` ## 4. Install Required Packages Use either pip or conda to install the required packages, depending on what's available on your system: Using pip: ``` pip install matplotlib pandas jupyterlab ``` Using conda: ``` conda install matplotlib pandas jupyterlab ``` ## 5. Verify Installation To verify that everything is installed correctly, create a new Python file in VSCode and try importing the packages: ```python import matplotlib import pandas import jupyterlab print("All packages imported successfully!") ``` ## 6. Set Up Jupyter Lab in VSCode 1. Open VSCode 2. Use the Command Palette (Ctrl+Shift+P or Cmd+Shift+P) and type "Create New Jupyter Notebook" 3. Select a kernel that uses your newly created environment (if applicable) ## 7. Cleanup After the Course After you've completed the course and no longer need the environment, you can deactivate and remove it to free up space on your system. ### Deactivating the Environment To deactivate the current Python environment after the course: If using conda: ``` conda deactivate ``` If using venv: - On Windows: `deactivate` - On macOS/Linux: `deactivate` ### Removing the Environment To completely remove the Python environment after the course (optional): If using conda: ``` conda remove --name hida --all ``` If using venv: Simply delete the environment folder: - On Windows: `rmdir /s /q hida` - On macOS/Linux: `rm -rf hida` ### Removing VSCode Extensions (Optional) If you no longer need the VSCode extensions for Python and Jupyter: 1. Open VSCode 2. Click on the Extensions icon in the left sidebar 3. Find the Python and Jupyter extensions 4. Click on the gear icon next to each extension and select "Uninstall" Note: Only remove these extensions if you're sure you won't need them for other projects. ## Troubleshooting - If you encounter any "command not found" errors, make sure your Python installation is in your system's PATH. - For Windows users with WSL (Windows Subsystem Linux): If you prefer using WSL, you can install Python and the required packages within your WSL environment and use VSCode's Remote - WSL extension to work with your WSL-based Python setup. In WSL, all the Linux / MacOS commands listed above work as well. Remember to keep any course materials or personal projects you've created before removing the environment. If you think you might need this setup again in the future, consider keeping the environment and just deactivating it instead of removing it completely. If you encounter any issues or need further assistance, please don't hesitate to ask for help during the course.